(Edit: This post was originally published on May 18, 2017; it has been updated on July 13, 2017 to reflect the recently announced changes in judged awards.)
All VEX competitions include trophies for the tournament winners and (if applicable) skills champion. They also include a number of “judged awards” that are based on criteria other than match play. Since many of these awards sound very similar to each other, I thought I’d write this post to summarize things.
- The judges & the competition judging process
- The interview
- Awards: The usual suspects
- Awards: Building and programming (Amaze, Build, Create, Think, Innovate)
- Awards: Team behavior & activity (Sportsmanship, Energy, Inspire, Service, Community, Teamwork)
- Awards: Individual (Mentor, Teacher, and Volunteer of the Year)
- ***Special rules for VEX Worlds***
- Resources & downloads
The Judges & Competition Judging Process
Each tournament will have recruited a number of adult volunteers to be judges; some of these people know a lot about VEX and robotics, and others do not (but do—most of the time—understand engineering, in my experience; sometimes not). Judges spend their day in a combination of reading through the engineering notebooks that teams have submitted in the morning when they register (see my separate, in-depth post about the Engineering Notebook); and walking around the pit area and talking to teams face-to-face (or in a separate judging room where teams come to them). Judges are generally assigned certain teams to speak with, to make sure that every team is visited during the day. Occasionally judges will talk to a team in the pit area more than once, or a second group will stop by later in the day; we usually take this as a good sign, that we are interesting enough for a second look.
As of the 2017-18 season VEX Awards Appendix has added a section on the front page entitled “Student-Centered Teams.” This section reiterates wording that was added to the game manual in the same year, emphasizing that work should be primarily done by students (even though most people would think you wouldn’t need to have to write this in the manual; apparently there are some teams out there not on the same page). Judges are instructed to observe in the pits whether coaches and mentors are offering help and guidance versus doing the work themselves. The stated emphasis is to make sure that “the purpose of the program is to enhance the learning process, not to win at all costs.”
VEX publishes a Judges Guide with the specifics of what types of questions judges may want to ask, and specific things they should be looking at and evaluating. According to the guidelines, judges should allot 10-15 minutes per robot team; in our experience this time is frequently far less than 10 minutes. At California State championships for Starstruck, we were one of the first teams interviewed in the morning, before competition started, and judges were only able to spend about 5 minutes with my girls. We felt that we got a little short-changed on that one, but in reality, the kids have to learn to go with unpredictability and be prepared for whatever happens.
But what if my team is in a match when they come around? Don’t worry, the judges will definitely come back when the team is done on the field. Here again, teams need to go with the flow; they may be in the pit area trying to fix the robot in a limited amount of time, and the judges may say that this is the only time they are available—these things happen, and the team should be prepared to improvise.
At some larger competitions (usually depending on whether the venue has the space available), the judges may be set up in one or more side-rooms, and teams must sign up in the morning for an interview time slot—and then manage their time so that they and the robot and any presentation materials get to their judging slot on time. In our experience, teams often have to sign up in advance of when the match schedule is handed out, which is a bit head-scratching, but again, the teams need to improvise and communicate with the judging team if they have a time conflict.
A coach from the VEX World Coaches Association Facebook group who is an “event partner” (VEX’s term for a person or organization that runs a tournament) describes the way she runs judging, which sounds very useful and streamlined:
One thing I’ve never liked in VEX is how judging is scattered amongst the matches all day. It is very jarring [for a team] to attempt to get composed for judging when matches are thrown in there.
One thing I’ve done at the tournaments I host, is that we have a 1-hour, stand-alone, dedicated time period at the start of day just for judging (after inspection) and every team is given a judging schedule with a prescribed time slot. I open robot skills during this time so teams can do those if they wish as they wait for their time slot. Then I queue teams 3-deep for each judging room so that the time slots run efficiently and have queuers stay on top of making sure that teams are ready to go for the their judged time when a team enters judging and we need that 3rd team queued.
This system has worked incredibly well for me at my tournaments. In an event with 20 teams, we finish judging in an hour ( I have 3-5 judge rooms with 3 judges each running concurrently) and then teams can just concentrate on competition the rest of the day. We have always finished no later than 4 pm with the entire tournament. Judges can then do pit judging if they feel the need. Plus then judges have all day to deliberate and watch matches etc. They are not scrambling to find teams they haven’t talked to at 2pm or later in the day.
At some events, only teams that have submitted an engineering notebook in the morning at check-in are eligible to sign up for a judging time-slot. Once those teams have been interviewed, judges then walk around the pits and talk to the remainder of the teams during the day.
They’re Watching You
Teams are not just evaluated in formal discussions with the judges in the pit area or interview room. Judges also look at teams and their robot during match play: What is their conduct around the event volunteers? What do their interactions with other teams look like? Are they good sports when they lose? Are they also good sports when they win? We have also had judges ask us at tournaments whether any other teams offered us assistance during the day, to find out more about how teams act, basically, when no one is looking.
This year there is a new document published for event volunteers: the “Field Note to Judges.” This form is an official, standardized way for volunteers out on the floor or at the competition fields to relay information about specific teams to the judges—either positive (GREEN) or negative (RED).
Judges keep their deliberations and evaluations of the specific teams 100% confidential, and at the end of the day, all written materials are generally shredded. Don’t expect to get feedback from the judges for an individual team; they have taken a vow of silence.
So what happens during the judges interview? What should my team be prepared to do?
I think the most important thing that teams can do is prepare an organized presentation of their robot before the competition. We have a PowerPoint presentation that we show on an iPad, and every team member is assigned a few slides to cover. Other teams we’ve seen use posters or display boards in lieu of an electronic presentation. Most teams, however, just put together a verbal description of the important features of the robot. However the team chooses do do things, the important part is to organize things ahead of time. Winging it will not produce the most high-quality interview.
Our team’s presentation starts with a brief overview of the team, and then goes over the various mechanical features of our robot (with pictures), discusses programming aspects they feel are important, talks about the team’s goals for the year, and so on. Our team finds that having a PowerPoint prepared ahead of time is a great, structured way for the girls to practice what they’re going to say and also to make sure that every team member has time to talk, every team member interacts with the judges, and people don’t talk over each other. These are all foundational aspects to having a good interview experience.
Having many different team members talking to the judges and answering impromptu questions is a way of demonstrating that every team member has been engaged with the robot build and design process. If one student does all of the talking, then the judges have no way to evaluate the level of teamwork and organization that has been done in the design/build process.
Awards: The Usual Suspects (Excellence, Design, and Judges)
In the VEX Awards Appendix, there is a long list of judged awards, but in reality most of them are not handed out at local tournaments. In my experience certain awards are predominantly given at large tournaments or state championships. Nearly every tournament, however, will include the Excellence Award, the Design Award, and the Judges Award; I’ve never been to a tournament in the last 5 seasons that did not include at least these three.
The Design Award
The Design Award is based on 2 components: the engineering notebook and the judges interview. There is a very specific scoring rubric for this award that is included on pages 21-21 pages of the Judges Guide. Once you read the rubric, there is really no mystery about what the judges are looking for, but there are a lot of details to think about.
Judges use the first page of the rubric to “separate the wheat from the chaff”. They use the scores for the engineering notebook criteria alone to narrow down the field to a list of the top 5 (or so) contenders. Then for just those people, they go on to page 2 of the rubric, the interview. So basically, page 1 of the rubric gets you into the game; page 2 determines the likely winner. Why “likely winner”? Because the rubric is still just a guide; page 7 of the Judges Guide states:
The rubrics are intended to be used by Judges to narrow down the field of contenders for each award. Multiple teams often score “perfect” 3’s on a rubric. While the rubric is quantitative in nature, Judges are expected to apply their qualitative Judgement when making a final decision on all awards.
The “engineering notebook” section of the rubric includes 8 items (discussed below), and the “student interview” includes 5 items. Teams can score up to 3 points in each line-item, based on whether their book or interview meets the criteria listed. Do the math, and the notebook/interview weighting is about 60/40, so merely having a fantastic engineering notebook is not enough if your team doesn’t also do reasonably well on the interview.
For the engineering notebook specifics, read my detailed post on this topic. The award criteria make clear what the judges are looking for: details. And they want details, with photos or drawings, at every stage of the process, from evaluating the game and developing a strategy to brainstorming to the actual design and build. It’s important to include details about “the road not taken”: include all of the team’s brainstorming ideas, not just the one they settled on in the end, and include an explanation of why the final design was chosen and the others not.
Once the building process starts, I can summarize what’s needed this way: details, details, details. For example, to earn 3 points in the “Design Process” item, “the notebook records the building and programming process in such detail that someone outside the team could recreate the robot by following the steps in the notebook.” Wow. Seriously, just wow, but that is the standard against which the notebooks are judged. Even on non-building days, details should be included; instead of “we had drive practice today,” one could write “we had drive practice today and our high score in 60 seconds is now xx points; we still need to work on blah blah blah.”
Starting in 2017, the Design Rubric added 3 bonus points for teams that have a bound engineering notebook. This change makes it explicitly OK for teams to have a 3-ring binder; in the past, there was inconsistency on how the “bound notebook is preferred” text was interpreted by judges. However, teams that choose to use a 3-ring binder will not receive the 3 extra points.
The judges interview is not only used to determine the Design Award winner; it’s used to determine all judged award winners. So while I stated above that the interview rubric is really only considered for the top Design contenders, it’s still important for every team to do well in this part. One of the best things your team can do, even without making a formal presentation, is to make a list of what they’d like to say about their robot, and divvy up that list among all team members.
Judges use the interview process as a way to confirm (or not) the information included in the engineering notebook. If a team has a fantastic notebook, but team members can’t adequately describe how their robot is put together, then that’s a red flag, indicating that the students may not have done all of the design work themselves (or possibly that the notebook entries may not be their own, exclusive efforts).
If the judges arrive at a team’s table and see that all 4 robots from their school are exact clones of each other, that’s another red flag that the design may not represent the exclusive work of the students on that robot team. (Note: it’s perfectly acceptable for all teams from the same school to have identical robots; however, they will not generally be considered for judged awards.)
At large tournaments, the Design Award can qualify a team for a spot at State Championships; however, that is not the case in many smaller local competitions.
See the Worlds section below for details on who is eligible and what happens there.
The Judges Award
This one can be summed up pretty succinctly (IMO): We thought you were great, but you didn’t fit into one of the other judging categories, but we thought you deserved an award anyway. PLEASE don’t take this as a put-down of this award—my team has a shelf full of them!
I have never been a judge myself—so I am speculating here—but I presume that we have won this award repeatedly because (a) we have a good engineering notebook (but not quite good enough to win the Design Award), (b) our team’s judges interview is generally great, (c) we have a good solid robot, (d) we’re generally good sports and interact with everyone in a respectful manner, and (e) our team does community outreach of various types.
The middle school team that we mentored for several years won the Judges Award in their first tournament ever: they had a Starstruck pushbot that was awesome; they had no engineering notebook whatsoever; and they made it to the finals. So an engineering notebook is definitely not a requirement for winning this award.
If your team was aiming for the Design Award or the Excellence Award and won the Judges Award instead (like us), it’s easy to feel like this is some sort of consolation prize. Disavow your team of this notion! “Hey, there were 36 teams at your tournament, they handed out only 3 awards, and you got one of them. That’s pretty awesome! There were 33 teams that did not win a judged award of any kind!”
The Excellence Award
I’ve left this one for last, because the team’s Design Award scoring factors into the judges evaluation for this award. Page 10 of the Judges Guide describes the key criteria for winning this award.
- Design award ranking
- Rank at the end of regular match play
- Skills challenge rank
- Rankings for other judged awards
- High quality VEX Robotics program
Judges are instructed to start with the top 5 contenders for the Design Award for their consideration. So the short version here is: if you’re not a contender for the Design Award, you will most likely not be the Excellence winner. The Judges guide states:
Judges will use their best judgement to choose the team they feel best exemplifies the best overall robotics program. … Would Judges want the team to be emulated by other teams?
It goes on to state explicitly that Judges should take into account the team’s behavior, sportsmanship, and professionalism—and that it includes students, mentors, and parents.
In most tournaments, the Excellence Award is handed out last, after the playoffs have determined the Tournament Champion, and in many (all?) tournaments, the Excellence Award will qualify a team for State/Regional Championships. (And the trophy is a *little bit* taller than all the rest.) See the Worlds section below for the details on the extra criteria required to be eligible for the Excellence Award at Worlds, as well as what happens at Worlds for teams in the running.
Building & Programming Awards: Amaze, Build, Create, Think, Innovate
OK, so lots of these awards sound very similar to each other, so here they are in a nutshell (described in detail in the VEX Awards Appendix). These awards are generally given out at very large tournaments, State Championships, and VEX Worlds. At large tournaments, when more than the standard 3 awards are given, we have seen just one or 2 of these awards given (such as Build and Think)—not the whole raft. In addition to the items described below, all of these awards take into account the team’s judges interview quality, professionalism, and teamwork displayed.
- Amaze Award. Kinda like it sounds. Your robot has to amaze the judges by being high-scoring, competitive, reliable/robust, with consistently successful autonomous code. A pretty high bar.
- Build Award. This is the “extra” award that we see most often here in Northern California. Its criteria stump me a little, as the description is sufficiently vague, but the criteria stress a professional, robust robot that holds up under competition conditions, with an elegant and efficient use of materials. The next bullet point they describe is that the robot was constructed “with a clear dedication to safety and attention to detail”; the “safety” part is a head-scratcher for me, honestly. Note that this criteria does not take into account robot performance, as the Amaze award does; there is no mention about it being competitive or high-scoring (though I assume those would help), but it does say that the robot needs to be reliable on the field and hold up under competition conditions.
- Create Award. This award is given to a team that has developed a creative solution to an engineering problem, a creative design process, or creative ways to play the game.
- Think Award. This one is pretty easy to describe; it’s for your team’s autonomous programming. The robot’s performance is one component of the judging, but the rest is made up of the team’s actual code (submitted in a 3-ring binder or similar with the Engineering Notebook), the team describing a clear programming strategy, and showing that it has a good programming management process (such as keeping track of version history).
- Innovate Award. This award recognizes “thinking outside the box” in terms of robot design: “Robot design demonstrates an ingenious and innovative piece of engineering.” This could be one specific component or subsystem of the robot, but it’s important that the innovative component is well-integrated into the rest of the robot, and that it is a well-constructed part of a well-constructed robot. Students should clearly state why they designed this interesting component—how is it related to their game strategy or some other restriction that makes it a good solution. Innovation-for-the-sake-of-innovation is not rewarded here.
VEX also publishes a judges scoring sheet (on pages 23 and 25 of the Judges Guide) that lays things all out in a grid. It’s a useful document to boil things way down. During the day, as judges interview teams, they put tick marks in the bold-titled-columns, indicating the rank of the teams that they speak to. If a new team they interview is superior to all the rest, they get 1 tick mark, and everyone else gets another tick mark added to their rank—the former #1 becomes #2 and so on. The same thing happens if the new team is better than, say #3 (but not better than #2). The new team gets 3 tick marks, and every team lower than them gets another tick mark added to their rank (so #3 becomes #4, and so on down the line).
Awards for Team Actions & Activities
Last, but not least, there are several awards for a team’s comportment at an event and interactions with other teams. I have seen the Sportsmanship Award given at local competitions, and the Energy Award given at state championships; I have never seen the remainder of these awards given at a local tournament, but they are given at VEX Worlds.
- The Sportsmanship Award is given to a team that demonstrates excitement and enthusiasm throughout the event; interacts with event volunteers in a respectful and helpful manner, is willing to help; and interacts with other teams in the friendly spirit of competition. We have often seen this award given to a team that finishes last (or close to it), but that has maintained their enthusiasm and good spirit nonetheless throughout the day. We’ve also seen this award given to a team-of-one that finished high in the rankings, so attributes other than being a good loser are clearly important.
- At VEX Worlds this award is given based on team voting; each team votes for the team they felt best demonstrated these qualities in their division.
- The Energy Award recognizes a team that shows a high level of enthusiasm, passion, excitement, and energy throughout the event. Additionally, this team’s passion for competition and robotics enhances the experience for other teams.
- The Inspire Award is given predominantly at World Championships to a team that “has inspired judges with their approach to competitive robotics” and communicates their passion for VEX and have a positive attitude throughout an event. This team must demonstrate high levels of integrity and sportsmanship. Additionally, this team shows that they “believe they can achieve what they set out to achieve through their diligence.”
- The Service Award is given to a team that is always willing to help other teams and share resources and knowledge. This team helps not just their alliance partners, but is of service to all teams, and has volunteered at local VRC events.
- At VEX Worlds this award is given based on team voting; each team votes for the team they felt best demonstrated these qualities in their division.
- The Teamwork Award recipient is a school or organization with multiple robots that has demonstrated cooperation, unity, and mutual respect among its robot teams, all season long.
- The Community Award is for a team making a difference in their community, supporting students and teams outside of their school.
- Partner/Volunteer of the Year is given to a person who has volunteered numerous hours to make local events happen and shows a devotion and commitment to their community. This award is given out at select local tournaments.
- Teacher of the Year goes to a teacher who shows true dedication and leadership, and ensures a valuable experience for all students.
- Mentor of the Year is given to a mentor who has helped students achieve goals that they thought were unattainable. This person is a role model who helps students expand their knowledge and solve problems in new ways.
See the Worlds section below for the time-sensitive, extra requirement that must be undertaken in order for an adult member to be considered for the Inspiration All-Star award at VEX World Championships (replaces former Mentor of the Year, Teacher of the Year, and Volunteer of the Year awards).
Special Rules for VEX World Championships
Read and understand this part—your team may be going to Worlds one day! VEX Worlds has specific instructions and requirements for certain awards that are different than/over-and-above what happens at local events, and they have hard deadlines associated with them, well in advance of the World Championship week.
Excellence Award at Worlds
First, only teams that have won an Excellence Award at their state/regional competition or at a “signature event” are eligible for this award at Worlds. So if you didn’t win an Excellence trophy on your road to Worlds, you don’t need to worry about these criteria, as you are not in the running (sorry.). REC knows from their computer system which teams are eligible, and automatically assigns them an interview slot at Worlds. Those teams must submit their engineering notebook upon check-in at Worlds.
The rules also note that the Excellence award is given to the entire school/organization, and not just to one robot team. Hence, even if multiple robots on your are eligible, your group will only be given one combined interview slot. The judges have a number of small (quiet) conference rooms outside the main expo hall in which to conduct team interviews.
Design Award at Worlds
Only teams that have won a Design Award or an Excellence Award at their state/regional/national qualifying event are eligible for this award at Worlds.
- As with the Excellence Award candidates, Design Award candidates also submit their engineering notebook upon check-in at Worlds.
- Once judges have reviewed all of the engineering notebooks submitted, the top-tier teams will have Design Award interviews with the judges in their pit areas (don’t call us, we’ll call you). There are no scheduled, sit-down interviews for this award; Excellence candidates are the only ones who sign up for an interview slot.
Special Requirements for Other Awards
All judged awards other than Design and Excellence that are given out at Worlds:
- Do not have scheduled interviews. All other awards only have interviews with judges as they walk around the pit areas, talking to teams in between matches. (At Worlds, there is generally a lot of time between matches, so it’s not the same squeeze-it-in-in-a-few-minutes-oh-my-god-we-have-to-go sort of thing that often happens at local events.)
- Do not involve an engineering notebook submitted upon check-in, or at any time during the event. Students may show their notebooks to the judges as they are interviewed at their pit location.
The Sportsmanship and Service Awards are given out by ballot at Worlds; each team present votes for the team in their division that they believe should win the award.
Mentor of the Year, Teacher of the Year, and Volunteer of the Year Awards are no longer presented at VEX Worlds. Starting in the Turning Point competition year (2018–19), they have been replaced by the STEM Hall of Fame – Inspiration All Star award (see page 10 of the Awards Appendix for criteria). Adults must be nominated for this award by students, parents, and volunteers via a 500-word essay. submitted online at robotevents.com/vexawards. Even though most teams will not know whether they are even going to Worlds at this point, the submission time period for nominations is September 1 – December 31, 2018. Late submissions will not be accepted.
Adherence to Award Criteria
What I’ve described here are the official VEX rules according to the judges guide and my personal experience about the competition judging and interview process. The Judges Guide says, in in bold on page 6:
Official events may not change award criteria from those listed below. Events not following the award criteria in this document will not qualify to higher level events..
From discussions on the VEX Forum, this statement does not appear to be universally adhered to, and when it does not, it usually causes an uproar online. At the same time, those talking on the Forum are not the judges who were in the deliberations, and no one after the fact can know what their exact rationale was for awarding Team X the Fill-In-The-Blank Award instead of Team Y.
At the same time, teams need to understand that the judges’ decisions are final, and that learning to accept answers that they don’t like is part of the process, however tremendously disappointing and frustrating it may be. For adults in particular, my recommendation is to try and really put themselves in the judges shoes for a bit. For everyone, I’d recommend going back and reading the judging criteria for the award in question very closely; what is being evaluated may be different, or much more specific than what they’ve assumed, or may not include the criteria they believe is in there (like robot performance or final rank).
If teams still feel like awards were given inappropriately (after re-reading the criteria closely and impartially), the best thing to do is to contact their local REC Foundation representative to discuss the issue in a calm, rational manner (waiting a day or two helps ensure this latter part). While doing so will not change the awards for the event in question, it will alert RECF that guidelines may not have been followed correctly, and that more event partner training or judges training is needed on these specific topics. A list of REC Foundation Regional Support Managers, including their email addresses, can be found about half-way down this page.
Multiple Awards for One Team
The Judges Guide says, at the bottom of page 8:
Awards are to be spread as equitably as possible among the teams, with no team winning more than one judged award. A team may win robot performance awards (Tournament Champion or Robot Skills awards) in addition to Judged awards, but no one team should win more than one Judged award.
This is a change in the 2018–19 season. Previous Judges Guides said that no team should win more than one judged award “if possible”, which resulted in varying interpretations of what “if possible” means. At one of the Starstruck tournaments my team attended, a home-team robot won Excellence and Think, out of a field of 60 robots. I have a hard time believing that no other robot in the place had autonomous programming good enough to qualify for the Think Award, and that the home-team Excellence winner was their only choice. It rubbed me the wrong way, and I am glad that they are now clear about this criteria.
Resources & Downloads
I’ve linked a number of VEX resources scattered throughout this post, so I thought I’d list them here in one location.
- Awards Appendix C. This is a must-have; it describes all of the awards in detail in one single document.
- Judges Guide. This document is designed for adults who are judges at an event. It includes all of the information in Awards Appendix C, along with starter questions for pit-area interviews as well as specific things that the judges are looking for, and a schedule of the judges’ day. It also includes:
- The Design Award Rubric is located on pages 20-21 of this document (it used to also be in the Awards Appendix, but as of this writing, it is ONLY available in the Judges Guide. If your team is interested in pursuing the Design Award, read this 2-page criteria before your write the first page in your engineering notebook.
- Scoring Sheet for Judged Awards are now included as pages 23 and 25 of this document. These pages show grids of the criteria for all judged awards, neatly fit into one location, with the concepts really boiled down.
- Award Descriptions for Judges’ Room have now also been incorporated into the Judges Guide, as pages 28-39. There is one page for each award, which have the criteria also boiled down to the essentials. At an event, these pages are taped to the wall in the room so the judges can easily reference it and put team numbers below it on post-it notes during their deliberations.
- Field Note to Judges. This new form is a way for volunteers and referees to communicate their observations about teams to the judges—positive or negative!
I hope that this encyclopedia of Judged Awards has laid things out a little more clearly for the novice coach, and that it will help you explain to your team the things that are important if they are aiming to win one of them.