(Edit: this post was originally published on May 1, 2017; it has been updated on July 19, 2018 to reflect the current VRC Awards criteria.)
It’s the new season, and teams that are getting a jump-start on things may be about to start on their engineering notebooks. Or maybe you’re a new team and you’re wondering, what the heck is an engineering notebook, and why do we need one?
In a generic sense, the notebook is what laboratory scientists, inventors, and engineers keep to record their experiment/build/design activities, as a record for themselves or for others. It’s a “daily journal” of sorts, where all important and relevant information about the project is included. Can’t remember what that test result was from last week? Just flip backwards in your notebook, and there it is.
- Why do I need one for VEX?
- (My) recommended format: A bound book
- Read this before writing page 1
- During the design & build process
- Daily entries
- Competition results
Why Do I Need One for VEX?
First off, it’s just a good idea; that’s why scientists and inventors throughout the ages have used them. For a robotics team, it also serves as a way to transfer information between team members if Person A did something last week, but isn’t here today, and Person B is picking it up now. On larger teams, often Person A is in charge of XYZ (e.g., CAD) and Person B is in charge of ABC (e.g., programming); everyone putting their information into the notebook is a way to collate all of that activity and knowledge.
From a trophy shelf-perspective, it’s hard to get any trophies for said shelf without an engineering notebook, straight up. If you are fortunate, your team can rack up tournament champion and skills champion bling, but almost all judged awards will expect to see a high-quality engineering notebook for your team to really even be in the running. The most obvious award relying on the notebook is the Design Award, but the Excellence, Build, Create, Innovate, Amaze, and Think awards all rely on the notebook as well. You are unlikely to beat out another team for one of these awards sans engineering notebook (and it’s required for Excellence, to boot). Judges want to see your team’s thought process, project management, testing, iterations, etc.
How Does It Get Judged?
At a typical competition, teams with an engineering notebook hand it in when they check their team in first thing in the morning; judges (praise the volunteer judges!) spend all day reading through them and (combined with the other award criteria) make their award determinations. At the very end of the day, generally when match play or alliance selection is complete, teams pick up their notebooks from some central location. So if there’s any vital information in the engineering notebook that your team needs during the day at a competition, be sure to have them write it down in a separate location, as they will not have any access to their notebooks during the tournament.
The Design Award has very specific criteria to evaluate the notebooks; that criteria can be found on pages 20-21 of the Judges Guide. If you are not familiar with this rubric, go and download it right now.
(My) Recommended Format: A Bound Book
There has been many, many discussions on the VEX Forum about whether a bound notebook is superior to a 3-ring binder, whether handwritten or typed is better, etc. (Search for “engineering notebook” on the VEX Forum if you’re inclined to read them.) The REC official description says, “A bound quad-ruled notebook is the preferred format. … The notebook should never be edited.” This simple description includes the words “preferred format”, which has been the source of much online disagreement. While that phrase remains unchanged, as of 2017–18 the Design Award Rubric now gives a 3-point bonus for a bound notebook in lieu of a 3-ring binder. Some teams still find, for many different reasons, that the binder approach works better for their team, and will continue doing things that way, with the understanding now that they will not get the 3 bonus points.
This clarification in the official scoring document will serve to alleviate some judging disparities and variability between events. At one Starstruck state championship event, the judges refused to accept any notebooks that were not bound; all teams that had maintained 3-ring notebooks throughout the year (and had been submitting them for evaluation at tournaments all year long) were immediately out of the running for most judged awards at that tournament. Needless to say, many teams were devastated (and irate). Now, judges will have no reason to reject a 3-ring binder or to assume that it is not allowed, and will be one step toward greater judging consistency.
My personal recommendation is to use a bound composition book from the get-go. REC even sends you one for free when you register your team! In my view, the benefit of a handwritten notebook is that it shows that the work was done by students, over time; printed materials in a 3-ring binder could be done by an “over-helping” adult for all they know (or all be done by one single team member), and could include retroactive additions which sort of negate the spirit of a laboratory notebook recording daily work and observations. But in the most practical sense, my team would use a bound notebook simply to get the 3 bonus points (hey, every little bit helps).
But what about stuff that doesn’t really fit into a bound notebook, like computer code? Several coaches on the VEX World Coaches Association Facebook group (which you should join if you’re a mentor reading this) said that their teams also maintain a small 3-ring binder with that other stuff (including outreach efforts, code, and anything else that’s many pages and/or not directly related to the design/build process), and rubber-band the two together and submit both at tournaments. Or if your team is using the REC-provided notebook, which is 3-hole punched, they can put the engineering notebook in the 3-ring binder and submit it all that way.
Read This Before Writing Page 1
The Design Award rubric (pages 20-21 of the Judges Guide) is crystal clear about what scores points for this award. The thing that our team did not realize in our rookie season is that there is stuff that must be at the beginning of the book if you want to score points in certain categories. Adding it later won’t work. [Note: the information below is all related to the first page of the Design Award rubric; the second page of the rubric is based on the team’s interview, which is not covered here.]
Table of Contents
In our rookie year, we used color-coded post-it flags in our book to flag major events in lieu of a TOC, but that doesn’t cut it. The REC-provided notebook (as well as commercially available lab notebooks, like the one we use now) includes a built-in TOC to make life easy for you. If your team’s book doesn’t include a pre-printed TOC, leave the first few pages blank for this purpose. The Awards Appendix states on page 5:
Students should include a number of items in their Engineering Notebook including: a table of contents …
That does make it pretty crystal clear…
While not specified in the award criteria, our engineering notebook includes a team introduction right at the start, with a group photo and description, followed by 3-to-a-page team member bios and photos (student age, school, interests).
At the bottom of each team member’s bio, we make sure to specify the team member’s role(s). We have a team with only 5-6 people on it, and in our rookie year we thought this was kind of stupid to give people specific roles, since we only have 6 people and everyone was involved in many aspects. HOWEVER, judges want to see specific roles identified as a sign of project management capabilities; so invent roles if you have to (“battery boy” was one role mentioned by a Worlds Excellence Award winner).
We are an all-girls team, and so we decided that the primary person responsible for a task, say, programming, would be Programming Queen, and the secondary person would be Programming Princess. Anyone else would be Programming Peon. Each person on our team had multiple roles, and if they are listed at the bottom of the person’s bio, it’s easy to add more as the year goes along.
The Design Award rubric states explicitly that it wants roles assigned in the “Resources” line-item:
The notebook shows good use of human resources by assigning members roles based on their strengths.
In our team section, we also include a list of our team’s goals for the coming year, which we decide as a group even before we start talking about the new game!
Design Process: Challenge
This was one we totally blanked on our first year. In the Design Award rubric, full points are scored for this line-item if the notebook
describes the challenge at the beginning of the notebook with words and pictures and states the team’s goals toward accomplishing that challenge
Whaa?? You have to describe the game? Don’t we already all know what the game is??? What does this really mean?
This step is actually a little microcosm of the real engineering world. The very first step in any project is to really, really understand what it is you’re tasked with doing. One of the coaches on the fabulous World Coaches Association Facebook group says:
One adage is … the most expensive mistakes are made on the first day of the [project]. If you have a great product but solve the wrong problem, you have failed—so understanding the problem is key.
So this is what the rubric is trying to encourage in this step: making sure that teams fully understand every aspect of the challenge—rules, scoring, construction requirements, and so on.
Only from that point can teams start an informed analysis of game strategy: “What is my team going to choose to pursue in this game, given these requirements?” Depending on the given challenge, a robot could focus on de-scoring, or could focus on picking up game objects the fastest, or picking up certain game objects the fastest and ignoring others. Another avenue of thought is, “What would make us a good alliance partner?” That discussion could include an evaluation of the more common strategies compared to scoring options (or robot behavior) that are valuable but not the obvious choice, including defensive strategies. None of those decisions, however, can be made with purpose and authority without fully understanding all aspects of the challenge.
Next, make sure your team writes it all down. You may be talking about this stuff for hours, and your team may do a great, thorough analysis of the game, but the judges will never know it unless they put it in the book, in their own words; simply regurgitating the rules as written in the manual does not convey the same level of mastery.
That same coach from the Facebook group also says:
This is the Systems Engineering process I do ALL THE TIME. (1) Understand the requirements (2) List options (3) Select option (4) Build option.
So this part of the rubric includes step 1 of the Systems Engineering process: Understand the requirements. Step 2, List options, is below (a.k.a. brainstorming).
Design Process: Brainstorming
Our first year, we also blanked on this one. Full points in this category are awarded if the notebook
generates an extensive list of possible approaches to the challenge with labeled diagrams
Our first year we weren’t so robust with the brainstorming, but it also never occurred to us to document the road not taken. Judges want to see that your team considered a lot of different ideas and didn’t just charge off and build the first thing they thought of. Be sure to include descriptions and sketches (they don’t have to be architecturally beautiful blueprints, just sketches) of each thing your team dreams up. If your engineering notebook is a graph-paper-lined book, this makes drawing in the book easier; if not, your team can draw them separately on graph paper and tape them into the book.
Make drawings in pencil, and then draw over them with ink when finalized.
Design Process: Select Approach
Here comes Step 3 of the Systems Engineering process: Select option. After your team’s brainstorming notebook entries, the next thing that needs to be in included is an evaluation of its various ideas from the brainstorming process. Full points are given if the notebook
explains why the selected approach was chosen, and why the other alternatives were not chosen
Again, note that last part—documenting the road not taken.
Include updated sketches if the design idea chosen has materially changed from the drawings previously included, or if the previous drawing is buried in earlier pages and is difficult to locate. In other words, make it easy for people to find and reference this drawing; it will be your team’s official starting design concept.
I recommend reading this page on the VEX Online Curriculum, 1.3: What Is the Engineering Design Process? This page has a WEALTH of information about all of these initial steps, including the concept of a weighted objectives table to make decisions. I learned a TON from this one page alone.
Project Management & Timeline
At this stage, your team should make a timeline that extends from now until your first tournament (or approximate date, if you don’t know yet), laying out exactly what should be accomplished at each stage. A Gantt chart is one idea, but in general, the more detail the team can include here, and the more foresight that is shown, the better. Full credit in the “Resources” category is given if the notebook
shows the team’s efficient use of time with an overall project timeline. The team uses checkpoints to help them know how well they are staying on schedule and readjusts their schedule as needed.
After your first tournament, your team may find that it needs another timeline, for modifications or programming that need to be completed between now and the next tournament. Be sure the team includes all subsequent timelines and project plans in the notebook.
During the Design & Build Process
Step 4 of the Systems Engineering process: Build the option you have selected. Here’s where achieving full credit gets a little crazy. Two separate line items in the rubric that net your team 3 points each are as follows:
- Design Process: Build and Program. The notebook
records the building and programming process in such detail that someone outside the team could recreate the robot by following the steps in the notebook.
- Usefulness. The notebook
is such a detailed account of the team’s design process that the reader could recreate the project’s history
Seriously. Could recreate the robot by following the steps in the notebook. I’m not sure how many teams achieve the 3 points in this category, but that is the standard against which teams are measured.
During the season, make sure that your team makes entries every day that you meet. We refer to our engineering notebook as “the journal” to emphasize its daily nature. Even if you did teeny-tiny amounts of stuff, it needs to be in there. Recommended for each day’s entry:
- Today’s date
- Meeting attendees
- Goals of the day’s meeting. Go back to the overall project timeline that the team created after the brainstorming process, and be sure to include those steps in the meeting notes, or any sub-tasks developed from the overall timeline.
- What the team did in the meeting today, including drawings or photos if applicable. Include as much data as you can, and try not to be complacent as the year goes on.
- Instead of “We had drive practice today,” write “We had drive practice today and can now score 20 points in 1 minute.” Or, “The lifting arm stalls when we lift xx pieces, and here’s what we will do about it.”
- When we were working on driver skills, one person not on the drive team would record each skills run in the notebook as the meeting was taking place, with a chart of how many points were scored, the time used, and notes about what went wrong or what needs fixing.
- At the end of the entry, make note of the goals not completed that will be carried over to the next meeting.
- The person writing the entry should sign & date the bottom of the page.
- Fill up the large white spaces on the page with diagonal lines (prevents unscrupulous scientists from adding content retroactively).
- Write in pen; if there are errors, just make a one-line strikethrough.
Make sure that your team’s members all take turns writing entries. It doesn’t help your team in the “Teamwork” scoring section to have all entries written in one handwriting, even if it is the neatest.
Test & Redesign
The Design Award rubric gives 3 points if the notebook
Describes in great detail the process of troubleshooting, testing, and redesigning through all iterations (cycles) of the process.
Once your team has a functioning robot, it’s easy to let the notebook entries become more generalized/less detailed. Be sure that your team keeps up the great work have done through the game analysis and initial build process. Instead of “Today we worked on fixing the arm”, a better description would be to describe exactly what the problem is, different ways you discussed to fix it, and a sketch or photo of the problem area. Two points are given in this area if the notebook
Captures the key results of the troubleshooting, testing, and redesign cycle.
You can see from these descriptions that including full documentation of the outcome/results of the testing and redesign process gets you part of the way there; being really specific and detailed about it gets that extra point.
I found it helpful to dedicate the last 15 minutes of the scheduled meeting to journal-writing and clean-up; everyone is doing one or the other, and we can all leave on time, and we don’t fall behind in journal entries.
But someday your team will fall behind. In that case, we found it useful to take notes (as detailed as possible in the time available) on a piece of scratch paper, and tuck it into the journal so that (hopefully) the next meeting, someone could enter them. The more you can avoid falling behind, the higher-quality your entries will be. If your team is 4 meetings behind, whoever takes the journal home to enter all the back-content will end up doing a less-thorough job than if those entries were done one at a time, in the lab, on the day of the meeting.
If you find one team member has nothing to do in the middle of the meeting, they can certainly get that day’s journal entry started, with attendees, meeting goals, and what has been done so far.
Don’t forget to include entries written by your team’s programmers. It’s easy to include only build/design entries, and forget to document the programming and autonomous testing process. It’s useful to include pseudo-code in your journal entries (full code printouts can be included in your add-on 3-ring binder), as well as keeping track of the values used for important variables or constants. These latter items tend to shift over time, and unless they are documented in the engineering notebook, no one will remember that holdPOT was originally 450 but now it’s 520, and hence will not realize that important shifts have taken place in the structure or functioning of the robot.
Be sure to document your team’s performance at competitions, and make sure that competitions are easy to find in the TOC. We always download our individual match scores, skills scores, and final competition and skills rankings into Excel, and print them out in a nice format and tape it into the book. We also make note of any awards received, or what alliance we were on in the playoffs.
Starting with the Nothing But Net season, we had a parent with a clipboard tracking exactly what our robot did during the match: the number of shots taken with regular balls, bonus balls, and driver loads, and whether they went into the high or low goal (or missed), and what happened in auton. [You can download a copy of our scoring sheet to see what we keep track of.] We included these statistics for each match in our journal as well. (Detailed statistics were a little hard to document in Starstruck, which was more like a game of Hot Potato.)
Teams should also include a “post-mortem” after each competition to say, in words, what went well, what was an epic fail, what required fixing, etc. at the tournament, as well as what they plan to do about it between now and the next competition (which may also involve an updated timeline for project management).
♦ ♦ ♦
I hope this post reaches some newbie teams in time to include some of those front-of-the-book items before diving in to documenting the build process in your engineering notebook, and I hope it has taken some of the mystery out of what it’s all about.