When I learn something new, all you readers get to learn it too. 🙂 This week I learned about RobotC’s datalog functionality. Thanks once again to the coaches on the VEX World Coaches Association Facebook group for guiding me along here. The datalog is one of the multitude of benefits I’m experiencing with RobotC over easyC, which we have used up to this season. [This post assumes readers are familiar with the Debugger windows already. I’ll write another post about that separately.]
So what is the datalog? In short, it’s print-to-screen on steroids in the most helpful way.
With easyC, you’re limited to either making good use of the LCD screen or having your robot tethered to your computer and doing a lot of print-to-screens, which zoom by at a million miles an hour, with data that’s really hard to get into a spreadsheet afterward.
RobotC’s datalog feature allows you to track up to 8 pieces of data (variables, sensor values, etc.) as the program executes, with super-easy, one-click exporting to a CSV file and automated, customizable graphs.
Since I’ve been using this for like a week, I’m sure I have only scratched the surface of what it’s capable of. The way I’ve been using it so far is to track data during the execution of a while loop. I happened to be working on writing a PID function, but you could use this feature any time you want to be able to look back through the execution of the program to see what *really* happened at each step.
- Finding the help page documentation
- Programming it
- The numerical output
- Wireless yay!
- Tuning a PID, and the mysterious kU and pU
- My use so far, and an epiphany
The Help Page’s Secret Hiding Place Revealed
One thing that has prompted me to write this post is that I find the RobotC datalog documentation difficult to locate via Googling, so I wanted to make finding it easier for everyone else. So, start here on the RobotC help page for VEX: http://help.robotc.net/WebHelpVEX/index.htm. I include the full path here instead of embedding the link because RobotC has various different wiki/help sections, and as shown in the URL, this one is specifically for VEX.
The confusing part of finding this datalog information is not done yet; from the main help page, in the left-hand column is a table of contents, and one of the items at the top level of navigation says “ROBOTC Debugger”. This is NOT the place to find datalog information (thanks a lot, RobotC). Instead, as shown in the screenshot below, you’ll want to navigate to Command Library – VEX Cortex > ROBOTC > Datalogging.
The datalog feature only works if you add code to your program to tell it to keep track of the stuff you want. The RobotC functions for the datalog can be found in the left-hand column of the help page above. There are not that many items to choose from. So far I’ve only used the datalogAddValue function, which I’ll talk about here. (There’s also datalogAddValueWithTimeStamp, which does the same thing with — duh — a time stamp, plus a few others I have not used.)
The datalogAddValue() function does just what it says: it adds data to the output listing. To make it work, these calls have to be sandwiched in between a “start” and an “end” instruction, like so, with one “add” statement for each variable you want to track:
datalogDataGroupStart(); datalogAddValue( 0, error ); datalogAddValue( 1, integral ); datalogAddValue( 2, derivative ); datalogAddValue( 4, SensorValue[rightEnc] ); datalogDataGroupEnd();
This block of code is inside the while loop I’m using to create my PID function, so that each time through the loop, it adds another row to the data with the current iteration’s values. A few specifics:
- The datalogAddValue() function takes 2 parameters: the index of the thing you’re keeping track of, and the thing you’re keeping track of.
- You can include variables or stuff like joystick and sensor values, as shown in this example.
- You can track up to 8 items; the index for each follows the usual way of counting the position in a matrix, so the possible values are 0 to 7 (and not 1 to 8).
- You can have blank columns if it helps you look at the output data. In the example above, there will be a blank column at index 3, because I found it easier to look at the output with error, integral, and derivative right next to each other, and then a blank column, and then show the encoder counts in the 5th column (index 4).
- The DataGroup start / end “bookends” do not take any arguments/parameters.
The Numerical Output
Finding the Datalog
In order to see any of the data you just recorded, you’ll need to set the correct debugger window to show. Go to Robot > Debugger Windows, and make sure that “Datalog” is selected.
Once you download a program with datalog code and have the appropriate debugger window selected to display, your screen will look something like this:
Once you run your program, you’ll get something like this, where you can scroll through the entire run and look at the data in the columns you’ve selected:
Doing Stuff with It
First you’ll need to expand the small “Program Debug” window by clicking “Show Datalog”; you’ll then see the options at right.
One great feature of the datalog is that you can 1-click export to CSV. In the lower-right corner of the popup window, click “Save Data Log“, and a window will open, allowing you to save the data to a CSV file. This is a good way to keep data from different runs, without them getting confused or merged, or if you want to do any analysis of what happened.
On the Program Debug small popup window, underneath the Start/Stop button, is the “Clear All” button. If you don’t click this in between your different test runs, the system will just concatenate your new data to the bottom of the old data. Since you can only see the top few lines of code in the Datalog window, if you don’t Clear All in between runs, then it will either look like your code changes had no effect, since the numbers are the same as last time, OR it will look like the data you expect, but really your most recent run’s data is way down the output listing.
In my runs so far, I’ve followed this process in order to just get the data I’m looking for:
- Download new program
- Click “Clear All” (which clears out encoder counts too, in case the robot got moved a little in repositioning)
- Click “Start”
- When the robot is done doing its thing, click “Stop”
- When I no longer want the given data (or have saved it to a file), click “Clear All” again
- Repeat and fade
The other cool-and-unbelievably-simple-to-use item is the “Show Graph” button. Seriously, you click the button, it shows you the graph (below). You can choose which pieces of information to show in the graph, which is useful if your data are on different scales, such as motor power and total encoder clicks travelled.
Saving Graphs and Data
It’s extraordinarily simple to save a graph and/or its associated data; RobotC has been kind enough to make buttons to do just that. These buttons save the graph and data for just the series on the screen now. You can save the output that is the most useful for your needs, and not get stuck with graphs showing all of your data. You can slice-and-dice your data to spit out each series in a different graph PNG and/or CSV if that’s what is most helpful. (The “Save Data Log” button mentioned above, on the other hand, spits out one file containing all the data.)
“Experiment” Tabs Above the Graph
But wait! There’s more! Immediately above the graph, there are tabs labeled “Experiment 1” through “Experiment 5”, which make the graph function even awesomer. You can keep up to 5 graphs of different trials and flip between them. It’s my dream come true. To use them:
- Click to a new, blank Experiment tab; do not close the graph window in between trials
- Click the “Clear All” button in the small popup window; as far as these tabs are concerned, “Clear All” actually means “clear all data and *this* experiment tab only” (i.e., it doesn’t actually mean “all”)
- Download your program and click “Start” from the debugger small popup window
- Run your robot
- Click “Stop”
- The Experiment tab you’re on will still be blank; click the “Show Graph” button from the popup window and boink! there it is.
- Now you can click back and forth between different runs to see via pictures what impact your last changes have made.
- Once you fill up all 5 tabs, never fear, you can still keep going! Actually, at any time on any graph, you can click to an Experiment tab, click “Clear All”, the existing graph will disappear. Do your run; click “Show Graph” as usual, and your new info is on this Experiment tab.
- A word to the wise: Keep notes (hmm…maybe in your engineering notebook) of which data or parameters are associated with which tab. Once you loop around past 5 graphs you’re going to have to start re-using those tabs, and things get muddled quickly.
Note: I have not yet played around with the “Analysis” tab that comes after “Experiment 5”; I’ll add that at a later date if it is of use.
One of the reasons my team switched to RobotC is the ability for wireless downloading (see my other post on that). I can’t shout loudly enough about how amazing it is in this process of running something over and over while making small changes (which would include auton programming too). Without the wireless option, every single iteration (“What if we changed parameter xyz from 0.9 to 0.85?”) requires getting the computer & robot together, unplugging the VEXnet key, plugging in the orange cable, downloading, VEXnet key back in, wait for joysticks to connect, get robot back to where it should be, and maybe throw in a power cycle.
Furthermore, without wireless, if you want to use this datalog feature, you need to keep your computer connected to the robot with the orange cable as it’s driving around, which involves some gymnastics that are potentially unsafe for the computer and team member holding the computer. With wireless, RobotC maintains connection to the robot continuously, so you can run your program, look at your datalog, make some adjustments, re-download, and run again without getting off the couch. (I’m all for not getting off the couch whenever possible.)
The new V5 system coming out in the next few months will have wireless downloading built in (super-Yay!), and it will also work on a Mac (also yay!). But if you’re not planning to upgrade right away, I recommend acquiring the “programming hardware kit” that makes wireless downloading work. It’s not cheap ($50), but given that many people will no longer have a need for them after switching to V5, I’m guessing you can pick one up on eBay for cheap (that’s where I got ours last year, for like $25, essentially brand new).
Tuning a PID, and the
Mysterious kU and pU
If you’ve read George Gillard’s excellent document, “An Introduction to PID Controllers,” and you’ve read all the way to the end, you’ve seen his description of “Mathematical Tuning” on page 15 (in lieu of trial & error tuning). He walks through the “Ziegler-Nichols” method (in case that’s a meaningful piece of information for you), and it starts as follows:
. . . Just as for the trial & error method, begin by disabling all three constants (set them to zero).
1. Increase kP until you get steady continuous oscillations. These need to be stable and consistent. Record this value as “kU” (ultimate or critical gain).
2. Measure the period of the oscillations caused by step 1. That is, the time it takes to do a full cycle from a point back to itself. You can measure this with a bit of programming, or with a stopwatch. It’s a good idea to measure many oscillations and then divide the time by the number of oscillations to get a good average. Record this number as “pU” (period for ultimate or critical gain).
On first reading—before I was aware of the graphing functionality of RobotC—step 1 seemed a little dicey. I mean, from looking at the robot, it’s hard to say whether the oscillations are “stable and consistent.”
Then you get to step 2 and it all goes out the window. You’re supposed to figure out how long one oscillation is. With your eyes and a stopwatch, this seems a highly unlikely prospect. If you’ve got a very good Kp, the oscillation is very small; it’s pretty hard to say what is the start or end of one. And, it relies on a person hitting “start” and “stop” on a stopwatch accurately.
As you can see from the graph above (under “Saving Graphs and Data”), the red line is my robot’s error as it drives. It’s quite a simple thing to get a very close measurement of kU (once I get past the “stable and consistent” step), and be able to follow the rest of the calculations used in this method.
This is a brand new day for me, people. I really had no hope of ever trying to do a mathematical tuning until I saw what you can get from these graphs. PID, here I come.
My Use So Far, and an Epiphany
As I said above, I’ve only made use of this feature so far to “tune” a PID algorithm for keeping left & right sides of a chassis driving straight; that is, I was trying to figure out what the parameters Kp, Ki, and Kd should be (via the “trial & error” method).
I had an epiphany while I was doing this. I went through many iterations, as one is wont to do in this process, and during each one I would stare at the robot and see if it was driving straight across my dining room floor or oscillating, etc., and then I would turn back to the computer and scroll through the data to see what my error was each time through the loop. Then I realized that looking at the robot in the very-fine-tuning stage is basically unnecessary—one need only look at the output data to see if your parameters are good enough. You don’t have to figure it out visually by watching the robot as it moves. This is something I had just never considered before, and it’s completely awesome.
My team tried to use PID in easyC for our Nothing But Net flywheel (it didn’t work for a variety of reasons) and in junior high my older daughter made a line-following robot for her science fair project, which of course used PID. In both of these instances the process was excruciating. Every teeny adjustment involved the whole download plugging–unplugging process, and then a lot of guessing as to what the next adjustment should be. Both adventures caused us all to view PID as too overwhelming to think about.
Having just gone through this process on our In The Zone robot, I can’t believe how easy it was. Yes, it will take a little bit of work for me to teach my team how to write the algorithm itself, but once they’ve got that down, they’ve got a piece of code that can be used again and again when the need arises, with the main work then being figuring out some new parameters.
If you haven’t yet tried out the RobotC datalog, I suggest giving it a go!