Arduino data logging and speed test

I’ve been working on a robotics project for a while* that requires me to log data from various analog sensors, and I’ve been doing it all with an Arduino so far. The objective is to basically use the Arduino as a data acquisition board and avoid using a more expense NI DAQ with LabVIEW or MATLAB. I don’t want to write to an SD card, because I want to be able to run repetitive experiments and graph the data in real time (through Processing) or close to real time (in Excel). I really need to log data very quickly and am having some issues getting above 200 Hz, so I thought I would simplify things and get back to basics to try to find the maximum speed I can write to a file. So I tried just using a photocell hooked up like one of the FSRs here , then I tried a speed test to see just how fast I could get data to print. While you can see data scroll in the serial monitor screen of Arduino, you can’t save it to a file directly from there. I’ve found that CoolTerm is the easiest way to do this, and it’s available in Mac, Windows, and Linux versions (yes, I’m still a PC).

So here’s the initial code:

const int analogInPin = A0; // Analog pin that the photocell is attached to
int sensorValue = 0; // value read from the photocell

// print the results to the serial monitor:
Serial.print(millis());
Serial.print(',');
Serial.println(sensorValue);
}

And here’s a section of the initial output:
953,828
954,828
954,827
955,828
956,828
957,828
957,827
958,828

The millis() function outputs milliseconds of time elapsed, and as you can see above, I’m getting some readings at the same ms! Which means this is printing faster than 1000Hz. So, is there a way to get more resolution in the time function so each time stamp is associated with only one sensor reading? Turns out there is! The micros() function outputs in, you guessed it, microseconds- sort of. According to the reference:

On 16 MHz Arduino boards (e.g. Duemilanove and Nano), this function has a resolution of four microseconds (i.e. the value returned is always a multiple of four).

So then I checked the Arduino Uno hardware page (the version I’m using) to check if it is a 16Mhz version, and it is indeed. So, lets give this another shot and replace millis() with micros() in the above code.

1064,824
1908,824
2776,824
3664,824

Bingo! Now this time is going by so fast that two sensor readings never get logged to the same time reading. Okay, now to make sense of the data given the resolution…
I was confused by the resolution part of the statement above, but a user on the Arduino forum post I made cleared that up:

All it means is that, in effect, instead of incrementing a variable by one every microsecond, the variable is incremented by four every four microseconds.

So, there are 1,000 microseconds in 1 millisecond, and 1,000 milliseconds in 1 second, so there are 1,000,000 microseconds in 1 second. So all I have to do is divide each reading by 1,000,000 to get seconds. For example, the first time reading I get is 1024 microseconds = 1024/1000000 seconds = 0.001024 seconds. If I do that calculation for the first few data points, I get a delay of about 0.000888 seconds between readings, which means I’m logging data at about 1150 Hz. Not bad, but not great, given that analogRead() happens at at the rate of about 10,000 Hz. So can it get better?

One possible thing slowing down the write is using 3 different commands to print the microseconds, comma, and sensor. I remember someone telling me to write data out all at once in a string. I had a little trouble figuring out how to make a string of a number, a comma, and another number at first, but finally the code below worked:


const int analogInPin = A0; // Analog pin that the photocell is attached to
int sensorValue = 0; // value read from the photocell
String toprint;

// print the results to the serial monitor:
Serial.println(toprint);
}


However, the frequency of data writing actually decreased to about 885 Hz. So then, I posted the above on the Arduino forum and sent out a tweet. I LOVE twitter helpers! Big thanks to @talldarknweirdo for this insight:
1150 is about what you should be getting, here’s the math…
115,200 is your serial rate, thats approximately in bits. 9 bits to a character so 115,200/9=12,800
9 characters to a complete reading, 12,800/9=1422. you’re losing a tad to overhead.
now, analogread returns a value that is really only 10 bits long (0-1023 1,024=2^10) and Millis returns a value 32 bits long.
And to those commented on the other forum thread I posted here.

So it seems that 1150 Hz is about the fastest I can write serial data to a file without getting into some tricky optimization of code and lower level programming. Here are some references on that for those who are interested:

https://github.com/practicalarduino/ScopeDigitalOptimized (thanks @jonoxer !)
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1208715493/11
http://www.arduino.cc/en/Reference/PortManipulation

*This is an attempt at blogging as note-taking and blogging as code repository, as mentioned by Matt Mights. Hopefully this will be useful to other people while also giving me a searchable reference for the next time I try this!

Let me know if there’s something I didn’t try that can make the regular code better without going into lower level programming! Thanks