Category Archives: * Developing NEW sensors *

I’m developing a family of environmental monitors for use in caves and underwater, but the basic three component logger platform will support a wide range of different sensors.

How to make ‘High-Resolution’ Sensor Readings with DIGITAL I/O pins

AN685 figure 8: The RC rise-time response of the circuit allows microcontroller timers to be used to determine the relative resistance of the NTC element.

For more than a year we’ve been oversampling the Arduino’s humble ADC to get >16bit ambient temperature readings from a 20¢ thermistor. That pin-toggling method is simple and delivers solid results, but it requires the main CPU to stay awake long enough to capture multiple readings for the decimation step. (~200 miliseconds @ 250 kHz ADC clock) While that only burns 100 mAs per day with a typical 15 minute sample interval, a read through Thermistors in Single Supply Temperature Sensing Circuits hinted that I could ditch the ADC and read those sensors with a pin-interrupt method that would let me sleep the cpu during the process. An additional benefit is that the sensors would draw no power unless they were actively being read.

Given how easy it is drop a trend-line on your data, I’ve never understood why engineers dislike non-linear sensors so much. If you leave out the linearizing resistor you end up with this simple relationship.

The resolution of time based methods depends on the speed of the clock, and Timer1 can be set with a prescalar = 1; ticking in step with the 8 mHz oscillator on our Pro Mini based loggers. The input capture unit can save Timer1’s counter value as soon as the voltage on pin D8 passes a high/low threshold. This under appreciated feature of the 328p is faster than typical interrupt handling, and people often use it to measure the time between rising edges of two inputs to determine the pulse width/frequency of incoming signals.

You are not limited to one sensor here – you can line them up like ducks on as many driver pins as you have available.  As long as they share that common connection you just read each one sequentially with the other pins in input mode. Since they are all being compared to the same reference, you’ll see better cohort consistency than you would by using multiple series resistors.

Using 328p timers is described in detail at Nick Gammons Timers & Counters page, and I realized that I could tweak the method from ‘Timing an interval using the input capture unit’ (Reply #12) so that it recorded only the initial rise of an RC circuit.  This is essentially the same idea as his capacitance measuring method except that I’m using D8’s external pin change threshold rather than a level set through the comparator. That should be somewhere around 60% of the rail, but the actual level doesn’t matter so long as it’s consistent between the two consecutive reference & sensor readings. It’s also worth noting here that the method  doesn’t require an ICU peripheral – any pin that supports a rising/ falling interrupt can be used. (see addendum for details) It’s just that the ICU makes the method more precise which is important with small capacitor values.

Using Nicks code as my guide, here is how I setup the Timer1 with ICU:

float referencePullupResistance=10351.6;   // a 1% metfilm measured with a DVM
volatile boolean triggered;
volatile unsigned long overflowCount;

ISR (TIMER1_OVF_vect) {  // triggers when T1 overflows: every 65536 system clock ticks
overflowCount++;
}

 

ISR (TIMER1_CAPT_vect) {     // transfers Timer1 when D8 reaches the threshold
sleep_disable();
unsigned int timer1CounterValue = ICR1;       // Input Capture register (datasheet p117)
unsigned long overflowCopy = overflowCount;

if ((TIFR1 & bit (TOV1)) && timer1CounterValue < 256){    // 256 is an arbitrary low value
overflowCopy++;    // if “just missed” an overflow
}

if (triggered){ return; }  // multiple trigger error catch

finishTime = (overflowCopy << 16) + timer1CounterValue;
triggered = true;
TIMSK1 = 0;  // all 4 interrupts controlled by this register are disabled by setting zero
}

 

void prepareForInterrupts() {
noInterrupts ();
triggered = false;                   // reset for the  do{ … }while(!triggered);  loop
overflowCount = 0;               // reset overflow counter
TCCR1A = 0; TCCR1B = 0;     // reset the two (16-bit) Timer 1 registers
TCNT1 = 0;                             // we are not preloading the timer for match/compare
bitSet(TCCR1B,CS10);           // set prescaler to 1x system clock (F_CPU)
bitSet(TCCR1B,ICES1);          // Input Capture Edge Select ICES1: =1 for rising edge
// or use bitClear(TCCR1B,ICES1); to record falling edge

// Clearing Timer/Counter Interrupt Flag Register  bits  by writing 1
bitSet(TIFR1,ICF1);         // Input Capture Flag 1
bitSet(TIFR1,TOV1);       // Timer/Counter Overflow Flag

bitSet(TIMSK1,TOIE1);   // interrupt on Timer 1 overflow
bitSet(TIMSK1,ICIE1);    // Enable input capture unit
interrupts ();
}

With the interrupt vectors ready, take the first reading with pins D8 & D9 in INPUT mode and D7 HIGH. This charges the capacitor through the reference resistor:

//========== read 10k reference resistor on D7 ===========

power_timer0_disable();    // otherwise Timer0 generates interrupts every 1us

pinMode(7, INPUT);digitalWrite(7,LOW);     // our reference
pinMode(9, INPUT);digitalWrite(9,LOW);     // the thermistor
pinMode(8,OUTPUT);digitalWrite(8,LOW);  // ground & drain the cap through 300Ω
LowPower.powerDown(SLEEP_30MS, ADC_OFF, BOD_ON);  //overkill: 5T is only 0.15ms

pinMode(8,INPUT);digitalWrite(8, LOW);    // Now pin D8 is listening

set_sleep_mode (SLEEP_MODE_IDLE);  // leaves Timer1 running
prepareForInterrupts ();
noInterrupts ();
sleep_enable();
DDRD |= (1 << DDD7);          // Pin D7 to OUTPUT
PORTD |= (1 << PORTD7);    // Pin D7 HIGH -> charging the cap through 10k ref

do{
interrupts ();
sleep_cpu ();                //sleep until D8 reaches the threshold voltage
noInterrupts ();
}while(!triggered);      //trapped here till TIMER1_CAPT_vect changes value of triggered

sleep_disable();           // redundant here but belt&suspenders right?
interrupts ();
unsigned long elapsedTimeReff=finishTime;   // this is the reference reading

Now repeat the process a second time with D7 & D8 in INPUT mode, and D9 HIGH to charge the capacitor through the NTC thermistor:

//==========read the NTC thermistor on D9 ===========

pinMode(7, INPUT);digitalWrite(7,LOW);     // our reference
pinMode(9, INPUT);digitalWrite(9,LOW);     // the thermistor
pinMode(8,OUTPUT);digitalWrite(8,LOW);  // ground & drain the cap through 300Ω
LowPower.powerDown(SLEEP_30MS, ADC_OFF, BOD_ON);

pinMode(8,INPUT);digitalWrite(8, LOW);    // Now pin D8 is listening

set_sleep_mode (SLEEP_MODE_IDLE);
prepareForInterrupts ();
noInterrupts ();
sleep_enable();
DDRB |= (1 << DDB1);          // Pin D9 to OUTPUT
PORTB |= (1 << PORTB1);    // set D9 HIGH -> charging through 10k NTC thermistor

do{
interrupts ();
sleep_cpu ();
noInterrupts ();
}while(!triggered);

sleep_disable();
interrupts ();
unsigned long elapsedTimeSensor=finishTime;   //this is your sensor reading

Now you can determine the resistance of the NTC thermistor via the ratio:

unsigned long resistanceof10kNTC=
(elapsedTimeSensor * (unsigned long) referencePullupResistance) / elapsedTimeReff;

pinMode(9, INPUT);digitalWrite(9,LOW);pinMode(7, INPUT);digitalWrite(7,LOW);
pinMode(8,OUTPUT);digitalWrite(8,LOW);  //discharge the capacitor when you are done

The integrating capacitor does a fantastic job of smoothing the readings and getting the resistance directly eliminates 1/2 of the calculations you’d normally do. To figure out your thermistor constants, you need to know the resistance at three different temperatures. These should be evenly spaced and at least 10 degrees apart with the idea that your calibration covers the range you expect to use the sensor for. I usually put loggers in the refrigerator & freezer to get points with enough separation from normal room temp. – with the thermistor taped to the surface of a si7051.  Then plug those values into the thermistor calculator provided by Stanford Research Systems.

Just for comparison I ran a few head-to-head trials against my older dithering/oversampling method:

°Celcius vs time [1 minute interval]  si7051 reference [±0.01°C resolution] vs. Pin-toggle Oversampled 5k NTC vs. ICUtiming 10k NTC.   I’ve artificially separated these curves for visual comparison, and the 5K was not in direct contact with the si7051 reference, while the 10k NTC was taped to the surface of chip – so some of the 5ks offset is an artifact. The Timer1 ratios delver better resolution than 16-bit (equivalent) oversampling in 1/10 the time.

There are a couple of things to keep in mind with this method:

si7051 reference temp (C) vs 10k NTC temp with with a ceramic 106 capacitor. If your Ulong calculations overflow at low temperatures like the graph above, switch to doing the division before the multiplication. Also keep in mind that the Arduino will default to 16bit calculations unless you set/cast everything to unsigned long.  Or you could make you life easy and save the elapsedTimeReff & elapsedTimeSensor values to your data file and do the calculations later in Excel.

1) Because my Timer1 numbers were small with a 104 cap I did the multiplication before the division. But keep in mind that this method can easily generate values that over-run your variables during calculationUlong MAX is 4,294,967,295  so the elapsedTimeSensor reading must be below 429,496 or the multiplication overflows with a 10,000 ohm reference. Dividing that by our 8mHz clock gives about 50 milliseconds. The pin interrupt threshold is reached after about one rise-time constant so you can use an RC rise time calculator to figure out your capacitors upper size limit. But keep in mind that’s one RC at the maximum resistance you expect from your NTC – that’s the resistance at the coldest temperature you expect to measure as opposed to its nominal rating.  (But it’s kind of a chicken&egg thing with an unknown thermistor right?  See if you can find a manufacturers table of values on the web, or simply try a 0.1uF and see if it works).  Once you have some constants in hand Ametherm’s Steinhart & Hart page lets you check the actual temperature at which your particular therm will reach a given resistance.  Variable over-runs are easy to spot because the problems appear & then disappear whenever some temperature threshold is crossed. I tried to get around this on a few large-capacitor test runs by casting everything to float variables, but that lead to other calculation errors.

2) This method works with any kind of resistive sensor, but if you have one that goes below ~200 ohms (like a photoresistor in full sunlight) then the capacitor charging could draw more power than you can safely supply from the D9 pin.  In those cases add a ~300Ω resistor in series with your sensor to limit the current, and subtract that value from the output of the final calculation. At higher currents you’ll also have voltage drop accross the mosfets controlling the I/O pins (~40Ω on a 3.3v Pro Mini), so make sure the calibration includes the ends of your range.

There are a host of things that might affect the readings because every component has a temperature, aging, and other coefficients, but for the accuracy level I’m after many of those factors are handled by the ratio-metric nature of the method. It’s almost independent of value of the capacitor, so you can get away with a cheap ceramic even though it’s value changes dramatically with temperature and age. (I’d still use at least an X7R or, a plastic film cap for measuring colder temps) Last year I found that the main system clock was variable to the tune of about 0.05% over a 40°C range, but that shouldn’t matter if the reference and the sensor readings are taken close to each other. Ditto for variations in your supply. None of it matters unless it affects the system ‘between’ the two measurements.

The significant digits from your final calculation depend on the RC rise time, so switching to 100k thermistors increases the precision. Processors with higher clock speeds should also boost your resolution.  You can shut down more peripherals with the PRR to use less power during the readings as long as you leave Timer1 running.  I’ve also found that cycling the capacitor through an initial charge/discharge cycle (via the 300Ω on D8) after long sleep intervals improved the consistency of the readings. That capacitor shakedown might be an artifact of the ceramics I was using but reading your reference twice at the start of each run might be a good way of eliminating errors that arise from the pre-existing conditions, though you’d also be heating up that resistor so it’s a bit of a trade-off.

Will this method replace our pin-toggled oversampling?  Perhaps not for something as simple as a thermistor since that method has already proven itself in the real world, and I don’t really have anything better to do with A6 & A7. And oversampling still has the advantage of being simultaneously available on all the analog inputs, while the ICU is a more limited resource.  Given the high resolution that’s potentially available with the Timer1/ICU combination, I might save this method for sensors with less dynamic range.  I already have some ideas there and, of course, lots more testing to do before I figure out if there are other problems related to this new method.  I still haven’t determined what the long-term precision is with our Pro Mini clones, and the WDT experiment taught me to be cautious about counting those chickens.


Addendum:   Using the processors built in pull-up resistors

After a few successful runs, I realized that I could use the internal pull-up resistor on D8 as my reference; bringing it down to only three components. Measuring the resistance of the internal pull-up is simply a matter of enabling it and then measuring the current that flows between that pin and ground. I ran several tests, and the Pro Mini’s the internal pullups were all close to 36,000 ohms, so my reference would become 36k + the 300Ω resistor needed on D8 to safely discharge of the capacitor between cycles. I just have to set the port manipulation differently before the central do-while loop:

PORTB |= (1 << PORTB0);     //  enable pullup on D8 to start charging the capacitor

A comparison of the two approaches:

°Celcius vs time: (Post-calibration, 0.1uF X7R ceramic cap)  si7051 reference [±0.01°C] vs. thermistor ICU ratio w 10k reference resistor charging the capacitor vs. the same thermistor reading calibrated using the rise time through pin D8’s internal pull-up resistor. The reported ‘resistance’ value of the NTC themistor was more than 1k different between the two methods, with the 10k met-film reference providing values closer to the rated spec. However the Steinhart-Hart equation constants from the calibration were also quite different, so the net result was indistinguishable between the two references in the room-temperatures range.

The internal pull-up probably isn’t a real resistor. More likely its a very thin semiconductor channel that simply acts like one, and I found the base-line temperature variance to be about 200 ohms over my 40°C calibration range. And because you are charging the capacitor through the reference and through the thermistor, the heat that generates necessarily changes the changes those values during the process.  However when you run a calibration, those factors get absorbed into the S&H coefficients provided you let the system equilibrate during the run.

As might be expected, all chip operation time affects the resistance of the internal pull-up, so the execution pattern of the code used for your calibration must exactly match your deployment code or the calibration constants will give you an offset error proportional to the variance of the internal pull-up caused by the processors run-time. Discharging the capacitor through D8, also generates some internal heating so those (~30-60ms) sleep intervals also have to be consistent.  In data logging applications you can read that reference immediately after a long cool-down period of processor sleep and use the PRR to reduce self-heating while the sample is taken.  Code consistency is always important with time based calibrations no matter how stable your reference is.

Another issue was lag because that pull-up is embedded with the rest of the chip in a lump of epoxy. This was a pretty small, with a maximum effect less than ±0.02°C/minute and I didn’t see that until temperatures fell below -5 Celsius.  Still, for situations where temperature is changing quickly I’d stick with the external reference, and physically attach it to the thermistor so they stay in sync.


Addendum:   What if your processor doesn’t have an Input Capture Unit?

With a 10k / 0.1uF combination, I was seeing Timer1 counts of about 5600 which is pretty close to one 63.2% R * C time constant for the pair.  That combination limits you to 4 significant figures and takes about 2x 0.7msec per reading on average.  Bumping the integrating capacitor up to 1uF (ceramic 105) multiplies your time by a factor of 10 – for another significant figure and an average of ~15msec per paired set readings.  Alternatively, a 1uF or greater capacitor allows you to record the rise times with micros()  (which counts 8x slower than timer1) and still get acceptable results. (even with the extra interrupts that leaving Timer0 running causes…) So the rise-time method could be implemented on processors that lack an input capture unit – provided that they have Schmitt triggers on the digital inputs like the AVR.

void d3isr() {
triggered = true;
}

pinMode(7, INPUT);digitalWrite(7,LOW);     // reference resistor
pinMode(9, INPUT);digitalWrite(9,LOW);     // the thermistor
pinMode(8,OUTPUT);digitalWrite(8,LOW);  // ground & drain the cap through 300Ω
LowPower.powerDown(SLEEP_30MS, ADC_OFF, BOD_ON);  // 5T is now 1.5ms w 10k

pinMode(8,INPUT);digitalWrite(8, LOW);    // Now pin D8 is listening

triggered = false;
set_sleep_mode (SLEEP_MODE_IDLE);  // leave micros timer0 running for micros()
unsigned long startTime = micros();
noInterrupts ();
attachInterrupt(1, d3isr, RISING); // using pin D3 here instead of D8
DDRD |= (1 << DDD7);          // Pin D7 to OUTPUT
PORTD |= (1 << PORTD7);    // Pin D7 HIGH -> charging the cap through 10k ref
sleep_enable();

do{
interrupts ();
sleep_cpu ();
noInterrupts ();
}while(!triggered);

sleep_disable();
detachInterrupt(1);
interrupts ();
unsigned long D7riseTime = micros()-startTime;

Then repeat the pattern shown earlier for the thermistor reading & calculation. I’d probably bump it up to a ceramic 106 for the micros method just for some extra wiggle room. The method doesn’t really care what value of capacitor you use, but you have to leave more time for the discharge step as the size of your capacitor increases.

‘No-Parts’ Temperature Measurement with Arduino Pro Mini

328p processor System Clocks & their Distribution pg26

Most micro-controllers use a quartz crystal oscillator to drive the system clock, and their resonant frequency is reasonably stable with temperature variations. In high accuracy applications like real time clocks even that temperature variation can be compensated, and last year I devised a way to measure temperature by comparing a 1-second pulse from a DS3231 to the uncompensated 8Mhz oscillator on a Pro Mini. This good clock / bad clock method worked to about 0.01°C, but the coding was complicated, and it relied on the ‘quality’ of the cheap RTC modules I was getting from fleaBay – which is never a good idea.

But what if you could read temperature better than 0.01°C using the Pro Mini by itself?

Figure 27-34:  Watchdog Oscillator Frequency vs. Temperature. Pg 346 (Odd that the frequency is listed as 128kHz on pg55?)  Variation is ~100 Hz/°C

The 328P watchdog timer is driven by a separate internal oscillator circuit running at about 110 kHz. This RC oscillator is notoriously bad at keeping time, because that on-chip circuit is affected by external factors like temperature. But in this particular case, that’s exactly what I’m looking for.  The temperature coefficient of crystal resonators is usually quoted at 10–6/°C and for RC oscillation circuits the coefficient is usually somewhere between 10–3/°C to 10–4/°C.  There’s plenty of standard sensors don’t give you a delta that large to play with!

To compare the crystal-driven system clock to the Watchdogs unstable RC oscillator I needed a way to prevent the WDT from re-starting the system.  Fortunately you can pat the dog and/or disable it completely inside its interrupt vector:

volatile boolean WDTalarm=false;
ISR(WDT_vect)
{
wdt_disable();  // disable watchdog so the system does not restart
WDTalarm=true;    // flag the event
}


SLEEP_MODE_IDLE
leaves the timers running, and they link back to the system clock.  So you can use micros() to track how long the WDT actually takes for a given interval.  Arduinos Micros() resolution cannot be better than 4 microseconds (not 1 µs as you’d expect) because of the way the timer is configured, but that boosts our detectable delta/° by a factor of four, and the crystal is far more thermally stable than the watch-dog. It’s worth noting that timer0 (upon which micros() depends) generates interrupts all the time during the WDT interval, in fact at the playground they suggest that you have to disable timer0 during IDLE mode sleeps. But for each time interval, the extra loops caused by those non-WDT interrupts create a consistant positive offset, and this does not affect the temperature related delta.

WDTalarm=false;
// Set the Watchdog timer                    from: https://www.gammon.com.au/power
byte interval =0b000110;  // 1s=0b000110,  2s=0b000111, 4s=0b100000, 8s=0b10000
//64ms= 0b000010, 128ms = 0b000011, 256ms= 0b000100, 512ms= 0b000101
noInterrupts
();
MCUSR = 0;
WDTCSR |= 0b00011000;    // set WDCE, WDE
WDTCSR = 0b01000000 | interval;    // set WDIE & delay interval
wdt_reset();  // pat the dog
interrupts ();
unsigned long startTime = micros();
while (!WDTalarm)  {    //sleep while waiting for the WDT
set_sleep_mode (SLEEP_MODE_IDLE);
noInterrupts ();  sleep_enable();  interrupts ();  sleep_cpu ();
sleep_disable();  //processor starts here when any interrupt occurs
}
unsigned long WDTmicrosTime = micros()-startTime;  // this is your measurement!

The while-loop check is required to deal with the system interrupts that result from leaving the micros timer running, otherwise you never make it all the way through the WDT interval. I haven’t yet figured out how many interrupts you’d have to disable to get the method working without that loop.

To calibrate, I use my standard refrigerator->freezer->room sequence for a repeatable range >30°.  Since the mcu has some serious thermal lag, the key is doing everything VERY SLOWLY with the logger inside a home made “calibration box” made from two heavy ceramic pots, with a bag of rice between them to add thermal mass:

1sec WDT micros() (left axis) vs si7051 °C Temp (right axis) : Calibration data selected from areas with the smallest change/time so that the reference and the 328p have equilibrated.

If you use reference data from those quiescent periods, the fit is remarkably good:

si7051 reference temperature vs 1 sec WDT micros() : A fit this good makes me wonder if the capacitor on the xtal oscillator is affected the same way as the capacitor in the watchdogs RC oscillator, with the net result being improved linearity. In this example, there was a constant over-count of 100,000 microseconds / 1-second WDT interval.

I’m still experimenting with this method, but my cheap clone boards are delivering a micros() delta > 400 counts /°C with a one second interval – for a nominal resolution of ~0.0025°C.  Of course that’s just the raw delta.  When you take that beautiful calibration equation and apply it to the raw readings you discover an inter-reading jitter of about 0.1°C – and that lack of precision becomes the ‘effective’ limit of the resolution.  It’s going to take some serious smoothing to get that under control, and I’ll be attacking the problem with my favorite median filters over the next few days. I will also see if I can reduce it at the source by shutting down more peripherals and keeping an eye on stray pin currents.

Noise from si7051 reference (red)  vs Cal. equation applied to raw WDT micros readings (blue). 

Doubling the interval cuts the the noise and the apparent resolution in half, and if you are willing to wait around for the watchdogs 8-second maximum you can add an order of magnitude. Of course you could also go in the other direction: a quarter second WDT interval would deliver ~100 counts/°C, which still gets you a nominal 0.01°C though the jitter gets worse. Note that you can’t use the ‘b’ coefficients from one interval to the next, because of the overhead caused by the non-WDT interrupts. That “awake time” must also be contributing some internal chip heating.

The si7051 reference sensor needs to be held in direct physical contact with the surface of the mcu during the room->fridge->freezer calibration; which takes 24 hours. Since my ref is only ± 0.1ºC accuracy, calibrations based on it are probably only good to about ± 0.2ºC.

There are a few limitations to keep in mind, the biggest being that messing with WDT_vect means that you can’t use the watchdog timer for it’s intended purpose any more. The other big limitation is that you can only do this trick on a voltage regulated system, because RC oscillators are affected by the applied voltage, though in this case both oscillators are exposed to whatever is on the rail, so a bit more calibration effort might let you get away with a battery driven system.

Self-heating during normal operation means that this method will not be accurate unless you take your temperature readings after waking the processor from about 5-10 minutes of power-down sleep. The mass of the circuit board means that the mcu will always have significant thermal lag. So there is no way to make this method work quickly and any non-periodic system interrupts will throw off your micros() reading.

Every board has a different crystal/capacitor/oscillator combination, so you have to re-calibrate for each one. Although the slopes are similar, I’ve also found that the raw readings vary by more than ±10k between different Pro Minis for the same 1sec WDT interval, at the same temperature. The silver lining there is that the boards I’m using probably have the cheapest parts available, so better quality components could boost the accuracy , though I should insert the usual blurb here that resolution and accuracy are not the same thing at all.  I haven’t had enough time yet to assess things like drift, or hysteresis beyond the thermal lag issue, but those are usually less of a problem with quality kit. If your board is using Y5V caps it probably won’t go much below -15°C before capacitor failure disrupts the method.

It’s also worth noting that many sleep libraries, like Rocketscreem’s Lowpower lib, do their own modifications to the watchdog timer, so this method won’t work with them unless you add the flag variable to their modified version of the WDT_vect.  To add this technique to the base code for our 1-hour classroom logger, I’ll will have to get rid of that library dependency.

Where to go from here:

  1. Turning off peripherals with PRR can save power and reduce heating during the interval.
  2. Switching from micros(), to timer based overflows could increase the time resolution to less than 100 ns;  raising nominal thermal resolution.
  3. Clocking the system from the DS3231’s temperature compensated 32khz output could give another 100 counts/°C and improve the thermal accuracy. My gut feeling is the noise would also be reduced, but that depends on where it’s originating.

Despite the limitations, this might be the best “no-extra-parts” method for measuring temperature that’s possible with a Pro Mini, and the method generalizes to every other micro-controller board on the market provided they have an independent internal oscillator for the watchdog timer.

Addendum:

As I run more units through the full calibration, I’m seeing about 1 in 3 where a polynomial fits the data better for the -15 to +25°C range:

si7051 reference temperature vs 1 sec WDT micros() : a different unit, but both clone boards from the same supplier

This is actually what I was expecting in the first place and I suspect all the fits would be 2nd order with a wider range of calibration temperatures. Also, this is the raw micros output – so you could make those coefficients more manageable by subtracting the lowest temperature reading from all those above. This would leave you with a numerical range of about 16000 ticks over 40°C, which takes less memory and is easier for calculations.

And just for fun I ran a trial on an unregulated system powered by 2xAA lithium batteries.  Two things happened: 1) the jitter/noise in the final Temperature readings more than doubled – to about 0.25°C  and 2) calibration was lost whenever the thermal mass of the batteries meant that the supply was actively changing – regardless of whether the mcu & reference had settled or not:

Red is the Si reference [left axis], Green is the calibration fit equation applied to the WDT micros() [left], and blue is the rail voltage supplied by 2xAA lithium batteries [right axis] (Note: low voltage spikes are caused by internal housekeeping events in the Nokia 256mb SD cards)

Addendum 2019-02-26

This morning I did a trial run which switched from micros() to timer1 overflows, using code from Nick Gammon’s Improved sketch using Timer 1. This increased the raw delta to almost 5000/°C, but unfortunately the width of the jitter also increased to about 500 counts. So I’m seeing somewhere near ±0.05°C equivalent of precision error – although my impression is that it’s been reduced somewhat because Timer1 only overflows 122 times per second, while the Timer0/micros had to overflow 100k times. So changing timers means less variability from the while-loop code execution. Next step will be to try driving the timer with the 32khz from the RTC…

Addendum 2019-02-27

So I re-jigged another one of Nicks counting routines which increments timer1 based on input from pin D5, using the WDT interrupt to set the interval. Then I enabled the 32.768 kHz output from a DS3231N and connected it to that pin. This pulse is dead-dog slow compared to the WDT oscillator, so I extended the interval out to 4 seconds.  This long-ish sample time only produced a delta of about 40 counts/°C.

Si7051 reference temp vs Timer1 counts of 32kHz output from DS3231N  (based on data selected from quiescent periods)

There wasn’t enough data to produce high resolution, but my thought was that since the DS3231N has temperature compensated frequency output, it eliminates the xtal as a variable in the question of where the jitter was coming from.  This approach also causes only 2-3 overflows on timer1, so the impact of code execution is further reduced.  Unfortunately, this experiment did not improve the noise situation:

DS3231 32khz clock tics vs 4sec WDT interval Raw reading jitter during a relatively quiescent period.

That’s about 8 counts of jitter in the raw, which produces readings about ±0.1C away from the central line.  That’s actually worse than what I saw with the Xtal vs WDT trials, but the increase might be an artifact of the pokey time-base.  The smoking gun now points squarely at variations in the WDT oscilator output as the source of the noise.

That’s kind of  annoying, suggesting it will take filtering/overhead to deliver better than about 0.1°C from this technique, even though higher resolution is obviously there in the deltas. The real trick will matching the right filter with all the other time lag / constraints in this system. Still, extra data that you can get from a code trick is handy, even if it sometimes it only serves to verify that one of your other sensors hasn’t gone squirrely.

—> just had a thought: oversampling & decimation eats noise like that for breakfast!
Just fired up a run taking 256 x 16ms samples (the shortest WDT interval allowed) with Timer1 back on the xtal clock. Now I just have to wait another 24 hours to see if it works…

Addendum 2019-02-28

OK: Data’s in from oversampling the WDT vs timer1 method.  I sum the the timer1 counts from 256 readings (on a 16msec WDT interval) and then >>4 to decimate. These repeated intervals took about 4 seconds of sampling time.

si7051 reference temperature vs 256x Oversampled Timer 1 reading on 16 msec WDT interval: Fit Equation

This produced 750 counts/°C for a potential resolution of 0.0013°, but as with the previous trials, the method falls down because the jitter is so much larger:

Variability on 256 Timer1 readings of 16msec WDT interval : During quiescent period

100 points of raw precision error brings the method back down to a modest ‘real’ resolution of only ±0.066°C at best.  The fact that this variability is so similar to the previous trials, and that oversampling did not improve it, tells me that the the problem is not noise – but rather the WDT oscillator is wandering around like a drunken sailor because of factors other than just temperature.  If that’s the case, there’s probably nothing I can throw at the problem to make it go away.

Several people pointed out that there is another way to measure temperature with some of the Atmel chips, so I decided to fire that up for a head-to-head trial against the WDT method.  Most people never use it because the default spec is ±10°C and it only generates 1 LSB/°C correlation to temperature for a default resolution of only 1°C.  Some previous efforts with this internal sensor produced output so bad it was used as a random seed generator.

But heck, if I’m going through the effort of calibration anyway, I might be able to level the playing field somewhat by oversampling those readings too:

si7051 reference temperature  °C  vs  4096 reading oversample of the internal diode: Fit equation

Even with 4096 samples from the ADC, this method only delivered ~75 counts /°C. But the internal diode is super linear, and the data is less variable than the WDT:

Variability from 4096 ADC readings of the internal reference diode : During quiescent period

Five counts of raw variability means the precision error is only  ±0.033°C  (again, this becomes our real resolution, regardless of the raw count delta) .  So even after bringing out the big guns to prop up the WDT, the internal reference diode blew the two-oscillator method out of the water on the very first try.

volatile uint16_t adc_irq_count;

ISR (ADC_vect)
{
adc_irq_count++;   //flag to track how many samples have been taken
}

 

internalDiodeReading=0;
adc_irq_count = 0;
unsigned long sum = 0;
unsigned int wADC;
ADMUX = (_BV(REFS1) | _BV(REFS0) | _BV(MUX3)); // Set the 1.1v aref and mux for diode
ADCSRA |= _BV(ADSC);  // 1st conversion to engage settings
delay(10);    // wait for ADC reference cap. to stabilize
do
noInterrupts ();
set_sleep_mode( SLEEP_MODE_ADC ); // Sleep Mode just to save power here
sleep_enable();
ADCSRA |= _BV(ADSC);  // Start the ADC
do{
interrupts();
sleep_cpu();      // Sleep (MUST be called immediately after interrupts()
noInterrupts(); // Disable interrupts so while(bit_is_set) check isn’t interrupted
} while (bit_is_set(ADCSRA,ADSC));  // back to sleep if conversion not done 
sleep_disable();  interrupts(); // Enable interrupts again
wADC = ADCW;  // Reading “ADCW” combines both ADCL & ADCH
sum += wADC;   // Add new reading to the total
} while (adc_irq_count<4096);  //sets how many times the ADC is read
ADCSRA &= ~ _BV( ADIE ); // No more ADC interrupts after this
internalDiodeReading=(sum >> 6); //decimation turns sum into an over-sampled reading

 

For now, I think I’ve kind of run out of ideas on how to make the WDT method more precise..? Oh well – It’s not the first time I’ve tried to re-invent the wheel and failed (and it probably won’t be my last….) At least it was a fun experiment, and who knows, perhaps I will find a better use for the technique in some other context.

I’ll spend some more time noodling around with the internal diode sensor and see if I can wrestle better performance out of it since it’s probably less vulnerable to aging drift than the WDT oscillator.  I’m still a bit cautious about oversampling the internal diode readings because the math depends on there being at least 1-2LSB’s of noise to work, and I already know the internal 1.1v ref is quite stable..?  I threw the sleep mode in there just to save power & reduce internal heating, but now that I think about it the oversampling might work better I let the chip heat a little over the sampling interval – substituting that as synthetic replacement for noise if there is not enough. The other benefit of the diode is that it will be resilient to a varying supply voltage, and we are currently  experimenting with more loggers running directly from batteries to see if it’s a viable way to extend the operating lifespan.

Addendum 2019-04-05

So I’ve kept a few units perking along with experiments oversampling the internal diode. And on most of those runs I’ve seen an error that’s quite different from the smoothly wandering WDT method. The internal diode readings have random & temporary jump-offsets of about 0.25°C:

si7051 (red) reference vs internal Diode (orange) over sampled 16384 reads then >>10 for heavy decimation.

These occur all all temperatures from +20 to -15C,  and I have yet to identify any pattern. The calculation usually returns to a very close fitting line after some period with the offset with about 50% of the overall time with a closely fit calibration. This is persistent through all the different oversampling intervals and stronger decimation does not remove the artifact – it’s in the raw data. No idea why…yet.

Using Arduino’s Internal EEprom to Store Data & Screen Fonts

This tutorial is the second in a series on adding displays to expand the capability of the Arduino data loggers described in our SENSORS paper earlier this year. As more of those DIY units go into the field, it’s become important that serial #s & calibration constants are embedded inside the files produced by each logger.  One can always hard-code that information, but with multiple sensors and screens for live output, space is getting tight:

This prompted me to look at the ATmega328p’s internal EEprom as a potential solution. The EEprom can only store one byte per location, so saving numbers larger than 255 requires you to slice them up and store them in consecutive memory locations. That takes two locations for a the “high byte” and a “low byte” of an int, and four memory locations for longs & floats. That’s a little clunky but it mirrors the way you read & write high-bit registers on I2C sensors, so there are lots of code examples out there to follow.  Piece by piece approaches also require memory pointers for retrieval and re-assembly of your variables.

A more elegant method is to roll them into a single ‘struct’  and store that with a generic function, but even with read_block & write_block I’d still be tweaking the code for each logger, since they often have dramatically different sensor combinations.  I wanted a more generic “cut & paste” method that would handle file headers and variable boiler plate info ( contact emails, etc.  ) without me having to update a bunch of pointers.  The real clincher was realizing that the equation constants generated by my thermistor calibration procedure had too many significant figures to store in an Arduino float variable anyway.

A char array provided the simplest way to achieve the flexibility I was after, and I wrapped that up into a little “EEprom loader” utility:             (Note: Github link at the end of this post)

#include <EEPROM.h>
#define PADLENGTH 1024
char eepromString [PADLENGTH+1];    //+1 to leave room for null terminator

void setup(){
strcpy(eepromString," ");
strcat(eepromString, "\r\n");      // a carriage return in Excel
strcat(eepromString,"#198, The Cave Pearl Project");
strcat(eepromString, "\r\n");
strcat(eepromString,"Etime=(raw*iSec)/(86400)+\"1/1/1970\"");
// NOTE: \ is an escape which lets you put "special" characters into the string 
strcat(eepromString, "\r\n");
strcat(eepromString,"Tres=(SeriesOhms)/((((65536*3)-1)/Raw16bitRead)-1)");
strcat(eepromString, "\r\n");
strcat(eepromString,"1M/100k@A7(16bitP32@1.1v),A=,-0.0003613129530,B=,0.0003479768279,C =,-0.0000001938681482");
strcat(eepromString, "\r\n");                 
// following this pattern, simply add more lines as needed (up to max 1024 characters)

// This fills the remaining unused portion of eepromString with blank spaces:
int len = strlen(eepromString);  // strlen does not count the null terminator
memset(&eepromString[len],' ',PADLENGTH-len);

// Now write the entire array into the EEprom, one byte at a time:
for (int i = 0; i < 1024; i++){ 
EEPROM.update(i, eepromString[i]); 
}
}

void loop() {
// nuthin here...
}
 

This is just a bare-bones example I whipped up, and it’s worth noting that strcat will overflow if you try to add more characters than eepromString can hold.  You can check your count at sites like lettercount.com  or try using snprintf() as an alternative without that problem. Since this is a ‘run-once’ utility, I haven’t bothered to optimize it further.  Bracketing calibration numbers with a comma on each side makes them directly usable when the CSV data file is loaded into Excel later because they end up inside their own cell.  It’s a good idea not to avoid putting EEPROM.write codes inside the main loop, or could accidentally burn through the limited write cycles of your internal EEprom. EEPROM.update is the safer than EEPROM.write, because it first checks the content of each memory location, and only updates it if the new information to be stored is different.

With the data stored in the EEprom, it only takes a few lines to transfer that information to the SD card when a new file gets created:

char charbuffer[0]=" ";  //a one character buffer
file.open(FileName, O_WRITE | O_APPEND);
for (int j = 0; j < 1024; j++) {
 charbuffer[0] = EEPROM.read(j);
 file.write(charbuffer[0]);
 }
file.close(); 

The spaces used to pad out the array so that it fills all 1024 bytes of the EEprom do create an extra blank line in the file, but that’s a pretty harmless trade-off for this level of code simplicity.

What else can we store in the EEprom?

In the post covering how to drive a Nokia5110 LCD using shiftout, I went into some detail on the way the fonts for that screen were created and displayed. Three simple cascading functions (originally from Julian Ilett) let you send any string of ASCII characters to the screen.

void LcdWriteString(char *characters)
{
while(*characters) LcdWriteCharacter(*characters++);
} 
void LcdWriteCharacter(char character)
{
for(int i=0; i<5; i++){
LcdWriteData(pgm_read_byte(&ASCII[character - 0x20][i])); 
}
LcdWriteData(0x00);            //one row of spacer pixels between characters
} 
void LcdWriteData(byte dat)
{
digitalWrite(DCmodeSelect, HIGH);    // High for data 
digitalWrite(ChipEnable, LOW);  
shiftOut(DataIN, SerialCLK, MSBFIRST, dat);  // transmit serial data 
digitalWrite(ChipEnable, HIGH);
} 

I modified that code that slightly with reduced font-set arrays stored in PROGMEM and introduced a method for displaying larger numbers on screen by printing the upper and lower parts of each number in two separate passes.  PROGMEM requires pgm_read_byte to get the data out of the 2-D arrays for printing.

Now, with a little bit of juggling, a “loader” script can store that font data in the 328P’s internal EEprom by converting the two dimensional font array into a linear series of memory locations:

const byte ASCII[][5] PROGMEM =  
{
{0x00, 0x00, 0x00, 0x00, 0x00} // 20 
,{0x00, 0x00, 0x5f, 0x00, 0x00} // 21 !
,{0x00, 0x07, 0x00, 0x07, 0x00} // 22 "
,{0x14, 0x7f, 0x14, 0x7f, 0x14} // 23 #
,{0x24, 0x2a, 0x7f, 0x2a, 0x12} // 24 $
,{0x23, 0x13, 0x08, 0x64, 0x62} // 25 %
,{0x36, 0x49, 0x55, 0x22, 0x50} // 26 &
,{0x00, 0x05, 0x03, 0x00, 0x00} // 27 '
,{0x00, 0x1c, 0x22, 0x41, 0x00} // 28 (
,{0x00, 0x41, 0x22, 0x1c, 0x00} // 29 )
,{0x14, 0x08, 0x3e, 0x08, 0x14} // 2a *
,{0x08, 0x08, 0x3e, 0x08, 0x08} // 2b +
,{0x00, 0x50, 0x30, 0x00, 0x00} // 2c ,
,{0x08, 0x08, 0x08, 0x08, 0x08} // 0x2d (dec 45) (-) in row 13 of source array
,{0x00, 0x60, 0x60, 0x00, 0x00} // 2e .
//...etc, a complete font is in the array, but I only store a 46 character "caps only"
// subset in the eeprom ranging from (-) at array row 13 to capital (Z) =array row 59
// this takes 235 bytes of EEprom memory storage (addresses 0-234)
} 

// move that sub-set of the font (13 to 59) from PROGMEM into the 328's internal EEprom
currentIntEEpromAddress=0;   // each letter is constructed from 5 byte-collumns of data
for(int i=13; i<59; i++){    // so each i value forces a 5-byte jump in EEprom address
for(int j=0; j<5; j++){      // while j value counts through each individual column
charbuffer=pgm_read_byte(&ASCII[i][j]);  // i&j used separately in the array
currentIntEEpromAddress=(((i-13)*5)+j);  // i&j combined for the EEprom address
EEPROM.update(currentIntEEpromAddress,charbuffer); //update to save unnecessary writes
}

Once the font has been transferred into the internal EEprom by the loader program, I only need to make two small changes to the original display functions so they pull that font from the EEprom. Note the calculation trick (character-0x2d) which uses the ASCII code for each character to calculate where the first of the five bytes is located for that character:

void LcdWriteCharacter(char character)
{
for(int j=0; j<5; j++){
LcdWriteData(EEPROM.read(((character -0x2d)*5)+j)); 
//we subtract 0x2d because the zeroth position in EE memory is (-) = ASCII(0x2d)
}
LcdWriteData(0x00);            
} 
// (character - 0x2d)*5 jumps 5 bytes at a time through the EEprom address space
// Add a fixed offset for other fonts stored at higher locations in the memory: 
// eg: LcdWriteData(EEPROM.read(235+((character -0x2d)*5)+j)); 
// the big#fonts are 11 byte-columns wide, so calc = (Offset+((character -0x2d)*11)+j)

This adds some delay, but because the Nokia 5110 is already dead-dog slow it’s not even noticeable.  A similar mod gets applied to the split print functions for the big number font, and moving both to the EEprom still leaves a very serviceable 500 characters for file header info.  The trick to stacking different kinds of information in the EEprom is figuring out what the resulting address offsets are so each reading function starts at the correct memory address. You skip over the memory locations holding the font data when transferring that file header text.

With fonts stored in EEprom, the remaining Nokia 5110 functions compile to just a little over 400 bytes of program storage and 10 bytes of dynamic (excluding EEPROM.h, which gets called by some of my other libraries even if the screen is not present)  That’s with three duplicate copies of each LCD function because I’ve used a set for the standard size font, and two more for printing the large numbers in upper & lower rows.  With a bit more optimization I could get that down to about 200 bytes, which I suspect is probably the smallest memory footprint achievable for adding live data output to my loggers.

I’ve posted a ‘EEprom loader utility’ which demonstrates the dual Font/Text approach at our Project’s GitHub repository  so people can modify it to suit their own projects. On loggers with a number of DS18b20 sensors, I’ll use a similar approach to store the sensor bus addresses, which I usually store in two dimensional arrays very similar to those shown here for screen fonts.

Using the Nokia 5110 LCD with an Arduino Data Logger

Here I’ve added a the 5110 LCD to a logger recording data from a BME280 & Tipping Bucket Rain gauge. If the BME  survives in our field environment, this will become a standard configuration for our climate stations. I’m not holding my breath though, as we’ve tested half a dozen RH sensors so far and none of them have gone the distance in high humidity environments that occasionally go condensing.

This year I want to tackle some projects that need live data out, so I’ve been sifting through the many display options available for Arduino. Unlike flashier projects, my goal was to find one that I could add to existing logger builds without sacrificing too much of the multi-year lifespan I had worked so hard to achieve. The low power winner by a fair margin was the Nokia 5110 Liquid-crystal Display  which you can pick up for around $2 from the usual sources. With the back-light off these displays pull between 100-400 μA, depending on the number of pixels turned on.

This screen uses a PCD8544 controller and the SPI protocol.  It will tolerate 5V, but it works best at 3.3V, which is perfect when you are driving it from an 8mhz ProMini. Each pixel on the display is represented by a single bit in the PCD8544’s RAM. Each byte in RAM correlates to a vertical column of 8 pixels. The X coordinate works on a per-pixel basis, and accepts values between 0 and 83. The Y coordinate accepts values of 0 – 5 which on this 48 pixel high screen, corresponds to 6 “rows of bytes” in the controller’s RAM. So bitmaps can only be displayed on a per row (& column) basis. The display is quite sluggish compared to competitors like the 0.96 I2C monochrome OLED and you have to handle any processing overhead on the Arduino.

Most hookup guides assume that you can spare six control lines to run the display, which is not the case when your logger already has three indicator LEDs, I2c devices, one wire sensors and a couple of voltage dividers on the go.  However if you are willing to add a few resistors and occasionally toggle the power, you can bring that down to three wires and a power pin.

So many libraries, so little optimization…

This screen’s been around for a very long time, so there’s are a huge number of easy to use, highly functional libraries for Arduino. But they tend to focus on things like speed or endless font options which are not important for most data logging applications. And these libs assume your project can afford to lose up to ⅓ of the available program & variable memory just driving the display. Most also require the hardware SPI lines, but our project needs those for SD cards, which are finicky enough without some pokey LCD gumming up the works: the 5110 maxes out at 4mbps, and this slows the bus significantly .

Those fat libs were non-starters for our project, and I had almost given up on this display when I found Ilett’s Ardutorial offering a bare-bones method more suitable for our resource limited data loggers. If you haven’t discovered Julians YouTube channel yet then you are in for a treat because if Andreas Spiess is the maker worlds answer to Werner Herzog, then Julian is surely their equivalent to Bob Ross.  I don’t know if he’s growing “Happy little trees” with his DIY hydroponics, but I can say that the gentle timbre of his “Gooood morning all” reduces stress faster than a warm cup of Tea.  And his “Arduino sandwiches” are brilliant examples of minimalist build technique.

Driving the Nokia 5110 with shiftout

Everything I’m presenting here builds on his tutorials, so grab a mug and give ’em a watch:

Tutorial #1 – Connecting and Initial Programming
Tutorial #2 – Getting Text on the Display
Tutorial #3 – Live Numerical Data

This software SPI method (originally from arduino.cc?) requires no library at all, and shiftout commands work with any combination of digital pins; saving those hardware SPI lines for more important jobs.

Initial setup is explained in video #1 using two functions

void LcdInit(void)
{
digitalWrite(RST, LOW);            // not needed with pin powering!
digitalWrite(RST, HIGH);           // see below for details
LcdWriteCmd(0x21);                 // extended commands 
LcdWriteCmd(0xB8);                 // set Vop(contrast) // you may need to tweak
LcdWriteCmd(0x04);                 // set temp coefficient 
LcdWriteCmd(0x14);                 // bias mode 1:40 // you may need to tweak this
LcdWriteCmd(0x20);                 // basic commands 
LcdWriteCmd(0x0C);                 // normal video
for(int i=0; i<504; i++) LcdWriteData(0x00);  // clear the sceen
} 
void LcdWriteCmd(byte cmd)
{
digitalWrite(DCmodeSelect, LOW);    // low for commands, high for data 
digitalWrite(ChipEnable, LOW);      // not need with pin-power
shiftOut(DataIN, SerialCLK, MSBFIRST, cmd);  // transmit serial data 
digitalWrite(ChipEnable, HIGH);     // not need with pin-power
} 

After that you need is a function to position the cursor and a font stored in a byte array (in this example called ASCII[][5])

void LcdXY(int x, int y)
{
LcdWriteCmd(0x80 | x);              // Column
LcdWriteCmd(0x40 | y);              // Row  
} 

Then three short cascading functions let you send a string of ascii characters to the display:

void LcdWriteString(char *characters)
{
while(*characters) LcdWriteCharacter(*characters++);
} 
void LcdWriteCharacter(char character)
{
for(int i=0; i<5; i++){
LcdWriteData(pgm_read_byte(&ASCII[character - 0x20][i])); 
}
LcdWriteData(0x00);            //one row of spacer pixels between characters
} 
void LcdWriteData(byte dat)
{
digitalWrite(DCmodeSelect, HIGH);    // High for data 
digitalWrite(ChipEnable, LOW);  
shiftOut(DataIN, SerialCLK, MSBFIRST, dat);  // transmit serial data 
digitalWrite(ChipEnable, HIGH);
} 

Julians original implementation included a 500 byte 5×7 font.h file (which you can find at several locations) and  I’ve rolled that font array into some code based on his work and posted it to the Cave Pearl Project’s repo .  You will find lots of other examples based on the shiftout method on Github, but for some reason many people insist in retooling that tiny bit of code into, you guessed it, even more libraries

You’ll also find plenty of other drop-in font definitions with Google, but for small 5×7’s, it doesn’t take that long to roll your own by clicking the boxes in an online font creator. and then copying the byte pattern into a bin-hex converter. This also gives you the option of creating custom icons by using a non-standard bitmap for some of the less frequently used ascii characters. Keep in mind that you don’t need to store the entire alphabet if you are only sending a few letters to the screen (like ‘T+P’ or ‘RH%’, etc …) extracting only the letters you need to a reduced font array could save a lot of memory.

So your reduced font array could look something like this:

const byte ASCII[][5] =
{
{0x7f, 0x09, 0x19, 0x29, 0x46}  // 52 R
,{0x7f, 0x08, 0x08, 0x08, 0x7f} // 48 H
,{0x23, 0x13, 0x08, 0x64, 0x62} // 25 %
,{0x01, 0x01, 0x7f, 0x01, 0x01} // 54 T
,{0x08, 0x08, 0x3e, 0x08, 0x08} // 2b +
,{0x7f, 0x09, 0x09, 0x09, 0x06} // 50 P 
};  

If you do that you’ll need to send text to the screen character-by-character because the ASCII based [character – 0x20] calculation won’t work any more.

I tweaked Julians code in a couple of important ways. First, I added PROGMEM to move the font(s) into the program memory space. Second, I added a method to print large numbers to the screen by repeating the same WriteString->WriteCharacter->WriteData pattern two times: once for the “upper half” of the numbers, and then again for the “lower half” of the numbers after re-positioning the cursor to the next line.

To make this limited large-number font I first composed a black & white bitmap for each number with a graphic editor, and then loaded that .bmp file into the LCD assistant program as described in this instructables tutorial.  I started with a bitmap that was 11 pixels wide, by 16 pixels high (though you can use any arbitrary size you want – just remember to leave the blank spacer row at the bottom) and for this two-pass ‘sliced-letters’ method I set vertical & little endian encoding in LCD assistant. I then put the top 11 bytes in the Big11x16numberTops[] array & the lower 11 bytes for each number in the Big11x16numberBottoms[] array.

It takes two passes to print each large number to the screen:

LcdXY (0,2);              //top half of the double-size numbers
LcdWriteBigStringTops(dtostrf(voltage,5,2,string));  
LcdXY (0,3);              //bottom half of the numbers 
LcdWriteBigStringBottoms(dtostrf(voltage,5,2,string));

And some slightly modified functions that refer to the corresponding number-font array:

void LcdWriteBigStringTops(char *characters)
{
while(*characters) LcdWriteBigCharacterTops(*characters++); 
} 
void LcdLcdWriteBigCharacterTops(char character)
{
for(int i=0; i<11; i++){
if((character - 0x2d)>=0){
LcdWriteData(pgm_read_byte(&Big11x16numberTops[character - 0x2d][i]));
}
}
LcdWriteData(0x00);  //one row of spacer pixels inserted between characters  
} 

Those number printing functions could be eliminated with better use of pointers, but I liked having the readability afforded by a few extra lines of code.

Reducing the number of control lines

The stuff I posted on Github assumes you are using a standard 6-pin arrangement shown in most Nokia 5110 hookup guides you will find on the web. But once I had that wrangled, I realized that it would be possible to reduce the number pins needed to drive the display.  You will have to tweak that default example by commenting out the RS & CS commands if you implement the pin-power changes I’m suggesting here… 

I use Deans micro-plugs  for multi-wire applications like this. The unused pin here is the normal Vcc, with the A0 pin-power supply indicated with a dash on the red line.

The shiftout method can be used with any pins you want, and most of my builds have A0-A3 available. With those dedicated wires the PCD8544’s chip select (CS) line can be connected to GND telling the screen that it’s always the selected device. This would be bad if it was connected to the hardware SPI lines shared with the SD card, but since we are using re-purposed anlaog lines, there is no conflict. One minor drawback is that since we are now using all the analog lines (I use A6 & A7 too) we can’t read a floating pin for RandomSeed().

Getting rid of the RESET line is a little trickier. The data sheet says that the RS line must be low while power stabilizes and should then be pulled high within 100ms of power on. Several people create an auto-reset situation by connecting the screens reset to the Arduino’s reset line. Others make the low-high transition with an RC network across the supply for a delayed rising signal. This can even be driven by the DC line (which is low in command mode and high in data mode) 

But I had something else in mind, since I wanted to power the entire display from a digital pin because the power draw with the display off is still about 70uA. This accumulates into a significant amount of wasted power over a multi-year deployment.

Reducing Back-light Current

OR … a photocell divider embedded in that clear epoxy would let you enable the backlight dynamically – with the appropriate mosfet for the connection type.

If you use the back-light in the default configuration, the screen can potentially draw up to 80mA (4 white LEDs at 20mA each). The back-light pin is usually connected to a transistor, so you can PWM all 4 LEDs at once for variable lighting control, but the peak currents are still too high for direct pin-powering unless you add some kind of series resistor.  A 10k pot gives you a simpler method to adjust the screen brightness, but I found that a 3k3 series resistor brought the total display current down to ~1mA with decent readability ( & blue LEDs are brighter than white).  Adding an in-line slide switch provides a way to completely disable the back-light for long deployments.  With the entire display safely below Arduino’s pin-current limit, you can then power it by writing a driver pin high or low in output mode.

#define n5110PowerPin  A0          // power the 5110 screen from pin A0 (RED)
#define n5110modeSelect A1         // 6.1.9 D/C: mode select (BLUE)
#define n5110SData A2              // 6.1.7 SDIN: serial data line (WHITE) 
#define n5110SCLK A3 ;             // 6.1.8 SCLK: serial clock line (YELLOW)
// lines not needed any more: 
// #define n5110RST  Now -> 4.7k to power A0
// #define n5110ChipEnable  Now -> GND

This gives you a way to perform a hard reset any time you want provided you tie the screens RS line to that switched power with a 4k7 pullup resistor, and re-run the initialization sequence after restoring power.

Boards with pin vias on both sides make it easier to add the RST & CE connections. The orange wires shown here thread through the housing to a slide switch which disables the backlight connection for surface deployments. This example connects power to the backlight, but with other screens you might have to connect BACKLIGHT to GND.

Enabling the screen now looks like this in setup:

pinMode(n5110PowerPin, OUTPUT);
digitalWrite(n5110PowerPin, HIGH);
pinMode(n5110modeSelect, OUTPUT);
pinMode(n5110SData, OUTPUT);
pinMode(n5110SCLK, OUTPUT); 
LcdInit(); // shiftout takes control of Mode, Data & SCLK lines at this point

To turn off the screen you pull all the control lines low:

digitalWrite(n5110PowerPin, LOW);        
digitalWrite(n5110modeSelect, LOW);                
digitalWrite(n5110SData, LOW);                
digitalWrite(n5110SCLK, LOW);                         

All four control lines must be brought low when you de-power the display or you will get a 13mA leak current through the controller after vcc goes low. Only the power pin needs to be driven high to start the screen later in the main loop, but don’t forget to run the init each time you power up.

My tests so far have shown reliable operation of pin-powered 5110’s through more than 8000 ‘long-sleep’ power cycles. In applications where I want to display data on the screen on for long periods of time,  I still depower the screen during the new sensor readings. This lets me know when the logger is capturing data and forces a periodic re-synch with the bus. I don’t know how long these displays would run continuously without that step, but I’m sure the coms would eventually go AWOL without some kind of regular reset.

Potting the Nokia 5110 display

Contraction of the epoxy created pressure burns on the LCD when I did a single large pour to pot the screen.

No screen is much use on our project unless it can withstand some bumping around in the real world, and ideally we want one that is dive-able. For several years my go-to solution has been to pot surface mounted LED’s and sensors in Loctite E30CL. I like this epoxy because the slow cure usually sets clear because bubbles have time to rise to the surface without a vacuum treatment. My first attempts looked great the night of the pour, but I got a nasty surprise the following morning. You see I usually mount sensors in small ½-1 inch wells, but the 5110 required a ring more than 2” in diameter. The contraction of the epoxy in this 10mm deep well caused pressure marks on the edges of the screen, and a significant brown spot in the center of the display where the text became inverted.

Successive small pours worked better. Here the back-light reflects off of the edges of the epoxy that seeped under the screen before it finished setting. The display in this photo has a 3k3 series resistor in the backlight circuit.

The next attempt was much more successful, as I built up the epoxy a few mm at a time like the layers of an onion. As each layer hardened, it protected the screen from the contraction of the subsequent layers above.  The trick was to bring the first pour to the base of the pcb, and the second pour to “just barely” cover the surface of the screen. The epoxy penetrates about 1/3 of the way into the display housing but this does not interfere with readability as those edges are invisible under natural lighting conditions. That epoxy is actually under the LCD, in the air gap between the transparent glass LCD sandwich and the white reflector plastic which holds the thin LCD in place between the metal rim and the PCB.  I’ll try future pours at different angles to see if that lets the space under the LCD fill completely. Looking at the epoxy penetration, it’s clear that the black edges in pour #1 were places where the LCD was compressed on both sides, and the brown discoloration was from pressure on top with no support below. 

Seawater caused severe fogging of the potting epoxy after only three days in service. Originally a concession to my aging eyes, the large fonts really saved our bacon when this reaction occurred.

The results for the second batch looked good and the screens worked beautifully with full marine submersion for about two days. Then some kind of chemical reaction with the sea-water started fogging the epoxy, and by day three I was glad I’d created the large number fonts because the 5×7’s were completely unreadable.   Once we were back home, a bit of elbow grease & 800 grit removed the foggy surface rind, and a layer of conformal coating restored clarity. I think my next builds will add the coating to the epoxy surface at the start.

I also noted some screen discoloration from pressure at about 3m depth, indicating that even a thick layer of epoxy bows too much for a deeper deployment. I’ve ordered some 1/4“ plexiglass disks to provide a surface with a bit more chemical resistance, and will post an update on how that works after the next fieldwork trip. I’m hoping that provides a bit more pressure protection too, but the shore hardness of the epoxy is 85, and PMMA (plexiglass) is only a few steps above that at 90. I might try polycarbonate as well.

Other Fun stuff:

There is so much more to explore with this screen, including live graphing libraries, and display controls so I expect it will keep me amused for a while since I can add it to any of the current logger builds. Several are out in the wild now for long term tests, and I’m currently working on a script to move those fonts (and a few other things) into the 328p’s internal eeprom. If all goes well I’ll release that ultra low memory footprint version of the code shortly. 

Cheers for now.

Addendum 2018-08-24

After several builds using the this LCD screen I finally got around to storing those font arrays in the Arduino’s internal EEprom. Works a treat, and frees a good chunk of PROGMEM space with very little change to the core functions. With fonts in EEprom, the remaining Nokia 5110 functions compile to a little over 400 bytes of program storage and 10 bytes of dynamic. (not counting EEprom.h) And that’s with three copies of the output functions because of the simple 2-pass method I’m using to display the large numbers.  A small price to pay for live data output on our loggers!

Addendum 2018-10-17

Because of that pressure problem with the 5110 I decided to try out the 0.96″ OLED screens which sell for about $3 on eBay.  When the first batch arrived I was pleasantly surprised by how well they stood up to pressure on their surface. Then I found the SPI version of the SSD1306 OLED can be driven by essentially the same code as the PCD8544 (with the exception of the init & XY functions which are specific to each controller).

Adding the SSD1306 OLED Screen to an Arduino Logger (without a library)

I’m connecting the OLED with the same analog line connections used for the Nokia, but I’ve added a delayed-high RC bridge because the OLED is pickier about the reset input than the Nokias. In hindsight a similar method is probably a good idea for the Nokia screens as well, though you might need to experiment a bit with the resistor/cap values to get the timing right.

Tutorial: How to Configure I²C Sensors with Arduino Code

I’ve spent the last year in the ‘uncanny valley’ of the Arduino. That’s the point where you understand the tutorials at Arduino.cc, but still don’t get much from the material on gitHub because trained programmers would never stoop to using the wire.h library when they could just roll their own in native C++ using the avr-g compiler.  The problem with establishing sensor communication at the level of the TWI peripheral inside the AVR is that there are so many fiddling details to keep track of that it quickly overruns the 7±2 things this average human can hold in his head at one time: Computers aren’t the only things that crash after a buffer overflow!  So this post is meant to be a chunking exercise for beginner-intermediate level people who want to get a new sensor working using the standard IDE.  I’ve tried to distill it all down to things that I run into frequently, but there’s still a lot of material here:  So pour yourself a cuppa before diving in...

The great strength of I2C is that you can put so many sensors on the same four wires. But for units with several pre-made modules connected you might have to remove a few smd resistors from the breakouts, or the pull-up on the bus might become too aggressive. Most of the time I just leave them on, so I can extend the wire length, or crank up the bus clock…

REGISTERS are simply memory locations inside an I²C device. The summary of how many registers there are in a given sensor, and what they control or contain is called a register map. Most of the information on the sensor’s datasheet is about explaining how each register functions, and they can be quite a slog to read through because the information is rarely presented in an intuitive way.

To give you a sense of what I mean by that: take a look at page 14 of the manufacturers datasheet for the ADXL345 accelerometer:

A document only a hardware engineer could love…

Then take a look at the interactive register map for that sensor over at the i2cdevlib site:

Even if you’ve never worked with registers before, jrowberg’s visual grid layout makes it easy to see how the sensor’s memory is divided into sections, which are doing different things.

There are many kinds of registers but for this introduction I am going to group them into three general types: Control, Data and Status registers, and provide brief examples of code that you can use to work with each of them. The functions named with the i2c_ prefix should be generic enough to work with most I²C sensors, but I’ll also be referring to a few specific cases to show how you might need to modify those basic functions.

1) Control Registers

Most sensors change how they operate based on the values stored in control registers. Think of control registers as banks of On/Off switches, which you turn on by setting a bit to 1 and turn off by setting that bit to 0.  I²C chip-based sensors often have a dozen or more operational settings for things like bit-depth, sampling speed, noise reduction, etc., so you usually need to set bits in several different control registers before you can actually take a reading. And sometimes there are “special chip functions” that perform some kind of post processing on those sensor readings that would be hard to replicate on the Arduino. These can add an extra layer of control settings to take care of when you initialize the sensor.

Arduino’s wire library can only transfer 8-bit bytes over the I²C bus, so that’s the smallest amount of information you can write into a register memory location at one time. This can potentially change eight of those control switches simultaneously and, for parameters that are controlled by more than one bit, sometimes it’s actually required that you set them in one register-writing operation.  Most people use byte variables for the sensor’s bus and register memory addresses, but once you’ve figured out the pattern you need to set up in control register switch-bits, it helps to write that information as a long form binary number (eg. 0b00001111) so you can see the on/off states when you read through your code. 

Writing a byte to a sensor’s control register can be done with four basic steps:

Wire.beginTransmission(deviceAddress);  // Attention sensor @ deviceAddress!
Wire.write(registerAddress);   // command byte to target the register location
Wire.write(dataByte);                           // new data to put into that memory register
Wire.endTransmission();

The I²C deviceAddress is set by the manufacturer but some can be modified from their defaults by connecting solder pads on the breakout board.  Since the bus address of a given sensor IC can vary from one module to the next I keep Rob Tillaart’s bus scanner handy to find them, and more importantly to discover when two sensors are fighting with each other by trying to use the same address on the bus.  The registerAddress moves a pointer inside the chip to the memory location you specified. You can think of this pointer as a read/write head and once that pointer is aiming at a specific register, the next byte you send along the wires will over-write the data that was previously stored there.

The startup default values for a given control register are often a string of zeros because all the chip functions being controlled by that register are turned off. Unfortunately this means you’ll find lots of poorly commented code examples out there where people simply write zero into a control register without explaining which of the eight different functions they were aiming for because seven of those were still at their default zero-values anyway.

Reading data from a sensors memory register(s) requires two phases:

Wire.beginTransmission(deviceAddress);    // get the sensors attention 
Wire.write(registerAddress);    // move your memory pointer to registerAddress
Wire.endTransmission();           // completes the ‘move memory pointer’ transaction

Wire.requestFrom(deviceAddress, 2); // send me the data from 2 registers
firstRegisterByte = Wire.read();             // byte from registerAddress
secondRegisterByte = Wire.read();       // byte from registerAddress +1

The first phase tells the I²C slave device which memory register that we want to read but we have complete the read operation in two separate steps because the wire library buffers everything behind the scenes and does not actually send anything until it gets the Wire.endTransmission(); command.  The second phase is the data reading process and you can request as many bytes as you want with the second parameter in Wire.requestFrom .  The memory location pointer inside the sensor increments forward automatically from the initial memory register address for each new byte that it sends.

These simple patterns are at the heart of every I²C transaction, and since they are used so frequently, they often get bundled into their own functions:


byte i2c_readRegisterByte (uint8_t deviceAddress, uint8_t registerAddress{
byte registerData;
Wire.beginTransmission(deviceAddress);              // set sensor target
Wire.write(registerAddress);                                     // set memory pointer
Wire.endTransmission();
// delete this comment – it was only needed for blog layout.   
Wire.requestFrom( deviceAddress,  1);     // request one byte
resisterData = Wire.read(); 
// you could add more data reads here if you request more than one byte
return registerData;           // the returned byte from this function is the content from registerAddress
}
// delete this comment – it was only needed to maintain blog layout
byte i2c_writeRegisterByte (uint8_t deviceAddress, uint8_t registerAddress, uint8_t newRegisterByte
 {
byte result;
Wire.beginTransmission(deviceAddress);
Wire.write(registerAddress);  
Wire.write(newRegisterByte); 
result = Wire.endTransmission(); // Wire.endTransmission(); returns 0 if write operation was successful
// delete this comment – it was only needed for blog layout.
//delay(5);  // optional:  some sensors need time to write the new data, but most do not. Check Datasheet.
if(result > 0)  
{ Serial.print(F(“FAIL in I2C register write! Error code: “));Serial.println(result); }
// delete this comment – it was only needed for blog layout. 
return result;    // the returned value from this function could be tested as shown above
//it’s a good idea to check the return from Wire.endTransmission() the first time you write to a sensor 
//if the first test is okay (result is 0), then I2C sensor coms are working and you don’t have to do extra tests

//NOTE: copy/pasting code from blogs/web pages is almost guaranteed to give you stray/302 errors because
//of hidden shift-space characters that layout editors insert. Look at the line your compiler identifies as
//faulty, delete all the spaces and/or retype it slowly and carefully ensuring you enter only ASCII characters.


Those two functions will let you control the majority of the I²C sensors on the market, provided you can figure out the correct pattern of bits to send from the datasheet. A common strategy for keeping track of the multi-bit combinations that you want to load into your sensor control registers is to declare them with #define statements at the beginning of your program, which replace the human readable labels with the actual binary numbers at compile time.

For example the ADXL345 can range from 3 samples per second to 1600 samples per second, depending on four bits in the ADXL345_BW_RATE register. A set of define statements to represent those bit combinations might look like:

byte ADXL345_Address=0x53;     // the sensors i2c bus address (as a hex number)
byte ADXL345_BW_RATE=0x2c;    // the memory register address
#define ADXL345_BW_1600  0b00001111
#define ADXL345_BW_800    0b00001110
#define ADXL345_BW_400    0b00001101
#define ADXL345_BW_200    0b00001100
#define ADXL345_BW_100    0b00001011
#define ADXL345_BW_50      0b00001010
#define ADXL345_BW_25      0b00001001
#define ADXL345_BW_12      0b00001000
#define ADXL345_BW_6        0b00000111
#define ADXL345_BW_3        0b00000110
etc…. Note that all of these combinations assume normal power mode (bit4=0)

So a command to set the sampling rate to 50 Hz could be written as:

i2c_writeRegisterByte(ADXL345_Address, ADXL345_BW_RATE, ADXL345_BW_50);

 The cool thing about using defines is that they do not use any ram memory like byte variables would. And you can usually find code examples on gitHub where someone has transcribed the entire register address list into a set of defines, which you can simply copy and paste into your own code. This saves you a great deal of time, though there’s always the chance they made a transcription error somewhere. Also note that typical datasheets & ‘c’ language examples express those numbers as hex “0x0F” instead of “0b00001111” and you can leave them in that format if you wish.

Writing a whole byte to a register is pretty straightforward, but it gets more complicated when you need to change only one of the bit-switches inside a control register. Then the standard approach is to first read out the register’s current settings, do some bit-math on that byte to affect only bit(s) you want to change, and then write that altered byte back into register’s memory location.

But bit-math syntax is one of those “devils in the details” that makes relatively simple code unreadable by beginners. The bit operators you absolutely must be familiar with to understand sensor scripts you find on the web are: the bitwise OR operator [|] , the bitwise AND operator [&], the left shift [<<] and the right shift [>>] operators.  Fortunately there is an excellent explanation of how they work over at the Arduino playground, with a set of bit-math recipes in the quick reference section that let you reach into a byte of data and affect one bit at a time.  Be sure to parenthesize everything when using bitwise operators because the order of operations can be counter-intuitive, and don’t worry if you have to look up the combinations every time because most people forget those details once they have their code working. I know I do. 

Two particularly useful procedures:

x &= ~(1 << n);   // AND inverse (~) forces nth bit of x to be 0. All other bits left alone
x |= (1 << n);       // OR forces nth bit of x to be 1.  All other bits left alone

And these let us add a third function to the standard set which will turn on or turn off one single bit switch in a sensors control register:

byte i2c_setRegisterBit ( uint8_t deviceAddress,  uint8_t registerAddress,  uint8_t bitPosition, bool state )  { 
 byte registerByte, result;
registerByte = i2c_readRegisterByte ( deviceAddress,  registerAddress ); // load the current register byte
// delete this comment – it was only needed to maintain blog layout
if (state) {   // when state = 1
  registerByte |= (1 << bitPosition);   //bitPosition of registerByte now = 1
//or use bitSet(registerByte, bitPosition); 
  }  
else {           // when state = 0
   registerByte &= ~(1 << bitPosition);   // bitPosition now = 0
//or use bitClear(registerByte, bitPosition); 
  }
// now we load that altered byte back into the register we got it from:
result = i2c_writeRegisterByte ( deviceAddress,  registerAddress,  registerByte );
return result;   // result =0 if the byte was successfully written to the register


The ADXL345 accelerometer supports low power modes that use about 1/3 less power than the ‘standard’ operating modes.  The sensor is not turned off, but the bandwidth is reduced somewhat, so the sensor responds more slowly to things like tap inputs.
An example which sets the single bit enabling this low power mode might look like:

i2c_setRegisterBit( ADXL345_ADDRESS,  ADXL345_BW_RATE,  5,  );

Many I2C sensors have power saving features like that which rarely get utilized. Note that bit position numbering starts with 0 and counts from the left OR the right hand side depending on the sensor manufacturer. 

Some devices have control registers that are 16-bits wide. These get treated as a pair of 8-bit bytes that are read-from or written-to sequentially. You only have to specify the device & register address once at the beginning of the process because the sensors internal memory pointers get incremented automatically during the process.

This adds an extra wire.write step to the basic register writing operation:

Wire.beginTransmission(deviceAddress);
Wire.write(registerAddress);
Wire.write(MSB_registerData);    // Send the “upper” or most significant bits
Wire.write(LSB_registerData);     // Send the “lower” or least significant bits
Wire.endTransmission();

The MCP9808 is a common temperature sensor that uses 16-bit control registers.  Setting “bit 8” of the CONFIG register to 1 puts the sensor into shut down mode between readings and setting that bit to 0 starts the sensor up again. (yes, that’s opposite to the usual on/off pattern…)  The 8-bit limitation of the I²C bus forces us to retrieve the register in two separate bytes, so bit 8 of the 16 bits described in the datasheet ends up in the zero bit position of MSB. 

A custom function shutting down the MCP9808 might look like this:  

#define MCP9808_i2cAddress          0x18    // defines in setup are an alternative to using variables
#define MCP9808_REG_CONFIG   0x01    // the compiler swaps the text-name for the # at compile time
// delete this comment – it was only needed to maintain blog layout
void mcp9808shutdown()      //since we used defines, we did not pass any byte variables into the function

 byte MSB, LSB;
 Wire.beginTransmission(MCP9808_i2cAddress);
 Wire.write(MCP9808_REG_CONFIG);
 Wire.endTransmission();
// delete this comment – it was only needed to maintain blog layout
 Wire.requestFrom(MCP9808_i2cAddress, 2); //request the two bytes
 MSB = Wire.read();       // upper 8 bits described in data sheet as 15-8
 LSB = Wire.read();        // lower 8 bits described as 7-0 in the datasheet
// delete this comment – it was only needed to maintain blog layout
 MSB |= (1 << 0); // bitmath forces MSB bit0 (which is ‘bit8’ in the datasheet) to value one
 // using MSB &= ~(1 << 0); here would start the sensor up again by forcing the bit to zero
// delete this comment – it was only needed to maintain blog layout
 Wire.beginTransmission(MCP9808_I2cAddress);  // now write those bytes back into the register
 Wire.write(MCP9808_REG_CONFIG);
 Wire.write(MSB);                          // the one we modified
 Wire.write(LSB);                           // unchanged
 Wire.endTransmission();
}


This ‘position x becomes position y’ translation is common stumbling block for beginners working with multi-byte registers – especially when you add reverse order position numbering into the mix.  But there’s another gotcha with 
control registers that’s even more frustrating if you don’t catch it on your first pass through the datasheet:  Sometimes there are special “write protection” registers that have to be set before you can change any of the other control registers, and these have to be changed back to their “protecting” state before those new settings take effect. You might not get any error messages, but nothing will work the way it should until you get the protection bits disabled and re-enabled in the right sequence. Fortunately less than 20% of the sensors I’ve worked with have this  feature.

Another thing to watch out for are old code examples on the web that appear to be using integer variables to store device and memory register locations, with statements like Wire.send((int)(eepromaddress >> 8));  The I²C wire library only sends bytes/uint8_ts, but people got away with this (int) cast  because it was being corrected behind the scenes by the library, which re-cast the value into a byte at compile time.  The (byte) data type on Arduino is interchangeable with the (uint8_t) variables you see in most C++ coding tutorials. 

2) Data registers

Unlike a control registers bank-of-switches, I think of data output registers as containers holding numbers which just happen to be stored in binary form. Since eight bits can only hold decimal system values from 0 to 255 you usually have to “re-assemble” larger sensor output values from bytes stored in consecutive memory locations. For sensors like the ADXL345 you can concatenate the two 8-bit bytes into one 16-bit integer variable by shifting the MSB left by 8 positions and merging in the LSB with a bitwise OR :

Wire.beginTransmission(deviceAddressByte);  // the pointer setting transaction
Wire.write(registerAddressByte);
Wire.endTransmission();

Wire.requestFrom(deviceAddressByte,2);       // request two bytes
LSB = Wire.read();                                                // byte from registerAddressByte
MSB = Wire.read();                                              // byte from registerAddressByte +1
int combined = (int)MSB;             // MSB now in rightmost 8 bits of combined int
combined = combined<<8;          // shift those bits to the left by 8 positions
combined |= LSB;     // logical OR keeps upper bits intact and fills in rightmost 8 bits

Those steps are usually written in one single line as:

int combined = (((int)MSB) << 8) | LSB;

There are several other ways to combine bytes and some sensors send the MSB first – so you have to check the register map in the datasheet to know the order of the bytes that arrive from the output registers when you request multiple.

Now if you are thinking that looked too easy – you’re right! Most hobby market I2C sensors only have a 12-bit ADC, and since memory is a limited resource there are often status register bits mixed in with the data held in the MSB. Since these bits are not part of the sensor reading, you need to &-mask them away before you combine the MSB & LSB. It gets trickier when the sensor output can be a positive or a negative number because signed and unsigned integers are distinguished from each other by a special “sign” indicator bit, which can accidentally be turned into a “number” bit by bit shifting. (see: ‘sign extension’ in that bit math tutorial )

The temperature data output register in the MCP9808 is a good example of both of these issues:

Bits 15-13 (which become the top 3 bits of the upperByte in the code below) are status indicator flags identifying when high & low temp. alarm thresholds have been crossed. Bit 12 is a sign bit (0 for +ve temperature or 1 or -ve temps). The remaining bits 11-8 (=bits 3-0 of the upperByte) are the most significant 4-bits of the 12-bit integer representing the temperature.

So a sensor-specific approach to reading the temp. from an MCP9808 might look like this:

int TEMP_Raw;
float TEMP_degC; 

// spacer comment for blog layout
Wire.beginTransmission(0x18);    // with mcp9808 bus address written in hex
Wire.write(0x05);                             // and the temperature output register
Wire.endTransmission(); 
Wire.requestFrom(0x18, 2); 
byte UpperByte = Wire.read();          // and sometimes the MSB is called the “highByte” 
byte LowerByte = Wire.read();          // sometimes called the “lowByte” 
// spacer comment for blog layout
UpperByte = UpperByte & 0b00011111;  // Mask away the three flag bits
//easier to read when the mask is written in binary instead of hex
// spacer comment for blog layout
//now we use a mask in a slightly different way to check the value of the sign bit:
if ((UpperByte & 0b00010000) == 0b00010000)  {          // if sign bit =1 then temp < 0°C
UpperByte = UpperByte & 0b00001111;                             // mask away the SIGN bit
TEMP_Raw = (((int)UpperByte) << 8) | LowerByte;    // combine the MSB & LSB
TEMP_Raw-= 256;   // convert to negative value: note suggested datasheet calculation has an error!
 }
else  // temp > 0°C  then the sign bit = 0  – so no need to mask it away
 {
TEMP_Raw= (((int)UpperByte) << 8) | LowerByte;
 }
// spacer comment for blog layout
TEMP_degC =TEMP_Raw*0.0625;


Typically a data output register will continue to hold the last sensor reading until it is refilled with the next one. If your sensor takes a long time to generate this new reading (30-250 ms is typical, while some can take up to a second) and you read the registers before the new data is ready, you can end up loading the previous sensor reading by mistake. That’s where status registers come to the rescue.

3) Status registers

These tell you if if a specified type of event has occurred and I think of these registers as a set of YES/NO answers to eight different questions. The most commonly used status register is data ready [usually labeled DRDY] which sets a bit to 1=true when a new sensor reading is available to be read from the related output registers. Another common status register is one that becomes true if a sensor reading has passed some sort of threshold (like a low temperature alert, or a falling/tilt-angle warning).

A function to check the true=1/false=0 state of a single DRDY bit inside an 8-bit status register might look like this: 

bool i2c_getRegisterBit (uint8_t  deviceAddress, uint8_t  registerAddress, uint8_t  bitPosition) {     
byte registerByte;
registerByte = i2c_readRegisterByte(deviceAddress, registerAddress);
 return ((registerByte >> bitPosition) & 0b00000001);  // or use (bitRead(registerByte, bitPosition))
 }
// delete this comment – it was only needed to maintain blog layout
 //  You could use i2c_getRegisterBit to check the DRDY status bit with a do-while loop
//  and only move on to reading the sensor’s data output registers after the DRDY bit changes to 1
// delete this comment – it was only needed to maintain blog layout
bool dataReady=0;
do {
dataReady= i2c_getRegisterBit(deviceAddress, statusRegAddress, DRDYbitPosition);  
} while ( dataReady==0 );        // processor gets cycled back through this loop until DRDY=1


Holding the processor captive in a status-bit-reading loop is very easy to do, but it is usually avoided unless you are trying to capture a series of sensor readings quickly.  Most status register bits can be mapped to physical alarm output lines on the sensor module, and these can be used to trigger a hardware interrupt pin (D2 & D3) on the Arduino.  This lets you to setup an interrupt service routine (ISR) which grabs that new reading even faster than a bit reading loop. And since hardware interrupts can be used wake a sleeping processor, the interrupt method also lets you put your data logger to sleep until something actually happens. 

The only drawback to the ISR method is that the sequence of register settings you need to follow to create hardware alarms is another multi-step process to add to your sensor initialization code.  The conceptual pattern is usually something like:

  1. Disable the sensor’s global interrupt control bit (if there is one)
  2. Enable the sensors triggering function   (eg:  a max. temperature alert)
  3. Load register(s) with the parameter value for that trigger (eg:  52.5°C)
  4. Turn on the status register that listens to that triggering function
  5. Map that status register bit to a hardware output line
  6. Re-enable the global interrupt control bit

This LSM303 combined accelerometer / magnetometer sensor has two alarm outputs in addition to DRDY. So you could map the Accelerometers DRDY signal to int1, and the Magnetometers DRDY to DRDY.  Just to make life interesting with this sensor, the 3-axis output data  registers are arranged in a different order  on the magnetometer than  they are on the accleerometer. This is typical for multi-sensor chips, which you handle like separate sensors even if they come in the same package – you can even put one to sleep mode while the other one is taking a reading.

Sensors can have many different status monitoring functions, but they usually have only one or two hardware alarm lines.  So the status register -> hardware output mapping (step 5) listed above sometimes involves its own sequence of register settings.  As example, the ADXL345 reads acceleration on three axes, and it has double-tap detection functions for each x,y,z direction. But the Arduino only has two incoming hardware interrupt lines. So generally speaking, you would map all three of those tap-detect status registers to the same outgoing alarm line on the sensor module, and then have the program figure out which axis actually triggered the alarm by reading the status registers later on. High & Low temperature sensor alerts are often mapped in a similar fashion because many breakouts only have one outgoing line: especially if the DRDY status register has been permanently connected to the only other physical alarm line.

A conceptual twist here is that most of the time, the hardware output actually moves the line LOW when the alarm is triggered, even if the status bit it’s mapped from is true=1=high when the actual event occurs. No matter what the status bit->alarm pattern is, any of the four possible interrupt triggers: HIGH, LOW, RISING & FALLING can be used to wake a sleeping 328p processor (though the datasheet states differently).  

Another thing to watch out for on the Arduino side is setting your ISR to respond to HIGH/LOW levels rather than RISING/FALLING edges: Level based interrupts will keep triggering as long as that line is HIGH/LOW. This could cause a sketch to run extremely slowly until the interrupt handler is disabled in your program. Even old analog reed-switched based sensors can suffer from this type of issue, as its not uncommon for something like a wind sensor to stop spinning right where the magnet is holding the reed-switch closed.  The thing that makes this choice somewhat tricky is that the most common type of sensor failure I see is one where the alarm stays on permanently.  If you set your interrupt to respond to LOW,  and the sensors starts self-triggering your event counters get pushed up to ridiculously large numbers – so it’s very easy to spot that failure in the data, and by the fact that the logger is usually kept awake till the batteries run dry.  If your ISR responds to FALLING, your counts go to almost zero in the same situation, and depending on the phenomenon you are recording it could be very easy to miss that a sensor problem has developed.  

For more information, there’s an excellent guide to interrupt handling over at the Gammon Forum. Probably the most important thing to keep in mind about using interrupts is that by default all interrupts are disabled once you are inside an interrupt subroutine so that the ISR can’t interrupt itself and create an infinite-recursion situation that over-runs the memory.  But the I2C bus relies on interrupts to function, along with timers and other important things.  So don’t try to change a sensor register while inside the ISR,  just set a volatile flag variable and deal with resetting registers later in the main loop.  The general rule of thumb is: “get in & get out ” as fast as possible, and I rarely have a sensor triggered ISR longer than this:

void  INT1pinD3_triggered()  {   INT1_Flag = true;   }

though sometimes I’ll also detachInterrupt(interrupt#) inside the ISR, to make sure it only fires once for things like button de-bouncing. 

Status registers are usually latched, and have to be reset by the I²C master after they are triggered. DRDY registers are cleared by reading information from the data registers they are associated with.  Most other status registers are cleared by reading the register’s memory location, which also turns off the hardware alarm signals that were mapped from them.  This is different from control registers which always have to be explicitly over-written to with new information to change them. If you are waking up a sleeping data logger based on something like a high temperature alert, you usually read the status registers to clear those alarms before enabling interrupts and putting your logger into a power-down state. Threshold based alarms allow interesting things like burst logging.

In Summary:

A good register map, and the four generic functions I’ve described here

  1. i2c_readRegisterByte
  2. i2c_writeRegisterByte
  3. i2c_setRegisterBit
  4. i2c_getRegisterBit

Should be enough to get a typical I²C sensor running, and you can easily tweak those functions to make custom versions for reading 16-bit registers and/or to mask the cruft out of data pulled from mixed registers.

After testing an I2C sensor combination, I pot them in epoxy. Detailed instructions here.

Initializing an I²C sensor is a multi-step process and the correct order of operations is often poorly explained in the data sheet because they are usually written “in reverse”.  Instead of a straightforward list saying “To get a reading from this sensor, do (1),(2),(3),(4), etc.” you find descriptions of the control register bits saying “before you set bit x in this register you must set bit y in this other control register”. When you look up that other control register you find that it too contains a sentence at the end saying “before you set bit y in this control register you must set bit z in this other control register”. So you have to work your way through the document, tracing all those links back until you find the things you were supposed to do first.  Finding the “prime control bit” can be such a time consuming process that it’s not unusual for people who figure out the sequence to wrap it all up into a sensor library so they never have to look at that damn datasheet ever again.

But if you use those libraries, keep in mind that they are probably going to configure your sensor to run at the highest possible bit-depth & data rate, unnecessarily burning away power in applications like data logging which might only need one reading every fifteen minutes.  So the majority of off-the-shelf sensor libraries should be seen as partial solutions, and you don’t really know what else your sensor is capable of until you read through the datasheet yourself.  As an example there are IMU’s out there that will do Euler angle calculations if you simply turn on those functions with the right control register. But libraries for those chips sometimes enable the bare minimum data output functionality, and then do computational handstands to accomplish those gnarly (long) calculations on the Arduino’s modest µC.

In addition there can be useful sensor functions hidden in plain sight, because the datasheet tells you how to turn them on & off, but gives you no clue when to do so. An example here would be humidity sensors like the HTU21D which has an on-chip heating element to help the sensor recover from long periods of condensation, but no status alert that would let you do this automatically. You could just run the heater once a day, but there is also no indication how long the sensor would last if you did that – just some vague references to “functionality diagnosis”. But then some manufacturers (Freescale and Sensirion come to mind…) commit more than just sins-of-omission, breaking away with non-standard I²C bus implementations to lock in customers. The logic there is that if you have to buy the one great sensor that only they make, it’s easier to buy the other four sensors for your device from them as well, rather than juggling low-level protocol conflicts. 

Another challenge when you are working with a new sensor is that Arduino’s C++ environment is not the same as vanilla C in some important ways. So many of the tutorials you find will describe methods that won’t work on an Arduino. Even when the code does compile, there are a number of different “styles” that are functionally identical when they pop out the other side of the compiler, so I’m still trying to wrap my head around the syntax that turns arrays into pointers when they get passed into functionsThat’s why I didn’t mention I2C eeproms in a post about memory registers: almost every multi-byte read/write example out there for EEprom’s uses array/pointer/reference thingies. If you absolutely have to read a series of sensor output registers into an array with a loop, my advice is to just make it a global until you really know what you are doing. And don’t try to store numbers in a char array, because the “temporary promotion” of int8_t’s to 16-bit during some operations can bung up the calculations.

But now it’s time to bring this thing to a close. While I’m still thinking about stuff I wish I’d known earlier, it occurs that a good follow-on to this post would be one about techniques for post-processing sensor data.  There are plenty of useful methods like Paul Badger’s digital smooth, and other code tricks like wrapping those functions in #ifdef #endif statements so those routines only get compiled when a sensor that actually needs them is connected to your logger.

That will have to wait for another day so for now I’ll just sign off with some links. Except for that last ranty bit, I’ve tried to stay out of the I²C handshaking weeds, because when you are up to your neck in bit banging, it’s easy to forget you were trying to measure the water level in a swamp.  But if that’s your thing, there’s some more advanced I²C code examples over at the Gammon Forum, an in depth reference to the Wire library at the Arduino playground , and some troubleshooting tips over at Hackaday.  Its also worth noting that I’ve used bit-shifting to extract bits, and concatenate 16-bit values from 8-bit registers. But you sometimes run into examples where people have uses structs & unions to do those tasks in a much more elegant way.

Addendum 2017-11-04

I wonder how many other sensors I could use this with? And if my pin-toggled oversampling method works on the ATtiny, this might provide better resolution than some commercial sensors; though I guess that would depend on how much I could squeeze into only 512 bytes of SRAM…

Somehow I always seem to run into a bunch of related material the day after I post something to this blog: There’s a cool little project over at Quad Me Up using ATtiny85 to turn an analog light sensor into an I2C slave device.  AN4418 from Maxim explains how to use I/O extenders to connect a compact-flash (CF) cards to the I2C interface, which is something I never thought I’d see. And then theres AN10658 from NXP with a method for sending I²C-bus signals over 100m. My own tests with the I²C sensors just hanging off the Arduino only reached about 20m.

 

Addendum 2017-11-05

Koepel over at the Arduino forum pointed out that the IDE supports some handy macros like bitSet(), bitClear(), and bitRead() that could replace the bit math & masking functions I described here:

byte resisterData = Wire.read();    // byte from register
bitSet(resisterData, 6);                   // change bit 6 to a 1

These handy macros are particularly helpful when setting the internal behavior of the processor on the Arduino board because all the bits in those registers have names that the IDE can translate into actual numbers: 

bitSet (DIDR0, ADC2D);     // disable input buffer on A2 by setting ADC2D bit in register DIDR0

Another highly useful trick is to to use bit_is_set or bit_is_clear macros to poll the value of a flag register inside a while loop to create short-duration conditional delays while some process occurs:

while (bit_is_set(ADCSRA,ADSC));   // waits here for ADC conversion to complete

There’s also word(h , l) to combine two bytes, or highByte() and lowByte() to divide 16-bit variables into 8-bit two pieces. These macros may not travel well outside the Arduino IDE, so most programmers avoid using them, preferring to write out the explicit code.

So bitSet replaces x |= (1 << n);  and bitClear can be used to replace x &= ~(1 << n); in in the standard I2C functions I described in the post. Many programmers do not like the code underlying these kinds of compiler substituted macros, so the bit math expressions are far more commonly used because they are generally faster to execute. The AVR can only shift bit positions by one bit per clock cycle (so <<n, takes n clock cycles) but that’s still faster.

There is one other bit-math expression I use frequently, when I want to toggle the state of I/O pins, for example:  PORTD ^= B00001000;  (toggles the state of only bits with a “1” on the right hand side of the expression)  That is a special use case for rapid port switching on the Arduino, rather than for sensor register bits.

Addendum 2017-11-06

I’ve also just found out that there are a small number of sensors there that require a ‘false’ modifier to be used at the end of an I²C transaction:  Wire.endTransmission(false);   This is called a repeated start, and the I2C master does not release the bus between writing the register-address and reading data with Wire.requestFrom();   The sensor responds to the I²C address with an acknowledge at the begin of the I²C transaction, and to each databyte that is written to the sensor, so the error code returned by endTransmission can still be used because it is a test if the I²C address was acknowledged by the sensor.

And there was another I²C quirk mentioned at the Gammon Forum:

“You can’t rely on the slave necessarily sending the number of bytes you requested. If you request 10 bytes, Wire.requestFrom() will return 10, even if only 5 have been sent. For the slave to terminate the communication early (ie. after sending less than the requested number of bytes) it would have to be able to raise a “stop condition”. Only the master can do that, as the master controls the I2C clock. Thus, if the slave stops sending before the requested number of bytes have been sent, the pull-up resistors pull SDA high, and the master simply receives one or more 0xFF bytes as the response…It does not help to use Wire.available() because that too will return the number of bytes requested.”

Addendum 2017-11-08

On my page about the DS3231 rtc I describe how to power that I²C chip from a digital pin during bus communication. That trick only works because the chip was designed to gracefully fail over to a backup coin-cell power supply. With other I²C sensors a leakage current might flow into the sensor through the pullup resistors, so you would have to power the bus pullups with the same digital pin to avoid this. And since the internal pullup resistors are enabled by default in the Wire library, you have to disable I²C before you could pin power that I²C device.  Also don’t try to de-power a whole module with decoupling capacitors through a digital output pin as that creates big current spikes and really needs proper switching with a PNP transistor of p-channel FET.  99.99% of the time its better to simply find a sensor with a really low sleep current sleep state that you can enter by setting a control register. The best sensors are ones that automatically go into these low current standby-states whenever they detect no traffic on the I2C bus: then you don’t have to do anything.

Another thing I discovered while working with that RTC was that it had a Wire.begin() call hidden in the library, but I was already starting the I²C bus normally during setup. So without knowing it the I²C hardware was being initialized a second time. As the I²C peripheral registers are set to the same value as in the first Wire.begin() call nothing bad happened. However I can see where it might get’s problematic if you call Wire.begin() accidentally because it was buried inside some sensor library while you were running a data transfer,  and the hardware is re-set to an idle state. 

Addendum 2017-11-09

Most of us are familiar with trying out different libraries to drive a sensor, but I’d be remiss if I didn’t mention that there are also some alternatives to Wire library for I²C. The one that gets the best reviews is the I2C Master Library developed by Wayne Truchsess at DSSCircuits. This lib has a faster throughput and a significantly smaller code size: the original Wire library adds about 796 bytes to your sketch when included whereas Wayne’s I²C library only adds 140 bytes. And it has built-in commands that replicate all of the functions I described in this post. For 16 bit registers Wayne points out:

“Technically when sending bytes to a slave device there is no difference between data and an address. In other words let’s say you have a three byte address and three bytes of data. You could use the write(address, regaddress, *data) by making the first byte of your multibyte address equal to regaddress and then combine the rest of the address and data together into *data.”

and that’s equally true with the wire library. The memory savings alone would be worth exploring, but perhaps I²C Master library’s most compelling feature is a ‘TimeOut’ parameter for all bus coms, which could keep your logger from getting stuck in a while-loop if one of your sensors goes AWOL, though I wonder if it still has the 0xFF problem mentioned above, if the sensor sends less than you requested?

Addendum 2017-11-10

I thought using an ATtiny to convert an analog sensor into an I²C device was a neat trick. But it seems that Andreas Spiess has taken the idea to a new level with three HC-SR04’saccessible through on a single AT.  His youTube video #42 with three Ultrasonic Sensors for Arduino walks through the process, with a vocal track that leaves you thinking Werner Herzog has started doing maker videos. I am impressed with what people achieve using those little processors.  The basic idea of reading an analog signal and turning it into PWM output is common to both amplifiers and sensor applications like light->frequency converters.

Addendum 2017-11-13

The IDE compiler has an annoying quirk when it runs into Wire.requestFrom in those I2C register routines because the compiler throws up warning messages whenever it feels it has to resolve an ‘ambiguous’ call:  (click to enlarge)

 Turns out that requestFrom has two different implementations, one that can take int arguments, and one that takes uint8_t arguments. If you put in something which has no type like a number (or something you declared with a #define) the compiler has to decide which implementation to use. In the case shown above it chose to use the (int,int) flavor even though device address was specified as uint_8 at the start of the function. 

Anyway, to make those warnings disappear, simply cast the two parameters in Wire.requestFrom to either (uint_8) or (int):

And all those compiler warnings will disappear.

Addendum 2017-12-14

Single I2C/SMBus Address Translator for those times when you have an unavoidable sensor bus address conflict. Or you can use an I²C multiplexer like the TCA9548A over at Adafruit which will let you use one I²C address to talk to the multiplexer and tell it which lane you want to enable.

 

 

 

Measuring EC ( electrical conductivity ) with Arduino

This post is a summary of my background research into electrical conductivity to serve as a backdrop for my own humble attempts at this interesting measurement challenge. I’m sure there are many other approaches that I’ve yet to discover, and if you know of one please leave a comment so that we can pass that knowledge on to others – Ed.

Obligatory blog-post monkey shot.

Pete & Trish doing profiles with a YSI EXO. As you might imagine, these puppies are pretty expensive. Now that we have A Flexible Arduino-Based Logging Platform to build on,  adding conductivity is our #1 priority.

The conduction of current through a water solution is primarily dependent on the concentration of dissolved ionic substances such as salt. Since most fresh water derives from relatively clean rainfall, variations in EC provide a way to track the chemical  and hydrological processes the water has been subjected to over time. High amounts of dissolved substances (usually referred to as salinity) can prevent the use of waters for irrigation and drinking, so conductivity ranks as one of the most important inorganic water quality parameters.

A huge number of resources are dedicated to measuring EC and rather than re-hashing all that material, I thought I’d start with links to a few good background reads:

Conductivity, Salinity & Total Dissolved Solids
-discusses the older TDS measurements in parts per million ( ppm ) which makes assumptions about the charge carriers that don’t reflect real world environments.  The conversion factor from EC (which is the thing you actually measure) to TDS changes for different dissolved solids, so instruments from different manufacturers often give you different TDS readings for the same solution, because the companies made different assumptions about what’s in your water.  Because of this confusion, straight EC measurements in siemens have been adopted as the standard by the international scientific community. One siemens is equal to the reciprocal of one ohm (S = 1/Ω)  and is also sometimes also referred to as the mho (℧) in older literature.

Conductivity Theory & Practice
-a white paper that covers basic probe designs, and mentions some non intuitive things like geometry/field effect errors.

Conductivity Sensing at PublicLab.org
-many groups at PublicLab.com have been working on different types of conductivity sensors and their overview page is another excellent introduction to DIY approaches. In fact it’s so good that I will be referring to several of those projects in this post.

Aqueous conductivity is commonly expressed in millisiemens/cm (mS/cm) and natural waters range from 0.05-1.5 mS/cm for freshwater lakes & streams up to about 55mS/cm for sea water. Water up to 3 mS/cm can be consumed, though most drinking/tap water is below 0.8 mS/cm.  Many of the Cave Pearl Project’s installations are in coastal areas where tidally driven haloclines require our instruments to cover that entire “natural waters” range.  Groundwater can vary even more, with measurements being complicated by organic acids and/or significant amounts of dissolved limestone.  Salt water is chemically aggressive and water hydrolyzes above 0.4v, so the probes for high-conductivity environments are usually made of resistant materials such as platinum, titanium, gold-plated nickel or graphite, making them somewhat expensive.

Ways to Measure Conductivity:

There are so many different approaches to measuring EC that it’s taken me a while to digest it all into some working categories.  I expect to build at least one prototype for each of these methods just to see if I can make it work.

Density Based Methods

Refractometers and density based hydrometers are used by aquarium hobbyists. Better quality acoustic doppler flow sensors can also calculate density based on the speed of sound through the water and infer salinity from that. Given then number of acoustic anemometer projects out there, I’m surprised someone has not already adapted the method for underwater applications, though this may be due to the timing limits of the affordable transducers.

Resistance Based methods:

a) Use submerged probes as part of a resistor divider / bridge :
This common approach measures the resistance between two probes using some type of voltage divider. Resistance =1/conductance, which allows you to derive conductivity with your cell K constant since conductivity=(conductance * length)/(area).  AC oscillators are tacked on to reduce electrode polarization, and this forces you to add even more electronics on the output side to convert the signal back to DC for reading. The resistance between the probes changes by several orders of magnitude in environmental waters so different probe surface areas & divider resistors are usually required to cover a significant conductivity range. Above 50% sea water, the resistance between the probes doesn’t change very much, so this method tends to get used more frequently for fresh water environments.

b) Change the pulse frequency of a 555 timer circuit:
You can use resistance between the electrodes as part of an RC relaxation oscillator and then measure the 555’s square wave output frequency to determine the resistance.

This circuit from Thomas Allen’s site provides galvanic isolation, uses AC measurement, and the output frequency varies from about 42 Hz with the probes in air to > 8000 Hz depending on conductivity. You can buy this circuit on pre-made a module for $24  from the EME site.

Circuits and instructions can also be found at at PublicLab.org and there are many good tutorial videos describing 555 based EC sensors on YouTube.  At this point I’ve run into so many projects using this chip that I’d  be willing to bet every environmental sensor I’ve ever heard of could be cobbled together from a few op-amps and a low voltage 555 timer . There are several frequency counting libraries available to help you get started, and if you are ready to sink your teeth into some code, Nick Gammon has produced the some elegant solutions for pulse/frequency timing. Note there are some duty cycle issues  (also see: Schroeder Thesis)

c) Time the discharge of a capacitor through the solution:
Jim Conner’s describes this method in his youTube video at
EC Probes – How they work, and how to build one


A circuit like this might be easy to implement on an Arduino if you can put the internal 1.1v reference onto the comparator that’s also built-in to the 328.  Microcontrollers count time with far better resolution than you get from their ADC’s, but that doesn’t mean there aren’t other issues to deal with. Given that you can try this method with practically no extra circuitry, I will definitely be prototyping a few of these.  Like the 555 based circuits, it will be interesting to see if the method bumps into timing & interrupt handling limits (100 kHz?) when you use it with seawater.

Capacitance based approaches:

You often see capacitance used for liquid level sensors and soil moisture probes, and some of these could be adapted for EC.  To me, the raindrop detection pcb’s you see on eBay have always looked like prime candidates for re-purposing as capacitive sensors.

The circuit described for the Chirp Moisture Sensor uses a fixed resistor and a non-contact probe to create a low pass filter whose cutoff frequency changes with capacitance,  which is affected by the electrolytes in solution. This filters an 1-8MHz square wave and the output voltage is accumulated other side of simple diode peak detector circuit for reading. 

You can also vary 555 timer output frequency by changing the capacitor in the tank circuit, or create more complicated oscillator circuits. No matter which cap-based method you use, the supporting electronics have to be located near the sensor – because just about any length of wire will add enough stray capacitance to throw off your measurements. Another thing to keep in mind is that common ceramic capacitors have some of the worst thermal coefficients and aging effects imaginable. Plastic film capacitors using Polyphenylene Sulfide (PPS ±1.5%) or Polypropylene (CBB or PP ±2.5%) are much better for sensor circuits like this, and having a digital capacitance meter on hand is probably a good idea too.

DIY capacitor plate sensors are going to be small values (picofarads ?) and the resulting RC time constants make these methods more suitable for fresh waters, as this leverages the relatively large solution resistance to give you more accurate interval timing.

Potentiometric (4 electrode) Methods

Four-electrode cells uses two “driver” pins to place an electric field across two other “reading” pins that lie between them: 

This paper describes a DIY 4-probe sensor that was used for soil moisture sensing, and you will find quite a few articles using potentiometric methods over at IEEE and Sensors

Nokia/Apple audio jacks came to mind as soon as I saw this diagram, and they might be available with gold plating.  4-electrode methods often measure the voltage between the read pins, which is divided by the exciter pin current to determine the solution’s impedance = 1/conductance.  To obtain the conductivity, the conductance is multiplied by the cell constant of the inner poles. Tracking the pin current lets you compensate for fouling on the plates, and the method can cover a wide range of conductivity. Like inductive methods, this approach tends to work better as the concentration increases. 

Inductive Methods

The conductivity measurement is made by passing an AC current through a toroidal drive coil, which induces a current in the solution. This induced solution current, in turn, induces a current in a second coil, called the pick-up toroid. The amount of current induced in the pick-up toroid is proportional to the solution conductivity. You get industrial grade performance out of this non-contact method in many different types of solutions, but you also need industrial amounts of power the drive the sender coil so it’s hard to implement on the kind of power constraints you see on stand-alone data loggers.  Inductive sensors require a 3 inch radius from any other surface (bio-fouling?) and you see this pretty clearly in the ‘donut on a stick’ sensor heads. It occurs to me that you see very similar components in a wireless charging system, but there’s a lot of devils hiding in those details – like shielding, etc.  It might be possible to press one of the production line proximity sensor chips into service for a low power solution, or simply try measuring changes in inductance due to the presence of salt water.

Off-the-shelf Solutions for Arduino:   (using 2-Electrode Resistance Methods)

TransparentSinglePixl
Atlas Scientific Conductivity Kit
A complete solution including calibration solutions, a range of probes and code libraries. All parts also sold separately: interface boards are ~$35 & EC probes come in around $120 each but they are durable enough for continuous long term submersion.  I2C data transfer is supported, so resolution is not limited to the Arduino’s ADC.  Whitebox labs Tentacle Shields ($35-$110) provide up to four galvanically isolated channels for full hydroponic rigs. Stand alone BNC carriers for $10.
$200
CN0349 Conductivity Measurement System
The EVAL-CN0349-PMDZ has total error less than 1% FSR after calibration. The digital output is fully isolated eliminating ground loop interference. Even if you are using a different circuit at the sensor, its work looking at how they did that.  Thanks for the tip about this one from Joshua Girgis: “Its designed to work as a benchtop sensor but one can easily wire it up to the i2c lines on an Arduino. The code is a little cumbersome but I have it working for taking temperature and conductivity measurements for sea water.”
$45
Gravity: EC Sensor Kit for Arduino (K=1)
Another complete K=1 kit solution, but the probes are not robust enough for long term submersion so several people replace the stock probe with the 208DH which is available on eBay for $35. Arduino ADC reads voltage.  The KnowFlow project uses the full set of DFrobot boards. DFR also has an inexpensive TDS kit, which cfastie has been testing over at publiclab.org
$70
Vernier CON-BTA EC probe
This 5v K=1 probe covers 0-20,000 μS/cm  in the high range, and the analog voltage output is read by the Arduino ADC. You need an inexpensive adapter board for the BTC connector, and they provide a basic library. One key feature is built in hardware temperature compensation with a 10k thermistor in the probe head. My tests show this reduces the usual 2% / °C reading variation down to about 0.5%/° C, so you still need to do your own calibration to get higher accuracy. Like Atlas Scientific, Vernier has many other interesting sensors that are Arduino compatible. (Much cheaper on Ebay for older stock)
$115
EC/pH Transmitters
This company offers a range of physically bulky turn-key solutions, with the $70 entry level unit claiming 0-5000 μS/cm (fresh waters) and continuous monitoring. Arduino ADC reads voltage. ~$200 units support PH with isolation.
$70-250
Sparky’s widgets MiniEC
An indie who makes several other useful sensor breakout boards, including PH. You have to build or locate your own probes, though they use a standard BNC connector like most EC probes.  Arduino ADC reads voltage output.
$24
EC-Salinity Probe Interface by Ufire
Designed around an ATtiny configured as an I2C slave, probably using the cap-discharge method.
$14.50
Hanna HI 73311 (K=1) Replacement probes
In the past we’ve used used these epoxy&graphite probes from Hanna DIST5 (HI 98311) and DIST6 (HI 98312) testers, which connect to a standard male audio jack.  You can also re-purpose one of the Vernier ABS/graphite probes if you get a used one cheap on eBay, and the Vernier probes have a 10k NTC thermistor built in, which you can read with a divider. The best completely DIY probes I’ve ever seen are the concentric electrodes built by Camilo Rada with epoxy & graphite rods.
$55
Comercial Standard Solutions
For fieldwork, it’s often easier to transport the dry packets, and mix them on location.  Atlas sells calibration sets, but at the twice the cost of standards when you buy them in larger volumes.  You can find recipes for homemade calibration solutions at Reefnet Central and PublicLab. For a classroom situation, it’s much cheaper to mix secondary “lab standards” in larger quantities, and then test the resulting solutions with a commercial probe that’s been calibrated against commercial solutions. 5.566g of dry NaCl in 1 litre of distilled water will create an ~10,000 μS/cm solution, which you can dilute down for lower concentration standards.
$14/500ml

This photo from Bitnitting gives you a sense of the physical space needed for the Atlas breakouts and a ‘mini’ form factor Arduino.

Hydroponics hobbyists have putting these kits to good use over the years with notable examples like the long running forum thread on Billies Hydroponic Controller, and the well documented adventures over at the Bitnitting Blog.  The people at OpenCTD and other academic projects have put the Atlas boards into real world deployments.

But to me these commercial solutions still leave you stuck with those expensive electrodes which sometimes cost more money than you would pay for a used 4-pole device. More annoying is the fact that those cell constants do not line up with my goal of measuring the entire “fresh” to “marine” range with one sensor, thought if I could extend it a bit the K=10 probe comes close.  This is illustrated by the following graph from Andy Connelly’s Blog which is worth digging through as he has posted lots of other interesting material on calibration, reproducibility, signal detection, etc. 

 

Of course the cell constant changes as your probes get older and dirtier, so you have to re-calibrate them with standard solutions just about every time you want to take a new reading. I’m pretty sure I will end up making my own probes, probably out of Nichrome 80 wire as the vaping fad has made it common on eBay. Some have had good EC results with gold plated PCB traces. Feedback on the Arduino.cc forum suggests that Platinum-Rhodium Thermocouple Wire is another good option.  I’ve also been wondering about Ag/AgCl which is highly resistant to seawater and is commonly used for non-polarizing electrodes in medical/bio applications. (EKG electrodes?) It might also be a good idea to cobble together a DIY magnetic stirrer, based a PC fan and an old hard drive magnet

DIY 2-Probe EC Circuits

The easiest circuits to build yourself are the 555 timer oscillators, but there are plenty of quad-opamp solutions out there for people comfortable with a breadboard. The oldest example I’ve seen is this one by M. Ahmon from the Sept 1977 issue of Electronics magazine which uses the resistance of the solution to modify opamp output:

This circuit uses the first stage of the quad opamp in a Wien-bridge oscillator, reducing errors caused by electrolysis with a 1-kHz signal that gets attenuated by the solution’s resistance before it reaches the driving amplifier A2.  Pot P1 controls oscillator amplitude, and P2 adjusts gain of A2.  A3-A4 form a precision rectifier giving output voltage equal to absolute value of input voltage. This one chip solution seems to have been the basis for many of the current EC projects on the web, including these two exceptionally well documented examples:

Octavia’s EC/TDS/PPM Meter On Limited Budget

Daniel Kramnik’s Digital Salinometer Project

Similar circuits can be found on the breakout modules from Sparky’s Widgets and DFrobot . Using the solution’s resistance in the feedback divider controlling an op-amp is a neat idea, but having only one opamp there imposes hard limits the range you can measure with a given K value probe. There is a more advanced multi-opamp approach over at pulsar.be that can step over several decades.

On more recent EC projects I’m seeing single supply RRIO opamps for the oscillator & gain stages, which are easier to integrate with battery operated Arduino’s. (though any dual supply opamp can be used as a single supply in a pinch; since voltage is relative the opamp doesn’t know whether V- is a negative voltage or ground) To keep using an AC signal, this requires a virtual GND at 1/2 VCC, but the integration also gives you the option of getting rid of the oscillator entirely, since you can use PWM output as your source.

This is beautifully illustrated by the circuit from bhickman’s Conductivity & Temperature Meter over at PublicLab:

Ranging is accomplished with the (red) bank of R1 resistors, and (yellow)R2’s 5/6 can be substituted in for the probe (R8) with those known resistances can be used to track drift. The AC–DC converter stage is built with precision peak detectors. I think this is the best voltage divider approach I’ve seen to date.  To simplify things a bit, you might replace that output stage with an RMS-DC converter; though I’ve not seen any breakouts for those, and I hate working with raw SMD parts.

Sources of Error: 

Even with a clever circuit like the one above you still need to address things like temperature compensation before you get an accurate, repeatable, and stable device. Electrical conductivity measurements are typically referenced to 25 °C using standard temperature compensation factors (α). The conductivity of natural waters exhibit strongly nonlinear temperature behavior, though in practice linear correction factors are most frequently used.  NaCl-based solutions typically have a temperature coefficient (α) of 0.02-0.0214 (~ 2% change/degree C). So to convert your “ambient” conductivity measurement into 25°C “specific” conductivity, the simple linear conversion is:

EC25=ECambient /[ 1 + α (tambient – 25) ], α= 0.02

Field effect errors are significant, causing read errors if bare 2-pole electrode get within 2-5 cm of the solution container: which will completely mess up your calibration and cell constant determination. This is one reason that virtually every EC probe is encased inside a plastic shroud of some sort. That causes field effect errors too, but at least its the same error every the time, rather than one that varies depending on how far you are from the edge of the beaker. Four probe methods also require a fixed volume of solution between the driver electrodes, so the shroud provides that.

Grazing through the hydroponics forums shows plenty of people struggling with cross-sensor interference.  Most notably when conductivity probes affects the accuracy of a PH probe in the same tank.  Any time two devices are immersed in the same environment differences between them can generate ground loop voltages and induce currents which degrade the readings and exacerbate corrosion.  Sometimes you can address these issues with optical or I2C isolators. One helpful contributor at Arduino.cc suggests:

pH electrodes are very high impedance devices and the cabling and connectors are all important – even flexing a decent cable will distort the readings…. Ground loops are the enemy of pH and any other specific ion electrode. I used them a lot in difficult situations and the most trouble-free solution is always to put a buffer op amp (FET type) as close to the electrode as possible – some commercial electrodes come already equipped. Find a decent op amp like the old MAX406, high impedance techniques like PTFE insulators or simply keep the input pin off the board. Modern FET’s take single-sided supplies and run at better than 2-microamps – a 3.6-V lithium cell will give you in excess of 5-year’s trouble and ground loop -free operation. Once you have buffered the signal, you can use any cable you like. As a bonus, you can convert a pH electrode into an ammonia electrode by separating the water from the electrode with PTFE tape as used by plumbers.”

Well, I think that covers most of the stuff I had in my notes, and hopefully gathering it all here saves someone else from burning away that time. I have been experimenting with conductivity quite a bit lately, and I think I might have  come up with an analog approach that will allow people to play with conductivity on shoestring budgets. I just have a little more calibration testing to do before I let that one out of the bag  🙂

Addendum 2018-12-06

Folks working on EC might want to check out our tutorial video showing how to build underwater connectors  ( part of the 2017 logger build series )   Near the end of that video we mount an Atlas EC probe on a long cable for a student project. 

Tutorial: Calibrating Oversampled Thermistors on an Arduino Pro Mini

Selecting a thermistor (& series resistor) value

Most of the material you find on thermistors makes the assumption that you are trying to maximize sensitivity and interchangeability. But oversampling gives you access to enough resolution that sensitivity is less critical, and interchangeability only makes sense if you are putting them in a product with good voltage regulation. In that case, precision thermistors like the ones from US sensor are a good option, but according to Campbell Scientific, that choice has other knock-on implications:

“The resistors must be either bought or selected to 0.02% tolerance and must also have a low temperature coefficient, i.e. 10 ppm or preferably 5 ppm/°C.”

Like many better quality components, these resistors are often only available in SMD format, with minimum order quantities in the thousands. If you use a typical 1% resistor with a T.C. of 50 ppm or more, you could introduce errors of ±0.1°C over a 50°C range, which defeats the point of buying good thermistors in the first place.

Still, if I was only building a few sensors, I’d spring for the good ones. But now that I have oversampling working on the Arduino, I’d like to add a thermistor to every logger in the field, and the mix of different boards already in service means I’ll have to calibrate each sensor/board combination. That time investment is the same whether I choose a 10¢ thermistor or $10 one.

Power consumption is also important, making 100kΩ sensors attractive although I couldn’t even find a vendor selling interchangeable thermistors above 50k.  A low temperature limit of 0°C (the units are underwater…) and putting 1.1v on aref to boost sensitivity,  requires a 688k series resistor, which is far from the 1-3x nominal usually recommended:

Here I’ve overlaid an image from Jason Sachs excellent thermistor article at Embedded Related, which shows I will only see about ⅓ of the sensitivity I would get if I was using a 100k series resistor. I highly recommend reading Jason’s post, despite the fact that I’m ignoring almost all of his good advice here…  🙂

Using the internal band-gap voltage as aref improves the ADC’s hardware resolution from 3.22mV/bit to 1.07mV/bit.  This trick gives you a extra bit of precision when you use it at the default 10bit resolution, and I figured I could do it again to compensate for the lost sensitivity due to that big series resistor.

In return, I get a combined resistance of at least 700k, which pulls only 4.7μA on a 3.3v system.  Such low current means I could ignore voltage drops inside the processor and power the divider with one of Arduino’s digital pins.  In practical terms, burning less than a milliamp-second per day means adding a thermistor won’t hurt the power budget if I leave it connected to the rails all the time; which you can only do when self-heating isn’t a factor.  This is quite handy for the bunch of old loggers already in service out there, that I want to retrofit with decent temperature sensors. 

Even 100 ohms of internal chip resistance would produce only 0.5mV drop,  so depending on your accuracy spec,  you could use 16-channel muxes to read up to 48 thermistors without worrying about cable length.  There aren’t many of us trying to connect that many temperature sensors to one Arduino, but using a 100k  thermistor also makes me wonder if you could mux a bank of different series resistor values, pegging the divider output at it’s maximum sensitivity over a very large temperature range.

What is a reasonable accuracy target?

Combining 5¢ thermistors & 1¢ metfilms, means my pre-calibration accuracy will be worse than ±1°C.  Cheap thermistor vendors only provide nominal & βeta numbers, instead of resistance tables, or a proper set of Steinhart-Hart coefficients. So I might be limited to ±0.4°C based on that factor alone.  And it took me a while to discover this, but βeta values are only valid for a specific temperature range, which most vendors don’t bother to provide either.  Even with quality thermistors, testing over a different temperature range would give you different βeta values.

In that context, I’d be happy to approach ±0.1°C without using an expensive reference thermometer.  Unfortunately, temperature sensors in the hobby market rarely make it to ±0.25°C.  One notable exception is the Silicon Labs Si7051, which delivers 14-bit resolution of 0.01°C at ±0.1°C.   So I bought five, put them through a series of tests,  and was pleasantly surprised to see the group hold within ±0.05°C of each other: 

Temps in °CCompared to what I usually see when I batch test temperature sensors, this is pretty impressive for an I2C chip that only cost $9 on Tindie.

Ideally you want your reference to be an order of magnitude better than your calibration target, but given the other issues baked into my parts, that’d be bringing a gun to a knife-fight. 

So my calculations, with oversampling, and the internal 1.1v as aref become:

1) MaxADCReading                  (w scaling factor to compensate for the two voltages)

= ( [2^(OverSampledADCbitDepth)] * (rail voltage/internal aref) ) -1

2) Thermistor Resistance        (w series resistor on high side & thermistor to GND)

= Series Resistor Value / [(MaxADCReading / OverSampledADCreading)-1]

3) Temp(°C)                                  (ie: the βeta equation laid out in Excel)

=1/([ln(ThermResistance/Tnominal R)/βeta]+ [1.0 / (NomTemp + 273.15)]) -273.15

Seeing the error in my ways

I knew that the dithering noise would have some effect on the readings, and all the other source ADC of error still apply.  Switching to 1.1v reduces the absolute size of most ADC errors, since they are proportional to the full scale voltage. But the internal reference is spec’d at ±0.1v; changing the initial (rail voltage/aref voltage) scale factor by almost 10%.  Since all I needed was the ratio, rather than the actual voltages, I thought I could address this chip-to-chip variability with the code from Retrolefty & Coding Badly at the Arduino.cc forum.  This lets Arduinos read the internal reference voltage using the rail voltage as aref.

I started testing units in the refrigerator to provide a decent range for the calibration:

Si7051 in blue vs 100K thermistor in red. The sensors were held in physical contact. ADC was read with 1024 oversamples providing approximately 15bit resolution. Temps in °C.

and strange artifacts started appearing in the log.  The voltage readings from both the main battery and the RTC backup battery were rising when the units went into the refrigerator, and this didn’t seem to make sense given the effect of temperature on battery chemistry:

Si7051 temp. in °C on the left, with the RTC backup battery (V) in green on the right axis. The CR2023 is monitored through a 2x10MΩ divider, using the 3.3v rail as aref. The large number of ADC readings needed for oversampling has the side benefit that it lets you read very high impedance dividers, but by the time you reach 10Meg ohms, you pick up 5-10 points of noise in the readings. Which is why that coincell voltage line is so thick.

I think what was actually happening was that the output from the regulator on the main board, which provided the  ADC’s reference voltage for the battery readings, was falling  with the temperature.

When I dug into what caused that problem, I discovered that temperature affects bandgap voltages in the opposite direction by as much as 2 mV/°C.  So heating from 0°C to 40°C (and some loggers will see more than that…) reduces the 328P’s internal reference voltage by as much as a tenth of a volt. In fact, bandgap changes like this can be used to measure temperature without other hardware.  This leaves me with a problem so fundamental that even if I calculate S&H constants from a properly constructed resistance table, I’d still be left with substantial accuracy errors over my expected range.  Argh!

Becoming Well Adjusted:  (Beta ain’t better…)

These wandering voltages meant I was going to have to use the internal voltmeter trick every time I wanted to read the thermistor.  It was mildly annoying to think about the extra power that would burn, and majorly annoying to realize that I’d be putting ugly 10bit stair-steps all over my nice smooth 15bit data. This made me look at that final temperature calculation again:

Temp(°C) =
1/([ln(ThermResistance/Tnominal R)/βeta]+ [1.0 / (NomTemp + 273.15)]) -273.15

which I interpret as:

 =fixed math(  [(ADC outputs / Therm. nominialR ) / Therm. βeta]  + (a #) ) – (a #)

Perhaps tweaking the thermistor’s nominal value (which I only know to ±5% anyway) and changing the (fictional) βeta values would compensate for a multitude of sins; including those voltage reference errors?  Then I could just pretend that (rail/aref) scaling factor had a fixed value, and be done with it:         (click image to expand)

So in my early tests, all I had to do was adjust those two constants until the thermistor readings fell right on top of the reference line.  Easy-peasy!

Well …almost. Repeat runs at 15bit (1024 samples) and 14bit (256 samples) didn’t quite yield the same numbers.  Applying the best fit Nominal and βeta values obtained from a 15bit run to 14bit data moved the thermistor line down by 0.05°C across the entire range (and vice versa). So the pin toggling method I used to generate the dither noise introduces a consistent offset in the raw ADC readings.  While that doesn’t completely knock me out of my target accuracy, I should generate new calibration for each oversampled bit depth I intend to use. It’s still good to know that the dithering offset error is consistent.

Throwing a Big Hairy Fit

I was pleased with myself for the simplicity of the Nominal/βeta approach for about two days; then I pushed the calibration range over 40° with a hot water bath:

Blue=Si7051 , Orange = 100k NTC thermistor.  1024 oversamples = ~15bit. Temps in °C.

This gave me targets at around 40, 20 and 5°C.  But no combination of Nominal & βeta would bring all three into my accuracy range at the same time.  Fitting to the 20 & 40 degree data pushed the error at 5°C beyond 0.2° :             (click image to enlarge)

…and fitting to 20 & 5, pushed the 40C readings out of whack.  After more tests I concluded that tweaking βeta equation factors won’t get you much more than 20° of tightly calibrated range. 

My beautiful plan was going pear-shaped, and as I started grasping for straws I remembered a comment at the end of that Embedded Related article

“… in most cases the relationship between voltage divider ratio and temperature is not that nonlinear. Depending on the temperature range you care about, you may be able to get away with a 3rd-order polynomial or even a quadratic..”

Perhaps it was time to throw βeta under the bus, and just black-box the whole system?   

To find out, I needed to prune away the negative temperature regions where the voltage divider had flat-lined, and remove the rapid transitions since the thermistor responds to changes more quickly than the si7051:                 (click image to inflate)

Then it was time for the dreaded Excel trend line:

Ok, ok. I can hear people inhaling through their teeth from here. But with 15 sigfigs, Excel seems like the height of luxury compared to the constraints in μC land.  I wonder what an advanced modeler like Eureqa would have produced with that dataset? 

The trick for getting workable constants is to right-click the default equation that Excel gives you, re-format it to display scientific notation, and then increase the number of displayed digits to at least six.  

Some people use the LINEST function to derive these polynomial constants but I’d advise against it because seeing the raw plot gives you a chance to spot problems before you fit the curve. When I generated the first Temp vs ADC graph, the horizontal spread of the data points showed me where the thermistor and the reference thermometer were out of sync, so I removed that data.  If I had generated the constants with =LINEST(Known Y values, X values^{1,2,3,4})  I could have missed that important step.

For the following graphs, I adjusted the trend line to display to nine insignificant digits:     

Blue =Si7051 reference, Orange is that 20&40 best fit from tweaking Nominal & Beta values, and the yellow line is the 4th order polynomial from Excel.   Temps in °C. (Click to embiggen)

It took a 4th order polynomial to bring the whole set within ±0.1° of the reference line and 5th order did not improve that by much.  Now I really have no idea where the bodies are buried!  And unlike the βeta equation, which just squeaks in under the calculation limits of an Arduino, it’s beyond my programming ability to implement these poly calcs on a 328 with high bit depth numbers. I certainly won’t be writing those lunkers on the bottom of each logger with a sharpie, like I could with a pair of nominal/βeta constants.

This empirical fit approach would to work for any type of sensor I read with ADC oversampling, and it’s so easy to do that I’ll use it as a fall back method whenever I’m calibrating new prototypes. In this case though, a little voice in my head keeps warning me that wrapping polynomial duct tape around my problems, instead of simply using the rail voltage for both aref & the divider, crosses some kind of line in the sand. Tipping points can only be predicted when your math is based on fundamental principles, and black-boxes like this tend to fail dramatically when they hit one.  But darn it, I wanted those extra 1.1v aref bits! Perhaps for something as simple as a thermistor, I’ll be able to convince the scientist in the family to look the other way.

Making the Steinheart-Heart equation work

Seeing that trend-line produce such a good fit to the temperature data, made me think some more about how I was trying to stuff those system-side errors into the βeta equation, which doesn’t have enough terms to cope.  By comparison, the Steinheart-Heart equation is a polonomial already, so perhaps if I could derive some synthetic S&H constants (since my cheap thermistors didn’t come with any…), it would peg that ADC output to the reference line just as well as Excel did?

I rolled the voltage offsets into the thermistor resistance calculation by setting the (rail voltage/internal aref) scale factor to a fixed value of 3, when in reality it varies from slightly below to slightly above that depending on the board I’m using:

1) MaxADCReading                  (w scaling factor to compensate for the two voltages)

=(2^(OverSampledADCbitDepth) * (3)) –1

2) Thermistor Resistance        (w series resistor on high side & thermistor to GND)

= Series Resistor Value / ((MaxADCReading / OverSampledADCreading)-1)

and I went back to that trimmed 40-20-5 calibration data to re-calculate the resistance values. Then to derive the constants, I put three Si7051 temp. & thermistor resistance pairs into the online calculator at SRS:

(Note: There are premade spreadsheets that you can download which will generate S&H constants, or you can build your own in Excel. There’s also coefficient calculators out there in C, Java, etc. if that’s your thing.)

With those Steinhart-Hart model coefficients in hand, the final calculation becomes:

3) Temp °C =1/( A + (B * LN(ThermR)) + (C * (LN(ThermR))^3)) – 273.15

and when I graphed the S&H (in purple) output against the si7051 (blue) and the 4th order poly (yellow), I was looking at these beauties:

and that fits better than the generic poly;  nearly falling within the noise on those reference readings. With the constants being created from so little data, it’s worth trying a few temp/resistance combinations for the best fit. And this calibration is only valid for that one specific board/sensor/oversampling combination;  but since I’ll be soldering the thermistors permanently into place, that’s ok.  I’m sure if I hunt around, I’ll find a code example that manages to do the S&H calculations safely with long integers on the Arduino. 

So even with cheap parts, oversampling offsets & bandgap reference silliness, I still made it below ±0.2°C over the anticipated temperature range.  Now, where did I put that marker…

Addendum 2017-04-27

Just a quick note to mention that you need to tape the thermistor to the si7051 sensor so they are held in physical contact with one another. The thermistors are tiny & react to temperature changes much faster than the si7051’s which have a much larger thermal mass because of the breakout board they are mounted on. So the temp/resistance pairs don’t match up as well as they could if the sensors are in physical contact with one another.

Addendum 2017-06-05

With 1.1v aref in the mix,  my 15bit oversampled resolution on those 100k thermistors varies between 0.002 and 0.004°  from 20-40°C. But I was throwing the bandgap aref in just to see if I could still make it work. From a calibration point of view, it’s better to better to use the rail voltage on aref, and remove that 3x ratio from the MaxADCReading calculation.  This will lower the resolution to somewhere between 0.006 to 0.012C with a 688k series resistor unless you bump up the oversampling to compensate. In addition to tripling my noise/toggle-pin current, how much extra power do I have to pay to get that resolution back if I’m using the 3.3v rail as aref?

In my oversampling experiments, I found that the Arduino ADC works well at 250 kHz, delivering just under 19230 ADC readings /second. For the purpose of estimation, assume the Pro-mini style boards I’m using draw about 5mA during the sampling time, and I take a reading every 15 minutes (= 96 readings per day) :

15bit= 1024 reads/19230 r/sec =0.053s*5mA =0.26 mAs*96/day=~ 25 mAs/day
16bit= 4096 reads/19230 r/sec = 0.213s*5mA =1.00 mAs*96/day= ~102 mAs/day
17bit= 16384 reads/19230 r/sec = 0.852s*5mA =4.26 mAs*96/day= ~408 mAs/day

so it would cost me another 385 mAs/day to reach a resolution slightly better than I was achieving with the 1.1v bandgap on aref. Given that a typical AA battery runs about 2000 mAh = 2000 mAh*3600 sec/hour =~7,000,000 mAs, it would be quite a while before that breaks the power budget.  Removing the ratio dependency also means that your S&H constants are for the resistor/thermistor pair only, making that calibration independent of what system you connect them to.

Using an Rnominial=100k series resistor would give about the same effective resolution boost as going to 17 bit, but that option costs you more power if you are leaving the thermistor powered all the time:

3.3v / 780k combined resistance  = 4.23μA x 86400 sec/day  = 366 mAs/day
3.3v / 200k combined resistance  = 16.5μA x 86400 sec/day  =  1425 mAs/day

You can power the thermistor from a digital pin, but since I’m already using digital-pin toggling to generate noise for the oversampling, I still need to test if I can combine pin power for the sensor with my oversampling technique. It’s possible that the thermistor bridge needs to be powered by the more stable rails, while I’m shaking aref inside the processor, because if the voltage on the divider started moving in sync with the ADC noise, the dithering noise will effectively disappear, and my oversampling would stop working.

Even before doing this test, I have a sneaking suspicion that 100k series vs. oversampling vs. other techniques  will end up converging on the same effective resolution in the end. And I’ll even hazard a guess that the point of diminishing returns is somewhere around 0.001°C, since that’s what you see offered by quite a few high-end temperature loggers.

Addendum 2017-09-24

Just posting an update about pin-powering the thermsitor dividers while using the 3.3v rail as aref: everything works, but as I suspected you need to stabilize the thermistor with a small 0.1uF capacitor or the dither noise vanishes.  This also requires you to take the RC time constant into account, waiting at least 5x T  for that parallel cap to charge before you start reading the divider. You can sleep the processor during this time, since I/O pin states are preserved.

Degree Celsius vs. Time with lines offset for easier visual comparison:  The blue line is over-sampled output from a pro-mini clone reading a 100k Thermistor /100k series voltage divider. Aref was set to the 3.3v rail, with a 100nF capacitor in parallel with the thermistor on the low side.  This RC combination has a time constant of ~10 milliseconds.  A 0.12 mA pin-current provided sufficient noise for dithering 1024 readings: to deliver an effective resolution of ~0.0028° at 24°C.  For comparison, the red line is the output from an I2C si7051 sensor on the same logger, with a resolution of 0.01°C.

So using a 100k series resistor with 3.3v aref really does deliver the same effective resolution as the 680k series/1.1v aref combination, and it does not suffer the problem of bumping into the aref voltage at a certain temp.  I’m using 100k termistors so the pin resistance (~40 ohms) will introduce less than 0.05% error over the range; though this pin-drop error would be higher for therms with lower Rnominal values.

Since I’m using cheap eBay 100k’s and a host of other no-name components, I have to calibrate each logger/thermistor/O.S. bit-depth combination.  This isn’t much of a burden for the overall workflow, since I always give new loggers a shake-down run, in fact, I usually do a fast sampling burn for at least a week before considering a unit ready for deployment:

That Degree vs Time image above was an excerpt from a calibration run like this. I’ve found that Freezer (morning)->Fridge (afternoon)->Room (overnight) is easier to manage than the reverse order, and gives enough time at each temperature to deal with thermal lag differences between the thermistors and the reference sensors.

As before, when I do the thermistor resistance calculation I make the assumption that everything in the system is behaving perfectly (which is obviously not true). So errors from things like pin drops, temp. coefficients, ADC gain, etc., are getting rolled into the S&H constants.  Essentially, I’m eliminating a host of different corrections in exchange for the interchangeability between sensors that I might have if I took all those factors into account individually. This makes it easier to standardize the code , and is a reasonable trade-off for loggers that I won’t be seeing again for several years, but if I have to swap some components at that time, I’ll need to do  another calibration.

The other factor is that every time you introduce one of the many possible corrections, you necessarily limit your final output to the stability, resolution, or number of significant digits in that correction.  In one case the limits of my rail voltage reading method produced random spikes in the record whenever that factor in the calculations had a brief toggle:

Note: spike errors are also diagnostic of calculation errors due to over-running your variables. The difference is that variable overflow problems are not random like the one shown above. They repeat regularly whenever the data passes some threshold in the calculation.

 

In more extreme cases this noise shows up as a overall thickening of the output from correction factors that toggle their relatively low-rez bits more frequently.  As an example I did some runs where I took a Vcc reading with the internal bandgap trick, and rolled that into the thermistor calculation to improve the accuracy. the net result was that the 4-digit Vcc reading placed a limit on the final output so that there was no “effective difference” in the thermal resolution between oversampling at 15bit & 16 bit because that VCC correction had been included.  (Note: You’ll run into this problem more often if you change aref voltages and forget leaving enough time for the aref capacitor to stabilize…)

The Arduino’s reference (and ADC) do not have a zero tempco.  However, if you make the “perfect” regulator/band-gap/ADC assumption the only limits placed on your resolution are the significant figures in your S & H constants.  Even so, there are so many other factors at play here, that I suspect that you can’t use my pin-toggle oversampling technique to push the Arduino’s ADC much past 16 “effective” bits before some other limitation occurs. Then there’s the issue of long term drift of the various components and the fact that it takes over 200ms each 16-bit reading; adding about 20 seconds of CPU run time to my logger’s daily duty cycle.  Remember that my goal here was a dirt cheap temp sensor that I could add to every logger with a modest accuracy in the 0.1-0.2C range.  If you need both resolution and accuracy, then you should switch to ratiometric measurements, with an instrumentation amp like the INA826, and a 24bit ADC.

Addendum 2017-11-05

Looks like Sensirion’s new STS35 has ± 0.1°C accuracy like the si7051 I’m currently using as a calibration reference. Since the Steinhart-Hart” equation has a built-in error of ~0.1°C and the si7051 ref is ~0.1°C, that might get me into the ball park of ±0.25°C accuracy.  Hopefully that shows up on Tindie soon.  Of course, it’s important to remember that we’re miles away from a real ITS-90 level calibration with a triple point cell.

Addendum 2018-03-14

I recently found out about a method using temperature-sensitive liquid crystals as thermal calibration references at 55, 75, and 90 deg°C. These were custom-made by Hallcrest UK (www.lcrhallcrest.com) and apparently the transitions were sharp enough to resolve 10 mK..?  That’s still a bit rich for my blood, but I also thinking about experimenting with virgin coconut oils (on amazon) which melt at ~24 °C  – the actual value is imprecise, but hopefully will remain constant for a given batch of oil.  So could provide a nice melting point plateau…we will have to see…

Addendum 2018-06-10

Still hunting for a good method to provide nice thermal plateaus for the calibration runs covering >30°C of range. The refrigerator gives a nice 5°C point, and of course room temp is easy, but getting that third calibration point up at ~35°C is a bit trickier because I want that peak to be long and slow.  In the winter that’s available on the house radiators, but during the summer I don’t have a ‘slow’ dry heat source in the right range.  I’ve been following some threads suggesting that you can convert a regular water bath into a “dry-bath” with copper coated BB shot, or aluminum pellets. Both would be a heck of a lot cheaper than lab grade dry bath beads, though for an application where i am simply looking for a slow temperature ramp (so hot & cold spots don’t matter) sand or rice might suffice to provide the thermal mass I need. And I could use an old bath from eBay for the job – these sometimes sell for as little as $25 if they have surface rust on them.  Or perhaps I could hack the temp sensor on a charity shop crock-pot to keep the temp really low….

Addendum 2019-03-25:

I’ve been developing a new method for reading thermistors with the Input Capture Unit on pin D8. Micro-controllers count time much more precisely than ADC’s measure voltage, so this new approach delivers more resolution than 16-bit oversampling in about 1/10th the time & power.