Category Archives: Teacher Resources

Pro Mini Classroom Datalogger [2020 update]

2020 update to the Cave Pearl Classroom logger. This is a combination of inexpensive pre-made modules from the open-source Arduino ecosystem, and can usually be assembled by beginners in 1-2 hours.

(Last Updated: Oct 26, 2020)

Covid has thrown a spanner into the works with hands-on learning because even if you have enough space to run a ‘socially distanced’ course, the students could still be sent home at any time.  With that in mind, we’ve divided the tutorial from 2019 into separate stages that make it easier to restructure the labs:

1) Component prep:  requires the equipment you  normally have access to in the lab space like soldering irons, heat guns, drills, etc.

2) Logger assembly: can be done remotely with scissors, wire strippers & a screwdriver. Connections are made by clamping stripped wire ends under screw terminals.

The complete tutorial can be run F2F, or if students are ‘distance learning’, the instructor can do the soldering steps (~15 minutes per set) and send out kits ready for assembly. Even that will be challenging through a zoom window, so you might want to add USB isolators to protect  tethered student laptops from accidental shorts. 

This is variation of the logger described in our 2018 paper but we’ve removed the regulator/ voltage divider and added screw terminals + breadboards for faster prototyping.  Bridging the I2C bus over the A2 & A3 pins leaves only two analog inputs, but A6 & A7 are still available if you add jumpers. However for ~$1 you can add a 16-bit ADS1115 which provides far more analog sensor capability, and uses a fixed internal reference independent of the supply rail.

The main components:

 (NOTE: complete parts list with supplier links can be found at the end of this post) You don’t necessarily need the cable glands if you are using sensors that will work inside the housing (light, temp, acceleration, magnetic, etc.)

Component Prep.  Part 1:   Pro Mini   ( 3.3v 8Mhz )                       (click any image to enlarge)

Solder the UART pins & test ProMini board with the blink sketch:  Set the IDE to (1) TOOLS> Board: Arduino Pro or Pro Mini (2)TOOLS> ATmega328(3.3v, 8mhz) in addition to the (3)TOOLS> COM port to match the # that appears when you plug in the serial adapter board.

Remove Limit Resistor to disable power indicator

With a known good Promini: Remove the power LED [in red square].                               SPI bus clock pulses will flash the pin 13 LED – so  leaving the pin13 LED connected will show you when data is being saved to the SD card.

Remove the voltage regulator with snips. Your system voltage will vary over time, but our starter script records that rail voltage without a voltage divider by comparing it to the 328p’s internal bandgap voltage.

Add header pins to the sides of the Pro Mini, but do not solder the two RESET pins.

Bridge the two I2C bus connections with the wire leg of a resistor. Connect A4->A2 & A5->A3.

A4 & A5 I2C bus bridged to side rails

Adding:   DIDR0 = 0x0F;  in Setup disables digital I/O on pins A0-A3 so they can’t interfere with I2C bus.

Lithium AA batteries are preferred because the slope of an alkaline discharge curve will reach the ProMini’s 2.7v brown-out with >50% of the battery capacity unused. (note that SD cards are safe down to ~1.8v) While the voltage of a ‘brand new’ Lithium AA is usually 1.8v/cell, that upper plateau usually settles at ~1.7 v/cell within an hour or two of starting the logger. That briefly dips below 1.6v/cell during >100mA SD card save events. At temps near 5°C (in my refrigerator) the SD write battery-droop reached about 1.5 v/cell while on the upper plateau. Lithium cells deliver ~50% of their rated capacity at temps below freezing, but that’s still a vast improvement over alkaline batteries.

The MIC5205 regulator is not efficient at low currents, so removing it extends your operating life by 30~50%. But it also forces you to deal with a constantly changing rail voltage. Thermal variation from 15 to 45°C will raise your rail voltage by about 100millivolts (on lithium cells that have been in service for several months)You will need to compensate for this in your calculations if your analog sensor circuits are not ratio-metric. Battery thermal mass can also cause hysteresis unless you read the reference resistor under the same conditions. Regulated promini’s usually see the rail vary by only 10-20 millivolts over a similar range of temperatures however the regs on cheap eBay modules are subject to other problems such as noise which can be even more problematic wrt the quality of your data. 

So making students deal with deal with supply  issues right from the start will save them from making more serious mistakes later on because every component in your logger is a temperature sensor.  It’s also worth noting that most chip based sensor modules carry their own regulators (usually a 662k LDO) and use internal bandgap reference voltages that are independent of the rail/supply.

Component Prep. Part 2:     Screw-Terminal Board & SD adapter

Reset terminals repurposed with jumpers under the board

At the UART end of the board: Use a tinned resistor leg to repurpose the RESET terminals: Join RST & GND pins on the digital side and link the pins labeled RST, 5v and Vin for the positive rail.

Label the repurposed screw terminals with red & black markers.

Gently rock the Pro Mini back to front (holding the two short sides) until the pins are fully inserted. Some ST shields have slightly misaligned headers so this insertion can be tricky.

Remove the last three ‘unused’ female headers to make room for the SD adapter

…which fits perfectly into that pocket

Remove the bottom 3 resistors from the SD adapter – leave the top resistor near the C1 label in place!

Connection map for analog side of Nano S-Terminal board

NOTE: The Screw Terminal board we use in this build was designed for the 5v Arduino NANO, so the shield labels don’t match the actual 3.3v Pro Mini pins on the ‘analog’ side. (the digital side does match) To avoid confusion may want to tape over those incorrect labels and hand write new labels to match the pattern above. Wire connections in this tutorial will usually be specified by ProMini pin labels:  D10-13 are used for the SD card, A4/A2 is the I2C Data line, and A5/A3 is the I2C clock line.

Technically speaking, bridging the I2C bus (A4=data & A5=clock) over top of A2 & A3 subjects those lines to more capacitance and pin leakage. (regardless of whether that channel is selected as input for the ADC p257).  However in practice, the 4K7 pull-up resistors on the RTC module handle that well at the 100 kHz default bus speed.

Adding DIDR0 = 0x0F; in setup disables digital I/O on pins A0-A3 to prevent interference with the I2C bus. If you want to disable only the digital IO on A2 & A3 add bitSet (DIDR0, ADC2D); bitSet (DIDR0, ADC3D); to your code.

Also note: On the UART adapter shown in the first image, the pins are in the reverse order to those on the Pro Mini board. This is a fairly common issue and if your sketches won’t upload flip the adapter around and try again. I have connected these oggers to UART modules the wrong way round many, many times, and not one of them has been harmed by the temporary reversal.

Component Prep.  Part 3:     RTC Module,  RGB  LED  &  Plano 3440 Housing

Remove two SMD resistors from the RTC board with the tip of your soldering iron. Note that this module includes 4k7 pullup resistors on SQW, SCL & SDA

Cut the Vcc leg on DS3231 chip (this optional step disables 32kHz output)

Add 90 degree header pins to the I2C cascade port

Clean away soldering flux residue from both the main & cascade header pins

Use a coin cell to determine the (negative) GND leg of a diffused common cathode RGB LED

Solder ~1k ohm limit resistor on the common GND. The precise value is not critical.

Add heat shrink, bend & trim the pins for connection to the screw terminal board.

Holes in the rear struts of the Plano 3440 Stowaway housing provide secure tie-down points.

Stepped drill bits make nice clean holes in the plastic  housing and accommodate thread diameter variations.

Only add cable glands if they match your planned lab exercises. We use them with waterproof DS18b20 sensors on 1m cables.

Add rubber washers to both the inside and the outside surfaces. PG7 size washers fit M12 cable glands.

A 3440 Plano Box configured for two external sensors with nylon M12 cable glands for 3-6mm cables. After threading the sensor cables I often seal the outside of the glands with a layer of silicone goop.

As with the ProMini, it’s worth testing the RTC modules  before you continue with the assembly. I usually keep a breadboard version of the logger handy for this step so I can test several modules at the same time time. You could even set the clocks at this point if the coin-cell is already in place.

Forcing the RTC to run from the backup coin cell reduces overall current by 0.09mA, bringing a “no-reg & noRTCvcc” build below 0.15 mA when the logger sleeps between readings. (with well behaved SD cards)  As a rough estimate, Lithium AA’s provide ~7 million milliamp-seconds of power, and your logger will burn about 12,960 mAs/day at 0.15mA. So you should approach a year before you fall off the ‘upper plateau’.  The clock will reset to Jan 1st, 2000 if you disconnect the RTC battery contact  with a hard bump. (note: the time stamps will be wrong after that kind of reset, the logger will continue running once the clock rolls around to the previous hour/minute alarm)   A CR2032 can power the RTC about four years but you have to set bit six of the DS3231_CONTROL_REG to 1 to enable alarms when running from the coin-cell. (and our starter code does this by default)

All of the soldering steps are now completed, and the components are ready for assembly.

Cutting the Vcc leg is optional: if you leave the RTC power leg attached you’ll see typical sleep sleep currents in the 0.25 mA range, which usually provides 4-6 months of operation before you trigger the low voltage shutdown. I’m being conservative here because run time also depends on sensors and other additions you make to the base configuration.

Visit our RTC page for more detailed information on this DS3231 module.


Assembly Part 1:  The Screw-Terminal Stack

Add 2-3 layers of double sided foam tape to the sides & center of the screw terminal shield. The tape needs to extend beyond the height of the solder points.

Add single wires from the 20cm Dupont cable.     Black =GND,   Purple=MISO,   Brown=CLocK,   Orange=MOSI,   Grey=CableSelect,         Red=3v3

A layer of double sided foam tape secures the individual  connectors & should extend over the solder pins to prevent accidental contact

Tape the SD adapter into place on the areas that’s been cleared at the back of  the screw terminal board

Bend the jumper wires into place, mark the length & trim the wires.

Score the wire insulation about 1cm back from the wire end but do not remove the insulation.

Gently ‘pull & twist’ the  scored insulation away from the wire to wind the thin strands together.

‘Fold-back’ about half of the stripped wire to provide more copper surface for the screw terminals to bite on to.

 

 

SD Connection pattern:    Grey (CS) to ProMini D10Orange (MOSI) -> pm D11,      Purple (MISO) ->pm D12,        Brown (CLK) -> pm D13  (NOTE the shield labels say A0-A3 which do not match the D10-13 pattern of the Pro Mini pins)

Bring the red SD wire over to the re-purposed RST power connection

and the black wire to GND on the digital side. At this point you can test the  connections by inserting an SD card & running the CardInfo utility.

 

 

Connect the legs of the indicator LED at: D3=red, D4=GND, D5=green, D6=blue

Strip & connect the male ends of the wires you trimmed from the SD jumper wires. Brown to A1, Grey to A0, Orange to D7, Purple to D8.

Add a layer of double sided tape to the back of the 2xAA battery holder

Attach battery holder wires to RED>Vin & black>GND

Connect Dupont jumper wires so that the metal & plastic retainer clips are facing upwards after the logger is assembled. That way you can diagnose connection issues with the tip of a meter probe and, if necessary, pull out & replace a single bad wire without taking everything apart. We’ve used the hardware interrupt port at D3 for the red LED channel, but if you have sensors that need that just shift the LED over by one. Any I/O pins can be used for the LED, but 3,5 & 6 have PWM so you can do some color tricks with the LED using AnalogWrite.

Assembly Part 2: RTC Module Jumpers

Attach Dupont jumper wires to the RTC module using White=I2C Data, Yellow = I2C Clock. Blue is the SQR alarm output line. Nothing is attached to the 32k output pin.

Use 20cm M-F jumpers on the input side and shorter 10cm M-F wires on the smaller cascade port.

 

Add a layer of double sided tape to secure the jumper shrouds, and provide housing attachment points for the module.

Add as small piece of soft heat shrink tubing to reinforce the modules contact spring. This spring connection can be bumped loose if the logger is dropped.

Write the installation date on the coin cell with a marker. A new coin cell should power the RTC for several years.          To avoid accidentally disconnecting the coin cell battery after the logger is assembled ->

Always check the backup voltage by placing the positive probe tip between the spring plates rather than on the surface of the cell.

Assembly Part 3:    Connect modules inside the housing:

Attach the screw terminal stack onto the upper left corner of the housing, and the 2xAA battery holder in the lower right corner.

Add the first mini breadboard so that it’s rear right edge aligns with the second rear support strut on the housing.

Connect the stack jumpers to that breadboard to keep them out of the way & tape the RTC module to the lower left side, just below the screw terminal stack.

Measure, mark and trim:  Red=3.3v, Yellow=A3/A5 I2C clock, & White=A4/A2 I2C data with enough wire to twist & fold the stripped ends under the terminals

RTC module power & I2C bus connections:  All ports on analog side of the Pro Mini / Shield combination  now have some kind of wire connection.

Connect the black wire to the re-purposed RST=GND, and the blue alarm wire to the D2 with some extra wire length in a loop for strain relief.

Use the other side of the trimmed blue jumper wire  to extend the D9 connection over to the breadboard.

Attach a 3/4 inch cable mount to the back of the housing, low enough that it does not interfere with closing the housing lid.

Loosely tie the long dupont wires to the rear mount. Add another cable mount near the center of the housing and attach the 2nd mini breadboard.


Your Logger is now ready for testing!

A typical sensor configuration with: BMP280 pressure, BH1750 lux & 0.96″ I2C OLED display – connected by short jumper wires made with a crimping tool. The combination shown above averages ~10mA with screen & cpu running, and a sleep current of 0.147 mA with a 1Gb Sandisk SD card. Without the SD, the sleep current on this unit was 37µA; with the sensor modules needing 2-3µA each & the sleeping 0.96″ OLED drawing ~7µA.  A 25µA sleep current from the ProMini clone hints that the MCU might be fake but with a AA power supply it doesn’t really matter.

(Note: Most of the time the tests listed below go well, however if you run into trouble at any point read through the steps suggested for Diagnosing Connection Problems at the end of this page.)

Install the Arduino IDE into whatever default directory it wants – we’ve had several issues where students tried to install the IDE into some other custom sub-directory, and then code wouldn’t verify without errors because the IDE could not find the libraries. The programming environment is written in Java, and the IDE installer comes with its own bundled Java runtime so there should be no need for an extra Java installation. However we have seen machines in the past which would not compile known-good code until Java was updated on those machines; but this problem is rare.

If you have not already done so, there are three things you need to set under the IDE>TOOLS menu to enable communication with the logger:

Note: that the “COM’ setting will be different for each computer, so you will have to look for the one that appears on your system AFTER you plug in the UART module.

This is the connection pattern inside the mini breadboard. Don’t accidentally connect jumper wires by plugging them into the same row. NEVER CONNECT the black GND & red 3.3v wires together or the short circuit may kill your logger, and possibly the USB port they are connected to . . .

The one that’s easy to forget is choosing the 328P 3.3v 8Mhz clock speed. If you leave the 328p 5v 16mhz (default), the programs will upload OK, but any text displayed on the serial monitor will be random garbled characters because of the clock speed mismatch.  Also be sure to disconnect battery power (by removing one of the AA batteries) whenever you connect your logger to a computer.  There is no power switch on the loggers, which are turned on or off via the battery insertion. Use a screwdriver when removing the batteries so that you don’t accidentally cause a series of disconnect-reconnect voltage spikes which might hurt the SD card.

1. Test the LED – the default blink sketch uses the pin13 LED, but because that pin is shared with the SD card’s clock line it’s recommended that you test the RGB indicator instead by adding commands in setup which set the digital pin 4 act as the ground line for the LED:
     pinMode(4, OUTPUT);   digitalWrite(4, LOW); 
You will also need to change LED_BUILTIN variable in the blink code example to one of the pins connected to your led module – either Red=3, Green=5, or Blue=6.

2. Scan the I2C bus with the scanner from the Arduino playgound. The RTC module has a 4K eeprom at address 0x56 (or 57) and the DS3231 RTC chip should show up at address 0x68.

The address of the eeprom can be changed via solder pads on the board, so it may have a different address. If you don’t see at least these two devices listed in the serial monitor when you run the scan, there is something wrong with your RTC module or the way it’s connected: It’s very common for a beginner to get some of the wire connections switched around during assembly but with the screw terminals this takes only a few moments to fix.

3. Set the RTC time, and check that the time was set – The easiest method would be to use the SetTime / Gettime scripts from our Github repository, but you first you need to download & install this RTC control library  The SetTime script automatically updates the RTC to the moment the code was compiled (just before uploading) so only run SetTime once, and then upload the GetTime sketch to remove SetTime from memory. Otherwise SetTime will keep setting the RTC to the old ‘code compile time’ every time it runs – and one of the quirks of the Arduino environment is that it restarts the processor EVERY TIME you open a serial window.

Note:  the RTClib by Mr. Alvin that we use has the same name as the Adafruit library for this RTC and this will give you compiler errors with our logger script if somehow the Adafruit library is already installed on your computer. You may have to uninstall the Adafruit library ‘manually’ before installing Alvin’s RTClib. This problem of ‘two different libraries with the same name’ was common back when this project started many years ago, but is rare today.

Typical Cardinfo output when the connections are correct                                  (click to enlarge)

4. Check the SD card with Cardinfo
Note that the SDfat library we use to communicate with SD cards prefers them to be formatted as fat16, so 4GB or larger cards may not work because they often get formatted as fat32. With a 15 minute sampling interval, most loggers generate ~ 5Mb of CSV format text files per year. Older, smaller SD cards in the 256-512Mb range often use less power. Note that we apply internal pullup resistors on some of the SD card lines in setup to help the SD cards go into low power sleep modes more reliably.

5. Calibrate your internal voltage reference with CalVref from OpenEnergyMonitor.

This logger uses an advanced code trick to read the positive rail voltage to ~11mv resolution by comparing it to an internal 1.1v bandgap reference inside the processor. That internal ref. can vary by ±10% from one chip to another, and CalVref gives you a numerical constant which usually brings the starter script’s rail=battery readings within ±20mv of actual. An accurate rail reading is more important when you are using ANALOG sensors where the positive rail directly affects the ADC output, but you can skip this procedure if you are only using digital sensors because they use their own internal reference voltages.

Typical CalVref output          (click to enlarge)

Load CalVref while the logger is running from USB power and then measure the voltage between GND and the positive rail with a  voltmeter. (this voltage will vary depending on your computer’s USB output, and the UART adapter you are using) Then type that voltage into the entry line at the top of the serial monitor window & press the enter key. Write down the reference voltage & constant which is then output to the serial monitor window. I write these ‘chip-specific’ numbers inside the logger with a black marker as they are related only to the 328p processor on the ProMini board used to make that particular logger. You then need to change the line #define InternalReferenceConstant 1126400L in our starter code to match the long number returned by CalVref. Alternatively you could just tweak the value of the reference constant ‘by hand’, increasing or decreasing the value till the reported rail readings match what you measure with a voltmeter. After you’ve done this once or twice you can usually reach the correct value with about 10 successive guesses.

6. Find a script to run your on logger. For test runs on a USB tether, the simplest bare-bones logger code is probably Tom Igoe’s 1-pager at the Arduino playground. It’s not really deploy-able because it never sleeps the processor, but it is still a useful ‘1-pager’ for teaching exercises and testing sensor libraries.  In 2016 we posted an extended version of Tom’s code for UNO based loggers that included sleeping the logger with RTC wakeup alarms. Our current logging “Starter Script” has grown since then to ~750 lines, but it should still be understandable once you have a few basic Arduino programming concepts under your belt.


Using the logger for experiments:

One positive aspect of the relatively loose fit of the Plano box lid is that it lets you run sensor tests quickly if you jumper your sensor module with thin 28-30 gauge wires:

A BMP280 pressure sensing module on long wires with crimped male dupont ends in the breadboard.

~1″ square of foam mounting tape with wires spaced evenly

 

Leave the red backing facing up as you fold the tape & wires over the corner edge.

The front corners of the box exert less pressure than the back corner shown here.

The sharp inner edge of the lid would cut the wire insulation if the tape was not there to protect, and even then you can only use this trick a few times.

The tape over the wires has to be replaced every time.

This gives you a chance to do some test runs before you commit to modifying the housing with holes or cable glands. For some indoor experiments this might be all you actually need, though I would still coat the ‘non-sensing’ parts of that dangling breakout with either conformal coating or clear nail polish.

After 1-2 minutes of kneading to mix the epoxy you have ~ 1 minute to work the putty into place. (it will become rock-hard within ~10 minutes). Be sure to leave yourself enough extra wire/space inside the housing so that you can open and close the lid easily without disconnecting anything after the putty hardens. This seal is not strong enough for underwater deployments, but it should easily withstand exposure to rain-storm events.

For a classroom project you could simply drill small a hole through the lid and stick the sensor/module on top of the housing, sealing the hole with double-sided tape. Thicker pass-through cables can be also be sealed reasonably well with plumbers epoxy putty which is non-conductive, and adheres quite well to  metal, glass & plastic surfaces->  This putty is also a quick way to make custom mounting brackets, or even threaded fittings if you wrap it around a bolt (which you carefully remove before the putty hardens completely) 

Breadboard connections are very easy to bump loose, so once you have your prototype working, it’s usually best to re-connect the sensors directly to the screw terminals before deploying a logger where it could get knocked around. In a pinch you can secure breadboard pins with a small drop of hot glue to keep them from wiggling.  Also remember that there are six ‘unused’ screw terminals on the back of the shield and these can be use to connect wires together securely.

2019 Logger mounted on a south-facing window. The top surface was covered with  white label-maker tape to act as a diffuser. 

[Click HERE] to read about the many types of sensors can be added to this logger The transparent enclosure makes it easy to do light-based experiments. Grounding the indicator LED through a digital pin allows it to be used as both a status indicator, and as a light sensorThe code we are using for this is from the Arduino playground. This polarity reversal technique relies on the very tiny parasitic capacitance inside the LED. (~50-300pF) This technique works better when the LED is connected directly to input pins because breadboards can add random capacitance at the same scale. Another thing to watch out for is moisture condensation inside your logger housing: this provides an alternate discharge path for the reverse-charge on the LED, which effectively shorts out the light level reading.

We have integrated this technique into the starter script on GitHub. I’ve tweaked the playground version with port commands so the loop execution takes about 100 clock cycles instead of the default of about 400 clock cycles.  The faster version was used to generate the following light exposure graph with a generic 5mm RGB LED, with a 4k7Ω limiter on the common ground.

Red, Green & Blue channel readings over the course of one day from the indicator LED in the logger photo above.  The yellow line is from an LDR sensor the same unit, that was over-sampled to 16-bit resolution. The LED sensor has a logarithmic response and the left axis on the graph is a time- based measurement where more light hitting the LED sensor results in a lower number. Note how the RED signal changes before/after Blue & Green at sunrise & sunset.  LED’s work well with natural full-spectrum light, but their limited frequency bands can give you trouble with the odd spectral distribution of indoor light sources. The peak response of LED’s is usually 30–50nm lower than their peak emission wavelength So the blue channel is actually recording in the near-UV range, the Green channel is responding at ~ 420nm (blue) and the red channel is actually responding to a wide band of yellow-green light. 

You can read more about LED based sensing techniques in the post about our leaf testing experiments which used two LEDs for a transmission-based variant of the NDVI ratio.

While the LED sensor idea is fun to work with, it’s a relatively slow method that can keep the logger running for many seconds when light levels are low. Figuring out how to take those light readings only during the day is a good coding exercise for students.

Note: VERY FEW light sensors can withstand exposure to direct sunlight. PTFE is an excellent light diffusing material which available in different sheet thicknesses.  The ‘divot’ on the lid of the Plano box is just a bit larger than 55mm x 130mm x 3mm (depth). The “teflon” tape that plumbers use to seal threaded joints can also be used. PTFE introduces fewer absorbance artifacts than other DIY diffusers like ping-pong balls, or hot melt glue. Most light sensors like the TSL2561 need 3-5mm of that PTFE sheeting to prevent the sensors from saturating in full sun. LED’s don’t saturate that badly, but you still lose all the useful detail in your data at peak brightness > 80,000 lux unless you add a diffusion layer to attenuate.

Full sun exposure can also cook your logger. Internal temps above 80°C may cause batteries to leak or damage the SD card.  So if you are leaving the logger in full sun, add a bit of reflective film or some aluminum foil around the outside to protect the electronics. Of course if you have a light sensor you’ll need to leave an un-covered ‘window area’ for it to take a reading. 

The RTC has a built-in temperature register which automatically gets saved with our starter script however that record only resolves 0.25°C, so we’ve also added support for the DS18b20 temperature sensor to the base code. A genuine DS18b20 (yes, fake sensors are a thing) draws very little power between readings and you can add many DS18b’s to the same logger.


Addendum: Diagnosing Connection Problems

If you successfully loaded the blink sketch to test the ProMini during your initial assembly, then issues during the testing stage are often due to incomplete connections to the I/O pins.

If you see only “Scanning I2C….. ” but nothing else appears when running the bus scanner, then it means that the ProMini can not establish communication with the RTC module. One common cause of this problem is that the white & yellow wires have been switched around at one end or the other. It’s also easy to not quite remove enough insulation from the wires to provide a good electrical connection under the screw terminals, so undo those connection and check that the wires were stripped, cleaned & wrapped together before being put under the terminals.

Scanner lockup can also happen if one of the I2C devices on the bus is simply not working: usually about 1 in 6 logger builds ends up with some bad component that you have to identify by process of elimination. (These are 99¢ parts from eBay…right?) It only takes a moment to swap in a new RTC board via the black Dupont connector and re-run the scan. If the replacement RTC also does not show up with the I2C scanner then it’s likely that one of the four bus lines does not provide a complete connection between the ProMini & the RTC module.

On this unit I measured 1 ohm of resistance on the I2C clock line between the ProMini A5 pin (on top of the board) and the SCL header pin on the RTC module. So this electrical connection path is good. It’s not unusual for each ‘dry’ connection to add 0.5-1 ohm of resistance to a signal path.

To diagnose: Unplug any power sources to the logger. Set a multi-meter to measure resistance and put one probe lead on the topmost point of the promini header pins, and the other probe on the corresponding header pin of the RTC module. If there is a continuous electrical connection between the two points then the meter should read one ohm or less. Higher resistances mean that you don’t have a good electrical path between those points even if they look connected:

1) the ground (black) wire should provide a continuous path from the ground pin on the digital side of the Promini board to the GND pin on the RTC module
2) the positive power (red) wire should provide a continuous path from the Promini positive rail pin (the one with the bundle of 4 red wires) to the VCC pin on the RTC
3) A4 (I2C data) near the 328P chip on the Promini must connect all the way through the screw terminal board and through the white Dupont wires to the SDA post on the RTC
4) A5 (I2C clock) nearest the UART end on the Promini must connect through through the yellow Dupont wire to the SCL header on the RTC .

You occasionally get a bad Dupont wire where the silver metal end is not in contact with the  copper wire inside because the crimp ‘wings’ did not fold properly. With a pair of tweezers, you can ‘gently’ lift the little plastic tab on the black shrouds holding the female Dupont ends in place, and then replace any single bad wire. Be careful not to break the little black tab or you will have to replace the entire shroud.

Also look at the little jumpers used to bridge the A4>A2 and A5>A3. If you have a ‘cold’ solder join, or an accidental bridge connection to something else, it could stop the bus from working. Remelt each connection point one at a time, holding the iron long enough to make sure the solder melts into a nice ‘liquid flow’ shape for each solder point.

The connection diagnosis procedures described above also apply to the connections for the SD adapter board. Sometimes you end up with an adapter that has a defective spring contact inside the SD module, but the only way to figure that out is to swap it with another one.

Here a jumper wire from the Promini pins is by-passing a bad connection.  This is also how you would break out A6 & A7 connections if you need them.

Sometimes those screw terminal boards have a poor connection inside the black female headers below the Promini. It’s also possible to accidentally over-tighten a terminal and ‘crack’ the solder connection below the board – or there may simply be a cold solder joint on one of the terminal posts. If you have only one bad connection, you can jumper from the Promini header pins on top, down to the other wires under the corresponding screw terminal. If you accidentally strip the threads on a screw terminal, you can use this same approach but move that set of wires over to one of the three ‘unused’ screw terminals at the far end of the board. (beside the SD card adapter) If you’ve gotten through all of the above steps and still have not fixed the problem, then it might be time to simply rebuild the logger with a different screw terminal adapter board.

 

If you do accidentally kill the ProMini by shorting a pin, etc, you can carefully lever it up away from the screw terminal shield and replace it without having to rebuild the whole logger.

I recommend that you build two loggers at a time, because that lets you determine whether problems are code related (which will affect both machines the same way) or hardware related. (which will only affect one of your two units) At any given time I usually have 2-3 units running overnight tests so that I can compare the effect of two different code/hardware changes the next morning.  As a general rule you want to run a new build for at least a week before deploying to get beyond any ‘infant mortality’, and reach the good part of the bathtub curve.

 


An I2C OLED is quite readable through the lid of the housing. I use Griemans text-only SSD1306Ascii library because it has a low memory footprint and sleeps well. While few loggers need output screens when they are actually deployed, it’s nice to have them when you are testing a new sensor configuration because they give you a way to view diagnostic messages when the logger is running on battery power.

Addendum:  A note about I2C sensors
The I²C bus is slow, so topology (star, daisy-chain, etc.) doesn’t matter much, but capacitance does. Length and number of devices increase capacitance. If you find that the devices work when you switch to a slower speed (e.g. 50 kHz), then this is probably your issue, and you need to minimise bus length and/or maybe decrease the combined resistance of the pull-ups to 2 kΩ or less. The DS3231 RTC module has 4k7 ohm pull-up resistors on the SDA & SCL lines & the Pro Mini adds internal 50k pull ups when the wire library is enabled. Typical I2C sensor modules usually add another set of 10k pullups so your ‘net pullup resistance’ on the I2C bus wires is usually:  50k // 4k7 // 10k = ~3k. With a 3.3v rail that means the devices draw 3.3v / 3k = 1 mA during communication which is fairly normal ( 3mA is max allowed) for total wire lengths below 1m. It’s common for pre-packaged sensors to arrive with housings at the end of about 1m of wire. If each sensor also adds another set of 10k pullups, the resistance generally compensates for the extra wire length, so the combination still works OK. But that depends on the cable too. A very bad cable might not even get to 0.5 meters and a very good cable (little capacitance to ground, no crosstalk between the wires) can go up to 6 meters.

For most sensor types there will be some options that draw much less power than others, and it’s always worth a look at the data sheet to make sure you are using one that will run longer.  The best chip based sensors automatically go into low current modes whenever the bus has been inactive, but more often you need to ‘manually’ put the sensors to sleep via specific commands. So it’s also important to check if your sensor library supports those ‘goto sleep’ & ‘wake up’ commands –  many common Arduino libraries do not.


Addendum:  The importance of moisture protection

I was noodling around in the garden recently and installed a few loggers without desiccants because it was only a short experiment. It rained immediately afterward and I noticed a small amount of moisture condensed inside the plano-box housing. While this didn’t prevent the logger from functioning, it completely disrupted the LED light sensors because it provided an alternate discharge path for the reverse bias charge:

Green channel data from a 5mm diffused RGB LED used as light level sensor. This logger was under some leaf cover, so there was considerable variability from the dappled light crossing over the sensor. An arbitrary cutoff of 200,000 was set in the code at low light levels.

After examining the O-ring I decided to add a little silicone to the channel holding the o-ring to improve the seal:

Gently pry the O-ring loose and apply sealant in the groove before replacing.

Bead only needs to be 3-4mm in diameter.

Close the housing & let the sealant set for a few days. The improved seal is especially visible at the corners

If you already have your logger assembled, try to find a silicone sealant that does not off-gas acetic acid (smells like vinegar) which could harm your circuits. If you are simply preparing empty boxes before assembly, then any regular bathroom sealant will do provided you give it about a week to finish curing.

Attach a mounting base to the lid so that a dessicant pack can be secured above the battery holder without interfering with any breadboard jumpers. Use a desiccant pack with color indicator beads, so you can check whether they are still working simply by looking through the transparent lid.

 


Addendum:  If you want to leave the original regulator in place

There are only a few of modifications to the tutorial above if you wish to leave the ProMini’s default MIC5205 regulator in place on your logger:

Use straight header pins on the RTC modules cascade port

Only bridge the unused RST terminals to the rail connections – Leave the Vin terminal separate for the raw battery input.

 

Add a 10/3.3 Meg voltage divider to read the raw battery voltage on A0

…and of course you will need more space for the extra batteries.

An alternative would be to add a better regulator to an intermediate battery connector. The the photo on the right shows two ceramic 105’s stabilizing an MCP1702-3302E/TO, while the 10/3.3M ohm divider provides a third output so an ADC channel can monitor the raw battery voltage. This is the simplest way to retro-fit a unit that was built without a reg, with the added benefit that the new regulator is far more efficient than the original MIC5205 on the ProMini.


Addendum:  Things to keep in mind when ordering parts

When a finished module arrives at your doorstep for less than you’d pay for any of its sub-components individually it’s because  you are doing the quality control.

My advice is to order at least 5-6 of each of the core components (Promini, RTC, SD module, screw terminal board, etc) with the expectation that about 10% of any cheap eBay modules will be DOA or have some other problem. I build loggers in batches of six, and one unit typically ends up with a bad part somewhere. Having replacement bits on hand is your #1 way to diagnose and fix these issues. I’ve noted over the years that bad parts tend to come “in bunches”, so if you scale up to ordering in quantities of 10’s & 20’s then spread those orders to a few different suppliers so you don’t end up with all your parts coming from the same flakey production run. 

 

The other thing I can’t stress enough is CLEAN ALL THE PARTS as soon as they arrive. Leftover flux is very hygroscopic, and pretty much guarantees that solder points will start to corrode the moment your logger gets exposed to atmospheric moisture. I usually give everything about 10 minutes in a cheap sonic bath with 90% isopropyl alcohol, rinse with water, and then dry the parts out in front of a strong fan for an hour. Clean parts that can’t take the sonic vibration (RTC modules, humidity sensors, accelerometers, etc) by hand with a cotton swab. Then store parts in a sealed container with a dessicant pack till you need them.  I also coat the non-sensing/non-contact surfaces with a layer of MG Chemicals 422B Silicone Conformal Coating and let that dry for a day before assembling the loggers.  One hint that you may have moisture issues is that the sensors seem to run fine in the house but start to act strangely when you deploy the unit outside.

Used nut containers make excellent “dry storage” once the parts have been cleaned – but any air-tight container will do.

Another insight I can offer is that the quality of a sensor component is often related to the current it draws – if your ‘cheap module’ is pulling significantly more power than the data sheet indicates, then theres a good chance it’s a junk part. Usually if the sleep current is on spec, then the sensor is probably going to work. It is much easier to check low currents with a µCurrent or a Current Ranger. (I prefer the CR for it’s auto-ranging feature) Sensors which automatically go into low current sleep modes take time to do – so you might need to watch the loggers sleep current for several seconds  before they enter their quiescent state. With SD cards, if the freshly formatted throughput drops too far below rated write speeds when tested with H2testw, then find another card to use with your logger. I avoid cards bigger than 2Gb because they draw too much current, and it’s rare to need that much space on a datalogger.

TransparentSinglePixl
Bill of Materials: ~$22.00
Plano 3440-10 Waterproof Stowaway Box
Sometimes cheaper at Amazon. $4.96 at Walmart and there are a selection of larger size boxes in the series. 6″ Husky storage bins are an alternate option.
$5.00
Pro Mini Style clone 3.3v 8mHz
Get the ones with A6 & A7 broken out at the back edge of the board. Just make sure its the 3.3v version because you can’t direct-connect the SD cards to a 5v board.
$2.20
‘Pre-assembled’ Nano V1.O Screw Terminal Expansion Board
by Deek Robot, Keyes, & Gravitech (CHECK: some of them have the GND terminals interconnected)  You will also need to have a few small flat head screw drivers to tighten those terminals down.  Since this shield is was originally designed for an Arduino Nano many of the labels on ST board will not agree with the pins on the ‘analog side’ of the ProMini.
$1.85
DS3231 IIC RTC with 4K AT24C32 EEprom (zs-042)
Some ship with CR2032 batteries already installed.  These will pop if you don’t disable the charging circuit!  
$1.25
CR2032 lithium battery  $0.40
SPI Mini SD card Module for Arduino AVR
Buy the ones with four ‘separate’ pull-up resistors so that you can remove three of them.
$0.50
Sandisk or Nokia Micro SD card 256mb-512mb 
 Test used cards from eBay before putting them in service. Older Nokia 256 & 512mb cards have lower write currents in the 50-75mA range. This is less than half the current draw on most cards 1gb or larger.
$2.00
Small White 170 Tie-Points Prototype Breadboard
These mini breadboards for inside the logger are also available in other colors.
$0.60
20cm Dupont 2.54mm M2F 40wire ribbon cable
Dupont connector hook-up wires might be expected to add an ohm or two of resistance and carry at most 100mA reliably with their thin 28-30 gauge wires.  Each 40-wire cable will let you make at least 2 loggers.
$1.55
10cm Dupont 2.54mm M2F ribbon cable
Sometimes these 10cm cables are harder to find, so you can just use the longer 20cm wires in a pinch. 
$1.00
2×1.5V AA Battery Batteries Holder w Wire Leads
If you are running an unregulated system on 2 lithium batteries, then you can use a 2x AA battery holder. If you need to keep the regulator in place to stabilize the rail voltage for particularly picky sensors, use alkaline batteries and a 4xAA battery holder.
$0.50
5mm Common cathode RGB LED  $0.10
M12 Nylon Cable Glands (pack of 20 pcs) 
You will also need some extra rubber washers.
$0.70 /2pcs
3.3V 5V FT232 Module
  *set the UART module to 3.3v before using it!* and you will also need a few USB 2.0 A Male to Mini B cables. You may need to install drivers from the FTDI website depending on your OS. Get at least 2-3 of these as they do wear out and you can kill them instantly with a short circuit. These boards can only supply ~50mA which can be tricky if your sensors need more for sustained periods.
$2.75
3M Double-side Foam Tape, 22awg silicone wireheader pins, 3/4 inch zip Tie Mounts, etc… $1.00
Some extra tools you may need to get started:                (not included in the total above)
2in1 862D+ Soldering Iron & Hot Air station Combination
a combination unit which you can sometimes find as low as $40 on eBay.
Or you can get the Yihua 936 soldering iron alone for about $25. While the Yihua is a so-so iron, replacement handles and soldering tips cost very little, and that’s very important in a classroom situation where you can count on replacing at least 1-2 tips per student, per course, because they let them run dry till they oxidize and won’t hold solder any more.  Smaller hand-held heat-shrink guns are available for ~$15 and $5 USB soldering irons are surprisingly useable.
$50.00
SYB-46 270 breadboards (used ONLY for soldering header pins )
Soldering the header pins on the pro-mini is MUCH easier if you use a scrap breadboard to hold everything in place while you work. I use white plastic breadboards that only have one power rail on the side since I won’t mistake them for my regular breadboards. I also write ‘for soldering only’ on them with a black marker.
$1.30
SN-01BM Crimp Plier Tool 2.0mm 2.54mm 28-20 AWG Crimper Dupont JST
I use my crimping pliers almost as often as my soldering iron –  usually to add male pins to component lead wires for connection on a breadboard. But making good crimp ends takes some practice.  But once you get the hang of it,  Jumper wires that you make yourself are always better quality than the cheap premade ones.
$16.00
Micro SD TF Flash Memory Card Reader
Get several, as these things are lost easily. My preferred model at the moment is the SanDisk MobileMate SD+ SDDR-103 or 104 which can usually be found on the ‘bay for ~$6.
$1.00
Side Shear Flush Wire Cutters & Precision Wire Stripper AWG 30-20
HAKKO is the brand name I use most often for these, but there are much cheaper versions.
$5-10
Dt380 Multimeter
Dirt Cheap & good enough for most classroom uses.
$3.50
Syba SY-ACC65018 Precision Screwdriver Set
A good precision screwdriver set makes it so much easier to work with the screw terminal boards. But there are many cheaper options.
$12.00
Donation to Arduino.cc
If you don’t use a ‘real’ Pro Mini from Sparkfun to build your logger, you should at least consider sending a buck or two back to the mother-ship to keep the open source hardware movement going…
$1.00

.. and the required lithium AA batteries are also somewhat expensive, so a realistic estimate is about $25-30 for each logger when you add a couple of sensors.


Addendum:  Using a more advanced processor

After you’ve built a few ProMini based loggers, you might want to try a processor upgrade. The 1284p CPU has twice the speed & 4x the memory, but still delivers comparable sleep current / longevity.

Creating a Normalized Vegetation Index Sensor with two LEDs

Here I’m using the 2019 classroom logger to create a custom ‘Leaf Transmittance Index’ based on readings from an IR LED and the red channel of the RGB indicator already on the logger. Although using generic LED’s introduces non-optimal aspects wrt frequency & bandwidth, the trial successfully distinguished ‘healthy’ vs ‘unhealthy’ plant leaves where a simple visual inspection could not.

When we released the 2019 version of the classroom logger  we updated the starter script to include a technique that uses the indicator LED as a light sensor. This under-appreciated technique leverages the timing capability of microprocessor inputs, rather than the more common approach of using an op-amp to amplify sensor output. Reversing the potential across a diode charges it’s internal capacitance, which can then be discharged by light photons hitting the surface.  In ‘reverse bias’ mode, the photon flux is linearly related to the discharge current, however this depletion method changes the voltage across the capacitor at the same time (+ other factors), so we see a response with exponential decay instead of linear.

Typical discharge curve for an LED charged up to 5 V and then discharged to a threshold of 1.7 V under artificial lighting (fluorescent tube).    Figure 8 from: Absorbance Based Light Emitting Diode Optical Sensors and Sensing Devices pg 2462

Electrically speaking, there is little difference between an LED and a typical photo-diode sensor however an LED’s capacitance is considerably smaller. ( 25 – 60 pF ) The tiny light sensing surface area of an LED (~0.1mm squared) only generates about 50 pA of discharge current in normal ambient conditions, and the reverse leakage through LED’s is exceptionally low (~0.002 pA). The net result is that LED’s are rather slow light detectors, and this phenomenon would be nothing more than a curiosity except for one important aspect: most LEDs detect a relatively narrow band of light wavelengths, making it possible to build a frequency-selective detector without the filters (or monochromators) you’d need to do the same job with photo-diodes or LDRs.

Illustration from that same Sensors review paper, but originally from: Novel fused-LEDs devices as optical sensors for colorimetric analysis.  Talanta 2004, 63, 167–173. Their next paper: Quantitative colorimetric analysis of dye mixtures using an optical photometer based on LED array Sensors and Actuators (2006), used a series of different emitter LEDs and a low band gap IR LED as a universal light detector.

This makes a host of new LED-based instruments possible at the DIY level, and Forest Mims demonstrated this with some  elegant experiments using near-IR LEDs detecting atmospheric water vapor, aerosols with twilight photometers, and he even proved that a single red LED reading provides a reasonable proxy for total PAR. Since Mim’s pioneering work in 1977, the number of applications for LED sensors has grown so fast that now it’s hard to keep up with the ‘review papers’,  let alone the individual publications.  Bench-top chemistry is seeing a host of fluorescence & reaction cell experiments based on frequency matched LED emitter-detector pairs.  By rapidly toggling the same LED between emitting and detecting light, several projects have created other types of sensors like ocean PH. ne can only imagine what will happen when up-converting nanoparticles get thrown into that mix.

What can we do with our logger
using this LED measurement technique
?

Here a day of raw readings from all three LED channels are compared to an LDR in same classroom logger. The unit was deployed in a south facing window with diffusing tape over the housing surface.

Light detectors are often used to make measurements of energy balance, usually by tracking solar insulation. Using the RGB indicator LED already on the logger means we only have a limited number of light frequencies to work with, so we can’t create a ‘full spectrum’ pyranometer unless we use a more advanced solution like SparkFun’s Triad Spectroscopy Sensor . Combining that with good mounting bracket , would provide enough frequency coverage to match some commercial instruments

Figure 2.1. LED-based Sun photometer for the GLOBE project. The rectangles with the green (505nm) and red (625nm) spheres represent the LEDs.  From The Contribution of Dutch GLOBE Schools to Validation of Aerosol Measurements from Space

Despite this limitation, a few dedicated groups have proven that LED photometers can still be quite capable. Most notably the 2-LED Globe program photometers by Brooks et al. at the Institute for Earth Science Research and Education It is quite inspiring to see students using hand-made instruments to produce research good enough to publish in peer-reviewed journals. 

The Globe device uses a more traditional op-amp approach to reading the LEDs, but several aspects of those instruments are directly transferable to other light-sensing projects:

  1. LED based sensors are exceptionally stable over time.
  2. Students can manually aim the detectors at the sun, enabling a basic instrument to do the work of more complicated “sun tracking” machines.
  3. In addition to that directional control, collumnators, or shadow bands are often used to measure diffuse irradiance. Two or more measurements are needed because it’s the difference between those readings that allows you to derive the property you were actually trying to measure.
  4. This also involves correcting for scattering/absorbance by the atmosphere ( Path Length = 1/cos(θ) ) based on the suns angle in the sky (also note: many diy PAR projects hack the white plastic domes out of old photometers as cheap cosine correctors.)

All of Forest Mims LED-based experiments could be replicated with the Cave Pearl classroom logger using the capacitance discharge method instead of op-amps. But there’s a lot more information to be had from LED-sensors by anyone willing to do a little tinkering, especially in the area of vegetation monitoring.

Spectral Characteristics of Vegetation:

Note the relatively absorption-free range: 550-630nm.

Photosynthetic pigments absorb significant amounts of the red and blue light falling on healthy vegetation. Many papers focus on chlorophyll, but there are several other ‘antenna pigments’ which also absorb light to enhance photosynthetic efficiency. Approximately half of the un-absorbed visible radiation is reflected, leading to the green appearance of leaves. Reflectance is much higher at near infrared (NIR) frequencies than in the visible region due to the cellular structures inside the leaves.

Normalized Vegetation Indexes:

A friend recently sent me a link to Rick Shory’s extensive work on the greenlogger which hit the in-box around the same time as SciAm’s article: Earth Stopped Getting Greener 20 Years Ago. Reading about the global decline in vegetation set me on a deep dive into how indexes are used in bio-physical monitoring:

NDVI was developed to estimate vegetation cover from remote sensing data. It is calculated from red and NIR spectral reflectance measurements, and the first key understanding is that spectral reflectances are normalized ratios of the reflected over the incoming radiation in each spectral band. Feeding those ratios into the NDVI calculation means that it can only produce values between -1 to +1.

The second key understanding is that using the ratio of the difference of the red & infrared over their sum corrects for the effect of the solar zenith angle. This eliminates irradiance from the equation, and largely corrects for differences due to topography and transmittance loss in the atmosphere. This allows the comparison of remote sensing data from different times of day, and different latitudes.

NASA uses NDVI as a an indicator of drought, When water limits vegetation growth, it has a different relative NDVI than when the same plant is hydrated because the spongy mesophyll layer deteriorates, and the plant absorbs more of that near-infrared light rather than reflecting it. This is a significant factor for agricultural yield prediction..

Moderate NDVI values represent low density of vegetation (0.1 to 0.3), while high values indicate dense vegetation (0.6 to 0.8)  Zero indicates the water cover, and lower values of NDVI (-0.1 and below) correspond to barren areas of rock, sand, or urban/built-up.  In addition to land cover classification, people use NDVI to infer parameters like Leaf Area Index (LAI), fractional light interception (fPAR), wildfire  burn-area, and other aspects of the biological environment.   But the NDVI index also has some limitations: Any time there’s very low vegetation cover (majority of the scene is bare earth), NDVI will be sensitive to that soil. On the other extreme, where there’s a large amount of vegetation, NDVI tends to saturate.

Over time NDVI has been tweaked in various ways and today there are a large number of different Broadband ‘Greenness’ Indexes that accent different aspects of plant physiology.  And the booming agricultural drone business seems to be inventing more by the day, with claims that somehow their camera tweak produces a new index that’s superior to those of it’s competitors, while their competitors make equally strident claims that company #1 doesn’t know what they are talking about. Public lab has an active community of people hacking cameras for this kind of imagery.

Can we use the RGB LED on the Classroom logger to measure a Vegetation index?

Fig.4.24, pg49, Approximated Emission and Sensitivity Spectra (of an OSRAM LH-W5AM RGB led)  from: Using an LED as a Sensor and Visible Light Communication Device in a Smart Illumination System, Dissertation by André Filipe Rato Bispo

The first challenge is figuring out what we can actually detect, since each index works with a given set of frequencies. LEDs generally have peak detection capability 20-60nm shorter than the wavelength they emit, but this information is rarely available from manufacturers because it’s a use case they were never designed for.  So there are not many sources that compare the emission and detection frequencies for the different LED chemistries. LEDs: Sources and Intrinsically Bandwidth-Limited Detectors (Figure 5) has a reasonable list of specific LEDs characterized as both sources and detectors, but even they didn’t bother to test a garden variety RGB.  Fortunately a few researchers working on visible light communication projects have tested them, generally finding that the blue emitter shifts into UV-A (320-400 nm) detection range, the green emitter shifts down to about 440-460 nm (so it’s now detecting blue…on top of chlorophyll B’s absorption band…), and the red LED channel shifts down to ~560-575nm, with a spectral spread 2 to 3 times wider than its emission band.

Testing to see how a 488nm dichroic mirror  (blue cut-off filter) affected readings on the Green LED detection channel. Note that in this case the round lens was also removed from the top of the LED with sandpaper to both collumnate and diffuse the incoming light. But with the PTFE tape layer added later, the sanding was unnecessary for the index measurements.

But was this true for my LED?  Since we didn’t have a “wavelength scanning monochromator” just lying around I tested the green channel with a blue-cutoff Dichroic Mirror Unlike regular filters, dichroic mirrors are intended to work only for incoming light that is normal to their surface – but given the tight 20 degree dispersion angle of typical 5mm leds, that’s probably OK.  If I was looking for a range of different filters on the cheap, I’d probably look at safety glasses designed for laser work – they usually come with a well specified transmission curves & very sharp cut-offs at certain frequencies.

Sure enough, the discharge on the ‘blue-shifted’ green channel took more than 10x longer with the filter in place, indicating that the green channel was sensitive to frequencies below the filters 488nm cut-off.  If the red LED channel also follows the pattern from Bipso’s paper, then red’s detection will include some green & some red, with a peak at yellow.  We might be able to use these frequencies for a BLUE vs GREEN variant of NDVI, but several sources indicating that blue indexes were sub-optimal because that chlorophyll absorption is strongly overlapped by carotene. So blue based indexes usually show less contrast between stressed versus non-stressed plants.  The loss of both the blue channel (now a UV detector) and the green channel (now a blue detector) meant that we need to add an IR led to have enough information for a viable index.

Distance sensor modules on eBay are one inexpensive source of IR LEDs.  These sensors are somewhat limited when used for their intended purpose, but a little cut & paste lets you merge the emitter LED with a daylight filter ‘cap’ cut from the photo-transistor on the same board:

IR detectors can be sensitive to visible light unless you add a daylight filter, and this little hack is much cheaper than buying a Wratten 87, 87B, or 87C. These distance modules usually have LEDs which emit at 940nm, but the same shorter wavelength-shift also applies to the IR LED, and this pulls the detection peak into the 920nm range – safely out of the absorbance well created by atmospheric water vapor. (something to keep in mind if you are actually trying to build a water vapor sensor – you probably need to select an emitter at least 25 nm above 940)

Note that the negative terminal of the IR LED is under the same screw terminal (D7) as the common cathode leg of the RGB indicator led. Columnating the LEDs with heat shrink tubing should also make them less sensitive to light reflections inside the logger. Note: our GitHub code uses a port command that assumes that shared ‘negative’ pin to be one of D3 to D7 on the pro mini. Here I’ve set #define LED_GROUND_PIN 7 as the common GND connection, RED_PIN 6, GREEN_PIN 5, and the positive side of the IR led is on pin D8. Set  #define BLUE_PIN 8 to force the former blue pin code to take the IR LED reading. Also note: that our code initially ‘lights’ the indicator LED via INPUT_PULLUP mode, instead of setting the pins to OUTPUT & HIGH, so limiting resistors are not necessary for that step.

Even with a filter and heat shrink tubing the IR LED was easily saturated, and it took 28 layers of plumbers tape to bring readings from the IR LED into approximately the same range as the RGB readings (which had only 1 layer of diffusing PTFE tape on it)  This was done with the logger tethered a laptop, displaying readings in the serial monitor window. I simply added one layer at a time until the readings under direct full sunshine for both the IR & Red channel were in the 300 to 500 range (using our logger code from Github). Then I add some heat shrink to hold those layers of teflon tape in place, leaving only the round dome of the LED(s) exposed.

Even with IR on board, the lack of a sharp red detector means we can’t produce the traditional NDVI. (unless we add a Wratten25) With the ‘red’ LED channel actually detecting from 525 to 650 (FWHM 560-610) and peaking around 575nm we have to invent our own pseudo index that might be described as  ‘lime-green’ NDVI.  This actually isn’t so bad, as several recent papers demonstrate that green-based NDVIs are more sensitive to chlorophyll than the red based indexes, and they have stronger correlation with total leaf area.

 

Other Challenges:

With a tinfoil wrap, light can only enter through the 5 x 8cm exposure window above the LEDs. One layer of blank label-maker tape on the lid diffuses the light and prevents the plastic struts inside the housing from creating hot-spots. A final outer wrap of clear packing tape protects the tinfoil.

Bandgap voltages vary with temperature, changing the LED emission wavelength by ~ 0.1nm/°C.  Detection wavelengths should follow suit, so it’s probably best to make sure the temperature varies as little as possible between our scale points and the target readings.  Since the LED detectors are inside a Plano box, there is potential for some frequencies to be lost, but materials like high density polyethylene (HDPE)  have remarkably smooth absorption curves that don’t become extreme until you reach UV. The fact that Vi’s are ‘a ratio of ratios’ means that we have to compare the raw sensor readings to direct insulation values before the index can be calculated. Housing losses should affect the high reference and the target readings in the same way, so it should not throw off the final index – essentially we treat it like transmittance loss in the atmosphere.

Since we can’t know the responsivity of our garden variety LEDs without lab testing, the best we can do is standardize the data by scaling it against a “maximum” reading (when the sun is shining directly on the top of the logger) and a second “minimum” point for each channel (obtained by covering the entire logger with tinfoil).  The dark point reading also address thermal leakage through the LEDs.  One drawback of this mcu-based method is that the raw discharge-time readings follow an exponential decay curve, so we need to take the log of those readings to linearize the data before scaling, and since our decay-time readings are inversely related to the photon flux, we need to apply  1/LN(reading)  to all data before the max/min scaling. Technically speaking, photon flux is not the same as irradiance, but the the index’s normalization sweeps that little issue under the rug too.

Using the sun as our high scale point necessitates that the readings are done under a clear blue sky. Clouds passing overhead between readings (or other haze / humidity variations) could change the ratio of IR to visible light more than the plants you are trying to monitor:

Rain is another potential complication, as water strongly absorbs IR.  So the readings have to be taken long enough after a rainfall that no water droplets are present on the surfaces. So with the basic 2x LED configuration I’m using here, you have to wait for good weather, and take the readings under a clear blue sky, between mid morning & mid afternoon.

Cloud induced variations could be compensated by putting two sets of sensors on the logger (one pointing up  &  one pointing down) for simultaneous correction of direct insolation vs surface reflectance, but for now this is just a prototype trial to see if a decent ‘lab exercise’ can be developed for the Cave Pearl Loggers with minimal additions to the basic build.

Does it work?

Backyard trial testing the reflectivity of my lawn in an area deliberately chosen as “unhealthy” grass .

With all the rough assumptions up to this point I was surprised to find the reflectance readings falling within the broad range of ‘textbook’ values. A reading 1m above a relatively healthy section of my lawn produced an index of 0.39 while a reading above a mangy half-dead section (photo: right) produced a much lower index value around 0.215   A patch of bare dirt read at 0.044, and my gravel driveway produced an index of -0.13   The front flower garden produced a reading of 0.292  It’s hard to know how representative this is given the wide range of values listed for different plant species in the various spectral libraries.

With the challenges of species variation & water condition, the use of verified bright & dark targets is pretty common in biophysical sensing. White panels coated with barium sulfate paint, Komatex, or Teflon are sometimes used because they have reflectance near 100% with very few  specular artifacts. NASA’s Aster Spectral Library suggests that most flat black paints have a similar response through visible and NIR, so some Rustoleum on a big sheet of cardboard might work as a 4% calibration point.

A better idea for the classroom:

Covering large areas with a grid of these single-shot readings would take a substantial amount of time, and even relatively short trials run into issues with trees and other large shadows creeping into the test patches throughout the day. So logging reflectance is more suited to long-term measurements of vegetation cover at single location. (or forest canopy transmittance)

After a bit more reading,  I began to notice a pattern:

(click to enlarge) Leaves cover the input window, secured to prevent wind shifting.

So index calculations ought to work with light that is transmitted through the leaves because the two curves contain very similar information. With the standard reflectance-based NDVI, ratings between 0 and 0.33 indicate unhealthy or stressed plants, 0.33 to 0.66 is moderately healthy, and 0.66 to 1 is very healthy. Flipping the calculation to a transmission-based version will shift those values significantly, but we can still use that trend as a rough guide to whether the method is working.

As luck would have it, this insight occurred just before another fieldwork trip, so it was more than a week before I could test the idea. The first trials used leaves from tropical almond trees which are dry-season deciduous. Re-capturing some of their ‘energetically expensive’ chlorophyll turns the leaves pinkish-red or yellow-brown due to the leftover pigments such as violaxanthin, lutein, and zeaxanthin. These Xanthophylls are yellow pigments from one of the two major divisions of the carotenoid group (the other division is formed by the carotenes). They act to modulate light energy and may also serve as a non-photochemical quenching agents to deal with excited states of chlorophyll that are overproduced during photosynthesis under the intense light conditions.

Leaf Number Red RGB channel: scaled
% of full sun reading
IR LED reading: scaled
% of full-sun reading
(gYr) “Transmission
based” NDVI
#1 (green) 37.24 82.81 0.380
#2 (orange) 59.47 83.13 0.166
#3 (red) 42.22 80.95 0.314
#4 (yellow) 67.62 82.91 0.102

All leaves were ‘fresh-picked’ from the tree, and the percent transmission numbers were averaged from five readings taken one minute apart. Natural light is notoriously variable, so most index sensors use considerably more sample averaging than that.

It’s not surprising that the yellow leaf was well discriminated, but the fact that the green & red leaves produced similar values highlights an issue with our rough prototype: the widened spectral spread of the red LED channel makes it difficult to distinguish between light frequencies on either side of the response curve.

Leaf (click to enlarge)  Red RGB channel: scaled
% of full sun reading
IR LED reading: scaled
% of full-sun read
(gYr) T-NDVI
#10 (Dead on ground) 26.96 65.38 0.416
#11 (yellow patch) 44.90 79.45 0.278
#12 (green & healthy) 34.27 76.17 0.379
#13 (>50% yellow) 55.56 77.79 0.167

Despite the limited red/green selectivity, this set shows that, at least for ‘fresh’ leaves, our custom index can still discriminate the loss of the chlorophyll. The dead leaf suggests that will only work up to a point. Dead vegetation reflects a greater amount of energy than healthy vegetation throughout the visible spectrum but reflects less IR, so any actual ‘dead-spots’ will raise a transmission-based index value. This is a limitation for our method – which does not produce a natural separation between healthy and dead vegetation like a reflectance based index would. This would be serious problem for remote sensing applications, but when a leaf that you can hold in your hand is that badly off, you probably don’t need a sensor to tell you the plant is not thriving.

Does this transfer to other plant species?

Leaf (click to enlarge)  Red RGB channel:
scaled % of full sun
IR LED reading: scaled
% of full-sun read
(gYr) T-NDVI
#14 (Green leaf – from
stressed plant1)
39.29 77.51 0.327
#15 (Brown spots – from stressed plant1) 37.71 76.23 0.338
#16 (Green leaf from
healthy plant2)
21.83 68.05 0.514
#18 (>50% yellow – other stressed plant3) 32.65 65.47 0.334

As expected, the overall light transmission numbers were different for palm than they were for the tropical almond leaves. So a calibration set would need to be created for each plant species to put these numbers into context. I’m assuming the green/yellow discrimination is due to chlorophyll levels in the almond leaves, but there could be other confounding factors like Anthocyanin in the hardier palm leaves. (Antho. also absorbs in our sensor band, and is abundant in senescing leaves) 

While #14 and #15 look different they were taken from same plant, and the yellow-brown spots on #15 make it clear the plant was under some kind of stress.  Visually, I would not have been able to distinguish leaf #14 from #16 but the index identified #14 as being from an “unhealthy” plant.  Given the relatively wide spectrum we are working with here, this is a remarkable result – suggesting that with a bit of homework to find LED’s with tighter detection bands, we could produce an inexpensive chlorophyll meter (like the SPAD?) or we could tune the idea for other pigments/applications. Using a white LED emitter above the leaf could enable a variant of our simple approach that was not dependent on the sun, and adding couple of reads using only green & red emitters above the leaf would enable us to distinguish which side of that wide detection curve we were on. Commercial chlorophyll meters sometimes use emitters at 660 nm & 940 nm , measuring both reflectance AND transmission so an accurate absorbance value can be calculated. Even then plants cease to create chlorophyll once a certain threshold has been reached, so these meters are often used to detect deficiencies (by comparison to a well fertilized control group) rather than concentrations.

And finally, a classroom lab could combine this kind of index-based characterization with paper chromatography to verify the pigments in the leaves via their Rf factors. A more advanced approach would do this quantitatively, turning this index into a true diagnostic tool. This could also be a good accompaniment to the many other Arduino based plant monitoring projects, as a growth or health verification stage. In northern climates this could even be done with house plants by taking readings ‘before’ and ‘after’ some experimental intervention. With those large temporal separations, you would want to take new max/min readings each time for the scaling & avoid using artificial light sources as these  introduce frequency artifacts that could interfere with the index.

Addendum 2020-10-28

“The same procedure is used for oxygen saturation measurement. Here the principle is to measure the absorption of the hemoglobin in the blood. Oxygenated hemoglobin (HbO2) has a significantly different absorption of light than non-oxygenated hemoglobin (Hb). To detect this difference, the skin is illuminated with one red and one IR LED light and a photodetector measures the absorption.”

Easy 1-hour Pro Mini Classroom Datalogger [Feb 2019]

Dupont jumper variant of the “fully soldered’ Classroom Data Logger from the Cave Pearl project: This version uses dupont jumpers to reduce assembly time to about 1 hour

Note: An updated version of the classroom logger was released in 2020:  CLICK THIS LINK to view the newest build tutorial.

It’s only been a couple of weeks since the release of the 2019 EDU logger, and we’re already getting feedback saying all the soldering that we added to that tutorial creates a resource bottleneck which could prevent some instructors from using it:

“Our classroom has just two soldering stations, and the only reason there are two is I donated my old one from home. So we simply don’t have the equipment to build the logger you described. And even if we did, some of my students have physical / visual challenges that prevent them from working with a soldering iron safely…”

Or goal with that design was to give students their first opportunity to practice soldering skills that are needed when repairing kit in the field. However helping people do science on a budget is also important – so that feedback sent us back to the drawing board.  After a little head scratching we came up with a new version that combines the Dupont jumper approach we used in 2016, with this flat-box layout. In the following video, I assemble one of these ‘minimum builds’ in about one hour.  To put that in perspective, the fully soldered version takes 2 – 2.5 hours for someone with experience.

Note: After you’ve seen the video to get a sense of where you are headed, it’s usually much better to work from the photos (below) when assembling your logger. Youtube videos make it look easier than it actually is when you are just starting out. So the first one you build could take several hours as you figure out what you are doing, the second will take half as long, and the third one you make usually takes less than two hours. With some practice you can easily make 4-5 of these things a day.

This variation of the basic 3-component logger is optimized for quick assembly, so the soldering has been reduced to adding header pins and bridging the A4/A5 I2C bus to the outer terminals.  An instructor could easily do that ahead of time with about 15 minutes of prep per unit, leaving only the solder-less steps for their students. After the header pins are in place, connections to the central Pro Mini are made by simply twisting stripped wire ends together and clamping them under screw terminals.

You should use Lithium AA batteries with a 2-cell unregulated supply because the slope of an alkaline batteries discharge would bring the system down to  ProMini’s brown-out of ~2.7v when only 40% of the battery capacity has been used. (note the SD card safe down to 1.8v) While the voltage of a ‘brand new’ Lithium AA is usually 1.8v/cell, that upper plateau provides a sleeping logger voltage of ~1.76 v/cell once the batteries settle, and that briefly dips to about 1.625v/cell under load during data saves. At low temps of about 5°C (in my refrigerator) those SD card voltage dips go down to about 1.525 v/cell. Unloaded readings of 1.5v on a Li AA = battery is dead.

This time reduction involves a few trade-offs, and bringing the I2C bus over to A2 & A3 leaves only two analog ports readily accessible ( although A6 & A7 are still available if you add some jumpers). Removing the regulator & battery voltage divider adds ~30% more operating life, but it also forces you to deal with a changing rail voltage as the Lithium AA batteries wear down. The daily variation is usually quite small, but for quantitative comparisons on monthly scales you will need to correct for the change in rail voltage over time if your sensor circuits are not ratio-metric. To do this voltage compensation multiply your raw sensor readings by the the ratio of (3300mv) / (current rail voltage).  Here 3300mv is just an arbitrary comparison point, which you could replace with any rail voltage reading from the data saved by your logger. Batteries have a lot of mass, so thermal lag in battery voltage can also cause hysteresis for analog temperature sensors unless you read the reference under the same conditions.

 (NOTE: complete parts list with supplier links are located at the end of this post)

Pro Mini Prep:

Solder the UART pins & test ProMini board with the blink sketch:  Set the IDE to (1) TOOLS> Board: Arduino Pro or Pro Mini (2)TOOLS> ATmega328(3.3v, 8mhz) in addition to the (3)TOOLS> COM port to match the # that appears when you plug in the serial adapter board.

Once you know you have a working Promini board: Remove the power LED [in red]. Removing the pin13 LED [yellow square above] is optional. Leaving the pin13 LED in place lets you know when data is being saved to the SD card because the SPI bus SClocK signal flashes the LED.

Remove the voltage regulator with snips. Your system voltage will vary over time, but our starter script records that rail voltage without a voltage divider.

Add pin headers to the sides & Serial input end of the Pro Mini.

Bridge the two I2C bus connections for side access with the leg of a resistor. Connect A4->A2 & A5->A3.

Adding DIDR0 = 0x0F; in Setup disables digital I/O on pins A0-A3 so they don’t interfere with I2C bus coms.

NOTE: The Screw Terminal board we use in this build was designed for the 5v Arduino NANO, so the shield labels don’t match the actual 3.3v Pro Mini pins on the ‘analog’ side. (the digital side does match) To avoid confusion may want to tape over those incorrect labels and hand write new labels to match the pattern above. Wire connections in this tutorial will be specified by ProMini pin labels:  D10-13 are used for the SD card, A4=A2 is the I2C Data line, and A5=A3 is the I2C clock line, A0 & A1 are not used.

Technically speaking, bridging the I2C bus (A4=data & A5=clock) over to A2 & A3 subjects those lines to the pin capacitance and input leakage of those analog pins (regardless of whether that channel is selected as input for the ADC p257). But in practice, the 4K7 pull-up resistors on the RTC module can easily handle that at the 100 kHz default bus speed. Adding DIDR0 = 0x0F; in setup disables digital I/O on pins A0-A3 to prevent interference with the I2C bus. If you want to disable the IO on A2 & A3 ‘only’ add bitSet (DIDR0, ADC2D); bitSet (DIDR0, ADC3D); to your code.

Also note: On the UART adapter in the picture above, the USB to TTL adapter pins are in the reverse order to the Pro Mini board. This is a fairly common issue with clones and if the blink sketch never uploads flip the adapter around and try again. I have connected 3.3v ProMinis to UART modules the wrong way round many, many times, and not one of them has been harmed by the temporary reversal.

Screw-Terminal Component Stack:

Add 3 layers of double sided tape so the tape is thicker than the solder pins.

Align RX&TX corner pins before inserting. The GND points on the screw terminal board may be interconnected (via the back-plane) & must match the ProMini’s GND pins.

Gently rock the Pro Mini back to front (holding the two short sides) until the pins are fully inserted. Some ST shields have mis-aligned headers so this insertion can be tricky.

Remove the last three ‘unused’ pin headers to make room for the SD adapter

Note: Screw-terminal board labels do not match the ProMini pins on the ‘analog’ side

Remove bottom 3 resistors from the adapter – leave the top one in place!

Separate Dupont Cable wires & click them into a 6-pin shroud.

Cable Color Pattern:     Black =GND,   Purple=MISO,   Brown=CLocK,   Orange=MOSI,   Grey=CSelect,      and   Red=3v3

 

 

 

Use foam tape to attach SD module to the Screw Terminal board. Metal tabs should be visible on top surface.

Measure, cut & strip the 4 SPI bus wires (NOTE the ‘Nano’ ST board labels say A0-A3 which does not match the D10-13 Pro Mini pins on this side of the board)

Grey (CS) to ProMini D10Orange (MOSI) -> pm D11,      Purple (MISO) ->pm D12,        Brown (CLK) -> pm D13    

 

 

Add three jumper wires to the red power line from the SD module, one with a male end pin. I often add Dupont ends with a crimping tool, rather than using a pre-made jumper.

Strip & twist the 4 red power wires together & add heat-shrink for strain relief. Bundling wires like this is easier if you make the stripped area a bit longer.

A short red jumper goes to RAW(pm)=VIN(st) to recruit the orphan capacitor.

Long red jumper bridges power to ‘unused’ screws on other side of the Terminal board, and the wire with the Dupont male end will go to the breadboard.

Add 2 extra wires to the black GND wire from the SD module (1 with a male Dupont end ). Jumper one black wire across the Promini to an unused terminal beside the red power wire.

The GND bundle completes Pro Mini / ST board / SD module stack. The ‘pinned’ Red & Black ‘pinned’ jumpers shown here are about 1 inch too short to reach the breadboard easily… make yours longer.

Note: You could connect the battery holder lead wires directly into the multi-wire Vcc & GND bundles: skipping the 2 jumpers crossing over to the other side of the ST shield.  But adding those jumpers provide extra Vcc/GND connection points & the ability to easily replace the battery holder later if you have a battery leak.

I always try to make my Dupont connectors so that the metal & plastic retainer clips are accessible (in this case facing upwards) after the logger is assembled. That way you can diagnose bad wire connection with the tip of a meter probe, and if necessary, pull out & replace a single bad wire in the Dupont connector without taking everything apart.

RTC module:

Remove two SMD resistors from the RTC board with the tip of your soldering iron.

The DS3231 modules often have flux residue – clean this off with 90% isopropyl.

 

Cable: Blk(gnd), Red(vcc), White(sda), Yellow(scl), Blue(sqw).  Shroud retainer clips face up & there is no wire on 32K output.

First tape layer

Next two tape layers

OPTIONAL: adding header pins to the cascade port provides a convenient attachment point for I2C sensors later.

Optional: After removing the two SMD resistors on the module, you can clip the Vcc leg on the RTC chip which forces the clock to run entirely from the backup coin-cell battery. This reduces the loggers overall power use by 0.09mA bringing a “no-reg & noRTCvcc” build below 0.1mA when the logger sleeps between sensor readings (this should run for more than 2 years on fresh lithium AA cells) . But the risk is that if you bump the RTC backup battery loose, that disconnection resets the clock time to Jan 1st, 2000. (note: while the time stamps will be wrong after that kind of reset, the logger will continue running after the next hour/min alignment occurs with the ‘old’ alarm time)   A couple of pieces of soft 1.6mm heat shrink tubing under the spring makes the negative coin-cell connection stronger, an a touch of hot melt glue will secure the battery on the top edges.  A CR2032 can power the RTC about four years but you have to set bit six of the DS3231_CONTROL_REG to 1 to enable alarms when running from the coin-cell. (our starter code does this by default) This modification also disables the 32.768 kHz output pin on the RTC.  Visit our RTC page for more detailed information on this clock module.

Final Assembly:
(Note: references here are to pin numbers/labels on the ProMini which do not match ST board labels on the analog side)

Attach the Pro Mini stack & RTC to housing with the double-sided tape.

Trim white & yellow I2C wires from the RTC & add 1 extra wire with dupont ends for each I2C line to bring the bus over to the breadboard

Attach yellow SCL line from the RTC beside the red 3v rail (ie to A3=A5 on the ProMini) then the white SDA line from the RTC to A2=A4.

The four extra jumper wires with male Dupont ends on Vcc, GND, & both I2C lines. These get patched over to the breadboard so you can add I2C sensors.

Each wire must be plugged into its own separate vertical column on the breadboard. Add a 2nd layer of foam tape to the bottom of the bread-board before attaching.

The RTC power line joins that short red jumper on RAW(pm)=VIN(st) at the end of the screw-terminal board.

Some of the box bottoms have slight bowing. If any component doesn’t stick well enough: add another layer of foam tape.

Attach the RTC’s black ground wire to GND & the blue SQW alarm line to ProMini pin D2

Attach 2xAA battery holder with 2 layers of foam tape. Trim wires to length. Use black 30lb Mounting Tape for extra strength.

Battery wires join the black & red jumpers from other side of the terminal board. All six of the ‘unused’ screw terminals we clipped earlier can be used to make secure ‘dry wire’ connections in this way.

Connections complete except the indicator. 5050 LED modules often come with pre-installed limit resistors which make them safer for the classroom. But you can use a raw 5mm LED if you only light the LED  with the INPUT_PULLUP command.

A ‘common cathode’ RGB LED module on pins  Red=D6, Green=D5,   Blue=D4, &  GND=D3.

Your Logger is ready for testing!

(Note: Most of the time the tests listed below go well, however if you run into trouble at any point read through the steps suggested for Diagnosing Connection Problems at the end of this page.)

Install the Arduino IDE into whatever default directory it wants to be in – we’ve had several issues where students tried to install the IDE into some other custom sub-directory, and then code will not verify without errors because it doesn’t know where to look for libraries. If you have not already done so, there are three things you need to set under the IDE>TOOLS menu to enable communication with the logger:

Note: that the “COM’ setting will be different for each computer, so you will have to look for the one that appears on your system AFTER you plug in the UART module.

The one that’s easy to forget is choosing the 328P 3.3v 8Mhz clock speed. If you leave the 328p 5v 16mhz (default), the programs will upload OK, but any text displayed on the serial monitor will just be a random bunch of garbled characters because of the clock speed mismatch.  Also be sure to disconnect battery power (by removing one of the AA batteries) whenever you connect your logger to a computer.

1. Test the LED – Edit the default blink sketch, adding commands in setup which set the digital pin 3 connected to the ground line of the LED to “OUTPUT” and “LOW”
      pinMode(3, OUTPUT);   digitalWrite(3, LOW); 
Since we removed the ‘default’ indicator led on the Pro Mini board, you will also need to change LED_BUILTIN variable in the blink code example to one of the pins connected to one of the color channels on your led module. (in this example change LED_BUILTIN to either 4, 5, or 6)

2. Scan the I2C bus with the scanner from the Arduino playgound. The RTC module has a 4K eeprom at address 0x56 (or 57) and the DS3231 RTC should show up at address 0x68.

The address of the eeprom can be changed via solder pads on the board, so sometimes it moves around. If you don’t see at least these two devices listed in the serial monitor when you run the scan, there is something wrong with your RTC module or the way it’s connected: It’s very common for a beginner to get at least one set of wire connections switched around during the assembly. With the screw terminals this takes only a few moments to fix. Also have a spare RTC module on hand in case you get a defective module…which is fairly common.

NOTE: Some sensors really need the stability provided by the on-board voltage regulator. Here is an alternative arrangement of parts for the classroom logger that leaves the 3.3v regulator in place on the ProMini and powers the logger from 4xAAA alkaline batteries (NOTE: regulated builds also leave out the short red jumper that was used to recruit the orphan capacitor after the reg was clipped.  The RTC & added sensors now get connected ONLY to the Pro-Mini’s regulated 3.3v rail )   We’ve designed the Cave Pearl Logger for maximum flexibility, so you can easily change components and positions like this to suit the needs of your design / experiment.

3. Set the RTC time, and check that the time was set – The easiest method would be to use the SetTime / Gettime scripts from our Github repository, but first you need to download & install this RTC control library  The SetTime script, automatically updates the RTC to the time the code was compiled (just before uploading) so you only run SetTime once, and then immediately upload the GetTime sketch to get the SetTime code out of memory after it’s done it’s job. Otherwise SetTime will reset the RTC to the ‘code compile time’  every time the Arduino restarts (and the Arduino restarts EVERY TIME you open the serial window…) 

There are dozens of other good Arduino libraries you could use to control the DS3231, and there is also a script over at TronixLabs.com that lets you set the clock without installing a library. [ in 24-hour time, & year with two digits eg: setDS3231time(30,42,21,4,26,11,14);  ] The trick with Tronix’s “manual” method is to change the parameters in the line: setDS3231time(second, minute, hour, dayOfWeek, dayOfMonth, month, year);  to about 2-3 minutes in the future, and then upload that code until about 20 seconds before your computers clock reaches that time (this compensates for delay caused by the compilers processing & upload time). Open the serial window immediately after the upload finishes, and when you see the time being displayed (and it’s not too far off actual…) upload the examples>blink sketch to remove the clock setting program from memory.

Another option would be to try setting the clock’s time using one of the serial window input utilities from Github.

4. Check the SD card is working with CardinfoChanging chipSelect = 4; in that code to chipSelect = 10;  Note that this logger requires the SD card to be formatted as fat16, so most 4GB or larger High Density cards will not work because they get formatted as fat32. Most loggers only generate 5 Kb of CSV format data per year when they are running, so you don’t need a big SD card. In fact older, smaller SD cards in the 256-512mb range often use less power if it’s a good name brand like Sandisk or Nokia.

Zip-Tie Mounting Bases are an easy way to add attachment points inside your logger to secure sensor cables, or desiccant packs. These adhesive cable tie mounts come in many varieties, and cost about 10¢ each at most hardware stores.

5. Optional: If you are running with no regulator & using analog sensors: Calibrate your internal voltage reference with CalVref from OpenEnergyMonitor.

This logger uses an advanced code trick to read the positive rail voltage to ~11mv resolution by comparing it to an internal 1.1v reference  inside the processor. That internal ref. can vary by ±10% from one chip to another, and CalVref gives you a constant which will make the rail=battery voltage reading more accurate. Load CalVref while the logger is running from USB power and then measure the voltage between GND and the positive rail with a good quality voltmeter. (this voltage will vary depending on your computer’s USB output, and the UART adapter you are using) Then type that voltage into the serial monitor window entry line & hit enter. Write down the reference voltage & constant which is output to the serial monitor window. I usually write these ‘chip-specific’ numbers inside the logger with a black sharpie. You will need to add that info to the core data logger code later by changing the line #define InternalReferenceConstant 1126400L to match the long number returned from CalVref. Alternatively you could just tweak the value of the constant ‘by hand’, increasing or decreasing the value till the reported rail readings match what you see with a voltmeter. After you’ve done this once or twice you can usually get within 15mv of actual with about 10 trials.

This calibration brings the starter script’s battery readings within ±15mv of actual but you can skip the CalVref procedure if you are only using digital sensors, as the script will still produce reasonably good battery readings with the default 1126400L value. Increasing the rail reading accuracy is more important when you are using ANALOG sensors which use that rail voltage to drive your sensors – so the +ive rail directly affects their output if you are not reading a ratiometric circuit.

6. Find a script to run your on logger. For test runs on a USB tether, the simplest bare-bones logger code is probably Tom Igoe’s 1-pager at the Arduino playground. It’s not really deploy-able because it never sleeps the processor, but it is still useful for teaching exercises and testing sensors after you set chipSelect = 10;  In 2016 we posted an extended version of Tom’s code for UNO based loggers that included sleeping the logger with RTC wakeup alarms. Our latest logging “Starter Script” has grown in complexity to ~750 lines, but it should still be understandable once you have a few basic Arduino programming concepts under your belt.

After 1-2 minutes of kneading to mix the epoxy you have ~ 60 seconds to work the putty into place. (it will become rock-hard within ~5-10 minutes). Be sure to leave yourself enough extra wire/space inside the housing so that you can open and close the lid easily without disconnecting anything after the putty hardens. This seal is not strong enough for underwater deployments, but it should easily withstand exposure to rain-storm events. This putty is also a quick way to make custom mounting brackets, or even threaded fittings if you wrap it around a bolt (which you carefully remove before the putty hardens completely)

In the previous tutorial we attached external sensors with a cable gland passing through the housing and epoxying them into a pvc cap.  Cheap cable glands are notoriously unreliable and I always add a second rubber oring on the inside – and often over-seal the outside with  conformal coating. But for a simpler classroom project you could simply drill small a hole through the lid and stick the sensor/module on top of the housing, seal the hole with double-sided tape. Thicker pass-through wires can be sealed reasonably well with plumbers epoxy putty from the hardware store. This putty is non-conductive, and adheres quite well to both metal & plastic surfaces. 

Remember that breadboard connections are very easy to bump loose, so once you have your prototype circuits  working, its usually best to re-connect the sensors directly to the screw terminals before deploying a logger where it could get knocked about. In a pinch you can secure breadboard pins with a ‘tiny’ drop of hot glue to keep them from wiggling around.

There is no power switch on the loggers, which are turned on or off via the battery insertion. Use a screwdriver, or some other tool, when removing the batteries so that you don’t accidentally cause a series of disconnect-reconnect voltage spikes which might hurt the SD card.

Using the logger for experiments:

Logger mounted on a south-facing window and held in place with double sided tape. Here the top surface of the housing was covered with two layers of white label-maker tape to act as a light diffuser. PTFE is another excellent light diffusing material available in different sheet thicknesses.  The ‘divot’ on the lid of the Plano box is just a bit larger than 55mm x 130mm x 3mm (depth). The “teflon” tape that plumbers use to seal threaded joints can also be used in a pinch. PTFE introduces fewer absorbance artifacts than other diy diffusers like ping-pong balls, or hot melt glue. Most light sensors like the TSL2561 need 3-5mm of that PTFE sheeting to prevent the sensors from saturating in full daylight. LED’s don’t saturate that badly, but you still lose all the useful detail in your data at peak brightnesses > 80,000 lux unless you add a diffusion layer to attenuate the light.

Many types of sensors can be added to this logger and the RTC has a built-in temperature register which automatically gets saved with our starter script. The transparent enclosure makes it easy to do light-based experiments. Diodes, LED’s & solar panels are basically the same device. So grounding the indicator LED through a digital pin allows it to be used as both a status light, and as a light sensor.   The human eye is maximally sensitive to green light so readings made with that LED channel approximate a persons impression of overall light levels.  Photosynthesis depends on blue and red light, so measurements using LED’s that detect those two colors can be combined for readings that compare well to the photosynthetically active radiation measurements made with “professional grade” sensors. In fact Forest Mimms (the man who discovered the light sensing capability of LEDs in the first place) has shown the readings from red LED’s can be used as a reasonable proxy for total PAR.  Photoperiod measurements have important implications for plant productivity, as do  measurements of transmittance through the plant canopy. Chlorophyll fluorescence is another potential application, and the response of plants to UV is fascinating.

The original code for using LEDs as sensors is from the Arduino playground. This polarity reversal technique does not require the op-amps that people typically use to amplify the light sensing response but it does rely on the very tiny parasitic capacitance inside the LED. (~50-300pF) This means that the technique works better when the LED is connected directly to the logger input pins rather that through the protoboard (because breadboards add random capacitance) . Another thing to watch out for is moisture condensation inside your logger housing: this provides an alternate discharge path for the reverse-charge on the LED, which effectively shorts out the light level reading.

We have integrated this into the starter script which you can download from GITHUB. I’ve tweaked the playground version with port commands so the loop execution takes about 100 clock cycles instead of the default of about 400 clock cycles.  The faster version was used to generate this light exposure graph with a generic 5mm RGB LED, with a 4k7Ω limiter on the common ground which was connected to pin D3:

Red, Green & Blue channel readings over the course of one day from the indicator LED in the logger photo above.  The yellow line is from an LDR sensor the same unit, that was over-sampled to 16-bit resolution. The LED sensor has a logarithmic response and the left axis on the graph is a time- based measurement where more light hitting the LED sensor results in a lower number. Note how the RED signal changes more quickly than Blue & Green at sunrise & sunset.  LED’s work well with natural full-spectrum light, but their limited frequency sensing bands can give you trouble with the spectral distribution of  indoor light sources. The peak spectral response of LED’s is usually around 30–50nm lower than their peak emission wavelength So the blue channel is actually recording in the near-UV range, the Green channel is responding at ~ 420nm (blue) and the red channel is actually responding to a wide band of yellow-green light. 

Characterizing light absorption and re-emission is also a primary technique in climate science . For example, measuring light intensity just after sunset with LEDs inside a heat-shrink tube pointed straight up can provide a measure of suspended particles in the stratosphere. An “ultra bright” LED has more than enough sensitivity to make these columnated readings, in fact on bright sunny days you usually have to place the LED/sensor beneath a fair thickness of white diffusing material (like PTFE tape) to prevent it from being completely saturated.  Older LEDs that emit less light can sometimes be easier to work with because they are less sensitive, so the discharge time does not go to zero in high-light situations. Other sensor experiments are possible with LED’s in the IR spectrum which can be used to detect total atmospheric water vapor. A light transmission-based variant of the NDVI ratio can be used to determine plant health after adding an IR LED to the logger.

Of course, full sun exposure can also cook your little logger: temps above 80°C may cause batteries to leak or damage the SD card. If you have to leave the logger in full sun, add a bit of reflective film or a layer of aluminum foil around the outside to protect the electronics. Though if use a light sensor you’ll need to leave a little un-covered window for it to take a reading. 

You might also find it handy to add a few holes to the Plano box as tie-down points:

and it’s always a good idea to add desiccant packs inside the box to prevent condensation. If you use a desiccant with color indicator beads, you can check whether they are still good simply by looking through the transparent housing.

If you clipped the RTC leg, your logger should pull less than 0.1mA while sleeping. Back of the envelope for Lithium AA’s is about 7 million milliamp-seconds of power, with your logger burning about 8600 mAs/day at 0.1mA.  So clipping the voltage regulator and cutting the vcc leg on the RTC should easily get you out to about a year before you fall off the “upper plateau” of the lithium burn-down curve, and more likely to ~two years if your sensors don’t use much power (…and you have a well behaved SD card).  With the RTC power leg still attached you’ll see sleep currents in the 0.18-0.22 mA range, so at least 6 months of operation before you come close to triggering the low voltage shutdown (which is embedded in the logger code). I’m being conservative here because it all depends on your sensors and other additions you make to the base configuration. For every type of sensor the market, there will be some brands that draw much less power than others, and it’s always worth a look at the data sheet to make sure you are buying one that will run longer on a given set of batteries. Good sensors automatically go into a low current sleep mode whenever the bus has been inactive for a period of time.

While the LED sensor idea is fun to work with, it’s a very slow method that keeps the logger running for many seconds per reading when light levels are low -> so reading all three color channels will probably cut your operating life in half again. Figuring out how to only take those light readings during the day is a good coding exercise for students that saves quite a bit of power. The RTC’s temperature record is pretty crude, so we’ve also added support for the DS18b20 temperature sensor in the base code. If you have a genuine DS18b20 (yes, crummy fake knock-off sensors are a thing) , these venerable old temperature sensors draw virtually no power between readings.


TransparentSinglePixl
Bill of Materials: $20.50
Plano 3440-10 Waterproof Stowaway Box
Sometimes cheaper at Amazon. $4.96 at Walmart and there are a selection of larger size boxes in the series. 6″ Husky storage bins are an alternate option.
$5.00
Pro Mini Style clone 3.3v 8mHz
Get the ones with A6 & A7 broken out at the back edge of the board. Just make sure its the 3.3v version because you can’t direct-connect the SD cards to a 5v board.
$2.20
‘Pre-assembled’ Nano V1.O Screw Terminal Expansion Board
by Deek Robot, Keyes, & Gravitech (CHECK: some of them have the GND terminals interconnected)  You will also need to have a few small flat head screw drivers to tighten those terminals down.  Since this shield is was originally designed for an Arduino Nano many of the labels on ST board will not agree with the pins on the ‘analog side’ of the ProMini.
$1.85
DS3231 IIC RTC with 4K AT24C32 EEprom (zs-042)
Some ship with CR2032 batteries already installed.  These will pop if you don’t disable the charging circuit!  
$1.25
CR2032 lithium battery  $0.40
SPI Mini SD card Module for Arduino AVR
Buy the ones with four ‘separate’ pull-up resistors so that you can remove three of them.
$0.50
Sandisk or Nokia Micro SD card 256mb-512mb 
 Test used cards from eBay before putting them in service. Older Nokia 256 & 512mb cards have lower write currents in the 50-75mA range. This is less than half the current draw on most cards 1gb or larger.
$2.00
Small White 170 Tie-Points Prototype Breadboard
These mini breadboards for inside the logger are also available in other colors.
$0.60
Dupont 2.54mm F2F 40wire ribbon cable Without Housing
Cheaper if you get the ones with the black plastic shrouds on the ends. Dupont connector hook-up wires might be expected to add an ohm or two of resistance and carry at most 100mA reliably with their thin 28-30 gauge wires. Poorer quality crimped ends can add significant contact resistance. Each 40-wire cable will let you make at least 2 loggers, and you”l need a couple of  6-pin connector shrouds.
$1.55
2×1.5V AA Battery Batteries Holder w Wire Leads
If you are running an unregulated system on 2 lithium batteries, then you can use a 2x AA battery holder. If you need to keep the regulator in place to stabilize the rail voltage for particularly picky sensors, use alkaline batteries and a 4xAA battery holder.
$0.50
5050 LED module (with built-in limit resistors) 
(Alternatively, you can also use cheaper 5mm diffused LEDs with a  4K7 limit resistor on the GND line – that you solder into place on the LED yourself)  
$0.75
3.3V 5V FT232 Module
  *set the UART module to 3.3v before using it!* and you will also need a few USB 2.0 A Male to Mini B cables. You may need to install drivers from the FTDI website depending on your OS. Get at least 2-3 of these as they do wear out and you can kill them with a short circuit. These boards can only supply about 50mA which can be tricky with sensors that draw more than that.
$2.75
3M Dside Mounting Tape, 22awg silicone wireheader pins, etc… $1.00
You might need some extra tools to get started:                (not included in the total above)
2in1 862D+ Soldering Iron & Hot Air station Combination
a combination unit which you can sometimes find as low as $40 on eBay.
Or you can get the Yihua 936 soldering iron alone for about $25. While the Yihua is a so-so iron, replacement handles and soldering tips cost very little, and that’s very important in a classroom situation where you can count on replacing at least 1-2 tips per student, per course, because they let them run dry till they oxidize and won’t hold solder any more.  Smaller hand-held heat-shrink guns are also available for ~$15.
$50.00
SYB-46 270 breadboards (used ONLY for soldering header pins )
Soldering the header pins on the pro-mini is MUCH easier if you use a scrap breadboard to hold everything in place while you work. I use white plastic breadboards that they only have one power rail on the side since they do not look like regular breadboards &  I write ‘for soldering only’ on them with a black sharpie to make sure I don’t get the heat-damaged boards mixed up with the good ones.
$1.30
SN-01BM Crimp Plier Tool 2.0mm 2.54mm 28-20 AWG Crimper Dupont JST
I use my crimping pliers almost as often as my soldering iron –  usually to add male pins to component lead wires for connection on a breadboard. But making good crimp ends takes some practice.  Once you get the hang of it,  Jumper wires that you make yourself are always better quality than the cheap premade ones.
$16.00
Micro SD TF Flash Memory Card Reader
Get several, as these things are lost easily. My preferred model at the moment is the SanDisk MobileMate SD+ SDDR-103 or 104 which can usually be found on the ‘bay for ~$6.
$1.00
Side Shear Flush Wire Cutters & Precision Wire Stripper AWG 30-20
HAKKO is the brand name I use most often for these, but there are much cheaper versions.
$5-10
Syba SY-ACC65018 Precision Screwdriver Set
A good precision screwdriver set makes it so much easier to work with the screw terminal boards.
$12.00
Donation to Arduino.cc
If you don’t use a ‘real’ Pro Mini from Sparkfun to build your logger, you should at least consider sending a buck or two back to the mother-ship to keep the open source hardware movement going…so more cool stuff like this can happen!
$1.00

When a finished module with 20 sub-components arrives at your doorstep for less than you’d pay for any one of those components individually, then the tradeoff is that you are the quality control.

.. and the required lithium AA batteries are also somewhat expensive, so a realistic estimate is about $25 for each logger before you add sensors & tools.

My advice is to order at least 5-6 of each of the core components (Promini, RTC, SD module, screw terminal board, etc) with the expectation that about 10-20% of any cheap eBay modules will be DOA due to some quality control problem. I build loggers in batches of six, and one unit typically ends up with a bad part somewhere. Having replacement parts on hand is your #1 way to diagnose and fix these issues. I’ve noted over the years that bad parts tend to come “in bunches”, so if you scale up to ordering in quantities of 10’s & 20’s then spread those orders to a few different suppliers so you don’t end up with all your parts coming from the same flakey production run. 

The other thin I can’t stress enough is CLEAN ALL THE PARTS as soon as they arrive. Leftover flux is very hygroscopic, and pretty much guarantees that solder joints will start to corrode the moment your logger gets exposed to any atmospheric moisture. So I usually give everything about 10 minutes in a cheap sonic bath with 90% isopropyl alcohol, rinse with water, and then dry the parts out in front of a strong fan for an hour. Store parts in a sealed container with a dessicant pack till you need them.  I clean parts that can’t take the sonic vibration (like RTC modules, humidity sensors, accelerometers, etc) by hand with cotton swabs. Then I coat everything I can with a layer of MG Chemicals 422B Silicone Conformal Coating and let that dry for a day before assembling the loggers.  One hint that you have moisture issues is that the sensors seem to run fine in the house but start to act weird when you deploy the unit outside.

Used nut containers make excellent “dry storage” once the parts have been cleaned – but any air-tight container will do.

Another insight I can offer is that the quality of a sensor component is often related to the current it draws – if your ‘cheap module’ is pulling much more power than the data sheet for that sensor indicates, then it’s probably a junk part. If the sleep current is on spec, then your part is probably going to work OK. It is much easier to check low current devices with at Ucurrent or a Current Ranger.  Keep in mind that sensors which automatically go into low current sleep modes usually take some time to do so after the bus has gone quiet – so you might need to watch the sleep current for 30 seconds or more before they fall down into their quiescent state. With SD cards, if the throughput drops below the typical write speed for that model with H2testw, then you should just find another card to use with your logger.

I strongly recommend that you build at least two loggers at a time, because that lets you isolate whether problems you run into during testing are code related (which will always affect both machines the same way) or hardware related. (which will usually only affect one of your two units) At any given time I usually have 2-3 units running overnight tests so that I can compare the effect of two different code/hardware changes the next morning.  As a general rule you want to run a new build for at least a week before deploying it to make sure you’ve made it past any ‘infant mortality’, and moved onto the good part of the bathtub curve.


Addendum: Diagnosing Connection Problems

If you loaded the blink sketch to test the Prominin during your initial assembly, you know that part of the logger is working, so issues during the testing stage are due to incomplete connections from the outside to the Promini I/O ports. Before you start your troubleshooting it’s worth remembering that after you’ve built a few of these units you can simply rebuild the whole logger in about a hour. So it really doesn’t make sense to spend much more time than that – especially when a few five-minute part swaps are likely to fix the problem. The  components are only worth a couple of bucks each, and your time is more valuable than that!

This is the connection pattern inside the white breadboard. Take care that you don’t connect the breadboard jumper wires together by accidentally plugging them in to the same row or – the I2C bus will stop working. NEVER CONNECT the GND & 3.3v wires or the short circuit will likely kill your logger, and possibly even the computer they are connected to . . .

If you see only “Scanning I2C….. ” but nothing else appears when running the bus scanner, then it means that the ProMini can not establish communication with the RTC module. The most common cause of this problem is that the white & yellow wires were switched around at one end or the other. It’s also easy to not quite remove enough insulation from the wires to provide a good electrical connection under the screw terminals, so undo those connection and check that the wires were stripped, cleaned & wrapped together ‘clean’ before being put under the  terminals. Those screws need to be clamped down relatively tight on the thin Dupont wires and if you are not careful, you might have accidentally cut away some of the thin copper wires inside the Dupont cable when you stripped the insulation.

Scanner lockup can also happen if one of the I2C devices on the bus is simply not working: usually about 1 in 6 logger builds ends up with some bad component that you have to identify by process of elimination. (These are 99¢ parts from eBay…right?) It only takes a moment to swap in a new RTC board via the black Dupont connector and re-run the scan. If the replacement RTC also does not show up with the I2C scanner then it’s likely that one of the four bus lines does not have a complete connection between the ProMini & the RTC module.

On this unit I measured 1 ohm of resistance on the I2C clock line between the ProMini A5 pin (on top of the board) and the SCL header pin on the RTC module. So this electrical connection path is good. It’s not unusual for each ‘dry’ connection to add 0.5-1 ohm of resistance to a signal path.

To diagnose this: set a multi-meter to measure resistance and put one probe lead on the topmost point of the promini header pins, and the other probe on the corresponding header pin of the RTC module. If there is a continuous electrical connection between the two points then the meter should read one ohm or less. Higher resistances mean that you don’t have a good electrical path between those points even if they look connected:

1) the ground (black) wire should provide a continuous path from the ground pin on the digital side of the Promini board to the GND pin on the RTC module
2) the positive power (red) wire should provide a continuous path from the Promini positive rail pin (the one with the bundle of 4 red wires) to the VCC pin on the RTC
3) A4 (I2C data) near the 328P chip on the Promini must connect all the way through the screw terminal board and through the white Dupont wires to the SDA post on the RTC
4) A5 (I2C clock) beside A4 on the Promini must connect through through the yellow Dupont wire to the SCL header pin on the RTC .

For ‘single-wire’ connections under the screw terminal with those thin dupont wires, strip enough insulation that you can ‘fold-back’ some extra wire to give the terminal more wire to bite onto.

You occasionally get a bad Dupont wire where the silver metal end is not in contact with the  copper wire inside because the crimp ‘wings’ folded over plastic insulation. With a pair of tweezers, you can ‘gently’ lift the little plastic tab on the black shroud holding the female Dupont ends in place, and then replace any single bad wire. Be careful not to break the little black tab or you will have to replace the entire shroud.

Also look at the little jumpers used to bridge the A4>A2 and A5>A3. If you have a ‘cold’ solder join, or an accidental bridge connection to something else, it could stop the bus from working. Remelt each connection point one at a time, holding the iron long enough to make sure the solder melts into a nice ‘liquid flow’ shape for each solder point.

A note about I2C sensors: The I²C bus is slow, so topology (star, daisy-chain, etc.) doesn’t matter much, but capacitance does. Length and number of devices increase capacitance. If you find that the devices work when you switch to a slower speed (e.g. 50 kHz), then this is probably your issue, and you need to minimise bus length and/or maybe decrease the combined resistance of the pull-ups to 2 kΩ or less. The DS3231 RTC module has 4k7 ohm pull-up resistors on the SDA & SCL lines & the Pro Mini adds internal 50k pull ups when the wire library is enabled. Typical I2C sensor modules usually add another 10k so your ‘net pullup resistance’ on the I2C bus wires is usually:  50k // 4k7 // 10k = ~3k. With a 3.3v rail that means the devices draw 3.3v / 3k = 1 mA during communication which is fairly normal ( 3mA is max) for total wire lengths below 1m. It’s  common for pre-packaged sensors to arrive with housings at the end of about 1m of wire. If each sensor also adds another set of 10k pullups, the resistance generally compensates for the extra wire length, so the combination still works OK. But that depends on the cable too. A very bad cable might not even get to 0.5 meters and a very good cable (little capacitance to ground, no crosstalk between the wires) can go up to 6 meters.

The connection diagnosis procedures described above also apply to the connections for the SD adapter board. Sometimes you end up with an adapter that has a defective spring contact inside the SD module, but the only way to figure that out is to swap it with another one.

Here a jumper wire from the Promini pins is by-passing a bad connection.  This is also how you would break out A6 & A7 connections if you need them.

Sometimes those screw terminal boards have a a poor connection inside the black female headers below the Promini. It’s also possible to accidentally over-tighten a terminal and ‘crack’ the solder connection below the board – or there may simply be a cold solder joint on one of the terminal posts. If you have only one bad connection, you can jumper from the Promini header pins on top, down to the other wires under the corresponding screw terminal. If you accidentally strip the threads on a screw terminal, you can use this same approach but move that set of wires over to one of the three ‘unused’ screw terminals at the far end of the board. (beside the SD card adapter) If you’ve gotten through all of the above steps and still have not fixed the problem, then it might be time to simply rebuild the logger with a different screw terminal adapter board.

 

Also note: The environment is written in Java, and the IDE installer comes with its own bundled Java runtime, so there should be no need for an extra Java installation. However we have seen machines in the past which would not compile working code due until Java was updated on those machines; but this problem is rare. More likely some permission limitation meant that you’ve ended up installing the IDE in a directory other than the default, and now it can’t find its libraries.


Addendum 2020-02-02:  Refining the build

The ‘solderless’ classroom build includes a breadboard for flexibility, but if you are building a logger for a single sensor application you can leave out the breadboard, and add some under board jumpers to avoid those finicky multi-wire bundles:

Using the tined leg of a resistor to add some under-board connections to give you more 3.3v & GND attachment points. Clean any residual solder flux thoroughly!

To repurpose those  screw terminals, leave the corresponding pins un-soldered so they can be  removed.

 

Here, we are repurposing the TX, RX, & RST terminals as GND connections. DO not repurpose TX & RX if you need serial comms with NEMA sensors like GPS.

Use a Black marker to label the repurposed GND points. NOTE: The GND-labeled terminals can not be repurposed because they are bridged via the back plane of the screw terminal board itself.

Label the connections with red. NOTE: ONLY join the Vraw & Vcc like this if you have removed the regulator. If you are keeping the reg. in your build then Vraw must remain separate, & you can only repurpose the reset pin for one extra 3.3v rail.

In a no-regulator build, you now have three (+)ive rail connections. This is handy if you accidentally strip one of the terminals by over-tightening.

Adding header pins to the RTC module’s cascade port provides a more convenient attachment point for I2C sensors, and removing the breadboard leaves room for a desiccant pack:

Example of a typical “dedicated sensor” build  without the breadboard, and with the under-board jumpers to provide extra (+)ive & GND terminals. This photo also shows a mosfet controlling the SD ground line, and a thermistor is tucked under the Promini board – but it’s probably better to tackle those additions when you have a few successful logger builds under your belt.

‘Single shroud’ 40-wire 20cm FF Dupont cables are ~$1 each – 1/3 the cost of the ones without housings.

We deliberately put students through the creation of custom sized Dupont connectors because it’s a useful thing to know about, however single shroud wires also work on the RTC and SD module because you are using the double sided tape to hold everything in place. I also tend to use RAW diffused LEDs as indicators, and simply light the individual channels with INPUT_PULLUP mode which uses the internal pullup resistor to limit current. However a single mistaken OUTPUT-HIGH can kill your Arduino if your not careful with that approach. I also leave one of my battery leads long enough that I can tap in to that line & measure sleep currents later on with a Current Ranger.

MCP1702-3302E/TO dead-bug hanging off a Deans style T-plug connector.

The unit pictured above has had its regulator removed so it can run more efficiently on 2x AA lithium batteries. But if you later find out that your sensors need strict voltage regulation, you can always replace the battery holder with a 4xAA, and add a regulator in-line on a battery connector. In the photo (right) two ceramic 105’s stabilize the MCP1700, while a 10/3.3M ohm divider provides a third output line so the ADC can monitor the raw battery status. This is the simplest way to retro-fit a regulator onto a unit that was built without one.

You occasionally get a plano box where the plastic ridges of the housing don’t mate well with the red rubber o-ring. In those cases it’s relatively easy to pull the o-ring out of its groove in the lid, and add some gasket compound to the recessed trough before carefully putting the o-ring back in. The additional height then compensates for casting error in the housings, though don’t add to much or you won’t be able to close the lid. One positive aspect of the relatively loose fit on that lid is that it lets you run prototype tests quickly if you jumper to your sensor module with thin 28-30 gauge wires:

This is a BMP280 pressure sensing module. Wires must extend beyond the edge

~1″ square of foam mounting tape with wires spaced evenly

 

Leave the red backing facing up as you fold the tape & wires over the corner edge.

The front corners of the box exert less pressure than the back

The sharp inner edge of the lid would cut the wires insulation if the tape was not there to protect

The tape has to be replaced every time.

This gives you a chance to do overnight run tests before you commit to modifying the housing with holes, putty, or pass-throughs with cable glands. And for indoor sensor deployments this might be all you actually need to do, though I would still coat the exposed solder joints on that dangling breakout module with either conformal coating or clear nail polish to prevent oxidation.


Addendum 2020-07-29:  Better moisture protection

I was noodling around in the garden recently and installed a few loggers without desiccants because it was only a short experiment. It rained immediately afterward and I noticed a small amount of moisture condensed inside the plano-box housing. While this didn’t prevent the logger from functioning, it completely disrupted the LED light sensors because it provided an alternate discharge path for the reverse bias charge:

Green channel data from a 5mm diffused RGB LED used as light level sensor. This logger was under some leaf cover, so there was considerable variability from the dappled light crossing over the sensor. An arbitrary cutoff of 200,000 was set in the code at low light levels.

After examining the O-ring I decided to add a little silicone to the channel holding the red o-ring to improve the seal:

 

Gently pry the O-ring loose and apply sealant in the groove before replacing.

Bead only needs to be 3-4mm in diameter.

Close the housing & let the sealant set for a few days. The improved seal is especially visible at the corners

If you already have your logger assembled, try to find a silicone sealant that does not off-gas acetic acid (smells like vinegar) which could harm your circuits. If you are simply preparing empty boxes before assembly, then any regular bathroom sealant will do provided you give it about a week to finish curing.


Addendum 2020-05-11:  Using a more advanced processor

After you’ve built a few ProMini based loggers, you might want to try a processor upgrade. The 1284p CPU has twice the speed & 4x the memory, but still delivers comparable sleep current / longevity.

Tutorial: Adding the SSD1306 OLED Screen to an Arduino Logger (without a library)

An SSD1306 OLED screen mounted on a climate station build.

This is the third installment in a series on adding output screens to Cave Pearl Data loggers. It builds on the Nokia 5110 LCD tutorial and the post describing how I store fonts in the Arduino’s internal EEprom to save program memory – so you might want to have a look at those two posts before diving in here.

While the Nokia 5110’s are a cheap, low-power option with great visibility in full sun, they reacted badly to pressure directly on the LCD surface.  Since this project deploys loggers in an underwater environment I went looking for something more robust and the SSD1306 OLED’s caught my attention. (@ ~$3.50 on eBay) These little screens are showing up in hacked toyscompasses, GPSanalog meters, ECG’s, and theres even a tiny oscilloscope project at Hackaday. But those applications typically use the u8g2 library which is fantastic for graphic output, but also quite memory intensive; what I need is a bare bones solution that uses the fewest system resources on a unit that’s already near memory limits. Though these 0.96 inch displays are quite small, 128×64 pixels lets you render several lines of readable text.

While the I2C variants of this screen are easy to use, the SPI version lets me re-purpose unused analog lines to drive this display without interfering with the sensor or SD card bus because it can be driven via shift-out on any pins that are available. Powering the screen from A0 brings the screen down to zero current while the logger is sleeping, and also lets me get rid of the Reset & Cable Select lines. The only thing to watch out for is that you bring all the control lines low when you cut power so that reverse bias doesn’t end up “back powering” the screen controller.

Connecting the OLED:                                {Click any image to see larger versions.}

The data sheet for the SSD1306 controller specifies that the reset input needs to be LOW, during initialization, after which the pin should be HIGH for normal operation. To achieve this low -> high  transition, I tie the Reset line to the middle of an RC bridge made from a 104 (0.1uF) capacitor and a 10k resistor. When you add the 25k inside the 328p used to pull A0 high, you get a 63.2% time constant of about 3.5ms.

Begin by tinning the pins, and bending them 90 degrees for alignment with the RC bridge which also connects to the CS pin.  Once that delayed rise circuit is soldered in place, also jumper the incoming GND line over to Cable Select. You would not do this if the display was on the normal hardware SPI bus, because it would exclude other devices on the SPI bus. But since we are using a separate set of wires for the display we can get away with this trick.

The connections for the SSD1306 OLED display are the same as the ones for the Nokia 5110: (D0=clock, DC=command, D1=data in)  Note that I’m not using the 3.3v rail from the pro-mini,  and YOU MUST BE USING A 3.3V ARDUINO FOR THIS WIRING TO WORK UNLESS YOUR OLED BOARD IS 5V TOLLERANT.  Some of the breakouts  from eBay are OK at 5v, and some are not – check THIS before you buy if you are using a 5V Arduino.  (Note that  most breakouts sold by Adafruit include regs & level shifters to handle both voltages.)

When all the incoming lines have been added to the board, thread them through the housing and verify that the screen is still working. The analog A0-A5 lines on an Arduino can be re-purposed as digital I/O so I usually break them out with a 6-pin Deans Micro Plug (incl. GND & Vcc)  In this case the screens Vcc line is being re-routed to pin-powering from A0 so the pro-mini’s rail voltage is not used.  Cut away or heat-shrink over the unused Vcc pass-through on the screen side of the connector (which is still visible & exposed in the photo on the right) as you don’t want it accidentally contacting something later on inside the housing.

After the operation test, cork the pass-through hole with a bead of plumbers epoxy putty  about the size of your fingertip. Wrap that putty so that it is on both sides of the wires before smoothing it down on the housing.  (The sensor cap shown below has four housing penetrations sealed by this method)  On the top of the sensor cap, wrap a grape-sized lump of well worked putty around the wires on the back of the screen. Then carefully press the screen down into the PVC well so that the putty compresses into a support pillar that holds the screen as near level as possible. The putty hardens in about 10 minutes and then you can pour the first layer of potting epoxy. Here I’ve used Loctite HYSOL E-60NC, with a 50ml applicator & MA6.3-21s mixing nozzles. The air space for the reflective back light forced me to use clear epoxy with the 5110 LCD screens, but since these OLED’s emit their own light, you can bring that black epoxy right to the edge of the display – covering the board & any ugly soldering… 🙂  Check the epoxy over the next couple of hours and pop any bubbles that rise to the surface with a pin. (unless you have a vacuum chamber)

Always TEST before potting!

Let that initial layer of epoxy cure for 24 hours, then drill a 7/32″ hole at a safe distance away from the screens edge. Mount an RGB indicator led inside the housing and seal it in place with plumbers putty. At this point you can simply add a second layer of clear epoxy to protect the screen, but in our experiments with the Nokia LCDs, seawater caused a serious fogging problem.  So I recommend a top surface of more chemically resistant material for that kind of application. Here I’ve used a 1/4″ thick plexiglass disk (~$0.35 each on eBay)
First apply a thin layer of clear epoxy over the screens surface so that it self-levels to thickness of about 1-2mm. Then hold the edge of the acrylic disk and carefully tilt it over the epoxy film with an even contact edge that moves across the disk slowly enough to let the air escape.  Don’t make that first layer of clear epoxy too thick or it will over-top the edge when the plexi settles into place and this will form blobs on the surface. Once the acrylic is in place, you can slowly add drops of epoxy on the side to bring the level up to match the edge.  Give the clear epoxy another 24 hours to cure.

The display draws more current as you increase the number of pixels turned on.  As a point of reference: the screen output shown above (generated from the code on Github ) draws 3.4 mA.  And I haven’t seen any nasty current surges that might exceed the pin limit when the display is first powered up.

The shore hardness of the plexiglass is not much higher than cured epoxy, but the optical clarity is considerably better. The photos in this post don’t capture it well, but this mounting method really shows off the razor-sharp output you get from an OLED, and it’s readable from just about any angle.  For surface loggers a 1/8″ disk should provide sufficient protection, but for the underwater units I will use 1/4″ thickness to resist the compression at depth. I won’t be able to test those underwater versions in the real world for a few months, but if the acrylic fails I will change to  a stiffer glass overlay.

A few might wonder why I added an indicator LED when the logger now has full display capability. There are plenty of situations where brief LED pips communicate the progression of duty cycle events that are too fast and too numerous for screen output. I also use the LED as a noise generator when I want to squeeze higher resolution from the ADC through oversampling.  These days I add an NTC thermistor to every build to capture ambient temperature.

The Code Example on Github uses a cascading method to break text strings up for single character rendering with : oledWriteString–>oledWriteCharacter–>oledWriteData  and I’ve co-opted that with a new split-character method to print larger numbers to the screen in a two-pass method.  The shift-out code for those functions is basically the same as that used for the Nokia 5110. I went over all that in some detail in the earlier posts so I wont re-fry those beans here. The key thing to keep in mind is that this new screen driving script uses EEPROM.read which assumes you’ve already loaded the font definition(s) into your Arduinos internal eeprom with this helper utility.  That utility loads a ‘reduced’ font set to leave 500 bytes of space for file header data in the EEprom as well. (so the 5×7 font is caps only, and large characters are #’s only).  If you don’t need that level of memory optimization in your build, you can switch back to simply storing the screen fonts in program memory;  just transfer the PROGMEM based string->character->data functions (and the font arrays) over from in the NOKIA 5110 code example.

The housekeeping functions that take care of  initializing the screen, clearing the memory, and setting the XY position are more complex for the OLED than those for the PCD8544 Nokia because the SSD1306 controller has more operating parameters.  But the overall approach transferred easily between the two controllers and I suspect this method could be re-used with most of the larger SPI OLED screens on the market (like the SH1106) provided they stay below the pin current limit on A0.

Time to go shopping!  🙂

Addendum 20181019:

A helpful commenter has informed me that it’s possible to try a similar approach with I2C screens because you can bit-bang I2C devices over any two spare wires.  Now that we’ve pulled off this isolation trick with SPI, I’ll look into bang’n as a way to shut down high current I2C sensors when they are not in use.  I’d want those off the regular I2C bus because it necessarily has a few “always on” devices that might not respond well to a lump of dead wood hanging off the same wires.

Addendum 20190423:

A year long experiment in OLED burn-in. Significant dimming of the most used pixels after 4 weeks, but the screens were still operational after one year.

Starting Points & Ideas for your Arduino STEM Curriculum

Arduino Starter Kits can be assembled from $10 to $100 per seat depending on the complexity of your course. We generally order parts about a six weeks beforehand and then spend a day pulling it all together into “ready to go” lab kits at the start of term.

Over the last decade the open source Arduino platform has been embraced by STEM educators, and there are a growing number of pay-per-use resources available with pre-made lesson plans, etc. (eg: becauselearning.com While most welcome viable business models in the sector, it struggles against the problem of ever shrinking education budgets. Where the rubber meets the road you are faced with the stark reality that many teachers now have to pay for teaching materials with their own money:

“Classroom teachers spent an average of $468 out of pocket on classroom supplies and equipment in the last year — amounting to nearly 1 percent of an average teacher’s salary in the United States. Nearly eight in 10 teachers — 77 percent — spent “at least” $200, with some as high as $5,000, according to the latest results of an annual survey.”

Teachers also have to teach themselves first, which is always a hard sell to program admins.  With those things in mind, I’ve started this list of links to STEM learning resources, curriculum, and activities for Arduino.  Unlike more formalized lists of this type, the focus here will be on creative IDEAS and resources that teachers can access for FREE ( eg: WeTeachNYC’s Gr9 lesson plans , textbooks online, What’s Going On in This Graph?, fizzixprof, NOAA, etc.

This page will grow over time as I find more material.


Arduino project IDEAS:

Any teacher worth their salt already knows how to make lesson plans, so the tough part is finding a theme that really motivates your students.  If you are looking for science project ideas, it wouldn’t hurt to browse through a few commercial data logger websites sites to see how people use loggers in the real world. (esp: Fondriest Environmental Learning Center ) Then search through the Arduino sensors forum and see if someone has already posted helpful information about the application your students find interesting.  Though the Cave Pearl Project is focused on environmental monitoring, you shouldn’t overlook the other cool things that people do with Arduinos for more information on integrating sensors (eg: building instruments like the TC1 slinky seismometer) Browsing through the Arduino project hub gives you some sense of the range.  A good number of artists create interactive pieces by adding motion, sensing, LEDs & sound. Wear-able projects are also pretty groovy and you are never too old for some fun & gamesOthers create simple robots with their Arduinos, and there are plenty of body/wheel/motor kits to get you rolling. Drones get all the media attention, but I think underwater ROV’s are also interesting.

There are lots of great maker resources to search through if can appreciate their sense of humor (though you might want to avoid clock projects 🙂  Intructables is heaving with Arduino projects which you can find simply by searching for “Arduino” + “sensor”, which will often take you to the larger sites like the Arduino Project hub. Sparkfun has moved their Inventor Kit tutorials over to a teachers resource platform called Workbench, which has recently been acquired by Google Classroom. To give you a sense of that scale, here’s a sampling of just their LED tutorials

If you find an Arduino book that sounds interesting, there is a good chance that there are sample projects on the web from the book that you can review.  GPS tracking opens up interesting possibilities and the folks over at the RIFFLE project have been pulling that location data out of digital camera photos, with their data logger hanging from a kite.  So really, the sky is the limit . . . or maybe not even that . . . commander Sparkles


Most ” Discovering Arduino ” resources follow a pattern something like this:

Introduction of the Arduino board. (hardware)
Introduction of the Arduino programming environment and the structure of a script. (software)
Introduction of the breadboard. (hardware)
Blinking the internal LED at pin 13. (software)
Connecting a LED to Arduino using a breadboard. (hardware)
Using a digital output pin to blink the LED. Using multiple digital output pins & leds (software)
Pulse-width modulation for fading LEDs. (software)
Connecting a button to the Arduino, with de-bouncing cap/resistor combination. (hardware)
Programming  to support the button input , if/then/else conditional behavior. (software)
Introducing the serial monitor for text output of events (software)
Connecting analog sensors to the Arduino with voltage dividers. (hardware)
Capturing sensor readings with the ADC and storing them in a variable. (software)
Using the serial plotter for ‘live’ display of sensor data (software)
Advanced programming concepts (e.g., for/while loops, counters, switch/case). (software)
Connecting digital bus sensors with pull-up resistors (usually I2C or OneWire ) (hardware)
Including code libraries to so you can read data from those digital sensors (software)

As you can see, learning the Arduino platform is like climbing a ladder, where each step you take toward understanding the hardware is matched by one learning how to write code.

Where To begin:
The instructables beginners guide is a good place to start, as is Udemy’s free Learn the Basics Arduino Tutorial. Actually instructables has been busy building a range of free beginners classes on subjects from the internet of things to 3D printingetc.  There are plenty of other “Getting started” videos available with  another free video course offered at the Programming Electronics Academy (also see their other Youtube videos). Many of these courses require some kind of registration, and given the nature of their business you can expect a fair amount of self promotion messages to be peppered throughout. And finally, don’t overlook the official Arduino example tutorials that come built into the IDE. There are some great learning examples in there like the Tone Pitch follower with tutorials by Massimo Banzi himself.

Special Mention:
Be sure to check out Jeremy Blum’s Arduino Tutorials which are essentially a complete course on the Arduino;  all the more impressive because he did the entire thing as a one-man-band while he was still a student. And he’s not alone, Schuyler St. Leger’s Arduino 101 is notable as perhaps the youngest student-made tutorial.   Jeff Feddersen & Tom Igoe have produced some of the the best quality videos available for Arduino for the ITP program at NYU. Jeff & Tom have been producing these videos for a good while now so there’s a lot of great material. If you are just getting started with Arduino, an excellent sub-set might be the Digital & Analog series, followed by their Sensors videos, and then their set on Serial Communication.  Paul McWhorter also has an extensive tutorial series for beginners, though I find his stuff pretty slow going.

And don’t forget to search for the many other tutorial videos that people have posted. Youtube has grown into a universal self-teaching tool and we’ve entered the game with clips from our fieldwork, and step-by step data logger build tutorials.

Arduino in a Nutshell is a free e-book resource worth looking into, as is the Programming Guide from instesre.org. And though I’m not sure if they are still a going concern, the old Earthshine Starter kit manual PDF can still be found floating around the internet. If e-books like that are your thing, and you are willing to shell out a few bucks, there are sometimes good Humblebundle deals, though those are often in weird combinations of topics, and the individual books also available the Make website.

Sparkfun is also a great place to look for teacher resources.

It’s a lot to wade through, but the Adafruit tutorial list  is another one of the best resources out there. Just be aware that they have developed their own library “system”, so sometimes their tutorials are tailored to that.

Tronixstuff has a large number of  specific hardware tutorials when you are ready to go further with your Arduino projects, and there are a host of cool Arduino projects to dig through at instructables site. I really believe that you can improve engagement and understanding by providing hands-on experience with real data, but there are plenty of other practical things you can do with the same basic setup.

If you google around, you can find curriculum documents, individual lesson plans, and other resources all over the place, like for example this conductivity lab over at teachengineering.org or this beginners set from Arduino 101.  The challenge is that most of the sites were developed for a different curriculum than yours, so first figure out what you want to tackle, then go sifting through the tutorial sites for material that matches your learning outcomes. Otherwise you will just get buried in the shear volume of it all.

If you want to abstract away the entire IDE interface for younger students, there are a few visual programming tools out there for the Arduino like Visuino, or MIT’s Scratch, for which there are plenty of tutorials on youtube.

Going further:
Once you know your way around the Arduino, Nick Gammon’s Microcontroller Forum is an invaluable resource.  There is also a good set of technical videos at Makecourse.com and on the Forcetronics channel. Though it’s a bit dry, All About Circuits has a complete electronics textbook online [see: Vol. I – Direct Current (DC) ] And if you really want to dig deep, several universities like Stanford, MIT & Berkeley have made full electronics courses available, though that goes well beyond the Arduino landscape. There is a good walk through the UNO sub-components at Rheingold Heavy’s ‘Build an Arduino From Scratch’ series, and Bruce Land’s AVR lectures take you right down to the “bare iron”.  And finally, I’d be remiss if I didn’t mention the excellent Mini-notebook series that that Forest Mimms wrote back in the 70’s. 


YouTubers on Science & Technology:

Like the other flavors of social media, YouTube can give a boost to your STEM lessons, provided you don’t go down that rabbit hole until after you already have clear lesson outcomes in mind.  It’s hard to pick a favorite, but Ben Krasnow @ Applied Science might take the title because he does incredible things without the over-the-top wow-yuck factor that the media seems to feel is the only way to make science interesting.  When Ben wants an electron microscope – he builds one.  Super Conductor? ditto. Plasma tube? easy-peasy.

Life, the universe, & everything:
Veritasium – An element of truth – videos about science, education, and anything else
Kurzgesagt – In a Nutshell – finding a new way to end the human race with every video
SciShow – delves into popular scientific subjects with lots of flash for younger audiences
Physics Girl – Physics videos for every atom and eve
Vsauce – Michael Stevens combines discussions of science and philosophy
BrainCraft – Vanessa Hill explains why we humans act the way that we do
ASAPscience  -explains topics in science with their trademark kinetic typography and drawings
MinutePhysics – physicist Henry Reich explains physics concepts simply  in a few minutes
MinuteEarth – stories about our awesome planet & the critters that live there
TedX Talks
It’s Okay To Be Smart
The Royal Institution – bringing science to the masses for over 200 years
Crash Course – history, science, physics, philosophy, and a bit of everything else
RSA animate
Nature video
Steve Mould

People who put things together:
Kevin Darrah – electronics w Arduino
Julian Ilett – the Bob Ross of Arduino makers
Andreas Spiess – he basically tests every sensor on the DIY market
Electronic Basics – by GreatScott
Adam Savage’s Tested – of Mythbusters fame
Peter Sripol – if it floats or flies, he will build one. FliteTest also worth a mention.
The Thought Emporium
NightHawkInLight
James Bruton

People who take things apart:
bigclivedotcom – random take apart videos from expensive toys to cheap junk from China
the Brain Scoop – warning: the Gross-o-meter ratings on those dissection videos are legit

People who explain how things work:
Engineerguy
Real Engineering
Practical Engineering
The Story of Stuff Project – animations explaining how ‘stuff’ affects our society
3Blue1Brown:  Brilliant animated introductions to complex math subjects
Facts in Motion
Numberphile – might give you an ‘irrational’ attraction to maths
PhET – interactive physics simulations
Stated Clearly – genetics & evolution

The Do-ers:
Smarter Every Day – engineer Destin Sandlin doing fascinating experiments
Mark Rober
Cody’sLab
The Slow Mo Guys
Mike Boyd
The Action Lab
Tom Scott

The Story Tellers:
Curious Droid (Paul Shillito)
LindyBeige
Knowing Better
The Good Stuff
TED-ed Animations
Today I Found Out
Nerdwriter – essays about movies, art & culture
Sideways – analysis of movie soundtracks
The Mechanical Universe – A classic older physics video series from Caltech
The Day the Universe Changed – hard to find, but Burke’s 1st series was by far his best
Cosmos – Tyson’s reboot works better for younger audiences. Original is great too.

The incomparable Colinfurze.  Close tie for the #1 spot because Colin is kind of like Ben Krasnow’s unhinged alter-ego from some parallel dimension where human beings have very short lifespans.  Like ArduinovsEvil , Plasma Channel, Nile Red, and the site-which-must-not-be-named, this highly inappropriate material is best viewed at 1am when you’ve already killed off any high-function brain cells with a 10-hour exam marking marathon. (ie: Don’t show Colin’s videos to impressionable young minds unless you want to be fired, and never show content from AvE, and IFLS without serious vetting)


Podcasts & Audiobooks:
Radiolab (stories & science)Quirks & Quarks (science news)The Science Show (science news)Freakonomics Radio (econ) , More or Less (stats)The Infinite Monkey Cage (science)The Age of PersuasionUnder the Influence (advertising’s effect on culture)EconTalkOpen Cultures science podcast list ,

LeVar Burton Reads
Free Audiobooks  (streams from Audible.com)
Free Audiobooks  (download to player from Librivox.org)
Free eBooks          (Project Gutenberg)


Using Social Media to find resources:
Like Pinterest, Reddit has grown into one of the most useful social media sites for leveraging other peoples knowledge to help you find useful resources.  I’ve compiled a short list of places that might be good starting points.  You can also find material with the right hashtags on twitter such as #DIYscience.  But social media like Twitter/Facebook/etc can easily waste as much of your time as it saves, so finding good material is directly related to how particular you are about following people who actually contribute resources to the community (as opposed to those who are merely talking about it)

A few Teacher & Maker sub-reddits:

Teaching & Education
/r/ScienceTeachers/ – Chemistry, Biology, Physics, Astronomy, and General Science for K-12
r/matheducation
r/education -news articles about America’s education system, from Pre-K through PhD
r/teaching

Electronics:
/r/arduino – Arduino and compatibles. Lots of projects posted, discussion leans more towards programming with dabbling in circuit design

/r/electronics – About electronic circuit design and occasionally embedded systems
/r/raspberry_pi – Discussion about the Raspberry Pi & /r/raspberrypi
/r/embedded – Similar to /r/arduino and /r/raspberrypi but not platform specific
/r/3Dprinting – 3D printers

Others that instructors may find interesting:
/r/diy – The granddaddy of them all. Largely focused on home improvement but has content from just about everything.
/r/crafts – Sewing, knitting, scrapbooking, kids crafts, etc. and /r/craftit – Smaller version of /r/crafts
/r/somethingimade – Largely arts & crafts related, occasional woodworking or self made website posted
/r/maker – For people who make things. Doesn’t seem to be very active.
/r/woodworking – 50/50 between using power tools and hand tools. Wide variety of projects posted from simple to “they must be professionals”
/r/metalworking – Stuff you can do with metal
/r/welding – Welders, machinists and all other enthusiasts of joining two things together
/r/outstruments – Musical instrument making
/r/lego – Lego is made of ABS plastic and you can use a tiny dab of ABS plumbing solvent (nasty stuff!) to weld it together into a custom bullet-proof housing for your Arduino project. Great for internal scaffolding too.  also see /r/AFOL – Custom designed LEGO creations.


Other inspiring links:

What’s the Maker Movement and Why Should I Care?

The Maker Movement in K-12 Education: A Guide to Emerging Research

Progressive Education and The Maker Movement

TED Talk – Massimo Banzi (the primary founder of Arduino) – How Arduino is Open-Sourcing Imagination and you can hear more from him at Arduino: The Documentary

Hans Rosling as an advocate for a “fact-based worldview” with his amazing bubble charts.

Open Culturea VAST compendium of free cultural & educational media on the web

Tutorial: Adding Sensors ( & Modules ) to Your Arduino Data Logger

This in-cave micro-climate recorder had pressure & temperature sensors mounted in little wells of Loctite E30-CL epoxy. This sensor potting method is described in our Pro Mini build tutorials.  Weather sensing stations are the most popular type of Arduino-based Sensor project on the instructables.com website.

This post isn’t another How-To tutorial for a specific sensor because the Arduino community has already produced a considerable number of resources like that You’d be hard pressed to find any sensor in the DIY market that doesn’t give you a dozen cookbook recipes to follow after a simple Google search. In fact, you get so many results from “How to use SensorX with Arduino” that beginners are overwhelmed because few of those tutorials help people decide which type of sensor suits their skill level. This post attempts to put the range of different options you can use with a Cave Pearl data logger into a conceptual framework, with links to examples that illustrate the ideas in text.

One thing to note before you start is that many modern sensors will only accept 3.3v inputs, so UNO based projects need to check if the sensor they want to use is 5v tolerant. Most sensors from vendors like Adafruit put regulators on their breakout boards to handle this 3.3v-5v translation, but you may have to place level shifters between some of the more advanced digital sensors and an UNO based logger. Occasionally you run into the opposite situation where the sensor requires 5v (or more) forcing Pro Mini based systems to do the same thing.

Also note: The first four videos from Tom & Jeff’s  sensor series (  Survey1&2, Datasheets & Interfacing ) make an excellent accompaniment to the material here.


Analog Sensors:

Some substances react to energy input by changing their physical or electrical properties. Arduinos can only read voltages, so to record these changes in the physical world some kind of circuit is needed to convert those properties into a voltage. Sensors that output continuously varying voltages in response to natural phenomenon are called analog sensors. Arduino pins A0 to A7 are analog input pins, and the ADC inside the microprocessor converts those voltages into a numerical value between 0 and 1023. It’s worth keeping in mind that those numbers are somewhat arbitrary depending on the reference voltage, and the behavior of your sensing circuit – so it’s up to you to figure out how to convert the raw ADC readings back into understandable units of the phenomenon you were trying to measure (like degrees Celsius for temperature, or m/second of wind speed, etc.)

A typical photoresistor divider from Sparkfun’s Voltage Divider Tutorial It’s worth noting that many LDRs go from 100 Ohm to over 1 MOhm. For a specific range of conditions, a common rule of thumb is: R-fixed = square root of (R-sensor min × R-sensor max)

The most common analog sensors are those that change their resistance in response to temperature (thermistors), light (photo resistors) or pressure (& it’s force / stretch / bend variants)  If a sensor varies in resistance, you can turn that into a voltage by adding a fixed resistor to create a voltage divider circuit.  The non-sensing resistor in the divider is usually chosen with a value near the midpoint of the sensing devices range.  For example, a photoresistor might vary between 1kΩ in the light and about 10kΩ in the dark, so a suitable resistor to pair it with would be ~5kΩ.  For analog sensors that change by really small amounts, more sensitive Wheatstone bridge arrangements combine 2 or 4 sensors in the same circuit to expand the delta, but it’s the same basic idea: you are converting a change in resistance into a change in voltage.

Divider methods are referred to as ratiometric because the output voltage from the circuit is some fraction of the supply voltage determined by the resistances of the components. If the input voltage is doubled, the output voltage is doubled, so these circuits work fine on 5v UNO and on a 3.3v Pro Mini. By default the Arduino ADC takes a reading by comparing it to the same rail voltage supplying the resistive divider, and sensor nerds like me get all excited about this because it means that variation in your power supply will have little effect on the readings. You can squeeze more sensitivity per bit out of the Arduino’s humble 10-bit ADC by changing to a lower internal reference voltage. However once the Aref is different from your supply, rail noise shows up on the divider output unless you squelch it out with a smoothing capacitor. (related method:  Capacitance Meter ) If your resistive sensor varies over a wide range, another common technique to increase sensitivity is to use a series string of resistors, and taking each junction to ground to  change the ratio of the voltage divider dynamically.

Some light sensors get used in conjunction with a emitter for reflectance and ranging applications. You can create a reasonably good color sensor by combining an RGB LED emitter with a simple photoresistor. And many red, yellow and green LEDs have a little known property: with reverse bias, they behave like a photo diode where the reverse current is proportional to the external light falling on them. So they can also be used to sense light because from a physics point of view, a rectifier diode, a light emitting diode and a photo-diode are basically the same device. While LEDs don’t produce the same signal strength as photodiodes designed for the job, they can be an inexpensive way to build spectrally selective detectors.  LEDs detect a much narrower band of light than they emit, having a peak sensitivity at a wavelength about 50nm shorter than their peak output wavelength.  Forrest Mims built some cool filterless photometers with LED sensors long before the mainstream media started waxing philosophical about ‘citizen science’.  (Also see: jpiat’s Li-Fi )

Most analog sensors are simple devices, but there are more complicated versions providing modified analog output, where some extra circuitry has been added to convert the highly non-linear response you get from typical resistance based sensors into the kind of straight y=m(x) relationship you get from a TMP36. This greatly simplifies the math required to convert your analog voltage readings into the real-world property you were actually trying to measure. Some analog sensors (like thermocouples) generate tiny voltages, but those signals may be so small that they need to be amplified before the Arduinos ADC can read them, so these analog sensors may also be sold with supporting electronic boards to boost the output.

Sensors can be mounted  inside a housing with a couple of layers of 3M Scotch Outdoor Mounting Tape. Sensors mounted this way have stayed in place for many years of deployment. The adhesive will sag somewhat under gravity if you expose your loggers to temps >55C.

At the top of the analog sensor food chain, there are complex Micro Electro Mechanical (MEMS) devices like accelerometers. In these sensors, silicon has been machined into very tiny physical devices made from springs, coils and flat sheets. These micro-cantilevers form capacitors that react to movement by changing a voltage and they are usually arranged in sets of three on x,y,&z axes. This means you need to read three separate input pins to capture a complete reading from the device. Since the Cave Pearl data loggers use pins A4 & A5 for communications with the RTC module, and A0 to track the main battery voltage, a complex analog sensor like the ADXL335 can use up all of the remaining analog inputs on the logger unless you build it with an Arduino that makes inputs A6 & A7 available. (the Pro Mini does, the UNO does not)  The limited number of analog input pins can motivate people to switch over to digital-bus sensors, though multiplexers provide another possible solution to the problem.

If you start with the project’s basic UNO logger script , adding a new analog sensor requires only three lines of code. Add

int AnalogSensorReading = analogRead(A0);  
// change A0 to match the input pin you connect the divider to

at the top of the main loop. Then add that new sensor data to the concatenated dataString which is saved to the SD card at the end of the main loop:

dataString += ", "; //comma separates new data from that already in the string
dataString = dataString + String(AnalogSensorReading);

That’s it. This simplicity is why analog sensors are usually the first ones people encounter when they are learning the ropes. Of course there are some advanced tricks you can play to supercharge Arduino’s humble 10-bit ADC, and you’ll find more useful tips over at Nick Gammon’s ADC tutorial.

Digital Sensors:

A bullet-proof de-bouncer from www.ganssle.com.    Compare this to the 5-key de-bouncing circuit from the IBM 705

Unlike analog sensors, digital sensors only output two voltages: High & Low. Usually the high voltage is the same as your power rail, and the low voltage is your system ground. In some ways that makes digital sensors easier to use, but there are some devils hiding in the details, and digital sensors cover the entire range from crude noob level devices to Gordian knots with more computational horsepower than the Arduino itself. Even the most complicated digital sensor usually has an analog sensor hiding somewhere at its core.

I group digital sensors into three conceptual categories:

Flippers,   Thumpers,  &   Thinkers

This is based on what kind of output they produce, rather than the complexity of their electronic circuits. And it’s not unusual for more advanced IC-based digital sensors to be as easy to use as the flippers & thumpers, because some kind soul has released a library that takes care of the gnarly low-level details.

1) Flippers

The humble push-button can be thought of as a crude pressure sensor that can be in only two states: open or closed. Add a couple of passive components for debouncing, and reed switches become the digital sensor of choice for event counting applications like the tipping-bucket rain gauges you find in weather stations.  IR break-beam switches are another common implementation with on/off output.

When you first look into digital sensors there seems to be a bewildering array of different ‘breakout boards’ and ‘sensor modules’ for the Arduino. These are often sold in bundles of twenty, thirty or even sixty different pieces. Once you get a closer look at them, you notice that many these cheap sensor modules look similar to each other:

That’s because most of those boards are simply a voltage divider connected to one leg of a five cent comparator circuit, with a twenty cent trimming pot setting the voltage on the comparatos other input:

These boards switch their high/low output when the sensor circuit voltage crosses the threshold set by the trimpot divider; and this changes the analog sensor voltage divider output into an environmentally responsive threshold alarm. It’s such a generic circuit, and you could connect other resistive sensor dividers and the board wouldn’t even notice. If you use these modules with the Cave Pearl loggers, look for boards that also break out that 4th analog pin so you can also read the sensor dividers output directly with the ADC if you want to.

Integrating simple on/off digital sensors to your logger code would use almost the same pattern as the analog sensor reading:

pinMode(PinNumber, INPUT);  // Declaring the pin as an digital Input
int DigitalSensorReading = digitalRead (PinNumber);

dataString += ", ";         //comma that separates new data
dataString = dataString + String(DigitalSensorReading);

These PIR sensors are usually 5v, but you can modify them for 3.3v operation.

Those eBay boards are somewhat redundant, since the Arduino has a built-in analog comparator that can do this job with pins D6 (reference) & D7 (sensor divider) .  However there are many high/low output sensors with more complicated circuits that are not as easily replicated.  Proximity sensors can have complex internal circuitry and perhaps the most common of these more-advanced-but-still-simple sensors would be the passive infrared (PIR) motion sensors that seem to occupy every corner of the modern world.  Adafruit has a fantastic tutorial on how to use them with an Arduino, which also demonstrates how the boolean HIGH or LOW value you get back from digitalRead() can be used with if statements to select different courses of action:

Reading = digitalRead (PinNumber);

if(Reading == HIGH)
  {   Serial.println("input is HIGH");   }

if(Reading == LOW)
  {   Serial.println("input is LOW");   }

All the I/O pins on an Arduino can be used as digital inputs  (including the analog lines) and the cool thing about that is the circuitry hidden behind those pins inside the microprocessor. The  Schmitt trigger on each pin has read-high vs read-low threshold voltages. This lets you replicate what those cheap eBay modules do by replacing the fixed resistor in your analog voltage divider with variable one, and then connecting the output of that divider to a digital input pin. Inferring resistance (or capacitance) by timing threshold crossovers on a digital I/O pin lets you make high resolution analog readings with digital pins because micro-controllers count time far more precisely than ADC’s measure voltage.

2) Thumpers:

Flippers change state slowly by microcontroller standards, and since they can be read with a single digitalRead() command, they won’t get you much cred at the local hacker-space. To get into the digital world’s caffeine-driven middle class you have to start working with Thumpers. These are sensors which convey information by varying the amount of time the sensor outputs a high voltage at a given frequency (called pulse-width modulation or PWM) OR by changing their output frequency with a fixed 50/50 split between on&off time (this is called frequency modulation or FM) .

This kind of output was common long before the Arduino existed because putting an analog sensor into the oscillator circuit feeding a 555 timer chip changes the pulses coming out the other end in proportion to the sensors resistance / capacitance / etc. You’d be hard pressed to find any environmental sensor that can’t be constructed with a couple of op-amps and a 555 (See: the conductivity sensing post for examples).

Three common methods for reading pulsed signals with an Arduino are:

  1. The pulseIn() Function
  2. External Interrupts
  3. Pin Change Interrupts

The output of the pulseIn() function is the time in microseconds that it took for the pin to go (or be) LOW, then go HIGH, then go LOW. This is the method of choice for PWM thumpers, and it is extremely easy to use provided the incoming signal is a clean square wave.
Unfortunately, it does not handle frequency modulation very well at the high end, because it’s susceptible to errors in timing when detecting the start and end of really short pulses.

Note: the RCWL-1601 is the 3.3v version of this board which is code compatible with the 5v HC-SR04. The 3.3v compatible board is also sold as the HSR04-P.

Range finding sensors often output PWM signals, and the most popular of those is the HC-SR04 which is used for collision avoidance by just about every Arduino-based robot on the planet. Self-balancing robots are one of the maker movements “killer apps”, and it doesn’t hurt that the SR04 transceivers just happen to look like a pair of eyes. There’s currently a bit of a turf war between the SR04 and slightly more expensive IR rangefinders. Ultrasonic energy is absorbed by soft materials, and SR04’s are susceptible to interference & multi-path issues in environments where there are lots of flat rigid surfaces. Infrared sensors have a much more focused beam so you get better results finding small objects…like the other robot you’re currently doing battle with.  (For more precise work you can go upscale to the VL53L0X Time of Flight distance sensors, and if money is no object, you can take that all the way to LIDAR.  To get the highest level range-finding merit badge of them all: MaxBotix Sonar sensors let you play the game under water…)

A code-side implementation for the HC-SR04 could be as simple as this:

digitalWrite(triggerPin, HIGH);       // send out (transmit) the pings
delayMicroseconds(10);                // give the sensor 10 ms to settle
digitalWrite(triggerPin, LOW);        // stop the outgoing pings
duration = pulseIn(echoPin, HIGH);    // listen for the echo and return time. 
Distance2reflectingSurface = (duration/2) / 29.1;
// Divide by 2 since the sound ping travels out & back = twice the distance
// Speed of sound in air = 340 meters per second or ~29 microseconds per centimeter
// Then divide the duration by 29 cm = distance in centimeters.

External interrupts handle both PWM and FM efficiently with the limitation that there are only two hardware interrupt lines on a typical Arduino. The Cave Pearl loggers are already using D2 for the RTC wakeup alarms, and that leaves only D3 available for hardware methods calling attachInterrupt().

It’s worth noting that there is also a near IR (940nm) sibling in the TSL family: the TSL245 though with IR sensors it can be tricky to extract temperature dependencies.

The TSL235R light-to-frequency sensor outputs a square wave (50% duty cycle) with a frequency proportional to light intensity. The TSL235 is self-contained, well calibrated, and very linear over the ultraviolet-to-visible light range of 320 nm to 700 nm. Calibration in manufacturing is something that most companies will try to avoid, and when you include the fact that this sensor works from 2.7-5.5v, you have a $3 sensor that’s nearly perfect for use with Arduino-based data loggers. Rob Tillaart has posted a simple bit of code that counts the interrupt pulses per second from this FM sensor over at the Arduino playground. It should be easy to integrate these functions into the Cave Pearl base code, and to modify it to work with any other FM output sensor. Data from light sensors usually requires post processing with somewhat complicated luminous efficiency calculations, but if you Google around you’ll find plenty of Arduino tutorials on those steps (also see: Insolation Models).

Only D2 & D3 support external interrupt signals by default, but with a little bit of extra code interrupt signals can be received on any of the Arduinos I/O pins.  Interrupts triggered from pins other than D2 & D3 are referred to as Pin Change Interrupts. Pin change interrupts are grouped into 3 ‘ports’ on the MCU. This means there are only 3 interrupt subroutines to handle input from all 20 pins. This makes the code somewhat more complicated than Rob Tillarts example, as it now needs to determine which pin triggered the ISR. That extra complication usually motivates people to use something like the PinChangeInt library for situations with a limited number of input pins.  Anemometers often use interrupt-based approaches because they work with output that’s so variable that it can’t really be classified as PWM or FM.

There are many great frequency counting libraries but it’s important to note the difference between ones which count the number of pulses during a fixed “gate interval” time, and those measuring the period of a single high frequency pulse.  Rob Tillart’s code uses the counting method, and this works well for relatively high frequencies, because many cycles are counted during the gate interval and this reduces error. At lower frequencies, very few cycles are counted, and the precision suffers, so measuring the elapsed time during a single cycle is a better option at low frequencies.

This image is from PJRC’s FreqCount Library page, which goes into more detail on the FM sensing process. It’s worth noting some of the other useful sensor libraries that Paul Stoffregen has released including: Onewire library for the ever popular DS18B20 temperature sensor, SerialFlash to simplify SPI memory builds, and their project blog is a font of other highly interesting things.

According to PJRC: Frequency Counting: works best for 1 kHz to 8 MHz and Period Measuring: works best for 0.1 Hz to 1 kHz.  There is some wiggle room there, and you should check your sensors data sheet to make sure your method matches the output range.

If you’d rather skip the libraries, you can get closer to the bare metal with the advanced timer over-flow methods described at www.fiz-ix.com and at Nick’s Timers & Counters page. Nick’s page describes a method to measure frequency with the input capture unit on pin D8. While that’s some pretty advanced code, it allows you to measure pulsed inputs to a resolution approaching the frequency of your system clock.

3) Thinkers

This Grove I2C hub connects several the sensors to the bus, so only one jumper set needs to be patched down to the logger platform. The mounting shown here was done with plumbers putty, which hardens quickly, and adheres well to most plastic enclosures.

Modern chip-based sensors offer high resolutions and complex signal processing capabilities that can be hard to replicate on the Arduino.  Most of these digital sensors send data using serial communication protocols over a common set of “bus” wires that are physically connected to all of the sensors. Serial protocols can be intimidatingly complex for beginners, but you rarely need to worry about the details because most of the vendors in the Arduino landscape release libraries to simplify the use of the sensors they sell. These libraries make it quite easy to work with complex sensors, and they are one of the reasons that companies like Adafruit and Sparkfun have such a dedicated following in the maker movement.

Newer versions of the Arduino IDE have a Library Manager which provides access to a large list of libraries with a one-click install. Sensors that have been used for a few years by the community often have a library available through the manager. However for new sensors, you usually have to download a library and manually place it in your

Username>Documents>Arduino>Libraries folder.

You can Learn more about installing Arduino libraries at:
Sparkfun :Installing an Arduino Library,    All About Arduino Libraries at Adafruit 
and  Installing Additional Arduino Libraries at Arduino.cc.

Note that these little bits of library code can be located in several different places on your hard drive but it’s best to keep the ones you add in your sketchbook folder because the Arduino Software (IDE) upgrades itself by first erasing everything in the program root directory: including any libraries that were stored there. Libraries in your personal document folders are not deleted during the Arduino Software (IDE) update process.

Here I’ve added a resistor which pulls the CS pin high to tell this ADXL345 accelerometer to communicate with the I2C protocol rather than SPI. The value of that pull-up resistor is not critical, so they can range from a 200Ω to 10kΩ.

The digital sensor protocols you are most likely to see used with an Arduino are SPI and I2C. It’s fairly common for chip-based sensors to support BOTH protocols and for those you usually add a pull-up or pull-down resistor to tell the chip which one to use.  SPI is preferred when fast communication is needed to move large amounts of data, but this is rarely the case for environmental monitoring. More importantly, the Arduino SD card libraries expect the SPI bus to be operating in Mode0. Adding a sensor to the Cave Pearl Logger which changes the SPI bus to one of the other three operating modes would prevent data from being saved until the bus was reset to Mode0. I have yet to find an SPI-only sensor that doesn’t have an I2C equivalent on the market.

 

I²C sensors are often the best choice for Cave Pearl Data Loggers.

The DS3231 RTC breakout module used on the Cave Pearl logger has a cascade port at one end, making a perfect attachment point for other I2C devices

The I²C bus or TWI (Two Wire Interface) allows a single master (the Arduino) to share communication lines with more than 100 slave devices (the sensors). Cave Pearls use an DS3231 RTC for timekeeping and the I²C breakout board carrying it provides 4.7k pullups resistors on the SDA and SCL communication lines. Each new I²C sensor gets connected to the same wires as the RTC board. If you have a good library to go with your sensor, about the only thing that might prevent it from working is a bus address conflict. Because I²C devices are all connected to the same wires, the Arduino needs a way to talk to only one device at a time. It does this using the I²C address of each sensor. (kind of like a phone number)

The first thing to do with a new sensor after connecting it to your Arduino, is run a bus scanner which queries every possible address to see if any devices are responding. If two devices are trying to use the same address, only one of them will show up in the scan, and sometimes neither of them will. Code for this basic utility for this can be found at the Arduino playground.

Running that bus scanner on a Cave Pearl data logger before any sensors are attached should produce:

This output screen tells us that the RTC breakout board is functioning and the I²C communications are working. It also tells us that the I²C “device addresses” are 0x57 ( this is the EEprom on that module ) and 0x68 (the DS3231 RTC).  Adafruit has compiled a list of typical I2C addresses for different sensors and scanning through that list for the two we are already using on the logger finds some potential conflicts:

0x57
MB85RC I2C FRAM (0x50 – 0x57)
MAX3010x Pulse & Oximetry sensor (0x57)  (uh-oh… this sensor will not work with our logger!)

0x68
This address is popular with real time clocks – almost all of them use 0x68!
AMG8833 IR Thermal Camera Breakout (0x68 or 0x69)
DS1307 RTC (0x68 only)
PCF8523 RTC (0x68 only)
DS3231 RTC (0x68 only)
MPU-9250 9-DoF IMU (0x68 or 0x69)
MPU-60X0 Accel+Gyro (0x68 or 0x69)
ITG3200 Gyro (0x68 or 0x69)

Some I²C devices have only one fixed address and but most offer a small range of different addresses that you can set by connecting different pins on the module to power or to ground. This will let you resolve an address conflict, but be make sure to make corresponding changes in your code if you change a sensor address away from it’s default. Most sensor libraries will have a modifiable parameter for the device address that is used to initialize the sensor. If you have a sensor with a fixed address, you will only be able to hook up one of those sensors to the logger at a time unless you add an I2C multiplexer to resolve the address conflict.

Once you’ve confirmed the sensors show up on a scan of the I²C bus, the next steps depend on the complexity of your sensor. I²C sensors that only do one thing can often be read with a minimal amounts of code after the #include <Wire.h> statement embeds the TWI library that’s built-in to the IDE . You often see this with a basic temperature sensors like the TC74

//All I2C coms start with a handshake transaction with the device @ address
Wire.beginTransmission(address);
Wire.write(0);  //Sends a bit asking for register 0, the data register of the TC74
Wire.endTransmission(); // nothing is sent over the wires until wire.end is executed

//then you request the temperature data from the TC74 sensor
Wire.requestFrom(address, 1);  //this requests 1 byte from the specified address
int celsius= Wire.read();

For sensors that do more complicated things there can be many more steps, especially during sensor initialization when you might have to configure the bit-depth of the readings, the sampling speed of the sensor, and a host of other options. I’ve posted an extensive tutorial about this for tech-savvy users (see: How to configure I2C sensors ) but for beginners the best approach to adding a new I²C sensor is

1) Find a suitable tutorial by typing “How to use SensorX with Arduino” in Google or by reviewing the tutorials available at: Hookup Guides at Sparkfun, the Sensor tutorials at Adafruit, or search for code examples and links at the sensors forum.

2) Download the associated sensor library and install it into your Documents>Arduino>Libraries folder

3) Add #include <SensorLibrary.h> to the start of your code

4) Initialize the sensor in startup following the code examples for your tutorial located on GitHub

5) Read the sensor in the main loop

Most libraries are written to provide InitializeTheSensor() and ReadTheSensor() functions that so steps 4) & 5) often end up adding only couple of lines to your code.

As a simple example look at the MCP9808 temperature sensor from Adafruit:

The Tutorial   &   The example code on Github

That script is quite small because the library condensed a lot of I2C handshaking down to

tempsensor.begin();  // initializes the sensor
tempsensor.readTempC();   // reads the sensor

For an example of a library driving a more complex sensor, look at the BMP180 pressure sensor from Sparkfun

Hookup guide   &  The example code on Github

There’s an important step at the start of the code:

#include <SFE_BMP180.h>
#include <Wire.h>
SFE_BMP180 pressure;    // creates an SFE_BMP180 object, called pressure
#define ALTITUDE 1655.0 // Altitude of SparkFun's HQ in Boulder, CO. in meters

In setup, theobjectname. appears before each call to library functions:

pressure.begin()       //initializes the sensor in setup

And then in the main loop, the sensor uses a four step process to complete one reading.

status = pressure.startTemperature();
delay(time);
status = pressure.getTemperature(T); 
// the temperature must be read before the pressure!

status = pressure.startPressure(3);  // with oversampling set to 3
delay(time);
status = pressure.getPressure(P,T);

Multi-step read procedures like that are quite common, because it takes time to capture high resolution readings, and in this case the temperature has to be sent as a correction factor for the pressure reading.

Then there are two more functions in the example program worth noting:

p0 = pressure.sealevel(P,ALTITUDE);
a = pressure.altitude(P,p0);

The pressure sensor returns absolute pressure, and Sparkfun have provided extra code in their library to do calculations which convert that number into sea level equivalent & altitude equivalent numbers.  That Sparkfun code example is pretty typical of what you get with libraries for more complex sensors, and it should not be too hard to just open two IDE windows to copy and paste the required pieces of code from the Adafruit & Sparkfun examples into the basic Cave Pearl Logger script on Github.  There is nothing magical about libraries: they are just pieces of code that you can read through yourself by opening the .cpp file listed in the same Github repository. I recommend that you always review the library code, as figuring out how someone else’s stuff works is an important part of learning how to program the Arduino.

Think about your housing-logger interconnections before you start your build.  My current favorite connectors are Deans Micro Plugs, which are available in 2,3,4,5,6,& 8 pin versions. Use a consistent color convention for different bus wires.

Most libraries will include a simple example sketch with the downloadable file. These show up in the IDE in the FILE>EXAMPLES> pull-down menu after the library is installed, so you don’t usually have to go all the way to Github like I did here. The included examples usually only initialize the sensor and print out some raw readings, but that’s exactly what you need to verify the sensor is working before you merge those bits into your own code.

The real benefit of a good library is not just the code, but the significant amount of time someone spent slogging through a sensors data sheet figuring out the correct sequence of operations.  Just because a library exists for your sensor does not mean that it is necessarily a good one – especially when you find them out in the wild. So you should test different libraries when you have options. I generally chose libraries that require the least amount of memory at compile time, and/or ones that give me access to the ‘raw’ sensor readings in addition to the processed output. Raw sensor readings let you do calculations later in Excel to make sure the library didn’t introduce an error somewhere.  Another thing to keep in mind is that sensor libraries don’t have to be continually upgraded like the software you run into on a more complex system. Once a sensor library is working, it will hang around for years with no updates because none are needed.

Well… this post swelled into another voluminous tome, but hopefully no one lost sight of the forest for the trees.  Generally speaking you can buy each type of physical sensor in all of the data output ‘flavors’ described in this post.  As an example, there are both analog (voltage) & digital (pulsed) anemometers, and the digital ones range from simple reed-switch thumpers to ones with onboard IC’s doing most of the raw signal processing to provide calibrated wind-speed numbers over an I2C bus.  Don’t mistake the Analog vs Digital divide as any indication that one kind of sensor is necessarily better for the job you are doing.  Same goes for my tongue-in-cheek categories for Flippers, Thumpers & Thinkers. They’re just conceptual tools to use when you are hunting through tutorials on instructables, or when you run into an intimidating wall of information like the Interfacing with Hardware page at Arduino.cc.

Although this post has been focused on capturing sensor data with a logger, you should also keep in mind that there are many different physical methods to measure the same phenomenon. Using the anemometer example, most people think of the traditional egg-cup spinners because that’s what they are used to seeing on rooftops, but heat-loss methods, and ultra sonic methods are also quite common. A Google search on ‘how to measure water level’ shows the incredible range of different sensors can be put to that simple task.  When you are faced with a range of methods like that, the ‘best sensor’ for the job is the one you can actually get working, and that usually boils down to the amount of programming complexity you are comfortable with. Good libraries can level the playing field quite a bit, making complicated sensors almost as easy to add to your data logger as basic analog voltage dividers. And it never hurts to review the difference between Resolution, Precision & Accuracy before you choose a sensor.

Addendum 20171218

A few people have commented about my use of string variables in the basic logger code, and the general consensus is that the String class should be avoided because it can lead to memory fragmentation. It is better to use character arrays, but there is a significant learning curve there and strings will let you build a working data logger when you are just starting out. Majenko has one of the most concise summaries of steps to address this issue, and there is a reasonably good introduction to character arrays, and many other helpful concepts at the Starting Electronics: Arduino Programming Course  (see: Section 18).  Personally, I find that having to re-jig sprintf() statements when I want to add another sensor to my logger is a pain in the backside. A more memory friendly approach could be to simply open the file and then save the variables directly to the SD card without the string concatenation steps.

dataFile.open(FileName, O_WRITE | O_APPEND);
dataFile.print(AnalogSensorReading); dataFile.print(",");
dataFile.print(DigitalSensorReading); dataFile.print(",");
[...]
dataFile.close();

An alternative way to address String memory issues is to use the Pstring library by Mikal Hart.   “Print-to-String” is a lightweight Print-derivative string class that renders text into a character buffer that you define at the start of your program.

char DATABuffer[30];  //This character array receives the ascii characters
// it's worth noting that you can't move more than 30 bytes at a time 
// over the I2C bus due to limitations of the wire library buffer, 
// so my receiving arrays are usually [30] bytes long.

The data concatenation steps I described previously for the basic UNO logger:

dataString += ", ";      //comma that separates new data
dataString = dataString + String(DigitalSensorReading);

are slightly different for the more advanced Cave Pearl logger code which uses the Pstring library:

PString str(DATABuffer, sizeof(DATABuffer));// set the array as the receiving buffer
str = "";                                   // this empties the receiving buffer
str.print(CycleTimeStamp); str.print(",");  // this data is already in ASCII format
str.print(DigitalSensorReading); str.print(",");     // this data is an integer
//add more variables as needed  up to the [30] char limit
// separating each additional sensor reading with a comma

It does not matter what what the source variable format is – float, integer, etc – it all gets rendered into ascii by the str.print statements. And Pstring will never cause a buffer overflow because any excess data that you try to add to the DATABuffer is simply discarded. That receiving buffer will always contain valid (i.e. NULL-terminated) C string data. This makes the method much friendlier for people who are new to programming.

To save the sensor data stored in the char array buffer to the SD card use file.write:

file.open(FileName, O_WRITE | O_APPEND);
file.write(DATABuffer, sizeof(DATABuffer));
file.close();

Addendum 201907:

The latest revision of our logger starter sketch dispenses with the string manipulation described above, and just writes the variables to the SD card directly with file print statements.

Tutorial: Using the Arduino UNO Serial Plotter for Live Data Acquisition

Using an UNO and an ADS1115 ADC module to monitor current use by a pro mini logger

Here, I’m using the basic UNO Logger as a tethered Data AcQuisition device, recording the current used by a second data logger. Since the second logger is ‘floating’ with no connection to the UNO’s ground line, the voltage drop across the 5Ω shunt resistor is recorded using a differential channel on the ADS1115. Differential readings are also useful for sensor applications that use a wheatstone bridge arrangement.

I recently picked up a ADS1115 breakout board, and it was fairly easy to use that with the serial data plotting capability of the Arduino IDE. It’s not often that something works this well on the first try, and I thought I would post about using  the combination as a kind of  ‘poor mans oscilloscope’.   The plotter’s vertical axis auto adjusts as the value of your output increases or decreases, while the X axis is fixed at 500 points, with each tick of the axis equal to an executed serial println command.  Having the ability to spool data to the screen with a simple print statement, turns the exercise into a “What happens if I do this?” kind of process, which is perfect for providing feedback to students learning how to program Arduinos.  I posted the code used to generate these graphs on GitHub, but you will have to noodle around with it to figure out what the threshold settings should be for your particular application.

Although I already have a good method to estimate the overall power consumption of my loggers, I was motivated by this Jeelabs post to see if there was a way I could look at individual events. Even if you have a nice Rigol to play with, it can still be tricky to get all the settings and timings right because the loggers can draw anything from 0.15mA while sleeping, all the way to up to 100mA during SD writing events.  Jeelabs elegant solution to this ranging problem uses two shunt resistors and a diode, but with 15bits of differential range on the 1115, the 8x gain setting can do the job with a single resistor. (…provided I don’t exceed the ±0.5 volt limit that PGA setting imposes…) 

After installing Rowbergs I2Cdev library, and running a couple of jumpers, my drip sensors were generating raw count output like this:

Cave Pearl data loggers

Of course that was scrolling by like crazy, and the re-scale feature meant that the y-axis was jumping round like a bullfrog on a hotplate. You can bring the vertical axis under control by adding a few delimiter separated constants before the final data println:

Serial.print(4000); Serial.print(” “);     // sets a stable upper value line
Serial.print(0); Serial.print(” “);             //this constant sets a stable lower value

But to prevent the x axis from scrolling forever, I had to setup a trigger threshold, that would only capture a new set of readings when an actual event was occurring:


void loop(void)
{
int sensorOneCounts=adc0.getConversion();  //note we already set the mux to differential in setup
if(sensorOneCounts >= LoopThreshold) {      //rapid sampling loop to capture the event
start=millis();
for (int Cycle = 0; Cycle < ADCcycles; Cycle++) {
currentADCreadings[Cycle]=adc0.getConversion();
}
elapsed=millis()-start;// elapsed gives you the timebase for your samples
}
//A separate loop to output of the samples to the text monitor or serial plotter
for (int Cycle = 0; Cycle < ADCcycles; Cycle++) {
Serial.println(currentADCreadings[Cycle]);
}
#ifdef ECHO_TO_SERIAL    //TEXT only output of the time it took for your data acquisition loop
Serial.print(F(“Time for “));Serial.print(ADCcycles);Serial.print(F(” readings: “));Serial.print(elapsed);Serial.println(F(” milliseconds”));
#endif

}

A few runs with text-only output showed the sleeping logger generating around 65 raw counts on the ADC, so I set the LoopThreshold to 75, and the plotter started behaving like an oscilloscope’s triggered sweep: scrolling 500 new readings across the screen each time the drip sensor woke up.  With a known shunt resistance, and the millivolt resolution per bit from the datasheet, I could convert the raw ADC counts to μA in the printing loop. Just for fun I also tried speeding up the I2C bus to see the temporal resolution of that sample loop at 400Khz:

Running ADS1115 with UNO as DAQ - with I2C bus running at 400HZ, the system outraces the ADCSo I was asking for data faster than the 1115’s top speed of 860sps could deliver it.  A few more trials showed that even with the Arduino’s default 100Khz bus, I had to add delayMicroseconds(400);  into the capture loop to keep the UNO from outpacing the ADC module.  I expect that will be different for Arduinos with different clocks like the promini, so one would have to make manual adjustments based on the elapsed times for each system.

These drip logger events are the result of straightforward code that follows the steps:

1) Wake from interrupt & clear the registers, update count variable
2) Turn indicator LED on & sleep for 15ms
3) Wake, turn LED off & sleep for another 32ms (to let impact vibrations die down)
4) Set accelerometer registers & go into deep sleep: wait for next accelerometer interrupt

Since you can drop as many constants on the plotter as you want, it’s easy to determine the current level at a specific point on the plot by intersecting it with a new line:

Cave Pearl data loggers

Though there were no surprises in the pattern per se, it was handy to find out that the “in-event” sleep current was about twice the normal standby state. (which I confirmed at 0.2mA using a DMM)  Now I can go digging through the code to figure out what’s drawing  that extra power.

After resetting I2C bus, I tweaked the delays to exactly match the 860 sps output from the ADC. Then I sat back and waited for one of the real time clock alarms to fire:

Cave Pearl data loggers

This was my first view of that complex logger event, with variable juggling, time stamp creation, eeprom buffering, counter resets, etc. With I2C devices being able to stretch the clock whenever they need to, I had no idea how long these kind of events actually took, and I certainly never managed to capture one so easily on the old clunker scopes I could get my hands on.  The comb pattern you see there is a result of sleeping the processor for 15ms after sending data to I2C device registers; instead of using a 6ms delay while the Arduino waits for the devices to respond.  This lengthens the overall duration of the event, but the time spent running at full power is reduced.

By how much? I reset the logger to use delays after I2C writes rather than sleeps and re-captured the event:

Cave Pearl data loggers

The horizontal axis is at the same scale as before. The start of the event now looks like a solid block and the overall event is about 50ms shorter.  For comparison, I’ve inset the previous graph with the sleep gaps cut out and the spikes moved together, and the two white rectangles at the top represent area equivalent to sleep gaps that were removed.  The grey area shows the power saved by using 15ms sleeps instead of 6ms delays.  To do this properly I would need to take another reading of the 3xAA power supply voltage and calculate the total power used (minus the loss on the shunt resistor), but you can see at least 25% improvement simply by comparing the plots. That kind of feedback is very helpful when you are trying to optimize your code.

Now, I can already hear people scoff that even $3 is too much to pay for a scope that won’t reach 1Khz. Indeed, with a minimum temporal resolution of just over a millisecond, the ADC frequently misses brief current peaks. I know my 8mhz promini based loggers usually draw more than 4mA while running, and you see the longer operations in the RTC/delays event pushing up towards 6 mA.

Cave Pearl data loggers

But even with the upper third lopped off those spikes by timing errors, I can still use this method to make a reasonable estimate of the power being consumed by logger events provided they don’t last more than 580 milliseconds. ( The serial plotter provides a window for only 500 samples and the UNO barely has enough variable memory for a 16bit integer array that size.) If you add the SD card library to your script you are only left with enough memory for about 100 of these samples in your capture loop, although you could try reducing the ADC readings down to 0-256 with a well tuned map function. Then you could use an uint8_t integer array, and store twice as many readings in the limited variable memory of the Arduino.  If you were just doing a classroom demo, that range would probably still give you a decent display on the plotter.

Despite these limitations, the setup is so simple that I think the serial plotter tool will become a go-to for quick looks at sensor data, perhaps even replacing some processing based demos.  The ability to connect four of these ADC modules to a single Arduino makes the platform look quite respectable as a classroom level data acquisition system, especially for environmental data that rarely requires high frequency sampling.  

If you switch to a beefier 1280 based Arduino like the Mega, or the Moteino, those memory limitations disappear and recording much longer events would become possible.  With the delays I had to put in, my gut feeling is that you could juggle a couple of these boards at a time; interleaving the readings to double or triple the effective sampling frequency. (at least until the input impedances start to mess with the shunt resistor…)  And finally, I think adding a couple of well chosen diodes to clamp the input voltage to the ±limits imposed by your PGA setting would be a good idea in the classroom, or you might loose a few of those ADS1115 modules.

It is worth noting that there is a comparator function in the ADS1115 that can be programmed to send a pin change alert when a given threshold is crossed. If I can figure out how to get ADS1115’s built in comparator working with the differential modes, this would even let me put the DAQ to sleep between sampling runs. The ADC module also has a ‘data ready’ alarm that can be enabled to tell you exactly when it has the next reading available.  I did not go that route as the generic approach used here is applicable to sensors without these features, including the ADC built into the Arduino. 

There are plenty of other serial data plotting programs, but using the free IDE version like this is an iterative exercise, forcing you to switch between the serial text monitor & the serial plotter a few times to tweak the delays and constants. If you are using an UNO in a classroom setting, this in itself is a useful activity.  And given how cheap basic DAQ modules are these days, choosing to use an Arduino is always about the showing the process rather than just getting a final result.

Addendum 2016-08-15:     A MUCH faster DAQ with the native UNO ADC

 Here I’m reading the voltage across a 10Ω resistor with the UNO’s built in ADC. To do that I need to join the ground lines because the Arduino can only take single-ended readings. That also forces me to put the shunt on the low side line so I don’t exceed the 1.1 vref. This arrangement causes the drip sensors’s effective ground line to jump around as the current through the resistor changes.


Here I’m reading the voltage across a 10Ω resistor with the UNO’s built in ADC. To do that I need to join the ground lines because the Arduino can only take single-ended readings. That also forces me to put the shunt resistor on the low side line so I don’t exceed the 1.1 vref. This arrangement causes the promini logger’s effective ground line to jump around as the current through the resistor changes.

The folks over at Measuring Stuff  posted a page called The Arduino DAQ Chronicles, which goes into some detail on the process.  It’s a good chunk of background reading even if you are using the ADS1115 ADC described above. The Arduino’s native ADC delivers 10-bit readings, (ie 0-1023) and compared to the relatively pokey ADS1115, a typical UNO can take several thousand analog readings per second; outperforming the ADS1115 by a fair margin. The challenge is balancing the reduced resolution with the other limits imposed by the method itself. Even if you increase the sensitivity of the native 10-bit ADC by changing to the internal 1.1vref, you only reach 1.1v/1024 steps = 1.07 millivolt per bit. Remember that my loggers are sleeping at 0.2 milliamp x 5Ω shunt resistor = 1 millivolt so we are operating right on the lower limit of the ADC resolution. With that 1.1v cap on the ADC input, and a potential 100mA peak current (during SD card data saves), the largest shunt resistor I can use is 1.1v/0.1a= 11Ω. So to get the Arduino to discriminate the low currents we need to use a pretty  large shunt resistor value, and allowing the voltage drop to get that high imposes another challenge in that the drip logger would only be left with 4.5v (the 3xAA battery) 1.1v drop on shunt = 3.4v. That’s right at the minimum input voltage needed by the 3.3v regulator on the Arduino promini board, risking a potential brown-out in the middle of the SD card writing process.

But hey, lets pop in some new batteries, and see what we get:

FirstRaw

Not too bad, but that’s a noisy plot with lots of jitter on the base line. Having 1-2 bits toggle like that is typical for ADC’s so there is probably nothing I can do to get rid of it. And the drip sensors resting state reads at zero when it should be registering at least two counts. It is also common for an ADC to under or over read by a few bits, so I added an offset adjustment into the code to bring that sleeping baseline up to the 0.2mA that I have already confirmed with a multimeter.

Then I realized I still had the 400us delay in there from the ADS1115, so I took that out:

Cave Pearl data loggers

Crikey! Even with the code overhead, that thing is taking over 8000 samples a second! That’s far too many readings for the serial plotter to handle if I want to view long events.  And that first spike goes upt to about 5mA – exactly where it should have been on the ADS1115 readings if that module had not missed them at 860sps.  So how do I maintain the Arduino’s ability to spot those rapid sample peaks, but only send 500 readings to the screen?

Instead of delays, I decided to create an “oversampling loop” that creates a running average using a low pass filter, but also checks for peaks throughout the process. You can dig into the code for this over at GitHub, but the basic idea that is if a peak occurs, then the highest reading in an interval becomes the final reading. If a decreasing trend is found, then the output from a leaky integrator becomes the final reading for that interval. So the readings get smoothed on the way down, but not on the way up.

Here is a try with the oversampling interval set to six ADC readings:

BothEvents6xoversampling

That’s the RTC event (with delays) and the drip triggered event at the same scale.

Now here’s another run with a 15:1 ratio:

OS15-900msPerloop

Squashing more than 7000 raw readings into a 500 line display really starts to distort the curve, but everything is still recognizable, and more importantly, the peaks did not get chopped off like they did with the ADS1115.  Woot!

Now we have a method that lets us adjust the serial plotter’s display to match the event we are trying to see, provided we don’t over-interpret the accuracy of those heavily averaged interval numbers. And rapid sampling can introduce other issues to deal with. For example: Dustyn Roberts has a useful post on creating very precise time stamps with the micros(); function when you are sampling faster than 1 millisecond. 

So within limits, a standard Arduino gives you good temporal resolution but can not detect less than 1 millivolt of change, while the ADS1115 gives you precision down to an impressive 0.0078 millivolts (@16x gain) but is not so great for tracking really brief events.  If you need both resolution & speed, you need opamp circuits designed for the specific situation, as in the Jeelabs example. It’s also worth mentioning that the Arduino ADC also supports a comparator that you could use for the threshold function more elegantly than I’ve done here.  And with such a low impedance input, you can also double or triple the sampling rate by changing the ADC clock prescalers (this is also in the code on GitHub) Both of those things are described in detail over at Nick Gammons excellent page on ADC conversion on the Arduino.

Addendum 2016-08-16

Hopefully they will add export options (like other serial tools) to the plotter in the future, but for now you can manually copy data out of the serial text monitor and paste it into Excel.  Before you start a long run for export to a spreadsheet, comment out the constants that you were printing to stabilize the plotters y – axis, so that only the numbers you are interested in are sent to the serial output.  If you are waiting for an event to happen, it often helps to add a threshold test so that the serial monitor is not just filling up with uninteresting data that you would have to prune away later anyway:thresholdprintline

Then after your DAQ has been capturing events for a while, click inside the serial monitor window,  select all the data & copy it (on a windows machine that would be [CTRL]+[A] followed by [CTRL]+[C] ) then open a new spreadsheet and simply paste the data into an empty column.

This lets you compare the events side by side:

CopySerialdataintoExcelIf you want to graph more than one number, then add a comma between them with Serial.print(“,”);  remembering to use println on the last variable being sent.  Then you need to add the intermediate steps of pasting the data into a text editor like notepad, and saving that text file. Then change the ending from .txt to .csv and Excel will open it directly. If you can’t change the filename, you can still import the text file and specify comma delimiters:

TextImportExcel

to put the numbers into adjacent columns. If you don’t have excel, there are plenty of other data plotting options out there. And programs like coolterm, can save you from having to do those cutting & pasting steps.

Addendum 2016-10-13

Gokul Shrinivas just posted a neat little bit of code at hackster.io that reads the Arduino ADC at a cracking 68k samples per second out of a Arduino Due by directly accessing the ADC ports as soon as the DataReady interrupt is triggered. 

He starts with defines:

#define ADC_MR * (volatile unsigned int *) (0x400C0004) /*adc mode word*/
#define ADC_CR * (volatile unsigned int *) (0x400C0000) /*write a 2 to start convertion*/
#define ADC_ISR * (volatile unsigned int *) (0x400C0030) /*status reg -- bit 24 is data ready*/
#define ADC_ISR_DRDY 0x01000000
#define ADC_START 2
#define ADC_LCDR * (volatile unsigned int *) (0x400C0020) /*last converted low 12 bits*/
#define ADC_DATA 0x00000FFF 

And the sampling loop becomes

ADC_CR = ADC_START ;
  for (i=0; i<320; i++){
    // Wait for end of conversion
    while (!(ADC_ISR & ADC_ISR_DRDY));
    // Read the value
    analog_data[i] = ADC_LCDR & ADC_DATA ;
    // start next
    ADC_CR = ADC_START ;
  }

He used an LCD screen for the scope, but I am wondering if it would be possible to refine my oversampling trick to squeeze that many data points on the serial plotter somehow…

And I wonder if this trick is even available on the 328p? I guess I will have to go looking at other Arduino Oscilloscope projects. The problem is that most of them seem to be using 8-bit signals from “free-running” modes, and I rarely have an application that needs less resolution. Here’s a project doing Interrupt-Driven Analog Conversion  with an ATMega328p, and Nick Gammon mentions it here, but no one does a speed comparison with the ten analog 10-bit readings you get per millisecond using plain old analogRead();

Even if there is no speed gain, perhaps I could do calculations while I was letting the ADC read continuously to speed up the overall sampling loop.  I could see myself getting out of sync pretty easily though, unless I deactivate the general interrupt flag and activate it again (with  cli() and sei()) or I would randomly miss readings.

Nick posted a code example of reading the Analog-to-Digital converter asynchronously that seems to solve the synchrony problem.  He’s also posted an interesting ‘sleep during ADC conversion’ example buried in there, but that’s only useful if you are not using timers & PWM at the same time, and I often do that…

Update: the Hackster.io project listing magically evaporated… and I just found exactly the same code at Bruce Land’s Hackaday project from 6 months ago. So I think proper attribution for the script above should go to Bruce.

Addendum 2017-01-10

I’ve been playing around with the ADC clock prescalars, and with the asynchronous reading code available over at Nick Gammon’s site to try to get better temporal performance out of my little UNO-scope.  With such a low value shunt resistor the ADC seems to read fine even if you crank it up to about 40k samples/second. Unfortunately you pass a point of diminishing returns due to the processing overhead with the interrupt based asynchronous readings,  so async reading actually reduces the number of readings per millisecond at the higher clock speeds. If you want to play with this approach yourself, I’ve posted the update with the prescalar settings to GitHub.

Addendum 2017-02-27

Anyone interested in using an Arduino as at data acquisition device will probably want to see my recent post: Enhancing ADC resolution with Dithering & Oversampling
For an unmodified UNO,  Qwerty’s triangular dither method is probably the easiest one to use, but for 3.3v boards I’d go for the Toggled pin dither method I outline at the bottom of the post.

Addendum 2019-09-27

Microsoft has added a ‘data streaming’ plugin that’s free for Office 365 users which will accept data directly from an Arduino. If you are a student, the 365 software itself is also free. But for those of us running older versions of Office, the cut and paste method described above still works fine.

Build your own Arduino Starter Kit for the Classroom

A typical student starter kit.

It’s August, and we’re assembling kits for another run of the instrumentation course.  Over time we have come to rely on 328p based microcontroller boards (aka: Arduinos) so I though I would post a list of the parts & materials we use to help others fire up their own classroom. More than a few people have requested this since I posted the UNO based logger tutorial.

This information should also be looking to buy an all in one solution for their first citizen science project.  Don’t reinvent the wheel if you don’t have to: there are lots of premade Arduino kits out there.  However we found that many had parts we simply didn’t need, as we wanted a minimal ‘starter kits’ designed to exactly match the lessons in a course focused on environmental monitoring.  We also hand out extra parts for some tutorials as needed through the course, and the students receive a second project box when they start building their stand-alone  projects. Cave Pearl data loggers Even if you want that level customization, it might worth looking at some basic electronic component kits, and building your classroom set on top of those. The DIY approach will save money, but you pay for it with time.  Most of the eBay parts shown here took about two weeks to ship, with some stragglers taking almost a month to arrive.

You will need more than an Arduino and a few components  to set up a classroom, so I am including tools & other miscellaneous bits of hardware that we use in the lab. I would not describe any of these as high quality equipment, but they are ‘good enough’ to get things rolling on a modest budget.  Even if you go with cheapest possible suppliers – you are doing great if you can run a fully developed course for $150 a seat. If you purchase a soldering station for every single student, and use UNO’s + a typical Pro Mini logger build, you’re looking at about twice that much for the initial run of your course.


Components for the UNO based Datalogger:

TransparentSinglePixl
UNO Data logger Kit: $25-50 / seat
UNO_100pxw Arduino UNO R3
The real thing, and tough as nails. All of my old UNO R1 boards are still operational despite years of abuse. If you can’t afford the real thing due to budget limitations, then at least donate a few bucks to help them keep the Arduino open source project going.
$24.95
UnoClone_100pxw UNO clone (incl. USB cable)
If your budget forces you to go this route, choose clone boards with ATmega16U2 or 8U2 UARTs (like those in the genuine UNOs) to avoid problems with the IDE. If you can deal with the driver issues, there are clones for as little $3 with alternate UART chips like the CH340 – but CH340 UARTS  do not work with Macs.
$7.20
PlatformBase_100pxw
Transparent Experimental Platform Base-plate
Last year we made our own base plates with M3 risers and Plastruct Styrene sheets. But these acrylic boards save a lot of time. Check left/right side orientation of the holes on the board before buying and the ones with the larger threaded nylon standoffs tend to be better options.
$1.50
MiniBreadboard_100pxw Mini Solderless Breadboard 400 Tie-points
Get at least two per student, in case they need to expand their projects. The plastic in these cheap breadboards tend to “warp” slightly over time and the adhesive layer on the back is not thick enough to accommodate this bending so they will slowly come away from the mounting surface. If you are attaching your breadboards to acrylic plates beside an UNO – add an extra layer of double sided foam tape to the bottom of the breadboards before sticking them down.
$1.20
40p10cmdupont 40wire Dupont 10cm Jumper cable
An alternative to loose jumpers wires, and they make the student kits look much nicer at the beginning of the term. Have students make longer or shorter jumper wires by hand when they need them to provide practice with the crimping tool. Jumpers you make yourself are always more reliable!
$1.00
JumperWire22AWG_100pxw 22AWG Solid Hook up Wire Kit
One box will cover the entire class and they should know how to cut and strip wires properly. Don’t bother with pre-cut solid core jumper wire sets as the set lengths are almost never the right size. Striveday also sells mixed boxes of solid core wire.
$20.00/box
SDmodule Micro SD Card Memory Shield Module
This shield is ONLY for UNO based loggers (not the ProMinis) Don’t forget to buy μSD cards to go with them. Test any used ones to make sure they are ok.
$1.00
RTC_100pxw DS3231 & AT24C32 IIC RTC Clock Module
I have a page describing these RTC board in some detail here. You will also need a CR2032 3v Lithium coin cell backup battery, after you remove the charging circuit resistor. The DS3231-S and -N chips are MUCH better than the DS3231M variant.
$1.00
Keyes3ColorRGBLED_100pxw 3-Colour Rgb Smd Led Module (with built-in limit resistors)
There is nothing special about this common cathode 5050 LED module, but it’s easy to remove & re-insert without bending the pins, and the low profile helps it fit in the kit boxes. The 200k limit resistors that come built-in on the green pre-mounted boards give you a current that approaches Arduino pin limits, so it wouldn’t hurt to add another 500-1k ohm on the GND line with these modules for extra safety.
$0.80
Tweezers_100pxw Vetus ESD-15 bent tip tweezers
There are many different kinds: ESD-15 have rounded ends that come to a sharp point. 
$1.00
ICbox_100pxw 10 Compartment Small Part Storage Case
To keep all the led’s, resistors, etc. from rolling around loose inside the larger clip boxes. Get ones where the section walls can be removed to make room for longer parts. There’s a range of other cheap fishing tackle and (non-waterproof) compartment boxes available.
$0.70
SteriliteBox Sterilite 1961 – Small Clip Box for Lab kits
Expensive, but they survive in a student back pack. You can find $1 clip box alternatives (like the photos at the top of this post)  at your local dollar store.  Various  other clip lock food / snack containers can also be used, and some can even be pressed into service as project housings.
$4.25
Outdoor Products Watertight Box
Measures 3.5 x 6.8 x 8 inches with 1.5-liter capacity so this one is for projects that need a lot of internal space –  it could also be pressed into service as a housing for larger UNO based projects going outdoors ( ie: lots of room for big batteries! ) Also see:  the $6.00 Husky 6″parts bin, 7×5.25×3.5″ Parts Organizer , and 4x4x2 in. Grey Sch.40 PVC junction boxes are another housing option, which are also available with 4 & 6 in. depths.  
$6.99
Plano 3440-10 Waterproof Stowaway (3400 Series)
Makes a reasonable “instant” waterproof housing for smaller projects and indicator LED’s can still shine right through: We use these on the Pro Mini based logger build. The actual usable “interior space” of the box is 8.1cm x 15.5cm x 3.5mm, which is just a a bit too small for the 83mm x120mm acrylic platform bases used for the UNOs (you can press them in, but they are very tight – better to cut them down a bit)   Pelican 1020 Micro Cases accomodate that UNO platform easily though they are the luxury polycarbonate option which cost more than 2x as much as the plano box. Polyc does allow you to put display screens and solar cells inside the housing because it passes all of the visible spectrum with little loss. Of course that also means that whatever’s inside the case will really cook (60-90°C) if you leave those housings exposed to full sun.
If your build is “skinny” enough, 2″ PVC compression couplings can be used as a housings with pvc plugs at each end. All these ideas require you to add cable glands and wire pass-through connections.
$4.96
CheapMultimeter_100pxw Basic Digitial Multimeter DT-830D
We have better meters in the lab (like the EX330 which is pretty good for ~$50) but at $3.00 each you don’t have to worry about students loosing or breaking them. Have replacement 9v batteries on hand, as people frequently forget to turn them off. Expect 1-2% error on all readings and don’t try to measure large resistors in the Meg ohm range (because the meter itself interferes with that reading) . Expect 20% of your order to arrive broken during shipment from China.
$3.40
UT120C Super Slim Pocket Digital Multimeters
Often you need a “shirt-pocket” sized multimeter during fieldwork, and we usually have a couple of these along for the ride. Cheap enough to be expendable, but functional enough for most jobs on site, but again not as accurate as the EX330. 
$15.00
eyeloupe Folding Magnifier (5x)
Useful for inspecting solder connections but not good for soldering because you inevitably need both hands to do the work. For the soldering stations we pick up cheap reading glasses from the local dollar store.
$0.55 ea.
Note: The stuff above this line goes into every UNO logger kit, but I am adding a few optional things to this list that might be appropriate for other courses:
MinimalKit_100pxw Minimal electronic components kit
Even though we don’t use them, I wanted to put this in here as an example of a minimum component package that could be a starting point for your custom kits.
$3.50
ElecfansKit Slightly less minimal component kit
Another one that’s almost tempting. Search eBay for ‘Electronics starter kit for Arduino’ and you find oodles of these. If you are only making a small number of kits these might be the way to go. But if you are making six or more, check the basic components list below, as most of the parts in these packages are pennies each if you buy them separately.
$7.70
6xAAbattery_100pxw 6xAA Battery Holder Box for Arduino
These will power an UNO for about 40 hours of continuous stand alone operation. You can stretch that out with processor sleeping & other tricks.
$2.45
PWradapters_100pxw 9 VDC 1000mA regulated power adapter
5.5mm/2.1mm barrel jack, positive tip. Expect these cheap ones to be noisy as heck compared to the ones from Adafruit. We run most of the UNO based lessons tethered to a USB cable, so these rarely get used.
$2.27
LongBreadboard_100pxw MB-102 Solderless Breadboard 830 Tie Points
They don’t go into the student kits, it’s a good idea to have a few of these longer breadboards around.
$2.00
TranspShield_100pxw Arduino UNO R3 Transparent Case
An alternative to the flat platform approach we use  would be to try these in combination with a breadboard shield
$1.60
BreadboardShield_100pxw Mini Breadboard Prototype Shield
With a little creativity you could squeeze low profile components for the data logger onto this if you find an SD card adapter, and an RTC with perpendicular pins.
$1.80

Basic Electronic Components:

Most electronic components can be had for pennies if you buy them in quantity. And the difference between low end parts and name brand stuff can be more than an order of magnitude.  That doesn’t mean that off the shelf kits are a bad thing, it just means that what you are really paying for is someone’s time selecting and packaging it.  That still might be the better option for you if you can afford it.

TransparentSinglePixl
Basic Components:
StanleyBins Stanley Removable 10 Compartment Organizer
Before you buy a bunch of tiny little parts, you need some way to organize them. I usually put the items into 3″x 5″ or 4″x 6″ bags, which then go into these compartments.  Pink 4Mil Anti-static Poly Bags come in all sizes, and work well for this. Being able to remove the section boxes from these organizers to lay out specific parts for a tutorial also comes in handy.  See: Organizer Bin Storage Unit See: Stacking organizers See: Tool Storage
$17.00
StanleyShallowOrg Stanley Shallow Organizer, 25 Compartment
This organizer is better for really small components & IC’s that you only have a few of, but you still need to bag them or they get jumbled.
$12.00
ELENCO WK-106 Solid Hook-Up Wire Kit 22 AWG
22 AWG works reasonably well with breadboards, get at least 6 colors/150 feet. Remember to specify how much to cut from the spool in your lab instructions, or your students will pull 2 feet of wire from the box when they only need a couple of inches on the breadboard. Get two boxes at the start of term.
$19.00
AdafruitWire_100pxw 26AWG Silicone Cover Stranded-Core Wire – 2m
This high strand count wire from Adafruit is the nicest stuff you are ever going to work with, and is my favorite after trying just about everything else on the market. Simply fold & cut those pre-cut 2m lengths four times and you end up with 12cm lengths which are perfect for including in kits. [Colors:Red,Black,White,Yellow,Green,Blue,Grey,Orange]
$0.50/m
CheapWire_100pxw 60m/box 26 AWG 10m x 6 colors Cbazy / Striveday Silicone Jacket Wire
As your projects get more advanced you can never have enough different colors of wire. And there are other box sets to add Orange, Brown & Purple . The thing to watch out for with wire sourced from eBay wire is that some vendors sell crappy seven-strand wire,  which will not stand up to frequent bending like the highly stranded Adafruit 26awg. I also keep some thinner 28AWG and 30AWG wire with high strand count around for finer soldering jobs (eg  here)
$17.00/box
CBAZY Stranded Hook up Wire 26-30 AWG Flexible Silicone Wire (108 Feet)
You can never have enough of certain colors, so I always order extra 30m rolls of RED & BLACK wire for battery connections.  [Colors:Red, Black, Blue, Brown, Green, Grey, Orange, Purple, White, Yellow]  16-30awg available. ~$0.33/m
$12/30m
Hshrinkspool_100pxw 1/16″ 2:1 Clear Heat Shrink (MIL-I-23053/2, class 2)
Unlike normal heat shrink, mil-spec is very thick and holds its shape after cooling; making it easier to route cables in a housing. Watch for eBay vendors that advertise mil-spec, and send you the thinner cheap stuff.   I think of 1/16″(=1.5mm) heat shrink as “1-wire sized”, 2mm = “2-wire size” and 1/8″(=3mm) as “3-wire size” when working with 26awg. Always buy clear heat-shrink tubing for the classroom – so that you can see when poor soldering has caused a problem with the circuits.
$12.00/100ft
I add male DuPont crimps & a bit of heat shrink to thin lead components to make the breadboard friendly

Resistors: Lead diameter makes a resistor breadboard friendly or not. You generally want 0.6 mm not 0.4mm leads, though it is inevitable that you will end up buying a few cheap multipacks with the thin leads. ( Note: You can add male DuPont crimp ends & a bit of heat shrink to components with thin leads to make them breadboard friendly : see inset photo) Brand-names like Vishay, Xicon, KOA Speer are usually very good, but you pay 2-10 cents each for them, and the Vishay leads are sometimes thick enough that you need to cut them at an angle just to insert them in the breadboard.  (Often you can find new-old stock bag lots of Xicon carbon resistors on eBay reasonably cheap)  The cheap old carbon resistors often tend to have thicker steel leads while the better brands have softer copper leads which bend more easily. I find 1/4 watt to be the most practical size and 1/2W metfilms are physically about the same size as 1/4W carbons. Beige-background carbon resistors seem to be much easier to read than blue-background metals, but if you need 1% tolerances, you won’t have much choice. 1/8watt metfilms are very tiny, which can be helpful if you need to put a resistor into a small place, like between two pins on a breakout board. Wherever possible, buy 1% tolerance (or better) resistors, rather than 5%, as this affects your sensor accuracy when you use them in voltage dividers – which is a common way to read sensors.

MetfilmResisorSet1 130 values 1-10MΩ 1/4W Metal Film Resistors Assortment
Crappy thin leads, but a huge range of values to have on hand at the beginning of a class – in fact I would order two of these to get started.   Then order 50-200 of the individual resistor values as needed for your labs/exercises. Be prepared for it to take 2-4 weeks for resistors to show up if you order them via eBay.
$10/2600pc
IndMetfilms MetFilm Resistor 1/4W 0.25W ±1% [Various Sizes]
100 pc of specific sizes with thin leads. We usually put five to ten of 330Ω, 1K, 4.7K & 10K in each kit, but you can tune this to your curriculum and/or hand out other specific sizes at the beginning of each lab.  After you get better at soldering, you will probably switch over to 1/8 watt resistors, as their smaller size lets you put them into tight spaces: like between the pins on a promini board. Also note that metal film resistors use a 5-band code to indicate their value, rather than the 4-band you see on carbon resistors. But we teach students to always check resistance values with a DVM to catch manufacturing errors, and because it’s quite common for them to cook their resistors, or bend the wires too much & break the connection,  when they are just learning to solder.
$1.89/100pc
HalfwattCarbons 1/2W Carbon Film Assortment (30value x10pc 1-3MΩ)
Sometimes the 1/2 watt size is easier for beginners to handle. It also helps to have “exercise specific” resistor values look physically different from the other resistors in the kits. These still have thin leads though.
$3.80/300pc
carbons Carbon Resistor 1/4W 0.25W 5% [Various Sizes]
Just a typical eBay search link to give you an idea what’s available. Usually these carbons have nicer thicker leads but the quality varies. Check for free shipping.
$1.00/100pc
Components: Most hobby market parts distributors have “Assortment Kits” for commonly used components and its a good idea to just buy a selection those when you are starting out. It might be a while before you actually dig into that mixed bag of capacitors for the odd value you need for that circuit you just found on the internet, but when it happens you will be glad you spend that $1 six months ago.

Where to buy them: I’m going to list several kits from Electrodragon as examples, but there is nothing special about them and you can often find a very similar sets on eBay for significantly less. A common problem with all the overseas suppliers is that it usually takes 3-4 weeks for stuff to arrive, and that can stretch to 8 weeks. So if you realize that you need something quickly, go to Sparkfun/Adafruit/Pololu/Amazon etc and just pay the $15 delivery charge. The thing that makes this complicated is that now a lot of domestic websites like Amazon, Digikey etc, often just acting as distributor for these exact same components from China. Another draw back with Amazon is that those easily recognized boxes are big targets for porch thieves.

My rule of thumb with dodgy flea market sites like Banggood, Dealextreme, AliExpress, etc. is: “If this order never arrives, how unhappy would I be about that?” If the answer to that question is “Quite a bit” then you should order that part from a reputable vendor.  More than 95% of the things I order from eBay do arrive…eventually…though it’s common for things like sensor modules to be significantly different than the photos shown in the listing. Overseas suppliers are best used where fakes are unlikely (or don’t matter) like resistors, LEDs, displays, pin headers, switches, wires, vanilla transistors, battery holders and similar items. Or a one-time thing you just want to play with to see how it works. For more complex sensors and microcontrollers it’s very easy to get counterfeits or rejects that kinda seem to work initially, but are way out of spec. In those cases I’m prepared for a percentage of duds, and so far I’ve only had one out of maybe 30  purchases. The cost of returning is usually more than the item’s worth, so you have to be prepared to bear the loss. There are even some complex components I wouldn’t buy anywhere else like the ESP’s, since these are manufactured in Shanghai, so you wouldn’t find these cheaper somewhere else.

LED's 5mm LED Light Diffused Assorted (5 Kinds*20PCs)
Having the colored plastic makes it easier for students to identify which LED is which. [red, green, yellow, blue, white]
$1.80/100pc