Category Archives: DIY Build a Pro Mini Logger

… from a few inexpensive components …

For an in-depth description of the Cave Pearl loggers and their performance
see: http://www.mdpi.com/1424-8220/18/2/530

Arduino Pro Mini Data Logger : with Dupont Jumpers (2016 Update)

Addendum 2019-02-21: 

In 2019, a teacher friend asked us for a build with minimal soldering because they didn’t have time in their course, or the budget for soldering stations. So we re-designed the 2016 ‘EDU build’ , adding a screw-terminal shield and a pre-made housing to reduce assembly time.  We also added support to the updated code base for using the indicator LED as a light sensor:

https://thecavepearlproject.org/2019/02/21/easy-1-hour-pro-mini-classroom-datalogger-build-update-feb-2019/

The schematic diagram is essentially the same, but the 2019 version is probably a better starting point for people building their first Arduino-based logger than this older model from 2016. For more background information see: A Flexible Arduino-Based Logging Platform for Long-Term Monitoring in Harsh Environments and the PDF is free to download from the open access journal Sensors.


Original post from 2016: Its been almost a year since the last stand-alone logger tutorial, and I continue to receive questions from people adopting the platform in education settings.  That feedback makes it pretty clear that the lack of soldering skills is a significant stumbling block for beginners, so I have come up with a build that uses pre-made DuPont style jumper cables wherever possible.

The core connections for this 3-Module logger are the same as those shown in our recent paper:  A Flexible Arduino-Based Logging Platform for Long-Term Monitoring in Harsh Environments, but here I’ve added colors to match the wires shown in this Pro-Mini based tutorial. In this diagram the battery monitoring divider is shown on A3, but that’s simply to make the diagram more readable. You can connect the battery dividers output to any analog pin. I’ve also added an RGB common cathode indicator LED, but a single color LED could be used.

This build uses a different SD card adapter than previous builds and I’ve changed the resistor location to make the overall assembly easier for beginners.This comes at the expense of having more wires to deal with on the limited real-estate of the knock-out cap platform, and re-positioning the modules to make room for the overhang of the DuPont connector housings. The overall result is a little uglier, and not as robust to knocking about as a unit where every connection is soldered in place, but the platform takes about a third less time to build (~2.5h) with a part cost under ten bucks before you add sensors. (NOTE: in 2019 we redesigned this classroom logger, with the new build taking only 1 hour to assemble.)

PARTS & MATERIALS

TransparentSinglePixl
Parts Total $8.25
Pro Mini Style clone 3.3v 8mHz $1.80
DS3231 IIC RTC with 4K AT24C32 EEprom (zs-042)
Some ship with CR2032 batteries which will pop if you don’t disable the charging circuit!
$1.25
SPI Mini SD card Module for Arduino AVR
Be sure to buy the ones with four ‘separate’ pull-up resistors.
$0.50
20cm Dupont Ribbon Cable 40pin F-F 2.54mm
You use ~1/2 of a 20cm cable per logger. Get the ones without the shrouds to save time.
$0.70
2xAA Battery Holder & 1xAA Battery holder $0.60/both
CR2032 lithium battery  $0.40
4″ Knock-Out Test Cap $0.40
Deans Ultra-T Style Battery Connector Plug $0.30
2x Nylon M2 12mm standoff, Nut & Screw M2 5mm $0.40
Micro SD card 64mb-1gb 
Older Sandisk  cards have lower sleep currents. Test used cards well  before putting them in service.
$1.00
Dupont Crimp PinsHousings
Most parts  cost about a 1-2 cents each, after you buy the crimping tool.
$0.30
Common Cathode Bright RGB LED 5mm 
A brighter bulb lets you use a larger limit resistor for the same light output.
$0.10
Double Sided Tape,  2x 4.7MΩ resistors, 26awg silicone wire, (104) 0.1uF caps, 4″ cable ties, heat shrink tubingheader pins, etc…etc… $0.50
Donation to Arduino.cc
It is now possible to build a decent data logger for less than $9 because of the hard work of many people around the world. If you don’t use a ‘real’ Promini from Sparkfun to build your logger, you should at least consider sending a buck or two back to the mothership to keep the open source hardware movement going…so more cool stuff like this can happen!
$1.00
Comment:  And a few tools you might need to get started:                     (not included in the total above)
CP2102 USB-UART Bridge module
This 6-pin connector module was specifically designed for compatibility with the Pro Mini. (note that most 2102 boards are not because they only have 5 pins) and these boards work with Mac & Windows after you install the drivers  Or for a safer option you could use the FTDI chip version.
***Be sure to confirm any UART board is set to 3.3v before using it!***
$2.20
Yihua 936b soldering station 110v
These have enough power to solder the large battery connectors.  Get extra handles & tips.
$23.00
DT-830D Digital Multimeter $3.00
Heaterizer XL-3000 Heat Gun $14.00
SN-01BM Crimping tool $22.00

COMPONENT PREPARATION

CLEANING:  Boards from reputable vendors like Adafruit / Sparkfun are usually clean, but cheap eBay modules are often covered with leftover flux from the manufacturing process and dirty contacts are guaranteed to corrode.  Most of us use 90% isopropyl alcohol to clean this residue, but some prefer Polyclens brush cleaner. A couple of five minute sessions in an ultrasonic bath with Iso90 usually works great, but you can also do it by hand with a cotton swab, etc.   (Never put vibration sensitive components like accelerometers into an ultrasonic bath)

Preparing the Main Board:

Begin by soldering header pins into place on the 3.3v Pro Mini style board.  This is much easier to do if you use a small breadboard to hold the pins in place, however you should not use that breadboard for anything else afterward because the transferred heat will ruin the contacts. I have a few of these old “for soldering only” breadboards lying around.  I prefer to use clone boards with A4-A7 broken out to the edges of the board to make the soldering the pin headers a bit easier, but those are not always available.      {Click on any images to see larger versions.}

Lay out the required header pins as in the images below. Arrange the pins for the sides of the Pro Mini board on a breadboard so that the two 3x rows extend sideways from the main board at the VCC line and the GND line on the analog side of the Arduino. This is a little tricky because you are working from the bottom of the board, not the top, so there is a left right side transformation. Also note that you can add more sidecar headers than I’ve shown here if your logger will need more connections to the power rails to supply sensors.

 {Click on any of the images below to see larger versions.}

Solder the straight riser pins into place along each edge. Then add a tiny amount of flux to the extra header pins and coat the ends of the pins with solder. Tin the wire on one end of a resistor and solder it to the GND header pin on the Pro Mini.

Then weave the resistor leg through the sidecar pins and use that wire to bridge the solder connection to all of the sidecar pins beside the ground pin on the Pro Mini. Repeat the procedure again for the VCC sidecar headers. Use a good amount of solder here to bridge the pins.  The sidecar pins are a bit fragile since they are not attached to the module, and could be accidentally broken when attaching Dupont connectors later on unless you provide some extra mechanical support with solder.

Now that the risers along the sides of the Pro Mini board are in place, we tackle the other riser pins on the board. These are a bit trickier because the A4 & A5 riser pins (needed for the I2C interface) are not aligned with standard breadboard pin spacing. Place the remaining pins into the board face up, then carefully turn the board over onto a flat surface making sure the loose pins stay in place.

It is good to have the six serial UART I/O pins bent slightly away from the other risers to make more room for connecting the UART board. Once the pins are in place clean the flux residue on the bottom of the board with some 90% isopropyl alcohol and a cotton swab.

On the adapter in this picture, the USB to TTL adapter pins are in the reverse order of the Pro Mini style board. This is a fairly common issue with clone boards and simply requires that you flip the adapter. I have connected 3.3v boards the wrong way round many, many times, and not one of them has been harmed by the temporary polarity reversal.

Before proceeding further with your logger build, take a few moments and connect the board to your computer via the UART adapter to test it out by uploading the Blink sketch. You need to install drivers to get the serial communications adapter working with your operating system, and this part of the process can be challenging for people who have never done that before. There are well written guides at Sparkfun & Adafruit if you are using boards with the FT232RL chip from FTDI. If you are using this CP2102 from Silicon Labs then you need to sort out driver installation on your own, but a quick google search should bring up lots of guides posted around the web. Once you have the drivers installed, the Arduino IDE should display a new com port when the adapter is plugged in to the usb port of your computer. At that point you can upload sketches to your Arduino through the serial adapter after selecting Board: Arduino Pro or Pro Mini   and  ATmega328(3.3v, 8mhz)   in addition to the new com port provided by the drivers. If you are ordering parts from eBay, then buy at least three of everything – it’s not uncommon to see a 10-15% failure rate on these budget parts…that’s why they are so cheap. I also order those backup parts from different vendors, since some will take months to ship, while others get the modules to you in a week.

Once you have confirmed that the board is working,  remove the power LED limit resistor and the resistor for the LED on pin 13. Simply hold the tip of the iron on one side of the small resistor and apply a gentle sideways pressure. When enough heat has conducted through the resistor, the solder will melt and you should be able to simply ‘flick’ the resistor off the board. Disabling these LEDs conserves battery power. (Note: these limit resistors are in different locations from one Arduino compatible board to the next, so you might have to hunt around to find them.)

Power LED limit Resistor

Pin 13 limit resistor

Trim the A4 & A5 pins flush to the board with a pair of side snips. Add two layers of double sided tape to the underside of the Arduino board so that it can be affixed to the platform. Due to the riser pins extending along the side of the pro-mini, two layers are needed: The first layer should fit “between the pins” and the second layer should extend past the pins to the outer edges of the board.

The RTC Module:

I don’t recommend using LIR2032 rechargeable batteries with this module as the charging circuit wastes a great deal of power, and a CR2032 will backup the RTC for many years.  Remove the charging circuit resistor and power LED limit resistor from the circuit board indicated with the red squares in the first picture below.  Next add four 90-degree header pins to the I2C cascade port on the RTC board. These four pins will become the connection point for any I2C sensors you add to the data logger:

rtc1 Cave Pearl data loggers rtc3
Cave Pearl data loggers Note the battery connector leg on the top surface is trimmed a bit Cave Pearl data loggers

Since the RTC board already has 4.7k pullups on the SDA (data) and SCL (clock) lines, you will not need to add them to your I2C sensors. This board also has a 4.7k pullup on the SQW alarm line.  The GND and VCC pins (3.3v) on that cascade port can also be tapped to power other sensors you connect to the data logger.

The SD Card Adapter:

This SD card module comes with small surface mount pullup resistors on the MOSI, MISO & SCK (clock) lines (which have already been removed from the dashed red line area in the diagram).  The Arduino SDfat library uses SPI mode 0 communication, which sets the SCK line low when the logger is sleeping. This would cause a constant drain (~0.33mA) through the 10K SCK pullup on the module (second smd from the bottom) if we did not remove it.  I prefer to pull MOSI & MISO high using the internal pullups on the ProMini’s 328P processor, so those physical resistors on the breakout board can also be removed. But be careful to leave the top-most resistor of the four in place to pull up the DAT1 & DAT2 lines. This keeps those unused pins on the μSD card from floating when the cards are accessed in SPI mode.

sd1 Only remove the bottom three pullup resistors. keep the top one sd3

Then add a layer of double sided tape to the bottom of the SD card module and trim it to size.

Dupont Connectors:

trimdupontcableThe RTC & SD card modules will be connected to the Arduino by custom jumpers that you assemble from a 20cm, 40pin F-F Dupont style ribbon cable and 0.1″ (2.54mm) crimp connector housings. Test your cable for end-to-end continuity, then cut the cable into three sections.  Then peel off individual 10cm wires and insert them into the black crimp connector ends to create the cables shown. The longer 20cm wires can be used later when we assemble the LED indicator cable, and for other sensor attachments.

For the I2C connections:
Blue:          SQW alarm signal.
Yellow:     SCL (clock) (to A4)
White:       SDA (data) (to A5)
RED:          3.3v regulated
BLACK:    Ground
For the SPI cables:
RED:           3.3v regulated
Grey:          Cable select (to D10)
Orange:     MOSI   (to D11)
Brown:      SClocK (to D13)
Purple:      MISO   (to D12)
BLACK:     Ground

I’ve left one of the six RTC connector shroud slots empty for future use with the 32K clock signal but you can change that to a 5-pin housing if you wish.  Single 0.1″ (2.54mm) crimp connector housings are shown on the ends of the red, black and blue wires ,but I find that putting heat shrink over single-wire DuPonts help them hold onto the pins better.

ASSEMBLING THE LOGGER PLATFORM

Attach Battery Holders & solder in series

On a 4” PVC knockout cap, arrange your battery holders as close to the edge as possible to maximize room for other parts on the platform, without letting the holders hang off the edge. Mount the battery holders in place with double sided tape.

Join the red wire from the bottom of the single battery holder to the black wire from the double battery holder, so that the three AA batteries are connected in series. Fold the soldered joint and the long RED wire into the gap between the battery holders. Use some heat-shrink to bind them so they stay together inside the gap. Mark and drill 1/8” pair of holes so that you can tie down the wires from the battery pack neatly. Secure the wires to the cap with a zip tie through the holes.

Cave Pearl data loggers Cave Pearl data loggers Cave Pearl data loggers

Risers & Tie Downs

Place the RTC board alongside the batteries with one corner near, but not extending over, the edge of the cap. You want the back edge (with the four-wire cascade port) as close as possible to the single battery holder and the outer edge of the platform. The larger 6-pin connector side of the RTC module will be at a slight angle  to make room for the long connector housing that will be placed over those pins. Mark two places for the nylon standoffs and drill 2mm holes through the knockout cap. Thread the nylon standoffs through the holes and attach them to the platform with the plastic nut on the underside of the cap.

Then add two pairs of ¼ inch holes in the platform to be used later to tie-down the connector cables:

Cave Pearl data loggers Cave Pearl data loggers Cave Pearl data loggers

Battery Pack Connector

Join the two pieces of the Deans battery connector, and add solder to the end tabs of the female side of the square battery connector. For big connectors like this I turn the iron up to ~700°F (370°C). Be sure to heat the part long enough for the solder to “flow” like a liquid, and for the flux to burn off, but not so long that the nylon starts melting. Then trim the battery wires so that they extend to a mid-point over the battery holders on the platform. Strip about 5mm of insulation from the wires and tin the ends. Build up some extra insulation thickness to protect the wires from bending stress by adding two or more layers of heat shrink to the wires just before the tinned ends. Thread two larger pieces of heat shrink over the wires and then solder the wires to the connector and seal the heat shrink over the exposed metal:

Cave Pearl data loggers Cave Pearl data loggers batterycon3

Note: the top of the ‘T’ shape on this connector is the positive (red wire) side. Set your iron back down to ~600°F(315°C) after you get the solder on the big connector tabs. As a general rule: the smaller the component – the less time it can withstand heat. It’s not unusual to see maximum tolerances of 5 sec @ 288C for tiny SMD’s.

Attach modules to the platform

Layout of main components on the 4 inch knock-out cap.

Remove the tape backing from the pro-mini and the SD card adapter and affix them to the platform. As you can see the space is pretty tight. The SD card adapter needs to be close to the battery holder, because the SDcard will extend another 5mm from the holder itself, and this must not extend past the edge of the disk.

Then attach the RTC board to the nylon risers. Check to make sure there is good clearance between the RTC pins and the Arduino pins.

Connect the RTC

After the modules are placed on the board begin the interconnections by affixing the I2C cables to the RTC.  Then add the white & yellow two wire connector to risers on A4 (white) and A5 (yellow) of the promini board. Gather the white wires and pull them into the gap between the RTC module and the battery holders. Trim the wires as a group, so that their length just reaches the single battery holder. Then strip, twist & solder the ends of the wires together, and cover the join with heat-shrink.

Repeat the process with the yellow wires, and thread the joined wire sets through a cable tie that is routed from the bottom of the platform. Do not tighten the cable tie loop too much, as the black and red power lines will go through that later. To finish the RTC connection, plug a single blue wire to pin D2 on the Arduino [int(0)],  and connect it to the blue RTC alarm line as you did for the other lines. Wrap the soldered join with heat shrink and thread it under the cable tie with the I2C bus wires.

Connect the SD card

Now we will join the SPI lines which connect the SD card adapter to four pins on the Arduino.   Connect the 4-wire SPI cable to pins D10-D13 of the Arduino so that the grey wire is connected to D10 and the brown wire is on pin D13.  Attach the 6-wire SPI cable to the SD card adapter so that the black ground wire is closest to the pro-mini board.

Take each matching color pair of wires and fold them over beside the SD adapter board. Strip, twist & solder the ends together and add heat shrink to the join. Then bind them to the platform with a cable tie.

Join the Power Rails

The final step for the platform interconnecting cable is to join the “power rails”. Connect a single red jumper to the VCC pin on the Arduino compatible board, and add a single black jumper wire to the GND pin (behind the yellow wire on A5). Then gather ALL the black wires together and trim them to length along the same channel used for the three RTC wires.  Strip, twist & solder the black wires together.

Repeat the procedure for the red (+3.3v) Vcc lines. After soldering the ends of these bundles together, cinch them out of the way with the same cable tie that you left loosely holding the RTC connection wires earlier:

Battery Connection with Voltage Divider

Note:  The battery connection wires see a lot of handling compared to the other connections on the logger platform.  Since I have crimping pliers to add my own DuPont ends, this is one place on the build where I prefer to use silicone wire that is more flexible than the PVC insulated wire. 

With the main interconnect wires completed the last part of the logger platform build is the Arduino side power connector. Put the male battery connector in a vise with a spare female side connected to prevent the nylon from sagging due to heat, then add solder to the terminal flaps of the male battery connector. You might need to turn your iron up to 700°F to tin the connections the large connector, but turn it back down to 600°F (315°C) before attaching the voltage divider resistors, or you will cook those smaller components.

Build a voltage divider from two 4.7MΩ resistors, bridging across the low side resistor with a 0.1µF (104) ceramic capacitor. Add a black wire to the capacitor side of the divider, a pink wire to the center, and a red wire to the opposite side. Then solder this divider across the pins on the male battery connector, being sure to connect the black wire side to the ground pin, and the red wire side to the (+) pin.  Blow on the solder to cool it down as soon as the join to the connector pins is completethere is more than enough residual heat stored in the battery connector plates to cook the little capacitor, which then usually becomes a short.

divider3 divider2 divider4

This voltage divider will cut the battery voltage in half, allowing us to read the status of the 3xAA battery pack on any analog input pin (via the pink wire in the photos above) even though the main battery voltage is above the Arduino’s 3.3v Aref.  The 4.7Meg resistors are large enough to limit losses on the bridge (to about 0.6uA), and because they exceed the input impedance limit of the ADC, the capacitor is needed to provide electrons to fill the ADC’s sample and hold capacitor when a reading is needed. The battery voltage calculation for this combination is:   float batteryVoltage = float((analogRead(A0)/ 511.5)*3.3);  The default MIC5205 regulator found on most promini style boards has ~200-300 mv dropout during SD card writes, so when your AA battery pack gets down 3.65 volts, it’s time to shut down the logger.

Add heat shrink to protect the connections, and then plug the connector into it’s mate on the logger platform. Trim the black wire to length for the ground pin beside D2 (with the blue RTC alarm wire) , and then add a female crimp connector to that wire. Do the same for the pink voltage divider wire which gets attached to your choice of analog inputs from A0-A3, and the red battery wire which gets attached to the RAW input pin near the serial/UART terminal.

Cave Pearl data loggers divider6

Indicator LED

Since we removed the pin 13 LED from the pro-mini, we will now construct an external indicator. This is another place where soft silicone wires are preferred, but if you do not have them, peel away 20cm Red, Green, Blue and Black ribbon cable wires and solder them into place on a common cathode RGB LED, matching the wire colors to the correct pin. Then add stiff heat shrink around the connection points to protect the wire from flexing. Place the ends of the Red, Green and Blue wires into a 3-connector housing.  The R-G-B connector attaches to digital pins 4-5-6 on the Arduino.

Cut the black wire near its mid-point, and insert a 20K resistor to limit the current through the LED. I often replace the bottom half of the wire with a softer flexible length of wire with a MALE crimp connector at its end. The male ground pin connects to the ground pin beside the D2 connector.

LEDwires

The MAIN LOGGER PLATFORM is complete

Typical pro-mini loggers built with this design sleep at 0.25mA, before extra sensors are added. At that current draw, the logger should deliver at least eight months of operation on three brand new AA batteries with a 15min duty cycle; depending on sensor load.

Now you can run testing utilities to verify that everything is working. But before you jump into that it’s worth keeping in mind that the entire construction can easily be disassembled for debugging if you suspect a bad connection somewhere, and it is even possible to pop the modules off with a small screwdriver and replace them:

Test continuity on suspect wires with a DVM.  If you carefully lift the little tabs holding the crimps inside the black housings, individual wires can be extracted from the connectors and replaced.  If your crimps don’t feel like they are securely in place when they slide over the riser pins – then you should replace that connector.

SENSORS & HOUSING

platform partsYou can build a robust, inexpensive waterproof housing for your logger from a couple of 4″ plumbing end-caps. I use Loctite E30-CL to pot sensors on the exterior of the housing and you can see a complete walk through of the process in the  Sensors & Housing post from last year. Note that I did not use the 4-pin Deans connectors you see there on this newer build because they are fairly expensive, and a few people have been having a hard time getting their hands on them. If you shell out for crimping pliers, you can make as many custom connectors as you need for pennies each.  Dupont’s are not as robust, but I’ve had loggers running for years with them, so they are not a bad place to start. If the male pins don’t mate tightly, put a thin layer of solder on them to make them a bit thicker. Quality varies quite a bit, and Pololu sells good crimp pins.

TESTING THE LOGGER

1. Test the LED
Connect your indicator LED, to pins 4(R) – 5(B) – 6(G), and the ground line to one of your spare GND connections. Open Blink with the LED connected to the Arduino. Change the LED to Pin 4. Verify and upload. The red LED should start flashing. Repeat with pins 5 and 6 to test green & blue respectively. Note that with the common ground connection, you can not light up two colors of the LED at the same time.

Make polarized (non reversable) connectors by mixing Male & Female connectors

You can make non reversible sensor connectors by mixing male & female pins in the same housing. But never use male pins on wires that supply power to a sensor circuit, or the connector could short out if those pins come into contact with a conductive surface.

2. Test the I2C bus
Use an I2C scanner  to determine if your devices are responding on the I2C bus. With the scanner running, specify P to return to output only the addresses that have devices, and specify S to run a Single Scan. The 4k AT24C32 eeprom on the RTC board is at address 0x57 and the DS3231 will show up at address 0x68. The 32K eeprom should show up at 0x50 and other sensors will show up at different addresses. Note that the 4K AT24C32 is only rated for 100khz operation, but the DS3231 is rated for 400khz bus speeds.

3. Test the RTC
Use the settime & gettime scripts to set and check the time on the RTC. Generally, I set my windows computer to UTC before uploading setTime so that the logger is operating on UTC. This is not possible on a Macintosh, so you will have to check if London time is currently in sync with GMT/UTC, or find some other city that is. Alternatively you could avoid the problem by using the new serial setting script from Mr Alvin’s gitHub.

4. Test μSD Card communication
Insert the microSD in the adapter and test it with the CARDinfo utility at the Arduino  playground.  Remember to change the CSelect variable to 10 to get the CARDino to operate properly, as that is the pin we connected to the μSD card Cable Select line.

text

Loggers with the default pro-mini’s MIC5205 regulator usually delivers sleep current in the 0.21-0.26mA range depending on the SD card (Promini~0.07mA, SDcard: ~0.05-0.1mA, RTC~0.09mA) That should give you at least eight months of operation on 3xAA’s, and with minor modifications you can pull that down into the 0.15mA range to run almost two years.  (Or you could pass the one year mark by powering the logger with 4xAA cells in series. If you do this you will have to change the resistor ratio on the main battery voltage divider to keep it’s output below the 3.3v aref voltage on the analog input pins. For higher input voltages I  use a 3.3/10 Meg Ohm combination) Write the build date & your initial sleep current on the bottom of the logger platform – it’s a handy piece of diagnostic information to have later.

5. Test the EEprom(s)
A utility to test I2C eeproms was posted by bHogan at the Arduino playground forum, but that version is now quite old and does not compile in the current IDE without editing. I’ve posted an updated version on gitHub. Setting #define EEPROM_ADDR 0x57 will test the 4k eeprom on the RTC board, and if you install the extra 32k eeprom that shows up on the bus at 0x50. The serial window will return  A through T & 33 through 48 if the eeprom being tested is working OK.

6. Check Sleep Current
#include the LowPower.h library from RocketScream at the beginning of a blank sketch and the put the following line at the beginning of the main loop:

 

LowPower.powerDown(SLEEP_FOREVER, ADC_OFF, BOD_ON);

Then enable some internal pull-up resistors in to the setup section of your code:
(be sure to do this before you call sd.begin)

// Always pullup CS:
pinMode(chipSelect, OUTPUT); digitalWrite(chipSelect, HIGH); //pullup the CS pin to tell the SD card go to sleep
//and you may need to pullup MOSI/MISO lines:
//pinMode(MOSIpin, OUTPUT); digitalWrite(MOSIpin, HIGH); 
//pinMode(MISOpin, INPUT); digitalWrite(MISOpin, HIGH);   

Upload that sketch and your logger will be put to sleep as soon as the program runs.  Note: the CS line should have a pullup when using the standard SD libraries, and the MOSI/MISO lines may also need to be enabled to lower the sleep current with some SD cards. But SPI is a complex protocol, and you might have to turn off these pull-ups if you have other SPI sensors attached.

7. Build your datalogger code
A good place to start would be Tom Igoe’s analog pin reading example at Arduino.cc  ( be sure  const int chipSelect = 10; for the build described in this tutorial) and there is a nifty little function  for automatically creating log file names over on Adafruit’s site.  For something a little more advanced, I have prepared a basic data logger script that puts the data logger to sleep and wakes it up again based on timed alarms from the real time clock. These are all just starting points for you to add to as you learn more about programming an Arduino.

The first major thing to tackle to achieve a decent operating lifespan is buffering your sensor data to the 100 kHz 4k EEprom on the RTC board. Examples of the code I use for this eeprom buffering is embedded the I2C eeprom tester example I’ve placed on gitHub. You don’t want to wake up the card more than once per day, since that operation draws up to 100mA, for 200-400ms, which is more than any other run-time power use on the logger.  Older Sandisk cards (<1Gb) tend to have lower sleep currents around 70µA, but it’s not unusual to see newer large capacity cards drawing ~200µA or more during sleep.

You can extend that power saving strategy by adding a larger 32K AT24C256 EEprom to the logger platform. Because all I2C devices use the same bus wires, simply add the four EEprom connections to the bundles as you make them:

In addition to saving power by reducing the number of SD write events, the larger EEprom also extends the operating lifespan by allowing you to safely accelerate the I2C bus to 400khz, and this can reduce run-time duty cycle length significantly.   Because all I2C bus devices are connected in parallel, simply add the four EEprom jumpers to the appropriate wire bundles as you make them.  Alternatively, you could add a high resolution temperature sensor like the TMP102, or the MCP9808 to the logger in exactly the same way.

To fully optimize your code it is helpful monitor current during “logging events”, but this usually requires an oscilloscope.  In this post, I have outlined a method to use an UNO with the serial monitor built into the IDE to capture and display these brief events.

Addendum 2017-06-22

I wouldn’t bother for just one or two builds, but Dangerous prototypes just introduced a Dirty Cables service to order custom cables. Be interesting if I could build something like this logger interconnect wire with their tool. Might be a good way to throw a few kits up on Tindie…

Arduino Pro Mini Data Logger : Part 4 : Power Optimization (2015)

The first three tutorials in this series show how to build a promini based data logger that should sleep around 0.25mA, depending on the quiescent current of your sensors and your μSD card. That will usually get you to at least 6 months of operating life before a bank of three AA batteries in series falls below the 3.5v cutoff on your regulator, and most of our field units make to about 11 months on a good quality set of alkaline AA’s. With their flatter discharge curve, lithium AAs would probably have carried those logger past 12 months,  but if you really want the logger to pass one year of operation you need to do a few other things that might be a bit of a stretch for beginners.  So I am posting them here as “additional” things to tackle once you have built a few of the basic loggers and have them working Ok.  Use the cheap parts till you get the hang of soldering and working out how you want your cables & housing to go together physically.  I find that things usually go well the third time I make a new prototype.

A DIY data logger

Usually takes me about 20 minutes to assemble, or less with solderless headers. At any given time I probably have 6-8 of these breadboard loggers running on the bookshelf to test different hardware and code combinations. Be careful not to bump the SD card connections though, as its easy to kill the card with unexpected power interruptions.

Without question the most important thing you can do to extend the life of your data loggers is to build yourself a breadboard “testing platform”.  This lets you determine the sleep current for each component on it’s own, making it easy to spot fake SD cards, or bad sensor breakout boards. And even good SD cards wear out with time , or are  damaged by high temperatures. Checking that cards and boards go to into a low current modes properly (after the main mcu sleeps) is the best diagnostic I have found to determine if the components are Ok. Good SD cards sleep between 0.05-0.07 mA, and these tend to be older Sandisk 128mb cards. Typical cards pull between 0.07-0.09 mA while sleeping. Any more than than that and I simply do not use the card in my data loggers. The difference between a good sensor breakout board and a bad one can be even more extreme, and you should always look for breakout boards that have a native 3.3v input line to avoid regulator losses. I already mentioned pulling up three of the SPI lines, and the breadboard unit lets you easily test how other pin configurations affect the logger. Pin states are generally preserved during sleep, though any timer dependent functions (like PWM) will be shut down when the associated timer stops, unless you use ‘idle’ mode.  Always avoid floating I/O pins.

Retrofit an MCP1700 voltage regulator to an ebay clone

All you have to do to retrofit any 3.3v mini-style board with a more efficient MCP1700 is connect the external regulators output directly to the 3.3v pin, which is the main power rail behind the default voltage regulator. This by-passes the onboard Vreg the same way that your UART board does when you are tethered to USB. When I do this modification I completely remove the original regulator from the Arduino board so there is no leakage current slipping through it while the logger is sleeping.  Also be sure to move the high side of the resistor divider you have monitoring the battery voltage to the new regulators Vin line. Replacing the MIC5205 with a MCP170x will save you ~0.05 mA of sleep current, depending on your particular board. That doesn’t sound like much, but it all adds up over time, and you have 100 mA more current available to power your sensors.

When you test your components individually you notice that there can be significant differences from one pro-mini board to the next (especially with cheep eBay clones) and much of this comes down to the voltage regulator. Sparkfun Pro-mini’s use a Micrel  MIC5205 150mA LDO Regulator, and there are more efficient options out there. I have had success with boards that use MCP1700  & MCP1703 regulators (datasheet) like the Rocket Scream Mini Ultra or the Moteino. Each board has a unique pin-out, so you will have to figure out where to put the jumpers for each particular board.  You can also simply bypass the pro-mini’s on-board regulator and use an external voltage regulator (see the image posted at the bottom of that thread by fat16lib – don’t forget the 1 µF caps.  MCP1700s @ < $0.50 ea here ).

Sleeping your processor & components at every possible opportunity is vital. If there is a power wasting delay statement left in your code anywhere, there ought be a really good reason for it (like waiting for your ADC reading to stabilize, etc)

YL-90_AT24C32eeprom1

You can flick the SCL and SDA pullup off the board easily with the tip of a soldering iron.

The next life extending technique is to add a larger I2C eeprom so that you can buffer more data before you write to the SD card. The functions I use to do this buffering is included with the I2C eeprom tester example I posted to gitHub.   Eeprom power consumption limits to how far you can take this strategy, but switching from the 4K AT24C32 on that RTC breakout to a 32K AT24C256  provides a $1 way to extend your operating life by 5-10% depending on the amount of sensor data you are handling. The two eeproms are from the same Atmel family, and the wire.h I2C library limits you to 32 byte page writes, so all you have to do is change the bus address in your code and you are done! Same applies to the AT24C512 all the way up to the 2Mb AT24CM02 if you can find them. You can also lift & jumper the address pins (A0,A1) to enable up to four of these Atmel boards on the same bus. (and it would require some code tweaking, but you might also be able to swap some of the larger Microchip brand I2C eeproms into the cheap press fit DIP boards if you wanted bigger chips to play with. Or you could just roll your own breakout board  for a whopping 1024Kb with the 24LC1025 and the 24AA1025 ) The logical end game if you go down the eeprom road is to abandon simple ASCII and start working with struct’s in C.  Then you could store a years worth of data without any SD cards at all.

 .
Isopropyleeprom1. Remove the two pull-up resistors from the YL-90 breakout (if that’s the one you are using ) as the RTC board pull-ups are sufficient.  Straighten the riser pins and trim them to about 1/2 length.

2. Cut 2” lengths of Black, Red, Yellow, and White wire. Solder them to GND, VCC, SDA, and SCL respectively. Shrink wrap the solder joints.

3. Clean the board with alcohol and let it dry.  Apply conformal coating if desired and put a patch of double sided tape on the bottom.

jumpering the eeprom board

4. The next step is to adhere the eeprom to your logger platform via the tape, so that you can cut the jumpers to length. If you left some excess wire protruding from the board when you added the bus interconnect for the sensor cap, it’s pretty easy to patch the four eeprom wires right onto the cascade port.

 

Sleeping & buffering are the low-hanging fruit, and after that you get into trickier techniques to improve the power budget. For example that new eeprom also lets you push the bus speed above the 100kHz default if your other I2C devices can handle it. I am still testing prototypes at 400kHz, even though that violates the Tlow spec on 8mHz AVR processors, so I am cautious about recommending that to everyone until I see those units deliver a year of good data. But the results have been promising so far...

Cave Pearl data loggersAnother useful modification is powering the RTC from a digital pin. For some reason I have not been able to dig out of the data sheet, the DS3231 pulls almost 0.1 mA when it is powered by the Vcc pin. Fortunately, you can force the IC into a miserly 3μA timekeeping mode if you draw that Vcc pin down, and if the Battery-Backed Square-Wave Enable register bit is set the RTC will still generate alarms when running off of the backup battery. But the soldering for this is a bit tricky:

Cave Pearl data loggers1. After removing the power LED & charge circuit resistors from the RTC board, wedge a fine tweezer tip behind the power pin and apply an iron to the pad on the board. When the solder melts gently lever the pin away from the board.

2. Then tin the pin, and solder a jumper wire to the lifted power pin, being careful not to bridge any of the other connectors in the process. I usually secure the jumper wire to the board with a zip-tie so that no physical stress can be transferred to that tiny solder connection later.

Cave Pearl data loggersAt that point you can attach the jumper to a free digital pin on your Arduino, and use digital.write(pin#,HIGH/LOW) to power to the chip only when the logger is awake.  My pin-powered builds have eight months under their belt now, and by spring 2016 I will know if the CR2032’s can provide enough power to drive the RTC in timkeeping mode for a full year. (2016 note: they all made it!)  I am trying to track the coin cell status with another voltage divider on the breakout board, but since lithium cells keep their nominal voltage until they are completely dead unless you provide a load, that record might not give me any useful information. I will post updates on those experiments on my RTC page as they become available. (Note: Even with the default MIC5205 reg on the promini, pin-powering the RTC like this should get your logger down to ~0.17mA sleep current)

By testing components, changing regulators, buffering, and pin powering the RTC, I am now seeing power curves like this:

036_inCave_PR&RH_20150324-0806

This unit had a TMP102 temperature sensor,  an MS5803-02 pressure sensor, and an HTU21D RH sensor attached, and it still took four months to burn off the over-voltage on 3 AA Duracells. (with a 15 minute sample interval) The logger slept at 0.15 mA because all those sensors all have great low power sleep states.

Though I have reached my original one-year goal, I still keep an eye out for other ways to save power. Several people have explored using a MOSFET to de-power the SD cards but according to the fellow who actually wrote the SdFat library, there are some issues wrt multiple re-starts of the SD library . He also warns that you have to close all files and allow at least one second for the SD card to power down before pulling the plug, just in case you accidentally trigger some internal housekeeping event with the file close command.  The folks over at OSBSS claim they can switch the low side without problems and Nick Gammon seems to be having success with his THL logger switching the high side, though those two examples leave me wondering which way to go.  Some set all the SPI lines HIGH & INPUT before powering down to prevent parasitic leakage after the cut, though the guys at Solarduino imply only the slave select line is vulnerable to the problem, and suggest that line won’t leak if you switch the low side (?)

Another potential factor is low 3.3v I’m using to control the FET, discussed here at  CMicrotek’s Low-power Design Blog:

“When using a P-channel FET to drive a load, a GPIO may not drive the gate high enough to completely turn off the FET so you may be leaking power through the FET. This can often go un-noticed since the amount of power is too low to activate the load. A P-channel FET of similar rated voltage and current as an N-channel FET will typically have 50-100% higher Rds(on) than the N-channel FET. With Rds(on) specs on modern FETs in the double-digit milliohm range even doubling the Rds(on) produces a fairly low value. However, that is simply wasted power that can easily be eliminated if low-side switching is an option for your application.”

Reading that makes me lean towards low side switching; though the 2n7000s in my parts bin probably can’t be fully turned on with only 3.3v on the base?  (Note:  there are beefier N-channels out there  that will work with a 3.3v system if you are also de-powering high current sensors)  Luke_I has proposed using SPI.end() to kill all SPI communications once all files are synced and closed but most people simply set the SPI control register SPCR=0;   When you restore power  after low side switching you have to reset SCLocK(D13), MOSI(D11) & CS back to OUTPUT (MISO stays at input) and set the SCLock line LOW to disable that pullup.  Then you reinitialize the SDcard with sd.begin(chipSelect, SPI_FULL_SPEED);  and reopen your files.   Several sources have suggested that only SdFat allows this re-initialization , while SD.h does not.  

One possibly important thing to note is that the sandisk datasheets states (pg 11) 

“Power must be applied to the VDD pin before any I/O pin is set to logic HIGH. In
other words, CMD, CLK, and DAT0-3 must be at zero (0) volts when power is
applied to the VDD pin. “ 

But perhaps this only applies when accessing the cards in their native SDIO mode? But this does make me wonder if you are supposed to turn on the mcu’s internal SPI peripheral before, or after you restore power to the SD card (?)  I think that datasheet suggest you should be doing it after, but they are switching the high side in that example case.

You can extend that strategy and cut power to the entire logger. As I mentioned in the RTC post, the most elegant way to do this would be using the RTC alarm (outputs low) to control a P-channel Mosfet on the high side of the main battery supply.  When the INT/SQW alarm goes low, this turns the mosfet on and powers everything including the main mcu board which would then goes to work taking samples and storing data. Unfortunately some of my builds use interrupts from both the RTC and the sensors, and there is a good chance that with these frequent startup initializations the resulting delays could miss the phenomenon I was actually trying to capture; like tipping bucket or wind sensor reed switches.

Addendum 2016-02-06

Just thought I would post another shot of a retrofit with an MCP1700 voltage regulator:

Cave Pearl data loggers

Note: I used a 6 volt input MCP1700 here, but the MCP1703-3302E/TO accepts up to 16v

…with the addition of a 2 x 4.7 M‎Ω  voltage divider to put 1/2 of the RAW input voltage on A0. Note that the onboard vreg has been removed (normally you would see it just under the cap in front: you can see some of the empty pads still there) and the two led limit resistors have also been pulled. Other than the mcu there is not much left but the crystal & some caps, and I now think of the board itself as simply a convenient breakout for the 328p. This retrofit with an MCP1700 looses the shutdown enable functionality of the default MIC5205 (marked KBxx or LBxx), and the noise suppression features, but I am hoping the caps are enough to deal with that.  The 1700’s are more efficient at low power than the 5205’s which limp along at about 20% efficiency below at 0.1mA, and loggers typically spend 99.9% of their time sleeping.  The XC6203E332PR is another low-ish standby power option if you need output currents in the 400mA range.

On more recent builds I have started moving the voltage regulator & battery divider from the Arduino board to the battery connectors. Note the ceramic 105's on the MCP1700 as per the proper spec.

On late 2016 builds I started moving the  replacement voltage regulator & battery divider away from the Arduino to the battery connector. Given that this is now separated from the caps on the main board, I’m using ceramic 105’s on the MCP1700 as per the spec. Three wires run from this deadbug back to the pro-mini: 3.3v, GND, and 1/2 battery voltage from that 2x10M divider, which gets read on an analog pin.

Keep track of the capacitors if you change the voltage regulator, as the ones on the outgoing side are still connected after you remove the original reg. Sparkfun promini’s have a pair of black 10µF C106 smd caps on either side of the MIC5205 (& an extra 0.1uF on the output side). Several of the clones I checked match that layout, but others had a pair of 0range & brown tantalum 475C SMD capacitors (4.7µF). The MCP1700’s call for a fairly small 1µF on either side. Since my loggers are powered by a relatively large battery with little chance of brown out and no input noise, I have not been too worried about adding a big cap on the input side (often people add up to 100uF..?). I could probably get by with the output side caps already on the Arduino board, but I have been adding the ceramic 104’s (0.1µF) anyway, as I’ve seen a few forum posts suggesting that you get better noise suppression by using multiple capacitor chemistry-types with different ESR on outputs. And I am still careful to add 0.1µF ceramic bypass capacitor from +V to GND on every sensor IC.

If you combine a pin powered RTC, with the MCP1700 retrofit, and you have a good SD card, it’s not unusual to see the  logger platform sleeping below 0.1 mA, even for units built with cheap eBay clones. The pro-mini style board is only responsible for about a third  of that after the retrofit.

Addendum 2016-04-21

It would be a good idea to re-read Nick Gammons post on Power saving techniques for microprocessors, as much of what I’ve done to optimize these loggers is covered in more detail there.  For folks who want to take it farther, there’s good background info on low power design in Granssle’s article: Issues in Using Ultra-Low Power MCUs. And Hackaday’s post on TI processors shows how far the art in low power operation goes. 

Addendum 2016-06-04

Kevin Darrah has been posting some brilliant you-tube tutorials on lowering the power consumption of an Arduino. This includes a kill power circuit, which has been on my to-do list for quite some time now. Definitely worth a look. The rub of course is that you then have power-up latencies for everything.  That’s probably around 75-100 milliseconds for a typical Arduino, SD card initialization would probably be about the same (ie about 5 milliamp seconds) and a couple of sensor inits could easily double or triple that total.  So you could burn up to 25 milliamp seconds for the restart. Even with a sleeping SD card drawing power, a logger built with the optimizations discussed here usually sleeps at 0.14 mA or less. So a regular startup is probably equivalent to ~3 minutes of sleep time, and it would take several more minutes of sleep time power to match a really bad 1 second startup if you had some pokey sensors. So it all depends on how much time you need to get everything operational.  As best I can tell, I am still getting operating life that compares well to some of the power down approaches people are using.

Addendum 2016-10-17

Another tip for saving power is to always use high output LEDs, and to use the green as a status indicator color whenever possible. Why? If you look at the typical rating for a high output RGB you see Luminosity numbers like: 800R, 4000G & 900B mcd.  That means that you can use a limit resistor that’s three times larger on the green channel for about the same light output. I often get away with 30K on the ground line, and an extra 20-30K on the green line of a common cathode LED, and I’m still able to see the indicators with pulses in the 10-15 ms range.  With that much limiting resistance, the LEDs don’t impact the power budget at all. LEDs tend to stay lit for a relatively long time after they are turned off so switching the LED on/off with a 50/50 duty cycle at a rate faster than 1Khz will cut the power by half with an imperceptible reduction in brightness. While I would never leave an led on all the time for a datalogger application, if your application needs a power on indicator, consider a slow flash of the LED (for ½ second every 3 seconds)  instead of having it on constantly.

Addendum 2016-10-28

With my hardware reaching reasonable sleep currents, I guess its finally time for me to look at reducing the run time power use by turning off unnecessary peripherals with the power reduction register.  Heliosoph posted tests results from his capacitor powered project, with a reminder about grounding unused pins.  There are some 3.3v numbers over at avrProgrammers and Nick Gammon weighs in with some sage advice on his forum. Each peripheral doesn’t use much on it’s own, but together they total up to ~1mA that is just being wasted.  Here’s a few of the things I’m currently experimenting with:

  1. Disable the digital input buffers you aren’t using with DIDR0
  2. Never leave floating pins – always use a pullup or pulldown resistor.
  3. Disable unused module clocks using the PRR register
  4. Especially the ADC with ADCSRA &= ~_BV(ADEN);
  5. Shut off the analog comparator: ACSR |=_BV(ACD);

Here’s how those look in code:

#include <avr/power.h>
//defines for DIDR0 setting
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif
// create variables to restore the SPI & ADC register values after shutdown
byte keep_ADCSRA;
byte keep_SPCR;
// then be sure to store the default register values during setup:
keep_ADCSRA = ADCSRA;   keep_SPCR=SPCR; 

//  1) where the ADC pins are being used, disable the digital input buffers 
//  from http://www.instructables.com/id/Girino-Fast-Arduino-Oscilloscope/step10/Setting-up-the-ADC/
sbi(DIDR0,ADC0D);  
sbi(DIDR0,ADC1D);  
sbi(DIDR0,ADC2D);  
//sbi(DIDR0,ADC3D);  //A3 not used as analog in
//sbi(DIDR0,ADC4D);  //A4= I2C data
//sbi(DIDR0,ADC5D);  //A5= I2C scl
//not needed for A6&A7 because they have no digital capability
//2) set unused analog pins to digital input & pullup to prevent floating
pinMode(A3, INPUT); digitalWrite(A3, HIGH);
//3) And pull up any unused digital pins:
pinMode(pin#, INPUT_PULLUP);

// Disable internal peripherals that you are not using:
// Note to self: Don’t mess with timer 0!
 power_timer1_disable();              // (0.12mA) controls PWM 9 & 10 , Servo library
 power_timer2_disable();             // (0.12mA) controls PWM 3 & 11
  power_adc_disable();                  // disable the clock to the ADC module 
  ADCSRA = 0;                                 // disable ADC by setting ADCSRA reg. to 0  (0.23mA)
  ACSR = B10000000;                  // disable comparator by setting ACD bit7 of ACSR reg. to one.
  power_spi_disable();                   // disable the clock to the SPI module
  SPCR = 0;                                        //disable SPI by setting SPCR register to 0 (0.15mA)
   #ifndef ECHO_TO_SERIAL
   power_usart0_disable();   //(0.08mA) but only if your are not sending any output to the serial monitor!
  #endif
// power_twi_disable();    // (0.18mA) I don’t usually turn off I2C because I use the bus frequently

// Shutting down ADC & SPI requires you to wake them up again if you need them:
// Before any SD card data saving:
power_spi_enable(); SPCR=keep_SPCR;  delay(1);
// Before ADC reading:
power_adc_enable(); ADCSRA = keep_ADCSRA;  // & throw away the first reading or two…

Still to explore:   Lower the CPU clock & then using a lower voltage power source.

Slowing down the processor doesn’t help much if you are trying to minimize processor up time,  but if you get into a situation where you are forced to keep the Arduino awake,  you could try to reduce the power consumption by slowing the main system clock.  I’m not brave enough yet to make new board definitionsmess with the CLKPR system prescaler, since I have so many sensor communication events.  Delay and millis I can deal with, but Timer0 PWM, ADC and probably a few other things have to be set in conjunction with CLKPR to keep them running at the right speed.  Slower clocks let you reduce the whole system voltage, but I still need to 3.3v because of the sensors & SD card.   Fuse setting still seems like the dark arts to me, and I’m not sure shutting down the BOD is a great idea for data loggers…

Addendum 2016-12-05

After you apply the optimizations described above the sleeping µSD card becomes the biggest power drain left in the system.  So I should probably add a reminder here that there is a huge difference between SD cards that sleep well, and those that don’t.  Sandisk cards smaller than 1Gb tend to have lower idling/sleeping current modes ~70µA, but it’s not unusual to see new large capacity cards drawing  200µA(I’ve seen spec sheets floating around the web for special  TwinMOS mircoSD cards, listing incredibly low sleep & write currents, but I have never managed to find any for sale.)   Cards between 64-256mb usually give me the lowest sleep currents, but because those older cards have to be purchased from eBay, there is also a significant difference in their speed when I test them with H2testw, with some being slower than 2.0MBytes/s, and others saving at 5MBytes/s or better. I presume that is because used cards get slower due to the onboard wear leveling circuitry working to avoid the accumulated bad spots, and the SD latency spec is worse than all of those, allowing a card to take as long as 250ms per write event. The default SD library in Arduino usually writes data at about 4500-5000 bytes per second, and even if SdFat performs significantly better even the slowest SD cards are much faster than the Arduino.  And my gut feeling is that the relatively slow I2C bus coms to the EEprom I’m currently using are sand-bagging the system so much that whole question of SD speed is moot until I start using SPI EEproms, or faster Fram

The point is, always try several different cards with your logger, so you can reject ones that do not sleep well.  With a good card you can get an optimized build below 0.1 mA between readings, but a bad sleeper will easily bump the logger back up into the 0.2 mA range, cutting your operating life in half.  It’s also worth remembering that there is a world of difference between how often you see problems with consumer grade vs industrial grade SD cards.

With regards to runtime current:  always format SD cards using SD Formatter, as OS level formatting utilities may adversely affect performance with the SdFat library. According to William Greiman (author of the SdFat library) “Performance of microSD cards is not easy to predict on Arduino.  Arduino uses SPI to access the cards and the SD library has a single 512 byte cache. Modern SD cards are designed to be used on the high speed 4-bit SDIO bus with very large (16 KB or larger) multi-block writes and reads so they must emulate the single block access that Arduino libraries use.  This can mean much internal data movement, erasing of flash, and rewriting of data.”  Once again, older, smaller SDcards suffer less from this problem than larger size cards, and it’s probably a good idea to try to match your data saves to that 512 byte block size.

Addendum 2017-01-20

One extension of getting your loggers down to a low sleeping current is that it becomes possible to power your data logger with solar cells, especially with 500F super caps  available for $3 each on eBay (& capacitor balancing boards at $1.50). A commonly seen conversion is 1Farad = 0.277mAh /V, so you can think of that 500F cap roughly equivalent to a 138mAh battery – somewhat similar to the capacity of a CR2032 coin cell.  Ignoring the pulse discharges for sampling, I’m seeing sleep currents in the 0.1-0.2mA range, implying more than a week of operation with that kind of power.   David Pilling has done some interesting experiments using cheap garden solar cells with super caps.   So has Nick Gammon, and the guys over at Solarduino.net and heliosoph.mit. 

The real question is how to get the logger to recover gracefully from a brown-out event if the sun goes away for an extended period of time, and then comes back and is able to charge the caps up again…

Addendum 2017-05-21

Well, I finally took the plunge and started cutting power to the SD cards: Switching off SD cards for Low Power Data Logging.   I left that step for last because as I was being cautious about anything that might put my precious data at risk, but so far (and I am still testing the heck out of it) is seems to be working well.

Addendum 2017-12-22:

It can take a reasonable amount of soldering & code Kung-fu to implement the power optimization methods described above, and if you haven’t quite reached that level the Adafruit TPL5110 Low Power Timer provides a alternative approach to power management for only $5.  Cfastie has been putting this board through it’s paces over at PublicLab.org.

Arduino Pro Mini Data Logger : Part 3: Sensors & Housing (2015)

Build Instructions – Part 3:  Sensors & Housing

Note: This is the third tutorial in a series providing detailed build instructions for a DIY data logger based on a Pro Mini style Arduino board.  Part 1 of this series covered preparation of the 3 core components for assembly, and Part 2 described connection of those components into a functioning logger. If you are landing on this page for the first time, it might be a good idea to start at the beginning of the series.

Status indicator: common cathode RGB LED

1. Test the common cathode LED with a 3v CR2032 coin cell battery by connecting the GND (usually the longest wire) to the negative side of the battery and each other wire to the positive side of the cell in turn. Note which lead is for red, green, and blue.  Usually the red lead is the one closest to the FLAT side of the led, and the ground pin is the longest lead but there is variation between manufacturers, etc.

LED12. Trim and add a 30 K ohm limit resistor to the GND line. On some RGB LEDs, green is 2-3 times brighter than the other colors, so you can add a second 20-30K ohm resistor to the Green LED line if you power budget is tight, although this will make the green indicator hard to see in full sunlight.

 

LED23. Cut 4” lengths of Red, Green, Blue, and Black wire. Solder them to the respective LED connections and use shrink wrap to protect the joins. Carefully bend the leads to a 90 degree angle to facilitate mounting in the housing cap later.

 

 

LED34. Trim the LED wires to the same length, then strip, twist and tin the ends.  The LED wires will be connected to the male side of a black WSD1241 Micro 4B plug connector, maintaining the same R, G, B, & GND pattern used on the corresponding logger side connector.  Make sure that the other side of the connector is attached while you are soldering so that the plastic doesn’t melt out of shape.

 

LED complete

You can connect and test your completed LED with your logger platform running the Blink sketch, by replacing the default pin number 13 in the code with 4, 5 & 6 respectively.


I2C Sensor Breakout Boards

Many sensors are available on I2C breakout boards. This guide shows pictures of several including an Adafruit MCP9808 temperature sensor and a MS5803-05 pressure sensor, but the four standard bus connections (VCC, GND, SCL & SDA) would be the same for any I2C sensor and they all get connected in parallel. The DS3231 RTC breakout board in this build puts 4.7k pull-ups on the SCL & SDA lines that we tap at the cascade port. So if you are using multiple sensors you may need to remove the pull-up resistors from some of your sensor boards because those resistors also end up in parallel, and this can place too much pull on those bus lines for the sensors to overcome. If you are only adding one sensor to your logger,  you can usually get away without bothering to remove them.

jumperedI2Cboards1. Cut 3-4” lengths of Red, Black, Yellow, and White wires. Strip, twist and solder those wires to red=VCC, black=GND, yellow=SCL, and white=SDA holes respectively. I find that this is much easier to do with the board in a rubber footed panavise, and the wires supported by ‘helping hands’. Where space is limited, I sometimes mark the wires with a sharpie so I can keep track of which leads go to which sensor from the underside of the pvc cap.

2. Trim and clean the soldered side of the board with isopropyl alcohol to remove flux residue. Trim the free ends of the wires to the same length, and strip & tin the ends. We will be adding a 4 pin deans connector later, but for now leave the ends of you sensor wires free. The sensors will be tested once the wires are inserted through the housing cap, and after the sensors pass the testing stage the connector will be attached to the leads


Mounting sensors on the housing

This part of the build should be done in a well ventilated area (ie: outside or in fume hood if you have one) as pvc solvent fumes are toxic. Make sure your sensor boards have been completely cleaned of any flux residue with q-tips and 90% isopropyl alcohol before potting them!

SensorCap11. Once lead wires are attached to your sensors, select 1cm tall pvc rings of sufficient diameter (1” dia. & 2x ¾” dia. shown here) to contain your sensors and arrange them on top of a 4” PVC drain cap. Often it is handy to leave room for and extra sensor well, as you may decide to add another sensor to the cap later.

 

2. Roughen the surface of the cap and the inside of the PVC rings with sandpaper to provide some tooth for the epoxy that will be used to pot the sensors later.  Sand the lower edges of the pvc rings so that they  are level and form a tight fit with the top surface of the 4” end cap.

Solvent13. Clean both surfaces, and give the two surfaces that will be brought together a light coating with clear pvc primer, then put a thin bead of pvc cement on the lower surface of the pvc ring and bring it into contact with the previously primed area on the large end cap. It is important that you use ‘just enough” of the primer and pvc cement to bond the rings. If you apply more than a thin bead, the solvent will fail to bond.

Cave Pearl data loggersVery quickly after applying the pvc solvent cement to the ring, put it into place on the surface of the end-cap. You will need to apply firm pressure to each ring with your fingers for about 2-4 minutes until the ring is bonded to the cap surface. PVC cement usually cures to reasonable strength in about 30 minutes, but I like to leave my solvent welds to set over night because residual PVC solvent can interfere with the curing of the potting epoxy and/or turn it an ugly yellow color.

Cave Pearl data loggers4. Once all of your sensor well rings are in place you will need to drill holes for the wires to pass through the cap for the indicator LED and sensor wires.  A 7/32” bit produces a hole slightly larger than a typical LED, allowing you to insert the led from the inside of the housing.

This size hole is also a reasonable size to accommodate the wires from you sensor breakout boards.

 

 

LeadWires1PVC is generally a very easy material to drill, but beware that the cap can be wrenched out of your hand violently if the bit bites into the cap the wrong way. Also be sure to wear eye protection whenever you operate a mechanized shop tool. Drill bits often break while you are using them, and then the little bits of metal go flying in all directions. If you have never operated a drill press before, seek guidance from someone with experience.

I2C test leads5. With the LED leads inserted through the cap, neatly strip, twist & solder together the four corresponding I2C bus wires at their ends. To these exposed wires attach the ends of an alligator lead adapter cable (photo below) that has your standard I2C bus connector on one end. It’s a good idea to make some of these cables for all of the connector styles you are using on your project so you can connect raw sensor wires to your logger platforms for testing.

testing16. With the logger connected to your computer via a USB-UART adapter, run a generic I2C bus scanning utility like the excellent multi-speed scanner by Rob Tillaart to see if the devices are reporting an address on the bus. Then run test utilities appropriate to your individual sensor to confirm that your sensor boards are operating normally and delivering data.  The software you use will depend on which sensors you are testing. Libraries are often provided by better sensor vendors like Adafruit & Sparkfun, and the Arduino.cc forums are a good place to look for useful test utilities that others have shared.

It is essential that testing be done at this stage BEFORE you permanently pot the sensors into place with epoxy!

testing27. Once the sensors are confirmed working, you can solder them more permanently to the male side of your connectors. I usually use the red Deans Micro-plug connectors for the I2C bus lines from the cascade port on the RTC so that I don’t mix them up with the black connectors I use on the LED lines. Be sure to follow the same connection pattern that you used previously. I like the Deans because they are keyed:  I always put the GND and Vcc lines in the same position with these connectors, so that even if I make an error when connecting the sensor cap – no part ever gets exposed to a polarity reversal.

putty compressed8. The next step is to mount the sensors more permanently on the sensor cap. I do this by potting the sensor breakout board into place with Loctite E-30CL epoxy. (which remains liquid for at least 2 hours) so we first need to seal the wire pass-through holes in the housing.

Cut and knead a pea-sized bead of plumbers epoxy putty until it is well mixed (it will become slightly warm to the touch at that point) and wrap it around the wires underneath each sensor board so that it completely surrounds the wires. Then press the board and the putty wrapped wires into the hole in the housing until it forms a complete seal.  It is important that the sensor breakout board be relatively level, parallel to the top surface of the pvc end cap, so that the epoxy that will be applied later does not accidentally cover the parts of the sensor that need to be exposed to the air. The putty usually takes about 5 minutes to set.  Repeat this procedure for your other sensor boards and let your top surface putty harden (~15min) before proceeding to put putty in the inside of the housings.

LEDputty9. Cut and knead another larger bead of putting and wrap it around the indicator LED as shown. Before the putty hardens press the led through the appropriate hole from the inside of the housing so that the putty forms a complete seal against the roof of the housing. Smooth the putty until the LED wires are pressed closely up against the roof of the housing and let it set (5min).

Cave Pearl data loggers10. Cut and knead more beads of putty and press these up against the other sensor wires passing through the housing. The goal is to have the wires laid out flat and smooth, rather than emerging perpendicular to the top of the sensor cap where they would interfere with the logger platform.

Let the plumbers putty cure for 15 minutes before proceeding. It’s worth noting that it is possible to remove this stuff later on if you have a sensor failure, but the results are usually a bit ugly.

Cave Pearl data loggers11. At this point you have the LED and sensors mounted to the top of the top of the cap, but they need to be potted to become moisture resistant. With an applicator gun and mixer nozzles that allow flow control down to the single drop level, fill the PVC sensor well rings from the bottom with epoxy until the circuitry of the breakout board is covered. I usually use Hysol E-30CL but any epoxy or urethane with very low moisture permeability should work. I even have loggers sealed with JB weld, that survived six month exposures to salt water, but a clear compound lets you inspect the units better as they age.  Let any potting compound completely cure before exposing your loggers to the environment. I usually wait a week or more (while running power consumption tests) before deploying a new unit.

RH sensorMany sensors such as temperature, magnetometers, and MEMs accelerometers can be completely encased in the potting epoxy and function well. However some sensors, such as those for PRESSURE and RELATIVE HUMIDITY require that certain surfaces remain exposed to the air for them to function properly.

RH with dust capYou must be careful not to spill any epoxy onto those sensor areas or they will be ruined!  Fill the sensor wells with epoxy VERY slowly from the bottom and stop when the epoxy just covers the top edge of the sensor breakout board. Sometimes it is helpful to use a thin wire to paint the epoxy over the soldered breakout board traces while avoiding the sensor itself. The image above is from a HTU21 RH sensor and gives some impression of how tricky this can be to do well.  That sensor is approximately 3 mm per side. If any epoxy had entered the small exposed hole at the center of the chip the RH sensor would have been destroyed. (yes, I have lost a few…) Some sensors also require a covering cap, and it is generally easier to use plumbers putty to affix these protective fabric caps into place after the epoxy has cured, rather than trying to do sub-millimeter adjustments with liquid epoxy.

MasonsSensorPottingThe indicator LED is easier to deal with as it is already sealed inside a clear plastic housing, so it only needs enough epoxy to seal the housing pass through. For sensors on cables, I usually fill the wells to the very top to provide some protection against flexing. You may need to clamp those wires in place so they remain at 90 degrees to the housing while the epoxy cures.  In the first hour or so after pouring the epoxy it’s a good idea to check it periodically and break any large bubbles that rise to the surface with a pin. Slower epoxies often let most of the bubbles escape on their own.

Slower curing epoxies are generally more moisture resistant, but they can take up to 24 hours to harden.  It is worth noting that some epoxies contract while hardening, and I have noticed that this sometimes produces a permanent offset in my pressure sensors.

platform parts12. After the epoxy has cured, you complete the logger assembly with a Fernco PQC-104 4″ Qwik Cap rubber boot which will hold the logger platform in place inside the PVC drain cap.

 

This unit detects drips down to 12cm drip fall

The round bottom of this housing is pretty stable when resting on a flat surface, but you can mount the logger more securely by threading a few zip-tie loops around the metal pipe clamp before you tighten it to seal the housing.  Then you can use those loops as tie down points. A 10 gram silica gel desiccant pack tucks nicely under the knockout platform. Get the ones with indicator beads so you know when they have expired.


Post assembly testing:

1. Test the LED
Open Blink again with the LED connected to the Arduino (Remember to set board and com port). Change the LED to Pin 4. Verify and upload. The red led should start flashing.
Repeat with pins 5 and 6 to test green and blue respectively.

2. Test the I2C bus
I usually use Rob Tillaart’s multi-speed scanner to determine if the devices are responding on the I2C bus:  https://github.com/RobTillaart/Arduino/tree/master/sketches/MultiSpeedI2CScanner
With the scanner running, specify P to return to output only the addresses that have devices, and specify S to run a Single Scan. The RTC breakout board has an AT24C32 eeprom at address 57 and the DS3231 RTC will show up at address 68. Your other devices should report other bus addresses as per their data sheets.

Note:  the AT24C32 is only rated for 100khz operation, but the DS3231 is rated for faster bus speeds up to 400khz. Also note that the I2c address of the DS3231 can be changed by joining solder pads on the breakout board if you have an address conflict with your sensor.

3. Test the DS3231 RTC
I use the library from https://github.com/MrAlvin/RTClib because it allows me to use sub minute alarms during debugging, but there are many good DS3231 libraries out there to use. The setTime & getTime code examples that came with older versions of MrAlvins lib provide a good way to test your RTC functionality.

  • Open & upload setTime File/Examples/RTClib/settime (from the link provided above) to set the RTC to your computers current time. Generally I set my computer to UTC before uploading this program so that the logger runs on UTC. The arduino restarts every time the serial window is opened so do not launch the serial monitor when settime is running or you will create a delay offset in your clock time.
  • Open & upload getTime File/Examples/RTClib/gettime  (from the link provided above) to check that the RTC has been set. Expect about a 10-15 second offset between your computers time and that read back from the RTC, as a result of the time needed between compiling the code and executing it on the Arduino.

Note: After the time is set you can try to make a sketch that reads the temperature register from the DS3231 with the example provided by Coding Badly at: http://forum.arduino.cc/index.php?topic=22301.0   While the resolution of the DS3231 is a modest ±0.25C, I have been quite impressed with its overall accuracy when compared to other much more capable temperature sensors.

4. Test μSD card communication
Insert a microSD card into the carrier and test it with the CARDinfo utility at: https://www.arduino.cc/en/Tutorial/CardInfo  You will need to change the CSelect variable to pin 10, and I usually have to add #include SPI.h to get CARDinfo to compile properly. After uploading the sketch to the Arduino, information about card size, files, etc. are reported to the serial monitor reported if the card is connected.

Note: While CARDinfo is a very useful testing utility, it depends on the SD.h library that I almost never use any more. I much prefer Greiman’s SdFat.lib for loggers that are deployed as it uses less memory.

5. Test the Eeprom on the RTC breakout board
There is a utility to test AT style I2C eeproms posted by bHogan at http://forum.arduino.cc/index.php/topic,18501.0.html  Set the EEPROM address to 57 and the serial window will return written A through T and 33 through 48 if the eeprom is working.

Note: Because of latency delays, wear leveling, and other factors the SD card often uses a significant amount of the power budget for a data logger. Buffering your data to the eeprom is a good approach to improving your operating lifespan.  If you use page-writes, rather than saving data to individual locations in the eeprom you can store 28 characters of data with the same time delay as saving a single byte (a page is actually 32 bytes, but some gets used in coms overhead & for carriage return characters, etc.). So I think about that 4K eeprom in terms of “pages”, where my first 16 characters of the first page is always the time stamp: “YYYY/MM/DD,00:00”. That means if your actual sensor data only needs 10 characters of space you could store a reading in one “page” of the eeprom which would let you store 128 readings/pages in the 4K eeprom on the RTC board before you had to flush the data out to the SD card. If you have a fairly typical 15 min sample cycle (96 readings per day) you would be “waking up” the SD card less than once per day to write data, as opposed to 96 times per day. This kind of 100 to 1 reduction in SD card write events results in a substantial improvement in the performance of your data logger. I2C eeprom write cycles use about 3mA for 5-6 ms,  and this often keeps your arduino processor awake an drawing power as well (usually @ 4-5 mA). My tests to date with much larger 32K eeproms buffering five days worth of data have only lead to a 10-15% overall improvement in the power budget when compared to one day buffering.  So you do eventually reach a point of diminishing returns with the eeprom buffering strategy, but definitely take advantage of the eeprom on that RTC board if you can.

6. Test your other sensors
This will depend on which sensors you are using, and which libraries you have to work with.

Testing sleep current

7. The final and perhaps most important test is to check how much current the logger draws while it is in sleep mode. The LowPower.lib from Rocketscream is a good way to do put your Arduino into deep sleep modes. If you want to see that library in action, I posted a bare bones logger script for the UNO based logger tutorial on the project’s Github which will run fine on this build, and that is the most minimal example you are likely to find of logger that sleeps, wakes on alarm, takes a reading, and then goes back to sleep.  A typical logger built with Pro Mini form factor boards will draw between 0.20 to 0.25 mA depending on the attached sensors and how much sleep current your SD card draws.

A logger that sleeps just below a quarter of a milliamp usually runs for about eight months on a set of 3x AA batteries before falling below the 3.4v minimum for the voltage regulator.  If your sleep current is higher than that you should look at whether you have a bad SD card, or if you have a sensor breakout board has a voltage regulator wasting large amounts of power. Since you already have the pro-mini style board delivering regulated 3.3v, try to buy sensors that accept 3.3v directly on their pins so that you don’t waste power with other support circuitry.  Also try buy sensors that can be put into low current sleep modes easily. The really good sensor IC’s sleep automatically if there is no I2C bus traffic so you don’t even have to do anything for them. If you are using analog sensors, try to de-power any voltage dividers during sleep with a transistor or a logic level mosfet.

Also keep in mind that each AA cell in bank of three series cells can only go down to 1.15v before you reach the 3.4v minimum input for your regulator. If you look at alkaline battery discharge curves that implies that you are only going to have access to about 75% of the battery capacity. With 2000 mAh being fairly typical for a AA, that means you should conservatively estimate that your power supply will only deliver about 2000*.75= 1500 mAh. I usually get more than that from fresh batteries, and lithiums give you a slight boost because of their flatter discharge curves, but they also tank very quickly at the end of their lifespan, so be sure you check them before doing any SD writing.  If the power fails in the middle of a data writing event – you loose all the data on the card. 

Bumping these loggers up to 4x AA batteries in series should get you past a year even if your logger is on the high side at 0.25mA sleep current.

8. Program and run your logger

If you have made it through all the tests, then the next step is to run your logger with a basic bare-bones script and then when you are ready, you can dig into the other more complicated code builds from the project.  I’m not going to hold everyone’s hand through all that, however I will add that the content of these tutorial pages by Nick Gammon were immensely useful when I was starting out, and I still refer to them frequently:

    1. http://gammon.com.au/interrupts –  on AVR microcontroller interrupts
    2. http://www.gammon.com.au/forum/?id=11504 – on AVR timers and counters
    3. http://www.gammon.com.au/forum/?id=11497 – on Power saving techniques

Once you have digested that information, a good learning exercise would be to start with the bare bones script, and see if you could copy & paste only the parts that you need for your sensor out of the more complicated code examples. Keep doing that, one piece at a time, and eventually you will understand the entire thing. And there are plenty of other great datalogger examples out there if you Google around and many of them were created by people who actually know what they are doing, most especially the moderators at the Arduino.cc forums

With all that in mind, Part 4 of this series covers techniques for optimizing power consumption, but there is some more complicated coding, and trickier soldering, so its probably a good idea to build a couple of the basic loggers and get them running properly, before trying all the things in the final tutorial.

Arduino Pro Mini Data Logger : Part 2 : Logger Platform Assembly (2015)

Build Instructions Part 2:  Connecting core components on a logger platform

Note: In Part 1 of this assembly guide we covered the preparation of the three main components being assembled here. If you have not seen that page yet you should probably start there.

Batteries11. On a 4” PVC knockout cap, arrange your battery holders as close to the edge as possible to maximize room for other parts, without letting them hang off the edge. Mount them in place with double sided tape.

 

 

 

 

2. Join the black and red wires so that the three AA batteries are connected in series. Fold the solder join and the remaining red wire into the gap between the battery holders. Mark and drill 1/8” pair of holes to tied down the wires from the battery pack. Secure the wires to the cap with a zip tie through the holes.
Batteries2       Batteries3

standoffs13. Place the RTC alongside the batteries with one corner near, BUT NOT EXTENDING OVER the edge of the cap. Mark two places for the M2 nylon standoffs and drill 2mm holes through the knockout cap. Thread the standoffs through the holes and attach them to the platform with a plastic nut on the underside of the cap.

 

standoffs2

4. Place the RTC board on the standoffs and fasten it into place with the battery side down. Then carefully drill two 1/8” or larger holes near the cascade port of the RTC board and fasten the I2C bus connector cable to the platform. Leave some play in the wires to that they are not under tension when the zip tie is tightened

5. With the RTC mounted on the nylon risers, add a thin layer of solder to each of the connector pins on the end of the RTC board.

platform16. The next step is to use  double sided tape to adhere both the Arduino, and the SD card adapter to the platform. The SD card must not hang off the end of the platform (so it’s a good idea to do this step with the card inside the holder to see the spacing) and the Arduino must not extend off of the platform. Do not push directly on the metal part of the SD card holder or you could damage it.
platform2

7. Once the components are in place it is time to start soldering the SPI bus wires to the bent riser pins on the Arduino.  The trick is to hold the wire in place with tweezers, and trim each wire to length while still leaving a bit of play in the wires. Don’t cut them too short or you will not be able to solder the ends in place on the Arduino.

platform3With the colors we used on the SD card adapter earlier:

  • Grey wire is the spi CSelect line connect this to pin D10
  • Green wire is the spi MOSI line connect this to pin D11
  • Orange is the spi MISO line connect this to pin D12
  • White wire is the spi SCLock line connect this to pin D13

If you used different colors on the SD adapter board you will have to figure out which of your colors correspond to each SPI line and connect accordingly. (check the wiring diagram)  These can be left without heat shrink as they are tucked under other wires when the platform is completed, so an accidental touchdown is unlikely. Clean the joins with a Q-tip after soldering if there is a great deal of flux residue.

Cave Pearl data loggers8. Now pinch one of the GND wires from the Arduino and the ground wire from SD Card adapter together and hold them in place over the ground line for the RTC breakout. Cut them both to length, strip the ends and twist & solder both of the two ground wires together.

 

 

 

CutPowerlines29. Do the same with the red VCC wires so that the 3.3v regulated power line from the Arduino joins the VCC line and the power wire from the SD card adapter.

Using a pair of pliers to hold the wires in place so you don’t burn your fingers, solder the combined VCC and GND wires to the matching connector pins on the RTC

 

 

RTC connections10. Now bring the (white) A4 wire from the Arduino to the pin SDA pin on the RTC board. Trim this wire to length, strip, twist and tin the end of the wire. Then solder this wire to the SDA pin.
Repeat this process for the (yellow) SCL wire from A5, and connect the (blue) interrupt pin 2 wire to the RTC pin labeled SQW.

 

securedWires11. After drilling a couple of holes in the platform, secure the three Red Green and Blue wires from pins 4, 5, & 6 together with the second black ground line from the Arduino. Also secure the battery power leads from side of the Arduino board with drilled holes and cable ties.

 

LED connector12. Trim the LED wires to the same length. Strip and Tin the ends, then solder those the ends to the female side of a black WSD1241 Micro 4B plug in the order R,G,B-GND (with ground on the dimpled side of the connector).

As with the I2C connector pre load the pins on the connector with solder & thread the heat shrink over the wires first. Be sure you are using the female connectors on the Arduino side of these interconnects because they are the ones delivering power.

Trimpower313. The final step on the main platform is adding connectors to the battery holders and the Arduino input leads. Start by trimming the battery leads to a position about ½ way across the battery holders. These wires are can be fairly thin so reinforce their insulation with a few layers of heat shrink before trying to solder them to the pre-tinned battery connector posts. Before soldering these beefy connectors make sure the male side has been inserted so that the nylon does not soften and let the metal contacts inside shift around when you are applying heat. They really get hot during this process!

PWRcon1

14. Trimpower4Now measure the wires from the Arduino against the connector that is already in place on the battery holder side.  Use the same procedure of reinforcing the wires with 1/16” poly shrink wrap before connecting to the main power tabs and adding a final layer of 1/8” 3:1 heat shrink tubing over the connections.  The Arduino should be on the male side of the main power connector.

Test battery holderBefore trying to run your data logger from that battery module, it’s a good idea to test the battery holder connections with some AA batteries. If your 3x AA battery module is above the promini voltage regulator 3.5v minimum, you can connect the logger to see if it is still running the blink sketch. (provided you did not remove the p13 led)

At this stage the main logging platform is ready although the RTC clock has not been set yet.

Note: In Part 1 of this assembly guide we covered the preparation of the three main components that were used here. This post covered connecting the core components of the main logger platform, and in Part 3 we will cover mounting sensors onto the housing assembly to finish the data logger build. Part 4 covers techniques for optimizing your loggers power consumption, but some of them are more advanced so its probably a good idea to build a couple of the basic three component loggers before tackling everything there.

Arduino Pro Mini Data Logger : Part 1: Component Preparation (2015)

Note: In 2019, we updated the design with Dupont jumper cables to reduce the total build time to about 1 hour. We’ve also added support to the code for using the indicator LED as a light sensor, so you can start monitoring the environment with the logger right away.

https://thecavepearlproject.org/2019/02/21/easy-1-hour-pro-mini-classroom-datalogger-build-update-feb-2019/


The connection diagram for the Cave Pearl loggers follows this general pattern:

Note: This diagram uses the pin-outs of a Rocket Scream Ultra 8Mhz board with μSD adapter pin numbering convention per this document.  I like the Ultras as they use a more efficient MCP1700 voltage regulator, but the whole point of the design is that you can build one of these loggers with any 3.3v ‘mini’ style board you can get your hands on simply by matching the connections to the pin labels [ Figure from:  Cave Pearl Data Logger: A Flexible Arduino-Based Logging Platform for Long-Term Monitoring in Harsh Environments Sensors 2018, 18(2), 530; doi:10.3390/s18020530 ]

This tutorial will show step by step construction of that basic plan using the more common Sparkfun pro-mini style Arduino board, which has different pin locations than the Rocket Ultra but the connections are all the same. We will be assembling the logger on a plumbing test cap platform to facilitate placement in a 4″ housing made from plumbing parts:

Basic 3 component data logger with promini style boards

Note: A list of build parts & suppliers can be found HERE, and I will try to embed direct links into these instructions later.

Also Note: I posted a bare bones logger script for the UNO based logger tutorial on the project’s Github, and that is probably about the most minimal thing you can run on these loggers and it should be easy for folks to modify. To see a more developed code, with support for multiple different sensors,  you can dig into the other code builds for the project.

And Also Note: In 2016 I posted an alternate build of this logger that uses Dupont style jumpers. It’s cheaper, takes about 1/3 less time to build, and is much easier for beginners to assemble. However the connections are not as robust as a soldered build, so if you are willing to pay the “iron price” the one shown here is better suited to rough & tumble of real world deployments.

After assembling more than fifty of these loggers, it now takes me about 4 hours (not counting epoxy cure time) to go from raw parts to a completely functioning logger. However, experience teaching other people how to make them has shown that if you can get your first one together in about 8 hours, you are doing great. Especially if you have never soldered before!  The unit in the photo above was built with eBay parts for about $13, and the PVC housing in this tutorial will set you back another $8-10 (and not counting UART adapters, sensor breakout boards, or the tools you need like soldering irons or epoxy applicator guns).  There are plenty of simple loggers out there in the $30$60 price range for things like temperature, pressure, etc., so the only reasons to build your own are 1) because you are doing something that is not already available on the market at a reasonable price  (An example would be having your logger change it’s sampling frequency in response to interesting events: a feature like that is rare even at the high end of the market, though it is easy to do when you have access to the code) or 2) for the fun of doing it!  And don’t underestimate that second one: few things are as addictive as getting beautiful time series data from equipment you built with your own hands.

My wire color conventions are:
Red=positive power (both raw & regulated), Black=Ground, White=I2C Data, Yellow=I2C Clock, I usually use blue or green for interrupt signals, and other colors for the SD card SPI lines. I try to keep indicator LED leads the same color as the light they are controlling. I usually cut a bunch of 12cm wire segments and tin them all with a solder pot before starting a build. Adafruit has the best multi-strand 26awg silicone wire, but you can get by just fine with the 7-strand pvc stuff if the budget is tight.

Before starting:
Clean flux residue from breakout boards with 90% isopropyl alcohol. Ultrasonic cleaning  in alcohol for 2x five minute sessions (outside & away from sources of ignition!) works well but you can also do it by hand with a Q-tips, etc. Brand name boards from reputable vendors like Adafruit or Sparkfun are usually very clean, but cheap knock-offs from eBay will be covered with goop that is guaranteed to corrode inside your loggers over time. Do not use ultrasonic cleaning on parts that are sensitive to vibration,  like accelerometers,  or you could damage them.


Component Preparation:
Each of the 3 main components is prepared with riser pins & jumper wires before the whole logger assembly takes place.

1)   The Arduino ProMini style board

PinLayout11. Remove the limit resistor for the power LED to disable it. Usually you can’t see these indicator lights because they are inside your data logger housings, so disabling the them will conserve battery power.
I usually also remove the limit resistor on the pin 13 indicator LED, although on your first build you should leave it there for testing purposes.  My concern is that current drawn by the LED  (because limit resistors vary from one board to the next) might interfere with SPI communications to the SD card.

Riser Pins2. Solder the pin headers into place. Use two extra long pins for GND & RAW, and bend them 90 degrees under the Arduino board so that they protrude from the side of the board. Cave Pearl data loggersThese under board connections will connect with the battery pack, leaving the vertical risers from the GND & RAW pins available for the connection of a resistor voltage divider to monitor the battery. I usually bend on the serial I/O pins slightly away from the other risers, just to make a little more room for connecting the UART boards.

Cave Pearl data loggers3. At this stage it makes sense to bend and add solder to several riser pins. Bend digital pins 10-13 away from the main board for later connection to the SD card wires. Bend pin A0,  RAW & GND pins in toward the processor so that the battery voltage divider can be located above the board.

 

4. Then add solder all of the following riser pins to make connecting the jumper wires easier later on:

  • A0 & vRAW & GND   on one side of the board
  • Vcc (3.3v), the other GND pin on the far side (you can never have to many ground connections!)
  • D 10-13 where you will be joining the spi SD card jumper wires later
  • D 2-9 (you may not use all of them, but it is much easier to tin those pins at this stage than doing after the logger is assembled)
  • Sometimes I also tin the rest of the vertical pins in case I need to connect something to them later. This also depends on whether I plan on adding analog sensors to the logger.

Cave Pearl data loggers5.  A4 & A5 are needed for the I2C bus, but on pro-mini style boards these are somewhat inconveniently located wrt the voltage divider we use to monitor the main battery status.  Soldering a white jumper wire directly to the board at A4, and a yellow jumper wire to A5 allows us to breakout these connections. I usually strip, twist & pre-tin the wire ends before putting then through the board for these connections. Otherwise a stray bit of wire might not make it through the hole, and cause bridging problems.

6. Prepare two 4.7 Meg ohm resistors and 0.1 μF capacitor (code 104) as shown to form the battery monitoring voltage divider.  The idea is to have this divider fit neatly within the board footprint as you connect to the appropriate the riser pins. Solder the end with all three together at A0.

Cave Pearl data loggersCave Pearl data loggers

Be sure to use an ohm meter to find resistors with values as close to each other as possible before you solder them together. It’s easy to end up with two opposing % errors that throw off your battery reading even if they are from the same batch.

Cave Pearl data loggers

It is very important that there is an air gap between the RAW & GND leads of the voltage divider (on the right hand side), so that they DO NOT MAKE CONTACT with each other. This would short out your power supply! It might be a good idea to put some heat shrink in there too.

7. First solder the middle of the divider securely to pin A0. With that A0 connection in place, bend an ‘L’ into the lone resistor leg so that it lines up with the  vRAW pin you bent and tinned earlier (ie: the one closest to the serial i/o pins). Try to get this connection fairly close to the Arduino board to make some room for the ground connection which will go above it.  Then do the same procedure for the combined cap/resistor pin, soldering that onto the ground line riser pin. After the solder joints are in place, trim away the excess wire leads, being careful not to accidentally cut the riser pins.

Note: see this Jeelabs Post  for more information on this method to track battery voltage

8. With the board level soldering finished, give the pin solder connections a thorough cleaning with 90% isopropyl alcohol to get rid of any flux residue. After drying, waterproof the Arduino board with a coating of silicone conformal coating (or nail polish) being careful not to get any on the risers or the reset button. Let the coating dry in a well ventilated area before proceeding to the next step.  This coating is optional, but a good idea for electronics you are going to deploy in the field.

 At this point you will be soldering some 3-4 inch jumper wires to the pre-tinned riser pins. 26AWG silicone jacket wire from Adafruit is my current favorite, but any thin multi-strand wire will do.  I use thicker 20 or 22 gauge wire for the two battery power lines to make them more robust for the handling they see when the loggers are in use. Putting heat shrink over pin-jumper connections protects from accidental bridging when you place the logger platform into the housing. I advise that you use transparent tubing, as this makes it easier to spot a failed solder joint.  Clear milspec 1/16”/1.5mm polyolefin heat shrink tubing is an excellent option, as it becomes very stiff when it cools, providing protection from flexing and holding a ‘bend’ in your wires that can help with cable routing.

Cave Pearl data loggers9. On the side with the voltage divider add a red jumper to the VCC pin. This is the 3.3v regulated power line that will provide power to all of your devices & sensors. Then add thicker battery connector wires to the two bent pins extending from below the board. Cut all your jumpers to about 4 inches, and TIN THE STRIPPED WIRE ENDS BEFORE YOU TRY TO SOLDER THEM to the riser pins. Having both sides of the connection already covered with solder before you bring them together makes it much easier to get a good connection. A cheap solder pot can be picked up from eBay for ~$15, which makes the stripping&tinning process much faster. Use a good quality no-clean flux.

DigitalPinJumpers2 On the opposite side of the board:

GND:   Solder TWO black wires for the RTC & LED.
Pin 2:  Interrupt wire for RTC alarm line (Blue)
Pin 3:  Optional interrupt wire for certain sensors (Green)
Pin 4:  Red power line for the RGB LED
Pin 5:  Green power line for LED
Pin 6:  Blue power line for LED

BoardTape110. After applying heat shrink tubing to the joints (optional), apply double sided tape to the bottom of the Arduino, in two layers. Cut the first layer so that it fits in between the stumps of the riser pins on the bottom of the board. Peel back the plastic backing on that first layer and apply a second layer of double sided tape that extends to the outer edges of the promini board.

 

BoardTape2The Arduino is now ready for the logger assembly stage.

Note: This would be a good time to test whether the Arduino is working. Connect the pro-mini to a PC via a 3.3volt UART adapter and make sure that the ProMini board is selected in the IDE (Tools > Board > Pro or ProMini). You may have to specify the com port that your computer has given to the UART adapter (Tools > Port) and/or you may have to download a driver to support your serial communications board.

11. Run the Blink program (File > Examples > Basics > Blink) on Pin 13. Make sure to verify the program before uploading it. Assuming you did not remove the P13 limit resistor earlier, you’ll see the pin13 LED on the Arduino blink with one second intervals if everything is communicating properly and your Arduino is working.


2)  The micro SD card adapter

Four SPI signal wires will connect the SD card adapter to the Arduino board on digital pins 10-13. For these cut 2-3” lengths of four different colored wire. In this guide we used Orange, White, Green, and Gray wire. In addition, you will need red & black power wires, but these will not be routed to the Arduino. Instead they will tap the 3.3v regulated power and GND lines on the RTC adapter.

1. Flip the adapter board over and apply a thin coating of solder over each of the nine connector pads. Do not linger too long with the soldering iron making these connections, as heat conducted through the board may be enough to reflow the small contact springs soldered to other side of the adapter board. Usually if this occurs you hear a ‘snap’ sound while soldering. If this happens you may have to start over with a new board, as the adapter will not be able to make electrical contact with the inserted microSD card. I have found that the boards from Adafruit are reasonably resistant to this problem

SDAdapter12. Then add a 30 – 50K pullup resistor to the two pads at the end of the board. These connections are not used when the Arduino is accessing the card in SPI mode, but if you leave them floating some SD cards draw excessive amounts of power in sleep mode. Solder one end of the resistor to connection 1, bending it 90 degrees and laying the lead wire across the board to the other pad.  Be careful to route the resistor lead so that it does not come in contact with any other connection vias on your adapter board.  Fold the resistor along the cut edge and solder the remaining end to connection in the middle of the board (which is the 3.3v power line) I often put shrink wrap over the exposed resistor lead. I am probably breaking some rule tying the unused DAT1 & DAT2 lines together like this, but it has been working ok over several years of operation at this point. Use two pullup resistors for this if you prefer.

Cave Pearl data loggers3. The ground line needs to be connected to the SD adapter board in two places. Cut another 1” of black wire. Strip, twist and solder the black wires together on one end and solder that joined connection to pad. With the two-wire end in place, cut the short jumper to length for the second pad, then strip and tin the wire end before bringing down to the other ground line pad. Note: Many of these uSD adapter boards already have the two ground pads connected with an internal jumper. In fact the one in this photo did, but I simply forgot to check for that with a meter before starting this tutorial.

4. Add a red jumper wire to the pad in the center of the adapter with the pull-up resistor, being careful that you don’t unseat the pull-up resistor lead in the process. Then solder the remaining jumper leads in place.  For this guide I have used:

  • Cave Pearl data loggersOrange at pad2 will be the spi MISO line and will connect to D12
  • White at pad5 will be the spi SCLock line and will connect to D13
  • Green at pad2 will be the spi MOSI line and will connect to D11
  • Grey at pad1 will be the spi CSelect line and will connect to D10
  • Black at pad6 with extra tail to pad 3, this will connect to the RTC GND pin
  • Red at pad5 (with pullup resistor), this will connect to the RTC VCC pin

Cave Pearl data loggers5. After the wires are soldered into place, clean away any flux residue with 90% isopropyl and let it dry. Be careful when applying silicone conformal coating to the soldered connections, as it may run to the other side of the board and coat the sprung SD connection springs, and prevent electrical connection. Make sure to dry it with the holder side up to prevent any liquid from getting inside the card holder. After the coating is dry, apply double sided tape to the SD card adapter.

Addendum:

pullups added to the SD card holder

Here is an example of the SD adapter modified with 30K pullups resistors on  CS, MOSI & MISO. I used 1/8 watt metal films because their small size makes it easier to squeeze them in

Reliable sources recommend pullups on the SPI lines, though I have been running my loggers for a long time without them.  I have noticed that some SD cards take a really long time to go to sleep unless CS, MOSI & MISO are pulled up, and some just refuse to sleep at all without the pullups. These weak pullup resistors are not described in this tutorial, but I suggest that you add them to your builds to sort out this problem with some cards.

If you already have loggers built without them, there is a workaround using the 328’s internal pull-up resistors that I current have in testing.  Despite the fact that only CS requires it, I found that all three lines (w MISO set to INPUT not output!) had to be pulled up before misbehaving cards went to sleep properly. So add these lines to the beginning of your setup before you run sd.begin

// pulling up the SPI lines at the start of Setup
pinMode(chipSelect, OUTPUT); digitalWrite(chipSelect, HIGH); //pullup the CS pin
pinMode(MOSIpin, OUTPUT); digitalWrite(MOSIpin, HIGH);//pullup the MOSI pin
pinMode(MISOpin, INPUT); digitalWrite(MISOpin, HIGH); //pullup the MISO pin
delay(1);

I found that enabling these three internal 20K pullup resistors raises sleep current by between 1-5 μA, so it should not hurt your power budget, and could potentially save far more power by helping your SD cards go to sleep much faster.  It’s worth noting that there is some debate about the necessity of this approach. Because SPI is a complex  protocol, there are situations where adding physical pull-ups could actually interfere with other SPI sensors on your data logger, and in that case the internal pullups provide a more flexible option.  That is one reason why I tend to stick to I2C & analog sensors on my builds, so the SPI lines are dedicated to the SD card.


3) DS3231 RTC Breakout board

These cheap DS3231 boards usually arrive with quite a bit of flux residue that you may need to clean off of the board.  There might also be a plastic tab on one end that can be removed.

RTCresistors2remove1. This board has a lithium battery charging circuit that can be disabled by removing as small surface mount resistor (201) on the right side of the board. The LED power indicator can be disabled by removing the (102) limiting resistor on the left side of the board.

Cave Pearl data loggers2. The I2C lines that we will attach to one side of this breakout board are provided on cascade connectors on the other side of the board with 4.7k pullup resistors already installed on the SDA and SCL lines.  To that cascade connector solder 4” lengths of Black, Red, White, and Yellow wire on the GND, VCC, SDA, SCL respectively, so that they emerge the battery side of the board. Leave a few mm of wire protruding from the solder joins, as these provide convenient attachment points for an I2C EEprom if you wish to add that later.

I2C connectors3. To the end of these four jumper wires add the four pin red WSD1242 Micro 4R Plug.  Note the pattern used with the black GND wire on the dimpled side of the connector. (pattern: Y-W-R-Bl) Place the connector in a vise with the other side of the connector in place to prevent the pins from shifting when the thermoset plastic gets hot. Pre solder each pin, then cut, strip and tin each wire end BEFORE you try to join the wires to the pins. Thread the heat shrink tubing over the wires before you solder to the pins.  BEFORE joining the wires to the connector in the pattern shown check that the female side of the connector is being soldered to the wires from the RTC.

As a general rule I always solder connectors in such a way that the side that is providing power is female. That way even when the connector is ‘live’ there are no exposed pins that could short out if the connector accidentally comes in contact with a metal surface.

Also note that there is nothing special about the Deans micro plugs I am using here, they are simply my current favorite after testing many others on the market. They stay secure with a good deal of knocking about, but at ~$1.50 per pair they are also pretty expensive,  so feel free to substitute different connectors for your build. Dupont crimp connectors are much cheaper, and I used those successfully for quite a while before switching over to the Deans.

RTC34. Cover the IC side of the breakout board with a thin layer of conformal coating and insert a fresh CR2032 3V coin cell battery.

 

 

 

At this point the three core components of the logger are ready for the assembly stage. In Part 2 of the build instructions, we will cover connecting these parts together on a flat platform with the battery holders. And in Part 3 we look at assembling a waterproof housing. Part 4 covers techniques for optimizing your loggers power consumption, but some of them are more advanced so its probably a good idea to build a couple of the basic loggers before tackling the material in that last post.

The DIY Arduino datalogger with low power shut down capability (2014)

Addendum 2017-02-20:

The original $10 DIY Arduino data logger was posted in 2014, and there have been several significant updates to the way I assemble the basic three component design since the early version described here.  If you are building your first data logger you might want to start with this page:  How to Build an Arduino Data Logger ☀  which links to more recent build instructions.  As the build improved, I stopped using the Pololu power switch described here, as it ended up using more power than the sleeping loggers themselves.

Also note: The Pololu power switch described in this post has been superseded by the vastly superior Adafruit TPL5110 Low Power Timer.  Cfastie has been putting this board through it’s paces over at PublicLab.org.


Original post from  2014-10-07:

Here is the latest version of my basic DIY datalogger, which adds a new component to the design:

A3 component DIY data logger

Note: There is a set of detailed build instructions for a pro-mini based data logger from this plan at:  Part 1 Part 2 Part 3 

NoteAdapter pin numbering follows this document: 1 = Chip Select, 2=Data in = MOSI, 3 = GND, 4 = VDD (supply voltage, not > 3.3v!), 5 =SCLK , 6 =GND , 7 =Data out =MISO
Several sources recommend pullups on CS, MOSI, MISO and SCK, though the loggers will usually work ok without them. But testing has shown that some SD cards take a very long time to go to sleep unless CS, MOSI & MISO are also pulled up

The addition of the power switch solves the last nagging problem I had with the original 3-component design: unpredictable system behavior when the power supply goes low. I knew from my bench tests that some boards (like the Rocket Ultra pictured here) gracefully go into a kind of holding state till they completely brown out, while others go into a kind of power cycling loop, alternating off & on, when the battery voltage gets too low for the regulators. Now, you can just read the divider and drive the control pin high to shut down the entire system. Of course, everything is a trade-off, and this soft switch almost doubles the sleep current, from ~0.35 mA (with a good SD card!) to ~0.6 mA.  Assuming 2000 usable mAh per AA, this brings my “back of the envelope” estimate down to 4-6 months on three cells (my design goal is of 1 year of operation). But given all the uncertainties I have dealt with regarding μSDcard power drains, battery leaks, and the rough and tumble of real world deployments, I think I will pay that price to protect my data. It’s not always that easy to get back into those caves on schedule. On future builds, I will try to find a simple latching relay circuit to get back the capacity I’m losing with the Pololu switch.

The SD card debacle forced me to make my first breadboard version of these loggers to measure power consumption at the component level . All my previous builds were soldered, and now that I know how fast bread-boarding is (~45 min , mostly for soldering the header pins) I realize that was a mistake.

The SD card debacle forced me to make my first breadboard version of these loggers to measure power consumption at the component level . All my previous builds were soldered, and now that I know how fast bread-boarding is (~45 min , mostly for soldering the header pins) I realize that was a mistake.

And I also finally got around to posting the drip logger script for this platform on the project’s Github. Even though the drip sensors only use an ADXL345 (for now), I left in all the other sensor support developed for the flow monitors, as the system is meant to be easily adapted to other applications. I also have two different versions of sleep code buried in there, one with the Rocket Scream low power library, which shuts off the BOD, and the other which just uses a variable WDT timer to wake from sleeps while the fuses are left on. I still have not decided if I am comfortable with BOD fuse shutdown in a data logger application. With a 10k limiter keeping the led current low, I sometimes leave the indicator pips on during sleeps without impacting the power budget too much. If you change the limiter to make the led brighter, make sure you shorten the time that the led is left on.

From the beginning I have focused on creating a fully functional data acquisition system from a small number of preexisting modules. Only interchangeable, non-proprietary components and formats are being used (AA batteries, csv files, SD cards, etc.) because I want my logger to have the flexibility of replacing any individual component with half a dozen other parts that are readily available.  To give some sense of this, I have compiled a “budget” and a “preferred” parts list:

Budget Parts:

Component Comment Cost

CloneBoard
*Clones work, but the “real” Sparkfun ProMini boards give me trouble with the SD cards. I suspect that a 22uF  decoupling cap between the SD card’s Vcc<->Gnd  lines might help the MIC5205 cover the current peaks.

3.3v ProMini clone*: And occasionally you catch them for <$2.50 on eBay. Make sure Vreg is 150mA or more, and watch for Vcc> 3.3v.  Usually 0.05-0.2 mA > sleep currents than Rocket Ultra boards, pin layouts can be irritating, remove power leds & inspect solder joints very carefully. It’s worth noting that you can  bypass the onboard regulator, and use an external MCP1700 vreg. $4.00 (+guilt)
RTC board RTC modules: wash them as they are often covered with blobs of leftover flux, remove charging circuit & pwr led resistors. Check my RTC page for further info on these modules. $1.50
 flipupAdapter Micro SD card adapter: try to get the ones that have Molex connectors, you need to leave room for the flap in your design, or you could try the self-spring type but these don’t seem to hold up to the soldering heat as well. If you hear a “snap” during soldering,  start over – you killed one of the spring contacts. $1.00
 cheepbatholder 3 x AA Battery Holders:  and don’t forget the connectors! I usually go for the deans style knock-offs,  but expect to throw more than a few of them out due to bad alignment. You need a good soldering iron to use them. $1.50
 3_colorled 3 color LED module:  The old round-style Keyes modules are the brightest I have found so far with a 10k limiter. You could buy raw common cathode leds for less, but I find the little breakout boards make it easier to mount them with JB plastiweld putty $1.00
 plasticbox There are so many different pre-built waterproof boxes out there to choose from I’ve lost count. If you build your own from PVC, you can make them far more rugged. $3.00
The first time I deliberately purchased something with DRM. The Muve Music cards are the only ones I have found that are usually authentic when I get them on eBay. My longest logger runs to date (5 months) have generated less than 50Mb of CSV data. So 1 gig is plenty. Test your SD cards extensively before any deployment to make sure they go to sleep properly! $2.00
 cp202 You will need at least one serial adapter, and I usually bring several for fieldwork. CP202’s are cheap but don’t work as well for me as FTDI’s. Be sure to get 3.3v and note that Windows drivers will brick fake FTDI’s $2.50
Adafruit sells 26awg silicone wire in 2m lengths. Perfect for starting a new project without breaking the bank. Deans micro plugs are the best connectors I have found and they are available from 2 to 6 pins. Resistors, wires, heat shrink, etc. The cheapest stuff I have found so far is at Electrodragon, but you get what you pay for wrt quality & 3 week  delivery times. $1.00

Preferred Parts:

When the budget isn’t too tight, I splash out for better components. After you build a few, and go through the effort to install them, you will start to realize that your time is worth more. Certainly your data is. But I would still make your first few builds with cheaper stuff, at least till you feel confident putting them together. Cable routing inside a housing is important, and it usually takes me a few tries before I get the physical build sorted out.

Component Comment Cost
miniultra The Rocket Scream Mini Ultra wins over other mini format boards with lower sleep current & 100 mA more juice at the high end. I also like the layout, which has an extra ground pin available for the voltage divider. 1284P’s like the Moteino MEGA (+built in RF!) are looking interesting, but I have not had a chance to play with them yet. $14.00
chronodot The Chronodot is still my gold standard as a trustworthy RTC. But using it takes more space, and requires a separate EEprom. Of course that allows you to go up to a 256k  AT24C256  possibly eliminating  the SD card. Or try a little FRAM for faster writes. And for only $33 more you could go all the way to a FE-5680A atomic clock but good luck fitting that into your housing 🙂 $17.00
 adafruitmicrosd I trust Adafruit’s quality control.  This board lets me tuck the SD under the RTC in tight builds if I use 15mm M2 standoffs.  $2.50
 BatteryHolders Battery holders:  Heavy duty aluminum holders from Keystone Electronics are my 1st choice. 3xAA 147’s are good, but singles give you more layout options. Don’t forget isolation diodes if you have multiple banks. $4.00
 Pololu I’m putting the Pololu Pushbutton Power switch in the “preferred” list because it’s not required and it adds 0.3 mA to the sleep current – that’s more than the logger itself! $6.00
 platform parts
The rubber flexes quite a bit under water and I have only tested these caps to about 5 m depth.
Waterproof Housing: 4″ PVC sewer drain cap, 4′ knockout cap to mount logger components inside & a Fernco Qwick Cap. I mount the LEDs & sensors in short lengths of 3/4 pipe which are solvent welded to the exterior, then filled with Loctite E30CL $6.00
 JY-MCU I like these tiny JY-MCU ‘s when I have a design that is really tight on space. But getting a particular 5050 is not easy on eBay due to unreliable vendors. $1.50
 wire1 My current favorite wire is Adafruit’s 26awg with Turnigy silicone & Cal Test Electronics Test Lead Wire also being good. Stiff PVC insulation gets irritating when you are trying to route wires into small housings. $2.00
 GroveHub Most of my builds go under water, so the sensors get potted into place on the housing. An interconnect system like the I2C hub from Groove Studios makes the “sensor caps” interchangeable. $3.00
 Sparkfunftdi These Sparkfuns are still my favorite serial adapters, they always seem to work, and the new pin connectors work are more robust. $15.00

Well, there it is: the simplest fully capable datalogger I could come up with. You have from 3 to 5 analog lines free depending on your board (1024 levels on the ADC), and two digital lines left for your 1-Wire sensors (and you still have a free interrupt line on pin 3, and the analog pins can be used identically to the digital pins, using the aliases A0, A1, etc) . I2C is already pulled-up (4.7k) & broken out for you on the RTC cascade port. SPI is still available if you watch your timing with the SD communication. (Both get SCK, MISO, and MOSI in parallel, But each device needs its own Chip Select line, as you can only communicate with one device at a time : SoftwareSPI is also an option but it looses the speed advantage). If you need better resolution on your ADC, you can add an ADS1015 to add four more 12 bit channels. If you need more output lines, you can look into adding a shift register, and there are plenty of transistor/relay breakouts if you want to control high power loads or LEDs.

Once you have a unit up and running, don’t forget to give the boards (but not the μSD adapter) a good dash of silicone conformal coating before they go into the field. And it never hurts to throw a desiccant pack or two in there while you are at it.

——————————————————————————————–

Addendum 2014-10-09:

I tested two breadboard builds (configured as drip sensors) to get a sense of the current draw of the individual components. Without the power switch, the ProMini clone logger drew a sleep current of 0.26 mA sleeping in powerdown mode using RocketScreams Low Power Library with ADC & BOD off.

ProMini clone +
Adxl345 accelerometer (50 hz, power save mode) :  
                          0.058 mA
Sandisk 256mb uSD card :                                                                         0.061 mA
DS3231 & AT24c32 eeprom combo  (with 4.7k pullups) :                   0.089 mA*

* I also tested an AT24c256 separately with a chronodot RTC, 10k pullups, and that eeprom draws about 0.03 mA when not being accessed. As you might imagine, I will be examining the RTC very carefully, as 0.089 mA is much higher than I would have expected for something that is supposed to run off of a coin cell. (The datasheet actually specifies a standby supply current of 0.11 mA at 3.3v)

Double lenth header pins make it easier to breadboard the SD card.

Double length header pins make it easier to breadboard the SD card.  DO NOT TAKE TOO LONG to solder jumpers onto any of these adapter boards. If the springs mounted inside re-flow from excess heat, the connections are toast.

Subtracting those figures from 0.26 mA indicates that the ProMini clone with its MIC5205 regulator uses about 0.052 mA when the unit is sleeping. I am guessing that most of that is due to losses on the regulator, since it is still providing power to the above components. 

Then I changed over to a RocketScream Ultra board, and the whole logger drew 0.22 mA with the same SD card, indicating that “sleeping” logger overhead with their MCP1700 regulator is around 0.012 mA. Significantly better than the 5205 based system, and you have another 100 mA of current capacity at the top end.

I then tested that Rocket ultra unit with the Pololu power switch in the circuit, and sleep current went up to 0.53 mA , indicating that the switch is adding 0.32 mA to the sleep current for this design (after I removed it’s power on LED). Just to be sure about this I measured the current on the other side of the Pololu switch, and the logger unit was still drawing 0.22 mA, so it was not noise, or some other weird side effect keeping the SD card from sleeping properly.

Addendum 2014-11-28

A month long test of one of these units binning at 15 minute intervals, with the Pololu installed (0.56mA sleep current) lowered the 3xAA power supply voltage by 181 mV.  Being conservative, this means we will only get about 4 months of run time with the no-name alkaline batteries I used. That’s a pretty dramatic reduction of run time for that one extra component so I probably will not be using the Pololu switch for my longer term deployments. Field test results so far indicate that a unit sleeping at 0.30 mA will deliver between 6-9 months of operation on a 3xAA battery pack.

Addendum 2015-01-20

Soldering Pullups and Jumpers on SD card adapterJust thought I would post a quick shot of how I jumper the pullup on that SD card adapter with the legs of the resistor itself. Super thin 30 AWG wire makes jumpering the ground line much easier as well. Once all the jumpers are on, I clean off leftover flux with 90% isopropl alcohol, and then coat the contacts with silicone conformal coating, being careful not to let it run to the other side of the board where it would block the connections. Then I adhere the boards to the platform with double sided tape, which has been working surprisingly well on ABS knockout caps.

 

Two resistors on ledsIt is also worth mentioning that the Green channel on the multi color common cathode LED’s is so much brighter than the red & blue colors (4000mcd vs 8-900 mcd), that you can actually use two limiting resistors – one 10K on the common ground line, and an extra 20K on the green line.  In fact, if you can live with a really faint indicator led, you can take the limiters to 20/40K or more if your LED is bright enough.  You don’t want to waste too much power on indicator LEDs when no one is around to see them, and this brings the green channel down to 110uA, while leaving the red & blue channels bright enough to see.  Luke Miller has an elegant method using a magnetic reed switch interrupt to enable & disable his status LED’s to save power, but I often need all the interrupt lines for my sensors.  I could probably work around this with a pin-change driven interrupt, but have not had a chance to dig into that yet.

Addendum 2015-02-12 

I just wanted to add a link here to my new pvc housing design. While not quite as simple as the rubber quick cap housing pictured in the table above, I am certain that this Formufit table-cap based design is the simplest diy housing you could build that will go to significant depth under water.

Addendum 2015-04-14

On my logger I use a voltage divider to monitor the main battery. This bleeds a few μA, but given the overall sleep current that’s a small price to pay for the simplicity of using two resistors & a cap to do the job.  However I just stumbled across a very good way to monitor the battery voltage with virtually no loss over at the LowPowerLab blog. About half way down the discussion user JC suggests ” a resistor divider network, with the high side gated by a FET. Turn the FET on when you want to read the battery voltage, off the rest of the time. The leakage is somewhere down in the femtoamps.”  Another elegant solution for me to look into if I ever get this project to the stage where I am actually implementing elegant solutions.

Addendum 2015-06-28

It will be a while before I sort out all the details of the V3 logger platform, but for a preview of what might be coming,  you can see a couple of builds currently on the test bench here. Also, after many successful field deployments without the power switch where the batteries died but the SD card data was still intact, I have dropped the Pololu power switch from the design. I just could not afford to loose all that power.

Addendum 2015-10-24 

I have produced a set of detailed build instructions for a pro-mini based version of this data logger which can be found at:  

  • Part 1 (component preparation)
  • Part 2 (logger platform assembly)
  • Part 3 (housing assembly)

I still prefer to use the rocket ultra as it’s voltage regulator is more efficient, but it is still possible to build a decent logger with the minis that should sleep below 0.25mA.

Addendum 2016-01-25:

Reliable sources recommend pullups on the SPI lines, though I have been running my loggers for a long time without them.  I have noticed that some SD cards take a really long time to go to sleep unless CS, MOSI & MISO are pulled up, and some just refuse to sleep at all without the pullups. These weak pullup resistors are not described in this tutorial. If you already have loggers built without them, there is a workaround using the 328’s internal pull-up resistors that I current have in testing.  Despite the fact that only CS requires it, I found that all three lines had to be pulled up before misbehaving cards went to sleep properly. So add these lines to the beginning of your setup before you run sd.begin

// pulling up the SPI lines
pinMode(chipSelect, OUTPUT); digitalWrite(chipSelect, HIGH); //pullup the CS pin
pinMode(MOSIpin, OUTPUT); digitalWrite(MOSIpin, HIGH);//pullup the MOSI pin
pinMode(MISOpin, INPUT); digitalWrite(MISOpin, HIGH); //pullup the MISO pin
delay(1);

  It’s worth noting that there is some debate about the utility of this approach. Because SPI is a complex protocol, adding internal or external pullups could actually prevent you from being able to use other SPI sensors with your data logger. That is one reason why I tend to stick to I2C & analog sensors on my builds, so the SPI lines are dedicated to the SD card only.