2023 is the ten-year anniversary of the Cave Pearl Project, with hundreds of loggers built from various parts in the Arduino ecosystem and deployed for Dr. Beddows research. During that time her EARTH 360 – Instrumentation course evolved from using commercial equipment to having students build the entire platform for labs on environmental monitoring. The experience of those many first-time builders has been essential to refining our educational logger design so in recognition of their ongoing and spirited enthusiasm, we call this model the e360.
Many parallel trends have advanced the open-source hardware movement over the last decade, including progress towards inexpensive and (mostly) reliable 3D printing. In keeping with the project’s ethos of accessibility, we use an Ender 3 for the rails and you can download that printable stl file directly from Tinkercad. Tinkercad is such a beginner-friendly tool that students are asked to create a mounting bracket as part of the Lux/LDR calibration lab. This directly parallels our ever increasing use of 3D prints for equipment installations on the research side of the project.
Cheap, simple, stand-alone loggers enable teaching and research opportunities that expensive, complex tools can not. However there are a few trade-offs with this minimalist design: Supporting only Analog & I2C sensors make the course more manageable but loosing the DS18b20, which has served us so well over the years, does bring a tear to the eye. Removing the SD card from the previous model means you have to think about memory constraints on run-time. The RTC’s one second minimum means this logger is not suitable for high frequency sampling – so you are not going to use it for experiments in eddy flux covariance or seismology. UV exposure makes the 50ml tubes brittle after about four months in full sun, and the coin cell limits operation to environments that don’t go much below freezing – although it’s easy enough to convert the logger to use two lithium AAA’s and we’ve tested those down to -15°C.
The lab kit:
Expect 10-25% of the parts from cheap suppliers like eBay or Amazon to be high drain, or DOA. We order three complete kits per student to cover defects, infant mortality, and replacement of things damaged during the course. Many students build a second or third logger for their final project.
Assembling the logger:
This build is based on the 2-Module logger we released in 2022 with changes and additions to support the course labs. That post has extensive technical details on the logger core that have been omitted here for brevity. But it’s a good idea to read through that background material when you have time.
Modifications to the RTC module:
Cutting the VCC input leg forces the clock to run on VBAT which reduces RTC to <1µA sleep, but currents can spike as high as 650µA when a temperature reading occurs. If the time reads 2165/165/165 instead of the normal startup default of 2000/01/01 then the registers are bad and the RTC will not function. Bridging Vcc to Vbat means a 3.3V UART will drive some harmless reverse current through older coin cells while connected. Macintosh users have smart USBc to USBa adapters which will go into shutdown if they detect any power back-fed from the coin cell when they are not powered from the USBc side. So people using those dongles must first disconnect at the UART-logger point before removing the adapter from the USBc port on their computer. DS3231SN RTCs drift 2ppm = max 61 sec/year while -M variants drift 5ppm, or a max of 153 seconds/year. If the RTC temperature reading is off by more than the 3°C spec. then the clock may drift more.
Modify & TEST the Pro Mini:
A Pro Mini style board continues as the heart of the logger, because they are still the cheapest low-power option for projects that don’t require heavy calculations.
Add the Sensors & LED:
These additions are optional, but provide excellent opportunities for pulse width modulation and sensor calibration activities.
We have a separate post describing how to calibrate these NTC thermistors
Connect the Modules via the I2C Bus:
As soon as you have the two modules together: connect the logger to a UART and run an I2C bus scanner to make sure you have joined them properly. You should see the DS3231 at 0x68, and the 4K EEprom at 0x57.
Add Rails & Breadboard Jumpers:
A video covering the full assembly process:
The Code [posted on GitHub]
In addition to the included NTC / LDR combination, the code has support for the Bmp280, Bh1750, and PIR sensors although you will need to install hp_BH1750 and BMP280_DEV via the library manager. Sensors are added / removed by uncommenting define statements at the beginning of the code. Each sensor enabled after the single-byte LowBat & RTCtemp defaults contributes two additional bytes per sampling event because every sensors output gets converted into a 16-bit integer.
The basic sensors cover light, temperature, and pressure – so you could teach an introductory enviro-sci course by enabling or disabling those sensors before each lab.
But limiting this tool to only the pre-configured sensors would completely miss the point of an open source data logger project. So we’ve tried to make the process of modifying the base-code to support different sensors as straight forward as possible. Edits are required only in the places indicated by call-out numbers on the following flowing charts. These sections are highlighted with comments labeled: STEP1, STEP2, STEP3, etc. so you can locate them with the find function in the IDE.
Those comments are also surrounded by rows of +++PLUS+++ symbols:
//++++++++++++++++++++++++++++++++++++++++++
// STEP1 : #include libraries & Declare Variables HERE
//++++++++++++++++++++++++++++++++++++++++++
In Setup()
A UART connection is required to access the start-up menu through the serial monitor in the IDE. This menu times-out after 8 minutes but the sequence can be re-entered at any time by closing and re-opening the serial monitor window. This restarts the Pro Mini if the UARTs physical connection to the DTR (data terminal ready) pin is good. The start-up menu should look similar to this:
The first menu option asks if you want to download data from the logger after which you can copy/paste (Ctrl[A]&Ctrl[C] then Ctrl[V]) everything from the serial window into a spreadsheet. Then, below the data tab in Excel, select Text to Columns to divide the pasted data at the comma separators. Or you can paste into a text editor and save a .csv file for import to other programs. While this transfer is a bit clunky, everyone already has the required cable and retrieval is driven by the logger itself. We still use the legacy 1.8x version of the IDE, but you could also do this download with a generic serial terminal app. You can download the data without battery power once the logger is connected to a UART. However, you should only set the RTC after installing a battery, or the time will reset to 2000/01/01 00:00 when the UART is disconnected. No information is lost from the EEprom when you remove and replace a dead coin cell.
A Unix timestamp for each sensor reading is reconstructed during data retrieval by adding successive second-offsets to the first record time saved during startup. It is important that you download any old data before changing the sampling interval because the interval stored in memory is used for the calculation that reconstructs each records time. This technique saves a significant amount of our limited memory and =(Unixtime/86400) + DATE(1970,1,1) converts those Unix timestamps into Excel’s date-time format. Valid sampling intervals must divide evenly into 60 and be less than 60. Short second-intervals are supported for rapid testing & debugging, but you must first enter 0 for the minutes before the seconds entry is requested. The unit will keep using the previous sampling interval until a new one is set.
Vref compensates for variations in the reference voltage inside the 328p processor. Adjusting that constant up or down by 400 raises/lowers the reported voltage by 1 millivolt. Adjust this by checking the voltage supplied by your UART with a multimeter while running the logger with #define logCurrentBattery enabled and serial output Toggled ON. Note the difference between the millivolts you actually measured and the battery voltage reported on screen, and then multiply that difference by 400 to get the adjustment you need to make to the default 1126400 vref for accurate battery readings. Save this new number with the [ ] Change Vref start menu option and re-run the test until the numbers on screen match what you measure with the DVM. This adjustment procedure only needs to be done once as the number is stored in the 328p EEprom for future use. Most loggers run fine with the default vref although some units will shutdown early because they are under-reading. It’s rare to get two constants the same in a classroom of loggers so you can use student initials + vref as unique identifiers for each logger. However if you do get a couple the same you can change the last two digits to to make unique serial numbers without affecting the readings. The battery readings have a resolution limit of 11millivolts, so that’s as close as you can get.
After setting the time, the sampling interval, and other operating parameters, choosing [ ] START logging will require the user to enter an additional ‘start’ command. Only when that second confirmation is received does the storage EEprom get erased by pre-loading every memory location with zero. Red&Blue LEDs then flash to indicate a synchronization delay while the logger waits for the first alarm to align with the current minute/hour. A zero-trap is required on the first byte of each record because the preloaded zeros also serve as the End-Of-File markers during download. If you leave the default LogLowestBattery enabled that is already taken care of.
In the main LOOP()
If all you do is enable sensors via defines at the start of the program you won’t have to deal with the code that stores the data. However to add a new sensor you will need to make changes to the I2C transaction that transfers those sensor readings into the EEprom (and to the sendData2Serial function that reads them back out later). This involves dividing your sensor variables into 8-bit pieces and adding those bytes to the wire transfer buffer. This can be done with bit-math operations for long integers or via the lowByte & highByte macros for 16-bit integers. The general pattern when sending bytes to an I2C EEprom is:
Wire.beginTransmission(EEpromAddressonI2Cbus); // first byte in I2C buffer
Wire.write(highByte(memoryAddress)); // it takes two bytes to specify the
Wire.write(lowByte(memoryAddress)); // memory location inside the EEprom
loByte = lowByte(SensorReadingVariable);
Wire.write(loByte); // adds 1st byte of sensor data to wire buffer
hiByte = highByte(SensorReadingVariable);
Wire.write(hiByte); // adds 2nd byte of sensor data to the buffer
–add more Wire.write statements here as needed for your sensor–
// You can add a total of 1, 2, 4, 8 or 16 DATA bytes to the I2C transaction. Powers of two increments are required because the recorded data must align with page boundaries inside the EEprom.
Wire.endTransmission(); // Only when this command executes does the data accumulated in the wire buffer actually get sent to the EEprom
The key insight here is that the wire library is only loading a memory buffer until Wire.endTransmission() is called. It does not matter how much time you spend doing calculations, or parsing variables, so long as you don’t start another I2C transaction while you are still in the middle of this one. Once that buffer is physically sent over the wires, the EEprom enters a self-timed writing sequence and the logger reads the rail voltage while the CR2032 is under load. This is the only way to accurately gauge the state of lithium coin cell batteries.
The data download function called in setup retrieves those separate bytes and concatenates them back into the original integers. The sequence of operations in the sendData2Serial function must exactly match the byte-order used to load the EEprom in the main loop.
Various Sensor Options:
By default, the logger records the RTC temperature at 0.25°C resolution and the battery voltage under load. These readings are compressed to only one byte each by scaling with a fixed offset. This allows ~2048 readings to be stored on the built-in 4k EEprom which allows 21 days of operation at a 15-minute sampling interval.
That 4k fills more quickly if your sensors generate multiple 2-byte integers but larger 32k (AT24c256) EEproms can easily be added for longer running time. These can be found on eBay for ~$1 each and they work with the same code after you update the EEpromI2Caddr & EEbytesOfStorage #defines at the start of the program.
Pullup resistors can be left on the sensor modules as the logger will operate fine with a combined pull below 2k. No matter what sensor you enable, always check that the total of all bytes stored per pass through the main loop is 1,2,4,8 or 16 or you will get a repeating data error whenever the bytes transmitted over the I2C bus pass a physical page boundary inside the EEprom. This leads to a wrap-around error which over-writes data at the beginning of that page/block.
Perhaps the most important thing to keep in mind is that breadboards connect the sensor module headers via tiny little springs which are easily jiggled loose if you bump the logger. Some sensors can handle this momentary disconnection but many I2C sensors require full re-initialization or they will not deliver any more data after a hard knock. So handle the logger gently while it’s running – no tossing them in a backpack full of books until after you’ve hard-soldered the sensor connections.
The code also includes the DIGITAL method we developed to read the NTC/LDR sensors. On this new build we used the internal pullup resistor on D8 as a reference to free up another digital pin. The blue jumper wire on D3 (the 2nd external interrupt) can wake the logger with high / low signals. This enables event timing and animal tracking. Pet behavior is a popular theme for final student projects.
These 0.49″ micro OLEDs sleep at only 6µA and usually draw less than a milliamp at 50% contrast. However, like all OLEDs they put wicked charge-pump spikes on the rail. A 220µF tantalum right next to them on the breadboard suppresses that noise completely. Sleep the CPU while the pixels are turned on to lower the current load on the coin cell.
While I2C sensors are fun, we should also mention the classics. It is often more memorable for students to see or hear a sensors output, and the serial plotter is especially useful for lessons about how noisy their laptop power supply is…
Unless you are run a lab tethered to the UART for power, any sensors you add have to operate within the current limitations of the coin cell powering the logger. This means they should take readings below 2mA and support low-power sleep modes below 20µA (ideally < 2µA). A GPS module or a CO2 sensor usually requires too much power, so to use those you will need one of the previous AA powered loggers from the project.
Logger Operation:
The logger typically sleeps between 5 – 10µA with a sensor module attached. Four 5mA*30millisecond sensor readings per hour gives an estimated battery lifespan of about one year. So the logger is usually more limited by memory than the 100mAh available from a Cr2032. The tantalum rail-buffering capacitor extends operating life about 20% under normal conditions, but it becomes much more important with power hungry sensors or in cold environments where the battery chemistry struggles:
The code sleeps permanently when the battery reading falls below that defined at systemShutdownVoltage which we usually set at 2.8v because many 328p chips trigger their internal brown-out detector circuit at 2.77v. And the cheap old-stock I2C EEproms you get from Ebay often have a lower operational limit at 2.7v.
When testing sleep current on a typical batch of student builds, some will appear to have anomalously high sleep currents in the 600-700µA range. Often that’s due to the RTC alarm being on (active low) which causes a constant drain through the 4k7 pullup resistor until the alarm gets turned off by the processor. Also, the 1000uF tantalum capacitors have a ‘saturation period’ where they have to be exposed to voltage for a several hours before they settle down to their typical 2µA leakage current. It is worth mentioning that tantalum capacitors are heat sensitive, so beginners can damage them while soldering. A typical logger should sleep below 5µA, and if they are consistently 10x that, replacing an overheated rail capacitor may bring that down to the expected sleep current. Also check that the ADC is properly disabled during sleep, as that will draw ~200µA if it is left on while sleeping. The Brown Out Detector typically draws 20µA if that is left on during sleep.
However, a few loggers will end up with hidden solder bridges that require a full rebuild. This can be emotionally traumatic for students until they realise how much easier the process is the second time round. Once you’ve made a few, a full logger can usually be assembled in about 1.5 hours. It’s even faster if you make them in batches so that you have multiple units for testing at the same time.
Running the labs:
Through no fault of their own, students usually have no idea what messy real-world data looks like, and many have not used spreadsheets before. So you will need to provide both good and bad example templates for everything, but that’s easy enough if you ran the experiment a dozen times at the debugging stage.
Even then students will find creative ways to generate strange results by using a cactus for the evapotranspiration experiment, or attempting the light sensor calibration in a room that never rises beyond 100 Lux. When students are doing a project it is critical that you make them download and send graphs of the data they’ve captured every day for feedback. Otherwise they they will wait until the hour before a major assignment is due before finding out that their first (and sometimes only) run didn’t give them any useable data. Your grading rubric has to be focused on effort and understanding rather than mechanical results, because the learning goals can still be achieved if they realize where things went wrong.
Important things to know:
Time: You need to start ordering parts at least three months ahead of time. Technical labs take a week to write, and another week for debugging. You can expect to spend at least an hour testing components before each lab. The actual amount of prep also depends on the capabilities of your student cohort, and years of remote classes during COVID lowered that bar several pegs. Have several spare ‘known good’ loggers on hand to loan out so hardware issues don’t prevent students from progressing through the lab sequence while they trouble-shoot their own build. Using multi-colored breadboards on those loaners makes them easy to identify later. Measuring logger sleep current with a DSO 138 scope ,a µCurrent or a Current Ranger will spot most hardware issues early, but students don’t really get enough runtime in a single course to escape the bathtub curve. On the research side of the project, we run our loggers to full memory shutdown several times, over a couple of weeks, before they are considered ready to deploy. In addition to data integrity, we also look for smooth predictable battery burn-down curves.
Money: Navigating your schools purchasing system is probably an exercise in skill, luck and patience at the best of times. Think you can push through dozens of orders for cheap electronic modules from eBay or Amazon? Fuhgeddaboudit! We have covered more than half of the material costs out of pocket since the beginning of this adventure, and you’ll hear that same story from STEM instructors everywhere. If you can convince your college to get a class set of soldering irons, multimeters, and perhaps a 3D printer with some supplies, then you are doing great. We bought nice DVMs at the beginning but they all got broken, or grew legs before we got enough course runs with them. We now use cheap DT830’s and design the labs around it’s limitations. Smaller tools like wire-strippers and side-snips should be considered consumables as few live long enough to see a second class. Cheap soldering irons can now be found for ~$5 (which is less than tip replacement on a Hakko!) and no matter which ones you get the students will run them dry every time. The up-side of designing a course around the minimum functional tools is that you can just give an entire set to any students who want to continue on their own after the course. That pays dividends later that are worth far more than any one years budget.
All that probably sounds a bit grim, but the last thing we want is for instructors to bite off more than they can chew. You will need to noodle with these loggers for a few months before you are ready to integrate them into your courses. Not because any of it is particularly difficult, but because you will need to work with them before you realize the many different ways this tool can be used. A summer climate station project with five to ten units running in your home or back yard is a great way to start and, if you do invest that time, it really is worth it:
Zigging while others zag:
Why did we spend ten years developing this DIY logger when the market is already heaving with data acquisition equipment? Our primary issue with polished educational products using pre-written software is that they are usually plug-and-play. The last thing you want in higher education is something that black-boxes data acquisition to the point that learners become mere users of the technology. While companies boast that students can take readings without hassle and pay attention only to the essential concepts of an experiment, that has never been how things work in the real world. Trouble shooting by process-of-elimination, combined with modest repair skills often makes the difference between a fieldwork disaster and a resounding success. So sanitized science equipment that generates uncritically trusted numbers just isn’t compatible with problem-based learning. Another contrast is the sense of ownership & accomplishment that becomes clear when you realize how many students gave their DIY loggers names, and then displayed them proudly in their dorm after the course. That’s not something you’ll buy off a shelf.
References:
ATmega328P Datasheet
Waterproofing your Electronics Project
Oregon Embedded Battery Life Calculator & Cr2032 testing
Nick Gammon: An excellent technical information source
A practical method for calibrating NTC thermistors
Underwater Arduino Data Loggers 
“with everybody still smiling because they still have no idea what they are in for.”
:-)))))
Pingback: The e360: A DIY classroom data logger for science #Arduino #Science « Adafruit Industries – Makers, hackers, artists, designers and engineers!
Pingback: A DIY classroom knowledge logger for science #Arduino #Science « Adafruit Industries – Makers, hackers, artists, designers and engineers! - artificialintelligence360
I honestly don’t know what to write here.
I’ve been following your work for… well, some time now. Using what I can for my own research, as well as teaching classes, and students, some of this. Here again, i’m at a loss on just HOW MUCH you are providing, and digging in deep. The instructions, the explanations, and the methods you’ve developed are wonderful, but your ability to convey them to others is critical here.
But the last few paragraphs… these are Gold. The hardware tips are wonderful, and software I’m frankly till trying to just catch up on constantly. But it’s the underlying design and educational philosophy that underlies it all that I think is the really critical takeaway from these. I need to remember this; and, I need a lot of other people in the educational world to read this, and understand it, and reach out and embody it.
OK, enough gushing for now… I need to dig in to your code, but I feel I’ll probably stick with bulkier slower less-power-efficient code for students. Not to mention how to run a PIR at 3.3v reliably (I’ve missed that trick in your previous posts and will have to dig; I’ve modified one to use but haven’t played with it much).
Again… “thanks” doesn’t begin to cut it. Now I just need to eliminate sleep and research so I can dig in enough to capture and build the next version of a course around this core (but a slightly different logger).
Just wait till you see what’s coming in 2024 🙂
Pingback: Data Logger Fai-Da-Te per le Scienze Ambientali - Elettronica In: La tua rivista di elettronica
Hello! I am trying to build a small underwater light, which goes approximately 200 m deep water. I am planning to use the same bottle as is in the DIY Classroom Data Logger. I am wondering if gets broken in this depth if filled with mineral oil?
That’s well below anything we dive to, and my little test chamber [ https://thecavepearlproject.org/2023/05/24/a-diy-pressure-chamber-to-test-housings/ ] doesn’t go beyond 90psi.
So I can’t say anything about 200m but in our shallow equivalent tests under oil the batteries had a dramatic voltage loss when the units were depressurized. So you can expect it to be a one way trip for the power supply(s) because any hydrogen, etc. that gets evolved as the units run under oil will form decompression bubbles inside the cells when the units are retrieved. And that’s even if you are very careful that there was no air trapped inside the logger housing at the surface.
Hello Ed Mallon,
First of all, thank you for such extensive writing in helping make dataloggers. The extensive documentation is such an incredible resource.
I am having trouble with a timing issue. The rtmc is accurate. My problem is with synching the time to true time. I am using your basic logger arduino code UNO-Breadboard-Datalogger/_20160110_UnoBasedDataLogger_v1/_20160110_UnoBasedDataLogger_v1.ino .
I want to set the rtmc/datalogger to the true time, within one sec or so. The problem is that the time is set to the computer time, upon compile and send of the arduino code. This always introduces a 10-20 sec error in the true time. This means I can not set the clock to the actual true time which I need to do, especially if using multiple loggers to track a phenomena.
I found a nice accurate way to synch the rtmc, but the clock set gets ruined when I then go back to running the above data logger code, ie it goes back to its delayed compile time delay.
‘synchrotime’ https://github.com/SergejBre/SynchroTime
I have looked and looked a lot at all your documentation, but can not find an answer to my problem. Could you please advise how I can set the datalogger to true time, close to one sec or less away from the actual time (desktop computer time etc). I would be very grateful for any assistance.
Sincerely, Raanan
Oh my gosh that’s a very old build you are using! On our newer code we have changed over to update the clock by entering digits manually through the serial monitor window. So you could just comment out the ‘RTC update’ parts in that older code in setup?
and copy over the function called startMenu_setRTCtime() in the latest codebuild: https://github.com/EKMallon/e360-Classroom-Data-Logger-2023
Note that you will also need the other functions that function calls – but you could also just copy those directly inline as well. The trick when setting the time is to enter the year, month, day etc. but then type in ‘seconds’ about 15 seconds ahead of actual. Then wait for your system clock to reach those seconds and hit enter. You will also need a HH:MM:SS display on your computer, and in windows I use an app like elevenClock to display the seconds. I can usually get them synced within 1/10 of a second using this ‘by hand’ method.
That synchrotime looks fantastic, but to this point in the project we have just lived with the fact that the 2ppm clocks drift, on average, about 30 seconds per year of operation. The 5ppm -M variants drift by about 2 minutes per year.
EDIT: I just looked at that old code – it has no commands that specifically sets the clock so this is being done by the RTClib rather than by my code. So you will have to comment out the ‘automatic time reset’ section in the library’s .ccp file and resave your modified version. This can be done inside any text editor.
Dear Mr Mallon,
Thank you for the full response. It is very helpful. If I may:
1 I am using for learning right now the arduino, with rtmc, as well as a sd card, not the current 2 component system. Could you please advise how I could adapt the suggested code above, to work with the sd card 3 component system? That is why I was trying to use the older code, but had lots of trouble with time setting described above.
2 Or maybe there is another newer code that is ideal for the 3 component system?
I would like to learn how to adapt the newer code, but it also might be helpful to have a much simpler, shorter basic code, to learn from, if one exists.
I greatly look forward to trying to implement your timing method described above.
Thank you,
Raanan
As I mentioned in the previous response the codebuild you are using has no commands that specifically sets the clock. So the update to compile time is being done by the library rather than by my code.
You could edit the library: commenting out the ‘automatic time reset’ sections in the .ccp file and resave your modified version. I suspect this section is checking the OSF (oscillator stop flag) in the RTC register and setting the clock to compile time IF the flag still shows a previous clock stoppage.
If it is the flag causing the auto-update in the RTClibrary, an alternative would be to clear the flag manually in your code by setting the bit 7 in the 0x0F status register to zero after starting the I2C bus but BEFORE calling the RTC library .begin. In fact you could just zero to the entire register – which also disables both alarms until you re-enable them:
Wire.beginTransmission(0x68);
Wire.write(0x0F);
Wire.write(0);
Wire.endTransmission();
OR you could switch to a different RTC library, and then just update the code wherever there is an RTC command, to the equivalent commands provided by that new library. There are many Arduino libraries for this RTC.