Yearly Archives: 2015

Field Report: 2014-12-11 Our First Real-World Drip Sensor Data!

Our deployment sites were within site of survey stations, so we could triangulate surface locations later.

Our sites were all close to survey stations, so we can work out the locations for top side sensor co-deployments later.

Trish retrieved the first generation of drip sensors from Rio Secreto yesterday, but we did not have time to open them up until this morning. So breakfast consisted of me concatenating the separate csv files, while she brought some substantial spreadsheet whammy to bear analyzing the data.  We were both pretty excited to find that the first generation of drip sensors had actually worked, and I can only imagine how our little science/nerd fest looked to the other folks at the Turtle Bay Cafe, as they nursed their Mexican holiday hangovers with caffeine.

Trish described how one of our installation sites had experienced a flooding event, which knocked over some of the the sensors. While she was re-assembling the scattered stations, she recorded another set of manual drip counts, so we could verify our numbers. Shortly after the deployment in August, I had made the unwelcome discovery that many of those early drip loggers had been deployed with fake SD cards, which use excessive amounts of power. But I had also installed lithium batteries to provide a bit more juice (~2900 mAh/cell), and I was happy to see that four of the original six units were still running. Whew!

I won’t bore you with the low level details, because I think the graphs speak for themselves. This drip count from is from DS01, binned to a 15 minute interval:Cave Pearl Drip Sensor DATA

Fernanda Lasas,  who kept an eye on our little loggers during this deployment, confirmed that there was indeed an event, with rainfall and a jump in the local water table, right around that first point of inflection, and that it was “pretty rainy” for more than a month after that. This resolved that question I had about the validity of data from DS03, which had failed early.  While there was considerable variability in drip counts from one stalagmite to the next (as predicted by my wife back in August…), we saw congruent trends in the different records from this chamber. With so few stations in this first deployment,  I am not sure what else one could say at this point.

The second chamber we instrumented had considerably lower drip rates, which was somewhat ironic considering that it was the one that flooded during the deployment. This record was from DS07 (shown in the photo above) with the count recorded every 15 min:DripSensor07_Data_RioSecreto_Aug-Dec_2014

While it doesn’t take a rocket scientist to see when the unit was knocked over, I have added a once-per-day x,y,z axis reading to the code, so we can be more certain that the sensors hold a fixed position on future deployments. You do see that spike reflected in the readings from the other chamber, although it was muted against the higher base rate. And it’s even good to see a long string of zeros in the data, as this tells me that my sensors are not generating false positives from internal noise. Accelerometers can be twitchy when you run them at high sensitivity.

The SD card debacle meant that most of these loggers burned through their batteries much faster than I would have liked. But after looking at the plots, I realized that we had a reasonably good natural experiment testing the capacity of the Energiser advanced lithium (EA91) batteries we used.  So in the spirit of “making lemonade”, I thought I would post the combined Battery Voltage (mV)  vs  Sleep Current (mA) results:

Battery Voltage vs Sleep CurrentWhile that’s not a Danish performance test, it shows me that the Cave Pearls really do need to get sleep currents below 0.3 mA if I’m going to get a year out of three AA’s. You do not see that distinct “shoulder & elbow” pattern (see DS05) in the discharge curve of alkaline batteries, which give you a nice gradual decline in voltage.  If a set of lithium cells is going to die while they are still at 4.4 volts, I need to raise the cut-off in the code to protect my data. Unit 3 did have a corrupt file on the SD card, so I probably lost a couple of weeks there by not intercepting the SD writes soon enough. I have reduced the number of records per file, which will put less information at risk from a lost file like this, though it means more work stitching the data together later.

It took us most of the day to sift through it all,  but in the evening I still had a time to fire up a few new units, and review the overnights from the ones I had started the day before. In the process I discovered a problem with the new humidity sensors I had cobbled together just before the trip, which were claiming that our room hit 200% RH sometime around noon. The raw data looked fine, so I suspected a problem with the floating point calculations on the Arduino; something I ran into with the temperature sensors. I locked them into a bag with some desiccant packages to see if forcing them into a low range over night would help me isolate the problem.  They were still running the code that had performed beautifully back home, but during those tests they not been exposed to such high humidity for significant amounts of time.

Still, after a banner day like this, I can’t get too torn up about it.  🙂

Addendum 2014-12-16

I am happy to report that all of these first generation drip sensors went down to ~0.33mA after I replaced the fake SD cards. And only one of those first gen units was built with Rocket Ultra, the rest were a variety of cheap Pro-Mini style clones from eBay.  I will leave the existing batteries in DS01, so that the next deployment gives me more of that discharge curve.

Addendum 2015-01-07

Another little gem lies in the record for DS04, which had a sleep current just under one milliamp.  That unit looks like it would have gone another couple of weeks before hitting the rapid fall-off at 4.4v , giving about 4 months of operation. Now 4(months) x 30(days) x 24 (hours) = 2880, which, more or less, is the number of milliamp hours you get from a single AA battery. But the drip loggers have three batteries, so my ‘live’ duty cycle power drain is roughly 2x that being consumed when the unit is sleeping. DS04 wrote about 10000 records to the SD card and with 64 records buffered to the 4k eeprom on the RTC, that translates to about 160 SD card write events. The cumulative drip count approached 11000. The drip-interrupt routine does not do very much so I think writing data to the eeprom & SD card are my biggest power users. The new generation units have 32k eeproms, and I have them set to buffer a range from 96 to 512 records to see if I can quantify that power use a bit better.

Addendum 2015-03-02

A friend put me onto a way of graphing battery capacity in a way I had never seen before: Ragone Plots. I am still wrapping my head around the idea, but the convergence of all those battery power curves seems to indicate that there is very little advantage to using lithium batteries in low power applications like data loggers unless you need to operate at very cold temperatures.

Addendum 2016-08-31

Just breezing over some of these older posts, and realized the mistake I made on 2015-01-07.  When you put batteries in series you add voltage, but not power, so 2880 mAh would have been the budget for the entire battery pack. Fortunately for me, the blog traffic was so low back then that no one pointed out the gaff.  I’m leaving it in, as it shows what kind of a learning curve I was on at that point, and the project continues to progress despite my many misunderstandings.

<— Click here to continue reading the story—>

Field Report 2014-12-10: Collecting Salt Water Pearls

We did not have scuba gear, so the fastest approach was simply to cut the anchors away.

Without scuba gear, the fastest approach was simply to cut the anchors. I wanted to inspect  those connectors anyway.

Our first day with boots on the ground, and we were quite keen to see what we had on the loggers that were deployed in back in August.  A fresh batch of data is always a great way to motivate yourself for fieldwork.  So Trish headed out to Rio Secreto to collect the 1st generation drip sensors, while I met up with Gabriel and Marco from CEA to see about retrieving the Beta flow sensors we put in Akumal bay back in August. As we waited for a boat to become available Gabriel showed me the fantastic records that they had kept, and the locations they had selected for the deployments of the other two flow meters. One had been placed in the shallower south side of bay on October 13th, and the third unit was deployed at the mouth of Yalku Lagoon on November 27th.

 

This was discovered on Aug, and the unit was re-installed on Nov 21st.

This pivot joint failure was discovered on Nov 7th, and the unit was re-installed on Nov 21st. (Photo courtesy: Centro Ecologico Akumal)

Opportunistic photos of the units every couple of weeks revealed that the constant roil of the surf had taken a toll: with both of loggers in the bay suffering failures on the anchor rods & pivot joints. I had designed the pearls for much gentler cave environments, so this was not unexpected. I was just thankful that the folks at CEA had been around to catch the problems while the “backup” bungee cords were still in place, or the loggers could have simply drifted away.  Sometimes all you get from the first deployment is an understanding of how to do the next one better, and patchy data is still 100% better than no data at all.  Of course, As I reviewed the photos with Gabriel, I could not help but wonder if the electronics had survived all that knocking about.

 

There was even a few small crabs crawling around on the surface.

A marine bio project if I ever saw one

Gabriel had other pressing business that day, so when the boat became available Marco and I set off to retrieve the flow meters.  At each site we did a quick check that the north orientation was still correct, and that there were no obvious signs of physical damage. It was a gorgeous day to be out, but the bright tropical sun made it impossible for me to determine if the leds were still piping.  The first unit in the bay looked great but the second unit (in much shallower water) had suffered an incredible amount of bio-accumulation in only two months. I had never seen this on a sensor in the caves and it made wonder it if would even be possible to deploy ambient light sensors on a reef without some kind of rigorous cleaning schedule. By mid afternoon we had all the babies on board, and were heading back to shore.

The Oring seats were still clear. I guess PVC tastes better than EPDM?

The 0-ring seats were still clear. I guess PVC tastes better to sea critters than EPDM?

I spent a couple of hours scraping the gunk off of the housings with isopropyl alcohol before I dared to break a seal. And it was tough going, even with a pot scrubber. During the cleaning I could see that the LED on unit three was lit, indicating that it had gone into some kind of error state. Unit 4 piped on schedule, but I saw no flash from Unit 5. After the cleaning, it still took a wrench and some colorful language to loosen those bolts.

Once they were open I had a chance to look at the data files.  All of them had saved at least 10,000 records, but unfortunately the data from Unit 3 consisted of the same four numbers, repeating over and over again.  Inspection revealed that the SCL line on the I2C bus was broken. This had terminated the internal communications, although the RTC interrupt continued to fire on schedule for at least a month before it got confused and reset itself. So the logger from the south of the bay did not give us anything useful.  Unit 4, the first to go in, was still running when I disconnected it and I was keen to how much power it had used in three months. (see: mV vs time in the graph below) These beta generation units were running some pretty hairy old code, and I knew they were probably pulling a few mA the whole time. I also had Unit4 on a five minute sample schedule, so it had saved almost twenty eight thousand records to the SD card:

B4_Battery

No surprises there, with another 2-3 months of operation before this unit powered down. But it is worth noting how much spread there is in the voltage reading. This generation of loggers sported a TinyDuino stack so I used the AVR’s internal 1.1 vref to monitor the battery, and I was not expecting to see so much variability with the bandgap voltage method (>70 mV of noise?). When I use a voltage divider to read Vbat on my other builds, the readings are much more stable. 

It will take me a while to chew the compass and accelerometer data into something useful but the temperature record really jumped out at me:

AkumalBay_unitB4_TempRecord

 *repairing the anchor rod failure left a two week data gap in Nov.

For almost two months the night-time lows stay above 28 C, with some of the highs reaching 31 degrees. And this sensor (DS18B20) is not on the surface, but down in the middle of the water column at about 3m depth, pretty close to that reef. I’m no biologist, but it seems to be getting a little toasty down there…

We had a little farmer tending the crop on unit3.

We had a little farmer tending the crop of algae that bloomed on Unit 3

Unit 5 was still running, although the LED ground line had been shaken loose, which I why I did not catch any pips. This build also had a 3.3v regulator one the power module, so I don’t have a battery voltage data to analyse. And finally, this unit did not go into the water till Nov 27th, so it’s flow data record is quite brief. However there was one other thing I could look at, before calling it a day: How much did those cheap eBay RTC’s I was using drift over the deployment?  I found a lag of about 30 seconds in the RTC on Unit4, and about 40 seconds had been dropped from Unit5. I probably caused some of that delay myself as I was not setting the clocks very carefully back then, but it is still gives me some indication that these RTC boards should be good for a year long deployment. Not bad for a board that only cost two bucks.

LoctiteYellowing

Beta Unit 4 has now been under water for almost 10 months. The JB marine weld & Loctite epoxy are starting to show their age, in fact if the units were not under water the whole time, I’d say they were suffering from UV exposure.  But I think they should still be water tight for a while, despite the fact that I exceeded any manufacturer specifications quite a while ago.  The plan is to keep these early builds in service till the housings finally fail, but I would like to lower that sleep current before I deploy these units are redeployed.  If memory serves, I never did get around to sleeping that bma250 in the Beta generation code (?)

<— Click here to continue reading the story—>

Project Update: Gearing up for field work 2014-11-26

Using Loctite E00-CL this time round.

Using Loctite E00CL this time round. The epoxy is weaker overall, but claims higher shear strength on PVC than E30-CL. The faster epoxy gets hotter, and contracts more, so there may be some risk of lifting the components. And the numbers in the individual data sheets do not exactly match those in Loctite’s own Plastics Bonding Guide, so who knows?

Another field trip is rapidly approaching, and I am scrambling to finish the bench tests before we have to stuff everything into a suitcase. The last three months have seen the project migrate away from the unregulated TinyDuinos as the heart of the data logging platform, to RocketScream based builds. Most of my sensors require a regulated 3.3 volt supply, and with only one MCP1700 voltage regulator in the mix, the Ultras have been delivering better sleep currents overall.  The MCP also gives me the ability to use lithium batteries in a pinch (who’s over-voltage would fry the unprotected processor on a TinyD board) , and it delivers up to 250mA if I end up with really power hungry sensors later on. Now that I have the same core logging platform in all the different Cave Pearl models, it is easier to shave down my code, as the compiles keep bumping up against the 32k limit for multi-sensor configurations like the pressure/temp/RH unit

But I have not forgotten how the TinyDuinos catapulted this project into viability back in 2013, and I am waiting to see if they release a generic I2C driver shield. Despite my rough handling of those early Tiny-based builds, most of them are still chugging along after months under water, a tribute to the quality of their build. I enjoy soldering my little bots together, but anything you have to do a hundred times begins to loose it’s luster.

Cave Pearl Flow Sensor

With new 32k EEproms in the mix, space on that logger platform is getting pretty tight and I have to trim the groove hub pcb to make more room.

Bench testing over the last few months has seen more sensor failures than I can throw a stick at, and I am sure that there are more to come if I keep using cheap eBay vendors.  The best overall diagnostic to identify good breakout boards continues to be shutdown mode current. If it’s on spec, and the board delivers a stable reading after wake-up, your golden.  Along the way, there have been so many little code tweaks I could not even begin to list them all. Some, like having the sensor reading LED pip change color to also indicate battery status, were effortless. But others, like determining the optimum number of times to use precious power up cycles to check that battery status, still have me scratching my head. We have more than 12 new loggers to deploy this time round, and I will be embedding plenty of little mini-experiments in the code to give me some empirical data for those questions.

You need at least a week of dry runs, as some sensors fail after a few days of operation.

You need at least a week of dry tests, as some sensors don’t fail till they have been running for a few days.

At this point I am focusing on micro amps, not milli amps, and the best drip sensor builds are coming in with sleep currents in below 15 μA (if I get all the sensors in to their low power modes and pin-power the RTC) That’s a heck of allot better than I was expecting for a few jumpers connecting off-the-shelf breakout boards. Even with the physical build coming together well, I still have a huge sensor calibration to-do list hanging over my head. But the tickets already bought, so that will have to wait till after the next set of field deployments. I also need to develop a new bench testing method that gives me the ability to discriminate how relatively subtle code changes affect a micro-power budget. Oscilloscopes seem to capture a time window that is too brief for the complex duty cycle of a data logger, and the power use ranges from a few μA of sleep current to many tens of mA for SD card writing during each cycle.

Hmmm…

<— Click here to continue reading the story—>