A year of pIoT, lessons learnt

About a year ago I posted about an Internet of Things platform I was developing for my home: pIoT.

The platform includes a simple hardware design based on the ATMega328 micro-controller (the same as Arduino UNO), an Arduino compatible firmware, and a server, actually two servers, one made in Java and another made in nodeJS.

In April 2015 the development of the platform was more or less completed, and I started deploying nodes around the house where I am living. Plus, I dedicated a small computer to the server and a node as a gateway between the server and the rest of the network.

So the situation, at the moment, is like this:

Server

it’s composed of a Raspberry PI and a node attached to it with a USB to TTL module.

IMG_20150905_171737.jpg

The software running in the server is the version based on nodeJS (see some screenshots below). As CSS framework I have used Bootstrap, so that the web page is also easily visible on mobile phones.
The server has several functionalities:

– it gives a summary of the status of the sensors and some measurements (dashboard)
– it allows connecting to the serial port and give basic input/output
– it parses the messages coming from the pIoT node (which are in JSON)
– it sends messages to the pIoT network, for example to control some actuator
– it stores all received data into a no-SQL database (nedb) and allows visualising data and plotting simple graphs
– it allows setting up rules so that when some data arrives, an action can be performed

This slideshow requires JavaScript.

The device is located in the conservatory of the house, this choice was due to the fact that it was half way between the router (giving Internet access) and most of the nodes.
I have also setup a dynamic DNS and a static port routing so that the server can be accessible from outside of the local network (no, I won’t give you the address).

Nodes

there are three nodes at the moment in my house (not considering the one connected to the server).
One node measures light intensity, temperature and soil moisture. It is located in the conservatory and also measures the moisture of the soil of a plant. The intention is to monitor the status of the plant and warn when the level of water is too low. The node is operated by two AA batteries and sends data each hour. In a previous post, I described how to build the moisture sensor.

IMG_20151210_093915

A second node is located outdoor. It measures light intensity, temperature, humidity, soil moisture and the presence of rain. Its main purpose is to give information about the weather. This node is operated by 3 rechargeable AA batteries (it needs a low-power voltage regulator to not burn the radio module) and a 6V solar panel. It also sends data each hour.

IMG_20151210_093844

A third node is located indoor, in my bedroom. It measures light, temperature and humidity and has a powerful RGB LED that can be activated remotely (in that sense it’s both a sensor and an actuator). The intention is to monitor the temperature in home and to have a sort of visual output of the system. This node is operated by a USB charger, so it includes a voltage regulator to bring the voltage down to 3.3V. The bedroom is located opposite to the conservatory in the house plant, so data transmission is quite difficult as there are some walls to be traversed. Data are sent each hour, but the node is continuously listening for incoming commands, that’s why the choice of powering it from the line, otherwise batteries would have been consumed very quickly.

IMG_20150905_162206

Collected data

So far I have received 27359 messages. Most of the messages (12801) are of type “hello” which is sent by all nodes and describes the internal status of the node. These messages include information like the number of packets lost, the operating voltage, the internal temperature as measured in the microcontroller.
Another big chunk (11360 message) is represented by “garden” messages which are sent by the outdoor node and the one in the conservatory. These messages include temperature, light, humidity, rain presence and soil moisture.
The other two categories of messages are “room” (3161 message), that contain information about the light, temperature and humidity of the bedoorm and “color” (only 37) which descirbe the status of the RGB LED.

Consider that the system was not always 100%, 24h/24 ON. It had some down times because I was updating something, or because the power went off or other reasons. But I can probably say that the system was ON at least 80% of the time within this year.

In total, the database occupies about 4 MB, which is prefeclty bearable by the Raspberry. Consider that nedb loads all the data into memory, so, if the DB grows a lot, probably I’ll have to archive some old data.

So now that we know what is the deployment and what data is there more or less, here’s the lessons learned.

Lessons learned:

Crappy batteries are… crap

I have used a wide range of batteries during this year. From expensive Duracell, to very cheap batteries bought in Poundland. Well, the difference is evident.

In the following graph you can see how fast a “good” battery is depleted on a node sending a message each hour.

batteries_good

That’s about 2 months. Now see what happens with cheap batteries:

batteries_bad

That’s only about 2 weeks!

In theory, at the rate I am sending data now, if you switch all modules off properly, batteries should last several months if not years. I am not sure why I am consuming power faster than foreseen, I have tried different approaches to improve it, but the situation doesn’t change. Changing batteries every 2 months is not too bad, but it’s not ideal either.

Solar works!

So, here a possible solution to power consumption: adding a solar panel.

And it works!

During winter I had to charge the batteries myself manually about 3 times. Since days have started to last longer (about March) I have never had to recharge the batteries again!

solar

In the graph you can see that the voltage is practically constant at 3.4V except one value, probably an artefact.

Packet loss is inevitable

pIoT configures the NRF24 module to retry transmitting a packet up to 7 times in case of failure. This gives a good amount of assurance that the packet is sent, if the connection is good enough. Even with this setting, packet loss happens.

I have computed the average packet loss for a the three nodes installed at home: for the node located in the conservatory, I get 18% of packets wasted, for the one outdoor: 24% and for the one indoor, almost 51%. That’s quite a big percentage, but it’s also expected. The nRF24 module was not designed for this range. Even though it says that it can reach 100 m, in reality it was thought for very short range communications, like for wireless keyboards or mouse. Still, it’s quite impressive that we can get useful data in an environment such as a house made of bricks.

… and sometimes it works unexpectedly well

It is not surprising that the node located in a room opposite to where the receiver is has such a high packet loss. Actually it is quite surprising that it receives data at all.

I have also observed that, when packets are sent by the server though, they are quite likely to be received by the node even though the packet loss on the other direction is pretty high. I don’t know exactly why, but it’s pretty reliable. And that’s also a very good news as, in fact, that node is used to warn the user when some action has to be taken (like pouring water to the plants,).

August is nicer than December

That’s a pretty obvious one, at least here in UK. Let’s see it from the perspective of natural light. The following two pictures show the light as measured by the outdoor node in a day of August 2015 and on in December 2016. You can see clearly that in August light starts at 4:00 and ends around 20:00, while in December it starts at 8:00 and ends at 16:00. Eight hours difference!

This slideshow requires JavaScript.

If we look at the outdoor temperature on two random days of August and December 2015:

This slideshow requires JavaScript.

Temperature in December spans from few degrees over 0 to 10 degrees (Celsius). In August we get temperatures between 10 and 20. It’s interesting to see that the range of variation in a day: in both months it’s about 10 degrees.

…but my room’s temperature is stable

the two charts below show the temperature in my room as measured by the indoor node. One chart is related to a day in August, the other to a day in Feburary.

As you can see, although there’s an obvious difference between the two days, the temperature, is quite stable throughout the day.

This slideshow requires JavaScript.

Micro-controller internal temperature needs calibration

I have used this trick to measure the internal temperature of the ATMega328. I have calibrated the raw values as received by the ADC with a thermometer I had at home and using an air dryer to raise the temperature. There was a very clear correlation between the raw values and the actual temperature and the formula I gathered was:  temperature = (ADCW * 0.9873) – 330.12. This calibration was done on one single MCU.

When using the same formula on other nodes, though, it stopped working. As the charts below show, there is an evident correlation between the temperature as measured internally and as measured with a more precise DHT, but the internal measurement is not correct. This means that, basically, each single microcontroller needs a different calibration.

This slideshow requires JavaScript.

s*#t happens!

Yes, and very often too!

Here’s an example of actual 💩:

shit

Why do these things happen?

I have no idea, but I can only speculate that, although the NRF24L01 has its internal checksum, some data corruption happens anyway. pIoT libraries do not add an extra checksum, so, if you need absolutely clean data, you’d better add it.

Electronics is incredibly resilient

Well, mostly.

Look at this node:

it has been in the wild for 16 months: rain or sun, winter or summer, with high humidity, insects of all sorts trying to build their home inside, and it was all covered by just a plastic box (actually of cotton buds). It’s a bit rusty inside, but still, it was able to survive and send data without (almost) any human intervention.

Soil moisture measurement is not sustainable

The outdoor node measures moisture thanks to two metal nails.That’s not unprecedented, but it’s not very reliable either. The nails tend to get rusty and to corrode over time (see picture).

IMG_20160828_191915

Nonetheless, as a rough estimation, it’s still good enough. The following charts show soil moisture and rain over the same period of time. There is some sort of correlation between the two, although not very strong.

This slideshow requires JavaScript.

 

Conclusion

My personal conclusion is that pIoT, as it is now, is a very interesting prototype, but it can’t compete with commercial, ready-made solutions. I wouldn’t rely on my pIoT installation to run critical things like theft alarm, or fire extinguishers, but it’s more than enough for things like gardening, or just for learning some electronics!

There are surely other interesting, and maybe less-obvious, findings that can be derived from the data collected over more than one year. For example, from the indoor node I may get some behavioural information (e.g. sleep patterns). So I may update this post with some other findings if they occur to me later. Meanwhile, if anybody wants to have a peek at my data for doing some research, I’ll be happy to give them !

A (tentative) GPS navigator for bicycles

Few weeks ago I participated at an hackathon about bicycles called “Pedalling Innovation Oxford“. A friend of mine and I presented the idea of a navigation system for bicycles.

The idea is very similar to some existing products like hammerhead, beeline, smart halo or onomo. We wanted to make everything open source, starting from the hardware.

The idea:

The initial idea was to have some kind of device to be mounted on the handle bar with Bluetooth connection to a smartphone. The device would have two bright lights, indicating when to turn (left or right), but the navigation software and the GPS would be in the phone. As microcontroller we chose RFduino because it includes Bluetooth LE and it’s low cost and low power. As navigation software we chose OSMAnd because it’s open source, it uses the wonderful Open Street Map maps (also offline) and has turn-by-turn navigation.

The reality:

we spent about 5 hours only trying to compile OSMAnd. It’s quite a complex project, with lots of dependencies and, eventually, we gave up.We decided instead to use some online navigation system, with turn-by-turn instructions like mapzen. It’s less optimal and requires  frequently querying the route (in case you miss the turn), but it’s OK for a first prototype!

As for the hardware, we managed to put together an RFduino with a couple of bright LEDs and a simple firmware that flashes the lights based on the received command.

I’ll try to put all the code on github, but it’ll need some polishing first…

 

Here’s some pictures of the event:

 

pIoT a pico/personal Internet of Things DIY platform

This post is a follow up and an evolution of this previous one. I am pretty proud to present here a work that has been going on for more than a year. It all started with some friends at MakeSpaceMadrid. We wanted to create a super-cheap, Arduino-compatible with embedded wireless communication, electronic board. We decided to adopt the very well-known Arduino UNO ATMega 328p as microcontroller and the nRF24L01 wireless chipset as communication module, both sold for about 1€ on the Chinese market. We called our project Sensorino. We did a lot of things, but the project took longer than expected. Its code is still on github and everybody is welcome to contribute. While Sensorino was slowly being developed, I forked it on a personal, simpler project, and called it pIoT.

So what´s pIoT?

It’s a collection of three things: a hardware design for custom boards with a microcontroller and the wireless communication chip, the firmware that runs on the microcontroller and a server that collects data, shows it and executes rules. The project is open source and available on github for anyone to download or contribute.

pIoT board

A pIoT board

At the moment most of the functionalities are there: the board design is extremely simple, it’s just the ATMega328p connected to an nRF24L01+ module, the firmware contains a library for exchanging messages over the nRF24 module, a library for managing energy consumption (that is, putting the microcontroller into sleep modes) and a library for exchanging JSON messages with a host computer, the server software is programmed in java and takes care of parsing packets, visualising data on web using GWT and implementing rules using the mvel expression language. A new version of the server for nodejs is also on its way.

server

A screenshot of the server interface

The topology of the network is simple, there is one “base” node connected to a host computer through a serial port, the computer running the server code, and a set of “peripheral” nodes, which can be both sensors or actuators, that send/receive data to the base. More complex topologies are possible, but routing is not supported at the moment (for this, I suggest the very interesting RadioHead project). I am writing a tutorial for helping set up the whole thing.

pIoT topology

pIoT topology

I have created a testbed at home, with an old laptop working as server (a Raspberry Pi would also do) and two nodes, one that senses data from a plant (as already published a in previous post) and one that acts as a remote controlled colour LED. It works, and, as far as now, battery consumption is quite good. I have had the plant sensor running on two AA batteries for a couple of months.

pIoT server

The pIoT server

Garden pIoT node

A pIoT node in the garden

 

There are still things missing of course, but a complete proof of concept is already there and promising. If you are interested in the project and willing to collaborate, I will be very pleased to help!

A simple Arduino garden sensor

This is not the first time Arduino is proposed for gardening, you can see nice examples here and here, or here, and even here. Anyaway, I would like to tell you how I made it even though it’s not that original.

My sensor has three elements: a moisture sensor for the soil, a light sensor and rain sensor.
Let’s see the three parts separately.

Light sensor:

it’s a very standard LDR (photoresistor) example, like this one. You connect an LDR between 5V and an analog pin and the pin to a 47KΩ resistor that goes to ground.

 

Moisture sensor:

I have used two nails as electrodes, as suggested in many places, like here. You need two long nails and a sponge. You solder the nails to two wires. If you can’t solder the wires, just wind them around the nails and solder just the wire. The you cut the sponge to obtain a rectangle of 5cm long and you put the nails inside, this way the wires won’t go away easily.

Then you connect one of the two wires to the 5V pin, the other to an analog pin and you connect the same pin to ground through a 10KΩ resistor.

The principle is simple, the two nails are put into the soil and act as electrodes. If the soil is completely dry, there will be no current between the electrodes and the value measured at the analog pin will be zero. With a lot of water there will be a relevant current between the nails and the measured value will be higher.

I have done some experiments and my values are like this:

dry soil: 0
some little water: 200-400
some fair water: 500-700
quite a lot of water: 700-750
soaking wet: 800-…

 

Rain sensor:

I was inspired by these kind of sensors they sell on the Internet. To emulate it, I used a proto perf PCB board the kind with holes connected in lines. I used a thin wire, peeled a piece of it, double the length of the board and “sewed” it around the lines in order to connect lines two by two. Then with another wire you do the same but with the lines that are not connected yet. This way half of the lines (the even) belong to one wire and the other half (the odd) to the other wire. When a drop of water falls between two lines, it generates a little current that can be detected by the Arduino. To wire it to Arduino, you connect one wire to 5V, the other to an analog pin and the same pin to ground with a 10kΩ resistor.

By measuring the values at the analog pin with different amounts of water I get these:

dry: 0
some drop: 300-400
fair a mount of water: 500-600
soaking wet: 800-…

 

At the end, when you put all together you have something like this:

Regarding the sketch to be used on the Arduino it’s up to you. At the moment I am just measuring the values of the analog pins and doing some experiments (the code is not even worth publishing).

 

Obnoxious flies shown on the Prado’s MediaLab facade

Maybe you remember my previous post on Obnoxious flies a Processing sketch that I programmed for the MediaLab of the Prado museum, well it seems that they have used it !

In this video, at minute 37:56 you can see it running on the facade of the building.

 

[vimeo https://vimeo.com/67309091]

 

By the way, they have opened again the call for 2014.

A minimal Arduino-compatible board with wireless connectivity

The subject of this post is a project I am involved in Makespace Madrid with some friends.

Our idea is to create a minimal Arduino-like board with wireless connection in. The project is being documented on the Makespace wiki (in Spanish). Basically we took an Atmega328p (the heart of Arduino Uno) with no strings attached and connected to a nRF24L01+, a super cheap, super low-cost 2.4GHz digital transceiver. The Atmega alone can be used as an Arduino, but at 8MHz. This post explains how to do it in detail.

This idea is not new. Just google “Arduino nrf24l01” and you’ll find examples like this, or this.

The connection between the Atmega and the RF board follows the instructions given in this post:

  • PD2 -> IRQ
  • PB0 -> CS
  • PB5 -> SCK
  • PB4 -> MISO
  • PB3 -> MOSI
  • PB1 -> CE
  • GND -> GND
  • VCC -> VCC

A quite raw prototype I have made is visible in the picture:
Image

I have also added a RGB LED to three pins of the Atmega with PWM.

As software, you will need a library that implements the communication with the RF chip (it works with SPI). I have tried the Maniacbug’s library RF24, but, a part from the basic examples that come with the library, I was not able to make my own wireless controller. I even tried some forks of the same library as suggested by Matthias Hertel in his post but wasn’t successful. I then tried the library provided by Mike McCauley here, and it worked very well at the first try!

As a demo I have created a simple sketch that can work in two ways: as a “client” that receives simple messages composed of 3 bytes, one per each color (RGB), and switches the LED on accordingly, and a “remote control” that reads these values from the serial connection or from three potentiometers and sends them. The code is available in my repository here.

Below you can see a video of the demo, the “client” is my prototype board with the LED and the “remote control” is a normal Arduino connected to a nRF24L01 and 2 potentiometers (I didn’t have a third one available !!).