pIoT

Intrusino, a new friend in pIoT-land

It’s been a while since I haven’t published anything about pIoT.

This is because I have beenquite busy with many things, and one of these it’s this shiny new pIoT node, I’ve called it intrusino.

It’s placed over the back door and its role is to detect when people open the door and pass by.

Its main sensors are two: a simple magnetic contact sensor and an extremely small, low power, low cost movement sensor (PIR).

Both sensors are attached to an interrupt. The node works this way: it stays in low power mode (except from the PIR, which is always on) and is woken up by either the PIR or the door sensor. If the status of the PIR or the door is different from the previous one, a message is sent. Also, each hour, the node sends its state anyway.

I have observed that the PIR sensor is not completely stable, which means that, if one passes next to it, it will produce a short burst of 1 and 0 zeros. In order to avoid sending too many packages, I have programmed the node to only send PIR updates if an update hasn’t been sent for more than 2 minutes.

In addition, I have also plugged the usual light sensor.

Here is how a single message looks like on the server:

piot_door

The “movement” field is an average of the PIR detections over the last hour. The higher the number, the more movement was detected. “DoorOpen” means that the door was opened when this packet was sent.

If we look at some graphs over a day:
piot_lightpiot_movementpiot_movement2

We can see clearly the light over the day and also the little tail around 8 PM that is the light coming from the kitchen. In terms of movement, you can see when people was moving around the backdoor, typically in the morning to go to work, or in the afternoon to go to do some shopping. That suspicious activity around 4 AM must be the cats!

The server can also be programmed with a rule to light up an LED in the bedroom whenever the door is opened. This can be used to tell people when someone is coming in, or as an intrusion detection at night!

Here’ s how the rule looks like:

piot_door_rule

 

I am powering this node with 3 AA batteries. This was necessary because 2 batteries were not enough to power the PIR sensor, but, be careful, the PIR needs 3.3V maximum, therefore a voltage regulator is necessary. Alcaline batteries last about 1 month. In the future I will likely try to install a small solar panel on it, given that the sensor is exposed to natural light all the day.

Advertisements

A year of pIoT, lessons learnt

About a year ago I posted about an Internet of Things platform I was developing for my home: pIoT.

The platform includes a simple hardware design based on the ATMega328 micro-controller (the same as Arduino UNO), an Arduino compatible firmware, and a server, actually two servers, one made in Java and another made in nodeJS.

In April 2015 the development of the platform was more or less completed, and I started deploying nodes around the house where I am living. Plus, I dedicated a small computer to the server and a node as a gateway between the server and the rest of the network.

So the situation, at the moment, is like this:

Server

it’s composed of a Raspberry PI and a node attached to it with a USB to TTL module.

IMG_20150905_171737.jpg

The software running in the server is the version based on nodeJS (see some screenshots below). As CSS framework I have used Bootstrap, so that the web page is also easily visible on mobile phones.
The server has several functionalities:

– it gives a summary of the status of the sensors and some measurements (dashboard)
– it allows connecting to the serial port and give basic input/output
– it parses the messages coming from the pIoT node (which are in JSON)
– it sends messages to the pIoT network, for example to control some actuator
– it stores all received data into a no-SQL database (nedb) and allows visualising data and plotting simple graphs
– it allows setting up rules so that when some data arrives, an action can be performed

This slideshow requires JavaScript.

The device is located in the conservatory of the house, this choice was due to the fact that it was half way between the router (giving Internet access) and most of the nodes.
I have also setup a dynamic DNS and a static port routing so that the server can be accessible from outside of the local network (no, I won’t give you the address).

Nodes

there are three nodes at the moment in my house (not considering the one connected to the server).
One node measures light intensity, temperature and soil moisture. It is located in the conservatory and also measures the moisture of the soil of a plant. The intention is to monitor the status of the plant and warn when the level of water is too low. The node is operated by two AA batteries and sends data each hour. In a previous post, I described how to build the moisture sensor.

IMG_20151210_093915

A second node is located outdoor. It measures light intensity, temperature, humidity, soil moisture and the presence of rain. Its main purpose is to give information about the weather. This node is operated by 3 rechargeable AA batteries (it needs a low-power voltage regulator to not burn the radio module) and a 6V solar panel. It also sends data each hour.

IMG_20151210_093844

A third node is located indoor, in my bedroom. It measures light, temperature and humidity and has a powerful RGB LED that can be activated remotely (in that sense it’s both a sensor and an actuator). The intention is to monitor the temperature in home and to have a sort of visual output of the system. This node is operated by a USB charger, so it includes a voltage regulator to bring the voltage down to 3.3V. The bedroom is located opposite to the conservatory in the house plant, so data transmission is quite difficult as there are some walls to be traversed. Data are sent each hour, but the node is continuously listening for incoming commands, that’s why the choice of powering it from the line, otherwise batteries would have been consumed very quickly.

IMG_20150905_162206

Collected data

So far I have received 27359 messages. Most of the messages (12801) are of type “hello” which is sent by all nodes and describes the internal status of the node. These messages include information like the number of packets lost, the operating voltage, the internal temperature as measured in the microcontroller.
Another big chunk (11360 message) is represented by “garden” messages which are sent by the outdoor node and the one in the conservatory. These messages include temperature, light, humidity, rain presence and soil moisture.
The other two categories of messages are “room” (3161 message), that contain information about the light, temperature and humidity of the bedoorm and “color” (only 37) which descirbe the status of the RGB LED.

Consider that the system was not always 100%, 24h/24 ON. It had some down times because I was updating something, or because the power went off or other reasons. But I can probably say that the system was ON at least 80% of the time within this year.

In total, the database occupies about 4 MB, which is prefeclty bearable by the Raspberry. Consider that nedb loads all the data into memory, so, if the DB grows a lot, probably I’ll have to archive some old data.

So now that we know what is the deployment and what data is there more or less, here’s the lessons learned.

Lessons learned:

Crappy batteries are… crap

I have used a wide range of batteries during this year. From expensive Duracell, to very cheap batteries bought in Poundland. Well, the difference is evident.

In the following graph you can see how fast a “good” battery is depleted on a node sending a message each hour.

batteries_good

That’s about 2 months. Now see what happens with cheap batteries:

batteries_bad

That’s only about 2 weeks!

In theory, at the rate I am sending data now, if you switch all modules off properly, batteries should last several months if not years. I am not sure why I am consuming power faster than foreseen, I have tried different approaches to improve it, but the situation doesn’t change. Changing batteries every 2 months is not too bad, but it’s not ideal either.

Solar works!

So, here a possible solution to power consumption: adding a solar panel.

And it works!

During winter I had to charge the batteries myself manually about 3 times. Since days have started to last longer (about March) I have never had to recharge the batteries again!

solar

In the graph you can see that the voltage is practically constant at 3.4V except one value, probably an artefact.

Packet loss is inevitable

pIoT configures the NRF24 module to retry transmitting a packet up to 7 times in case of failure. This gives a good amount of assurance that the packet is sent, if the connection is good enough. Even with this setting, packet loss happens.

I have computed the average packet loss for a the three nodes installed at home: for the node located in the conservatory, I get 18% of packets wasted, for the one outdoor: 24% and for the one indoor, almost 51%. That’s quite a big percentage, but it’s also expected. The nRF24 module was not designed for this range. Even though it says that it can reach 100 m, in reality it was thought for very short range communications, like for wireless keyboards or mouse. Still, it’s quite impressive that we can get useful data in an environment such as a house made of bricks.

… and sometimes it works unexpectedly well

It is not surprising that the node located in a room opposite to where the receiver is has such a high packet loss. Actually it is quite surprising that it receives data at all.

I have also observed that, when packets are sent by the server though, they are quite likely to be received by the node even though the packet loss on the other direction is pretty high. I don’t know exactly why, but it’s pretty reliable. And that’s also a very good news as, in fact, that node is used to warn the user when some action has to be taken (like pouring water to the plants,).

August is nicer than December

That’s a pretty obvious one, at least here in UK. Let’s see it from the perspective of natural light. The following two pictures show the light as measured by the outdoor node in a day of August 2015 and on in December 2016. You can see clearly that in August light starts at 4:00 and ends around 20:00, while in December it starts at 8:00 and ends at 16:00. Eight hours difference!

This slideshow requires JavaScript.

If we look at the outdoor temperature on two random days of August and December 2015:

This slideshow requires JavaScript.

Temperature in December spans from few degrees over 0 to 10 degrees (Celsius). In August we get temperatures between 10 and 20. It’s interesting to see that the range of variation in a day: in both months it’s about 10 degrees.

…but my room’s temperature is stable

the two charts below show the temperature in my room as measured by the indoor node. One chart is related to a day in August, the other to a day in Feburary.

As you can see, although there’s an obvious difference between the two days, the temperature, is quite stable throughout the day.

This slideshow requires JavaScript.

Micro-controller internal temperature needs calibration

I have used this trick to measure the internal temperature of the ATMega328. I have calibrated the raw values as received by the ADC with a thermometer I had at home and using an air dryer to raise the temperature. There was a very clear correlation between the raw values and the actual temperature and the formula I gathered was:  temperature = (ADCW * 0.9873) – 330.12. This calibration was done on one single MCU.

When using the same formula on other nodes, though, it stopped working. As the charts below show, there is an evident correlation between the temperature as measured internally and as measured with a more precise DHT, but the internal measurement is not correct. This means that, basically, each single microcontroller needs a different calibration.

This slideshow requires JavaScript.

s*#t happens!

Yes, and very often too!

Here’s an example of actual 💩:

shit

Why do these things happen?

I have no idea, but I can only speculate that, although the NRF24L01 has its internal checksum, some data corruption happens anyway. pIoT libraries do not add an extra checksum, so, if you need absolutely clean data, you’d better add it.

Electronics is incredibly resilient

Well, mostly.

Look at this node:

it has been in the wild for 16 months: rain or sun, winter or summer, with high humidity, insects of all sorts trying to build their home inside, and it was all covered by just a plastic box (actually of cotton buds). It’s a bit rusty inside, but still, it was able to survive and send data without (almost) any human intervention.

Soil moisture measurement is not sustainable

The outdoor node measures moisture thanks to two metal nails.That’s not unprecedented, but it’s not very reliable either. The nails tend to get rusty and to corrode over time (see picture).

IMG_20160828_191915

Nonetheless, as a rough estimation, it’s still good enough. The following charts show soil moisture and rain over the same period of time. There is some sort of correlation between the two, although not very strong.

This slideshow requires JavaScript.

 

Conclusion

My personal conclusion is that pIoT, as it is now, is a very interesting prototype, but it can’t compete with commercial, ready-made solutions. I wouldn’t rely on my pIoT installation to run critical things like theft alarm, or fire extinguishers, but it’s more than enough for things like gardening, or just for learning some electronics!

There are surely other interesting, and maybe less-obvious, findings that can be derived from the data collected over more than one year. For example, from the indoor node I may get some behavioural information (e.g. sleep patterns). So I may update this post with some other findings if they occur to me later. Meanwhile, if anybody wants to have a peek at my data for doing some research, I’ll be happy to give them !