9 ms·
Build an Air Quality Monitor with InfluxDB, Grafana and Docker on a Raspberry Pi
- Uhhrrr 8y agoThe Docker part seems thoroughly unnecessary.
- tylerflick 8y agoI can't understand why they would add anymore overhead on such a low powered device.
- jjeaff 8y agoEase of setup and reproducibility probably. And docker doesn't add much overhead, especially when running natively on Linux.
- Gigablah 8y agoWhat overhead? "Docker induces no significant overhead on CPU nor memory usage, compared to a native execution (worse observation: -4%; 0% on all others)" [1] [1]: https://roudier.io/2015/08/docker-vs-kvm-vs-native-performance-comparison/ https://roudier.io/2015/08/docker-vs-kvm-vs-native-performan...
- krageon 8y agoThink about how the stack works. I'm not saying this measurement is definitely wrong, but if you're finding exactly 0 overhead then you have to suspect there is something weird going on with how you're measuring it.
- Gigablah 8y agoThe processes are still running natively. The most common overheads would be due to network and storage driver and those can be mitigated with some simple settings. The Docker daemon is more or less a process supervisor at this point.
- krageon 8y agoI think we have a fundamentally different view of what "natively" means and what it means to be "more or less a process supervisor". That's fine, but it also means we won't get to an agreement that we are both at peace with in this case.
- tinco 8y agoWhat feature of Docker makes you consider it significantly more than a process supervisor?
- viraptor 8y agoZero and original raspberries can be low-powered. 3+ is a quad core with 1gb memory. Docker doesn't even show up on CPU monitor and takes 2% shared memory in my case. The overhead of a namespace is not really noticeable either, unless you're doing lots of processing. (My influx+grafana+monitors sit under 5% CPU almost all the time)
- dikei 8y agoAgree, an Ansible playbook would have easily suffice.
- flo123456 8y agoWhile I do like Ansible, it‘s not really the right tool for a tutorial about sensors. It‘s a lot easier to explain that you need to pull & run some image than it is to explain installing python, get your ansible host configuration right and then run a playbook. There’s just a lot more variance (and therefore margin for error) that you don’t want to deal with when you’re explaining something else entirely.
- joshvm 8y agoOne downside of Ansible on the Pi is that a lot of stuff involves building from scratch, because aren't armhf packages available, or what's available in apt is hopelessly outdated. I have a stack at work that takes a day to build on the hardware. With Docker I can build once and pull the image later. I guess you could do similar with Ansible and copy build artefacts over, but Docker is a simpler solution.
- alexandros 8y agoHey there - BalenaOS is built to run containerised workloads on small devices and is quite stripped down otherwise, kinda like coreOS. So in that sense our architecture allows you to have less stuff you don't need floating around, saving you overhead. We focused on Docker containers since balenaCloud is built for fleets. It's important that our stack supports not just one but many copies of the same device running the same code, and then it can be updated in production etc, just like a set of servers. If one is optimizing for a single device, just naked Raspbian will do fine of course. Our approach has a bit of overhead up front in terms of setup, but you get a production ready setup from day one in return.
- viraptor 8y agoIt gives you easy upgrades and nice dependency separation. It's not required in this case, but it sure makes operations easier. (Doing the same for multiple services on rpi)
- StreakyCobra 8y agoIf you do not want to need 10 different RPIs for running 10 different functionalities (Pi-hole, RetroPie, remote access, cloud, side-projects, air quality monitoring, etc.), docker is a good solution to easily deploy several services on a single RPI, and it prevents dependency hell.
- gpm 8y agoWhat do people think about the recommended sensor? Is it reasonably accurate? Are there better alternatives?
- gpm 8y agoFor anyone else who goes looking for the datasheet - the link from the shop is broken but is saved in the wayback machine. https://web.archive.org/web/20180130060734/https://ae-bst.resource.bosch.com/media/_tech/media/datasheets/BST-BME680-DS001-00.pdf https://web.archive.org/web/20180130060734/https://ae-bst.re...
- deleted 8y ago[deleted]
- gpm 8y agoSomeone replied with this interesting and then deleted their comment. It doesn't include the same sensor, but it does include a related one and it's a wonderfully detailed analysis: Thank you for linking it! https://www.kandrsmith.org/RJS/Misc/Hygrometers/calib_many.html https://www.kandrsmith.org/RJS/Misc/Hygrometers/calib_many.h...
- peferron 8y agoThe sensor doesn't appear to measure particulate matter, so I'd take the air quality reading with a large grain of salt.
- gpm 8y agoAgreed - it's pretty explicitly just a volatile organic compound (gas) detector. On the other hand VOCs, especially as a proxy for CO2 and air circulation, are also interesting.
- ekianjo 8y agoIt does not even measure Ozone concentration which is one of the key things you would want to look at for air quality...
- flo123456 8y ago
- shawnps 8y agoIf you're interested in purchasing an air quality monitor, Awair is nice: https://getawair.com/ https://getawair.com/ I learned about it from DHH's talk on air quality: https://www.youtube.com/watch?v=MRqh8oLY7Ik https://www.youtube.com/watch?v=MRqh8oLY7Ik
- bhargav 8y agoDid you simply use one Awair device around your entire house?
- borumpilot 8y agoUnfortunately, only shipping to the US & Canada.
- Luc 8y agoDid you try https://getawair.co.uk/ https://getawair.co.uk/ ?
- zantana 8y agoI was really keen on the Awair, but the necessity to be connected to wifi to function was a deal killer for me. From what I've seen this will never change which is disappointing.
- deleted 8y ago[deleted]
- wuyishan 8y agoA similar, yet completely different project: https://luftdaten.info/en/home-en/ https://luftdaten.info/en/home-en/ Can be seen as a "worldwide crowd sourced air quality measurement projects using cheap DIY sensor/data collector"
- bayesian_horse 8y agoI'm working on similar things, and at the moment I prefer Redis and Postgres. Redis just got streams, a bit like Kafka, and Postgres has extensions for timeseries.
- Animats 8y agoWhat's impressive is that you can get a sensor for temperature, air pressure, "air quality", and humidity for $12.80 in quantity 1. "Air quality" is strange. It measures volatile organic components, not CO, CO2, or particulates. Sensors for those cost much more. So this is measuring what's easy to measure, not what's useful for, say, HVAC control. 8GB and Docker, plus a "cloud" server, seems a bit much.
- arcturus17 8y agoCan anyone recommend a better sensor? I’d like to build an upgraded version of this - that way I’ll also learn more.
- TheSpiceIsLife 8y agoAssuming I have no idea about these things, which is true—I don’t—how much is a decent CO2 sensor that can be connected to a Pi or similar setup? I recently upgraded to double glazed windows in half the house, including all the bedrooms, and put a heat pump in each of the two used bedrooms. So I want to keep the windows closed for heat retention, but also want to keep them slightly open to reduce CO2 build up. I sleep in my small bedroom with my two 16kg dogs, so it seems reason the CO2 level may rise to suboptimal during the night if the window is shut. What I would like to do is make a DIY ventilation system that can move fresh, HEPA filtered, air from outside, in to the room. Or am I over complicating things? Could I just use a small fan, something sized to replace, say, half the volume of the room every few hours, and just leave it constantly on? Or should I put a larger fan on a timer so it runs for a few minutes every hour? Does any of this really matter? Edit to add: any good resources to lose myself down a rabbit hole?
- joshvm 8y agoA good CO2 sensor that uses infrared absorption to measure concentration costs about $100. https://www.co2meter.com/collections/co2-sensors https://www.co2meter.com/collections/co2-sensors
- TheSpiceIsLife 8y ago
- TheSpiceIsLife 8y agoI'm curious to hear from anyone who runs air quality monitors at home: how has knowledge of the quality of the air in your house affected your behaviour? My house is 56 years old. I'm guessing the kitchen upgrade is at least 5 years old. The carpet is probably at least a 5 years old. There isn't much fresh paint in here. Am I about right to assume I probably don't have much in the way off off-gassing from anything in here, or does carpet / furniture / melamine continually off-gas for it's entire life? Anyway, I've got some idea that CO2 buildup is a thing, and humidity, and off-gassing, so... I tend to ventilate the house frequently even during winter. Even when it's -5 outside, I'll occasionally open one or more windows / doors and turn the kitchen exhaust fan on in order to draw fresh air in and across the house. I'll also occasionally open multiple doors / windows for 10 minutes or so in an effort to replace the majority of the air in the house. I tend to eschew yet-another-device that needs maintenance / power / charging / monitoring / fiddling because I tend to do a lot of that at work and would rather just come home and not-have-to-maintain-another-97-machines. So I'm interested in the on average / sort of good enough / sometimes over-shoot-the-mark behavioural changes that would result if I did have air quality monitors.
- hadlock 8y agoMost air quality measurement involves particulate matter, specifically PM 2.5. This is primarily wood smoke, car exhaust and a couple of other combustion-related. When the news talks about air quality, most commonly they are referring to PM 2.5. If you are looking for out-gassing (paints, carpets, furniture etc) you probably need to invest in formaldehyde and other gas sensors. In your case you ought to look at your furniture and electronics as they will produce the most out-gassing. This is different from the above particulate matter.
- msisk6 8y agoWe have a new house--about a year old now--here in Texas with all the efficiency and insulation stuff. I noticed after working in my office with the door and windows closed I didn't feel so good after awhile. I got one of those trendy Awair monitors and stuck it on my desk and hooked up the iPhone app. And sure enough, when I started not feeling well it was because the CO2 was over 1500 ppm. I don't know about the accuracy of the actual CO2 measurement, but I figure the delta is at least good enough to spot trends. And yeah, those times I don't feel well is always associated with a rise of CO2. I "fix" it by either opening the door and/or window or running the fan of the central air system. Our system also has a "bleed air" adjustment for outside air. I turned that up a bit, too. The Awair also does "chemicals" and PM2.5. I'm surprised our new house doesn't an issues with those, but I do see PM2.5 go up when the window in my office is open and "chemicals" fluctuates throughout the day but at a low level. I'm not exactly sure what "chemicals" means in this context; I haven't really looked into it yet. You might consider the Awair; I've been happy with it. I have several thousand systems that need tending at work and, same as you, I don't need to come home and do more. My wife's Windows gaming rig needs enough of that as it is...
- pookieinc 8y agoWe've been on the hunt for the most accurate Air Quality Monitor since we've just moved to the Middle East and have had a really tough time finding something accurate. Aside from building one and having it receive questionable ratings, does anyone have any suggestions on a super reliable machine they've been happy with that has proven its accuracy in some way?
- joshvm 8y agoWhat do you want to measure? Saying you need it to be super accurate doesn't mean much on its own. Air quality is subjective in any case. The BME680 (like other chips) measures VOC content - organic stuff like alcohol. You can also measure dust concentration, for example if you lived in China, PM2. 5 would be good to know. And you can also use CO2 concentration, but good CO2 sensors cost a lot. Some chips measure "equivalent CO2" but it's not the same.
- mike_h 8y agoHere’s a lab writeup comparing a few models from about a year ago: https://smartairfilters.com/en/blog/are-cheap-particle-counters-accurate/ https://smartairfilters.com/en/blog/are-cheap-particle-count... Depending on which part of the Middle East you’re in, you may need a PM10 (dust) monitor. The devices here just do PM2.5 (smog).
- zensavona 8y agoProbably want to add a PM (particulate matter) sensor to the mix. I've built a few similar devices using ESP chips and various laser dust sensors, highly recommend the Plantower PMS5003 - laser defraction, PM1-PM10 accuracy, ~$20 and there are good libraries available on GitHub for interacting with it. Edit: https://twitter.com/zensavona/status/1091949965306257409 https://twitter.com/zensavona/status/1091949965306257409
- luma 8y agoI'm using a very similar stack! I've built 20 devices using the PMS5003, Spec Sensors O3 digital gas sensor, and a cheap GPS module connected to an ESP32. 6 have been mounted in the field (more coming soon!). They report sensor readings via LoRaWAN to a public-facing dashboard visible here: https://graqm.org https://graqm.org Devices being tested at an EPA air quality monitoring station: https://imgur.com/a/joQvj https://imgur.com/a/joQvj Desktop version: https://imgur.com/a/yT1IK https://imgur.com/a/yT1IK
- bugsense 8y agoDitch Grafana+Influx for Netdata and you are golden :)
- alexandros 8y agoThat's a good point! We should try that as an alternative next :)
- tpxl 8y agoReal question, why? I'm running grafana + influx and it's doing the job well. The netdata git repo is full of marketing fluff and fails to convince me it's any better.
- betimsl 8y agoMeasuring a sensor is so complicated...Influx, Grafana and Docker is not enough. You may want to throw in AWS, some Google API and not to forget: Tensorflow. A pinch of opencv with GPU support might make it real software. Cross compile all this for windows that runs on RPi so corporate clients can trust it -- because buying a license always reflects seriousness.
- Ysx 8y agoI'd have solved it same way as the article, as they're tools I'm familiar with - feels like overkill though. How would you have managed it?
- IanSanders 8y agoArduino nano, a few sensors, display, and a few buttons
- mschaef 8y agoNot OP, but here's how I did solve it: https://github.com/mschaef/metlog https://github.com/mschaef/metlog This was mainly an exploratory thing to learn some new technologies, but the gist is that it's a Clojure/Clojurescript application with HSQLDB on the back end. It runs reliably and has for years, but the more 'roll your own' experience means continually finding out that there's some other bit of missing functionality you either need to implement or live without. A couple specific observations on various elements of the tech stack: * HSQLDB - Nice little database, well documented, and has served its purpose well. Just now getting to the point where query lengths on long running time series are a performance issue. I think my mitigation strategy will be to do more in memory. * core.async - I use this in the front end to manage sourcing data from the server and getting it into the graphing components. Three or four different approaches later, I'm still not sure I fully get how or why this should be used. * Reagent - Definitely the right choice. About the perfect level of abstraction. Wish I'd spent a bit more time reading the docs ahead of time, though. * Clojure - This has been a great choice, but to be honest, there's not much code on the server side, so it's not at all heavily used. * ClojureScript - Also a good choice (Figwheelr helps), but it's fundamentally a different language with different core data structures sitting in a land that's solidly entrenched in JavaScript. That impedence mismatch is a continual low grade annoyance. * HTML Canvas - Easy to use, but lots of tricky little edge cases to worry about. (The latest being how to find a way to get non-blurry pixel accurate display on a Retina laptop.)
- djhworld 8y agoHas anyone been running this in the field for a few months/weeks. I'm wondering how the SD card gets on with the constant writes into InfluxDB (assuming the samplesa are taken frequently)
- Already__Taken 8y agoSD cards fail but they're not a house of cards. I've logged into couch dB every minute for nearly 5 years. only about a million entries bit since it's replicated to other nodes a failed card is ok.