28 ms·

Monitoring energy usage with smart plugs, Prometheus and Grafana

354 points by hddherman 2y ago

dainiusse 2y ago
No need for that mate, just deploy home assistant or something similar and you will get this (and more) out of the box
- mindslight 2y ago
  Right up until the Home Assistant UI turns into a lagfest, the installation dies, and you can't debug why because Docker. At least that's what happened to me. And no, it wasn't RPi SD power issues. This happened on an otherwise-stable amd64 server. The Home Assistant authors' hostility towards simple native distributions is now a show stopper for me. Long term reliability is more important than quick initial setup.
  - cyberax 2y ago
    HA is actually pretty debuggable. Just install the SSH plugin, then SSH into the HA box, and then simply "docker exec" into the target HA container.
    - mindslight 2y ago
      ... and then not have any of your usual development tools, environment, system layout, or repair techniques because you're inside someone else's "works on my system" that they threw over the wall. It's obviously possible to debug what goes on inside a Docker image. It's just not something I'm particularly interested in dealing with, especially under duress.
      - TeMPOraL 2y ago
        > because you're inside someone else's "works on my system" that they threw over the wall. FWIW, this can also be called stable state you can retreat to. And build upon, e.g. adding a layer of debugging tools. I don't really like to deal with Docker, but at least I have reasonable certainty it'll work. I prefer system package manager or MSI, but if not that, it beats having to build something when it's near-guaranteed that what I'll get is not the binary the authors had in mind, if it even runs at all. (Then again, I routinely rebuild Emacs to stay on the bleeding edge. But it took a while to work out all the usual dependency mess, and I even broke my system once doing it.)
        mindslight 2y ago
        It's certainly within my gamut to jump into an embedded system to debug it, bringing/building tools as I go. I'm just not looking to opt into doing that on something that doesn't need to be that complex in the first place. Same reason I run one decently powerful amd64 server that does many things rather than a stack of Raspberry Pis, one per software package.
        cyberax 2y ago
        But how would you do it differently? You need to host a bunch of daemons (MQTT, ZWave and ZigBee bridges, and whatever else you might need). And a bunch of these daemons can have their own gnarly dependencies (e.g. they can be written in JS and built with NPM, ugh). So you kinda _need_ to use Docker to make it at least sane. And if you're using Docker for the plugins, then why not use it for the HA core itself? And once you do that, you don't really need much from the host system. So why not use a minimalistic OS instead of something like Debian?
        mindslight 2y ago
        At the time my setup didn't require other daemons like that. But if I had been in that position, I would have just set up the other daemon under Debian and pointed HA at it. These days I'd say that NixOS captures that requirement, allowing orchestration of many daemons and other system config to be abstracted into a packaged solution (eg NixOS Mailserver), that the user can override as much or as little as they'd like. I believe NixOS does package (or at least attempts to package) HA, but given my past experience and what I believe is still the throw-it-over-the-wall desire of the HA maintainers, I'm wary of adopting it as an overarching solution. I'm certainly not ruling it out for performing some functions, like UI. I just would rather set up my automation efforts as MQTT-first, keep logging and automation rules as their own separate things, and not be fully committed to HA.
        cyberax 2y ago
        > At the time my setup didn't require other daemons like that. But if I had been in that position, I would have just set up the other daemon under Debian and pointed HA at it. You can do that just fine even now. I'm doing experiments with voice control, and I run the complete AI stack locally on my computer. So I just set up everything as regular background processes. You just can't expect HA to be able to do autoupdates for these daemons. The other problem is that most of required dependencies are not packaged in Debian. So you'll have to install multiple NodeJS servers and tons of NPMs somewhere on your system. > These days I'd say that NixOS captures that requirement, allowing orchestration of many daemons and other system config to be abstracted into a packaged solution (eg NixOS Mailserver), that the user can override as much or as little as they'd like. You can do that with HA as well. Just push in a new image, and tag it appropriately. The last time I played with Nix, it needed to download tens of gigs of data for a few programs. I don't think this is acceptable for HA. You can definitely do HA in a piecemeal fashio, but there's just no way it can be done as a reproducible system that you can give to your grandmother. Given these constraints, HAOS is actually pretty remarkable. > I just would rather set up my automation efforts as MQTT-first, keep logging and automation rules as their own separate things, and not be fully committed to HA. Raw MQTT still needs a UI that is user-friendly. And even with MQTT you'll need to run ZWave and ZigBee bridges.
      - cyberax 2y ago
        > ... and then not have any of your usual development tools, environment, system layout, or repair techniques because you're inside someone else's "works on my system" that they threw over the wall. The thing is, the "it works" is reproducible because of containers. Which is a step above just hoping that it works. HA is also easy to "patch". You can just install your custom components in `config/custom_components`, it can also be used to "override" core HA files. Finally, if you are doing intrusive development, you can easily launch HA locally. macOS, Linux, and WSL are supported. You will lose the ability to install add-ons via the addon manager, but that's about it. FWIW, I had the same aversion to their custom OS and their crazy container-based setup initially. For a couple of years, I used to run HA as a Python app and managed the dependencies manually. Then I tried the HAOS and it... kinda just worked.
  - baq 2y ago
    It's a Python app, of course being distributed as a docker image is the sanest way of doing it. I don't see why you couldn't just pip install it if you really wanted, but having been a Python developer for close to two decades, I wouldn't want to.
    - mindslight 2y ago
      I'm talking about distribution package managers, not pip.
      - darkwater 2y ago
        I happily ran a dockerized HA on a Debian for years now, no need to do any complicated debugging (and even if I did, it would not be difficult to inspect it properly)
        mindslight 2y ago
        Dockerized HA on Debian is exactly what died on me. About 5-6 years ago. I'm sure it works just fine for most people. Just once bitten, twice shy.
    - Izkata 2y ago
      That was the standard way a long time ago, and the first startup would take a really long time because it would install even more stuff. And sometimes fail. It wasn't very reliable if you used any addons, and some required a ton of extra steps that it couldn't automate like the modern deployments do now.
  - tw04 2y ago
    Nobody is preventing you from running Home Assistant core and deploying everything else yourself manually. Demanding the authors who gave you the software for free also provide support for an installation method they've offered up with no support is a bit ridiculous, don't you think? That attitude is what causes open source projects to die though...
    - mindslight 2y ago
      What do you mean "demanding support" ? I remember Home Assistant authors being actively hostile to people packaging their software outside of the official Docker or RPi images. Which is why it wasn't in the Debian repository, pushing me down that Docker path in the first place. Here's the same dynamic on an associated project in 2021: https://github.com/NixOS/nixpkgs/pull/126326 https://github.com/NixOS/nixpkgs/pull/126326 If anyone chimes in and says they've been running Home Assistant from nixpkgs (where I am now) for several years with no hiccups, then I will certainly reconsider my opinion. But based on my experience and what I've continued to read since, it feels like trying to do that is an uphill battle. One I'm not looking to take on, especially for automation I'm relying on.
      - tw04 2y ago
        So... your example is a developer of HA stating that he sees major flaws with how they're distributing his package, and that he has absolutely no interest in supporting users that pull his code in a way that is unmaintainable by him. YOU believe he should support this anyway, because of various "we promise end-users won't reach out to you" which is comically incorrect because history has shown repeatedly that a user's first step when something is broken is to google package_name broken - which will absolutely turn up the author's name. BECAUSE he doesn't want to support his software being repackaged in a way he believes isn't supportable, you're upset. You want him to support your unicorn config because that's what you want to do, and his refusal to comply makes him a bad person. Thank you for reinforcing EXACTLY why open source devs burn out. He has a workflow that he is willing and able to support and doesn't want to support anything outside of that. Your response is: but you need to do it for me because it's what I want.
- Havoc 2y ago
  Grafana is a hell of a lot nicer & controllable than HA HA is great, but it's not the answer to everything
  - madaxe_again 2y ago
    Agreed. I live off grid, so energy monitoring is a big deal for me. HA is fine for “at a glance”, but if I want any kind of detail, I use grafana. I actually have my old openhab instance still running purely as I can’t be faffed setting up all the piping from MQTT into influx again. It’s also possible to integrate the usage over time using a dynamic time window to get Wh figures from wattage, which is enormously useful for me, and is more accurate than the figures HA gives in their power system. HA is dead useful for getting alerts when the laundry finishes, though - dumb machine, smart plug, look for a sudden drop in power. Also does all our climate control. So different tools for different jobs.
  - zimpenfish 2y ago
    Seconded - HA's graphs are great for a simple "is this going up or down" glance but when you want to put a whole bunch of things together for comparison or perform aggregations or calculations, that's when you want Grafana et al.
  - Cyph0n 2y ago
    Why not both? You’ll need to run a server either way. HA can export data to Prometheus. Setting up and running HA is much easier than figuring out how to get a set of different smart devices to export metrics to Prometheus/Influx. Let HA deal with that.
  - icehawk 2y ago
    It might be, but for all of the examples in the blog post, HA does this out of the box.
- Banditoz 2y ago
  Why not both?
- whitehexagon 2y ago
  I'm also looking at a custom solution for my current migration from WiFi sockets to Zigbee. It seemed impossible to do an offline installation of home assistant, and discouraging signs for running it without an internet connection. There seems to be a sonoff usb stick that might act as a hub and allow command-line monitoring of all devices, should be perfect for feeding into grafana/prometheus.
  - baq 2y ago
    HA will happily run offline; if you mean HAOS then I don't know what it does but it's an unorthodox Linux distro, but once it installs it should also run offline without issues. I'm also using their skyconnect zigbee coordinator and it works very well.
    - whitehexagon 2y ago
      Yeah one of the tests was a RPi image and it wouldnt complete without a LAN internet connection (only got 4G). And it seemed far too weighty for a bit of home automation. I recall the online requirement was for some ntp server requests that cant be disabled.
      - baq 2y ago
        Yeah that's more of a rpi hardware requirement as it doesn't have a battery and you realistically want to have accurate time on your smart home controller, even - especially - after it cold boots after power loss.
Xerox9213 2y ago
Yeah, not to mention the ability to automate things. My latest automation: when the white noise machine is on for the baby, the doorbell volume is turned down.
- alchemist1e9 2y ago
  Could you expand on how prometheus or grafana helps you automate? I didn’t know either enabled that.
  - Jleagle 2y ago
    I think this was supposed to be a reply to one of the Home Assistant threads
    - Xerox9213 2y ago
      Correct. My mistake.
john61 2y ago
Missing from the blog. The power reading of the Tosmota device should be calibrated: https://tasmota.github.io/docs/Power-Monitoring-Calibration/ https://tasmota.github.io/docs/Power-Monitoring-Calibration/
- jhenkens 2y ago
  I would love a semi-automated way to generate a power-profile for ESP-Home. Find a smart room heater with 3 levels perhaps, and use home assistant to gather values at "Off", "1/3", "2/3", "3/3", with a downstream power plug as reference (and a known consumption of the downstream plug as well). So I can just take my EspHome plug and very quickly generate a standard set of mapping values for voltage and wattage.
  - ssl-3 2y ago
    The easy way is with a resistive space heater and a multimeter. I keep a big, dumb, thrifted "oil-filled radiator" space heater around just to use as a big, safe 3-speed dummy load with reasonably OK repeatability (nichrome heaters do not have perfect temperature coefficients, but they're stable-enough that using them to measure temperature quickly begins to be a non-starter). The level of integration you choose is entirely up to you. I don't do this kind of thing much, so I'm OK with kludging together a test rig as-needed with a handheld meter and tearing it apart when I'm done. This makes good use of my own time and tools, according to my personal proclivities. But if I were doing it often, then I might buy the equivalent of the HOPI meter that Big Clive uses in many of his videos. It displays current and voltage, multiplies them to get power, and also displays power factor -- concurrently, on separate digital displays, in real time. Or I might build something: A box with a current shunt with some panel-mount meters and appropriate connectors would not be too challenging to put together in an afternoon with parts from Amazon and Lowes, depending on one's ability and desire to deal with sheet metal at home. (I use galvanized steel handy boxes and cover plates from Lowes for all kinds of small-ish stuff. They're cheap, common, and durable-enough.) Whatever the approach, a simple space heater with multiple literal-speeds seems like a cheap and useful way to make it happen unless you're trying to automate every part of it. (But by then, making a dumb multi-speed space heater into a "smart" multi-speed space heater that can be activated programmatically with software like ESPHome and some relays is probably pretty much a no-brainer, isn't it?)
ars 2y ago
If someone else wants to try this, I strongly recommend against using WiFi for the plugs, instead use Z-Wave or Zigbee. Wifi is just not meant for this use case, it will be unhappy if you start adding a lot of devices, and it will slow down your main use of WiFi for your phone.
- seszett 2y ago
  Not the first time I hear this but I have about 20 ESP devices on my WiFi, almost all of them pushing data regularly and polling for instructions (I don't use HA, but a simpler home made solution) and I have no problem at all.
  - meatmanek 2y ago
    It helps that the IoT things almost invariably use 2.4GHz while your data-hungry computers and phones usually use 5GHz.
dsab 2y ago
Guys what are your favourite smart plug? I need one with easy integration with grafana, not sucking, and shipment to EU country?
- tetris11 2y ago
  ZigBee list of good devices: https://www.zigbee2mqtt.io/supported-devices/ https://www.zigbee2mqtt.io/supported-devices/
- ckolkey 2y ago
  I'm a huge fan of https://www.shelly.com/ https://www.shelly.com/ - they have a built in webserver and can be controlled via POST requests. No cloud needed.
  - septic-liqueur 2y ago
    I also like their products very much. I installed a few of them in my sockets. Some people argue that it's not safe to put in your sockets walls and the 16A limit is realistically lower before they can overheat and cause fire. That scared me a little bit but I think most of the reports are from bad wiring like using thin wires or not tightening the clamps enough
    - spockz 2y ago
      I have a Shelly pro 1PM in my Breaker box for the car charger. It gets to 80 degrees Celsius easily at 16A. I have two other gripes with it: 1: the slots/clamps are small which makes fitting 4mm2 wires a chore already. 6mm3 is impossible. 2: the overcurrent protection is very trigger happy. Due to solar panels in the street that voltage can vary significantly. Apparently the charger doesn’t always keep up exactly resulting in a current of 16.0001A which is more than the limit of 16 and poof, off it goes. Not sure whether this is an actual fault in the charger or some rounding error.
  - theshrike79 2y ago
    Big +1 for Shelly stuff. IIRC it was started by a hobbyist who got annoyed at the crap from Aliexpress and decided to do it properly. And that they have done.
leeoniya 2y ago
i'm trying to nudge Grafana into the direction of IoT/SCADA control, so we can be both, a great way to viz (data sources) and to control (data sinks). not personally a huge fan of having to recommend Home Assistant for that use case :) (i work at Grafana Labs)
- recursinging 2y ago
  This would be great. Were using Grafana dashboards for thermal vacuum testing. Everyone is always asking for simple SCADA functionality.
- hackernewds 2y ago
  could you expand on your comment? what is data sinks as applied to your idea?
  - leeoniya 2y ago
    similar to how you can make plugins for data sources, you can make plugins for data sinks, like write over modbus, or POST to http API or, publish to some kind of MQTT broker, etc. since IoT devices have limited storage, usually the metrics are dumped into another system like Prometheus. but that Prometheus data source used plotting cannot be used to control the device, which will have another endpoint and another API, so we need some kind of concept of data sinks. at least that's my idea right now, allow data links configured in the panels to poke some "data sink" with values that are available in the DataFrame or custom-entered into the UI, like we do with traceID for Exemplars, etc.
    - nagisa 2y ago
      Have you seen/know about NodeRED? I myself have moved away from HA/OpenHAB-likes to NodeRED as that allows me control that's more... restricting. At its heart NodeRED is a visual programming tool (and I think it wouldn't be a terrible fit for Grafana), but what NodeRED has and Grafana doesn't is a community which built a very significant number of packages that integrate with various protocols and devices used in home automation. I see a potential for some integration and/or idea sharing here. But it is also important to understand that the set of people who want to fiddle with their homes is much, much larger than the set of people who (are able to) program. Which is why HA and the likes are so popular.
stavros 2y ago
I got a Zigbee power breaker and hooked it up so all my flat's power goes through it, and then made an e-ink display to show my power consumption: https://www.stavros.io/posts/making-the-timeframe/ https://www.stavros.io/posts/making-the-timeframe/
- shrx 2y ago
  I'm curious, which power breaker do you have?
  - stavros 2y ago
    This one: https://www.aliexpress.com/item/1005005657383634.html?spm=a2g0n.order_detail.order_detail_item.3.29d9f19cVGy4lV https://www.aliexpress.com/item/1005005657383634.html?spm=a2...
    - wannacboatmovie 2y ago
      You're running all your flat's power through that $14 Chinese rubbish and never assumed that shortcuts were taken or quality would be an issue? How do you know it will continue to function as a circuit breaker and isn't just a piece of wire inside? For the uninitiated, CE marking is meaningless (it allows for self-certification). I'd like to see Big Clive do a teardown of one of those.
      - theshrike79 2y ago
        You can get less shit ones that aren't made of Chinesium from Shelly
        wannacboatmovie 2y ago
        Great suggestion. Some of their modules appear to be UL certified.
        2y ago
ryall 2y ago
It's annoying that smart plugs/bulbs etc use wifi when Powerline exists
- whitehexagon 2y ago
  I have a box of old X10 devices here, one of the most reliable home automation systems I ever set up. I only switched to WiFi when my iobridge X10 controller failed and I couldnt work out the RS232 protocol for a different controller.
- Dalewyn 2y ago
  Powerline ethernet dumps an utterly horrifying amount of electrical noise into both the wiring and the surrounding area. Please don't use powerline unless you have no other solution. Note: Powerline ethernet should not be confused with Power Over Ethernet which is perfectly fine.
  - baq 2y ago
    I'm using it on two outlets after not having enough prescience to install ethernet in my wardrobe-turned-office - got any reading materials?
    - Dalewyn 2y ago
      Don't have any to link off hand, but the basic gist is this: Electrical wiring of any sort are all antennas to varying degrees, transmitting and receiving electromagnetic signals. Most mains electrical wiring is also unshielded, meaning they readily transmit and receive electromagnetic signals. Powerline ethernet basically puts ethernet data on mains electrical wiring by utilizing the bands that aren't used for carrying power. This data is very, very noisy in electrical noise terms, and because most wiring also acts as an antenna that noise also gets broadcast everywhere. Simple electronics like your coffee machine or microwave oven won't care, but more sensitive electronics like radios can in turn receive interference from both the power line and the noise broadcast into the air.
      - 2y ago
WesolyKubeczek 2y ago
You need to be careful with what plugs you choose, though, because they each have, let's say, their own peculiarities. For instance, their overvoltage protection might not align well with what the local regulations say. For example, in my region of EU, the upper voltage tolerances are such that 264V must trigger an instant poweroff, and also anything producing power must shut off if the average voltage over the last 10 minutes was 253V or more. However, TuYa sockets which pretty much are the only in-wall variety I was able to find on the local market, shut off at 260V. This tends to be somewhat problematic in an area saturated with PV installations, like the one I'm living in. This problem is compounded by the fact that the reported measurements of sockets sitting on the same phase tend to differ quite a bit. Some sockets tend to overstate the voltage compared to neighboring sockets sitting on the same wires. Thus, they shut off when they think it's 260V, while it might just as well be 255V. Just saying that if you put lots of those in your walls, you might suddenly find yourself in a need to prepare some automations to try and bring the sockets on once the voltage is back to normal. This particular variety of sockets won't come back on after the voltage drops.
- baq 2y ago
  Shelly relays can be configured to do all these things and the voltage safety threshold itself is also configurable (at least in the Plus devices) but they aren't zigbee. Otherwise great little devices.
  - darkwater 2y ago
    I cannot really understand why Shelly doesn't offer ZigBee (and now Matter) options, only Wifi. This has always been the biggest blocker for me.
    - nagisa 2y ago
      They have z-wave options, though.
  - WesolyKubeczek 2y ago
    Cannot find in-wall options much.
richardjennings 2y ago
I scrape power usage metrics from Tapo P110s and push them to Grafana Cloud using https://github.com/richardjennings/tapmon https://github.com/richardjennings/tapmon - although as other commenters have noted - using Wifi for smart plugs has its rough edges.
- rkagerer 2y ago
  I second that. I replaced a bunch of wall switches with Leviton WiFi smart ones and would never go WiFi again. They're totally unreliable. My Meross smart plugs fair a little better but still lose connectivity now and then (got a bit better with updates).
  - ssl-3 2y ago
    I hear a lot about reliability issues with various wifi smart devices. I've had perfectly lovely reliability with wifi smart devices in my mixture of zigbee/wifi at home, such that I don't really have a preference. Except for one cheap ESP8266-based wifi relay module that had some liquid damage (not the module's fault), and the LED driver in my very first RGBW light bulb finding death after being used for a few years (a common-enough tale regardless of connectivity choice), they seem to Just Work. It's all semi-random brands of devices, bought over time. I'm not doing anything particularly fancy with the network itself: It's just a couple of hardwired dual-band Mikrotik access points, with one upstairs at the back of the house and one downstairs at the front of the house (perhaps non-obviously, on non-overlapping channels). A Pi 4 with OpenWRT quietly does the packet-slinging. Like in many other places, the 2.4GHz band is approximately ruined where I live these days. It's noisy and slow. But it all works well enough to reliably toggle a relay on or off, at least. Am I just lucky? Are others just unlucky? Or is there an actual pattern here?
    - nucleardog 2y ago
      I’ve tried a handful of WiFi light bulbs, smart plugs, and other things. They were connected to a dedicated 2.4GHz access point (MikroTik as well). WiFi signal was fine—had good coverage (access point was central to the small wood frame house, and checking signal strength at device locations showed a great signal) and I live in the middle of a forest and there’s nothing else within range to interfere. Basically best case scenario outside of a lab or something. I haven’t had a single device that worked reliably. Some worked fairly consistently, but only after a long (and variable) delay. Many others failed to work often enough that, combined with the delays, the workarounds became the normal way of using things. At this point I’m running basically everything over Z-Wave (via Home Assistant) and it’s been rock solid for me. Especially with the ability to set up direct associations, things like “this dimmer’s state should be synchronized to that dimmer” are very responsive and reliable, not involving my controller or Home Assistant at all. Hard to say whether you’re lucky or I’m unlucky—most people having a good experience aren’t going to take to the internet to start a crusade about it—but I do occasionally see someone recommending or saying that Z-Wave or Zigbee has been reliable for them… I think yours is the first I’ve read where someone’s been happy with WiFi.
recursinging 2y ago
One step further. I just installed the Emporia Vue 2 in my distribution box. 16 CTs plus the three mains phases. It's ESP32 based and there is a great ESPHome project that you can flash it with for local only reporting. Add some HA and VictoriaMetrics, and now I can see how the whole house behaves with Grafana. Next up, Zero-Export using this data to steer my little OpenDTU solar plant. We live in such cool times!
- septic-liqueur 2y ago
  Opendtu solar plant - care to elaborate?
  - recursinging 2y ago
    OpenDTU is an open source project using an ESP32+CMT2300A for talking to Hoymiles Micro-Inverters. Local Only.
    - jauntywundrkind 2y ago
      I did some scouting about for what microinverters would be usable without a full professional install, for a small under half kilowatt playing around. I was hoping I could snap up some used enphases & try stuff out with a 200w panel & my existing batteries. But I really didn't turn up much; most discussion online made it seem like you needed special installer access to get anywhere with Enphase. Exciting to hear maybe this microinverter idea might not be totally dead in the water.
- pzduniak 2y ago
  Is anyone aware of any other OSHW alternatives to this? Preferably with Ethernet. ESPHome would be preferable. The clones I can find are roughly the same price as the "original" hardware. ATM90E32AS seems to be ~$1 per channel on JLCPCB, so I'd imagine this could be pretty cheap with SMT assembly. My use case is like ~60 circuits.
  - anupcshan 2y ago
123pie123 2y ago
i've just bought a cheap esp32 with a light sensor connected to it. then connected light sensor (ie bluetac) to my electicity meter that pulses every 1/1000 1KW/Hr, it uploads the data to google sheets which graphs the output - works great I also have another esp32 at my elderly relatives house with a pir sensor connected to it, it's also sending the movement data to a different sheet on the google sheets site, so that i can monitor some sort of movement. i'm i expecting google to discontinue this service at anytime - yes, but its working for now. you can write and read data from the google sheets via json via the esp32, not very inutitive but doable (and free!!)
- gog 2y ago
  I have a Frient Zigbee device that does the same, but sends data to Home Assistant.
  - 123pie123 2y ago
    Home assist looks good, but my requirements was that my elderly relative was being monitored by all my family (ie they're not on my home wifi/lan) hence the data on the movement needed to be on the internet for all to see
- whitehexagon 2y ago
  nice, I like the simple approach. My old meter only had a rotating disc and it took all my effort to get a sensor connected to my arduino that could detect the black mark on the edge of the disc. There was just enough mem for a http service to allow me to pull that value into my iobridge for remote monitoring.
eMerzh 2y ago
Homeassistant + Power calc (https://github.com/bramstroker/homeassistant-powercalc https://github.com/bramstroker/homeassistant-powercalc) really does wonder here, you can "simulate" power of fairly stable appliances. Then you chart that in a nice Sankey chart or in standard charts and enjoy
- sifttio 2y ago
  This is a good solution, but I'm having trouble estimating the power draw from charging all my fleshlights daily.
sponaugle 2y ago
I used IoTaWatt devices, which can be installed in panels. It is a great solution for by circuit monitoring, and has direct influxdb integration so you can use Grafana. Per plug monitoring is cool however for getting specific devices on a circuit. (Short video about the setup: https://www.youtube.com/watch?v=-tcbJCvuJG8 https://www.youtube.com/watch?v=-tcbJCvuJG8)
- gibbonsrcool 2y ago
  Wow it’s a small world. I just discovered your channel recently and love the content. Probably watched the home lab video like 10 times. I have a historic building with 3 vacation rentals that I’m in the process of adding some intelligence and monitoring to, including a raspberry pi kiosk based on one of your videos. Thanks for making them!
  - sponaugle 2y ago
    That is cool! Indeed there is a good cross section of home automation , software development, power monitoring, and hacker news. I just did a new video on building a 68030 computer that I suspect will have a much smaller audience!
- RatchetWerks 2y ago
  I gotta chime in also. I just learned about your channel literally today when a buddy DM'd me a link on your YouTube homelab tour. We have pretty similar interests. I also have an IOTAWATT and I love it. My only grip is I wish they had the ability to log more channels with the same unit. ie Approximately 30 channels.
  - sponaugle 2y ago
    Sweet! Yea, I have several panels that have 2 of them for that reason. I have debated making a larger one since the code is open source. It would be great to have one that has more ports, and perhaps the ports combined into less cables.
aksss 2y ago
Surprised there’s not more mention of Shelly monitors in these comments. They’re great for whole house (service entrance) and circuit power monitoring. Pretty open integration, OOTB integration with HA. I think it makes more sense to use “dumb” OTS circuit breakers in your house and augment with add-on monitor than combining the capabilities into a tightly-coupled single device.
- AdrianB1 2y ago
  I use a dozen of their relays and I am happy with it; for power metering they don't have a great solution, the 3 phase clamp is very expensive and there is no option for more, like Emporia Vue with 8 or 16 sensors. So yes, Shelly is worth mentioning, but not for fine grained power metering.
  - aksss 2y ago
    The deployment model is very different, and within that model, the capabilities are far different. Comparing Shelly's system to Vue really starts with understanding your requirements and long-term automation plans. I think the key differences are in cloud requirements, control, and granularity. Price appears higher than Vue, though it's not when considering that Shelly is far more capable in all of the above categories. The Shelly 3EM, which I use for split phase monitoring (in the US) at the meter is $109. That's not bad for its capacity, capability, and build quality. Mine is outdoors though enclosed (necessarily) so endures some pretty extreme temp swings and humidity changes. There is no cloud requirement, and it offers contactor control. The Vue 2 from Emporia is built for a single indoor panel, provides no control over the circuits, and requires the cloud to operate unless you get out the soldering iron and reflash it with ESPHome, which isn't horrible, but the OOTB cloud requirement is your starting point. (I think that's still the case, but if I'm wrong, happy to be corrected) For Shelly, their additional clamp solutions are all high-amp (50-100+) so not really designed to be put on individual circuits, especially at $50 per. You might use these on whole panels, breaking your whole house calculation down into zones by panel or monitoring high-amp devices. If you tried to replicate the Vue with only these devices from Shelly, then yes, it would be an order of magnitude more expensive. But I don't think that's a good way to use them or the right way to replicate the Vue capabilities. For the high-grained monitoring, it's hand-in-hand with control. Shelly's deployment model for fine-grained resolution is to put power metering down closer to the consuming devices, and with it, a relay for integration into home automation. You can add power metering along with control to outlets, switches, or hard-wired devices in house with things like the 1PM Mini G3 for $13 per. In this sense, you get far greater granularity than something like the Emporia Vue which can get no higher resolution than a whole circuit. Here again it must be stressed that with Shelly, there's no cloud requirement for that price and you're also getting remote control of the power distribution. On the balance, you're getting far more for your money with this approach. For my needs, I combine Shelly's approach with some cheaper sonoff/tasmota plugs with power metering for things like the washing machine (alarming mostly). But for more critical devices or always on devices (like freezers), or switched devices like ventilation fans or lights, I think the build quality and deployment model of the Shelly devices is a better fit. But definitely different target audiences. If the Vue is the right fit, it's the right fit.
fidotron 2y ago
Facetious but half serious reaction: surely grafana usage nullifies any possible benefit from the monitoring? If there is one piece of software regularly using way more resources than seem reasonable it's that one.
- baq 2y ago
  You can always export into Splunk or Datadog and burn way more dollars than seems reasonable.
- sifttio 2y ago
  Simple fix is to run an extension cord from the neighbors house and use that to host your monitoring equipment.
kasey_junk 2y ago
I nerded out on this a few years ago and ended up buying a not well known device called a rainforest automation eagle (https://www.rainforestautomation.com/rfa-z114-eagle-200-2/ https://www.rainforestautomation.com/rfa-z114-eagle-200-2/). Its a pretty straight forward little linux device that reads your smart meter (after being enrolled via your utility). It exposes an xml api that I bridge to Prometheus (https://github.com/kklipsch/reagle https://github.com/kklipsch/reagle). I also bridge my utility (ComEd's) pricing feed to prometheus (https://github.com/kklipsch/comed_exporter https://github.com/kklipsch/comed_exporter). Between those 2 I get pretty good whole home utilization and pricing info graphed into Prometheus (and thus into Grafana).
- alexgorbatchev 2y ago
  Could you share how do you combine the hourly cost with wattage readings?
  - sifttio 2y ago
    kwh x $ Price is generally a dollar figure for 1000W per hour.
    - alexgorbatchev 2y ago
      Thanks! Unfortunately all my data readings are in kW. I've been trying to figure out how to get hourly roll-up working and then multiply each hourly reading by price in Prometheus.
33282334 2y ago
[flagged]
killme2008 2y ago
It's interesting and cool! Looks like the software stack is still running on the local server. Looking for Prometheus and Grafana on the cloud? Check out GreptimeCloud! https://greptime.com/ https://greptime.com/
Gravityloss 2y ago
Hmm, just as a note, at least over here normal electricity companies provide the total level on their web site... no need to install any hardware.
- NortySpock 2y ago
  In real time? Mine only gives me a month-to-date, end of month for billing, and month-over-month for the last two years (plus the outside temperature for comparison).
  - Gravityloss 2y ago
    I haven't bothered to check but I think on about hourly level is common.
  - Symbiote 2y ago
    My Danish electricity supplier gives hour-by-hour usage, with the most recent hour being 23-24 yesterday, so I assume it updates at midnight. I don't know if there's an API or something to get almost-current data.
VyseofArcadia 2y ago
I'm really impressed to see the home server power consumption in the article. I always sort of expect these things to end with, "it turns out I could save even more power by not having a home server".
lucascacho 2y ago
Just last week I was geeking out at Jeff Gluon's home lab implementation, and he released a nice video walking through his power monitor, which also uses Grafana. https://www.youtube.com/watch?v=-tcbJCvuJG8 https://www.youtube.com/watch?v=-tcbJCvuJG8
deleted 2y ago
[deleted]

354 points by hddherman 2y ago

dainiusse 2y ago

No need for that mate, just deploy home assistant or something similar and you will get this (and more) out of the box

mindslight 2y ago
Right up until the Home Assistant UI turns into a lagfest, the installation dies, and you can't debug why because Docker. At least that's what happened to me. And no, it wasn't RPi SD power issues. This happened on an otherwise-stable amd64 server. The Home Assistant authors' hostility towards simple native distributions is now a show stopper for me. Long term reliability is more important than quick initial setup.
- cyberax 2y ago
  HA is actually pretty debuggable. Just install the SSH plugin, then SSH into the HA box, and then simply "docker exec" into the target HA container.
  - mindslight 2y ago
    ... and then not have any of your usual development tools, environment, system layout, or repair techniques because you're inside someone else's "works on my system" that they threw over the wall. It's obviously possible to debug what goes on inside a Docker image. It's just not something I'm particularly interested in dealing with, especially under duress.
    - TeMPOraL 2y ago
      > because you're inside someone else's "works on my system" that they threw over the wall. FWIW, this can also be called stable state you can retreat to. And build upon, e.g. adding a layer of debugging tools. I don't really like to deal with Docker, but at least I have reasonable certainty it'll work. I prefer system package manager or MSI, but if not that, it beats having to build something when it's near-guaranteed that what I'll get is not the binary the authors had in mind, if it even runs at all. (Then again, I routinely rebuild Emacs to stay on the bleeding edge. But it took a while to work out all the usual dependency mess, and I even broke my system once doing it.)
      - mindslight 2y ago
        It's certainly within my gamut to jump into an embedded system to debug it, bringing/building tools as I go. I'm just not looking to opt into doing that on something that doesn't need to be that complex in the first place. Same reason I run one decently powerful amd64 server that does many things rather than a stack of Raspberry Pis, one per software package.
        cyberax 2y ago
        But how would you do it differently? You need to host a bunch of daemons (MQTT, ZWave and ZigBee bridges, and whatever else you might need). And a bunch of these daemons can have their own gnarly dependencies (e.g. they can be written in JS and built with NPM, ugh). So you kinda _need_ to use Docker to make it at least sane. And if you're using Docker for the plugins, then why not use it for the HA core itself? And once you do that, you don't really need much from the host system. So why not use a minimalistic OS instead of something like Debian?
        mindslight 2y ago
        At the time my setup didn't require other daemons like that. But if I had been in that position, I would have just set up the other daemon under Debian and pointed HA at it. These days I'd say that NixOS captures that requirement, allowing orchestration of many daemons and other system config to be abstracted into a packaged solution (eg NixOS Mailserver), that the user can override as much or as little as they'd like. I believe NixOS does package (or at least attempts to package) HA, but given my past experience and what I believe is still the throw-it-over-the-wall desire of the HA maintainers, I'm wary of adopting it as an overarching solution. I'm certainly not ruling it out for performing some functions, like UI. I just would rather set up my automation efforts as MQTT-first, keep logging and automation rules as their own separate things, and not be fully committed to HA.
        cyberax 2y ago
        > At the time my setup didn't require other daemons like that. But if I had been in that position, I would have just set up the other daemon under Debian and pointed HA at it. You can do that just fine even now. I'm doing experiments with voice control, and I run the complete AI stack locally on my computer. So I just set up everything as regular background processes. You just can't expect HA to be able to do autoupdates for these daemons. The other problem is that most of required dependencies are not packaged in Debian. So you'll have to install multiple NodeJS servers and tons of NPMs somewhere on your system. > These days I'd say that NixOS captures that requirement, allowing orchestration of many daemons and other system config to be abstracted into a packaged solution (eg NixOS Mailserver), that the user can override as much or as little as they'd like. You can do that with HA as well. Just push in a new image, and tag it appropriately. The last time I played with Nix, it needed to download tens of gigs of data for a few programs. I don't think this is acceptable for HA. You can definitely do HA in a piecemeal fashio, but there's just no way it can be done as a reproducible system that you can give to your grandmother. Given these constraints, HAOS is actually pretty remarkable. > I just would rather set up my automation efforts as MQTT-first, keep logging and automation rules as their own separate things, and not be fully committed to HA. Raw MQTT still needs a UI that is user-friendly. And even with MQTT you'll need to run ZWave and ZigBee bridges.
    - cyberax 2y ago
      > ... and then not have any of your usual development tools, environment, system layout, or repair techniques because you're inside someone else's "works on my system" that they threw over the wall. The thing is, the "it works" is reproducible because of containers. Which is a step above just hoping that it works. HA is also easy to "patch". You can just install your custom components in `config/custom_components`, it can also be used to "override" core HA files. Finally, if you are doing intrusive development, you can easily launch HA locally. macOS, Linux, and WSL are supported. You will lose the ability to install add-ons via the addon manager, but that's about it. FWIW, I had the same aversion to their custom OS and their crazy container-based setup initially. For a couple of years, I used to run HA as a Python app and managed the dependencies manually. Then I tried the HAOS and it... kinda just worked.
- baq 2y ago
  It's a Python app, of course being distributed as a docker image is the sanest way of doing it. I don't see why you couldn't just pip install it if you really wanted, but having been a Python developer for close to two decades, I wouldn't want to.
  - mindslight 2y ago
    I'm talking about distribution package managers, not pip.
    - darkwater 2y ago
      I happily ran a dockerized HA on a Debian for years now, no need to do any complicated debugging (and even if I did, it would not be difficult to inspect it properly)
      - mindslight 2y ago
        Dockerized HA on Debian is exactly what died on me. About 5-6 years ago. I'm sure it works just fine for most people. Just once bitten, twice shy.
  - Izkata 2y ago
    That was the standard way a long time ago, and the first startup would take a really long time because it would install even more stuff. And sometimes fail. It wasn't very reliable if you used any addons, and some required a ton of extra steps that it couldn't automate like the modern deployments do now.
- tw04 2y ago
  Nobody is preventing you from running Home Assistant core and deploying everything else yourself manually. Demanding the authors who gave you the software for free also provide support for an installation method they've offered up with no support is a bit ridiculous, don't you think? That attitude is what causes open source projects to die though...
  - mindslight 2y ago
    What do you mean "demanding support" ? I remember Home Assistant authors being actively hostile to people packaging their software outside of the official Docker or RPi images. Which is why it wasn't in the Debian repository, pushing me down that Docker path in the first place. Here's the same dynamic on an associated project in 2021: https://github.com/NixOS/nixpkgs/pull/126326 https://github.com/NixOS/nixpkgs/pull/126326 If anyone chimes in and says they've been running Home Assistant from nixpkgs (where I am now) for several years with no hiccups, then I will certainly reconsider my opinion. But based on my experience and what I've continued to read since, it feels like trying to do that is an uphill battle. One I'm not looking to take on, especially for automation I'm relying on.
    - tw04 2y ago
      So... your example is a developer of HA stating that he sees major flaws with how they're distributing his package, and that he has absolutely no interest in supporting users that pull his code in a way that is unmaintainable by him. YOU believe he should support this anyway, because of various "we promise end-users won't reach out to you" which is comically incorrect because history has shown repeatedly that a user's first step when something is broken is to google package_name broken - which will absolutely turn up the author's name. BECAUSE he doesn't want to support his software being repackaged in a way he believes isn't supportable, you're upset. You want him to support your unicorn config because that's what you want to do, and his refusal to comply makes him a bad person. Thank you for reinforcing EXACTLY why open source devs burn out. He has a workflow that he is willing and able to support and doesn't want to support anything outside of that. Your response is: but you need to do it for me because it's what I want.
Havoc 2y ago
Grafana is a hell of a lot nicer & controllable than HA HA is great, but it's not the answer to everything
- madaxe_again 2y ago
  Agreed. I live off grid, so energy monitoring is a big deal for me. HA is fine for “at a glance”, but if I want any kind of detail, I use grafana. I actually have my old openhab instance still running purely as I can’t be faffed setting up all the piping from MQTT into influx again. It’s also possible to integrate the usage over time using a dynamic time window to get Wh figures from wattage, which is enormously useful for me, and is more accurate than the figures HA gives in their power system. HA is dead useful for getting alerts when the laundry finishes, though - dumb machine, smart plug, look for a sudden drop in power. Also does all our climate control. So different tools for different jobs.
- zimpenfish 2y ago
  Seconded - HA's graphs are great for a simple "is this going up or down" glance but when you want to put a whole bunch of things together for comparison or perform aggregations or calculations, that's when you want Grafana et al.
- Cyph0n 2y ago
  Why not both? You’ll need to run a server either way. HA can export data to Prometheus. Setting up and running HA is much easier than figuring out how to get a set of different smart devices to export metrics to Prometheus/Influx. Let HA deal with that.
- icehawk 2y ago
  It might be, but for all of the examples in the blog post, HA does this out of the box.
Banditoz 2y ago
Why not both?
whitehexagon 2y ago
I'm also looking at a custom solution for my current migration from WiFi sockets to Zigbee. It seemed impossible to do an offline installation of home assistant, and discouraging signs for running it without an internet connection. There seems to be a sonoff usb stick that might act as a hub and allow command-line monitoring of all devices, should be perfect for feeding into grafana/prometheus.
- baq 2y ago
  HA will happily run offline; if you mean HAOS then I don't know what it does but it's an unorthodox Linux distro, but once it installs it should also run offline without issues. I'm also using their skyconnect zigbee coordinator and it works very well.
  - whitehexagon 2y ago
    Yeah one of the tests was a RPi image and it wouldnt complete without a LAN internet connection (only got 4G). And it seemed far too weighty for a bit of home automation. I recall the online requirement was for some ntp server requests that cant be disabled.
    - baq 2y ago
      Yeah that's more of a rpi hardware requirement as it doesn't have a battery and you realistically want to have accurate time on your smart home controller, even - especially - after it cold boots after power loss.

Xerox9213 2y ago

Yeah, not to mention the ability to automate things. My latest automation: when the white noise machine is on for the baby, the doorbell volume is turned down.

alchemist1e9 2y ago
Could you expand on how prometheus or grafana helps you automate? I didn’t know either enabled that.
- Jleagle 2y ago
  I think this was supposed to be a reply to one of the Home Assistant threads
  - Xerox9213 2y ago
    Correct. My mistake.

john61 2y ago

Missing from the blog. The power reading of the Tosmota device should be calibrated: https://tasmota.github.io/docs/Power-Monitoring-Calibration/ https://tasmota.github.io/docs/Power-Monitoring-Calibration/

jhenkens 2y ago
I would love a semi-automated way to generate a power-profile for ESP-Home. Find a smart room heater with 3 levels perhaps, and use home assistant to gather values at "Off", "1/3", "2/3", "3/3", with a downstream power plug as reference (and a known consumption of the downstream plug as well). So I can just take my EspHome plug and very quickly generate a standard set of mapping values for voltage and wattage.
- ssl-3 2y ago
  The easy way is with a resistive space heater and a multimeter. I keep a big, dumb, thrifted "oil-filled radiator" space heater around just to use as a big, safe 3-speed dummy load with reasonably OK repeatability (nichrome heaters do not have perfect temperature coefficients, but they're stable-enough that using them to measure temperature quickly begins to be a non-starter). The level of integration you choose is entirely up to you. I don't do this kind of thing much, so I'm OK with kludging together a test rig as-needed with a handheld meter and tearing it apart when I'm done. This makes good use of my own time and tools, according to my personal proclivities. But if I were doing it often, then I might buy the equivalent of the HOPI meter that Big Clive uses in many of his videos. It displays current and voltage, multiplies them to get power, and also displays power factor -- concurrently, on separate digital displays, in real time. Or I might build something: A box with a current shunt with some panel-mount meters and appropriate connectors would not be too challenging to put together in an afternoon with parts from Amazon and Lowes, depending on one's ability and desire to deal with sheet metal at home. (I use galvanized steel handy boxes and cover plates from Lowes for all kinds of small-ish stuff. They're cheap, common, and durable-enough.) Whatever the approach, a simple space heater with multiple literal-speeds seems like a cheap and useful way to make it happen unless you're trying to automate every part of it. (But by then, making a dumb multi-speed space heater into a "smart" multi-speed space heater that can be activated programmatically with software like ESPHome and some relays is probably pretty much a no-brainer, isn't it?)

ars 2y ago

If someone else wants to try this, I strongly recommend against using WiFi for the plugs, instead use Z-Wave or Zigbee. Wifi is just not meant for this use case, it will be unhappy if you start adding a lot of devices, and it will slow down your main use of WiFi for your phone.

seszett 2y ago
Not the first time I hear this but I have about 20 ESP devices on my WiFi, almost all of them pushing data regularly and polling for instructions (I don't use HA, but a simpler home made solution) and I have no problem at all.
- meatmanek 2y ago
  It helps that the IoT things almost invariably use 2.4GHz while your data-hungry computers and phones usually use 5GHz.

dsab 2y ago

Guys what are your favourite smart plug? I need one with easy integration with grafana, not sucking, and shipment to EU country?

tetris11 2y ago
ZigBee list of good devices: https://www.zigbee2mqtt.io/supported-devices/ https://www.zigbee2mqtt.io/supported-devices/
ckolkey 2y ago
I'm a huge fan of https://www.shelly.com/ https://www.shelly.com/ - they have a built in webserver and can be controlled via POST requests. No cloud needed.
- septic-liqueur 2y ago
  I also like their products very much. I installed a few of them in my sockets. Some people argue that it's not safe to put in your sockets walls and the 16A limit is realistically lower before they can overheat and cause fire. That scared me a little bit but I think most of the reports are from bad wiring like using thin wires or not tightening the clamps enough
  - spockz 2y ago
    I have a Shelly pro 1PM in my Breaker box for the car charger. It gets to 80 degrees Celsius easily at 16A. I have two other gripes with it: 1: the slots/clamps are small which makes fitting 4mm2 wires a chore already. 6mm3 is impossible. 2: the overcurrent protection is very trigger happy. Due to solar panels in the street that voltage can vary significantly. Apparently the charger doesn’t always keep up exactly resulting in a current of 16.0001A which is more than the limit of 16 and poof, off it goes. Not sure whether this is an actual fault in the charger or some rounding error.
- theshrike79 2y ago
  Big +1 for Shelly stuff. IIRC it was started by a hobbyist who got annoyed at the crap from Aliexpress and decided to do it properly. And that they have done.

leeoniya 2y ago

i'm trying to nudge Grafana into the direction of IoT/SCADA control, so we can be both, a great way to viz (data sources) and to control (data sinks). not personally a huge fan of having to recommend Home Assistant for that use case :) (i work at Grafana Labs)

recursinging 2y ago
This would be great. Were using Grafana dashboards for thermal vacuum testing. Everyone is always asking for simple SCADA functionality.
hackernewds 2y ago
could you expand on your comment? what is data sinks as applied to your idea?
- leeoniya 2y ago
  similar to how you can make plugins for data sources, you can make plugins for data sinks, like write over modbus, or POST to http API or, publish to some kind of MQTT broker, etc. since IoT devices have limited storage, usually the metrics are dumped into another system like Prometheus. but that Prometheus data source used plotting cannot be used to control the device, which will have another endpoint and another API, so we need some kind of concept of data sinks. at least that's my idea right now, allow data links configured in the panels to poke some "data sink" with values that are available in the DataFrame or custom-entered into the UI, like we do with traceID for Exemplars, etc.
  - nagisa 2y ago
    Have you seen/know about NodeRED? I myself have moved away from HA/OpenHAB-likes to NodeRED as that allows me control that's more... restricting. At its heart NodeRED is a visual programming tool (and I think it wouldn't be a terrible fit for Grafana), but what NodeRED has and Grafana doesn't is a community which built a very significant number of packages that integrate with various protocols and devices used in home automation. I see a potential for some integration and/or idea sharing here. But it is also important to understand that the set of people who want to fiddle with their homes is much, much larger than the set of people who (are able to) program. Which is why HA and the likes are so popular.

stavros 2y ago

I got a Zigbee power breaker and hooked it up so all my flat's power goes through it, and then made an e-ink display to show my power consumption: https://www.stavros.io/posts/making-the-timeframe/ https://www.stavros.io/posts/making-the-timeframe/

shrx 2y ago
I'm curious, which power breaker do you have?
- stavros 2y ago
  This one: https://www.aliexpress.com/item/1005005657383634.html?spm=a2g0n.order_detail.order_detail_item.3.29d9f19cVGy4lV https://www.aliexpress.com/item/1005005657383634.html?spm=a2...
  - wannacboatmovie 2y ago
    You're running all your flat's power through that $14 Chinese rubbish and never assumed that shortcuts were taken or quality would be an issue? How do you know it will continue to function as a circuit breaker and isn't just a piece of wire inside? For the uninitiated, CE marking is meaningless (it allows for self-certification). I'd like to see Big Clive do a teardown of one of those.
    - theshrike79 2y ago
      You can get less shit ones that aren't made of Chinesium from Shelly
      - wannacboatmovie 2y ago
        Great suggestion. Some of their modules appear to be UL certified.
        2y ago

ryall 2y ago

It's annoying that smart plugs/bulbs etc use wifi when Powerline exists

whitehexagon 2y ago
I have a box of old X10 devices here, one of the most reliable home automation systems I ever set up. I only switched to WiFi when my iobridge X10 controller failed and I couldnt work out the RS232 protocol for a different controller.
Dalewyn 2y ago
Powerline ethernet dumps an utterly horrifying amount of electrical noise into both the wiring and the surrounding area. Please don't use powerline unless you have no other solution. Note: Powerline ethernet should not be confused with Power Over Ethernet which is perfectly fine.
- baq 2y ago
  I'm using it on two outlets after not having enough prescience to install ethernet in my wardrobe-turned-office - got any reading materials?
  - Dalewyn 2y ago
    Don't have any to link off hand, but the basic gist is this: Electrical wiring of any sort are all antennas to varying degrees, transmitting and receiving electromagnetic signals. Most mains electrical wiring is also unshielded, meaning they readily transmit and receive electromagnetic signals. Powerline ethernet basically puts ethernet data on mains electrical wiring by utilizing the bands that aren't used for carrying power. This data is very, very noisy in electrical noise terms, and because most wiring also acts as an antenna that noise also gets broadcast everywhere. Simple electronics like your coffee machine or microwave oven won't care, but more sensitive electronics like radios can in turn receive interference from both the power line and the noise broadcast into the air.
    - 2y ago

WesolyKubeczek 2y ago

You need to be careful with what plugs you choose, though, because they each have, let's say, their own peculiarities. For instance, their overvoltage protection might not align well with what the local regulations say. For example, in my region of EU, the upper voltage tolerances are such that 264V must trigger an instant poweroff, and also anything producing power must shut off if the average voltage over the last 10 minutes was 253V or more. However, TuYa sockets which pretty much are the only in-wall variety I was able to find on the local market, shut off at 260V. This tends to be somewhat problematic in an area saturated with PV installations, like the one I'm living in. This problem is compounded by the fact that the reported measurements of sockets sitting on the same phase tend to differ quite a bit. Some sockets tend to overstate the voltage compared to neighboring sockets sitting on the same wires. Thus, they shut off when they think it's 260V, while it might just as well be 255V. Just saying that if you put lots of those in your walls, you might suddenly find yourself in a need to prepare some automations to try and bring the sockets on once the voltage is back to normal. This particular variety of sockets won't come back on after the voltage drops.

baq 2y ago
Shelly relays can be configured to do all these things and the voltage safety threshold itself is also configurable (at least in the Plus devices) but they aren't zigbee. Otherwise great little devices.
- darkwater 2y ago
  I cannot really understand why Shelly doesn't offer ZigBee (and now Matter) options, only Wifi. This has always been the biggest blocker for me.
  - nagisa 2y ago
    They have z-wave options, though.
- WesolyKubeczek 2y ago
  Cannot find in-wall options much.

richardjennings 2y ago

I scrape power usage metrics from Tapo P110s and push them to Grafana Cloud using https://github.com/richardjennings/tapmon https://github.com/richardjennings/tapmon - although as other commenters have noted - using Wifi for smart plugs has its rough edges.

rkagerer 2y ago
I second that. I replaced a bunch of wall switches with Leviton WiFi smart ones and would never go WiFi again. They're totally unreliable. My Meross smart plugs fair a little better but still lose connectivity now and then (got a bit better with updates).
- ssl-3 2y ago
  I hear a lot about reliability issues with various wifi smart devices. I've had perfectly lovely reliability with wifi smart devices in my mixture of zigbee/wifi at home, such that I don't really have a preference. Except for one cheap ESP8266-based wifi relay module that had some liquid damage (not the module's fault), and the LED driver in my very first RGBW light bulb finding death after being used for a few years (a common-enough tale regardless of connectivity choice), they seem to Just Work. It's all semi-random brands of devices, bought over time. I'm not doing anything particularly fancy with the network itself: It's just a couple of hardwired dual-band Mikrotik access points, with one upstairs at the back of the house and one downstairs at the front of the house (perhaps non-obviously, on non-overlapping channels). A Pi 4 with OpenWRT quietly does the packet-slinging. Like in many other places, the 2.4GHz band is approximately ruined where I live these days. It's noisy and slow. But it all works well enough to reliably toggle a relay on or off, at least. Am I just lucky? Are others just unlucky? Or is there an actual pattern here?
  - nucleardog 2y ago
    I’ve tried a handful of WiFi light bulbs, smart plugs, and other things. They were connected to a dedicated 2.4GHz access point (MikroTik as well). WiFi signal was fine—had good coverage (access point was central to the small wood frame house, and checking signal strength at device locations showed a great signal) and I live in the middle of a forest and there’s nothing else within range to interfere. Basically best case scenario outside of a lab or something. I haven’t had a single device that worked reliably. Some worked fairly consistently, but only after a long (and variable) delay. Many others failed to work often enough that, combined with the delays, the workarounds became the normal way of using things. At this point I’m running basically everything over Z-Wave (via Home Assistant) and it’s been rock solid for me. Especially with the ability to set up direct associations, things like “this dimmer’s state should be synchronized to that dimmer” are very responsive and reliable, not involving my controller or Home Assistant at all. Hard to say whether you’re lucky or I’m unlucky—most people having a good experience aren’t going to take to the internet to start a crusade about it—but I do occasionally see someone recommending or saying that Z-Wave or Zigbee has been reliable for them… I think yours is the first I’ve read where someone’s been happy with WiFi.

recursinging 2y ago

One step further. I just installed the Emporia Vue 2 in my distribution box. 16 CTs plus the three mains phases. It's ESP32 based and there is a great ESPHome project that you can flash it with for local only reporting. Add some HA and VictoriaMetrics, and now I can see how the whole house behaves with Grafana. Next up, Zero-Export using this data to steer my little OpenDTU solar plant. We live in such cool times!

septic-liqueur 2y ago
Opendtu solar plant - care to elaborate?
- recursinging 2y ago
  OpenDTU is an open source project using an ESP32+CMT2300A for talking to Hoymiles Micro-Inverters. Local Only.
  - jauntywundrkind 2y ago
    I did some scouting about for what microinverters would be usable without a full professional install, for a small under half kilowatt playing around. I was hoping I could snap up some used enphases & try stuff out with a 200w panel & my existing batteries. But I really didn't turn up much; most discussion online made it seem like you needed special installer access to get anywhere with Enphase. Exciting to hear maybe this microinverter idea might not be totally dead in the water.
pzduniak 2y ago
Is anyone aware of any other OSHW alternatives to this? Preferably with Ethernet. ESPHome would be preferable. The clones I can find are roughly the same price as the "original" hardware. ATM90E32AS seems to be ~$1 per channel on JLCPCB, so I'd imagine this could be pretty cheap with SMT assembly. My use case is like ~60 circuits.
- anupcshan 2y ago

123pie123 2y ago

i've just bought a cheap esp32 with a light sensor connected to it. then connected light sensor (ie bluetac) to my electicity meter that pulses every 1/1000 1KW/Hr, it uploads the data to google sheets which graphs the output - works great I also have another esp32 at my elderly relatives house with a pir sensor connected to it, it's also sending the movement data to a different sheet on the google sheets site, so that i can monitor some sort of movement. i'm i expecting google to discontinue this service at anytime - yes, but its working for now. you can write and read data from the google sheets via json via the esp32, not very inutitive but doable (and free!!)

gog 2y ago
I have a Frient Zigbee device that does the same, but sends data to Home Assistant.
- 123pie123 2y ago
  Home assist looks good, but my requirements was that my elderly relative was being monitored by all my family (ie they're not on my home wifi/lan) hence the data on the movement needed to be on the internet for all to see
whitehexagon 2y ago
nice, I like the simple approach. My old meter only had a rotating disc and it took all my effort to get a sensor connected to my arduino that could detect the black mark on the edge of the disc. There was just enough mem for a http service to allow me to pull that value into my iobridge for remote monitoring.

eMerzh 2y ago

Homeassistant + Power calc (https://github.com/bramstroker/homeassistant-powercalc https://github.com/bramstroker/homeassistant-powercalc) really does wonder here, you can "simulate" power of fairly stable appliances. Then you chart that in a nice Sankey chart or in standard charts and enjoy

sifttio 2y ago
This is a good solution, but I'm having trouble estimating the power draw from charging all my fleshlights daily.

sponaugle 2y ago

I used IoTaWatt devices, which can be installed in panels. It is a great solution for by circuit monitoring, and has direct influxdb integration so you can use Grafana. Per plug monitoring is cool however for getting specific devices on a circuit. (Short video about the setup: https://www.youtube.com/watch?v=-tcbJCvuJG8 https://www.youtube.com/watch?v=-tcbJCvuJG8)

gibbonsrcool 2y ago
Wow it’s a small world. I just discovered your channel recently and love the content. Probably watched the home lab video like 10 times. I have a historic building with 3 vacation rentals that I’m in the process of adding some intelligence and monitoring to, including a raspberry pi kiosk based on one of your videos. Thanks for making them!
- sponaugle 2y ago
  That is cool! Indeed there is a good cross section of home automation , software development, power monitoring, and hacker news. I just did a new video on building a 68030 computer that I suspect will have a much smaller audience!
RatchetWerks 2y ago
I gotta chime in also. I just learned about your channel literally today when a buddy DM'd me a link on your YouTube homelab tour. We have pretty similar interests. I also have an IOTAWATT and I love it. My only grip is I wish they had the ability to log more channels with the same unit. ie Approximately 30 channels.
- sponaugle 2y ago
  Sweet! Yea, I have several panels that have 2 of them for that reason. I have debated making a larger one since the code is open source. It would be great to have one that has more ports, and perhaps the ports combined into less cables.

aksss 2y ago

Surprised there’s not more mention of Shelly monitors in these comments. They’re great for whole house (service entrance) and circuit power monitoring. Pretty open integration, OOTB integration with HA. I think it makes more sense to use “dumb” OTS circuit breakers in your house and augment with add-on monitor than combining the capabilities into a tightly-coupled single device.

AdrianB1 2y ago
I use a dozen of their relays and I am happy with it; for power metering they don't have a great solution, the 3 phase clamp is very expensive and there is no option for more, like Emporia Vue with 8 or 16 sensors. So yes, Shelly is worth mentioning, but not for fine grained power metering.
- aksss 2y ago
  The deployment model is very different, and within that model, the capabilities are far different. Comparing Shelly's system to Vue really starts with understanding your requirements and long-term automation plans. I think the key differences are in cloud requirements, control, and granularity. Price appears higher than Vue, though it's not when considering that Shelly is far more capable in all of the above categories. The Shelly 3EM, which I use for split phase monitoring (in the US) at the meter is $109. That's not bad for its capacity, capability, and build quality. Mine is outdoors though enclosed (necessarily) so endures some pretty extreme temp swings and humidity changes. There is no cloud requirement, and it offers contactor control. The Vue 2 from Emporia is built for a single indoor panel, provides no control over the circuits, and requires the cloud to operate unless you get out the soldering iron and reflash it with ESPHome, which isn't horrible, but the OOTB cloud requirement is your starting point. (I think that's still the case, but if I'm wrong, happy to be corrected) For Shelly, their additional clamp solutions are all high-amp (50-100+) so not really designed to be put on individual circuits, especially at $50 per. You might use these on whole panels, breaking your whole house calculation down into zones by panel or monitoring high-amp devices. If you tried to replicate the Vue with only these devices from Shelly, then yes, it would be an order of magnitude more expensive. But I don't think that's a good way to use them or the right way to replicate the Vue capabilities. For the high-grained monitoring, it's hand-in-hand with control. Shelly's deployment model for fine-grained resolution is to put power metering down closer to the consuming devices, and with it, a relay for integration into home automation. You can add power metering along with control to outlets, switches, or hard-wired devices in house with things like the 1PM Mini G3 for $13 per. In this sense, you get far greater granularity than something like the Emporia Vue which can get no higher resolution than a whole circuit. Here again it must be stressed that with Shelly, there's no cloud requirement for that price and you're also getting remote control of the power distribution. On the balance, you're getting far more for your money with this approach. For my needs, I combine Shelly's approach with some cheaper sonoff/tasmota plugs with power metering for things like the washing machine (alarming mostly). But for more critical devices or always on devices (like freezers), or switched devices like ventilation fans or lights, I think the build quality and deployment model of the Shelly devices is a better fit. But definitely different target audiences. If the Vue is the right fit, it's the right fit.

fidotron 2y ago

Facetious but half serious reaction: surely grafana usage nullifies any possible benefit from the monitoring? If there is one piece of software regularly using way more resources than seem reasonable it's that one.

baq 2y ago
You can always export into Splunk or Datadog and burn way more dollars than seems reasonable.
sifttio 2y ago
Simple fix is to run an extension cord from the neighbors house and use that to host your monitoring equipment.

kasey_junk 2y ago

I nerded out on this a few years ago and ended up buying a not well known device called a rainforest automation eagle (https://www.rainforestautomation.com/rfa-z114-eagle-200-2/ https://www.rainforestautomation.com/rfa-z114-eagle-200-2/). Its a pretty straight forward little linux device that reads your smart meter (after being enrolled via your utility). It exposes an xml api that I bridge to Prometheus (https://github.com/kklipsch/reagle https://github.com/kklipsch/reagle). I also bridge my utility (ComEd's) pricing feed to prometheus (https://github.com/kklipsch/comed_exporter https://github.com/kklipsch/comed_exporter). Between those 2 I get pretty good whole home utilization and pricing info graphed into Prometheus (and thus into Grafana).

alexgorbatchev 2y ago
Could you share how do you combine the hourly cost with wattage readings?
- sifttio 2y ago
  kwh x $ Price is generally a dollar figure for 1000W per hour.
  - alexgorbatchev 2y ago
    Thanks! Unfortunately all my data readings are in kW. I've been trying to figure out how to get hourly roll-up working and then multiply each hourly reading by price in Prometheus.

33282334 2y ago

[flagged]

killme2008 2y ago

It's interesting and cool! Looks like the software stack is still running on the local server. Looking for Prometheus and Grafana on the cloud? Check out GreptimeCloud! https://greptime.com/ https://greptime.com/

Gravityloss 2y ago

Hmm, just as a note, at least over here normal electricity companies provide the total level on their web site... no need to install any hardware.

NortySpock 2y ago
In real time? Mine only gives me a month-to-date, end of month for billing, and month-over-month for the last two years (plus the outside temperature for comparison).
- Gravityloss 2y ago
  I haven't bothered to check but I think on about hourly level is common.
- Symbiote 2y ago
  My Danish electricity supplier gives hour-by-hour usage, with the most recent hour being 23-24 yesterday, so I assume it updates at midnight. I don't know if there's an API or something to get almost-current data.

VyseofArcadia 2y ago

I'm really impressed to see the home server power consumption in the article. I always sort of expect these things to end with, "it turns out I could save even more power by not having a home server".

lucascacho 2y ago

Just last week I was geeking out at Jeff Gluon's home lab implementation, and he released a nice video walking through his power monitor, which also uses Grafana. https://www.youtube.com/watch?v=-tcbJCvuJG8 https://www.youtube.com/watch?v=-tcbJCvuJG8

deleted 2y ago

[deleted]