7 ms·

>The owner of the panels and inverters can meanwhile establish a connection with that manufacturer using an app or website, and via the manufacturer see how the

>The owner of the panels and inverters can meanwhile establish a connection with that manufacturer using an app or website, and via the manufacturer see how their own panels are doing > It wasn’t necessary from a technical standpoint to let everything run through the manufacturer’s servers, but it was chosen to do it this way. (emphasis from article) I'm working on IoT cloud system. It was chosen to be done this way because netither consumers nor installers have any expertise whatsoever to setup their own network or any devices to be acessible from outside (and they want their panels to be accessible when they are outside their home). I can do it, most readers of HN could do it, but typical consumer or installer can't. Sad but true.

Nextgrid 2y ago
The cloud can operate as a dumb TURN relay relaying E2E-encrypted traffic. Then the worst the cloud can do is deny service to remote management (and even then, local management would still work), but it wouldn't be able to send direct control commands to the equipment since they don't have the authentication nor encryption keys. This also makes it simpler from a programming point of view - instead of having separate cloud sync & local control protocols, you just have one local protocol and you merely tunnel it through the (dumb) cloud if you can't connect directly.
- yetihehe 2y ago
  It could, but this requires to store historical data about usage on devices. If you store that encrypted data in cloud, then getting it to your mobile phone is super slow. If you store it in cloud, you can get historical data even if your device is dead or has 256 BYTES of memory and 1 megabit of flash storage. We have such devices, very effective at managing local municipal heating network and controlling several thermal controllers each via rs232 or rs485. Fortunately we preemptively moved everything into VPN'ed mobile network, we need special approval to touch anything on that network and can't connect without them granting access, so after EU started moving with cybersecurity this year, we are covered. > This also makes it simpler from a programming point of view - instead of having separate cloud sync & local control protocols, you just have one local protocol and you merely tunnel it through the (dumb) cloud if you can't connect directly. Having only cloud protocol is even simpler, I've done all of the above (I do backend and our firmwares).
  - Nextgrid 2y ago
    > we preemptively moved everything into VPN'ed mobile network Unless your device itself is handling the VPN, I have bad news for you if you trust the mobile network to not open your devices up to malicious attackers: https://berthub.eu/articles/posts/5g-elephant-in-the-room/ https://berthub.eu/articles/posts/5g-elephant-in-the-room/
    - yetihehe 2y ago
      We consider "they hacked the mobile network VPN's AND had time to reverse our protocol before being booted out of network" as too high a level to be resolved by us. If someone has enough resources to do this, he will also just hack into standard-level secured server at municipal office and there will probably be no one there to stop him or discover what went wrong.
      - adrianN 2y ago
        Do you at least fuzz your software?
      - DoctorOetker 2y ago
        reversing the protocol can be done in advance, if they order your product
- wmf 2y ago
  I don't think E2E is simpler to program if you want to get it right. There are entire companies whose raison d'être is actually managing keys properly (e.g. Signal, Tailscale).
- akira2501 2y ago
  This should be the basic model. A fully third party TURN service. You pay $20/mo to keep your home connected, and all devices and providers can use a standard protocol, and users remain fully in control of their data.
bdamm 2y ago
For IoT stuff in general; I can do it, and I don't want to because I'd rather spend my time doing other things (although yeah, I totally did learn everything I could about my solar array, because it is a source of power, after all. But for the other stuff...)
danielovichdk 2y ago
These plants or farms are usually built around and on top of industrial IEC protocols and SCADA controllers which is a lot more low level than what any cloud IoT privider offers. I have done a controller for a 40 foot container battery and it wasn't like we received any API from Hitachi (battery manufactor). We had to write everything ourselves.
pheatherlite 2y ago
If we've learned anything from the security cam and baby cam scandals, then it's that convenience is king and we as a society would rather risk everything than be arsed to take few additional steps to setup/learn something to prevent such basic breaches. We (the society) don't even want to change the default password on most things.
- yetihehe 2y ago
  > We (the society) don't even want to change the default password on most things. Like you wouldn't believe. My most memorable case of insecure IoT devices - wifi socket was sending wifi ssid and password of the network in cleartext in every ping packet to chinese servers.
- titzer 2y ago
  People gonna be people. It's up to engineers and product designers to make things user friendly but also safe-by-default. If something needs to be configured, then provide instructions on how to configure it. Instead of pretending that it's society's fault (can't be arsed), maybe ask why the IT industry can't make instructions that are written out--explicit, fairly standard, and easy to follow--like the manual for putting together a piece of furniture. Or why the stupid device doesn't come with a randomly-generated strong password taped to it.
  - yetihehe 2y ago
    > fairly standard, and easy to follow--like the manual for putting together a piece of furniture. And people still have problems following that instructions. We gave a lot of instructions, but some people just don't read them. Or can't understand and follow them. Example from this week: Yeah, main support guy for our client is on holidays this week and only a girl from sales is available and she doesn't know why our device doesn't work, she tried resetting her phone wifi but still can't pair our zigbee smart hub connected with usb modem stick (problem: no one told her she needs to message us to actually activate sim card before installing device for end client). Yes, you can solve this one problem, but there are many more we didn't see yet. Consumer support does not scale and you can't write tests for something you don't know will be a problem.
lucianbr 2y ago
What's the reasoning for not allowing both control paths, via cloud but also locally? So that people who can and want to, will use the local control.
- yetihehe 2y ago
  Cheapness. It would require to be at least semi secure, application on phone would need to find those devices locally and it should be synchronized with cloud anyway, synchronization is error prone and we had problems with devices sometimes responding twice or very slowly through local interface (through cloud was much faster, no idea why, not our firmware). Also not enough people requesting that feature, most don't care and think that losing internet is not often enough to warrant worrying about this.
  - MostlyStable 2y ago
    Why not offer an either/or rather than both? Some people (I am one of them) actively do not want these kinds of things to be managed through the cloud servers. I don't want it to sync, I want to fully turn that off. I want to locally host, and I'm willing to take responsibility for that feature, including when it breaks. All I want is access to whatever the data reporting and control APIs are. I get that I'm a tiny minority, and that very few customers want what I want. But A) it seems like giving me what I want should be very cheap (i.e should not entail ongoing customer support costs beyond normal, and in fact would get rid of the small cloud hosting cost) and B) I'd be willing to pay a premium to get it.
    - wmf 2y ago
      In some areas like cameras there are a decent number of cloud-free alternatives. Hopefully as the IOT market grows we'll get cloud-free versions of everything. I think you're too optimistic about costs though. Providing any support at all, even one-time during the install, is expensive and cloud-free IOT is going to require support due to home networks being broken.
      - 2y ago
bee_rider 2y ago
Also administering a bunch of IOT systems is a pain. If something is an open source community project, ok, I’ll play. If somebody is selling a product they are responsible for making sure it works.
- Gibbon1 2y ago
  You could put an sql database on a local device and just access it remotely like anything else. But you are correct you're stuck with administering each and everyone one of them. The standard go to a raz pi solution will up and die every few months. And half the time you'll need physical access to get it back. It takes a lot of work to develop an embedded system that has enough reliability.
MathMonkeyMan 2y ago
To be fair, I can do it only if I have time and physical access to the network. Home routers have different gateway IPs, different web interfaces, different password policies (e.g. there might be an admin password and an additional password for changing anything), etc. It reminds me of <https://xkcd.com/627/ https://xkcd.com/627/>, but when you're launching a product that isn't good enough. It's hard enough to open up a port even with uPNP (typically disabled) and other made-for-purpose tech. Torrent clients end up trying to poke holes and such. Service discovery might work via local UDP broadcast, or it might not. LAN clients might live at 10.* or 192.* or be isolated by default. It's easier to just go onto the public internet and contact some mysterious server. Botnet by design.
- BlueTemplar 2y ago
  You mention IPv4. We're in 2024, this is getting ridiculous. Governments should have done the same thing as with digital TV transition(s) : first ban selling devices that can't do IPv6, then ban selling (most) devices advertising they can do IPv4.
  - yetihehe 2y ago
    Here comes Matter protocol to the rescue, it supports IPv6 natively. It's even more complicated than Zigbee and of course doesn't specify all the devices available (but 1/4 of protocol specification is dedicated to smart fridge functionality because one fridge producer actually had someone to do any collaboration with protocol makers) and allows for "manufacturer specific fields" which means all manufacturers will have incompatible implementations of some fields anyway and you can't control them universally.
kkfx 2y ago
Victron (to cite an NL vendor) actually can perfectly operate in LAN only via MQTT and ModBUS also offering a (bad) WebUI locally for settings pretty anything, including a display for the said WebUI in a framebuffer with an embedded mini-keyboard. It's up to the installer decide to go with their cloud offer or not. The sole remark I have against them (beside the not so good software quality it's the impossibility for individual owners to do offline updates, we can upgrade via VRM portal but not downloading fw and flash it locally even if the needed device is on sale, because they offer fw files only to registered vendors. Fronius (to remain in the EU) have a local WebUI witch need a connection only for fw updates, even if differently from Victron it's not a Debian based system with sources available but a closed source one, they unfortunately offer only a very limited REST API and a very slow ModBUS but still anything con be do locally. I'm not sure, since I haven't any myself by SMA (Germany) and Enphase (USA) seems to been able to operate offline as well. Stated that, yes, you are damn very right in saying most installers have no competence, thankfully where I live self-installation is allowed (at least so far), but that's simply demand better UIs and training for them perhaps avoiding the current state of the industry with an immense amount of CRAP at OEM level, with most "state of art" systems not at all designed to be used in good ways (see below) and absurdly high prices to the customer at a level it's not interesting installing p.v... 4 years ago I paid my system 11.500€ for 5kWp/8kWh LFP, the smallest offer to have it designed and built by someone else was ~30.000€ the most expensive ~50.000€ and all the 6 offers I tried shows some unpleasant issues and incompetence. About OEMs just observe how ABSURD is that there is no damn DC-to-DC direct car charger. Most EVs now have 400V batteries, the same of stationary batteries, with equal BMS comms. Why the hell not sell an MPPT-to-CSS combo direct solution? Ok, we do not ONLY charge from the Sun, than it's perfectly possible have a compo charging station with DC for p.v. and AC for the grid, switching from one to another as needed. It's ~30% energy lost in double conversion. Why no DC-to-DC high power appliance who still run DC internally (A/C, hot-water heat-pump heaters etc)? Why not a modern standard protocol for integration of anything instead of building walled gardens? Long story short OEMs have choose the cloud model partially because most installers are electricians able to use desktop holding the mouse with one hand and clicking with the other, but also because they have no intention to made user-interesting solution in an open market...
- ansgri 2y ago
oezi 2y ago
The key failure is that despite the IPv6 transition we don't have static IPs at home and can start hosting servers at home. Certainly this requires a lot of progress to secure the IOT space, but we can allow the enshitification of clouds to continue.
crabmusket 2y ago
I agree, and I wish it were otherwise. Why is it so difficult for me to have a home network where things can just work? Why is it a mess of configuration and self signed certificates? It seems like nobody is incentivised to provide this, because nobody providing me with devices, services, and so on lives in my house with me. They need my data and my control pathways to go via them, not to stay in my house.
USiBqidmOOkAqRb 2y ago
>they want their panels to be accessible when they are outside their home I call bullshit. They've been conditioned to think that they want it, because all product brochures have it. What kind of tangible benefit could there be to know how bright the sun is at your home while you're not there? A cool party trick to virtue signal or a break between doomscrolling, I suppose, but it's not like you're gonna jump up and drive back to... what even could you do if you knew?
- yetihehe 2y ago
  > I call bullshit. They've been conditioned to think that they want it, because all product brochures have it. You gave reason WHY they want it. Maybe consumers were conditioned to want access, but they still want access. If you give them similar devices, they will chose the one which has application or webpage to see how their big investment is actually working. It's not about current state of device, it's all about historical data and month-by-month savings presented as a nice graph. They will check this maybe every week or month (later every several months), but buyers still want to know what their installation did for them.
pjc50 2y ago
OK, so key question: why is there a control plane in there at all? I can understand people wanting to be able to see the metering live, but remote control of the panels just seems like a security incident waiting to happen. I'm quite glad I have a non-internet-connected inverter.

Nextgrid 2y ago

The cloud can operate as a dumb TURN relay relaying E2E-encrypted traffic. Then the worst the cloud can do is deny service to remote management (and even then, local management would still work), but it wouldn't be able to send direct control commands to the equipment since they don't have the authentication nor encryption keys. This also makes it simpler from a programming point of view - instead of having separate cloud sync & local control protocols, you just have one local protocol and you merely tunnel it through the (dumb) cloud if you can't connect directly.

yetihehe 2y ago
It could, but this requires to store historical data about usage on devices. If you store that encrypted data in cloud, then getting it to your mobile phone is super slow. If you store it in cloud, you can get historical data even if your device is dead or has 256 BYTES of memory and 1 megabit of flash storage. We have such devices, very effective at managing local municipal heating network and controlling several thermal controllers each via rs232 or rs485. Fortunately we preemptively moved everything into VPN'ed mobile network, we need special approval to touch anything on that network and can't connect without them granting access, so after EU started moving with cybersecurity this year, we are covered. > This also makes it simpler from a programming point of view - instead of having separate cloud sync & local control protocols, you just have one local protocol and you merely tunnel it through the (dumb) cloud if you can't connect directly. Having only cloud protocol is even simpler, I've done all of the above (I do backend and our firmwares).
- Nextgrid 2y ago
  > we preemptively moved everything into VPN'ed mobile network Unless your device itself is handling the VPN, I have bad news for you if you trust the mobile network to not open your devices up to malicious attackers: https://berthub.eu/articles/posts/5g-elephant-in-the-room/ https://berthub.eu/articles/posts/5g-elephant-in-the-room/
  - yetihehe 2y ago
    We consider "they hacked the mobile network VPN's AND had time to reverse our protocol before being booted out of network" as too high a level to be resolved by us. If someone has enough resources to do this, he will also just hack into standard-level secured server at municipal office and there will probably be no one there to stop him or discover what went wrong.
    - adrianN 2y ago
      Do you at least fuzz your software?
    - DoctorOetker 2y ago
      reversing the protocol can be done in advance, if they order your product
wmf 2y ago
I don't think E2E is simpler to program if you want to get it right. There are entire companies whose raison d'être is actually managing keys properly (e.g. Signal, Tailscale).
akira2501 2y ago
This should be the basic model. A fully third party TURN service. You pay $20/mo to keep your home connected, and all devices and providers can use a standard protocol, and users remain fully in control of their data.

bdamm 2y ago

For IoT stuff in general; I can do it, and I don't want to because I'd rather spend my time doing other things (although yeah, I totally did learn everything I could about my solar array, because it is a source of power, after all. But for the other stuff...)

danielovichdk 2y ago

These plants or farms are usually built around and on top of industrial IEC protocols and SCADA controllers which is a lot more low level than what any cloud IoT privider offers. I have done a controller for a 40 foot container battery and it wasn't like we received any API from Hitachi (battery manufactor). We had to write everything ourselves.

pheatherlite 2y ago

If we've learned anything from the security cam and baby cam scandals, then it's that convenience is king and we as a society would rather risk everything than be arsed to take few additional steps to setup/learn something to prevent such basic breaches. We (the society) don't even want to change the default password on most things.

yetihehe 2y ago
> We (the society) don't even want to change the default password on most things. Like you wouldn't believe. My most memorable case of insecure IoT devices - wifi socket was sending wifi ssid and password of the network in cleartext in every ping packet to chinese servers.
titzer 2y ago
People gonna be people. It's up to engineers and product designers to make things user friendly but also safe-by-default. If something needs to be configured, then provide instructions on how to configure it. Instead of pretending that it's society's fault (can't be arsed), maybe ask why the IT industry can't make instructions that are written out--explicit, fairly standard, and easy to follow--like the manual for putting together a piece of furniture. Or why the stupid device doesn't come with a randomly-generated strong password taped to it.
- yetihehe 2y ago
  > fairly standard, and easy to follow--like the manual for putting together a piece of furniture. And people still have problems following that instructions. We gave a lot of instructions, but some people just don't read them. Or can't understand and follow them. Example from this week: Yeah, main support guy for our client is on holidays this week and only a girl from sales is available and she doesn't know why our device doesn't work, she tried resetting her phone wifi but still can't pair our zigbee smart hub connected with usb modem stick (problem: no one told her she needs to message us to actually activate sim card before installing device for end client). Yes, you can solve this one problem, but there are many more we didn't see yet. Consumer support does not scale and you can't write tests for something you don't know will be a problem.

lucianbr 2y ago

What's the reasoning for not allowing both control paths, via cloud but also locally? So that people who can and want to, will use the local control.

yetihehe 2y ago
Cheapness. It would require to be at least semi secure, application on phone would need to find those devices locally and it should be synchronized with cloud anyway, synchronization is error prone and we had problems with devices sometimes responding twice or very slowly through local interface (through cloud was much faster, no idea why, not our firmware). Also not enough people requesting that feature, most don't care and think that losing internet is not often enough to warrant worrying about this.
- MostlyStable 2y ago
  Why not offer an either/or rather than both? Some people (I am one of them) actively do not want these kinds of things to be managed through the cloud servers. I don't want it to sync, I want to fully turn that off. I want to locally host, and I'm willing to take responsibility for that feature, including when it breaks. All I want is access to whatever the data reporting and control APIs are. I get that I'm a tiny minority, and that very few customers want what I want. But A) it seems like giving me what I want should be very cheap (i.e should not entail ongoing customer support costs beyond normal, and in fact would get rid of the small cloud hosting cost) and B) I'd be willing to pay a premium to get it.
  - wmf 2y ago
    In some areas like cameras there are a decent number of cloud-free alternatives. Hopefully as the IOT market grows we'll get cloud-free versions of everything. I think you're too optimistic about costs though. Providing any support at all, even one-time during the install, is expensive and cloud-free IOT is going to require support due to home networks being broken.
    - 2y ago

bee_rider 2y ago

Also administering a bunch of IOT systems is a pain. If something is an open source community project, ok, I’ll play. If somebody is selling a product they are responsible for making sure it works.

Gibbon1 2y ago
You could put an sql database on a local device and just access it remotely like anything else. But you are correct you're stuck with administering each and everyone one of them. The standard go to a raz pi solution will up and die every few months. And half the time you'll need physical access to get it back. It takes a lot of work to develop an embedded system that has enough reliability.

MathMonkeyMan 2y ago

To be fair, I can do it only if I have time and physical access to the network. Home routers have different gateway IPs, different web interfaces, different password policies (e.g. there might be an admin password and an additional password for changing anything), etc. It reminds me of <https://xkcd.com/627/ https://xkcd.com/627/>, but when you're launching a product that isn't good enough. It's hard enough to open up a port even with uPNP (typically disabled) and other made-for-purpose tech. Torrent clients end up trying to poke holes and such. Service discovery might work via local UDP broadcast, or it might not. LAN clients might live at 10.* or 192.* or be isolated by default. It's easier to just go onto the public internet and contact some mysterious server. Botnet by design.

BlueTemplar 2y ago
You mention IPv4. We're in 2024, this is getting ridiculous. Governments should have done the same thing as with digital TV transition(s) : first ban selling devices that can't do IPv6, then ban selling (most) devices advertising they can do IPv4.
- yetihehe 2y ago
  Here comes Matter protocol to the rescue, it supports IPv6 natively. It's even more complicated than Zigbee and of course doesn't specify all the devices available (but 1/4 of protocol specification is dedicated to smart fridge functionality because one fridge producer actually had someone to do any collaboration with protocol makers) and allows for "manufacturer specific fields" which means all manufacturers will have incompatible implementations of some fields anyway and you can't control them universally.

kkfx 2y ago

Victron (to cite an NL vendor) actually can perfectly operate in LAN only via MQTT and ModBUS also offering a (bad) WebUI locally for settings pretty anything, including a display for the said WebUI in a framebuffer with an embedded mini-keyboard. It's up to the installer decide to go with their cloud offer or not. The sole remark I have against them (beside the not so good software quality it's the impossibility for individual owners to do offline updates, we can upgrade via VRM portal but not downloading fw and flash it locally even if the needed device is on sale, because they offer fw files only to registered vendors. Fronius (to remain in the EU) have a local WebUI witch need a connection only for fw updates, even if differently from Victron it's not a Debian based system with sources available but a closed source one, they unfortunately offer only a very limited REST API and a very slow ModBUS but still anything con be do locally. I'm not sure, since I haven't any myself by SMA (Germany) and Enphase (USA) seems to been able to operate offline as well. Stated that, yes, you are damn very right in saying most installers have no competence, thankfully where I live self-installation is allowed (at least so far), but that's simply demand better UIs and training for them perhaps avoiding the current state of the industry with an immense amount of CRAP at OEM level, with most "state of art" systems not at all designed to be used in good ways (see below) and absurdly high prices to the customer at a level it's not interesting installing p.v... 4 years ago I paid my system 11.500€ for 5kWp/8kWh LFP, the smallest offer to have it designed and built by someone else was ~30.000€ the most expensive ~50.000€ and all the 6 offers I tried shows some unpleasant issues and incompetence. About OEMs just observe how ABSURD is that there is no damn DC-to-DC direct car charger. Most EVs now have 400V batteries, the same of stationary batteries, with equal BMS comms. Why the hell not sell an MPPT-to-CSS combo direct solution? Ok, we do not ONLY charge from the Sun, than it's perfectly possible have a compo charging station with DC for p.v. and AC for the grid, switching from one to another as needed. It's ~30% energy lost in double conversion. Why no DC-to-DC high power appliance who still run DC internally (A/C, hot-water heat-pump heaters etc)? Why not a modern standard protocol for integration of anything instead of building walled gardens? Long story short OEMs have choose the cloud model partially because most installers are electricians able to use desktop holding the mouse with one hand and clicking with the other, but also because they have no intention to made user-interesting solution in an open market...

ansgri 2y ago

oezi 2y ago

The key failure is that despite the IPv6 transition we don't have static IPs at home and can start hosting servers at home. Certainly this requires a lot of progress to secure the IOT space, but we can allow the enshitification of clouds to continue.

crabmusket 2y ago

I agree, and I wish it were otherwise. Why is it so difficult for me to have a home network where things can just work? Why is it a mess of configuration and self signed certificates? It seems like nobody is incentivised to provide this, because nobody providing me with devices, services, and so on lives in my house with me. They need my data and my control pathways to go via them, not to stay in my house.

USiBqidmOOkAqRb 2y ago

>they want their panels to be accessible when they are outside their home I call bullshit. They've been conditioned to think that they want it, because all product brochures have it. What kind of tangible benefit could there be to know how bright the sun is at your home while you're not there? A cool party trick to virtue signal or a break between doomscrolling, I suppose, but it's not like you're gonna jump up and drive back to... what even could you do if you knew?

yetihehe 2y ago
> I call bullshit. They've been conditioned to think that they want it, because all product brochures have it. You gave reason WHY they want it. Maybe consumers were conditioned to want access, but they still want access. If you give them similar devices, they will chose the one which has application or webpage to see how their big investment is actually working. It's not about current state of device, it's all about historical data and month-by-month savings presented as a nice graph. They will check this maybe every week or month (later every several months), but buyers still want to know what their installation did for them.

pjc50 2y ago

OK, so key question: why is there a control plane in there at all? I can understand people wanting to be able to see the metering live, but remote control of the panels just seems like a security incident waiting to happen. I'm quite glad I have a non-internet-connected inverter.