3 ms·
Depends. I have a cluster serving about half a million clients that send binary sensor data every 1-3 minutes. Sometimes the hardware has bugs and send erroneou
by keithnoizu 7y ago
Depends. I have a cluster serving about half a million clients that send binary sensor data every 1-3 minutes. Sometimes the hardware has bugs and send erroneous payloads while the majority just continue running. So instead of spending a lot of time figuring out all of the possible malformed inputs I just let the processes restart in those cases and otherwise resume.
There are certainly ways you can get run away reactions that eventually take down the otp tree regardless of your settings and I probably do more input validation than strictly called for my the erlang guide books but not spending all of my time worrying about those edge cases does have it's upside with out negatively impacting other clients.
- phtrivier 7y agoInterested: do you also do heavy business logic with the data you receive ? How easy is it to refactor this logic ? Or is your cluster more of an agnostic broker / message pump that lets other systems do the logic ?
- keithnoizu 7y agoit calculates running and time segment based average/high/low/tally values and drives alerts off of them. Processing is done in stages it's not particularly hard to change how one stage works as long as you aren't talking about major struct type changes or message format changes as those are a bit harder to rollout if not harder to implement.