3 ms·

Author here. Any time you want to store historical data, and query that data later. Very similar to how you might imagine using a database for a website, just

by geedy 11y ago

Author here. Any time you want to store historical data, and query that data later. Very similar to how you might imagine using a database for a website, just at a smaller scale. The project that inspired this work was a water metering project in Kelowna, BC. Basically, a friend of mine took a bunch of micro-controllers with soil moisture sensors, shoved them in the ground, and tried to come up with better demand-based watering schedules. It took them two months to develop the data management code, and knew it could have been days or hours with a proper database.

tptacek 11y ago
Thanks! What would the storage look like for this? (I have an ulterior motive for asking, which is that I will probably end up shoplifting this code and embedding it into the next set of levels for our CTF game, which is serverside-emulated AVR).
- geedy 11y ago
  I did a fair amount of testing with arduino-based SD-card storage, but as long as you had some sort of way to store bytes to your preferred medium, that shouldn't be too hard to hack yourself. Check out https://github.com/graemedouglas/LittleD/blob/master/src/dbstorage/dbstorage.c https://github.com/graemedouglas/LittleD/blob/master/src/dbs... EDIT: Please let me know how/if you end up using it! Or if you run into bugs! :)
VLM 11y ago
If I'm reading you right and extending your example more concretely, you're doing IoT SIMD parallelization, right? So rather than 1M soil moisture sensors continuously sending records to a central repo, you're giving the mesh network something exciting to do by distributing "select * from db where moisture<200" or whatever then collating all the responses, if any... There's a classic crypto thought experiment illustrating SIMD for cracking keys where the Chinese distribute 100M boom boxes (well, its an old thought experiment) and they randomly test keys and when the red light turns on indicating factor found, the owner of the radio turns it in for a substantial reward. And thats how SIMD key cracking gets a 27 bit parallelism speedup over one box. In the IoT era I assume its going to be a normal thing for lightbulbs and toasters to get powned to mine bitcoins and the like. You could probably extend both the real story and the old crypto thought experiment to help provide some docs. Then again I read your unit tests and they kind of document the system pretty well. Nice tests. In true supercomputer fashion, by going SIMD you've taken a formerly CPU bound problem, and turned it into an IO bound problem, but for little microcontrollers not classic supercomputer hardware, which I thought was pretty funny.
- geedy 11y ago
  In the example given, it was more about needing to collect data over a large area, such as across multiple city parks. They needed to manage data at the source(s) to inform good watering decisions.

by geedy 11y ago

tptacek 11y ago

Thanks! What would the storage look like for this? (I have an ulterior motive for asking, which is that I will probably end up shoplifting this code and embedding it into the next set of levels for our CTF game, which is serverside-emulated AVR).

geedy 11y ago
I did a fair amount of testing with arduino-based SD-card storage, but as long as you had some sort of way to store bytes to your preferred medium, that shouldn't be too hard to hack yourself. Check out https://github.com/graemedouglas/LittleD/blob/master/src/dbstorage/dbstorage.c https://github.com/graemedouglas/LittleD/blob/master/src/dbs... EDIT: Please let me know how/if you end up using it! Or if you run into bugs! :)

VLM 11y ago

If I'm reading you right and extending your example more concretely, you're doing IoT SIMD parallelization, right? So rather than 1M soil moisture sensors continuously sending records to a central repo, you're giving the mesh network something exciting to do by distributing "select * from db where moisture<200" or whatever then collating all the responses, if any... There's a classic crypto thought experiment illustrating SIMD for cracking keys where the Chinese distribute 100M boom boxes (well, its an old thought experiment) and they randomly test keys and when the red light turns on indicating factor found, the owner of the radio turns it in for a substantial reward. And thats how SIMD key cracking gets a 27 bit parallelism speedup over one box. In the IoT era I assume its going to be a normal thing for lightbulbs and toasters to get powned to mine bitcoins and the like. You could probably extend both the real story and the old crypto thought experiment to help provide some docs. Then again I read your unit tests and they kind of document the system pretty well. Nice tests. In true supercomputer fashion, by going SIMD you've taken a formerly CPU bound problem, and turned it into an IO bound problem, but for little microcontrollers not classic supercomputer hardware, which I thought was pretty funny.

geedy 11y ago
In the example given, it was more about needing to collect data over a large area, such as across multiple city parks. They needed to manage data at the source(s) to inform good watering decisions.