5 ms·

Something like this is so sorely needed. I've been doing some work in ag-tech for the past few years (having previously worked in ISP/telco, general SME web/mo

Something like this is so sorely needed. I've been doing some work in ag-tech for the past few years (having previously worked in ISP/telco, general SME web/mobile development, then consumer travel tech). The tools available to farmers/growers for what should be quite basic things - i.e., web-connected weather stations and monitoring systems for soil moisture, frost alarms and irrigation control are terrible. The industry is full of small players who take a look and think "I know how to build stuff with [Arduino/Raspberry Pi/Zigbee etc], I can build an ag-tech monitoring/control product easily". So everyone builds a hardware product from scratch, then as a quick afterthought, a web app with an SQL database and Highcharts to show data graphs. You have an MVP within a few months, then a few paying customers in your local community, then suddenly your customers start asking for features you'd never thought of, like sensor inputs you didn't know existed, or combined displays of different kinds of data you'd never considered anyone would want, or needing it to be faster (SQL turns out to be really slow for this kind of data) or more reliable (bugfixing is hard when your devices are hundreds of miles away and connected via weak 3G/4G data links), and you realise your hardware and software architecture doesn't support any of this, and you'd have to completely start over to actually deliver something that customers really need, but you're out of money and energy. The result is a lot of farmers/growers dissatisfied/frustrated at never being able to get the monitoring systems they need (some of them having tried 3+ different vendors), and a lot of new companies trying to build new solutions and going bust. I've been thinking there needs to be some widely accepted open standards in the industry for hardware and software platforms, so solutions providers can avoid trying to re-invent everything and instead focus on integration of tried-and-true building blocks. If this initiative brings that about, it would be a big win for everyone in the industry.

bshipp 5y ago
You appear to have had the exact same experience as I have. I work in the fresh vegetable side of things and the tools that are available are completely closed source and locked in. Even silly things like humidity controls and whatnot. there's so much potential for the ag sector to benefit from all these open data sources and systems.
- krapht 5y ago
  You can't make money selling hardware with open API, though. It'll get cloned and manufactured in China, and people buying in bulk will naturally choose the cheapest option. Nobody makes money in open-source selling software, either. Everything is either consulting, or SAAS.
tracyhenry 5y ago
Can you elaborate on the kind of data and queries that SQL is slow for?
- openthc 5y ago
  Likely the sensor data being stuffed into a "standard" SQL schema -- loads of these bespoke solutions aren't fully dialed in with tools like timescaleDB, or Prometheus for these metrics. Even with slower (eg 240s interval) sensors the data builds up -- and slows the systems (w/o indexes).
- void_mint 5y ago
  The problem that arises with a lot of these "pull data from sensors, pump it into a database" is schemas and data integrity have to be kind of a second-class problem behind storage. When you can't push an update to whatever is ingesting data, and that ingestion tool is also ingesting with an invalid format, you can't just ignore the data (or fix the problem). So your store has to accommodate semi structured and unstructured data gracefully. I do not agree that SQL is "slow" for these types of problems. I've built a number of systems that support this issue effectively. You _could_ use a tool that has schemaless/unstructured data as a first-class feature, but if your goal is to reduce complexity a Postgres instance is just fine. As with all data projects, indexing is important and needs to be thoughtful (from the beginning). For sensor data, it's also a good idea to think about data retention and removal policies immediately (keep your metrics/aggregates, move raw data to cold storage after a while).
ahepp 5y ago
Are you aware of any promising companies or start ups in this area?
- dementiev 5y ago
  I'll do a bit of self-marketing, but we at GeoPard Ag https://geopard.tech https://geopard.tech work on this. Have some big ag companies as clients
avip 5y ago
Wow. I worked in an agtech startup and this is exactly how it went. Hopefully your comment stays on top for others to read.
void_mint 5y ago
The best work I've ever done was for one of the biggest energy/agg companies in the world. They wanted an outside contractor to build something from scratch and ignore their normal corporate bureaucracy. They thought it was insane how much better the system we (I) built for them was than their normal in house tooling. The crazy part was the solutions were mostly just normal cloud tech. Postgres, DynamoDB, some AWS tools. They paid over a million dollars (nothing to them) for a set of tools that weren't technically advanced at all. In my experience, the problems are a result of the manufactured constraints on the physical hardware/sensor devices. Mountings and power and connectivity. Receiving data from an array of devices into a shared storage system for analysis is a well solved problem. Receiving data from multiple sensors on a fixed interval, when the sensors may be made by different companies/work totally differently, is where the complexity lives. Combined with each company trying to build their own awful closed source proprietary data system on top of their sensors, you've got a really terrible time. > SQL turns out to be really slow for this kind of data I think this is just poor modeling. SQL is just fine for the work you're talking about.
- ethbr0 5y ago
  > Combined with each company trying to build their own awful closed source proprietary data system on top of their sensors, you've got a really terrible time. The more I think about right to repair, the more I become convinced it's a symptom of hazy interface specifications. Radical idea: Ban sensor / actuator companies from building software on top of them in-house. They're welcome to offer a turn-key solution to market, but it must (1) have hardware and software built by two separate, independent companies & (2) publish its interface specs, between those two companies, to all customers or end users. Things would cost more. But I'm not convinced this would be a worse world, in aggregate.
  - lukeschlather 5y ago
    Kind of simpler to me, is just making closed-source drivers illegal. I like the idea of being able to buy something cheap because they were too lazy to actually write a spec for how it works. But not being able to read the de facto spec - the source code for the driver - is just silly. I've paid for the device, the driver is basically useless except for that device. Give me the code.
walleeee 5y ago
There is a bit of progress towards open standards: e.g., MIAPPE, BrAPI. But you're right, we have a long way to go. Programmers and software/hardware engineers are sorely needed but ag struggles to attract them, and the startup model is not very well suited for the space imo- we need to build a collaborative ecosystem of open source tooling that farmers, biologists, breeders, and others in ag/plant sciences can hack for their particular use case Check out Phenome Force, they do weekly webinars featuring the open source/DIY tools people are building: https://phenome-force.github.io/PhenomeForce/ https://phenome-force.github.io/PhenomeForce/
- mch82 5y ago
  Is remote work common in AgTech?
  - walleeee 5y ago
    I can't comment on the situation prior, but post-covid it's pretty common afaik. I've been entirely remote since last March, in both software developer and grad student roles, and although word is students will be back in the office/classroom this fall at my institution I think there is likely a lot of flexibility w.r.t. software-oriented positions I know a few people who work outside of academia, e.g. agtech companies under the Flagship Pioneering umbrella (Inari etc, same parent as Moderna and various other ag/biotech) and the software teams are very remote friendly I believe If you're interested in a staff position at a university there's a chance my PI might be hiring for full time dev positions later this year (dependent on luck with grant applications), lmk if interested
dementiev 5y ago
That's the issue indeed. The entry threshold and the amount of minimum needed features for agtech products are very high. Moreover, the seasonality of ag business makes the situation for agtech startups even more difficult (you can usually sign new test clients only in between ag seasons)
rsj_hn 5y ago
You know what I'd like to see is there can be other non-dev support for these types of efforts. For example: * technical writers * security pentesting/code reviews * hosting and infrastructure support services for open source projects Pretty sure that there are a lot of people who would be happy to donate their time to opensource projects that benefit farmers, so there is a bigger opportunity for matchmaking here.
dopidopHN 5y ago
Interested question : would those folks be interested in low bandwidth / long distance network over the 800khz band ( Lora ) ? There is incentive to deploy those network now. A agreement to deploy is all what is needed ( private property style ). Electricity is a plus but that work well on solar too.

bshipp 5y ago

You appear to have had the exact same experience as I have. I work in the fresh vegetable side of things and the tools that are available are completely closed source and locked in. Even silly things like humidity controls and whatnot. there's so much potential for the ag sector to benefit from all these open data sources and systems.

krapht 5y ago
You can't make money selling hardware with open API, though. It'll get cloned and manufactured in China, and people buying in bulk will naturally choose the cheapest option. Nobody makes money in open-source selling software, either. Everything is either consulting, or SAAS.

tracyhenry 5y ago

Can you elaborate on the kind of data and queries that SQL is slow for?

openthc 5y ago
Likely the sensor data being stuffed into a "standard" SQL schema -- loads of these bespoke solutions aren't fully dialed in with tools like timescaleDB, or Prometheus for these metrics. Even with slower (eg 240s interval) sensors the data builds up -- and slows the systems (w/o indexes).
void_mint 5y ago
The problem that arises with a lot of these "pull data from sensors, pump it into a database" is schemas and data integrity have to be kind of a second-class problem behind storage. When you can't push an update to whatever is ingesting data, and that ingestion tool is also ingesting with an invalid format, you can't just ignore the data (or fix the problem). So your store has to accommodate semi structured and unstructured data gracefully. I do not agree that SQL is "slow" for these types of problems. I've built a number of systems that support this issue effectively. You _could_ use a tool that has schemaless/unstructured data as a first-class feature, but if your goal is to reduce complexity a Postgres instance is just fine. As with all data projects, indexing is important and needs to be thoughtful (from the beginning). For sensor data, it's also a good idea to think about data retention and removal policies immediately (keep your metrics/aggregates, move raw data to cold storage after a while).

ahepp 5y ago

Are you aware of any promising companies or start ups in this area?

dementiev 5y ago
I'll do a bit of self-marketing, but we at GeoPard Ag https://geopard.tech https://geopard.tech work on this. Have some big ag companies as clients

avip 5y ago

Wow. I worked in an agtech startup and this is exactly how it went. Hopefully your comment stays on top for others to read.

void_mint 5y ago

The best work I've ever done was for one of the biggest energy/agg companies in the world. They wanted an outside contractor to build something from scratch and ignore their normal corporate bureaucracy. They thought it was insane how much better the system we (I) built for them was than their normal in house tooling. The crazy part was the solutions were mostly just normal cloud tech. Postgres, DynamoDB, some AWS tools. They paid over a million dollars (nothing to them) for a set of tools that weren't technically advanced at all. In my experience, the problems are a result of the manufactured constraints on the physical hardware/sensor devices. Mountings and power and connectivity. Receiving data from an array of devices into a shared storage system for analysis is a well solved problem. Receiving data from multiple sensors on a fixed interval, when the sensors may be made by different companies/work totally differently, is where the complexity lives. Combined with each company trying to build their own awful closed source proprietary data system on top of their sensors, you've got a really terrible time. > SQL turns out to be really slow for this kind of data I think this is just poor modeling. SQL is just fine for the work you're talking about.

ethbr0 5y ago
> Combined with each company trying to build their own awful closed source proprietary data system on top of their sensors, you've got a really terrible time. The more I think about right to repair, the more I become convinced it's a symptom of hazy interface specifications. Radical idea: Ban sensor / actuator companies from building software on top of them in-house. They're welcome to offer a turn-key solution to market, but it must (1) have hardware and software built by two separate, independent companies & (2) publish its interface specs, between those two companies, to all customers or end users. Things would cost more. But I'm not convinced this would be a worse world, in aggregate.
- lukeschlather 5y ago
  Kind of simpler to me, is just making closed-source drivers illegal. I like the idea of being able to buy something cheap because they were too lazy to actually write a spec for how it works. But not being able to read the de facto spec - the source code for the driver - is just silly. I've paid for the device, the driver is basically useless except for that device. Give me the code.

walleeee 5y ago

There is a bit of progress towards open standards: e.g., MIAPPE, BrAPI. But you're right, we have a long way to go. Programmers and software/hardware engineers are sorely needed but ag struggles to attract them, and the startup model is not very well suited for the space imo- we need to build a collaborative ecosystem of open source tooling that farmers, biologists, breeders, and others in ag/plant sciences can hack for their particular use case Check out Phenome Force, they do weekly webinars featuring the open source/DIY tools people are building: https://phenome-force.github.io/PhenomeForce/ https://phenome-force.github.io/PhenomeForce/

mch82 5y ago
Is remote work common in AgTech?
- walleeee 5y ago
  I can't comment on the situation prior, but post-covid it's pretty common afaik. I've been entirely remote since last March, in both software developer and grad student roles, and although word is students will be back in the office/classroom this fall at my institution I think there is likely a lot of flexibility w.r.t. software-oriented positions I know a few people who work outside of academia, e.g. agtech companies under the Flagship Pioneering umbrella (Inari etc, same parent as Moderna and various other ag/biotech) and the software teams are very remote friendly I believe If you're interested in a staff position at a university there's a chance my PI might be hiring for full time dev positions later this year (dependent on luck with grant applications), lmk if interested

dementiev 5y ago

That's the issue indeed. The entry threshold and the amount of minimum needed features for agtech products are very high. Moreover, the seasonality of ag business makes the situation for agtech startups even more difficult (you can usually sign new test clients only in between ag seasons)

rsj_hn 5y ago

You know what I'd like to see is there can be other non-dev support for these types of efforts. For example: * technical writers * security pentesting/code reviews * hosting and infrastructure support services for open source projects Pretty sure that there are a lot of people who would be happy to donate their time to opensource projects that benefit farmers, so there is a bigger opportunity for matchmaking here.

dopidopHN 5y ago

Interested question : would those folks be interested in low bandwidth / long distance network over the 800khz band ( Lora ) ? There is incentive to deploy those network now. A agreement to deploy is all what is needed ( private property style ). Electricity is a plus but that work well on solar too.