20 ms·

Tractor Hacking: Documentary About Farmers Fighting for the Right to Repair

352 points by goodmachine 9y ago

dancek 9y ago
There was an article about these John Deere hacking farmers a year ago, and this is a follow-up. There was good discussion on HN back then: https://news.ycombinator.com/item?id=13925994 https://news.ycombinator.com/item?id=13925994
pdelbarba 9y ago
I work in the embedded field and this drives me nuts. I personally feel that companies should be required to go one step further and be compelled to release their embedded keys and source code for any product that they no longer provide complete support for. The embedded field is strange because the lock-in is generally on the side of the hardware design and tooling. Nobody is going to try to clone a tractor because they have the source code for it. They also need the CAD drawings, machine paths, tooling, supply chain, etc... Yet just about every company in the space feels that their code is absolutely sacred above all else. Just about everything you own running embedded software that has some update functionality has a bootloader containing cryptographic keys such that the firmware patches/images cannot even be disassembled. For a few devices like routers, some toys, a couple handheld radios and the like, people have been able to modify them through exploits that cause firmware dumps from the uC's memory but otherwise it's typically prohibitively time intensive to rewrite new firmware from scratch without having access to the device schematics (and even then). Worse, just about all these devices have some access to JTAG ports such that you could easily program them yourself if you wanted to and had something useful to flash them with. I've run into so many products I use on a regular basis with bugs or simple but badly needed features that I could have easily fixed/hacked in with access to the source code but alas, I cannot.
- deepsun 9y ago
  I believe the change should start from consumers, especially big institutions. For example, a big municipality or government agency may impose limits on buying new software/hardware that isn't open to refactor/repair by their competitors, to avoid lock-in to a single supplier, operate on more open market, and consequently drive the costs down.
  - pdelbarba 9y ago
    I like the idea. I'm worried that this ship has sailed though as it's becoming harder to even get phones without cryptographically verified bootloaders, and god forbid you want to mess with the baseband/modem source. This is only going to get worse too, as software becomes the part that's actually valuable. AI will mean that everything on the processor will be a trade secret x1000 and everything that can possibly be kept from the local processor will be hidden away in the cloud, forever dependent on continued server access.
- rhizome 9y ago
  I personally feel that companies should be required to go one step further and be compelled to release their embedded keys and source code for any product that they no longer provide complete support for. Sounds great, but it's something the abandonware[1] scene has been fighting for over the past 20 years, to no avail. 1. https://en.wikipedia.org/wiki/Abandonware https://en.wikipedia.org/wiki/Abandonware
  - Slansitartop 9y ago
    It's a good idea, so people should keep fighting for it, even if it takes more than 20 years.
  - pdelbarba 9y ago
    I think the difference here is that when it comes to embedded software, very expensive capital can be tied up, as is the case in the above video. Not many commercially available software packages cost >$10k and very few people are exposed to that risk. Meanwhile, there are hundreds of millions of cars in the US.
  - archgoon 9y ago
    It'll probably take one generation to raise the issue (and find an example for enough people to care), another to debate, and finally a third to agree about the common sense solution. So give it another 40 years.
- aplorbust 9y ago
  "Just about everything you own running embedded software that has some update functionality..." "Worse, just about all these devices have some access to JTAG ports such that you could easily program them yourself if you wanted to and had something useful to flash them with. I've run into so many products I use on a regular basis with bugs or simple but badly needed features that I could have easily fixed/hacked in with access to the source code but alas, I cannot." IMO, it is a massive waste. Short-sighted, short-term thinking. There are people who for whatever reason do not want you to have access to the software ("firmware") and to read/fix/improve upon it. Some of these people comment on HN. They become active during debates over whether users would derive any benefit from being able to read and edit source code.
  - dpwm 9y ago
    The problem seems to be that software is viewed as a trade secret. Big old companies have CEOs and boards that see embedded software in the same way many legislators and judges see it. Firstly they don't really understand it. But in any case it's part of the product and people have no reason to tinker with it. Now software is being used to enforce what manufacturers could previously have only dreamed of: monopolizing repairs. With hindsight it seems obvious that things converged onto the embedded computers of consumer products: think microwaves and TVs rather than general purpose computers. The problem with tractors is that all the farmers I know have a much better idea of how a tractor works and how to fix it than the average motorist does about their vehicle. I am reminded of the era when windscreen-mounted GPS navigation systems started becoming mainstream. Most of the models had very similar hardware but the software was all that differed. I never looked into it but I suspect almost zero had source code published. To the more typical consumer, the unit is as a whole and the software comes with it. If it routinely takes you the wrong way down a one way street or tries to take you down a set of steps, it's defective and needs to be replaced. To the more technical consumer it's clear that either the software or the data is at fault and either could be fixed with the right resources. I still feel a sadness at this, yet this waste is insignificant compared to a tractor or any other vehicle.
    - carlmr 9y ago
- grkvlt 9y ago
  > embedded field [...] companies should be [...] compelled to release their embedded keys and source code awesome! my botnet malware that exploits and roots your smart meter or set top box can now install new firmware with a permanent backdoor and nothing will complain because all the malware binaries are signed - by the manufacturer's key, no less - so they must be secure, right?
  - M_Bakhtiari 9y ago
    It would be beyond retarded to use the exact same key for every unit. That's not what people are asking for. People are asking for keys to their exact units so they and only they can sign software for them.
    - evancox100 9y ago
      While GP's snark is uncalled for, he is correct as that's exactly how this works. What you're asking for is like saying that a web server would provide a unique HTTPS cert to every distinct visitor.
      - M_Bakhtiari 9y ago
        >What you're asking for is like saying that a web server would provide a unique HTTPS cert to every distinct visitor. It's not similar at all. The equivalent would not be the certificate but the session key, and you do get your own session key in order to prevent what the person above is describing. My HN session key is useless for decrypting your HN password even if I could intercept the traffic. There is no technical reason why the devices can't ship with not only the manufacturer's public key, but also a key pair generated for each unit that comes off the assembly line for the customers to use to sign their own firmware images (and if they wish, delete the manufacturer's public key). But they will never be able to sign images for any device but their own because they simply don't have any signing keys that can produce a signature that will be accepted by any device but their own.
- jabl 9y ago
  Not to defend Deere et al., but one argument I've seen bandied about is that the manufacturers consider their software to be their "secret sauce" value add. If machined steel were all there is, there's nothing preventing the Chinese (or whoever happens to be the current outsourcing bogeyman) from producing perfectly good equipment at a fraction of the price. Personally I'm not convinced this is a really sustainable "moat"; surely the Chinese (boo!) can do embedded programming as well.
  - squarefoot 9y ago
    That's why a lot of people are advocating for a fair law that forces companies to release sources after a reasonable amount of time or when they cease to support the product, so that they can profit for some years from their IP but users aren't screwed when the product becomes obsolete. And of course being repairable and/or upgradeable means that a piece of technology won't be thrown away.
- M_Bakhtiari 9y ago
  >I personally feel that companies should be required to go one step further and be compelled to release their embedded keys and source code for any product that they no longer provide complete support for. The embedded team where I work struggles with the same thing. Frankly this should be included with the first delivery of any industrial system. Under NDA if they absolutely must, though I don't like it. I also don't like consumer systems being locked up, but at least it's somewhat defensible. But treating farmers as dumb, inbred "hold my beer and watch this firmware hack" hillbillies and not the educated industry professionals that they are is beyond condescending, disrespectful and is completely indefensible. Exactly who is deemed qualified to hack farm equipment firmware is something for the farming trade associations and their insurance companies to work out, but as far as John Deere and the regulators are concerned, we're talking about professional embedded developers writing code to run on their equipment to support an industrial process, something that is completely uncontroversial in many other fields. That you're not supposed to intentionally or inadvertently convert your combine into a man-eating killbot is surely already covered by insurance policy clauses, labour laws and public safety laws.
  - mikepurvis 9y ago
    I completely agree with you, but I am a little sympathetic to the "bad PR" angle. It's really, really not that hard to imagine a scenario where a well-meaning farmer (or their contractor) loads something onto the embedded processor that does cause a malfunction, and before you know it there's a splashy news story featuring pictures of a big green tractor not living up to its brand. I work for a robotics company making indoor self-driving vehicles, and we worry about exactly this issue— from an ROI perspective, our product is a slam dunk, so it's safety and reliability are the first and second most significant questions which have to be answered during the sales process. When we invest substantially in associating our brand with those values, we really can't afford the possibility that an unauthorized user modification compromises them in a way that's outside of our control. Hence closed source, FDE, and all the rest of it.
    - squarefoot 9y ago
      That is a very specific scenario, but agreements can be made to release software in exchange of loss of warranty and liability protection: I give you the code and from that moment (either if you use/modify it or not) you lose all warranties and can't sue me even if the product burns your house and kills all your kittens.
- gjvc 9y ago
  "Nobody is going to try to clone a tractor because they have the source code for it." Thus is the funniest thing I've read in ages. It's also 100% correct and this whole situation shows perhaps clearer than many, how Stallman's warnings were accurate.
  - guardian5x 9y ago
    While that is true, i think the manufacturers are more afraid of competitors, which could in some way "clone" at least the software.
    - mrguyorama 9y ago
      I'm doubtful that DRM in any space has ever been about competitors.
- _pmf_ 9y ago
  > Nobody is going to try to clone a tractor because they have the source code for it. In the automotive industry, people and companies will do this (at the component level). Enabling features via that the OEM has disabled due to its marked segmentation strategy via aftermarket components is done (and would be done to a greater extend if OEMs did not protect the part of their diagnostic stack that handles feature activation). For example, XCP or UDS security access modules are sometimes distributed by the Tier 1 as a DLL so that even the OEM has no source code for the concrete authentication mechanism that is used.
AdieuToLogic 9y ago
A similar situation exists for consumer grade WiFi routers in the US. For about a year now, it's well-nigh impossible to flash routers with DD-WRT, OpenWRT, and friends due to the new FCC conditions manufacturers paid^H^H^H^H lobbied to get put in place.
- vuln 9y ago
  You've peaked my interest. Has then been new legislation that has enabled router manufacturers lock down their hardware to a point an end-user cannot change it? All three organizations you name are open source and community driven. They cannot possibly write or port their firmware to every platform.
  - plasticchris 9y ago
    Parent likely refers to the 5ghz dfs rules, see https://arstechnica.com/information-technology/2015/09/fcc-open-source-router-software-is-still-legal-under-certain-conditions/ https://arstechnica.com/information-technology/2015/09/fcc-o...
    - AdieuToLogic 9y ago
      I wish that were the case, but alas it is not. The article you link was unfortunately an optimistic interpretation of what not yet known to come[0]. 0 - https://www.wired.com/2016/03/way-go-fcc-now-manufacturers-locking-routers/ https://www.wired.com/2016/03/way-go-fcc-now-manufacturers-l...
  - ynezz 9y ago
    http://blog.true.cz/2017/02/free-your-router-again/#background http://blog.true.cz/2017/02/free-your-router-again/#backgrou...
  - wmf 9y ago
    A history of the FCC router lockdown thing: https://arstechnica.com/information-technology/2016/08/fcc-forces-tp-link-to-support-open-source-firmware-on-routers/ https://arstechnica.com/information-technology/2016/08/fcc-f...
akshayB 9y ago
This practice is pretty much rampant across all industries ranging from cellphones to luxury cars. One simple answer is big companies just want to extend their profits. Another reasons is they want their distributors or dealerships or middleman to thrive as well without which their business may get stagnant.
- nas 9y ago
  This is a good point. In the ag business, the dealers of equipment make little to no money actually selling the equipment. They make their profit on selling parts and doing servicing. So, keeping the necessary software tools locked to authorized dealers would be popular with the dealerships. It ensures you have to go to the dealer and not some 3rd party mechanic shop.
- kevin_b_er 9y ago
  Because copyright and software can be used to perform a backdoor attack to seize ownership of physical objects from you. You are looking at a full-on attack on the concept of ownership. You own nothing: It is licensed to you at the whims of the true owners. Why wouldn't corporations, which are sociopathic "persons" defined exclusively by their avarice not want to do so? It is more profitable to prevent repair. It more profitable to force you into buying more.
igor47 9y ago
I'm "excited" about the day that I'll be forced to choose between getting a life-saving medical device implanted, or instead throwing a hissy fit about getting the source code to something that's going in my body. I hope we can figure this shit out with tractors instead...
- anotherevan 9y ago
  That's already happening. https://youtu.be/5XDTQLa3NjE https://youtu.be/5XDTQLa3NjE https://youtu.be/8wPAHu_zYDw https://youtu.be/8wPAHu_zYDw
LeonM 9y ago
As a software engineer with an embedded systems background this kind of OEM behavior driver me up the wall. I've had OEM's: - refusing to provide interface specs of a device of which we just ordered hundreds of units. (But hey, here is the binary windows (!) driver for this embedded device) - refusing to provide the calculation method of a checksum value their device returned. - Stopping software development (tooling, firmware, drivers) on embedded products, even when there are known issues. - Dropping support on a device, even when hundreds of them are still being used in the field. It's not just tractors, its everywhere. From a business standpoint, it's just so dangerous to work with hardware vendors who don't offer implementation details or source code. If they drop support or stop existing, you're screwed...
- pdelbarba 9y ago
  Couldn't agree more. The industry is so stuck in various dogmas that all of this is totally acceptable in the pursuit of securing source code from presumably China and anyone with a screwdriver.
- nas 9y ago
  As a former farmer (roughly 4000 acres of dryland farming in Canada) and an engineer who also did embedded control system development, I would like to add some of my concerns. For new ag machinery, embedded control systems are absolutely essential for the machine to do its job. The ECU on the engine is generally reliable and have long life. The other stuff, I'm highly dubious of the long-term maintainability of it. On our farm, we ran a lot of older equipment and did a lot of repairs ourselves. For new equipment, that will be near impossible as the electronic control systems are black boxes. Even the cabling is complicated and is generally not documented. After the machine gets a decade or two old, good luck fixing it. I suppose you could just drive the machine in the junk pile. However, when a new combine harvester is costing around 3/4 of a million USD, that seems a bit wasteful. I don't know what the solution is though. The farmers buying new equipment don't care so much, their resale value is not suffering too badly. The manufacturers are looking for ways to sell new equipment and adding more features via electronic controls is a relatively cheap way to enhance the product. Most farmers say they don't want all these new electronics on their machines. When Deere announced their S700 series combines, they had a video bragging about all the new improvements: https://youtu.be/l5xJOoNyLoU https://youtu.be/l5xJOoNyLoU If you study it, you will notice that most of the new features are due to software changes or to electronic control systems. Little of what the machine actually does to harvest the grain has changed (metal parts, etc). This automation features are nice when they work but are a disaster when something goes wrong. The operator will have trouble to determine what the machine is doing (never mind trying to figure out how to fix it). However, even though many farmers say they don't want it, the new machines are selling well. So, maybe you could argue the market is working. To me, it feels like there could be some externalization of costs going on. These new machines are very much less valuable as they get older vs the previous era of farm equipment. I guess the same thing has happened to automobiles. You can take a car from the 1940s and fix it up into perfect condition. If you take a 2017 car loaded with electronics, would you have any hope of fixing it to new condition in 50 years from now?
wmf 9y ago
Is there any farm equipment (from China perhaps) that isn't DRMed? In addition to discussing the responsibilities of vendors, maybe we can also discuss the responsibility of customers to understand what they're buying.
- mveety 9y ago
  There are no new tractors that aren't DRMed if they don't have electronics. Hell, it's even hard to get service documents. Talking about consumer responsibility is great and all, too, but there is no choice so it doesn't matter.
- userbinator 9y ago
  from China perhaps In this case their general lack of effort around security (see the whole IoT mess) may be a good thing, as it means any DRM is more hackable. The flipside of that is the firmware may also be buggier, but then again, you're also more likely to be able to fix those bugs due to the hackability. The old Douglas Adams quote comes to mind: "The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at and repair."
SteveGerencser 9y ago
I own a small farm and work in tech. We are struggling with this issue right now as we look at the options for a new tractor vs an older serviceable model. While many of the larger corporate farms are content to buy new equipment every 10 years, those of us at the small end of the scale have begun a push to keep older equipment up and running. My tractor is from 1996, my hay baler from the early 70s and my hay cutter is even older. These all work just fine even if a little slow. The challenge of not being able to work on my equipment currently outweighs the marginal speed gains I would see on a farm our size. The same goes for nearly all of my neighbors. The last "new" tractor in the local area was bought a couple years ago by one of the older guys who was buying the last tractor he will ever own and he thought it would be nice to have a top of the line, new, tractor just once.
- anubisresources 9y ago
  What do you grow? Hay mainly? Acreage?
  - SteveGerencser 9y ago
    We grow hay to feed our beef cattle on about 100 acres +/- a bit. But all around us are plenty of farms growing small farm (under 500 acres) or row crops/beans etc. We were having this very discussion today when picking up some hay to help us get through till spring with a guy that cuts about 500 acres of hay 3 times a year. He just keeps rebuilding his equipment rather than get sucked into the new stuff he can't work on. This is becoming another huge issue that affects smaller farms more than it affects larger corporate type farms. When a piece of equipment goes down it's needed back in service in hours to days, not the potential weeks that hauling it to a repair depot can cause.
    - anubisresources 9y ago
      You weren't kidding about the small farm! The dealers in your area can't do field repairs?
- vvanders 9y ago
tyingq 9y ago
This story comes up quite often. It seems an opportunity for someone to launch a farm equipment business that makes it's margin on the initial sale instead of the service. Even if it's just refurbished pre-drm tractors.
bitmapbrother 9y ago
According to the video lawyers from Microsoft and Apple showed up at the legislature to voice their objections to the Right To Repair bill. Interesting that these 2 companies would be so anti-consumer.
- dfg0987098x7 9y ago
  What's so interesting about it, are you surprised? Apple have been fighting the ability to run your own code on the device since day one. This is part of the business model of the "app store" purchase model and not very surprising. I'm sure even you have heard that Microsoft have been anti-consumer once or twice over the years.
emcrazyone 9y ago
Companies generally don't want you in the software for safety reasons. While true there is intellectual property around the guidance and swath generating algorithms, companies hate lawsuits. Lawsuits tie up resources, can cause stop shipments, generally lost profits, and tarnish the brand. Look at the Chrysler Jeep news... You hack or change software on a vehicle and cause the auto-guidance to run over and kill someone or you circumvent a safety check in software unbeknownst to you due to your software changes and a spinny sharp thing that should have stopped doesn't and hurts or mames you or calibration changes cause engine to to over-heat and burn up and generally the company is held responsible for it. As much time is spent testing software as is writing software. All our testing is there to make sure the software works as expected. Millions are spent on test equipment and millions of hours and man hours are spent making sure software works as expected. I happen to work for a major agricultural company and I'm a lead software architect for their guidance + autonomous vehicles + precision farming embedded devices. We are one of John Deere's major competitors.
- Lxr 9y ago
  That's a poor argument in my opinion. Should my washing machine have no serviceable parts because I might electrocute myself?
  - eksu 9y ago
    OP was talking about tractors software and machines that can run people over and kill them, not washing machines or serviceable parts.
    - jdc 9y ago
      I agree; unlike washing machines, apparently, these tractors are made to be unserviceable.
    - vvanders 9y ago
      Yup, tractors and most farm equipment are fucking dangerous. Even the compact/subcompacts regularly maim/kill people. I can totally understand wanting to minimize every possible new vector.
EvanAnderson 9y ago
I've done contract work for a manufacturer of electronic control systems for excavation and mining equipment. They've aggressively implemented DRM in their new products to support renting time-limited and geofenced access to optional features. Their revenue models for the newest generation of equipment are based in large part on the recurring revenue stream enabled by DRM.
drunkencarolina 9y ago
A low tech alternative, depending on one's needs could be draft horses. My wife's family worked teams of belgians for about 80% of their 400-acre farm. https://www.motherearthnews.com/homesteading-and-livestock/farming-with-horses-zmaz87jazgoe https://www.motherearthnews.com/homesteading-and-livestock/f...
ktta 9y ago
This is only going to get worse. A company called Blue River Technologies, which hopes to automate weeding was acquired by John Deere(the company being discussed in the article). Their software is obviously more complex than simple firmware which can be 'hacked' by the average joe. If this weeding systems is profitable and becomes popular, these farmers are screwed. Here's[1] a nice longread on that company. It's a bit fluffy, but I recommend it. [1]: https://www.bloomberg.com/news/features/2018-01-11/this-army-of-ai-robots-will-feed-the-world https://www.bloomberg.com/news/features/2018-01-11/this-army...
jaclaz 9y ago
A similar/related thread: https://news.ycombinator.com/item?id=14074894 https://news.ycombinator.com/item?id=14074894 Risking to cite myself: https://news.ycombinator.com/item?id=14077327 https://news.ycombinator.com/item?id=14077327
tomohawk 9y ago
What if there was a law that any device containing embedded software would have to either be (a) open source, with source code and docs freely available, or (b) closed source, with source code and docs in escrow in the event that the manufacturer ceases support, at which time the code and docs would immediately become open source?
InitialLastName 9y ago
I'm involved in the design of embedded systems for a relatively consumer market, and I've looked into making my company's designs more accessible to modification/software manipulation. There are a few (sometimes major) costs to allowing the modification of software that people tend to ignore in this discussion: - Support: If you want to make your product more accessible to repair, you need to provide support for that. This costs developer time (making the code nice enough to be externally visible, maybe putting together an SDK, making sure any actual secret sauce or anti-counterfeiting systems are blocked off), customer support time (you WILL get calls when somebody buys a used system with modified firmware that doesn't work), and increases the hardware costs (want to provide a JTAG port? that's PCB space and BOM cost). - Brand reflection: We've all seen how much of a ripple the wrong person having a bad experience with a product can have on the market. Say somebody buys a modified Initech Widget (tm) that has a bug or burns some other expensive equipment that it has to work with. That person complains on their friendly local social media network that Initech Widgets have a habit of (say) turning off furnaces in the middle of winter so the user's pipes freeze. Suddenly Initech is dealing with a media storm because they allowed modification of their software, which brings us to... - Liability. What happens if somebody modifies the software in my product and somebody dies, burns down their house, or (* forbid) makes it possible to cause harmful interference and the FCC finds out? How do I prove to that user's (or the FCC's) lawyers that I shouldn't have to pay the damages? Let's assume that, as in the case of tractors or mobile baseband modems (which seem to get brought up a lot in this case) that software is a FUNDAMENTAL part of ensuring the safety/compliance of the unit. Every time I've been through that calculus, the answer at the end has been, "I should do everything in my power to prevent the firmware of my devices from being modified". I can understand and sympathize with the arguments in the other direction, and as a consumer I'm both qualified and interested in modifying the software on my widgets, but it would take a lot of change in the way our society works before most companies will put themselves at risk by supporting or even allowing it for their products.

352 points by goodmachine 9y ago

There was an article about these John Deere hacking farmers a year ago, and this is a follow-up. There was good discussion on HN back then: https://news.ycombinator.com/item?id=13925994 https://news.ycombinator.com/item?id=13925994

pdelbarba 9y ago

I work in the embedded field and this drives me nuts. I personally feel that companies should be required to go one step further and be compelled to release their embedded keys and source code for any product that they no longer provide complete support for. The embedded field is strange because the lock-in is generally on the side of the hardware design and tooling. Nobody is going to try to clone a tractor because they have the source code for it. They also need the CAD drawings, machine paths, tooling, supply chain, etc... Yet just about every company in the space feels that their code is absolutely sacred above all else. Just about everything you own running embedded software that has some update functionality has a bootloader containing cryptographic keys such that the firmware patches/images cannot even be disassembled. For a few devices like routers, some toys, a couple handheld radios and the like, people have been able to modify them through exploits that cause firmware dumps from the uC's memory but otherwise it's typically prohibitively time intensive to rewrite new firmware from scratch without having access to the device schematics (and even then). Worse, just about all these devices have some access to JTAG ports such that you could easily program them yourself if you wanted to and had something useful to flash them with. I've run into so many products I use on a regular basis with bugs or simple but badly needed features that I could have easily fixed/hacked in with access to the source code but alas, I cannot.

deepsun 9y ago
I believe the change should start from consumers, especially big institutions. For example, a big municipality or government agency may impose limits on buying new software/hardware that isn't open to refactor/repair by their competitors, to avoid lock-in to a single supplier, operate on more open market, and consequently drive the costs down.
- pdelbarba 9y ago
  I like the idea. I'm worried that this ship has sailed though as it's becoming harder to even get phones without cryptographically verified bootloaders, and god forbid you want to mess with the baseband/modem source. This is only going to get worse too, as software becomes the part that's actually valuable. AI will mean that everything on the processor will be a trade secret x1000 and everything that can possibly be kept from the local processor will be hidden away in the cloud, forever dependent on continued server access.
rhizome 9y ago
I personally feel that companies should be required to go one step further and be compelled to release their embedded keys and source code for any product that they no longer provide complete support for. Sounds great, but it's something the abandonware[1] scene has been fighting for over the past 20 years, to no avail. 1. https://en.wikipedia.org/wiki/Abandonware https://en.wikipedia.org/wiki/Abandonware
- Slansitartop 9y ago
  It's a good idea, so people should keep fighting for it, even if it takes more than 20 years.
- pdelbarba 9y ago
  I think the difference here is that when it comes to embedded software, very expensive capital can be tied up, as is the case in the above video. Not many commercially available software packages cost >$10k and very few people are exposed to that risk. Meanwhile, there are hundreds of millions of cars in the US.
- archgoon 9y ago
  It'll probably take one generation to raise the issue (and find an example for enough people to care), another to debate, and finally a third to agree about the common sense solution. So give it another 40 years.
aplorbust 9y ago
"Just about everything you own running embedded software that has some update functionality..." "Worse, just about all these devices have some access to JTAG ports such that you could easily program them yourself if you wanted to and had something useful to flash them with. I've run into so many products I use on a regular basis with bugs or simple but badly needed features that I could have easily fixed/hacked in with access to the source code but alas, I cannot." IMO, it is a massive waste. Short-sighted, short-term thinking. There are people who for whatever reason do not want you to have access to the software ("firmware") and to read/fix/improve upon it. Some of these people comment on HN. They become active during debates over whether users would derive any benefit from being able to read and edit source code.
- dpwm 9y ago
  The problem seems to be that software is viewed as a trade secret. Big old companies have CEOs and boards that see embedded software in the same way many legislators and judges see it. Firstly they don't really understand it. But in any case it's part of the product and people have no reason to tinker with it. Now software is being used to enforce what manufacturers could previously have only dreamed of: monopolizing repairs. With hindsight it seems obvious that things converged onto the embedded computers of consumer products: think microwaves and TVs rather than general purpose computers. The problem with tractors is that all the farmers I know have a much better idea of how a tractor works and how to fix it than the average motorist does about their vehicle. I am reminded of the era when windscreen-mounted GPS navigation systems started becoming mainstream. Most of the models had very similar hardware but the software was all that differed. I never looked into it but I suspect almost zero had source code published. To the more typical consumer, the unit is as a whole and the software comes with it. If it routinely takes you the wrong way down a one way street or tries to take you down a set of steps, it's defective and needs to be replaced. To the more technical consumer it's clear that either the software or the data is at fault and either could be fixed with the right resources. I still feel a sadness at this, yet this waste is insignificant compared to a tractor or any other vehicle.
  - carlmr 9y ago
grkvlt 9y ago
> embedded field [...] companies should be [...] compelled to release their embedded keys and source code awesome! my botnet malware that exploits and roots your smart meter or set top box can now install new firmware with a permanent backdoor and nothing will complain because all the malware binaries are signed - by the manufacturer's key, no less - so they must be secure, right?
- M_Bakhtiari 9y ago
  It would be beyond retarded to use the exact same key for every unit. That's not what people are asking for. People are asking for keys to their exact units so they and only they can sign software for them.
  - evancox100 9y ago
    While GP's snark is uncalled for, he is correct as that's exactly how this works. What you're asking for is like saying that a web server would provide a unique HTTPS cert to every distinct visitor.
    - M_Bakhtiari 9y ago
      >What you're asking for is like saying that a web server would provide a unique HTTPS cert to every distinct visitor. It's not similar at all. The equivalent would not be the certificate but the session key, and you do get your own session key in order to prevent what the person above is describing. My HN session key is useless for decrypting your HN password even if I could intercept the traffic. There is no technical reason why the devices can't ship with not only the manufacturer's public key, but also a key pair generated for each unit that comes off the assembly line for the customers to use to sign their own firmware images (and if they wish, delete the manufacturer's public key). But they will never be able to sign images for any device but their own because they simply don't have any signing keys that can produce a signature that will be accepted by any device but their own.
jabl 9y ago
Not to defend Deere et al., but one argument I've seen bandied about is that the manufacturers consider their software to be their "secret sauce" value add. If machined steel were all there is, there's nothing preventing the Chinese (or whoever happens to be the current outsourcing bogeyman) from producing perfectly good equipment at a fraction of the price. Personally I'm not convinced this is a really sustainable "moat"; surely the Chinese (boo!) can do embedded programming as well.
- squarefoot 9y ago
  That's why a lot of people are advocating for a fair law that forces companies to release sources after a reasonable amount of time or when they cease to support the product, so that they can profit for some years from their IP but users aren't screwed when the product becomes obsolete. And of course being repairable and/or upgradeable means that a piece of technology won't be thrown away.
M_Bakhtiari 9y ago
>I personally feel that companies should be required to go one step further and be compelled to release their embedded keys and source code for any product that they no longer provide complete support for. The embedded team where I work struggles with the same thing. Frankly this should be included with the first delivery of any industrial system. Under NDA if they absolutely must, though I don't like it. I also don't like consumer systems being locked up, but at least it's somewhat defensible. But treating farmers as dumb, inbred "hold my beer and watch this firmware hack" hillbillies and not the educated industry professionals that they are is beyond condescending, disrespectful and is completely indefensible. Exactly who is deemed qualified to hack farm equipment firmware is something for the farming trade associations and their insurance companies to work out, but as far as John Deere and the regulators are concerned, we're talking about professional embedded developers writing code to run on their equipment to support an industrial process, something that is completely uncontroversial in many other fields. That you're not supposed to intentionally or inadvertently convert your combine into a man-eating killbot is surely already covered by insurance policy clauses, labour laws and public safety laws.
- mikepurvis 9y ago
  I completely agree with you, but I am a little sympathetic to the "bad PR" angle. It's really, really not that hard to imagine a scenario where a well-meaning farmer (or their contractor) loads something onto the embedded processor that does cause a malfunction, and before you know it there's a splashy news story featuring pictures of a big green tractor not living up to its brand. I work for a robotics company making indoor self-driving vehicles, and we worry about exactly this issue— from an ROI perspective, our product is a slam dunk, so it's safety and reliability are the first and second most significant questions which have to be answered during the sales process. When we invest substantially in associating our brand with those values, we really can't afford the possibility that an unauthorized user modification compromises them in a way that's outside of our control. Hence closed source, FDE, and all the rest of it.
  - squarefoot 9y ago
    That is a very specific scenario, but agreements can be made to release software in exchange of loss of warranty and liability protection: I give you the code and from that moment (either if you use/modify it or not) you lose all warranties and can't sue me even if the product burns your house and kills all your kittens.
gjvc 9y ago
"Nobody is going to try to clone a tractor because they have the source code for it." Thus is the funniest thing I've read in ages. It's also 100% correct and this whole situation shows perhaps clearer than many, how Stallman's warnings were accurate.
- guardian5x 9y ago
  While that is true, i think the manufacturers are more afraid of competitors, which could in some way "clone" at least the software.
  - mrguyorama 9y ago
    I'm doubtful that DRM in any space has ever been about competitors.
_pmf_ 9y ago
> Nobody is going to try to clone a tractor because they have the source code for it. In the automotive industry, people and companies will do this (at the component level). Enabling features via that the OEM has disabled due to its marked segmentation strategy via aftermarket components is done (and would be done to a greater extend if OEMs did not protect the part of their diagnostic stack that handles feature activation). For example, XCP or UDS security access modules are sometimes distributed by the Tier 1 as a DLL so that even the OEM has no source code for the concrete authentication mechanism that is used.

AdieuToLogic 9y ago

A similar situation exists for consumer grade WiFi routers in the US. For about a year now, it's well-nigh impossible to flash routers with DD-WRT, OpenWRT, and friends due to the new FCC conditions manufacturers paid^H^H^H^H lobbied to get put in place.

vuln 9y ago
You've peaked my interest. Has then been new legislation that has enabled router manufacturers lock down their hardware to a point an end-user cannot change it? All three organizations you name are open source and community driven. They cannot possibly write or port their firmware to every platform.
- plasticchris 9y ago
  Parent likely refers to the 5ghz dfs rules, see https://arstechnica.com/information-technology/2015/09/fcc-open-source-router-software-is-still-legal-under-certain-conditions/ https://arstechnica.com/information-technology/2015/09/fcc-o...
  - AdieuToLogic 9y ago
    I wish that were the case, but alas it is not. The article you link was unfortunately an optimistic interpretation of what not yet known to come[0]. 0 - https://www.wired.com/2016/03/way-go-fcc-now-manufacturers-locking-routers/ https://www.wired.com/2016/03/way-go-fcc-now-manufacturers-l...
- ynezz 9y ago
  http://blog.true.cz/2017/02/free-your-router-again/#background http://blog.true.cz/2017/02/free-your-router-again/#backgrou...
- wmf 9y ago
  A history of the FCC router lockdown thing: https://arstechnica.com/information-technology/2016/08/fcc-forces-tp-link-to-support-open-source-firmware-on-routers/ https://arstechnica.com/information-technology/2016/08/fcc-f...

akshayB 9y ago

This practice is pretty much rampant across all industries ranging from cellphones to luxury cars. One simple answer is big companies just want to extend their profits. Another reasons is they want their distributors or dealerships or middleman to thrive as well without which their business may get stagnant.

nas 9y ago
This is a good point. In the ag business, the dealers of equipment make little to no money actually selling the equipment. They make their profit on selling parts and doing servicing. So, keeping the necessary software tools locked to authorized dealers would be popular with the dealerships. It ensures you have to go to the dealer and not some 3rd party mechanic shop.
kevin_b_er 9y ago
Because copyright and software can be used to perform a backdoor attack to seize ownership of physical objects from you. You are looking at a full-on attack on the concept of ownership. You own nothing: It is licensed to you at the whims of the true owners. Why wouldn't corporations, which are sociopathic "persons" defined exclusively by their avarice not want to do so? It is more profitable to prevent repair. It more profitable to force you into buying more.

igor47 9y ago

I'm "excited" about the day that I'll be forced to choose between getting a life-saving medical device implanted, or instead throwing a hissy fit about getting the source code to something that's going in my body. I hope we can figure this shit out with tractors instead...

anotherevan 9y ago
That's already happening. https://youtu.be/5XDTQLa3NjE https://youtu.be/5XDTQLa3NjE https://youtu.be/8wPAHu_zYDw https://youtu.be/8wPAHu_zYDw

LeonM 9y ago

As a software engineer with an embedded systems background this kind of OEM behavior driver me up the wall. I've had OEM's: - refusing to provide interface specs of a device of which we just ordered hundreds of units. (But hey, here is the binary windows (!) driver for this embedded device) - refusing to provide the calculation method of a checksum value their device returned. - Stopping software development (tooling, firmware, drivers) on embedded products, even when there are known issues. - Dropping support on a device, even when hundreds of them are still being used in the field. It's not just tractors, its everywhere. From a business standpoint, it's just so dangerous to work with hardware vendors who don't offer implementation details or source code. If they drop support or stop existing, you're screwed...

pdelbarba 9y ago
Couldn't agree more. The industry is so stuck in various dogmas that all of this is totally acceptable in the pursuit of securing source code from presumably China and anyone with a screwdriver.
nas 9y ago
As a former farmer (roughly 4000 acres of dryland farming in Canada) and an engineer who also did embedded control system development, I would like to add some of my concerns. For new ag machinery, embedded control systems are absolutely essential for the machine to do its job. The ECU on the engine is generally reliable and have long life. The other stuff, I'm highly dubious of the long-term maintainability of it. On our farm, we ran a lot of older equipment and did a lot of repairs ourselves. For new equipment, that will be near impossible as the electronic control systems are black boxes. Even the cabling is complicated and is generally not documented. After the machine gets a decade or two old, good luck fixing it. I suppose you could just drive the machine in the junk pile. However, when a new combine harvester is costing around 3/4 of a million USD, that seems a bit wasteful. I don't know what the solution is though. The farmers buying new equipment don't care so much, their resale value is not suffering too badly. The manufacturers are looking for ways to sell new equipment and adding more features via electronic controls is a relatively cheap way to enhance the product. Most farmers say they don't want all these new electronics on their machines. When Deere announced their S700 series combines, they had a video bragging about all the new improvements: https://youtu.be/l5xJOoNyLoU https://youtu.be/l5xJOoNyLoU If you study it, you will notice that most of the new features are due to software changes or to electronic control systems. Little of what the machine actually does to harvest the grain has changed (metal parts, etc). This automation features are nice when they work but are a disaster when something goes wrong. The operator will have trouble to determine what the machine is doing (never mind trying to figure out how to fix it). However, even though many farmers say they don't want it, the new machines are selling well. So, maybe you could argue the market is working. To me, it feels like there could be some externalization of costs going on. These new machines are very much less valuable as they get older vs the previous era of farm equipment. I guess the same thing has happened to automobiles. You can take a car from the 1940s and fix it up into perfect condition. If you take a 2017 car loaded with electronics, would you have any hope of fixing it to new condition in 50 years from now?

wmf 9y ago

Is there any farm equipment (from China perhaps) that isn't DRMed? In addition to discussing the responsibilities of vendors, maybe we can also discuss the responsibility of customers to understand what they're buying.

mveety 9y ago
There are no new tractors that aren't DRMed if they don't have electronics. Hell, it's even hard to get service documents. Talking about consumer responsibility is great and all, too, but there is no choice so it doesn't matter.
userbinator 9y ago
from China perhaps In this case their general lack of effort around security (see the whole IoT mess) may be a good thing, as it means any DRM is more hackable. The flipside of that is the firmware may also be buggier, but then again, you're also more likely to be able to fix those bugs due to the hackability. The old Douglas Adams quote comes to mind: "The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at and repair."

SteveGerencser 9y ago

I own a small farm and work in tech. We are struggling with this issue right now as we look at the options for a new tractor vs an older serviceable model. While many of the larger corporate farms are content to buy new equipment every 10 years, those of us at the small end of the scale have begun a push to keep older equipment up and running. My tractor is from 1996, my hay baler from the early 70s and my hay cutter is even older. These all work just fine even if a little slow. The challenge of not being able to work on my equipment currently outweighs the marginal speed gains I would see on a farm our size. The same goes for nearly all of my neighbors. The last "new" tractor in the local area was bought a couple years ago by one of the older guys who was buying the last tractor he will ever own and he thought it would be nice to have a top of the line, new, tractor just once.

anubisresources 9y ago
What do you grow? Hay mainly? Acreage?
- SteveGerencser 9y ago
  We grow hay to feed our beef cattle on about 100 acres +/- a bit. But all around us are plenty of farms growing small farm (under 500 acres) or row crops/beans etc. We were having this very discussion today when picking up some hay to help us get through till spring with a guy that cuts about 500 acres of hay 3 times a year. He just keeps rebuilding his equipment rather than get sucked into the new stuff he can't work on. This is becoming another huge issue that affects smaller farms more than it affects larger corporate type farms. When a piece of equipment goes down it's needed back in service in hours to days, not the potential weeks that hauling it to a repair depot can cause.
  - anubisresources 9y ago
    You weren't kidding about the small farm! The dealers in your area can't do field repairs?
vvanders 9y ago

tyingq 9y ago

This story comes up quite often. It seems an opportunity for someone to launch a farm equipment business that makes it's margin on the initial sale instead of the service. Even if it's just refurbished pre-drm tractors.

bitmapbrother 9y ago

According to the video lawyers from Microsoft and Apple showed up at the legislature to voice their objections to the Right To Repair bill. Interesting that these 2 companies would be so anti-consumer.

dfg0987098x7 9y ago
What's so interesting about it, are you surprised? Apple have been fighting the ability to run your own code on the device since day one. This is part of the business model of the "app store" purchase model and not very surprising. I'm sure even you have heard that Microsoft have been anti-consumer once or twice over the years.

emcrazyone 9y ago

Companies generally don't want you in the software for safety reasons. While true there is intellectual property around the guidance and swath generating algorithms, companies hate lawsuits. Lawsuits tie up resources, can cause stop shipments, generally lost profits, and tarnish the brand. Look at the Chrysler Jeep news... You hack or change software on a vehicle and cause the auto-guidance to run over and kill someone or you circumvent a safety check in software unbeknownst to you due to your software changes and a spinny sharp thing that should have stopped doesn't and hurts or mames you or calibration changes cause engine to to over-heat and burn up and generally the company is held responsible for it. As much time is spent testing software as is writing software. All our testing is there to make sure the software works as expected. Millions are spent on test equipment and millions of hours and man hours are spent making sure software works as expected. I happen to work for a major agricultural company and I'm a lead software architect for their guidance + autonomous vehicles + precision farming embedded devices. We are one of John Deere's major competitors.

Lxr 9y ago
That's a poor argument in my opinion. Should my washing machine have no serviceable parts because I might electrocute myself?
- eksu 9y ago
  OP was talking about tractors software and machines that can run people over and kill them, not washing machines or serviceable parts.
  - jdc 9y ago
    I agree; unlike washing machines, apparently, these tractors are made to be unserviceable.
  - vvanders 9y ago
    Yup, tractors and most farm equipment are fucking dangerous. Even the compact/subcompacts regularly maim/kill people. I can totally understand wanting to minimize every possible new vector.

EvanAnderson 9y ago

I've done contract work for a manufacturer of electronic control systems for excavation and mining equipment. They've aggressively implemented DRM in their new products to support renting time-limited and geofenced access to optional features. Their revenue models for the newest generation of equipment are based in large part on the recurring revenue stream enabled by DRM.

drunkencarolina 9y ago

A low tech alternative, depending on one's needs could be draft horses. My wife's family worked teams of belgians for about 80% of their 400-acre farm. https://www.motherearthnews.com/homesteading-and-livestock/farming-with-horses-zmaz87jazgoe https://www.motherearthnews.com/homesteading-and-livestock/f...

ktta 9y ago

This is only going to get worse. A company called Blue River Technologies, which hopes to automate weeding was acquired by John Deere(the company being discussed in the article). Their software is obviously more complex than simple firmware which can be 'hacked' by the average joe. If this weeding systems is profitable and becomes popular, these farmers are screwed. Here's[1] a nice longread on that company. It's a bit fluffy, but I recommend it. [1]: https://www.bloomberg.com/news/features/2018-01-11/this-army-of-ai-robots-will-feed-the-world https://www.bloomberg.com/news/features/2018-01-11/this-army...

jaclaz 9y ago

A similar/related thread: https://news.ycombinator.com/item?id=14074894 https://news.ycombinator.com/item?id=14074894 Risking to cite myself: https://news.ycombinator.com/item?id=14077327 https://news.ycombinator.com/item?id=14077327

tomohawk 9y ago

What if there was a law that any device containing embedded software would have to either be (a) open source, with source code and docs freely available, or (b) closed source, with source code and docs in escrow in the event that the manufacturer ceases support, at which time the code and docs would immediately become open source?

InitialLastName 9y ago

I'm involved in the design of embedded systems for a relatively consumer market, and I've looked into making my company's designs more accessible to modification/software manipulation. There are a few (sometimes major) costs to allowing the modification of software that people tend to ignore in this discussion: - Support: If you want to make your product more accessible to repair, you need to provide support for that. This costs developer time (making the code nice enough to be externally visible, maybe putting together an SDK, making sure any actual secret sauce or anti-counterfeiting systems are blocked off), customer support time (you WILL get calls when somebody buys a used system with modified firmware that doesn't work), and increases the hardware costs (want to provide a JTAG port? that's PCB space and BOM cost). - Brand reflection: We've all seen how much of a ripple the wrong person having a bad experience with a product can have on the market. Say somebody buys a modified Initech Widget (tm) that has a bug or burns some other expensive equipment that it has to work with. That person complains on their friendly local social media network that Initech Widgets have a habit of (say) turning off furnaces in the middle of winter so the user's pipes freeze. Suddenly Initech is dealing with a media storm because they allowed modification of their software, which brings us to... - Liability. What happens if somebody modifies the software in my product and somebody dies, burns down their house, or (* forbid) makes it possible to cause harmful interference and the FCC finds out? How do I prove to that user's (or the FCC's) lawyers that I shouldn't have to pay the damages? Let's assume that, as in the case of tractors or mobile baseband modems (which seem to get brought up a lot in this case) that software is a FUNDAMENTAL part of ensuring the safety/compliance of the unit. Every time I've been through that calculus, the answer at the end has been, "I should do everything in my power to prevent the firmware of my devices from being modified". I can understand and sympathize with the arguments in the other direction, and as a consumer I'm both qualified and interested in modifying the software on my widgets, but it would take a lot of change in the way our society works before most companies will put themselves at risk by supporting or even allowing it for their products.