6 ms·

I'm working on a project involving physical manufacturing and fulfillments, and I was honestly shocked how low tech a lot of it is. Like we communicate with on

by skohan 4y ago

I'm working on a project involving physical manufacturing and fulfillments, and I was honestly shocked how low tech a lot of it is. Like we communicate with one of our warehouses by passing XML documents back and fourth over FTP. It seems like the whole thing was probably set up in 1994 and nobody has touched it since. Working with our own operations people, I have the sense that a lot of it is inertia. People in their field had to figure out how to do all of this before the internet existed for example. And now they busy enough getting their job done through these various arcane systems they're not going to slow down for a few months to learn a new way of doing things. I think as software engineers, we have an intuition about how software could solve a lot of these logistical types of problems efficiently, so we assume it already works that way. Probably we would be shocked how much of the world still runs on handshakes, phone calls, and hand-maintained spreadsheets.

trashtester 4y ago
> It seems like the whole thing was probably set up in 1994 and nobody has touched it since. If you write code that doesn't need to be touched over a 28 year period, chances are that code has produced more value (ROI) than 99% of the code that is written every day will ever do. Even 10 years is pretty good. > Like we communicate with one of our warehouses by passing XML documents back and fourth over FTP. Sending XML, CSV or JSON files over sftp is a simple and relatively robust for exchanging data, especially when a latency of minutes to hours is acceptable and the data volumes are non-trivial. > they're not going to slow down for a few months to learn a new way of doing things. New ways of doing things are not always better. Specifically they're not better for all instances. Often, they're radically worse. RESTful API's have many benefits, especially when volumes are low and latency requirements are strict. But if something works perfectly and efficiently, there is often little reason to change it. I've seen a number of cases where some juniorish devs come into a business, identify that some technical system is using an unfashionable stack, started a project to migrate it, and then failed spectacularly. By just focusing on the unfashionable tech involved, many fail to realize all the clever tricks and deep thought that has gone into the original setup, and in many cases they don't even know the old tech well enough to be able to read the original code. Often the orginal setup was the output of a startup full of motivated, clever and sometimes brilliant people who ended up being fairly successful. The team ending up tasked with migrating that system are often overconfident in their skills and superior technical ability compared to the orignal team, but are often staffed by mostly run-of-the-mill developers who are not really of the same callibre as the original startup. Within a few months, they start to realize SOME of the complexity they overlooked the first time, and a couple of years later, most of the new team quit, after realizing they failed. Hopefully after learning a lesson. I've seen instances where MULTIPLE teams have been set up to migrate some legacy platform, each team spending more effort than the original technology cost, and ALL failing.
- skohan 4y ago
  > If you write code that doesn't need to be touched over a 28 year period, chances are that code has produced more value (ROI) than 99% of the code that is written every day will ever do. Even 10 years is pretty good. Just because it's been in use for a long time doesn't mean it's any good. It only means it's good enough that they can keep enough clients to stay in business. And when it comes to logistics, a lot of times the quality of the tech integration is way down the list in terms of decision factors, like price or the ability to handle some special needs you might have. We have plenty of problems with this system, some of which require our ops people to get on the phone and for instance find out why the number of orders they see in weekly reports don't perfectly match the reporting we have through the EDI. > Sending XML, CSV or JSON files over sftp is a simple and relatively robust for exchanging data, especially when a latency of minutes to hours is acceptable and the data volumes are non-trivial. It's fine as a data transfer protocol, but it's terrible as a real-time communication interface. Yes you can use sftp to exchange messages, but if we want to send an order out and give the customer confirmation as soon as possible that it's been confirmed, REST is the right tool for that job and sftp is a square peg that's been trimmed down to fit inside a round hole. Using REST to solve a problem like this is not about being "fashionable", it's simply about using the right tool for the job. We're not talking about GraphQL and protocol buffers, REST is an old and extremely well-tested approach to web services, and has won because it's so well fit to the problem it solves. And there's a huge benefit to standardizing web service interfaces. It means that any 3rd party you work with already knows how to talk to your system, and can just focus on which data has to be passed back and fourth. You don't have to teach your clients how to communicate with you via some arcane in-house protocol which has weird corners you only find out about once you start to work with it, like the system breaks if the documents come in the wrong order or something. By not offering the modern standard, these kinds of businesses make it slower and more expensive for clients to work with them. If I am trying to get a product to market, give me a REST API and I can have warehouse integrated in a week or two. With some proprietary protocol who knows how long it is going to take. This system is also solvable without a rewrite. If your system is so complicated and arcane that it's impossible to re-write it, just wrap it in a REST service. Your REST service can talk to sftp or whatever crazy system you want to come up with. But there is zero benefit to deviating from the standard from the client's perspective.

by skohan 4y ago

trashtester 4y ago

> It seems like the whole thing was probably set up in 1994 and nobody has touched it since. If you write code that doesn't need to be touched over a 28 year period, chances are that code has produced more value (ROI) than 99% of the code that is written every day will ever do. Even 10 years is pretty good. > Like we communicate with one of our warehouses by passing XML documents back and fourth over FTP. Sending XML, CSV or JSON files over sftp is a simple and relatively robust for exchanging data, especially when a latency of minutes to hours is acceptable and the data volumes are non-trivial. > they're not going to slow down for a few months to learn a new way of doing things. New ways of doing things are not always better. Specifically they're not better for all instances. Often, they're radically worse. RESTful API's have many benefits, especially when volumes are low and latency requirements are strict. But if something works perfectly and efficiently, there is often little reason to change it. I've seen a number of cases where some juniorish devs come into a business, identify that some technical system is using an unfashionable stack, started a project to migrate it, and then failed spectacularly. By just focusing on the unfashionable tech involved, many fail to realize all the clever tricks and deep thought that has gone into the original setup, and in many cases they don't even know the old tech well enough to be able to read the original code. Often the orginal setup was the output of a startup full of motivated, clever and sometimes brilliant people who ended up being fairly successful. The team ending up tasked with migrating that system are often overconfident in their skills and superior technical ability compared to the orignal team, but are often staffed by mostly run-of-the-mill developers who are not really of the same callibre as the original startup. Within a few months, they start to realize SOME of the complexity they overlooked the first time, and a couple of years later, most of the new team quit, after realizing they failed. Hopefully after learning a lesson. I've seen instances where MULTIPLE teams have been set up to migrate some legacy platform, each team spending more effort than the original technology cost, and ALL failing.

skohan 4y ago
> If you write code that doesn't need to be touched over a 28 year period, chances are that code has produced more value (ROI) than 99% of the code that is written every day will ever do. Even 10 years is pretty good. Just because it's been in use for a long time doesn't mean it's any good. It only means it's good enough that they can keep enough clients to stay in business. And when it comes to logistics, a lot of times the quality of the tech integration is way down the list in terms of decision factors, like price or the ability to handle some special needs you might have. We have plenty of problems with this system, some of which require our ops people to get on the phone and for instance find out why the number of orders they see in weekly reports don't perfectly match the reporting we have through the EDI. > Sending XML, CSV or JSON files over sftp is a simple and relatively robust for exchanging data, especially when a latency of minutes to hours is acceptable and the data volumes are non-trivial. It's fine as a data transfer protocol, but it's terrible as a real-time communication interface. Yes you can use sftp to exchange messages, but if we want to send an order out and give the customer confirmation as soon as possible that it's been confirmed, REST is the right tool for that job and sftp is a square peg that's been trimmed down to fit inside a round hole. Using REST to solve a problem like this is not about being "fashionable", it's simply about using the right tool for the job. We're not talking about GraphQL and protocol buffers, REST is an old and extremely well-tested approach to web services, and has won because it's so well fit to the problem it solves. And there's a huge benefit to standardizing web service interfaces. It means that any 3rd party you work with already knows how to talk to your system, and can just focus on which data has to be passed back and fourth. You don't have to teach your clients how to communicate with you via some arcane in-house protocol which has weird corners you only find out about once you start to work with it, like the system breaks if the documents come in the wrong order or something. By not offering the modern standard, these kinds of businesses make it slower and more expensive for clients to work with them. If I am trying to get a product to market, give me a REST API and I can have warehouse integrated in a week or two. With some proprietary protocol who knows how long it is going to take. This system is also solvable without a rewrite. If your system is so complicated and arcane that it's impossible to re-write it, just wrap it in a REST service. Your REST service can talk to sftp or whatever crazy system you want to come up with. But there is zero benefit to deviating from the standard from the client's perspective.