4 ms·

I worked at a major third party logistics company from 2012-2015 and they were less than halfway through a massive project to rewrite their COBOL mainframe syst

by kyllo 7y ago

I worked at a major third party logistics company from 2012-2015 and they were less than halfway through a massive project to rewrite their COBOL mainframe systems in Java, and I doubt they're anywhere near being done yet. The industry also runs on FTP transfers of flatfiles and ANSI X12 EDI messages. The most "modern" partner API I saw was SOAP. No one I talked to had even heard of REST. Meanwhile, on multiple occasions I heard senior managers express terror at the prospect of Amazon entering their market and competing with them. Rightfully so, given the size of Amazon's resources. Making money off a logistics network is an economy of scale game.

duxup 7y ago
>Making money off a logistics network is an economy of scale game. And yet there are so many small niche providers.... you'd think that would have gone away long ago... Such a weird world.
- kyllo 7y ago
  Yes, but the niche providers flourish when the market is good, but they're the first to get killed off when it's bad. The post-2008 economic collapse saw mass consolidation in the industry as smaller players either went bankrupt or got bought up by competitors. There's also a big difference between asset-owning and non-asset-owning carriers. You can start a logistics company with very little capital--just licensing, bonding, and insurance. In that case, you're basically a travel agent for freight. Some of these agents have grown into very large companies, however, because they help big customers allocate freight across many asset-owning carriers and mitigate their risks, and they handle dealing with international customs and compliance regimes. They're like an outsourced supply chain operations department. In air and ocean, the niche asset-owning carriers generally have some statutory protection. A good example is Jones Act carriers--there's a law called the Jones Act that requires that only American flagged carriers can transport freight between any two American ocean ports. That means there are only a couple of ocean shipping companies that are allowed to go to places like Hawaii, Alaska, Guam and Puerto Rico.
  - sokoloff 7y ago
    They are allowed to go to Puerto Rico (or AK/HI/Guam), but they couldn’t then continue to Miami or Norfolk (or Long Beach) or another US port from there, which makes it uneconomical for them to stop in PR/AK/HI due to the Jones Act.
  - Itaxpica 7y ago
    Interestingly, similar laws come in to play with cruise ships - non-US-flagged passenger ships that depart from a US port aren't allowed to return to a US port without first stopping at a foreign port. Almost all American cruise ships are flagged in countries outside of the US. The practical effect of this is that the many cruises that travel between the west coast and Alaska all have to stop over in Canada before they can return to the west coast, which is a major driver of tourism to Victoria, BC - since that happens to be the largest port city between Alaska and the mainland US.
    - 0xffff2 7y ago
      IANAL, but I don't think it's "similar" laws, but the exact same law. Cruise ships are just (very inefficiently) transporting human cargo.
  - BurningFrog 7y ago
    The Jones Act is such a shit show. Almost no one in the mainland US knows about this Crony Capitalist act than enriches a tiny group of people, but makes life in Puerto Rico, Guam, American Samoa etc quite seriously more impoverished than there is any need for it to be. https://reason.com/tag/jones-act/ https://reason.com/tag/jones-act/
  - repiret 7y ago
    The problem with the Jones Act isn’t so much that the ships have to be US flagged - that’s a natural requirement for a domestic operation. The problem is that it requires them to be US built but the US no longer has a competitive shipbuilding industry.
- cpitman 7y ago
  Yeah, it is really strange. When it comes to trucking, the vast majority of the industry is individuals or very small companies with a handful of trucks. I would have though the benefits of scale would cause mass consolidation, but that isn't the case so far.
  - nevir 7y ago
    There's a lot of inertia in the industry behind the traditional brokerage model (very sales-centric, and all about relationship building). Consolidation is also hard on the trucker side, where there's a ton of fragmentation in the kind of work they perform and access to it
    - petra 7y ago
      Can you please explain your last paragraph?
sonnyblarney 7y ago
Can concur. I urge any technical or product manager to dig into this mess of an industry, it defies everything. The way the markets are set up, the inherent localization, lack of standards. In 2019 tons of 1-3 day short hauls are negotiated over the phone, trucker to shipper, and neither of them may have exact understanding of market rates, especially given variables like the ability for the trucker to return with a partial load. And of course the unbelievable lack of tech. It is truly mystifying. That said, it may very well be that Amazon can conquer it, possibly by imposing regular standards for all sorts of things and efficient clearing market, a common tech platform etc. etc. etc.. But I'm far more confident that Amazon will conquer 1 day delivery than actually fix trucking. All of this may be moot by the time autonomous truckers arrive, and I suggest they might hit before cabs as it's possible 'just driving 2000km on a highway no-stop' might be easier than navigating Manhattan. Autonomous trucking may come with a whole new set of systems and procedures which will probably be more advanced. When I did some research with a friend into this industry, we visited some brokers and dealers I swear I was in a Vince Vaughan comedy.
tcarn 7y ago
I work in the enterprise payments space that functions on ftp transfers of xml, edi and text files and it's not necessarily a bad thing, just as long as it works. Much easier to troubleshoot why you got bad data if you have a ftp to look at with a daily xml file that gets posted.
- microcolonel 7y ago
  Yeah, I've dealt with SFTP + CSV workflows and it's not so bad. I ended up writing a virtual SFTP server which was not backed by a filesystem, and which would prevent malformed data from being written, and make closed/authoritative files immutable. It's obviously not ideal, compared to a well-thought-out purpose-built API and a complete set of tools, but that takes work, and isn't always better faster than the refined hack.
  - cm2187 7y ago
    Wouldn't it be simpler to upload a file with a tmp name and then at the end rename the file if there was no error? Renaming is pretty much an atomic operation.
    - mjevans 7y ago
      It would be, but they might not have the latitude to dictate the connection AND it might be multiple-file batches.
- fragmede 7y ago
  Except that failure modes of ftp (technically not ftp-the-protocol itself, but what the server chooses to do with the file) is not well defined. What happens if the connection dies half way through? Is the partial file processed? None of it? Does the file get moved after the upload is done or after it has been fully processed? Does every single last company in this space's ftp site behave the same in the face of errors? Also is this literally ftp and not anything more recent that includes encryption, but if it includes encryption, what ciphers are supported? Nevermind the files may be in ebdic or something else wonderfully obscure... (I also work in payments. Bank's SFTP sites have under-defined failure modes.)
heavenlyblue 7y ago
>> The most "modern" partner API I saw was SOAP. No one I talked to had even heard of REST. How is REST better than SOAP per se?
- kyllo 7y ago
  The amount of incidental complexity. Reading through their SOAP service spec and coding up a program to interact with it took several times as much effort as coding against a similar REST API would have.
alasdair_ 7y ago
>I worked at a major third party logistics company from 2012-2015 and they were less than halfway through a massive project to rewrite their COBOL mainframe systems in Java, and I doubt they're anywhere near being done yet. Expeditors International perchance?

by kyllo 7y ago

duxup 7y ago

>Making money off a logistics network is an economy of scale game. And yet there are so many small niche providers.... you'd think that would have gone away long ago... Such a weird world.

kyllo 7y ago
Yes, but the niche providers flourish when the market is good, but they're the first to get killed off when it's bad. The post-2008 economic collapse saw mass consolidation in the industry as smaller players either went bankrupt or got bought up by competitors. There's also a big difference between asset-owning and non-asset-owning carriers. You can start a logistics company with very little capital--just licensing, bonding, and insurance. In that case, you're basically a travel agent for freight. Some of these agents have grown into very large companies, however, because they help big customers allocate freight across many asset-owning carriers and mitigate their risks, and they handle dealing with international customs and compliance regimes. They're like an outsourced supply chain operations department. In air and ocean, the niche asset-owning carriers generally have some statutory protection. A good example is Jones Act carriers--there's a law called the Jones Act that requires that only American flagged carriers can transport freight between any two American ocean ports. That means there are only a couple of ocean shipping companies that are allowed to go to places like Hawaii, Alaska, Guam and Puerto Rico.
- sokoloff 7y ago
  They are allowed to go to Puerto Rico (or AK/HI/Guam), but they couldn’t then continue to Miami or Norfolk (or Long Beach) or another US port from there, which makes it uneconomical for them to stop in PR/AK/HI due to the Jones Act.
- Itaxpica 7y ago
  Interestingly, similar laws come in to play with cruise ships - non-US-flagged passenger ships that depart from a US port aren't allowed to return to a US port without first stopping at a foreign port. Almost all American cruise ships are flagged in countries outside of the US. The practical effect of this is that the many cruises that travel between the west coast and Alaska all have to stop over in Canada before they can return to the west coast, which is a major driver of tourism to Victoria, BC - since that happens to be the largest port city between Alaska and the mainland US.
  - 0xffff2 7y ago
    IANAL, but I don't think it's "similar" laws, but the exact same law. Cruise ships are just (very inefficiently) transporting human cargo.
- BurningFrog 7y ago
  The Jones Act is such a shit show. Almost no one in the mainland US knows about this Crony Capitalist act than enriches a tiny group of people, but makes life in Puerto Rico, Guam, American Samoa etc quite seriously more impoverished than there is any need for it to be. https://reason.com/tag/jones-act/ https://reason.com/tag/jones-act/
- repiret 7y ago
  The problem with the Jones Act isn’t so much that the ships have to be US flagged - that’s a natural requirement for a domestic operation. The problem is that it requires them to be US built but the US no longer has a competitive shipbuilding industry.
cpitman 7y ago
Yeah, it is really strange. When it comes to trucking, the vast majority of the industry is individuals or very small companies with a handful of trucks. I would have though the benefits of scale would cause mass consolidation, but that isn't the case so far.
- nevir 7y ago
  There's a lot of inertia in the industry behind the traditional brokerage model (very sales-centric, and all about relationship building). Consolidation is also hard on the trucker side, where there's a ton of fragmentation in the kind of work they perform and access to it
  - petra 7y ago
    Can you please explain your last paragraph?

sonnyblarney 7y ago

Can concur. I urge any technical or product manager to dig into this mess of an industry, it defies everything. The way the markets are set up, the inherent localization, lack of standards. In 2019 tons of 1-3 day short hauls are negotiated over the phone, trucker to shipper, and neither of them may have exact understanding of market rates, especially given variables like the ability for the trucker to return with a partial load. And of course the unbelievable lack of tech. It is truly mystifying. That said, it may very well be that Amazon can conquer it, possibly by imposing regular standards for all sorts of things and efficient clearing market, a common tech platform etc. etc. etc.. But I'm far more confident that Amazon will conquer 1 day delivery than actually fix trucking. All of this may be moot by the time autonomous truckers arrive, and I suggest they might hit before cabs as it's possible 'just driving 2000km on a highway no-stop' might be easier than navigating Manhattan. Autonomous trucking may come with a whole new set of systems and procedures which will probably be more advanced. When I did some research with a friend into this industry, we visited some brokers and dealers I swear I was in a Vince Vaughan comedy.

tcarn 7y ago

I work in the enterprise payments space that functions on ftp transfers of xml, edi and text files and it's not necessarily a bad thing, just as long as it works. Much easier to troubleshoot why you got bad data if you have a ftp to look at with a daily xml file that gets posted.

microcolonel 7y ago
Yeah, I've dealt with SFTP + CSV workflows and it's not so bad. I ended up writing a virtual SFTP server which was not backed by a filesystem, and which would prevent malformed data from being written, and make closed/authoritative files immutable. It's obviously not ideal, compared to a well-thought-out purpose-built API and a complete set of tools, but that takes work, and isn't always better faster than the refined hack.
- cm2187 7y ago
  Wouldn't it be simpler to upload a file with a tmp name and then at the end rename the file if there was no error? Renaming is pretty much an atomic operation.
  - mjevans 7y ago
    It would be, but they might not have the latitude to dictate the connection AND it might be multiple-file batches.
fragmede 7y ago
Except that failure modes of ftp (technically not ftp-the-protocol itself, but what the server chooses to do with the file) is not well defined. What happens if the connection dies half way through? Is the partial file processed? None of it? Does the file get moved after the upload is done or after it has been fully processed? Does every single last company in this space's ftp site behave the same in the face of errors? Also is this literally ftp and not anything more recent that includes encryption, but if it includes encryption, what ciphers are supported? Nevermind the files may be in ebdic or something else wonderfully obscure... (I also work in payments. Bank's SFTP sites have under-defined failure modes.)

heavenlyblue 7y ago

>> The most "modern" partner API I saw was SOAP. No one I talked to had even heard of REST. How is REST better than SOAP per se?

kyllo 7y ago
The amount of incidental complexity. Reading through their SOAP service spec and coding up a program to interact with it took several times as much effort as coding against a similar REST API would have.

alasdair_ 7y ago

>I worked at a major third party logistics company from 2012-2015 and they were less than halfway through a massive project to rewrite their COBOL mainframe systems in Java, and I doubt they're anywhere near being done yet. Expeditors International perchance?