4 ms·

Spot on. I was an early eng and first VP of Product at Flexport. Global logistics is inherently complicated and involves coordinating many disparate parties. T

Spot on. I was an early eng and first VP of Product at Flexport. Global logistics is inherently complicated and involves coordinating many disparate parties. To complete any step in the workflow, you're generally taking in input data from a bunch of different companies, each of which have varying formats and quality of data. A very challenging context if your goal is process automation. The only way to make progress was exactly the way you described. At each step of the workflow, you need to design at least 2 potential resolution pathways: 1. Automated 2. Manual For the manual case, you have to actually build the interfaces for an operator to do the manual work and encode the results of their work as either: 1. Input into the automated step 2. Or, in the same format as the output of the automated case In either case, this is precisely aligned with your "reuinifying divergent paths" framing. In the automated case, you actually may wind up with N different automation pathways for each workflow step. For an example at Flexport: if we needed to ingest some information from an ocean carrier, we often had to build custom processors for each of the big carriers. And if the volume with a given trading partner didn't justify that investment, then it went to the manual case. From the software engineering framing, it's not that different from building a micro-services architecture. You encapsulate complexity and expose standard inputs and outputs. This avoids creating an incomprehensible mess and also allows the work to be subdivided for individual teams to solve. All that said – doing this in practice at a scaling organization is tough. The micro-services framing is hard to explain to people who haven't internalized the message. But yeah, 100% automation is a wild-goose chase. Maybe you eventually get it, maybe not. But you have to start with the assumption that you won't or you never will.

initplus 2y ago
Sounds like a really interesting problem space. I'm curious if you have any comments about how you approached dealing with inconsistencies between information sources? System A says X, system B says Y. I suppose best approach is again just to bail out to manual resolution?
- seanlinehan 2y ago
  In the early days, we bailed out to manual resolution. In the later days, we had enough disparate data sources that we built oracles to choose which of the conflicting data was most likely to be correct. For example, we integrated with a data source that used OCR to scan container numbers as they passed through various way points while they were on trains. The tech wasn't perfect. We frequently got reports from the rail data source that a train was, for example, passing through the middle of the country when we knew with 100% certainty that it was currently in the middle of the pacific ocean on a boat. That spurious data could be safely thrown out on logical grounds. Other cases were not as straightforward!

initplus 2y ago

Sounds like a really interesting problem space. I'm curious if you have any comments about how you approached dealing with inconsistencies between information sources? System A says X, system B says Y. I suppose best approach is again just to bail out to manual resolution?

seanlinehan 2y ago
In the early days, we bailed out to manual resolution. In the later days, we had enough disparate data sources that we built oracles to choose which of the conflicting data was most likely to be correct. For example, we integrated with a data source that used OCR to scan container numbers as they passed through various way points while they were on trains. The tech wasn't perfect. We frequently got reports from the rail data source that a train was, for example, passing through the middle of the country when we knew with 100% certainty that it was currently in the middle of the pacific ocean on a boat. That spurious data could be safely thrown out on logical grounds. Other cases were not as straightforward!