4 ms·

Will someone please convince the trucking industry that "communicating" by swapping text files over FTP in an absolutely incomprehinsible, proprietary format is

Will someone please convince the trucking industry that "communicating" by swapping text files over FTP in an absolutely incomprehinsible, proprietary format is simply no way to live? Especially with the recent push by the fed to put electronic logging systems in every truck, this system is absolutely ripe for disruption. Downside is you'll be fighting entrenched companies like IBM for ground.

dchuk 8y ago
I work in the trucking (telematics) industry. I've dealt with a decent amount of ftp file transfers, but most stuff I've seen has been some sort of API based (SOAP, REST, etc). Can you give a few more specific examples? Are you in the industry now? Working at a carrier?
- nulagrithom 8y ago
  I work in intermodal. Whenever one of our larger customers wants to give us a load to move, we use X12 EDI. This is standard throughout at least the intermodal industry, though from what I gather that may also be true of over-the-road as well. An example 204 EDI (Load Tender) looks like this: ISA*01*0000000000*01*0000000000*ZZ*ABCDEFGHIJKLMNO*ZZ*123456789012345*101127*1719*U*00400*000003438*0*P*> GS*SM*4405197800*999999999*20111219*1747*2100*X*004010 ST*204*0001 B2**XXXX**9999955559**PP B2A*04 L11*NONPRIMARY*OK MS3*XXXX*B**M NTE**FROZEN GOODS SET TO -10d F N1*PF*XYZ CORP*9*9995555500000 N3*31875 SOLON RD N4*SOLON*OH*44139 N7**NONE*********FF****5300 S5*1*CL*27800*L*2444*CA*1016*E L11*9999001947*DO L11*9999670098*CR L11*9999001866*DO L11*9999669887*CR G62*69*20111218 N1*SH*XYZ CORP*9*9991555550000 N2*TERMINAL FREEZER N3*5555 TERMINAL RD N4*CLEVELAND*OH*44023 S5*2*PU*3042*L*312*CA*146*E L11*9999001866*DO L11*9999595358*PO L11*9999669887*CR G62*70*20090728 N1*ST*1 EDI SOURCE*93*9990055555 N3*31875 SOLON RD N4*SOLON*OH*44139 OID*9999669887*99999595358**PC*312*L*3042*E*146 L5**FREIGHT G61*IC*FEEDBACK*EM*FEEDBACK@1edisource.com S5*3*CU*24758*L*2132*CA*870*E L11*9999001947*DO L11*9999008881*PO L11*9999670098*CR G62*70*20111218 N1*ST*1 EDI SOURCE*93*9990055555 N3*55555 5TH AVE N4*MAYFIELD*OH*44244 OID*9999670098*999608881**PC*2132*L*24758*E*870 L5**FREIGHT G61*IC*FEEDBACK*EM*FEEDBACK@1edisource.com L3*27800*G*******1016*E*2444*L SE*46*0001 GE*1*2100 IEA*1*000002104 I haven't proper parsed it, but I believe that's going from Cleveland to Mayfield. One of those L11 segments is probably a reference number. There's no MS1 segment so it's likely over the road? Anyway, it's not exactly descriptive or even human readable... A reply accepting a load looks like this: ISA*01*0000000000*01*0000000000*ZZ*ABCDEFGHIJKLMNO*ZZ*123456789012345*101127*1719*U*00400*000003438*0*P*> GS*GF*4405197800*999999999*20111219*1742*000000003*X*004010 ST*990*000000003 B1*XXXX*9999919860*20111218*A N9*CN*9999919860 SE*4*000000003 GE*1*000000003 IEA*1*000000003 These are commonly exchanged as text files over FTP sites. Some of our more forward-thinking, larger customers are considering moving to AS2, which I believe is sent over HTTP vs FTP. A cursory Google search doesn't really turn up any clear examples on AS2, which doesn't exactly comfort me, but at least there's an RFC[0] for it, whereas for the X12 spec you have to pay[1] to see certain parts of it. Not that anyone follows the "spec" anyway. We code special handling for every single one of our customer's EDI transmissions. I wish everything was REST, or at least JSON. That would be 10x easier. Instead we spend weeks going back and forth on silly things like what a 07 means in the ATS segment, or what character to use for line endings (wish I was kidding -- we've been blocked for two months on the line ending character). What's more is with the ELDs in all our trucks, customers are increasingly wanting GPS updates. I'd love to offer them a streaming socket with GPS data -- it's completely feasible considering our ELD backend. Instead everyone is wondering how we can send updates in 15 minutes increments over FTP, especially when these transactions are often batched in 5 minute loops on both ends in the first place. It kills me a little. We could be doing so much more. I can't believe we aren't pushing for real time. I can't believe five to fifteen minute batching loops are acceptable. [0]: http://www.ietf.org/rfc/rfc4130.txt http://www.ietf.org/rfc/rfc4130.txt [1]: http://www.x12.org/x12-work-products/x12-edi-standards.cfm http://www.x12.org/x12-work-products/x12-edi-standards.cfm
  - dhruvkar 8y ago
    I work for a stone distributor and we have issues with trucking, but had never delved into it. This was interesting to read. Donyou have more sources for gaining more insight into how this all works? I'd love to take a crack at parts of this.
    - nulagrithom 8y ago
      If you Google '204 EDI specification' you should find a lot of random specs in PDF out there for various companies. 214 will show you some of the status updates too. Stuff like this: http://www.shipfsp.com/media/pdf/it/EDI_214_X12_V4010.pdf http://www.shipfsp.com/media/pdf/it/EDI_214_X12_V4010.pdf I've found this EDI notepad useful for parsing flat files: https://www.liaison.com/products/integrate/edi-notepad/ https://www.liaison.com/products/integrate/edi-notepad/ And this npm library actually parses EDIs pretty well too: https://www.npmjs.com/package/x12 https://www.npmjs.com/package/x12
  - hboon 8y ago
    Good old EDI. I last worked with it more than 15 years ago. I used to have several thick, bound EDI specs for each message my team was working with on our desks. After a while you get really good at counting the number of separators... For small messages and loads, it's entirely reasonable to work towards near real-time processing. A possible reason why some EDI folks you talk to might be reluctant to think in that direction is because EDI payloads can go upwards to 10s or 100s MB. Yes, text. Instead of trucks, think ship cargo manifests. That could blind-side them.
  - dchuk 8y ago
    Thanks for the reply! That certainly does look like a pain in the ass. Being on the telematics side specifically, we deal less with EDI stuff, but I'm sure down the road I will have to, and I agree that a structured API would be much better. If you don't mind me asking, what ELD are you running? My company specifically builds one of those, amongst the rest of the productivity suite necessary for a driver to do their work.
    - 8y ago
greenleafjacob 8y ago
https://freight.uber.com/ https://freight.uber.com/

dchuk 8y ago

I work in the trucking (telematics) industry. I've dealt with a decent amount of ftp file transfers, but most stuff I've seen has been some sort of API based (SOAP, REST, etc). Can you give a few more specific examples? Are you in the industry now? Working at a carrier?

nulagrithom 8y ago
I work in intermodal. Whenever one of our larger customers wants to give us a load to move, we use X12 EDI. This is standard throughout at least the intermodal industry, though from what I gather that may also be true of over-the-road as well. An example 204 EDI (Load Tender) looks like this: ISA*01*0000000000*01*0000000000*ZZ*ABCDEFGHIJKLMNO*ZZ*123456789012345*101127*1719*U*00400*000003438*0*P*> GS*SM*4405197800*999999999*20111219*1747*2100*X*004010 ST*204*0001 B2**XXXX**9999955559**PP B2A*04 L11*NONPRIMARY*OK MS3*XXXX*B**M NTE**FROZEN GOODS SET TO -10d F N1*PF*XYZ CORP*9*9995555500000 N3*31875 SOLON RD N4*SOLON*OH*44139 N7**NONE*********FF****5300 S5*1*CL*27800*L*2444*CA*1016*E L11*9999001947*DO L11*9999670098*CR L11*9999001866*DO L11*9999669887*CR G62*69*20111218 N1*SH*XYZ CORP*9*9991555550000 N2*TERMINAL FREEZER N3*5555 TERMINAL RD N4*CLEVELAND*OH*44023 S5*2*PU*3042*L*312*CA*146*E L11*9999001866*DO L11*9999595358*PO L11*9999669887*CR G62*70*20090728 N1*ST*1 EDI SOURCE*93*9990055555 N3*31875 SOLON RD N4*SOLON*OH*44139 OID*9999669887*99999595358**PC*312*L*3042*E*146 L5**FREIGHT G61*IC*FEEDBACK*EM*FEEDBACK@1edisource.com S5*3*CU*24758*L*2132*CA*870*E L11*9999001947*DO L11*9999008881*PO L11*9999670098*CR G62*70*20111218 N1*ST*1 EDI SOURCE*93*9990055555 N3*55555 5TH AVE N4*MAYFIELD*OH*44244 OID*9999670098*999608881**PC*2132*L*24758*E*870 L5**FREIGHT G61*IC*FEEDBACK*EM*FEEDBACK@1edisource.com L3*27800*G*******1016*E*2444*L SE*46*0001 GE*1*2100 IEA*1*000002104 I haven't proper parsed it, but I believe that's going from Cleveland to Mayfield. One of those L11 segments is probably a reference number. There's no MS1 segment so it's likely over the road? Anyway, it's not exactly descriptive or even human readable... A reply accepting a load looks like this: ISA*01*0000000000*01*0000000000*ZZ*ABCDEFGHIJKLMNO*ZZ*123456789012345*101127*1719*U*00400*000003438*0*P*> GS*GF*4405197800*999999999*20111219*1742*000000003*X*004010 ST*990*000000003 B1*XXXX*9999919860*20111218*A N9*CN*9999919860 SE*4*000000003 GE*1*000000003 IEA*1*000000003 These are commonly exchanged as text files over FTP sites. Some of our more forward-thinking, larger customers are considering moving to AS2, which I believe is sent over HTTP vs FTP. A cursory Google search doesn't really turn up any clear examples on AS2, which doesn't exactly comfort me, but at least there's an RFC[0] for it, whereas for the X12 spec you have to pay[1] to see certain parts of it. Not that anyone follows the "spec" anyway. We code special handling for every single one of our customer's EDI transmissions. I wish everything was REST, or at least JSON. That would be 10x easier. Instead we spend weeks going back and forth on silly things like what a 07 means in the ATS segment, or what character to use for line endings (wish I was kidding -- we've been blocked for two months on the line ending character). What's more is with the ELDs in all our trucks, customers are increasingly wanting GPS updates. I'd love to offer them a streaming socket with GPS data -- it's completely feasible considering our ELD backend. Instead everyone is wondering how we can send updates in 15 minutes increments over FTP, especially when these transactions are often batched in 5 minute loops on both ends in the first place. It kills me a little. We could be doing so much more. I can't believe we aren't pushing for real time. I can't believe five to fifteen minute batching loops are acceptable. [0]: http://www.ietf.org/rfc/rfc4130.txt http://www.ietf.org/rfc/rfc4130.txt [1]: http://www.x12.org/x12-work-products/x12-edi-standards.cfm http://www.x12.org/x12-work-products/x12-edi-standards.cfm
- dhruvkar 8y ago
  I work for a stone distributor and we have issues with trucking, but had never delved into it. This was interesting to read. Donyou have more sources for gaining more insight into how this all works? I'd love to take a crack at parts of this.
  - nulagrithom 8y ago
    If you Google '204 EDI specification' you should find a lot of random specs in PDF out there for various companies. 214 will show you some of the status updates too. Stuff like this: http://www.shipfsp.com/media/pdf/it/EDI_214_X12_V4010.pdf http://www.shipfsp.com/media/pdf/it/EDI_214_X12_V4010.pdf I've found this EDI notepad useful for parsing flat files: https://www.liaison.com/products/integrate/edi-notepad/ https://www.liaison.com/products/integrate/edi-notepad/ And this npm library actually parses EDIs pretty well too: https://www.npmjs.com/package/x12 https://www.npmjs.com/package/x12
- hboon 8y ago
  Good old EDI. I last worked with it more than 15 years ago. I used to have several thick, bound EDI specs for each message my team was working with on our desks. After a while you get really good at counting the number of separators... For small messages and loads, it's entirely reasonable to work towards near real-time processing. A possible reason why some EDI folks you talk to might be reluctant to think in that direction is because EDI payloads can go upwards to 10s or 100s MB. Yes, text. Instead of trucks, think ship cargo manifests. That could blind-side them.
- dchuk 8y ago
  Thanks for the reply! That certainly does look like a pain in the ass. Being on the telematics side specifically, we deal less with EDI stuff, but I'm sure down the road I will have to, and I agree that a structured API would be much better. If you don't mind me asking, what ELD are you running? My company specifically builds one of those, amongst the rest of the productivity suite necessary for a driver to do their work.
  - 8y ago

greenleafjacob 8y ago

https://freight.uber.com/ https://freight.uber.com/