3 ms·

Every time someone complains about file formats I bring up a similar story. It's hard to believe fixed width files are still being used today. Parsing these kin

Every time someone complains about file formats I bring up a similar story. It's hard to believe fixed width files are still being used today. Parsing these kind of files with 45 million records, each 600+ bytes is a huge pain. I chuckle when I think about the first time I dealt with these files and thought I could open them in an editor...

DaiPlusPlus 9y ago
> It's hard to believe fixed width files are still being used today. Parsing these kind of files with 45 million records, each 600+ bytes is a huge pain. I chuckle when I think about the first time I dealt with these files and thought I could open them in an editor... Depending on the business-application, fixed-width formats are fantastic because they allow for genuine O(1) random-access to records without needing to precompute an index first. Writing parsers for fixed-width formats is also much simpler, for example, because it eliminates ambiguity around null-terminators vs length-prefix in text/string/array data - for resource-constrained environments it's great because you can guarantee you won't need to dynamically-allocate buffers. I feel the only real arguments against fixed-width records are concerned with wasted space - which are mitigated if your system lets you compress them somehow - because they'll typically compress very, very well.
- krylon 9y ago
  > I feel the only real arguments against fixed-width records are concerned with wasted space Sometimes, fields can turn out to be too short. I have received a few letters from business with my last name truncated, because somebody way back when decided eight or nine characters are definitely enough for a surname.
marktangotango 9y ago
Ha! I wrote an editor for fixed length files, it lets you map to a cobol copybook, or create the layout manually. I never released it. Anyone interested?
- wolfgang42 9y ago
  If it works on Windows, I'd love to give it a try! Our ERP system is written in COBOL and I have to deal with fixed-width files all the time. Does it also handle REDEFINES and level 88?
  - marktangotango 9y ago
    Yes it handles redefines and 88s, my emails in my profile, I have a Windows build.

DaiPlusPlus 9y ago

> It's hard to believe fixed width files are still being used today. Parsing these kind of files with 45 million records, each 600+ bytes is a huge pain. I chuckle when I think about the first time I dealt with these files and thought I could open them in an editor... Depending on the business-application, fixed-width formats are fantastic because they allow for genuine O(1) random-access to records without needing to precompute an index first. Writing parsers for fixed-width formats is also much simpler, for example, because it eliminates ambiguity around null-terminators vs length-prefix in text/string/array data - for resource-constrained environments it's great because you can guarantee you won't need to dynamically-allocate buffers. I feel the only real arguments against fixed-width records are concerned with wasted space - which are mitigated if your system lets you compress them somehow - because they'll typically compress very, very well.

krylon 9y ago
> I feel the only real arguments against fixed-width records are concerned with wasted space Sometimes, fields can turn out to be too short. I have received a few letters from business with my last name truncated, because somebody way back when decided eight or nine characters are definitely enough for a surname.

marktangotango 9y ago

Ha! I wrote an editor for fixed length files, it lets you map to a cobol copybook, or create the layout manually. I never released it. Anyone interested?

wolfgang42 9y ago
If it works on Windows, I'd love to give it a try! Our ERP system is written in COBOL and I have to deal with fixed-width files all the time. Does it also handle REDEFINES and level 88?
- marktangotango 9y ago
  Yes it handles redefines and 88s, my emails in my profile, I have a Windows build.