4 ms·

That works until you can't pay your bills unless you take a new contract where you have to deal with a large amount of historical text files from various source

• cheeze 2y ago
  Even then, at that point I'm writing an adapter to convert into UTF-8 JIT in front of my service. Or modifying my program when that contract comes up, but I'm not going to waste time proactively safeguarding against a what-if that may come up at some point.
  • fl7305 2y ago
    > I'm not going to waste time proactively safeguarding against a what-if The article doesn't say that you should. It clearly states that for many cases, the input format is known or explicitly stated in the input headers. The article talks about cases where the input files are in an unknown input format. Even then, it states: "Perhaps there is a case to be made that csvbase's auto-detection should be a more explicit user interface action than a pre-selected combo-box." But for the case where the requirements call for heuristics, the article then talks about how that can be done. > at that point I'm writing an adapter to convert into UTF-8 JIT in front of my service. And at that point you might need the advice in the article, right?
    • hot_gril 2y ago
      The article is overly broad, and if the point is that not every single system ever uses UTF-8... well of course.
  • kelnos 2y ago
    > at that point I'm writing an adapter to convert into UTF-8 JIT in front of my service Right, and at that point you're probably gonna need these statistics-based heuristics to write that adapter. Unless you know specifically what other encoding each bit of input is. If you do, then, again, you are not the target audience for this article.
• kazinator 2y ago
  If you have to deal with a large number of historic text files that are not in UTF-8, why, that's a chance to promote UTF-8!
• ahi 2y ago
  > a large amount of historical text files from various sources Sounds like academia. "You" can't pay your bills anyway.