4 ms·

Stability seems like an orthogonal issue to the standard string representation being "quite possibly the least efficient (non-contrived) representation of text

Stability seems like an orthogonal issue to the standard string representation being "quite possibly the least efficient (non-contrived) representation of text data possible". As a production Haskell user, what do you do when you have to load a large amount of text data?

tome 10y ago
> As a production Haskell user, what do you do when you have to load a large amount of text data? Use Text https://hackage.haskell.org/package/text https://hackage.haskell.org/package/text "The Text type represents Unicode character strings, in a time and space-efficient manner. This package provides text processing capabilities that are optimized for performance critical use, both in terms of large data quantities and high speed."
- StefanKarpinski 10y ago
  Isn't "hackage" the not-fully-vetted / unstable namespace of packages? Is it recommended to use hackage packages in production systems?
  - dack 10y ago
    Hackage is like npm. Anyone can put something up there, but people expect them to be depended on for production stuff. There is another thing called "stackage" - it's a curated set of packages from hackage that are known to work together (due to being run in a continuous build environment), but not all packages are on there.
  - hyperpape 10y ago
    No, hackage is just one place where documentation lives, for both base and non-base things. For instance, very much base things have docs there. https://hackage.haskell.org/package/base-4.9.0.0/docs/Data-List.html https://hackage.haskell.org/package/base-4.9.0.0/docs/Data-L.... Clarity about what lives where is one area where the Haskell ecosystem is a bit deficient, in my opinion. Documentation in general doesn't seem to be great.
  - Chris_Newton 10y ago
    In the case of the text package, it’s a well-respected de facto standard regardless of its official status, and probably as safe a dependency as you’re ever going to get in practice. Anecdotally, everything serious I write in Haskell uses Data.Text by default these days, and just converts to and from other representations like String as necessary. It’s mildly inconvenient for things like writing Show instances or integrating with a library that uses a different convention, but still better in almost any context than using String by default IME.
Buttons840 10y ago
Use a string type that is appropriate. Like Text or ByteString. You have to be aware of the issue, but you usually don't have to do any extra work because there are, for instance, already functions that load a large amount of text into an efficient representation. You just have to know to use them.

tome 10y ago

> As a production Haskell user, what do you do when you have to load a large amount of text data? Use Text https://hackage.haskell.org/package/text https://hackage.haskell.org/package/text "The Text type represents Unicode character strings, in a time and space-efficient manner. This package provides text processing capabilities that are optimized for performance critical use, both in terms of large data quantities and high speed."

StefanKarpinski 10y ago
Isn't "hackage" the not-fully-vetted / unstable namespace of packages? Is it recommended to use hackage packages in production systems?
- dack 10y ago
  Hackage is like npm. Anyone can put something up there, but people expect them to be depended on for production stuff. There is another thing called "stackage" - it's a curated set of packages from hackage that are known to work together (due to being run in a continuous build environment), but not all packages are on there.
- hyperpape 10y ago
  No, hackage is just one place where documentation lives, for both base and non-base things. For instance, very much base things have docs there. https://hackage.haskell.org/package/base-4.9.0.0/docs/Data-List.html https://hackage.haskell.org/package/base-4.9.0.0/docs/Data-L.... Clarity about what lives where is one area where the Haskell ecosystem is a bit deficient, in my opinion. Documentation in general doesn't seem to be great.
- Chris_Newton 10y ago
  In the case of the text package, it’s a well-respected de facto standard regardless of its official status, and probably as safe a dependency as you’re ever going to get in practice. Anecdotally, everything serious I write in Haskell uses Data.Text by default these days, and just converts to and from other representations like String as necessary. It’s mildly inconvenient for things like writing Show instances or integrating with a library that uses a different convention, but still better in almost any context than using String by default IME.

Buttons840 10y ago

Use a string type that is appropriate. Like Text or ByteString. You have to be aware of the issue, but you usually don't have to do any extra work because there are, for instance, already functions that load a large amount of text into an efficient representation. You just have to know to use them.