5 ms·

I tried writing a bunch of F# once before, but one major issue that came up was working with JSON. There's a lot of good libraries for serializing/deserializing

I tried writing a bunch of F# once before, but one major issue that came up was working with JSON. There's a lot of good libraries for serializing/deserializing, but sometimes you need to do something like injecting a value into a an object, and that's just not easy or type safe to do. How hard is working with JSON in Haskell? Any particular libraries/extensions that make it easy? Another question that came to mind: how good is the ecosystem for Haskell? Eg, if you wanted to deal with, say, generating PDFs, would you be more or less confident that you could get something working over a weekend? E: Completely forgot to mention, sumi.news looks great!

nine_k 5y ago
> injecting a value into an object That's something lenses look purpose-built to do. Did they fail you, too?
- square_usual 5y ago
  The thing is, I needed a very specific kind of output for the JSON request I needed to make, and from messing around with lenses I couldn't get it work out. I don't remember what the exact usecase was, but for a hypothetical example, consider an awful POST endpoint that needs the authorization token to be in the json body. So you would have a request like: { "page": 1, "token": "eyJ..." } But ideally, you'd abstract out the auth stuff and put it in a central location which injects it into all requests that need it, eg. if you're using axios you'd add an interceptor for it. We couldn't find a way to do this without making the request into a Dictionary<string, obj>. It's entirely possible this could work with lenses, but from what I looked at I couldn't make it happen. As an aside, I chuckled at the 'Did they fail you, too?'; it made me wonder how lenses could've failed you.
  - solomonb 5y ago
    Lenses are definitely not necessary to do what you want here. In haskell, you typically keep the JSON data to the very periphery of your program. You would probably have some Request with a ToJSON instance and Token types such as: data Request = Request { url :: URI , method :: Method , headers :: [Header] , body :: ByteString ... } instance ToJSON Request where toJSON = ... newtype Token = Token ByteString Then you might write a function to add the token header: applyAuthToken :: Token -> Request -> Request applyAuthToken (Token tok) req = req { headers = ("token", tok) : req headers } Or if you really wanted to use a lens you could do: headersL :: Lens' Request [Header] headersL = lens headers (\req hdrs -> req { headers = hdrs }) applyAuthToken :: Token -> Request -> Request applyAuthToken (Token tok) req = over headersL ("token", tok) :) req Then you call applyAuthToken right before serializing your Request type to a JSON ByteString. edit: sorry I see that you wanted the auth token in the body not the headers. Its the exact same concept but you would make the `body` field in `Request` into some other specific type with a ToJSON instance and a slightly different `applyAuthToken` function.
  - lmm 5y ago
    > But ideally, you'd abstract out the auth stuff and put it in a central location which injects it into all requests that need it, eg. if you're using axios you'd add an interceptor for it. We couldn't find a way to do this without making the request into a Dictionary<string, obj>. I do this kind of thing in Scala using Shapeless records sometimes. You can work pretty easily with types like "this case class, plus a field called token of type String", though writing those types down is a pain (they're basically recursive hlists of fields tagged with singleton types for their names). Apparently GHC.Generic is the closest Haskell equivalent?
  - Eugeleo 5y ago
    Not that it matters now, but I’ve recently learned about generic-data-surgery [1], a library that let’s you take a record X and return a record X with an added field Y. Pretty cool stuff, considering it’s all strongly typed. You would need to do the JSON serialisation _after_ the addToken step, but that shouldn’t be a problem. [1]: https://hackage.haskell.org/package/generic-data-surgery-0.3.0.0/docs/Generic-Data-Surgery.html https://hackage.haskell.org/package/generic-data-surgery-0.3...
klysm 5y ago
I think I’m functional languages, or more generally languages that require you to very carefully express your types, that you have to avoid dealing in JSON except at the very edges of your program. If you do truly need dynamic content that can be a real pain but usually there is an okay way to express it - you just have to throw away the structure.
- square_usual 5y ago
  I'm okay with throwing away structure sometimes - it does make sense when, if you're at the point where you need to manipulate the output JSON while being abstract enough to not care about the input, I could see why you'd want to ignore it. But when the language syntax makes that hard or inelegant to do it's kind of a pain. As a point of comparison, ReasonML and ReScript both have great ways to express JS-like objects with types (OCaml, which both are based on, suffers from a slightly more verbose syntax), and TypeScript of course leads the way in making dynamic-ish objects still work. Ideally I'd find a functional language with row polymorphism that also had a great ecosystem, but that's asking for too much.
  - Zababa 5y ago
    > Ideally I'd find a functional language with row polymorphism that also had a great ecosystem, but that's asking for too much. Your best bet here would be Rescript, OCaml or Purescript and leverage the JS ecosystem. There's also http://www.ocamljava.org/ http://www.ocamljava.org/, which seems to be not really active. I'm not sure if I would call any option great though.
solomonb 5y ago
I can't speak for F#, but in Haskell we typically use the Aeson library for working with JSON data. You define a custom type to represent your JSON data and then you write a parser to go from JSON data serialized as `ByteStrings` into your custom type. The parser is implemented in a Typeclass and you get to leverage adhoc polymorphism to map your serialized JSON into whatever type (or complex set of nested types) you need in a given context. Now to your specific question, Aeson comes with a type called `Value` which represents JSON data directly. You can see the definition of `Value` [here](https://hackage.haskell.org/package/aeson-1.5.6.0/docs/src/Data.Aeson.Types.Internal.html#Value https://hackage.haskell.org/package/aeson-1.5.6.0/docs/src/D...). `Value` is not particularly ergonomic but what you can do is parse the serialized JSON into `Value`, inject your additional JSON values into the `Object` HashMap, and then finally parse the `Value` term into whatever custom type you created for your application.
tofukid 5y ago
Thank you for the compliment on sumi.news. Haskell has a mature library for JSON, Aeson. I often use it to interface with APIs. You write your type, then functions to serialize/deserialize. Records using simple types (Text, Int, Maybe, etc.) can automatically derive the serialize/deserialize functions using generics. Funny you mention PDF, because I’m doing that right now in Haskell for a resume builder project. There’s a few libraries for working with PDFs. HPDF (https://hackage.haskell.org/package/HPDF https://hackage.haskell.org/package/HPDF) supports all the basic functionality you’d expect to create PDFs.
alephu5 5y ago
It seems like you're asking a more general question unrelated to JSON. Rather you want to know how to manipulate data without all the ceremony of a strong type system, as you would in JavaScript or python. This is completely valid and is the best choice for simple jobs. If you have a set of key value pairs where all the values are the same, you can use a Map or HashMap. These are both easily serialisable. I'm not sure there's a way of having key-value pairs without a schema and with different types of values. If it has a schema you can use a record of course. If you just want to add a single value, such as pdfUrl for example, you can include this as a nullable (Maybe Text) value in the record. Receive the data without it and return it with it. If you wanted to, you could also use a type parameter to guarantee that a pdfUrl is always included in the response record.
- square_usual 5y ago
  > If you have a set of key value pairs where all the values are the same, you can use a Map or HashMap. These are both easily serialisable. This is ultimately the choice most people suggested back then, but I have two issues with it: 1. It leads to a ton of boilerplate, first in the step that converts the request record to a map, then adding stuff to the map and then serializing it. Instead, consider what just adding a field to what you know is a record is in F#: let newRecord = {| oldRecord with token = token |} This is vey close to the spread syntax of JS, and is super intuitive and elegant to write. 2. You lose type safety: say your JSON object has a bunch of things of different types within it, like a record with a few records or arrays inside. If you want to convert that to a map, you effectively have to make it a Dictionary<string, obj>, or whatever the Haskell equivalent for obj would be (obj is the "base class" in F#: everything is an obj.) That means you completely lose type information from that point on. The closest any functional language gets to what I would consider "ideal" in this scenario is ReScript, which allows you to type row polymorphic open objects and use JS-like syntax to work with them. This is in addition to good ol' records, of course. Reason is pretty good, in this aspect, too. The closest any functional language gets to this
  - lalaithion 5y ago
    Haskell's type-level programming is pretty strong, and so if you're willing to use some language extensions you can get let newRecord = oldRecord & #token .~ token as seen in https://hackage.haskell.org/package/row-types https://hackage.haskell.org/package/row-types The syntax isn't great, admittedly, but if you're willing to write a template haskell macro you could probably get something like let newRecord = [json| oldRecord with token = token |] working.

nine_k 5y ago

> injecting a value into an object That's something lenses look purpose-built to do. Did they fail you, too?

square_usual 5y ago
The thing is, I needed a very specific kind of output for the JSON request I needed to make, and from messing around with lenses I couldn't get it work out. I don't remember what the exact usecase was, but for a hypothetical example, consider an awful POST endpoint that needs the authorization token to be in the json body. So you would have a request like: { "page": 1, "token": "eyJ..." } But ideally, you'd abstract out the auth stuff and put it in a central location which injects it into all requests that need it, eg. if you're using axios you'd add an interceptor for it. We couldn't find a way to do this without making the request into a Dictionary<string, obj>. It's entirely possible this could work with lenses, but from what I looked at I couldn't make it happen. As an aside, I chuckled at the 'Did they fail you, too?'; it made me wonder how lenses could've failed you.
- solomonb 5y ago
  Lenses are definitely not necessary to do what you want here. In haskell, you typically keep the JSON data to the very periphery of your program. You would probably have some Request with a ToJSON instance and Token types such as: data Request = Request { url :: URI , method :: Method , headers :: [Header] , body :: ByteString ... } instance ToJSON Request where toJSON = ... newtype Token = Token ByteString Then you might write a function to add the token header: applyAuthToken :: Token -> Request -> Request applyAuthToken (Token tok) req = req { headers = ("token", tok) : req headers } Or if you really wanted to use a lens you could do: headersL :: Lens' Request [Header] headersL = lens headers (\req hdrs -> req { headers = hdrs }) applyAuthToken :: Token -> Request -> Request applyAuthToken (Token tok) req = over headersL ("token", tok) :) req Then you call applyAuthToken right before serializing your Request type to a JSON ByteString. edit: sorry I see that you wanted the auth token in the body not the headers. Its the exact same concept but you would make the `body` field in `Request` into some other specific type with a ToJSON instance and a slightly different `applyAuthToken` function.
- lmm 5y ago
  > But ideally, you'd abstract out the auth stuff and put it in a central location which injects it into all requests that need it, eg. if you're using axios you'd add an interceptor for it. We couldn't find a way to do this without making the request into a Dictionary<string, obj>. I do this kind of thing in Scala using Shapeless records sometimes. You can work pretty easily with types like "this case class, plus a field called token of type String", though writing those types down is a pain (they're basically recursive hlists of fields tagged with singleton types for their names). Apparently GHC.Generic is the closest Haskell equivalent?
- Eugeleo 5y ago
  Not that it matters now, but I’ve recently learned about generic-data-surgery [1], a library that let’s you take a record X and return a record X with an added field Y. Pretty cool stuff, considering it’s all strongly typed. You would need to do the JSON serialisation _after_ the addToken step, but that shouldn’t be a problem. [1]: https://hackage.haskell.org/package/generic-data-surgery-0.3.0.0/docs/Generic-Data-Surgery.html https://hackage.haskell.org/package/generic-data-surgery-0.3...

klysm 5y ago

I think I’m functional languages, or more generally languages that require you to very carefully express your types, that you have to avoid dealing in JSON except at the very edges of your program. If you do truly need dynamic content that can be a real pain but usually there is an okay way to express it - you just have to throw away the structure.

square_usual 5y ago
I'm okay with throwing away structure sometimes - it does make sense when, if you're at the point where you need to manipulate the output JSON while being abstract enough to not care about the input, I could see why you'd want to ignore it. But when the language syntax makes that hard or inelegant to do it's kind of a pain. As a point of comparison, ReasonML and ReScript both have great ways to express JS-like objects with types (OCaml, which both are based on, suffers from a slightly more verbose syntax), and TypeScript of course leads the way in making dynamic-ish objects still work. Ideally I'd find a functional language with row polymorphism that also had a great ecosystem, but that's asking for too much.
- Zababa 5y ago
  > Ideally I'd find a functional language with row polymorphism that also had a great ecosystem, but that's asking for too much. Your best bet here would be Rescript, OCaml or Purescript and leverage the JS ecosystem. There's also http://www.ocamljava.org/ http://www.ocamljava.org/, which seems to be not really active. I'm not sure if I would call any option great though.

solomonb 5y ago

I can't speak for F#, but in Haskell we typically use the Aeson library for working with JSON data. You define a custom type to represent your JSON data and then you write a parser to go from JSON data serialized as `ByteStrings` into your custom type. The parser is implemented in a Typeclass and you get to leverage adhoc polymorphism to map your serialized JSON into whatever type (or complex set of nested types) you need in a given context. Now to your specific question, Aeson comes with a type called `Value` which represents JSON data directly. You can see the definition of `Value` [here](https://hackage.haskell.org/package/aeson-1.5.6.0/docs/src/Data.Aeson.Types.Internal.html#Value https://hackage.haskell.org/package/aeson-1.5.6.0/docs/src/D...). `Value` is not particularly ergonomic but what you can do is parse the serialized JSON into `Value`, inject your additional JSON values into the `Object` HashMap, and then finally parse the `Value` term into whatever custom type you created for your application.

tofukid 5y ago

Thank you for the compliment on sumi.news. Haskell has a mature library for JSON, Aeson. I often use it to interface with APIs. You write your type, then functions to serialize/deserialize. Records using simple types (Text, Int, Maybe, etc.) can automatically derive the serialize/deserialize functions using generics. Funny you mention PDF, because I’m doing that right now in Haskell for a resume builder project. There’s a few libraries for working with PDFs. HPDF (https://hackage.haskell.org/package/HPDF https://hackage.haskell.org/package/HPDF) supports all the basic functionality you’d expect to create PDFs.

alephu5 5y ago

It seems like you're asking a more general question unrelated to JSON. Rather you want to know how to manipulate data without all the ceremony of a strong type system, as you would in JavaScript or python. This is completely valid and is the best choice for simple jobs. If you have a set of key value pairs where all the values are the same, you can use a Map or HashMap. These are both easily serialisable. I'm not sure there's a way of having key-value pairs without a schema and with different types of values. If it has a schema you can use a record of course. If you just want to add a single value, such as pdfUrl for example, you can include this as a nullable (Maybe Text) value in the record. Receive the data without it and return it with it. If you wanted to, you could also use a type parameter to guarantee that a pdfUrl is always included in the response record.

square_usual 5y ago
> If you have a set of key value pairs where all the values are the same, you can use a Map or HashMap. These are both easily serialisable. This is ultimately the choice most people suggested back then, but I have two issues with it: 1. It leads to a ton of boilerplate, first in the step that converts the request record to a map, then adding stuff to the map and then serializing it. Instead, consider what just adding a field to what you know is a record is in F#: let newRecord = {| oldRecord with token = token |} This is vey close to the spread syntax of JS, and is super intuitive and elegant to write. 2. You lose type safety: say your JSON object has a bunch of things of different types within it, like a record with a few records or arrays inside. If you want to convert that to a map, you effectively have to make it a Dictionary<string, obj>, or whatever the Haskell equivalent for obj would be (obj is the "base class" in F#: everything is an obj.) That means you completely lose type information from that point on. The closest any functional language gets to what I would consider "ideal" in this scenario is ReScript, which allows you to type row polymorphic open objects and use JS-like syntax to work with them. This is in addition to good ol' records, of course. Reason is pretty good, in this aspect, too. The closest any functional language gets to this
- lalaithion 5y ago
  Haskell's type-level programming is pretty strong, and so if you're willing to use some language extensions you can get let newRecord = oldRecord & #token .~ token as seen in https://hackage.haskell.org/package/row-types https://hackage.haskell.org/package/row-types The syntax isn't great, admittedly, but if you're willing to write a template haskell macro you could probably get something like let newRecord = [json| oldRecord with token = token |] working.