4 ms·

Serde really is one of the gems of the Rust ecosystem. It is a (de)serialization framework that can be quite easily implemented for various serialization forma

Serde really is one of the gems of the Rust ecosystem. It is a (de)serialization framework that can be quite easily implemented for various serialization formats like JSON, MessagePack, Yaml, toml, ... It enables automatic and very performant (de)serialization of you data, into different formats. Often with a simple: ` #[derive(Serialize, Deserialize)] struct Data { ... } `

jadbox 10y ago
I really really would love Avro support added... I'd do it if I had more Rust experience. The icing on the cake is if it could also interact with the Avro registry somehow.
- steveklabnik 10y ago
  What's Avro?
  - orthecreedence 10y ago
    Looks like a data serialization format: https://avro.apache.org/ https://avro.apache.org/ (Never heard of it either)
  - MasterIdiot 10y ago
    It allows you to define a schema, and then compile it to various languages and have multiple applications in multiple languages "speak" the same class, down to the binary level (for RPC, blob storage etc).
  - lobster_johnson 10y ago
    It's not as popular as Protocol Buffers, Thrift, MessagePack etc., probably because it started out in Java land (I believe it came from Doug Nutch of Lucene fame). I've never used it, but one benefit of Avro is that it can embed the schema in the serialized output, which allows clients to consume data even though they don't have the IDL, which isn't possible with Protobuf and Thrift. MessagePack does include types and map keys, but doesn't have a schema.
    - spearo77 10y ago
      s/Doug Nutch/Doug Cutting/
      - lobster_johnson 10y ago
        Oops. That should have been "Doug Cutting of Nutch and Lucene fame".
- dtolnay 10y ago
  Avro support, just like all other Serde data formats, lives in its own crate. https://crates.io/crates/serde-avro https://crates.io/crates/serde-avro
- bluejekyll 10y ago
  I keep picking this up to try and help and then dropping it. I wish I had more time in the day. It looks like deserialization is mostly there, but not serialization.
problems 10y ago
Really glad to see some cool libraries coming up for Rust. When I last tried to learn it, I got the basics then sort of just gave up because there didn't seem to be anything cool or unique I could do with it.
vvanders 10y ago
For existing formats Serde is indeed pretty awesome. However when I tried to pick it up I found the lack of support for fixed arrays >32 a show-stopper for me. I just found out recently that there's a couple work-arounds but none of them were obvious and for a framework that is zero-allocation based not having great support for arrays was a non-starter since that's 90% of the structures I was working with. I love Rust but I feel like arrays in general are somewhat of an unfinished part of the language, you run into similar problems with Clone and Debug which can be frustrating.
- steveklabnik 10y ago
  Active work is going into integer generics, which will solve this issue. It has taken a while but isn't exactly simple either!
  - vvanders 10y ago
    Glad to hear it! I guess it's just jarring since the rest of Rust is so well crafted it felt really abrupt to run up against the 32 length fixed size limits. It's also somewhat annoying to have to coerce it into a slice via &array[..] instead of &array. It's workable and when I finally get the library to a stable state I definitely plan to share what went well and where I saw pain points.
    - steveklabnik 10y ago
      Yup, totally hear you. It's totally achievable, but not simple; Haskell has this issue too, for example. There's been a few iterations of the design already. The current approach is to cut scope as much as possible to ship the basics sooner.
      - RcouF1uZ4gsC 10y ago
        It is this lack of non-type template parameters, template templates (ie higher minded types), and variadics that make Rust feel like a step backwards compared to C++. When, Rust gets these, it will remove a a major blocker for me.

jadbox 10y ago

I really really would love Avro support added... I'd do it if I had more Rust experience. The icing on the cake is if it could also interact with the Avro registry somehow.

steveklabnik 10y ago
What's Avro?
- orthecreedence 10y ago
  Looks like a data serialization format: https://avro.apache.org/ https://avro.apache.org/ (Never heard of it either)
- MasterIdiot 10y ago
  It allows you to define a schema, and then compile it to various languages and have multiple applications in multiple languages "speak" the same class, down to the binary level (for RPC, blob storage etc).
- lobster_johnson 10y ago
  It's not as popular as Protocol Buffers, Thrift, MessagePack etc., probably because it started out in Java land (I believe it came from Doug Nutch of Lucene fame). I've never used it, but one benefit of Avro is that it can embed the schema in the serialized output, which allows clients to consume data even though they don't have the IDL, which isn't possible with Protobuf and Thrift. MessagePack does include types and map keys, but doesn't have a schema.
  - spearo77 10y ago
    s/Doug Nutch/Doug Cutting/
    - lobster_johnson 10y ago
      Oops. That should have been "Doug Cutting of Nutch and Lucene fame".
dtolnay 10y ago
Avro support, just like all other Serde data formats, lives in its own crate. https://crates.io/crates/serde-avro https://crates.io/crates/serde-avro
bluejekyll 10y ago
I keep picking this up to try and help and then dropping it. I wish I had more time in the day. It looks like deserialization is mostly there, but not serialization.

problems 10y ago

Really glad to see some cool libraries coming up for Rust. When I last tried to learn it, I got the basics then sort of just gave up because there didn't seem to be anything cool or unique I could do with it.

vvanders 10y ago

For existing formats Serde is indeed pretty awesome. However when I tried to pick it up I found the lack of support for fixed arrays >32 a show-stopper for me. I just found out recently that there's a couple work-arounds but none of them were obvious and for a framework that is zero-allocation based not having great support for arrays was a non-starter since that's 90% of the structures I was working with. I love Rust but I feel like arrays in general are somewhat of an unfinished part of the language, you run into similar problems with Clone and Debug which can be frustrating.

steveklabnik 10y ago
Active work is going into integer generics, which will solve this issue. It has taken a while but isn't exactly simple either!
- vvanders 10y ago
  Glad to hear it! I guess it's just jarring since the rest of Rust is so well crafted it felt really abrupt to run up against the 32 length fixed size limits. It's also somewhat annoying to have to coerce it into a slice via &array[..] instead of &array. It's workable and when I finally get the library to a stable state I definitely plan to share what went well and where I saw pain points.
  - steveklabnik 10y ago
    Yup, totally hear you. It's totally achievable, but not simple; Haskell has this issue too, for example. There's been a few iterations of the design already. The current approach is to cut scope as much as possible to ship the basics sooner.
    - RcouF1uZ4gsC 10y ago
      It is this lack of non-type template parameters, template templates (ie higher minded types), and variadics that make Rust feel like a step backwards compared to C++. When, Rust gets these, it will remove a a major blocker for me.