chaokunyang

8 ms·

91.
by chaokunyang 2y ago
For implemetation details, https://github.com/apache/incubator-fury/blob/main/java/fury... can be taken as an example
92.
by chaokunyang 2y ago
For spec details, please see fury meta string spec: https://fury.apache.org/docs/specification/fury_xlang_serial...
93.
by chaokunyang 2y ago
In rpc/serialization systems, we often need to send namespace/path/filename/fieldName/packageName/moduleName/className/enumValue string between processes. Those strings are mostly ascii strings. In or
94.
Meta String: A more space-efficient string encoding than UTF-8 in Fury(fury.apache.org)
75 points by chaokunyang 2y ago|62 comments
95.
by chaokunyang 2y ago
See https://fury.apache.org/blog/fury_0_5_0_release for more details about release notes
96.
by chaokunyang 2y ago
We're excited to announce the release of Fury v0.5.0. This release incorporates a myriad of improvements, bug fixes, and new features across multiple languages including Java, Golang, Python and JavaScript. It further refines Fury&#x27
97.
Apache Fury – fast serialization framework – 0.5.0 released(github.com)
2 points by chaokunyang 2y ago|2 comments
98.
by chaokunyang 3y ago
No reflection needed even for native binary, generate code at graalvm native image build time. 50x faster than native jdk serializtion
99.
Fast Cloud Native Java Serialization:Fury JIT and GraalVM Native Image AOT(github.com)
1 points by chaokunyang 3y ago|2 comments
100.
by chaokunyang 3y ago
Graal Native Image is a technology to compile Java code ahead-of-time to a lightweight native binary executable, which has the advantages of smaller resources, starts in milliseconds, no warmup needed, and reduce attack surface. The seriali
101.
Fury Serialization 0.3.1 Released: support Python 3.11&12(github.com)
1 points by chaokunyang 3y ago|0 comments
102.
by chaokunyang 3y ago
See https://github.com/alipay/fury/blob/main/docs/guide/scala_gu... for scala serialization user doc
103.
by chaokunyang 3y ago
For a long time, scala serialization is always tricky, only fully-reliable way is to use JDK serialization. But it's too slow, and the serialized size is too large. Although we have frameworks such as twill, but the support for scala t
104.
by chaokunyang 3y ago
Fury is a blazing fast multi-language serialization framework powered by jit and zero-copy. We just released 0.3.0 version: https://github.com/alipay/fury/releases/tag/v0.3.0 With this new release, we su
105.
Fury Scala: Fast binary serialization framework by JIT for any Scala 2/3 objects(github.com)
1 points by chaokunyang 3y ago|3 comments
106.
by chaokunyang 3y ago
Thank you so much for your response and valuable insights! The suggestions about providing as much context for issues, and help people with their PRs are very useful. I will take it across the whole open-sourcing. The suggestions about shar
107.
by chaokunyang 3y ago
Another issue about json is that json is not storage efficient. For a float 0.3333333333333335, json needs 18 bytes, but fury needs only 4 bytes.
108.
by chaokunyang 3y ago
Thanks for your kind words. I also want to find more people to join us tro maintain the library. But I still don't know how to involve more people to join the fury community.
109.
by chaokunyang 3y ago
Another issue for json format is that it will bloat the payload. For example, for an int `1111122222` will take 10 bytes in json, but only 4 bytes in fury. `0.3333333333333333` will need 18 bytes in json, but only 4 bytes in fury.
110.
by chaokunyang 3y ago
Good questions. 1)Fury support versions. 2) The schema will be included in the data, and will be used for deserializing/validating the data. The schema will be sent to peer once for a tpc connection.
111.
by chaokunyang 3y ago
No, I just mean it's not easy to use. When using fury, you just need one line code for serialization: `fury.serialize(javaObject)`. When using SBE, it need to much coding, it needs define schema xml, compile code, and so on.
112.
by chaokunyang 3y ago
Not exactly. Protobuf has compression for string/numbers. Fury use different compression protocol. Another thing is that Protobuf generated code are not efficient enough. Fury generated the code at runtime, which has more information a
113.
by chaokunyang 3y ago
For python, the speedup won't be so huge, we get a 50%~100% speed up for dataclasses. We haven't put much efforts on it. In future I believe the speed up will be larger.
114.
by chaokunyang 3y ago
We used https://github.com/janino-compiler/janino to compile the generated code at runtime It's stable and the compiler used by spark/flink. Janino can generated the bytecode for fury generated java code. I m
115.
by chaokunyang 3y ago
Looking forward to your library
116.
by chaokunyang 3y ago
Good point, I used `lambda metafactory` too in fury, but only for some `constructor/getters`. I implements some Function utils to generate lambda function, see https://github.com/alipay/fury/blob/main&#x2
117.
by chaokunyang 3y ago
Yes, it should be an alternative. In fury, we disabled it by default, and all those benchmarks doesn't enable it. Most objects are value bounded, not binary data bounded. Zero-copy won't have a big speed up too. But if the objects
118.
by chaokunyang 3y ago
Yeah, you are right. Fury supports polymorphism and circular/shared references, which are not supported by json/jackson. I did a benchmark with jackson before, here are the results: 1) Fury is 41.6x faster than jackson for Struct
119.
by chaokunyang 3y ago
Json doesn't support polymorphism and circular/shared references, those scenarios are more suitable for fury. And json is a text protocol, which is inefficient compared to fury binary protocol. Taking jackson json lib as an exampl
120.
by chaokunyang 3y ago
Good question! Part of code are generated using `MethodHandle` such as some field getters, or constructor. But the overall code are generated as source code first at runtime instead of bytecode. This is because binary protocol are very comp

More ›