6 ms·

A Closer Look at Transit

122 points by _halgari 12y ago

simi_ 12y ago
What's with the predilection for unreadable fonts? http://blog.cognitect.com/blog/2014/7/22/transit http://blog.cognitect.com/blog/2014/7/22/transit http://swannodette.github.io/2014/07/23/a-closer-look-at-transit/ http://swannodette.github.io/2014/07/23/a-closer-look-at-tra...
- _halgari 12y ago
  Not sure I understand, it looks perfectly fine on my box.
  - joshuacc 12y ago
    On FF in Windows, the fonts are extremely thin and difficult to read. I had to zoom in Nolen's blog four times to make it readable.
    - derefr 12y ago
      Huh. Here it is on a Mac, FYI: http://i.imgur.com/swOvBka.png http://i.imgur.com/swOvBka.png
  - ejr 12y ago
    The main body of the text was extremely difficult to read. There's nothing wrong with the font itself, but the size is far too small and colour too faint. I had to turn off the font stylesheet to get to the text. It's too bad the same font as the sidebar section wasn't used throughout the body.
  - snogglethorpe 12y ago
    It's "possible to read" for me (FF on Debian), but it's not particularly "readable"... it seems to be using fonts that don't render very well for "traditional resolution" (~100dpi) displays, at an uncomfortably small size (maybe the author tests using a browser that tends to inflate text sizes?).
thesorrow 12y ago
Have you thought about JSON-LD ? Having some semantic web meaning on top of it would be really cool !
- swannodette 12y ago
  Yes comparisons to JSON-LD has come up a few times and semantic web & HATEOAS are the reasons why the Transit spec includes the link type. JSON-LD looks interesting but it does not seem to address the basic performance concerns that Transit does.
  - thesorrow 12y ago
    Thanks for your answer. Can I create extensions for Schema.org types then ?
    - swannodette 12y ago
      I don't see why not - I mean these are unlikely to get included in the official Transit spec, but sure, being able to build a common set of shared extensions seems mighty powerful to me - I had a bit of fun with this today when putting together the example for the blogpost https://github.com/swannodette/transit-js-example/blob/master/shared/handlers.js#L40 https://github.com/swannodette/transit-js-example/blob/maste...
jacobolus 12y ago
Looking at http://jsperf.com/json-vs-transit/2 http://jsperf.com/json-vs-transit/2 ... Why does Transit performance take such a hit in Mobile Safari? (In desktop Safari on my Mac, performance seems to be about the same between JSON+hydrate vs. Transit. But the charts below show that especially on later versions of mobile Safari Transit is quite a bit slower.)
- swannodette 12y ago
  I just checked on my iPhone 5S running iOS 7.1.2 & Mobile Safari 7.0 - the transit-js results look good and now are reflected in the benchmarks. I have no idea what's up with the browsers showing up as Mobile Safari 7.1 & 7.2 (which are dev releases as far as I know).
lbarrett 12y ago
Why do they say JSON is faster than MessagePack and then link to a comparison for only Javascript? Most servers are not running Javascript, and most data goes its entire lifetime untouched by Javascript. There's some reason to believe that MessagePack is noticeably faster than JSON on every language but Javascript. Edit: s/entirely/entire/
- swannodette 12y ago
  No one is saying MessagePack is slower than JSON across the board. In fact for languages that have fast MessagePack implementations, server programs can speak to each other using the Transit MessagePack encoding. But having a good story to and from JavaScript was a big design constraint and Transit delivers this as well.
  - papaf 12y ago
    I thought that the big advantage of MessagePack was that it was smaller and quicker over networks. Network effects are the biggest cause of delay when transferring data to remote machines and are much bigger than serialization performance. The localhost testing done in the article will not show this advantage of MessagePack.
    - swannodette 12y ago
      Transit JSON payloads benefit from standard gzipping techniques for the cases where the latencies are quite large as is the case when communicating with client browsers. For the cases where latencies are not quite so big we have yet to see evidence that Transit JSON suffers dramatically.
- jmtulloss 12y ago
  It is faster for most languages, and could be made faster in server-side JS. However, this article is talking about performance for browser JS, where lack of robust string decoding destroys msgpack performance.
Skinney 12y ago
Hey Swannodette, I've currently got a Clojure backend talking edn with a browser running ClojureScript. Would it make any sense to switch to transit? More specifically, are there any advantages to use transit besides performance, and how big would the improvement in performance potentially be?
- swannodette 12y ago
  For Clojure & ClojureScript users Transit delivers little new in terms of feature set over EDN but the performance gains are quite significant. transit-cljs is up to 18-20X faster than cljs.reader/read-string on the EDN equivalent. Under V8 JS environments (Chrome, Opera, Node.js) I consistently see transit-cljs outperform JSON.parse on the equivalent data. In addition transit-cljs is a great way to consume JSON endpoints from ClojureScript without the massive perf hit of cljs.core/js->clj. You can also trivially spit out JSON from ClojureScript via transit-cljs as long as your map keys are strings and not keywords using a json-verbose Transit writer.
  - Skinney 12y ago
    Nice :)
  - tel 12y ago
    The speed difference is largely a result of caching correct?
    - swannodette 12y ago
      It's a part of it - the full story is more like: caching + array representation + array wrapping + preventing internal cache misses.
phpnode 12y ago
this jsperf revision is slightly more interesting to me http://jsperf.com/json-vs-transit/3 http://jsperf.com/json-vs-transit/3 it's effectively the same input, but optimised for JSON.parse() rather than transit. This means it purely decodes into objects with fast properties (in V8 at least), so objects never have to undergo the -> hash table transition. It seems consistently faster, but you can argue that it's not doing quite the same thing.
- swannodette 12y ago
  These results aren't actually that different when you include all the other browsers. In fact when you look at Chrome 38 (Canary) the results are exactly the same for both versions of the benchmark. As a general rule the optimizations in transit-js take all browsers into consideration - not any particular one.
dcre 12y ago
It would be nice to include the encoded Transit version, just to see what it looks like. Edit: the tour linked at the bottom is basically what I was looking for. http://cognitect.github.io/transit-tour/ http://cognitect.github.io/transit-tour/
jacobolus 12y ago
To what extent are legal EDN and Transit compatible? Are there types that appear in one and not the other? How does the extension mechanism compare? Would it be possible to make a 2-way converter between them which would retain full fidelity of the Transit data when round-tripping, so that we could use EDN for actually inspecting/writing (e.g. config files), but then send Transit over the wire to a browser?
- puredanger 12y ago
  EDN and Transit are different formats but share a common philosophy towards types and extensions (as they originate from the same place). There are likely small differences in what types appear in both. The extension mechanisms are different but of equivalent expressive power. I don't think it would be difficult to convert between EDN and Transit using existing readers and writers for both. The common meeting point is values in your language.
- dgreensp 12y ago
  From the examples, it looks like JSON and EDN are both first-class "transport" formats for Transit: https://github.com/cognitect/transit-format/tree/master/examples/0.8/simple https://github.com/cognitect/transit-format/tree/master/exam...
kbd 12y ago
I haven't looked that closely at Transit yet, but one very small thing that irked me at first glance is that the tag for floats is 'd' and the tag for arbitrary precision decimals is 'f', which seems entirely backwards.
- rubiquity 12y ago
  I think this likely has to do with how confusing the implementation of floats and doubles in programming languages has been over the years. A lot of languages will call a double precision decimal a float, but I don't know of any languages that call a single precision decimal a double. I think the general ambiguity around float is why they chose to use it as any precision float.
latchkey 12y ago
I'm curious how Transit handles cyclic object graphs for optimizing the payload size. Would be nice to see JSOG integration. https://github.com/jsog/jsog https://github.com/jsog/jsog
- puredanger 12y ago
  I don't think Transit does or plans to handle cyclic object graphs.
  - tel 12y ago
    The name generation from caching ought to allow it, though...
noelwelsh 12y ago
I can imagine using Transit as a format to communicate between servers, but I'm less sold on the JS story. If I'm exposing an API endpoint to the browser I'm also exposing it as an API endpoint. In this case I need a really good reason to not use JSON, which everyone understands and can parse. Adding a barrier to people using my API just doesn't make sense. It's fairly standard for single page apps to consume a public API, so I think this will be a fairly common objection. Also, Computer Modern is so so so ugly (IMHO of course).
- swannodette 12y ago
  This is a winner-take-all narrative I simply cannot understand. I recall when all endpoints were XML and JSON.parse was called eval. I can easily imagine a endpoint providing both Transit JSON and plain JSON. I know which one I'd rather consume from the client when building a sophisticated application.
SimeVidas 12y ago
Am I the only one who wants to know how transit data looks like (compared to JSON) over the wire?

122 points by _halgari 12y ago

simi_ 12y ago

What's with the predilection for unreadable fonts? http://blog.cognitect.com/blog/2014/7/22/transit http://blog.cognitect.com/blog/2014/7/22/transit http://swannodette.github.io/2014/07/23/a-closer-look-at-transit/ http://swannodette.github.io/2014/07/23/a-closer-look-at-tra...

_halgari 12y ago
Not sure I understand, it looks perfectly fine on my box.
- joshuacc 12y ago
  On FF in Windows, the fonts are extremely thin and difficult to read. I had to zoom in Nolen's blog four times to make it readable.
  - derefr 12y ago
    Huh. Here it is on a Mac, FYI: http://i.imgur.com/swOvBka.png http://i.imgur.com/swOvBka.png
- ejr 12y ago
  The main body of the text was extremely difficult to read. There's nothing wrong with the font itself, but the size is far too small and colour too faint. I had to turn off the font stylesheet to get to the text. It's too bad the same font as the sidebar section wasn't used throughout the body.
- snogglethorpe 12y ago
  It's "possible to read" for me (FF on Debian), but it's not particularly "readable"... it seems to be using fonts that don't render very well for "traditional resolution" (~100dpi) displays, at an uncomfortably small size (maybe the author tests using a browser that tends to inflate text sizes?).

thesorrow 12y ago

Have you thought about JSON-LD ? Having some semantic web meaning on top of it would be really cool !

swannodette 12y ago
Yes comparisons to JSON-LD has come up a few times and semantic web & HATEOAS are the reasons why the Transit spec includes the link type. JSON-LD looks interesting but it does not seem to address the basic performance concerns that Transit does.
- thesorrow 12y ago
  Thanks for your answer. Can I create extensions for Schema.org types then ?
  - swannodette 12y ago
    I don't see why not - I mean these are unlikely to get included in the official Transit spec, but sure, being able to build a common set of shared extensions seems mighty powerful to me - I had a bit of fun with this today when putting together the example for the blogpost https://github.com/swannodette/transit-js-example/blob/master/shared/handlers.js#L40 https://github.com/swannodette/transit-js-example/blob/maste...

jacobolus 12y ago

Looking at http://jsperf.com/json-vs-transit/2 http://jsperf.com/json-vs-transit/2 ... Why does Transit performance take such a hit in Mobile Safari? (In desktop Safari on my Mac, performance seems to be about the same between JSON+hydrate vs. Transit. But the charts below show that especially on later versions of mobile Safari Transit is quite a bit slower.)

swannodette 12y ago
I just checked on my iPhone 5S running iOS 7.1.2 & Mobile Safari 7.0 - the transit-js results look good and now are reflected in the benchmarks. I have no idea what's up with the browsers showing up as Mobile Safari 7.1 & 7.2 (which are dev releases as far as I know).

lbarrett 12y ago

Why do they say JSON is faster than MessagePack and then link to a comparison for only Javascript? Most servers are not running Javascript, and most data goes its entire lifetime untouched by Javascript. There's some reason to believe that MessagePack is noticeably faster than JSON on every language but Javascript. Edit: s/entirely/entire/

swannodette 12y ago
No one is saying MessagePack is slower than JSON across the board. In fact for languages that have fast MessagePack implementations, server programs can speak to each other using the Transit MessagePack encoding. But having a good story to and from JavaScript was a big design constraint and Transit delivers this as well.
- papaf 12y ago
  I thought that the big advantage of MessagePack was that it was smaller and quicker over networks. Network effects are the biggest cause of delay when transferring data to remote machines and are much bigger than serialization performance. The localhost testing done in the article will not show this advantage of MessagePack.
  - swannodette 12y ago
    Transit JSON payloads benefit from standard gzipping techniques for the cases where the latencies are quite large as is the case when communicating with client browsers. For the cases where latencies are not quite so big we have yet to see evidence that Transit JSON suffers dramatically.
jmtulloss 12y ago
It is faster for most languages, and could be made faster in server-side JS. However, this article is talking about performance for browser JS, where lack of robust string decoding destroys msgpack performance.

Skinney 12y ago

Hey Swannodette, I've currently got a Clojure backend talking edn with a browser running ClojureScript. Would it make any sense to switch to transit? More specifically, are there any advantages to use transit besides performance, and how big would the improvement in performance potentially be?

swannodette 12y ago
For Clojure & ClojureScript users Transit delivers little new in terms of feature set over EDN but the performance gains are quite significant. transit-cljs is up to 18-20X faster than cljs.reader/read-string on the EDN equivalent. Under V8 JS environments (Chrome, Opera, Node.js) I consistently see transit-cljs outperform JSON.parse on the equivalent data. In addition transit-cljs is a great way to consume JSON endpoints from ClojureScript without the massive perf hit of cljs.core/js->clj. You can also trivially spit out JSON from ClojureScript via transit-cljs as long as your map keys are strings and not keywords using a json-verbose Transit writer.
- Skinney 12y ago
  Nice :)
- tel 12y ago
  The speed difference is largely a result of caching correct?
  - swannodette 12y ago
    It's a part of it - the full story is more like: caching + array representation + array wrapping + preventing internal cache misses.

phpnode 12y ago

this jsperf revision is slightly more interesting to me http://jsperf.com/json-vs-transit/3 http://jsperf.com/json-vs-transit/3 it's effectively the same input, but optimised for JSON.parse() rather than transit. This means it purely decodes into objects with fast properties (in V8 at least), so objects never have to undergo the -> hash table transition. It seems consistently faster, but you can argue that it's not doing quite the same thing.

swannodette 12y ago
These results aren't actually that different when you include all the other browsers. In fact when you look at Chrome 38 (Canary) the results are exactly the same for both versions of the benchmark. As a general rule the optimizations in transit-js take all browsers into consideration - not any particular one.

dcre 12y ago

It would be nice to include the encoded Transit version, just to see what it looks like. Edit: the tour linked at the bottom is basically what I was looking for. http://cognitect.github.io/transit-tour/ http://cognitect.github.io/transit-tour/

jacobolus 12y ago

To what extent are legal EDN and Transit compatible? Are there types that appear in one and not the other? How does the extension mechanism compare? Would it be possible to make a 2-way converter between them which would retain full fidelity of the Transit data when round-tripping, so that we could use EDN for actually inspecting/writing (e.g. config files), but then send Transit over the wire to a browser?

puredanger 12y ago
EDN and Transit are different formats but share a common philosophy towards types and extensions (as they originate from the same place). There are likely small differences in what types appear in both. The extension mechanisms are different but of equivalent expressive power. I don't think it would be difficult to convert between EDN and Transit using existing readers and writers for both. The common meeting point is values in your language.
dgreensp 12y ago
From the examples, it looks like JSON and EDN are both first-class "transport" formats for Transit: https://github.com/cognitect/transit-format/tree/master/examples/0.8/simple https://github.com/cognitect/transit-format/tree/master/exam...

kbd 12y ago

I haven't looked that closely at Transit yet, but one very small thing that irked me at first glance is that the tag for floats is 'd' and the tag for arbitrary precision decimals is 'f', which seems entirely backwards.

rubiquity 12y ago
I think this likely has to do with how confusing the implementation of floats and doubles in programming languages has been over the years. A lot of languages will call a double precision decimal a float, but I don't know of any languages that call a single precision decimal a double. I think the general ambiguity around float is why they chose to use it as any precision float.

latchkey 12y ago

I'm curious how Transit handles cyclic object graphs for optimizing the payload size. Would be nice to see JSOG integration. https://github.com/jsog/jsog https://github.com/jsog/jsog

puredanger 12y ago
I don't think Transit does or plans to handle cyclic object graphs.
- tel 12y ago
  The name generation from caching ought to allow it, though...

noelwelsh 12y ago

I can imagine using Transit as a format to communicate between servers, but I'm less sold on the JS story. If I'm exposing an API endpoint to the browser I'm also exposing it as an API endpoint. In this case I need a really good reason to not use JSON, which everyone understands and can parse. Adding a barrier to people using my API just doesn't make sense. It's fairly standard for single page apps to consume a public API, so I think this will be a fairly common objection. Also, Computer Modern is so so so ugly (IMHO of course).

swannodette 12y ago
This is a winner-take-all narrative I simply cannot understand. I recall when all endpoints were XML and JSON.parse was called eval. I can easily imagine a endpoint providing both Transit JSON and plain JSON. I know which one I'd rather consume from the client when building a sophisticated application.

SimeVidas 12y ago

Am I the only one who wants to know how transit data looks like (compared to JSON) over the wire?