7 ms·
URL Object Notation: A better JSON for URLs
- bdfh42 15y agoI like the concept. However JSON works because it can be directly converted into a JavaScript object and (without too much trouble) a JavaScript object can be converted to JSON. I thus think that to succeed this approach needs some support on the server side to convert the notation into an object that can be interrogated from code. OK, that would need to be different for each server runtime but things like this tend to pick up support fairly quickly. I use JSON to communicate with ASP.NET web services because the .NET runtime provides a great de-serializer that can convert the JSON directly to one of my server side classes and vice versa
- vjeux 15y agoI am not sure to properly understand your second paragraph. Here's an answer: URLON supports both "stringify" and "parse" operations. So you can go both ways. You can even do fun things like URLON -> Javascript Object -> JSON if you want to. The fact that it is 100% compatible with JSON is a great benefit. As for my use, the URLON is in the hash part of the URL (after the #) so everything in running in the client. This is useful to give URLs that hold the current state of the page.
- roberthahn 15y agoI thought the work you did was cool but honestly? Hashes are not the place to store page state. See http://www.webmonkey.com/2011/02/gawker-learns-the-hard-way-why-hash-bang-urls-are-evil/ http://www.webmonkey.com/2011/02/gawker-learns-the-hard-way-... for an example. If you need to store page state RESTfully, I would suggest you create a page state resource and PUT your state there. If you need a page's state, ask for it by name. This also has the benefit of keeping URLs shorter and cleaner.
- vjeux 15y agoI'm not sure to understand what you mean. Could you tell me how I would implement that on a concrete example: http://db.mmo-champion.com/items/#table__search_results_item=2%3A-category http://db.mmo-champion.com/items/#table__search_results_item... I want to store: page number, sorted-column, reverse.
- roberthahn 15y agoCreate a new /items/pageState URI that accepts POST, PUT, and GET requests. POST to /items/pageState to get a handle - this could be a randomly generated small sequence of characters. A response will contain a Location: header with a URL: /items/pageState/aA1 (the aA1 is just an example for this description - each user would get an unused sequence of characters) Anytime the user changes the state of the page, PUT the page number, sorted-column, and reverse fields to /items/pageState/aA1. Now, the url of the items page to http://db.mmo-champion.com/items/?pageState=aA1 http://db.mmo-champion.com/items/?pageState=aA1. When that page loads, the JS will make an Ajax GET request to /items/pageState/aA1 to fetch the state of the page, and re-render it appropriately. If you're concerned about speed, well, don't be. Add support for eTags on /items/pageState, and while the state is unchanged, that data will be fetched from browser cache instead of the network.
- vjeux 15y agoOne state per user is not what I want. If the user sorts in one way, give a link, then sorts in another way and put a link back. I want both links to be differents. Also, your solution adds a lot of extra server calls. The goal of client-side sorting & pagination is to avoid those. I don't want to get them back just for the sake of being RESTful.
- roberthahn 15y agoAh, new constraints on the problem! :) Love it! My solution does not have to add a lot of extra server calls - it all depends on the caching strategy you choose. For example, if you use maxAge, you would only be adding one extra request per new state created, or per new fetch; all subsequent fetches for the same state would automatically pull from the browser cache. To satisfy the linking requirement, I would redesign my solution to just POST the state to /items/pageState, and get back a URI that represents only that state (which can be shared across all users). Combine this with the maxAge, and yes, you would have to make more requests, but the extra overhead would still be way less significant than adding a link to an image on every page (especially in terms of bandwidth) But, chacun à son goût! Everyone prefers their own tradeoffs :)
- SoftwareMaven 15y agoNobody turns a JSON object directly in JavaScript (I'm assuming you mean eval, otherwise, both are doing the same parsing). Security considerations have long since moved us away from that. Also, this is really meant for web client apps that are maintaining their state in the URL (that crazy AJAX thing). The app would parse it and make whatever JSON requests are needed with the server.
- adeelk 15y agoTo me, this: _table_achievement_column=instance&ascending:true is less clear than: table[achievement][column]=instance&table[achievement][ascending]=true which also has the benefit of being parsed properly by most web frameworks.
- vjeux 15y agoThe URL strings are mostly read only. Once generated, they are not going to be edited by the users anymore. I find it better to have less readable output as long as there is a win in term of length. Because no one likes big urls. Also, what is less visible in URLON is the object structure. But usually when you want to edit something, you are only interested in changing the values, not the structure itself.
- adeelk 15y agoIf you want to optimize length over readability, then how about: col=instance&asc=1 Or even better: c=instance&a=1 :)
- vjeux 15y agoIt is still possible to make such small structure with URLON URLON.stringify({c: 'instance', a: 1}) == '_c=instance&a:1' We also benefit from the typing: 1 is a number an not a string :)
- adeelk 15y agoBut URLON.stringify({c: 'instance', a: '1'}) == '_c=instance&a:1' as well, right?
- vjeux 15y ago@adeelk: 1, the number, is converted to :1, but "1", the string, is converted to =1. This way I can tell whether it's a string or number upon reconstruction. You can actually run this on the blog page URLON.stringify({c: 'instance', a: '1'}) == '_c=instance&a:1' It will return false
- deleted 15y ago[deleted]
- kablamo 15y agoWhy would want json (or html) in your urls? Example?
- vjeux 15y agoI want to store the current state of the page. For example, if there's a table in the page, I want to keep track of what page I'm in, what column I sorted ... http://db.mmo-champion.com/items/2/#table__search_results_item=1%3A-category http://db.mmo-champion.com/items/2/#table__search_results_it...
- DougWebb 15y agoIf that's all you're trying to do, then why not store the json using local storage and a randomly generated key, and just put the key into the url's fragment?
- Robin_Message 15y ago_foo_bar=Something&baz=Else could parse as either of: {foo:{bar:"Something",baz:"Else"}} {foo:{bar:"Something"}, baz:"Else"} which seems to ruin things. You need a character to end an object, maybe ; or & would work. Also, I don't like using : in query strings as a separator since that makes it a bit ugly and not quite a query string. How about using = for both and using say ! to identify non-strings, and @item@item@item for arrays? So your example becomes: _user_name=Bob%20Smith&age=!47&sex=M&dob=5/12/1956&pastimes=@golf@opera@poker@rap&;children_bobby_age=!12&sex=M&;sally_age=!8&sex=F
- vjeux 15y agoI thought that I could get away without terminations for objects & arrays but I was wrong. I've added ; as you suggested. I made it safe to remove ; at the end in order to keep size small for flat structures. @item1@item2@item3 is a really good idea. It's added :)
- vaporstun 15y ago> _foo_bar=Something&baz=Else > could parse as either of: > {foo:{bar:"Something",baz:"Else"}} > {foo:{bar:"Something"}, baz:"Else"} I don't think this is right, or at least, it shouldn't be right. I think _foo_bar=Something&baz=Else should only parse as {foo:{bar:"Something"}, baz:"Else"}. {foo:{bar:"Something",baz:"Else"}} should be _foo_bar=Something&_foo_baz=Else It should be possible for the writer of the article to fix this without major changes.
- Robin_Message 15y ago> {foo:{bar:"Something",baz:"Else"}} should be _foo_bar=Something&_foo_baz=Else This makes sense, but is not what I or the author was suggesting. Note that it is somewhat repetitive (foo appears twice). In contrast the new suggestion simply has an possible ; to disambiguate these possibilities.
- josscrowcroft 15y agoAre you planning to make a server-side implementation so that these URLs could be parsed on the server? Love the idea.
- vjeux 15y agoIf your server side is written in Javascript, you can use the implementation here: https://github.com/vjeux/URLON/blob/master/urlon.js https://github.com/vjeux/URLON/blob/master/urlon.js If not, someone need to write a URLON library for your language.
- josscrowcroft 15y agoOne more point - `JSON.stringify` returns an error when attempting to stringify a circular structure such as the `window` object... I tried `URLON.stringify(window)` and nothing happened, no error or return value (bad sign) so hit it a few more times, saw a couple stack overflow errors, and the tab crashed. If possible it should definitely throw a RangeError instead of attempting to stringify a circular structure.
- vjeux 15y agoYou can use Douglas Crockford cycle tools in order to deal with this. https://github.com/douglascrockford/JSON-js/blob/master/cycle.js https://github.com/douglascrockford/JSON-js/blob/master/cycl... As the only way to know if an object has already been visited is in linear time in Javascript, it makes cycle detection a O(n^2) process. This is way too costly to add by default unfortunately.
- josscrowcroft 15y agoThanks for the link, I see what you mean about costly. Perhaps a way to hack-fix this is to set a parameter for maxRecursion, to prevent the errors - seems like a pretty solid way of preventing endless loops (RangeErrors) but is probably something that should be off-by-default and switched on globally via a setting or once via a parameter... I can add it and submit a pull-request for your consideration if you like?
- vjeux 15y agohttp://jsperf.com/object-cycle-detection http://jsperf.com/object-cycle-detection I've added a jsPerf to see how bad it performs. For small objects it's reasonable but if the object has 10 000 elements the Javascript test will take 1 second, which is not acceptable. I'd rather give people a detection cycle detection library if their input may be circular.
- dramaticus3 15y agoWhy wouldn't you just base64encode your json and include that in the URI and the base64decode it when you want to de-json it ?
- vjeux 15y agoI've tried many things but I wanted it to be readable, small, and good looking. JSON: {"user":{"name":"Bob Smith","age":47,"sex":"M","dob":"5-12-1956"},"pastimes":["golf","opera","poker","rap"],"children":{"bobby":{"age":12,"sex":"M"},"sally":{"age":8,"sex":"F"}}} Base64: eyJ1c2VyIjp7Im5hbWUiOiJCb2IgU21pdGgiLCJhZ2UiOjQ3LCJzZXgiOiJNIiwiZG9iIjoiNS0xMi0xOTU2In0sInBhc3RpbWVzIjoNClsiZ29sZiIsIm9wZXJhIiwicG9rZXIiLCJyYXAiXSwiY2hpbGRyZW4iOnsiYm9iYnkiOnsiYWdlIjoxMiwic2V4IjoiTSJ9LA0KInNhbGx5Ijp7ImFnZSI6OCwic2V4IjoiRiJ9fX0= JSON + URIEncode %7B%22user%22:%7B%22name%22:%22Bob%20Smith%22,%22age%22:47,%22sex%22:%22M%22,%22dob%22:%225-12-1956%22%7D,%22pastimes%22:%5B%22golf%22,%22opera%22,%22poker%22,%22rap%22%5D,%22children%22:%7B%22bobby%22:%7B%22age%22:12,%22sex%22:%22M%22%7D,%22sally%22:%7B%22age%22:8,%22sex%22:%22F%22%7D%7D%7D URLON _user_name=Bob%20Smith&age:47&sex=M&dob=5-121956;&pastimes@=golf@=opera@=poker@=rap;&children_bobby_age:12&sex=M;&sally_age:8&sex=F
- dramaticus3 15y agoFair enough, though json | base64 is the easiest for others to implement json | gzip | base64 saves another 33 bytes :) and you already know urlon is the shortest
- sethg 15y agoYou can translate the base64 version into a readable form from the terminal using widely available libraries. For example, using a Python interpreter you can do this: >>> from base64 import b64decode >>> from json import loads, dumps >>> print dumps(loads(b64decode('eyJ1c2VyIjp7Im5hbWUiOiJCb2IgU21pdGgiLCJhZ2UiOjQ3LCJzZXgiOiJNIiwiZG9iIjoiNS0xMi0xOTU2In0sInBhc3RpbWVzIjoNClsiZ29sZiIsIm9wZXJhIiwicG9rZXIiLCJyYXAiXSwiY2hpbGRyZW4iOnsiYm9iYnkiOnsiYWdlIjoxMiwic2V4IjoiTSJ9LA0KInNhbGx5Ijp7ImFnZSI6OCwic2V4IjoiRiJ9fX0=')), indent=2) { "pastimes": [ "golf", "opera", "poker", "rap" ], ## [etc.] }
- jamesmoss 15y agoThis smells bad. Passing in a load of JSON (or URLON) seems to go against RESTful practices and really reduces the hackability of the URL.
- vjeux 15y agoThis has been designed to be used in a hash context (after the #) so it's not used in REST application.
- arethuza 15y agoI thought that the URL structure is orthogonal to whether the service is RESTful or not. I know that RESTful services tend to have nice URL formats but, as far as I understand it, you really shouldn't be caring about URL formats - you have a single entry point and everything is driven by "hypertext" from there: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte...
- stdbrouw 15y agoIn cases where I really need to have data structures in URLs, I rather like Rison: http://mjtemplate.org/examples/rison.html http://mjtemplate.org/examples/rison.html which has existed for quite some time now and has parsers/generators in Python (https://github.com/stdbrouw/python-rison https://github.com/stdbrouw/python-rison), Ruby and JavaScript.
- vjeux 15y agoThanks! I didn't know about Rison, it is trying to solve the same issue :) Edit: I've added Rison to the list of translations in the article. It is far more readable than URLON but I find that it doesn't feel like it's a part of an url.
- skrebbel 15y ago> It is far more readable than URLON but I find that it doesn't feel like it's a part of an url. Should it?
- stdbrouw 15y agoIf you know what kind of structure each argument will have, the variants O-Rison and A-Rison further cut down on the parentheses. For example: http://example.com/service?query=q:'*',start:10,count:10&pretty=false Looks pretty natural to me.
- sethg 15y agoThe standard URL query-string syntax, supported by libraries in every programming language worth using, uses & and = to construct “key1=value1&key2=value2”-style parameters. URLON completely breaks that syntax: if I see something like _table_achievement_column=instance&ascending:true my immediate reaction is “so there are two parameters, one called ‘table_achievement_column’, and one called ‘ascending’... wait... why isn’t there an equals sign between ‘ascending’ and ‘true’?” The syntax is just similar enough to query-string syntax that it can mislead to someone trying to parse it by eye. And if you’re trying to pass complicated recursive structures (the kind of structures that JSON was invented to describe) through the URL, they’re not going to be parseable by eye in any format.
- aubergene 15y agoSo the main advantage over rison is that it urlify should work correctly in as many clients as possible? It would be good to start a test matrix to verify this.
- autarch 15y agoRight, cause no one ever uses an underscore in parameter identifiers.
- Kilimanjaro 15y agoA slight mod can make it easier to read/parse. First the rules: everything is a string unless specified like num# date% bool! list* and objects are delimited by : and ; Here is an example: user:name=Bob,age#=47,sex=M,dob%=5-12-1956;pastimes*:golf,opera,poker,rap;children:bobby:age#=12,sex=M;sally:age#=8,sex=F
- Kilimanjaro 15y agoBtw, people complain about the usability or applicability of such a format, well it is more about the mental challenge to solve a problem in a clean way and the steps to get to the best solution, even if the problem is non-existent or silly or retarded, the neural exercise is worth it.
- underwater 15y ago'#' denotes a fragment in a URL. Browsers will only send 'user:name=Bob,age' to the server.
- Kilimanjaro 15y agoThen lets use $, we still have a few chars available
- drdaeman 15y agoI think it would be easier to declare that several forms like (true, false, ISO-8601 strings or decimal numbers) are "special" and if there's a string that looks like them it must be specifically noted as so. This complicates parsing, but feels more natural. In my understanding, most of time there's an occurence of "age=47", it actually means {"age": 47}, not {"age": "47"}. But if we'd want to represent the latter, we could easily do it with "age*=47". So it'll be something like: user:name=Bob,age=47,sex=M,dob=1956-12-05;pastimes:golf,opera,poker,rap;children:bobby:age=12,sex=M;sally:age=8,sex=F
- drdaeman 15y agoUnreadable, ambiguous and just plain ugly. First of all, in my understanding URLON is primarily made for URI fragments. No browser would send a URLON-encoded data, and if you're doing AJAX, then URLs are barely a concern. While the JSON is overly-verbose, the URLON is way too human-unfriendly: JSON: {"table":{"achievement":{"ascending":true,"column":"instance"}}} URLON: _table_achievement_ascending:true&column=instance Humans suck at parsing grammars. We even suck with deeply nested parentheses (that's why editors highlight matching parens), and counting underscores is way more counter-intuitive. Oh, and key names with underscores will look ugly. If one'd want to be "URLish", and concise, he'd write something like table[achievement[ascending=true&column=instance]] While not perfect, at least, a human could understand what's going on from a first glance.
- jrockway 15y agoSo having a " in your URL is worse than having an &? Neither are representable in HTML, so you're going to have to deal with escaping whether you want to or not. And if you're already escaping & to &, it's pretty easy to escape " to ". Which you should already be doing anyway. (Never put a literal [&<>"'], in your (x)html document, please.)