7 ms·

> We got NoSQL (esp. JSON based ones) instead of XML. Close enough though. JSON is basically XML with 50% less bullshit. It does less but it does it so much mo

> We got NoSQL (esp. JSON based ones) instead of XML. Close enough though. JSON is basically XML with 50% less bullshit. It does less but it does it so much more elegantly than XML - and has the further benefit of a syntax similar to that of structured data in such languages as Javascript (duh), Python and Ruby - that it was a shoo-in to squeeze XML out of most non-enterprise uses, at least on the data transfer format side. However, to this day most data still resides in SQL databases, which have the net advantages of speed, stability and reliability that come from long lead time. What has changed is that object-relational mapping has gotten vastly better than it was in 2004, so programmers can mostly enjoy the best of both worlds. There's also the fact that SQL is an easy enough DSL to understand, read and write that it isn't a barrier to use even in edge cases where ORMs fall down.

SonicSoul 14y ago
> object-relational mapping has gotten vastly better than it was in 2004 Maybe so, but it still seems like it's only good for prototyping. With each new version of ORM i get excited and try to implement it, but sooner or later it comes to bite me in the ass, and long term maintenance becomes more painful than not using ORM with all the low level hacks that need to be added and tricky management of the mapping config.
- ryanbrunner 14y ago
  I think the key difference, at least among most developers I work with, is that ORM has evolved from a complete abstraction of all relational querying, into more of a productivity tool that can be useful in certain circumstances. The differences between something like Hibernate and something like ActiveRecord is pretty vast, at least in terms of usage, and I think one of the key learnings we've made since 2004 is that it's not practical or feasible to treat ORM as a silver bullet. Honestly, no one should be surprised that ORM isn't a silver bullet. NoSQL isn't either - there's plenty of caveats that should be considered before deciding on using NoSQL. I'd still argue that about 80% of database interactions can be solved by a decent ORM. Anything that lets me ignore 80% of my trivial problems and focus on the 20% that's actually challenging is a net win in my book.
edu222 14y ago
Have you tried Django's ORM? From what I understand its not perfect, but for the most part it really rocks.
- hcarvalhoalves 14y ago
  I've been working with Django for 5 years, and I would say Django's ORM is pretty pedestrian, when not downright painful. It fails at simple things like allowing you to GROUP BY without falling back to SQL.
  - amatix 14y ago
    That used to be the case, it's much better now: see QuerySet.annotate() and QuerySet.aggregate()
    - hcarvalhoalves 14y ago
      I'm aware. These features are not replacement for GROUP BY.
jimbokun 14y ago
It's true that JSON document databases very closely follow the model set out by earlier XML document databases, and have succeeded for the reason Yegge spelled out. The big lack is the equivalent of XQuery and XPath, in my opinion. Javascript plays this role in some systems, but I don't think there is anything for Javascript matching the power and flexibility of XQuery/XPath for querying and transforming data. I would be happy to learn about query languages for JSON document databases that prove me wrong.
- wildgift 14y ago
  JSON is traditional: dictionaries, arrays. OO languages have built-in syntax to deal with those, via [] and dot notation, and the array objects nowadays have ways to do map() apply() grep() etc. on the collections. The main thing missing is that XML has attributes, so you can query against those. I won't say that the JS stuff is better, but it's one less thing to learn, and what you learn is directly applicable to writing apps in JS.
corresation 14y ago
JSON is basically XML with 50% less bullshit. I don't get this. XML has the capacity of being as simple as you want it to be. You don't need schemas, namespaces, xpath, and so on, and can make your XML life every bit as easy as JSON. In many ways easier given that JSON doesn't even have a bloody date or time type. But if you do want strong-type validation, schemas and namespaces are there for you. If you want an easy, standard way to query, XPath/XQuery are there for you. If you want to wholesale completely restructure entire documents/data blobs, XSLT is there for you. I have never gotten the hate for XML. I suspect -- and this is not some sort of slur against those who don't like something I like, but rather is an observation about general tech trends -- that many programmers were simply too lazy to understand all of XML, so rather than work in an area where they felt ignorant (again, nothing stops you from making bespoke XML at your desire), JSON and things like it seem so appealing.
- reginaldo 14y ago
  I think the 50% less bulshit figure is pretty generous. Let's not forget that, unless you opt out of it, most XML parsers, when given user-generated input, will do lots of crazy stuff. In the best case scenario, the machine that parsed the input will fall victim to DoS. In the worst, we have good old remote code execution. Tipically, an attacker will be able to read lots of files from the servers and make arbitrary network connections, many times from the viewpoint of a machine inside a corporate firewall. As a spare-time security researcher, let's say I absolutely love XML. As a developer, I despise it.
  - corresation 14y ago
    I'm sorry, but the vague handwaving about security issues with XML is utter nonsense. There have been issues, just as there have been issues with JSON, HTML, or any other parsed or shared content. Alluding to some vast chasm of danger is absurd, made especially obvious that XML remains the lingua franca or most enterprise systems and interchanges (meaning the ones that people are most interested in trying to compromise).
  - reginaldo 14y ago
    corresation: I can't answer your post directly, so I'll answer my own. It's absolutely not vague handwaving. HTML or JSON parsers can't make arbitrary network connections. XML parsers often can. XML remains the lingua franca or most enterprise systems and interchanges (meaning the ones that people are most interested in trying to compromise That is one of the reasons attackers are so happy. They are not trying to compromise. They are succeeding. Show me a SOAP/XMLRPC web service and I will show you a compromised machine, with very high probability. See, for instance, last year's BlackHat presentation about SAP. Root with 1 request. Granted, there was an overflow involved, but the entry point was XML. Many more instances of this are available (I've compromised tens of systems in the last six months through the "magic" of XXEs, but unfortunately can't talk about them). http://media.blackhat.com/bh-us-12/Briefings/Polyakov/BH_US_12_Polyakov_SSRF_Business_Slides.pdf http://media.blackhat.com/bh-us-12/Briefings/Polyakov/BH_US_...

SonicSoul 14y ago

> object-relational mapping has gotten vastly better than it was in 2004 Maybe so, but it still seems like it's only good for prototyping. With each new version of ORM i get excited and try to implement it, but sooner or later it comes to bite me in the ass, and long term maintenance becomes more painful than not using ORM with all the low level hacks that need to be added and tricky management of the mapping config.

ryanbrunner 14y ago
I think the key difference, at least among most developers I work with, is that ORM has evolved from a complete abstraction of all relational querying, into more of a productivity tool that can be useful in certain circumstances. The differences between something like Hibernate and something like ActiveRecord is pretty vast, at least in terms of usage, and I think one of the key learnings we've made since 2004 is that it's not practical or feasible to treat ORM as a silver bullet. Honestly, no one should be surprised that ORM isn't a silver bullet. NoSQL isn't either - there's plenty of caveats that should be considered before deciding on using NoSQL. I'd still argue that about 80% of database interactions can be solved by a decent ORM. Anything that lets me ignore 80% of my trivial problems and focus on the 20% that's actually challenging is a net win in my book.

edu222 14y ago

Have you tried Django's ORM? From what I understand its not perfect, but for the most part it really rocks.

hcarvalhoalves 14y ago
I've been working with Django for 5 years, and I would say Django's ORM is pretty pedestrian, when not downright painful. It fails at simple things like allowing you to GROUP BY without falling back to SQL.
- amatix 14y ago
  That used to be the case, it's much better now: see QuerySet.annotate() and QuerySet.aggregate()
  - hcarvalhoalves 14y ago
    I'm aware. These features are not replacement for GROUP BY.

jimbokun 14y ago

It's true that JSON document databases very closely follow the model set out by earlier XML document databases, and have succeeded for the reason Yegge spelled out. The big lack is the equivalent of XQuery and XPath, in my opinion. Javascript plays this role in some systems, but I don't think there is anything for Javascript matching the power and flexibility of XQuery/XPath for querying and transforming data. I would be happy to learn about query languages for JSON document databases that prove me wrong.

wildgift 14y ago
JSON is traditional: dictionaries, arrays. OO languages have built-in syntax to deal with those, via [] and dot notation, and the array objects nowadays have ways to do map() apply() grep() etc. on the collections. The main thing missing is that XML has attributes, so you can query against those. I won't say that the JS stuff is better, but it's one less thing to learn, and what you learn is directly applicable to writing apps in JS.

corresation 14y ago

JSON is basically XML with 50% less bullshit. I don't get this. XML has the capacity of being as simple as you want it to be. You don't need schemas, namespaces, xpath, and so on, and can make your XML life every bit as easy as JSON. In many ways easier given that JSON doesn't even have a bloody date or time type. But if you do want strong-type validation, schemas and namespaces are there for you. If you want an easy, standard way to query, XPath/XQuery are there for you. If you want to wholesale completely restructure entire documents/data blobs, XSLT is there for you. I have never gotten the hate for XML. I suspect -- and this is not some sort of slur against those who don't like something I like, but rather is an observation about general tech trends -- that many programmers were simply too lazy to understand all of XML, so rather than work in an area where they felt ignorant (again, nothing stops you from making bespoke XML at your desire), JSON and things like it seem so appealing.

reginaldo 14y ago
I think the 50% less bulshit figure is pretty generous. Let's not forget that, unless you opt out of it, most XML parsers, when given user-generated input, will do lots of crazy stuff. In the best case scenario, the machine that parsed the input will fall victim to DoS. In the worst, we have good old remote code execution. Tipically, an attacker will be able to read lots of files from the servers and make arbitrary network connections, many times from the viewpoint of a machine inside a corporate firewall. As a spare-time security researcher, let's say I absolutely love XML. As a developer, I despise it.
- corresation 14y ago
  I'm sorry, but the vague handwaving about security issues with XML is utter nonsense. There have been issues, just as there have been issues with JSON, HTML, or any other parsed or shared content. Alluding to some vast chasm of danger is absurd, made especially obvious that XML remains the lingua franca or most enterprise systems and interchanges (meaning the ones that people are most interested in trying to compromise).
- reginaldo 14y ago
  corresation: I can't answer your post directly, so I'll answer my own. It's absolutely not vague handwaving. HTML or JSON parsers can't make arbitrary network connections. XML parsers often can. XML remains the lingua franca or most enterprise systems and interchanges (meaning the ones that people are most interested in trying to compromise That is one of the reasons attackers are so happy. They are not trying to compromise. They are succeeding. Show me a SOAP/XMLRPC web service and I will show you a compromised machine, with very high probability. See, for instance, last year's BlackHat presentation about SAP. Root with 1 request. Granted, there was an overflow involved, but the entry point was XML. Many more instances of this are available (I've compromised tens of systems in the last six months through the "magic" of XXEs, but unfortunately can't talk about them). http://media.blackhat.com/bh-us-12/Briefings/Polyakov/BH_US_12_Polyakov_SSRF_Business_Slides.pdf http://media.blackhat.com/bh-us-12/Briefings/Polyakov/BH_US_...