8 ms·

Simplicity and Utility, or Why SOAP Lost

120 points by keithba 12y ago

tfederman 12y ago
I feel like I spend a troubling amount of time as a software developer dealing with and/or avoiding solutions that are much more complicated than the problems they're trying to solve.
- blt 12y ago
  tell me about it... I've been searching for a C++ library that will encode x86 instructions from syntax that looks moderately similar to assembler code. Not a disassembler, not a JIT toolkit, not a special cross-platform IR that gets compiled to real instructions. Just an instruction encoder with a moderately pretty syntax. No luck so far.
  - s_kanev 12y ago
    Have you seen xed [1]? It seems to fit the bill nicely from what I understand about your requirements. [1] https://software.intel.com/sites/landingpage/pintool/docs/67254/Xed/html/main.html https://software.intel.com/sites/landingpage/pintool/docs/67...
    - blt 12y ago
      yes, I don't need its decoding capability and I don't think its syntax is pretty but if I ever decide to write my own library I'll probably wrap xed in some C++ sugar.
      - _dps 12y ago
        I'm not sure I understand your requirements but might DynASM be useful? It's one component of the JIT library behind LuaJIT but many people use it for run-time code generation completely outside a JIT setting. http://luajit.org/dynasm.html http://luajit.org/dynasm.html
        blt 12y ago
        DynASM looks cool but its preprocessing step and fancy C integration definitely place it outside the description "Just an instruction encoder with a moderately pretty syntax." I guess it's not fair to say that XED and DynASM are "much more complicated than the problems they're trying to solve." They are much more complicated than the problem I'm trying to solve. But I am surprised that there is no minimal X86 encoder with nice C++ syntax out there.
    - blt 12y ago
      also Xed does not seem to be open source. It ships with a bunch of headers that say "Intel Open Source License" at the top but there is a binary library instead of source files.
markolschesky 12y ago
SOAP is still common in healthcare as many IHE profiles rely on SOAP for Cross-community document/data exchange. They are a pain to work with if you don't have a toolset that handles it well.
Eric_WVGG 12y ago
The S Stands for Simple — https://web.archive.org/web/20140710041718/http://wanderingbarque.com/nonintersecting/2006/11/15/the-s-stands-for-simple/ https://web.archive.org/web/20140710041718/http://wanderingb... a web classic, can never be linked enough times
- smacktoward 12y ago
  Compared to what came before it, SOAP was simple! It's just that REST + JSON turned out to be even simpler.
- larrys 12y ago
  "can never be linked enough times" Off topic question here. Does an archive.org article get cached so that when it's linked to it comes up faster? Or does it display at the same speed of a random archive.org page? That page came up pretty fast for me don't know if that is random or because other people are hitting the link from HN.
  - burnte 12y ago
    Archive.org does caching. A page is pulled from long term slow storage, and cached.
  - deno 12y ago
    Yes. < X-Page-Cache: HIT
entelechy0 12y ago
Heh...we're using SOAP and WSDL for this current project...
- Pxtl 12y ago
  You have my sympathy.
  - entelechy0 12y ago
    Thank you. And believe me - it was certainly not my choice.
treve 12y ago
I think SOAP has lost in a sense that it's no longer very popular around many web developers. But has it really lost? I still see it used a lot in large corporate and high tech environments. I was never a big fan of SOAP myself, but that's also partly because when I started, the tooling wasn't great and I got mostly a negative user-experience. But this is many years ago. My understand is that these days it's quite good, and it has a pretty good user-experience. Furthermore, everything is backed by xml schemas, which makes it easier to create and require strictly valid xml being sent back and forward. While some equivalents exist in REST-ish API's, it's arguably not very common and with SOAP it's a pretty much a free and built-in feature that everybody uses. Modern web API design is great for getting things up and running fast, but it's not amazing for rigorously robust design. I don't really like the "SOAP is bloated and over-engineered" narrative. It is uninteresting now, and was uninteresting 10 years ago. I think it's much more interesting to learn about the things that SOAP did get right, and see how we can apply some of these learnings in our API's.
- keithba 12y ago
  I think the momentum is not on SOAP's side. But you are correct that it isn't going away, at least for legacy. I'm writing something up on what was done right with SOAP and WS-*. (Arguably the code generation was useful for quite a few scenarios.)
- delluminatus 12y ago
  I work on an SOA platform for a very large software company. We offer both REST and SOAP versions of our RPCs. In my experience, SOAP services are okay plug-and-play if you own both the provider and the consumer. But consuming external APIs can be nightmarish, especially with SOAP 1.2. SOAP exists in our organization for legacy reasons but we recommend all new consumers use RESTful services. Our mobile clients like REST because it's faster and easier to consume from outside the .NET platform than SOAP. Even within the .NET platform, a client can get up and running consuming a "RESTful" service in fewer lines of code than a SOAPy one. Not only SOAP tooling, but HTTP client tooling in general, is a lot easier to use these days. WSDLs are cool because they are a fairly standard metadata format. Our clients like them because they can perform data validation with proxy/firewall systems before the request ever enters the corporate network. I've yet to see a concrete benefit from that kind of strictness, though. IMO, SOAP schema validation is so brittle that it causes more problems than it fixes. Of course, you can also use non-SOAP document description formats like RAML for describing your RESTful services. It's not like SOAP offers anything you can't get for REST. But once you start adding all that on top of your REST API stack, you might as well use SOAP 1.1. (Never use SOAP 1.2). I don't think the main problem with SOAP is that it's over-engineered, although it is. The issue is that the supposed benefit of "interoperability" was never really realized. It's supposedly protocol-agnostic, but nobody cares. It's supposedly interoperable, but everyone implements it differently.
ultramancool 12y ago
Sure, SOAP's complexity is a problem, but the issue with "RESTful" APIs is that they're so loosely defined, almost all the way to the other extreme. Being able to generate code and have a type-checked API, which is as easy to use as any other local API was a very nice feature we've now lost. I wish there were a better compromise.
- keithba 12y ago
  I think we'll see more and more schema related projects for JSON-based API (like swagger). It will be interesting if they can compose in a way that keeps the complexity down, as opposed to what happened with SOAP.
- Blackthorn 12y ago
  There is a better compromise (in fact, several): * Originally, Google Protocol Buffers. These are still in wide use; unfortunately, they never published/blessed an official RPC stack. * Apache Thrift, aka Facebook's answer to Protocol Buffers. * Capn Proto, by one of the Protocol Buffers authors, now has an RPC standard! After having used Protocol Buffers extensively, I could never go back to untyped APIs. I'd at least use Apache Thrift for everything. Hopefully Capn Proto gets more languages supported for its RPC soon.
  - blt 12y ago
    Protocol Buffers is great. Thrift generates much nicer C++ code but performs slower and wants you to do everything inside their networking world. I'm excited for Cap'n Proto. It's a good time to be doing binary IPC. I'm still uncertain about melding the message format and RPC mechanics though. Part of me feels like network communications is just too big of a deal to abstract away into something that looks like a normal procedure call. But hey, even if that's always true for big systems, it probably won't be for smaller apps.
seanalltogether 12y ago
My guess is that more and more people are coming to the realization that xml is great for fluid text and document formatting, but too ambiguous to serve as a data structure format. XML nodes and children are just too slippery. I've actually argued with enterprise developers about why serving the following format was a terrible idea. <user> <id>123456</id> <name>Joe</name> <account>Account 1</account> <account>Account 2</account> </user>
- dietrichepp 12y ago
  The other problem is that you never know how a data structure would map to XML. <user id="123456" name="Joe"> <account name="Account 1"/> <account name="Account 1"/> </user>
- Igglyboo 12y ago
  Is there a hard and fast rule for deciding when something is an attribute and when it is a child? I would have personally done <user id="123456" name="Joe"> <account>Account 1</account> <account>Account 2</account> </user> but I've never actually produced XML, only consumed.
  - seanalltogether 12y ago
    Both ways still leave the structure ambiguous. Should <user> be mapped to an associative array of mixed objects, or to a fixed object using a custom mapper to shove the account objects into their own sub array.
    - treve 12y ago
      If you're looking for a generalized way to map an XML structure to primitive types, you're going to have a bad time.
  - pjungwir 12y ago
    My rule is that things are almost never attributes. :-) There is less flexibility there. Escaping seems harder, and you can't have sub-elements or lists. Avoiding attributes makes things more verbose, but then that's XML for you.
bradleybuda 12y ago
I think this is largely accurate, but one thing the article misses is that SOAP was never as interoperable as it was claimed to be. If you had a Microsoft client and a Microsoft server, things would work great, but as soon as you started mixing vendor implementations, you quickly ended up in configuration hell. For simple SOAP messaging you could usually get it to work with enough elbow grease, but for any of the WS-* protocols you might as well be doomed. I lost days, if not weeks, of work to failed interop of WS-Security across two different vendors (both Java, and both with the code under my control).
- liveoneggs 12y ago
  throw in perl or python and really have some fun
  - untothebreach 12y ago
    Oh god, I have to deal with a SOAP service from a perl client...try dumping out a SOAP::Lite data structure once, and you will be wondering what in gods name the SOAP::NotLite one would have to look like.
- Pxtl 12y ago
  And from a developer persepctive, the Microsoft stack wasn't very good anyways. WCF stands for Windows Configuration Fuckup.
  - silverbax88 12y ago
    WCF is a mess and poorly documented. I have several significant implementations of WCF I support now and I really often wonder WTF Microsoft was thinking.
    - tdicola 12y ago
      It was a peak of the architecture astronaut era unfortunately.
    - 12y ago
qwerta 12y ago
Apples,oranges and orangutans. Doing transactions and data format evolution in REST is hard, in SOAP it is simple.
nolok 12y ago
The entire thing hide itself behind a "don't worry about how that works, the tools and libs are supposed to do that for you" philosophy, and then those would fail and break so often it was almost comical. You would then end up debugging the entrails of a SOAP exchange, which is one of the most terrible things I've seen.
Pxtl 12y ago
“The essence of XML is this: the problem it solves is not hard, and it does not solve the problem well.” – Phil Wadler, POPL 2003 SOAP takes XML's problems to the next level.
debacle 12y ago
http://www.google.com/trends/explore#q=wsdl%2C%20restful&cmpt=q http://www.google.com/trends/explore#q=wsdl%2C%20restful&cmp... SOAP is past its peak but far from dead. In the Microsoft ecosystem, SOAP is nearly transparent, except for a bit of fiddling - you publish your WSDL from your definitions, I consume it to generate an API class on my end, and after maybe ten minutes on either side, we don't even think about SOAP again.
mpweiher 12y ago
Not sure why the article equates XML with SOAP. Request: GET /customers/43456 HTTP/1.1 Host: www.example.org Response: HTTP/1.1 200 OK Content-Type: text/xml; charset=utf-8 <customer>Foobar Quux, inc</customer> Seems fine to me. In fact, having named elements without the requirement for a dictionary makes parsing straight to an object-representation (without an intermediate property-list) much easier.
- delluminatus 12y ago
  It never equates XML with SOAP. The point is that even while SOAP was being pushed by Microsoft, JSON was gaining mindshare among API developers, and API design was moving away from complex XML documents. Using a simpler format for object serialization into XML is definitely an option, and it's a perfectly fine middle-ground between SOAPy verbosity and JSON compactness, but IMO it doesn't really have a lot to offer over a JSON version of the same API. Many API providers support both formats using content-type detection.
rayiner 12y ago
It's funny that the joke starts with a dig at CORBA. When I was worked on a CORBA-based system a decade ago, I thought I'd never see something more over-designed. But this whole document-oriented web services thing makes me feel like: https://screen.yahoo.com/unfrozen-cave-man-lawyer-1-223412426.html https://screen.yahoo.com/unfrozen-cave-man-lawyer-1-22341242....
bch 12y ago
The article lists 1) JSON coming up to replace XML as a serialization technology, and 2) and Ruby on Rails (etc) I remember even serious pressure from w/i the XML community in the sense of XML-RPC, and the joke that the entire spec for XML-RPC was smaller than the Table of Contents for the SOAP spec. SOAP was tough. It was confusing to use (fwiw, I had the best luck w/ Perls SOAP::Lite) and difficult enough to implement that devs I knew simply abandoned it.
binarymax 12y ago
In my opinion, WS-* lost not because of the format or verbosity. WS-* lost because the concept is about the continuation of RPC, where you expose custom methods that can be invoked from a client. When state is altered in custom ways, complexity goes through the roof - to the point where you need a DSL (WSDL) to define these methods and how to interact with them. REST simplified things because the concept is about limiting the method invocation to the bare minimum and always have an explicit understanding of the state that is being changed or communicated. The XML/JSON question is not as important as the method/resource question.
- keithba 12y ago
  This is very insightful. WCF and WS-* tried very hard to handle both RPC and more asynchronous scenarios, but the tooling continued to be heavily RPC based.
- troels 12y ago
  The whole baked-in type system also adds a lot of complexity. On top of that you get the complexity that is xml it self (namespaces, while theoretically a good idea, are pure insanity).
- deleted 12y ago
  [deleted]
cowardlydragon 12y ago
You can't argue that PUT, DELETE, and POST methods are usable in a browser, and once a rest api starts going hogwild with headers, the GET really doesn't work either. The fact REST didn't provide some way for all methods to be invoked in a simple browser was to me a step towards the inaccessibility of SOAP.
- twic 12y ago
  Rails addresses this by interpreting a special _method parameter: http://api.rubyonrails.org/classes/ActionView/Helpers/FormTagHelper.html#method-i-form_tag http://api.rubyonrails.org/classes/ActionView/Helpers/FormTa... That is, a POST containing _method=delete will be interpreted as a DELETE. It seems faintly silly; if you're exposing an API to the browser, why use methods that you can't use directly and then have to go to the effort of emulating them, when you could just use supported methods directly? Is the use of the proper methods really that important? But it does work.
kakoni 12y ago
New Finnish gov digital infrastructure is going to built using Estonian X-road (well next year). And yes, its only SOAP, so I guess either Finns are pretty crazy or SOAP is making a comeback. Go figure.
userbinator 12y ago
One nice example of how MS really made things more complex with SOAP is the MSN Messenger Protocol (MSNP), which started out quite simple and sane, then got extremely SOAPy in the later versions; compare protocol interactions pre-SOAP: http://msnpiki.msnfanatic.com/index.php/MSNP8:Getting_Details#Example_SYN_Responses http://msnpiki.msnfanatic.com/index.php/MSNP8:Getting_Detail... and doing the same thing with the SOAP'd version: http://msnpiki.msnfanatic.com/index.php/MSNP13:Contact_List http://msnpiki.msnfanatic.com/index.php/MSNP13:Contact_List
- im2w1l 12y ago
  I think this is a great example of SOAP's failing. While it is simple, it looks difficult. As you scroll through all that XML your eyes begin to glaze and you don't even notice the small section at the bottom. Despite being the most important, the section is at the bottom, has weird grammar, and feels like an afterthought. >Here the WSDL and XML schemas for the web service descripted here, you can use them to generate a web service binding for your programming language. >MSN Addressbook/Sharing Service WSDL & XSD files This is how simple it actually is: http://www.diveintopython.net/soap_web_services/index.html http://www.diveintopython.net/soap_web_services/index.html
  - userbinator 12y ago
    I wouldn't consider requiring a ton of mostly redundant and useless information in messages (each namespace needs a whole damn URL every time it's referenced? Does the parser use it, or even care that the entire URL is correct?) and XML parsing (which isn't so bad, in comparison to all that redundantly redundant redundancy) to be "simple" at all. WSDL/XSD is yet another layer on top of this mess of complexity and I suppose you're referring to the fact that it looks simple if everything goes according to plan, but when it doesn't it's anything but simple. Trying to get any kind of performance out of it is another counterpoint to "simplicity"... This is all from the experience of someone who has worked with XMLA-based OLAP systems. That link you gave has a summary part ( http://www.diveintopython.net/soap_web_services/summary.html http://www.diveintopython.net/soap_web_services/summary.html ) that says "SOAP web services are very complicated."
fit2rule 12y ago
Why I never used SOAP: Endian issues.
ChuckMcM 12y ago
Hold on to this, "tldr: Simplicity and utility trump large corporate backing." When your manager tells you that your API needs to handle something "just in case", when your customer asks, "But what if I want to use this API with mumblefratz?", and when you look at your code and say, "Gee, if I make this a variable I could handle any special edge case by encapsulating its specialness in the variable ..."
ownedthx 12y ago
The fact that SOAP is not reasonably supported in a browser was a huge reason it fell down. When SOAP first came on the scene, complicated AJAX based applications were not typical. But as more and more JSON/HTTP APIs emerged in conjunction with browsers becoming more powerful and rich apps build built in them, the meaninglessness & complexity of SOAP grew. Also, another major issue with SOAP is that most all of the popular tools would generate classes based on a WSDL. This creates a toolchain issue that is readily solved for someone experienced, but can really suck if you are new to the idea of generated code working it's way into your project. Much worse was the scenario of WSDL versioning in conjunction with these tools that generated classes. If the API never broke backwards compatibility, you should be OK and can just use the newer class representations, counting on the service and tools to deal with null fields appropriately (not always true unfortunately). But if version bumps of an API/WSDL broke backwards compatibility... then you had to have separate class hierarchies for different versions of the WSDL; what an intense headache. Contrast that to web REST APIs; without a formal schema and a lack of class-centric tooling, client libraries would often let the author stuff in the params themselves and let the serializer stuff in the body based on the params; so yes your API is not as formal, but this isn't a big issue but offers infinite flexibility in dealing with one-off versioning issues or interop issues. As someone else mentioned, some platforms couldn't interop with others (differences as it related to simply stuff like nullable fields or primitives existed all over; total nightmare), and some features of WSDLs didn't translate well to certain languages. A great WSDL author would know to make their WSDL as simple as possible, because they had spent time working with various language toolchains and new the limitations out there, but that's asking way too much and not realistic for someone to have to spend all their time trying to understand all the ways someone in language XYZ might use their WSDL.
- PantaloonFlames 12y ago
  Turns out the "definition language" (WSDL) and the presumption that it would be simpler to deal with than just showing people the messages that needed to be placed on the wire... was the broken assumption. We wanted automagic seriaization and de-serialization and we wanted every language-based construct (linked lists, arrays of structures that contain arrays) to be supported. Different tool vendors built that differently, hence the interop problems. In REST/JSON, we don't have an IDL. We don't have a JSON Schema (yet! thank god) We generate human-readable documentation containing examples, for the definition. And developers build to those examples. There's nothing - no runtime or static tool - that gets in the way of developers serializing their data into JSON, the way they need to.
jcfrei 12y ago
I've had the "fortune" to work with SOAP as well (java implementation) and I never could shake the feeling that it was designed with the intention of being obscure. It seems you could employ an entire batch of consultants just to deal with defining, implementing and changing SOAP APIs (kind of a misnomer I know). The whole change from SOAP 1.1 using an ActionHeader to SOAP 1.2 using an action parameter seems downright misleading. Or downloading WSDL files to automatically create Java classes sounds great in theory but now your program has additional obscure classes with very weird syntax and you have to read through numerous reference guides just to know how you can modify the header of an outgoing request. Such a waste of time.
- ebbv 12y ago
  I also get to deal with SOAP occasionally. I recently got the joy of dealing with it for a CA's API. It's clearly the result of developers and managers more interested in what the technology CAN do than what it should do.
  - ownedthx 12y ago
    In my experience, it wasn't that. Larger companies put value in supporting standards, because it is an important checkbox in their marketing, and SOAP is definitely a standard.
    - ebbv 12y ago
      I was talking about how SOAP itself was designed.
      - ownedthx 12y ago
        ... oh :)
mrfusion 12y ago
I always liked XMLRPC. It seemed really simple. At least in Python they simply mapped to function calls and all of the XML stuff was handled for you. It took minutes to get an API set up with that from any program. I never understood why it's out of favor?
Animats 12y ago
It's all about who has the power. With JSON, if the data returned by the server doesn't exactly match the documentation, it's the client's problem. With SOAP, if the data returned by the server doesn't exactly match the documentation, the server is broken. The client, reading the WDSL file which documents the API, will report an error and refuse to continue. This means bug reports the server provider cannot make go away by the usual defect-denial techniques.
corbinpage 12y ago
SOAP Web Services around still frequently used in a lot of large corporations, where technology evolves a little slower and Microsoft products are still very popular. Being a Rails fan myself, I have always enjoyed using HTTP Web Services and interacting with tech companies' Restful APIs so easily. This post provided a clear, side-by-side comparison of the technologies including the historical context of SOAP, which I found really useful and enlightening. If I could give you some Bitcoin for writing it, I would. Have a new Twitter follower instead. It's hard to find succinct, objective technology articles in this day and age. Most reviews I find are either unconstructively biased ("dynamic typing sucks!") or too long and convoluted to be effective.
cratermoon 12y ago
Let's hope SAML follows soon.
lkrubner 12y ago
Pete Lacey's parody of SOAP is both hilarious and also very accurate: http://harmful.cat-v.org/software/xml/soap/simple http://harmful.cat-v.org/software/xml/soap/simple
jprince 12y ago
The first thing I did as a developer was to write a SOAP wrapper gem for Propertyware's API. It was here that I learned the concept of Pain.
Yhippa 12y ago
I still have to use SOAP more often than I'd like to these days. It's a pain testing and dealing with all the different tools you have to use just to stand up a basic request. A lot of vendors I've worked with will publish REST web services for public data but as soon as any private information needs to be accessed they use SOAP. I don't know if they just don't trust the tools to make REST secure or what but it's disappointing for sure. Also the people who wrote the SOAP code are long gone which makes support for them even more difficult when I'm trying to get bugs fixed. REST server code has not been anywhere as bad to deal with.

120 points by keithba 12y ago

I feel like I spend a troubling amount of time as a software developer dealing with and/or avoiding solutions that are much more complicated than the problems they're trying to solve.

blt 12y ago
tell me about it... I've been searching for a C++ library that will encode x86 instructions from syntax that looks moderately similar to assembler code. Not a disassembler, not a JIT toolkit, not a special cross-platform IR that gets compiled to real instructions. Just an instruction encoder with a moderately pretty syntax. No luck so far.
- s_kanev 12y ago
  Have you seen xed [1]? It seems to fit the bill nicely from what I understand about your requirements. [1] https://software.intel.com/sites/landingpage/pintool/docs/67254/Xed/html/main.html https://software.intel.com/sites/landingpage/pintool/docs/67...
  - blt 12y ago
    yes, I don't need its decoding capability and I don't think its syntax is pretty but if I ever decide to write my own library I'll probably wrap xed in some C++ sugar.
    - _dps 12y ago
      I'm not sure I understand your requirements but might DynASM be useful? It's one component of the JIT library behind LuaJIT but many people use it for run-time code generation completely outside a JIT setting. http://luajit.org/dynasm.html http://luajit.org/dynasm.html
      - blt 12y ago
        DynASM looks cool but its preprocessing step and fancy C integration definitely place it outside the description "Just an instruction encoder with a moderately pretty syntax." I guess it's not fair to say that XED and DynASM are "much more complicated than the problems they're trying to solve." They are much more complicated than the problem I'm trying to solve. But I am surprised that there is no minimal X86 encoder with nice C++ syntax out there.
  - blt 12y ago
    also Xed does not seem to be open source. It ships with a bunch of headers that say "Intel Open Source License" at the top but there is a binary library instead of source files.

markolschesky 12y ago

SOAP is still common in healthcare as many IHE profiles rely on SOAP for Cross-community document/data exchange. They are a pain to work with if you don't have a toolset that handles it well.

Eric_WVGG 12y ago

The S Stands for Simple — https://web.archive.org/web/20140710041718/http://wanderingbarque.com/nonintersecting/2006/11/15/the-s-stands-for-simple/ https://web.archive.org/web/20140710041718/http://wanderingb... a web classic, can never be linked enough times

smacktoward 12y ago
Compared to what came before it, SOAP was simple! It's just that REST + JSON turned out to be even simpler.
larrys 12y ago
"can never be linked enough times" Off topic question here. Does an archive.org article get cached so that when it's linked to it comes up faster? Or does it display at the same speed of a random archive.org page? That page came up pretty fast for me don't know if that is random or because other people are hitting the link from HN.
- burnte 12y ago
  Archive.org does caching. A page is pulled from long term slow storage, and cached.
- deno 12y ago
  Yes. < X-Page-Cache: HIT

entelechy0 12y ago

Heh...we're using SOAP and WSDL for this current project...

Pxtl 12y ago
You have my sympathy.
- entelechy0 12y ago
  Thank you. And believe me - it was certainly not my choice.

treve 12y ago

I think SOAP has lost in a sense that it's no longer very popular around many web developers. But has it really lost? I still see it used a lot in large corporate and high tech environments. I was never a big fan of SOAP myself, but that's also partly because when I started, the tooling wasn't great and I got mostly a negative user-experience. But this is many years ago. My understand is that these days it's quite good, and it has a pretty good user-experience. Furthermore, everything is backed by xml schemas, which makes it easier to create and require strictly valid xml being sent back and forward. While some equivalents exist in REST-ish API's, it's arguably not very common and with SOAP it's a pretty much a free and built-in feature that everybody uses. Modern web API design is great for getting things up and running fast, but it's not amazing for rigorously robust design. I don't really like the "SOAP is bloated and over-engineered" narrative. It is uninteresting now, and was uninteresting 10 years ago. I think it's much more interesting to learn about the things that SOAP did get right, and see how we can apply some of these learnings in our API's.

keithba 12y ago
I think the momentum is not on SOAP's side. But you are correct that it isn't going away, at least for legacy. I'm writing something up on what was done right with SOAP and WS-*. (Arguably the code generation was useful for quite a few scenarios.)
delluminatus 12y ago
I work on an SOA platform for a very large software company. We offer both REST and SOAP versions of our RPCs. In my experience, SOAP services are okay plug-and-play if you own both the provider and the consumer. But consuming external APIs can be nightmarish, especially with SOAP 1.2. SOAP exists in our organization for legacy reasons but we recommend all new consumers use RESTful services. Our mobile clients like REST because it's faster and easier to consume from outside the .NET platform than SOAP. Even within the .NET platform, a client can get up and running consuming a "RESTful" service in fewer lines of code than a SOAPy one. Not only SOAP tooling, but HTTP client tooling in general, is a lot easier to use these days. WSDLs are cool because they are a fairly standard metadata format. Our clients like them because they can perform data validation with proxy/firewall systems before the request ever enters the corporate network. I've yet to see a concrete benefit from that kind of strictness, though. IMO, SOAP schema validation is so brittle that it causes more problems than it fixes. Of course, you can also use non-SOAP document description formats like RAML for describing your RESTful services. It's not like SOAP offers anything you can't get for REST. But once you start adding all that on top of your REST API stack, you might as well use SOAP 1.1. (Never use SOAP 1.2). I don't think the main problem with SOAP is that it's over-engineered, although it is. The issue is that the supposed benefit of "interoperability" was never really realized. It's supposedly protocol-agnostic, but nobody cares. It's supposedly interoperable, but everyone implements it differently.

ultramancool 12y ago

Sure, SOAP's complexity is a problem, but the issue with "RESTful" APIs is that they're so loosely defined, almost all the way to the other extreme. Being able to generate code and have a type-checked API, which is as easy to use as any other local API was a very nice feature we've now lost. I wish there were a better compromise.

keithba 12y ago
I think we'll see more and more schema related projects for JSON-based API (like swagger). It will be interesting if they can compose in a way that keeps the complexity down, as opposed to what happened with SOAP.
Blackthorn 12y ago
There is a better compromise (in fact, several): * Originally, Google Protocol Buffers. These are still in wide use; unfortunately, they never published/blessed an official RPC stack. * Apache Thrift, aka Facebook's answer to Protocol Buffers. * Capn Proto, by one of the Protocol Buffers authors, now has an RPC standard! After having used Protocol Buffers extensively, I could never go back to untyped APIs. I'd at least use Apache Thrift for everything. Hopefully Capn Proto gets more languages supported for its RPC soon.
- blt 12y ago
  Protocol Buffers is great. Thrift generates much nicer C++ code but performs slower and wants you to do everything inside their networking world. I'm excited for Cap'n Proto. It's a good time to be doing binary IPC. I'm still uncertain about melding the message format and RPC mechanics though. Part of me feels like network communications is just too big of a deal to abstract away into something that looks like a normal procedure call. But hey, even if that's always true for big systems, it probably won't be for smaller apps.

seanalltogether 12y ago

My guess is that more and more people are coming to the realization that xml is great for fluid text and document formatting, but too ambiguous to serve as a data structure format. XML nodes and children are just too slippery. I've actually argued with enterprise developers about why serving the following format was a terrible idea. <user> <id>123456</id> <name>Joe</name> <account>Account 1</account> <account>Account 2</account> </user>

dietrichepp 12y ago
The other problem is that you never know how a data structure would map to XML. <user id="123456" name="Joe"> <account name="Account 1"/> <account name="Account 1"/> </user>
Igglyboo 12y ago
Is there a hard and fast rule for deciding when something is an attribute and when it is a child? I would have personally done <user id="123456" name="Joe"> <account>Account 1</account> <account>Account 2</account> </user> but I've never actually produced XML, only consumed.
- seanalltogether 12y ago
  Both ways still leave the structure ambiguous. Should <user> be mapped to an associative array of mixed objects, or to a fixed object using a custom mapper to shove the account objects into their own sub array.
  - treve 12y ago
    If you're looking for a generalized way to map an XML structure to primitive types, you're going to have a bad time.
- pjungwir 12y ago
  My rule is that things are almost never attributes. :-) There is less flexibility there. Escaping seems harder, and you can't have sub-elements or lists. Avoiding attributes makes things more verbose, but then that's XML for you.

bradleybuda 12y ago

I think this is largely accurate, but one thing the article misses is that SOAP was never as interoperable as it was claimed to be. If you had a Microsoft client and a Microsoft server, things would work great, but as soon as you started mixing vendor implementations, you quickly ended up in configuration hell. For simple SOAP messaging you could usually get it to work with enough elbow grease, but for any of the WS-* protocols you might as well be doomed. I lost days, if not weeks, of work to failed interop of WS-Security across two different vendors (both Java, and both with the code under my control).

liveoneggs 12y ago
throw in perl or python and really have some fun
- untothebreach 12y ago
  Oh god, I have to deal with a SOAP service from a perl client...try dumping out a SOAP::Lite data structure once, and you will be wondering what in gods name the SOAP::NotLite one would have to look like.
Pxtl 12y ago
And from a developer persepctive, the Microsoft stack wasn't very good anyways. WCF stands for Windows Configuration Fuckup.
- silverbax88 12y ago
  WCF is a mess and poorly documented. I have several significant implementations of WCF I support now and I really often wonder WTF Microsoft was thinking.
  - tdicola 12y ago
    It was a peak of the architecture astronaut era unfortunately.
  - 12y ago

qwerta 12y ago

Apples,oranges and orangutans. Doing transactions and data format evolution in REST is hard, in SOAP it is simple.

nolok 12y ago

The entire thing hide itself behind a "don't worry about how that works, the tools and libs are supposed to do that for you" philosophy, and then those would fail and break so often it was almost comical. You would then end up debugging the entrails of a SOAP exchange, which is one of the most terrible things I've seen.

Pxtl 12y ago

“The essence of XML is this: the problem it solves is not hard, and it does not solve the problem well.” – Phil Wadler, POPL 2003 SOAP takes XML's problems to the next level.

debacle 12y ago

http://www.google.com/trends/explore#q=wsdl%2C%20restful&cmpt=q http://www.google.com/trends/explore#q=wsdl%2C%20restful&cmp... SOAP is past its peak but far from dead. In the Microsoft ecosystem, SOAP is nearly transparent, except for a bit of fiddling - you publish your WSDL from your definitions, I consume it to generate an API class on my end, and after maybe ten minutes on either side, we don't even think about SOAP again.

mpweiher 12y ago

Not sure why the article equates XML with SOAP. Request: GET /customers/43456 HTTP/1.1 Host: www.example.org Response: HTTP/1.1 200 OK Content-Type: text/xml; charset=utf-8 <customer>Foobar Quux, inc</customer> Seems fine to me. In fact, having named elements without the requirement for a dictionary makes parsing straight to an object-representation (without an intermediate property-list) much easier.

delluminatus 12y ago
It never equates XML with SOAP. The point is that even while SOAP was being pushed by Microsoft, JSON was gaining mindshare among API developers, and API design was moving away from complex XML documents. Using a simpler format for object serialization into XML is definitely an option, and it's a perfectly fine middle-ground between SOAPy verbosity and JSON compactness, but IMO it doesn't really have a lot to offer over a JSON version of the same API. Many API providers support both formats using content-type detection.

rayiner 12y ago

It's funny that the joke starts with a dig at CORBA. When I was worked on a CORBA-based system a decade ago, I thought I'd never see something more over-designed. But this whole document-oriented web services thing makes me feel like: https://screen.yahoo.com/unfrozen-cave-man-lawyer-1-223412426.html https://screen.yahoo.com/unfrozen-cave-man-lawyer-1-22341242....

bch 12y ago

The article lists 1) JSON coming up to replace XML as a serialization technology, and 2) and Ruby on Rails (etc) I remember even serious pressure from w/i the XML community in the sense of XML-RPC, and the joke that the entire spec for XML-RPC was smaller than the Table of Contents for the SOAP spec. SOAP was tough. It was confusing to use (fwiw, I had the best luck w/ Perls SOAP::Lite) and difficult enough to implement that devs I knew simply abandoned it.

binarymax 12y ago

In my opinion, WS-* lost not because of the format or verbosity. WS-* lost because the concept is about the continuation of RPC, where you expose custom methods that can be invoked from a client. When state is altered in custom ways, complexity goes through the roof - to the point where you need a DSL (WSDL) to define these methods and how to interact with them. REST simplified things because the concept is about limiting the method invocation to the bare minimum and always have an explicit understanding of the state that is being changed or communicated. The XML/JSON question is not as important as the method/resource question.

keithba 12y ago
This is very insightful. WCF and WS-* tried very hard to handle both RPC and more asynchronous scenarios, but the tooling continued to be heavily RPC based.
troels 12y ago
The whole baked-in type system also adds a lot of complexity. On top of that you get the complexity that is xml it self (namespaces, while theoretically a good idea, are pure insanity).
deleted 12y ago
[deleted]

cowardlydragon 12y ago

You can't argue that PUT, DELETE, and POST methods are usable in a browser, and once a rest api starts going hogwild with headers, the GET really doesn't work either. The fact REST didn't provide some way for all methods to be invoked in a simple browser was to me a step towards the inaccessibility of SOAP.

twic 12y ago
Rails addresses this by interpreting a special _method parameter: http://api.rubyonrails.org/classes/ActionView/Helpers/FormTagHelper.html#method-i-form_tag http://api.rubyonrails.org/classes/ActionView/Helpers/FormTa... That is, a POST containing _method=delete will be interpreted as a DELETE. It seems faintly silly; if you're exposing an API to the browser, why use methods that you can't use directly and then have to go to the effort of emulating them, when you could just use supported methods directly? Is the use of the proper methods really that important? But it does work.

kakoni 12y ago

New Finnish gov digital infrastructure is going to built using Estonian X-road (well next year). And yes, its only SOAP, so I guess either Finns are pretty crazy or SOAP is making a comeback. Go figure.

userbinator 12y ago

One nice example of how MS really made things more complex with SOAP is the MSN Messenger Protocol (MSNP), which started out quite simple and sane, then got extremely SOAPy in the later versions; compare protocol interactions pre-SOAP: http://msnpiki.msnfanatic.com/index.php/MSNP8:Getting_Details#Example_SYN_Responses http://msnpiki.msnfanatic.com/index.php/MSNP8:Getting_Detail... and doing the same thing with the SOAP'd version: http://msnpiki.msnfanatic.com/index.php/MSNP13:Contact_List http://msnpiki.msnfanatic.com/index.php/MSNP13:Contact_List

im2w1l 12y ago
I think this is a great example of SOAP's failing. While it is simple, it looks difficult. As you scroll through all that XML your eyes begin to glaze and you don't even notice the small section at the bottom. Despite being the most important, the section is at the bottom, has weird grammar, and feels like an afterthought. >Here the WSDL and XML schemas for the web service descripted here, you can use them to generate a web service binding for your programming language. >MSN Addressbook/Sharing Service WSDL & XSD files This is how simple it actually is: http://www.diveintopython.net/soap_web_services/index.html http://www.diveintopython.net/soap_web_services/index.html
- userbinator 12y ago
  I wouldn't consider requiring a ton of mostly redundant and useless information in messages (each namespace needs a whole damn URL every time it's referenced? Does the parser use it, or even care that the entire URL is correct?) and XML parsing (which isn't so bad, in comparison to all that redundantly redundant redundancy) to be "simple" at all. WSDL/XSD is yet another layer on top of this mess of complexity and I suppose you're referring to the fact that it looks simple if everything goes according to plan, but when it doesn't it's anything but simple. Trying to get any kind of performance out of it is another counterpoint to "simplicity"... This is all from the experience of someone who has worked with XMLA-based OLAP systems. That link you gave has a summary part ( http://www.diveintopython.net/soap_web_services/summary.html http://www.diveintopython.net/soap_web_services/summary.html ) that says "SOAP web services are very complicated."

fit2rule 12y ago

Why I never used SOAP: Endian issues.

ChuckMcM 12y ago

Hold on to this, "tldr: Simplicity and utility trump large corporate backing." When your manager tells you that your API needs to handle something "just in case", when your customer asks, "But what if I want to use this API with mumblefratz?", and when you look at your code and say, "Gee, if I make this a variable I could handle any special edge case by encapsulating its specialness in the variable ..."

ownedthx 12y ago

The fact that SOAP is not reasonably supported in a browser was a huge reason it fell down. When SOAP first came on the scene, complicated AJAX based applications were not typical. But as more and more JSON/HTTP APIs emerged in conjunction with browsers becoming more powerful and rich apps build built in them, the meaninglessness & complexity of SOAP grew. Also, another major issue with SOAP is that most all of the popular tools would generate classes based on a WSDL. This creates a toolchain issue that is readily solved for someone experienced, but can really suck if you are new to the idea of generated code working it's way into your project. Much worse was the scenario of WSDL versioning in conjunction with these tools that generated classes. If the API never broke backwards compatibility, you should be OK and can just use the newer class representations, counting on the service and tools to deal with null fields appropriately (not always true unfortunately). But if version bumps of an API/WSDL broke backwards compatibility... then you had to have separate class hierarchies for different versions of the WSDL; what an intense headache. Contrast that to web REST APIs; without a formal schema and a lack of class-centric tooling, client libraries would often let the author stuff in the params themselves and let the serializer stuff in the body based on the params; so yes your API is not as formal, but this isn't a big issue but offers infinite flexibility in dealing with one-off versioning issues or interop issues. As someone else mentioned, some platforms couldn't interop with others (differences as it related to simply stuff like nullable fields or primitives existed all over; total nightmare), and some features of WSDLs didn't translate well to certain languages. A great WSDL author would know to make their WSDL as simple as possible, because they had spent time working with various language toolchains and new the limitations out there, but that's asking way too much and not realistic for someone to have to spend all their time trying to understand all the ways someone in language XYZ might use their WSDL.

PantaloonFlames 12y ago
Turns out the "definition language" (WSDL) and the presumption that it would be simpler to deal with than just showing people the messages that needed to be placed on the wire... was the broken assumption. We wanted automagic seriaization and de-serialization and we wanted every language-based construct (linked lists, arrays of structures that contain arrays) to be supported. Different tool vendors built that differently, hence the interop problems. In REST/JSON, we don't have an IDL. We don't have a JSON Schema (yet! thank god) We generate human-readable documentation containing examples, for the definition. And developers build to those examples. There's nothing - no runtime or static tool - that gets in the way of developers serializing their data into JSON, the way they need to.

jcfrei 12y ago

I've had the "fortune" to work with SOAP as well (java implementation) and I never could shake the feeling that it was designed with the intention of being obscure. It seems you could employ an entire batch of consultants just to deal with defining, implementing and changing SOAP APIs (kind of a misnomer I know). The whole change from SOAP 1.1 using an ActionHeader to SOAP 1.2 using an action parameter seems downright misleading. Or downloading WSDL files to automatically create Java classes sounds great in theory but now your program has additional obscure classes with very weird syntax and you have to read through numerous reference guides just to know how you can modify the header of an outgoing request. Such a waste of time.

ebbv 12y ago
I also get to deal with SOAP occasionally. I recently got the joy of dealing with it for a CA's API. It's clearly the result of developers and managers more interested in what the technology CAN do than what it should do.
- ownedthx 12y ago
  In my experience, it wasn't that. Larger companies put value in supporting standards, because it is an important checkbox in their marketing, and SOAP is definitely a standard.
  - ebbv 12y ago
    I was talking about how SOAP itself was designed.
    - ownedthx 12y ago
      ... oh :)

mrfusion 12y ago

I always liked XMLRPC. It seemed really simple. At least in Python they simply mapped to function calls and all of the XML stuff was handled for you. It took minutes to get an API set up with that from any program. I never understood why it's out of favor?

Animats 12y ago

It's all about who has the power. With JSON, if the data returned by the server doesn't exactly match the documentation, it's the client's problem. With SOAP, if the data returned by the server doesn't exactly match the documentation, the server is broken. The client, reading the WDSL file which documents the API, will report an error and refuse to continue. This means bug reports the server provider cannot make go away by the usual defect-denial techniques.

corbinpage 12y ago

SOAP Web Services around still frequently used in a lot of large corporations, where technology evolves a little slower and Microsoft products are still very popular. Being a Rails fan myself, I have always enjoyed using HTTP Web Services and interacting with tech companies' Restful APIs so easily. This post provided a clear, side-by-side comparison of the technologies including the historical context of SOAP, which I found really useful and enlightening. If I could give you some Bitcoin for writing it, I would. Have a new Twitter follower instead. It's hard to find succinct, objective technology articles in this day and age. Most reviews I find are either unconstructively biased ("dynamic typing sucks!") or too long and convoluted to be effective.

cratermoon 12y ago

Let's hope SAML follows soon.

lkrubner 12y ago

Pete Lacey's parody of SOAP is both hilarious and also very accurate: http://harmful.cat-v.org/software/xml/soap/simple http://harmful.cat-v.org/software/xml/soap/simple

jprince 12y ago

The first thing I did as a developer was to write a SOAP wrapper gem for Propertyware's API. It was here that I learned the concept of Pain.

Yhippa 12y ago

I still have to use SOAP more often than I'd like to these days. It's a pain testing and dealing with all the different tools you have to use just to stand up a basic request. A lot of vendors I've worked with will publish REST web services for public data but as soon as any private information needs to be accessed they use SOAP. I don't know if they just don't trust the tools to make REST secure or what but it's disappointing for sure. Also the people who wrote the SOAP code are long gone which makes support for them even more difficult when I'm trying to get bugs fixed. REST server code has not been anywhere as bad to deal with.