18 ms·
90% of software engineering done today is integrating poorly documented APIs
- deleted 5y ago[deleted]
- ladyattis 5y agoI would argue that we shouldn't be writing APIs at all and should be writing constraint and goal oriented programs which can be remotely tested for resource utilization. Basically, we need to stop thinking about reinventing the wheel just because we don't like the wheel we're given and figure out how to adapt it to our uses in a manner that doesn't require explicit human written code. I know this all is just wishful thinking on my part, but it would be nice to just have programs just be a set of constraints and goals as I mentioned early and then have the compiler and runtime figure out the details of execution on its own.
- stakkur 5y agoNo true Scotsman.
- NaughtyShiba 5y agoGoogle SmartLock/Onetap was probably my worst ever experience. Facebook login, while documented, was also terrible due to all the errors and bugs.
- pcmaffey 5y agoThat’s like your choice man. Use services with well-worn paths. Or build them yourself. Make more sustainable architectural decisions and you should be able to get that 90% (lol) number much much lower.
- CitrusFruits 5y agoThat's a pretty big assumption that the developer gets to choose their tooling and APIs. A lot of times you don't have the choice and just have to use what the other 99% of the codebase is using, even if it's terribly documented.
- marvin 5y agoI'll try to do that the next time I'm going to create a banking product combining the services of the reserve bank, the tax authority and the stock exchange in my country of 4 million people! Well, I suppose it's a bit of a choice to work on that kind of product, but an awful lot of economic activity stems from that kind of product.
- tosh 5y agoreminds me of what Sussman said re MIT moving from Scheme to Python for 6.0001 > Nowadays you muck around with incomprehensible or nonexistent man pages for software you don’t know who wrote. You have to do basic science on your libraries to see how they work, trying out different inputs and seeing how the code reacts. This is a fundamentally different job, and it needed a different course. https://www.wisdomandwonder.com/link/2110/why-mit-switched-from-scheme-to-python https://www.wisdomandwonder.com/link/2110/why-mit-switched-f...
- bsedlm 5y agoit really is becoming more like an experimental science than a mathematical science. it's become easier to make experiments instead of deducting things from first principles... What's really surprising, is that everything about the program is constructed from first principles! It's kinda like if math became 'math archeology' i.e. imagine if maths were really about trying to understand clay tablets and papyrus writings in order to do calculations.
- curuinor 5y agoI guess we're all the Mechanicus of Warhammer 40k now
- v-erne 5y ago>>it's become easier to make experiments instead of deducting things from first principles... Is this really suprising for You? I mean I always believed that our deduction skills as humans were really limited and primitive. Just look at conveys game of life or any game as a matter of fact (chess?). The emergent properties of those games with few basic rules are so elusive that we need generations od geniuses to work on them to discover anyhthing beyond simplest things. How can we expect anything better when the rules gets really complex and fuzzy (as in real life)
- TeeMassive 5y agoWe're becoming more and more like the Adeptus Mechanicus from Warhammer 40k. The secrets of the technology we are using lost to time, using "prayers" in the hopes one sticks.
- ec109685 5y agoSilly tweet. Most spent building UI’s and business logic.
- Jensson 5y agoAsk yourself, why do you need programmers to build UI's? Poorly documented API's, there is no reason you'd need programmers to build UI's otherwise. Either the API the UI is working against is so poorly documented that only a programmer can understand what to do with it, or the UI tooling itself is so bad that only a programmer can understand it.
- Ekaros 5y agoIsn't UIs at least on Web essentially integrations towards APIs? Which are poorly documented...
- thesausageking 5y agoAusten runs a coding bootcamp. I'd bet this is true for his students.
- hkt 5y agoResponding only to the title: not true in my case. In my case, 100% of software "engineering" is this.
- photochemsyn 5y agoI don't really think a lot of what goes on under the umbrella of 'software development' can really be classified as 'engineering'. Some of it certainly can, but once you reach a certain point of abstraction away from the underlying hardware, it's more like 'composing' than 'engineering', or if you want to have a more prestigious label, 'architecturing' or maybe 'designing'. https://study.com/academy/lesson/what-is-engineering-definition-types.html https://study.com/academy/lesson/what-is-engineering-definit...
- selfhifive 5y agoAgreed. It feels like fitting pre-cut puzzle pieces. Reading code/documentation has become the most important skill. The amount of working software engineers who don't understand basic things like paging or the TCP/IP protocol or file systems is unnerving.
- dotAJ 5y agoWhat could be the reason though? Too much abstraction these days? I have worked with experienced devs that have a background in Computer Science, who don't grasp basic concepts (especially when it comes to computer networking). Could it be that the emphasis in current CS education is more on theory rather than application in the real world? These days, there are more Software Engineering curriculum style studies popping up at Universities that get rid of the hardcore academic CS fluff you'd only need if you plan to stay in academia, and focus on real world applications. Perhaps that would help more.
- abledon 5y agotheres too much stuff to memorize nowadays... its getting more complex every decade. e.g. If you have to load up in your brain XYZ security details about a certain cloud provider, and also understand caching, scaling, load balancing etc in a cloud environment, why would the person actively keep their 'OS paging' knowledge fresh the mind, its wasted real-estate. maybe certain developers with photo-graphic memory can do it, where they never forget _anything_ they learn. but most developers I work with have forgotten what they haven't used in 5-10 years
- alexashka 5y agoSure, that we all know. Go ahead and answer why and what you're doing about it.
- jseban 5y ago90% of my software engineering is fighting with incomprehensible framework magic to automates simple tasks that don't need to be automated, to "save me" from just writing some simple code. It's literally rube goldberg all the way, the most over complex contraptions to achieve the simplest of tasks. "You don't want to write "raw" html/js/sql/Java" Uhm I thought that was my job.
- ozmodiar 5y agoThis strikes a nerve with me. Lately my work has started taking apart our website and shifting functionality to 3rd party vendors in an effort to limit liability. We're going to end up with something far less versatile, uglier and ironically less secure than what we started with. My focus is shifting to a bunch of API stuff that I just don't find very rewarding.
- omginternets 5y agoI tell every junior colleague two things: 1. There is software engineering, and then there’s working at the digital factory. 2. A good way to set yourself on the path that leads to the former is to get intimately familiar with the HTTP protocol. A good way to start is to reject fancy frameworks, and to get comfortable reading RFCs. Being the guy who really knows HTTP is a powerful career move. Becoming that person almost inevitably kicks off a virtuous cycle that steers bright/motivated devs well clear of writing REST endpoints for the rest of their working lives.
- urthor 5y agoIt does feel like the barrier to entry for that kind of deep knowledge career is increasing these days. Maybe it's because I live outside the United States, but it seems like it's much harder to build a "pathway" into work that demands that intensive, deep understanding of engineering on computers. At least one that'll pan out in my 9-5. The supply of problems that require deep knowledge vs framework Kung Fu... my impression from ground level is they're tilting more towards GitHub martial arts every year. The supply of companies outside the United States with a deep interest in these topics is sparse, to say the least.
- 5y ago
- likortera 5y agoNah, that's just 20%. The other 80% is dealing with the Scrum Agile bullshit.
- kps 5y agoIt's ‘Agile’ as in ‘People's Democratic Republic’.
- tinco 5y agoIf you phrase it that way, you make it seem that somehow the poor documentation has anything to do with it. It does not and thinking that wat leads to horrible mistakes like HATEOAS. 90% (not really, but some arbitrary and probably admittedly high percentage) of software engineering is integrating API's. You could state that as a problem but it's not really. Integrating API's is just a highly effective thing to do as a programmer. Consider the alternative, if 90% of software engineering is writing things from scratch that likely already exist elsewhere. Integrating API's is also not a trivial sort of cost that we need to eliminate from our daily work to be more efficient. Integrating API's requires the same sort of deep thought and careful consideration as do the other types of programming we do in our profession. Sure it could be made easier often, but really the part where we have to figure out the finicky details of some API is not the hard part. The hard part is figuring out if and how we can make our use the API within our requirements. That doesn't change even if all API's were some click and drag Lego block system.
- sto_hristo 5y ago... to 0% documented enterprise applications. But, you know, it's "clean code", right?
- moltar 5y agoIt was always like that though. It’s just now “API” means “an HTTP endpoint serving JSON”. And before it was SOAP. And before it was MS COM. And before it was some mainframe app in COBOL.
- nix23 5y ago>And before it was some mainframe app in COBOL. Yeah but at least you could speak with that guy over phone...for the next 3 days, and often even get the source-code aka tapes ;)
- throwawayboise 5y agoDon't forget CORBA. Actually on second thought, please do.
- thom 5y agoI suspect this is true, but it's worth noting that often we're in control of poorly documented API A & B in the first place, and could probably have avoided the much of the pain of later integrating them.
- civilized 5y agoAlmost perfect, just need to remove that extraneous "today".
- is_true 5y agoYoutube, thanks for all the fish
- adamnemecek 5y agoToday? You don’t think that when people were writing kernels back in the day that they were not dealing with CPU “APIs”? Etc etc.
- zwieback 5y agoI'm part of the remaining 10 percent.
- thrower123 5y agoAll I can say is thank god most of the stuff I work against has the source published on Github, and most of the rest is in decompiler-friendly languages. This leaves the completely inscrutable and mostly undocumented REST apis that I've got to trial-and-error at, and always ask whether I did something wrong, the other party made a breaking change, or its just a Tuesday and the infrastructure is derped up.
- conjecTech 5y agoI think the emphasis on the documentation is a bit of a red herring. Sure, documentation for most things could definitely be better. But when you are building a system that is a composition of subcomponents, the bulk of the complexity is almost always going to come from the interactions. Those aren't things that can be readily captured in the documentation of any single component. This conclusion is actually pretty obvious when you consider it from a testing/dimensionality perspective. If you have 2 components with N inputs, an interesting application using both may have close to 2N inputs. Sounds pretty tame, but the size of the space grows _exponentially_ with dimension. And for a bounded space, almost all of the volume will be near the surface, which would mostly correspond to non-trivial combinations of the inputs. I think we tend to lump most of the code needed for this complexity into "business logic".
- pc86 5y agoThe system should have its own documentation though. It's fine if that documentation is basically just explaining all the interactions (especially if that's the bulk of your system). But that doesn't mean the individual components shouldn't have good documentation.
- saiya-jin 5y agoAh, the variant of classic 'code documents itself'. Which is true unless you come to a dark place where stuff stops working randomly and you have no clue why. Or you interact with other systems which are just blackboxes for you. Its fairly trivial to write on few lines piece of (bad) code that is really hard to grok for anybody else and even for you after few months / years. What good documentation does, and it can take just few lines here and there, is explain why things are supposed to happen or not, expectations, corner cases etc. Haven't heard any good argument against that in past 20 years and doing this myself diligently even on code that is purely for me. Always thanking myself when debugging something old.
- conjecTech 5y agoI didn't mean to diminish the value of documentation. It's hugely valuable. My point was that even if the documentation for all systems was good, the bulk of work in a mature software environment would still be integration, since that's where most of the complexity emerges.
- awinter-py 5y agobig reason for success of nocode tool space imo -- there's no standard for full stack plugins unless you're using django/rails in like, 2013 with no frontend JS and no external services API doesn't mean 'drop-in compatibility', whereas nocode is designed to integrate with everything in your stack with relatively low lift by the buyer
- Zigurd 5y agoI'm currently working on a communications product that spans a lot of on-device APIs to use them in interesting ways and in novel combinations. Part of the challenge is that complete documentation of behaviors is very difficult. Some behaviors depend on OEM and MNO choices. Some depend on drivers and hardware. They all depend on user choices for enabling various radios on a device. All of these choices can shift over time, and not just on OS updates. Even running hardware/firmware changes on a particular model can change behavior. Many of these things will appear in different combinations on new devices and different user environments. Internet APIs have fewer excuses, so challenges are more annoying in some cases. But it's always challenging to build apps that are resilient to loosely specified behavior. Such is life.
- mlindner 5y agoAs a notable side point, much actual mechanical-related (and other) engineering is integrating poorly documented physical parts. That software engineering is finally getting to the point where you open up the part catalog and pick your parts is a sign of maturity in the industry.
- remoquete 5y agoHire API technical writers. Involve them in API design processes. A developer portal should not be an automated afterthought.
- Terry_Roll 5y agoDocumentation can be like going to college and university and what I mean by that is it can blinker your thought processes. The best way to learn is to see what errors api's & language functions throw at you. Then you know where it wont work! The people who docs typically also dont code, but those that do code and document tend to produce the best docs, but its not a given.
- FpUser 5y agoI remember doing my first electronic related projects using discrete transistors and other elements. Now it is mostly microcontrollers, integrated circuits and blocks. Which is a very good thing. The thing I could do now in a matter of days would take months / years to implement using old approach. Poor documentation is bad of course but a fact that one can assemble sizeable project integrating various existing APIs / libs holds huge value from a productivity standpoint.
- mountainriver 5y agoGood APIs are incredibly hard to build because your always fighting upstream against the feature farm.
- unfocused 5y agoI read that comment as: Nobody is hiring technical writers to do documentation. Side question: Are companies even hiring technical writers, or is it something most budgets don’t account for?
- ChuckNorris89 5y ago>Are companies even hiring technical writers? In more traditional tech businesses, where customers expect proper documentation, YES.
- drpixie 5y agoMost companies - no. There is very little tech writing and of that, it's of very poor quality. It's now normal for API documentation to be generated from the definitions, with a few explanatory comments thrown in (eg. see the Android API doco.) I've even seen this regarded as a good thing! The software makers don't "waste" time, and they've created an "opportunity" for 3rd parties - an opportunity for someone to write a "Dummies guide to XXX".
- foobiekr 5y agoThe networking industry hires a ton of tech writers to write manuals. Admittedly the quality has dropped like a rock over the last 10 years.
- notreallyserio 5y agoAnd IME 50% of that is dealing with generic error messages, or worse completely silent errors.
- phkahler 5y agoHow many of those APIs are actually wrappers around some database tables? This could be replaced with ONE API, some access controls, and a published schema. That is until someone makes a table of "issued API commands" and then we've come full circle with more complexity.
- zozbot234 5y ago> This could be replaced with ONE API GraphQL?
- purplerabbit 5y agoYou might be interested in thin.dev (no affiliation)
- mkl95 5y ago95%+ of SaaS engineering seems to be some form of that. I've been sick of it since day one but it's where the big bucks seem to be.
- a_c 5y agoSkimming through the comments, since when does API equate with HTTP API? Aren't we always integrating some API? Standard library anyone? What is poorly documented anyway? Put it another way, what is NOT an API?
- barnabee 5y agoI guess the interesting question is the ratio of “glue” and layout code to business logic and algorithms. There’s definitely too much glue (and somewhat relatedly, too much branding and unique “design”) in software, IMO.
- thenerdhead 5y agoIsn’t that what software is? Framework APIs, HTTP APIs, Hardware APIs, SDK APIs. Is the tweet author suggesting crud or web APIs only then? I think it’s always been a meme that most software work is glorified CRUD.
- Animats 5y agoRight. The hard line on this comes from aerospace. If A won't connect to B, you check the spec. If A isn't compliant with the spec, A is wrong and the vendor of A has to fix it. Same for B. If you can't tell who's wrong from the spec, the spec is wrong. This is why you can unplug a Pratt and Whitney engine from a aircraft and plug in a Rolls-Royce engine. Now, this worked a lot better when aerospace companies and the USAF had real clout with manufacturers. If it wasn't in compliance with the spec, you put a red tag on it and rejected it. The supplier didn't get paid. The way it works today is that if the supplier is bigger than you are, it's your problem.
- drewcoo 5y agoAnd that kind of spec'ing and compliance is ultimately rooted in legal threats: loss of life leading to loss of money. Until software actively kills people (enough that it costs someone a lot of money), we won't see movement in this area.
- protastus 5y agoTo your point, most software comes with a license that says "this comes with no guarantee for any purpose, we are not liable under any circumstances". I remember reading licenses as a teenager and wondering about their legality.
- YeGoblynQueenne 5y agoThat's how computer hardware used to work: see "IBM PC". I have no idea if it still works that way. I haven't seen the insides of a computer for ages now. Hardware is constrained by the costs of manufacturing at scale. I can't make my own computer as I want it (can't find a hand-cranked punched-card reader, or a steam-powered dot matrix printer. Oh why). With software though, everyone can write whatever they like and send it out there were someone is bound to pick it up and reuse it. A little like the way Covid-19 spreads, I guess. No constraints, no respect for specs.
- javajosh 5y agoThe problem of software specification in that comprehensive sense is unsolved.
- arealaccount 5y agoI work at a business thats in the business of connecting APIs. Id say the bulk of the work is the tooling around these in a generic enough way where it makes sense for our processing. IMO this is a fun thing to work on. Some dev frameworks on the other hand that turn 4 lines of CORS directives into 5 lines of library code + 14 dependencies annoy me.
- flappyeagle 5y agoThis is simply untrue. It’s not even true in spirit. It ignores all UI programming, game development, and line of business code. This might be true for an extremely narrow band of startup and consumer facing web programming. It might be true for the generally low quality programmers getting churned out of Austen’s bootcamp, because they can’t do much more than the stated activity.
- dragonwriter 5y ago> It ignores all UI programming, game development, and line of business code. It is certainly not the case that UI programming and LOB code categorically does not involve integrating poorly documented APIs. I'd be surprised if that was true categorically of gamedev, either.
- barbazoo 5y agoIt's just someone trying to find the next edgy tweet to please their followers. Making such broad generalizing statements in a few characters is rarely successful.
- tomatowurst 5y agosame can be said of the parent's comment. just say something contrarian and not back it up.
- onlyrealcuzzo 5y agoUI programming - let's take websites - the DOM can be considered an API. Game development, using literally any pre-existing function can be considered an API (libraries are APIs). This really boils down to saying - the average line of code is used many times. How many, I guess, depends on how poorly designed the APIs are. If the average function takes longer to use than it did you write (I'm skeptical) - then it would be <9 usages per line. If it takes less time to use a function than to write it - then it would be >9 times.
- tomatowurst 5y agoyou haven't worked in the industry long enough if you haven't encountered poorly constructed/documented API integration from vendors. the ideal world you described does not exist on this planet.
- Ericson2314 5y agohttp://langsec.org/ http://langsec.org/ Interfaces are more important than implementations. That the "implementation" are just duck taping other interfaces together proves the point!
- foobiekr 5y agoIt’s the other way around. Implementation is and always has been everything. Performance, security, reliability, efficiency - these are all implementation aspects where the same interface can cover a world ranging from total crap to excellence. Langsec is something I strongly agree with btw, but it’s not interface-supremacy and the world doesn’t look like that.
- Ericson2314 5y agoI'm not disputing the implementations are more lines of code than interfaces. What I am saying instead is interfaces and composition needs more attention. Shitty APIs are a lack of attention to interfaces. Shitty marshalling ultimately stems from a lack of attention to competitionality, and the cost is that people are incentivized to commit anti-lang-sec sins. > these are all implementation aspects where the same interface can cover a world ranging from total crap to excellence. This is narrowly true, but completely misleading. You have to focus on incentivization and emergent behavior. Good interfaces can be used safely by lazy people. Bad interfaces will never be used will by the most careful people. Every implementation tests in interfaces below it, and so quality will percolate upwards.
- goodpoint 5y ago70% integrating APIs thanks to SaaS. 30% fiddling with yaml files thanks to modern devops tools. The remaining 0% is real software engineering.
- deleted 5y ago[deleted]
- Groxx 5y agoTrue or untrue in the same way as saying 90% of art is not new. Or many things. Almost everything humans do isn't actually novel. It's remixing things we've done before, overwhelmingly in almost-but-not-quite-exactly the same way as many others. tbh I think that's fine. But it can be a bit depressing when you first realize it about something you care about.
- 1vuio0pswjnm7 5y agoThere should be a web of 100% JSON^1 endpoints. Let users choose their own clients. The JSON could be used in any way the user chooses. Maybe they want to make simple HTML from it. Or maybe they want to embellish it with remote Javascript-powered interactivity. It could be their choice. What I see now as an ordinary end user, not a biased web developer, is myriad websites that are mostly just remote Javascript-powered shells for generating requests to JSON endpoints (and, via remote Javascript, doing nasty stuff with advertising and tracking). As someone who prefers text-only, more and more, albeit slowly, the "real" web is becoming JSON endpoints, not the HTML pages with links to remote Javascripts and no useful data. To me, "poorly documented APIs" sounds like a euphemism for people hiding public data behind a gauntlet of opinionated and manipulative design geared toward optimal advertising and tracking. They are not focused on providing simple, direct access to public data. They are more focused on dictating "UI" and "UX" and forcing acceptance of advertising and tracking. Overall though, I think the rise of endpoints is good. Hopefully, in time, the Javascript and other remote, unwanted cruft will fall away. Maybe someday the web will become a more efficient means of sharing public data. In the meantime, we have Gemini and other alternatives. 1. Or a better text-only format.
- 1vuio0pswjnm7 5y agoThe important idea here, IMO, is that users are not effectively restricted to choosing from amongst a small handful of clients issued by the same companies that sell online advertising services or otherwise profit from user data mining, online ads and tracking. IMO, it is this effective control over the client that is the root cause for so many of the complaints HN commenters have about the www today and the direction it is heading, namely because such control over the client itself has become a necessary in the user-hostile strategies of these corporations. They are now relying on that control.
- zzo38computer 5y agoOne idea I have is <link rel="data"> (or <a rel="data">), if your web page is a data-oriented web app. You can include it in your own web pages if it is appropriate to do so. (This way, you can have a default view but allow the user to choose their own instead if they wish to do so. Comment 31228726 mentions they have to have empathy to ordinary users; <link rel="data"> will allow it to work both ways, I think.) (It could use any format, whether JSON, CSV, SQLite, etc; you can indicate this as the type attribute. And, you can have more than one, as different types.) For API documentation (or data format documentation), you can have <link rel="documentation"> (or <a rel="documentation">) too; if you limit it to a simpler subset of HTML or to plain text, then it can be easily read (or printed out, if wanted) in simpler viewer programs too. I have other ideas too, such as <html application="...">; you can use this to identify instances of a web app that may be on multiple servers; this can be used for user CSS and for other purposes (e.g. substituting native code if the client has it in order to improve efficiency of computation, or using software that the end user has installed (or has written and/or customized) specifically for this use). And then there is the "feature" attribute, to indicate that it implements a feature that the end user might already have in which case it can be ignored (I had written a list of some of these, e.g. feature="usercss" and feature="uploadname"). Another idea I have is a Interpreter header, but that will require that web browsers implement it (unlike the other three things above, which are backward compatible). This can allow to serve any file format and it can be displayed in the modern kind of web browsers while simpler ones could also display them in their own way; it can also reduce the number of requests required for some things, improve caching in some ways, allow polyfilling video/audio codecs in a cleaner way, etc. (It could also allow a user to install command-line programs to convert a file if you are using WebAssembly with only a small subset of the Interpreter API.) There does need better web browsers written, too. Half of the things will need to be changed (often, implemented differently than the specification says; e.g. do not allow enumerating devices but if it is requested, the browser might say request audio input enter a system command to produce the audio (which might be one that converts a audio file to stdout, or that converts audio from a microphone to stdout, etc)); better way is a way that is meant for only advanced users, assuming that you know what you are doing and better interoperability that user can specify UNIX pipes, etc, without trying to stop you from doing things; another thing that would be helpful is the ARIA view mode. Some things were also mentioned by FSF, such as that the end user should be allowed to substitute their own JavaScript codes in a web page; I think that it is also a good and necessary idea. The web developer console is already a good end-user-oriented feature, but by itself it is not good enough. Having other web browsers of course does not mean that all users will have to use it; we will need to have the choices and controls!!! (The ability for the end user to write native extensions in C (and run them as native code) would also be helpful; WebExtensions are too limited, and might not be as efficient as a C program.) It would seem needing to write a new browser in C, but it might be possible to modify existing ones (Netsurf seems the best idea at this time but it might not be good enough), but that would require changing half of the program, and might need to maintain and pay attention to changes to decide if you will include those changes, and to get them to work. Furthermore, other people can make providing the API endpoints for many things even if they do not have, and to be not too complicated, that then you can use your own. When there will be a lot of them, then you may be able to access using SQLite virtual table extensions (even accessing two sources at once in a joined query, if wanted), Gemini, Telnet, Gopher, JSON, CSV, NNTP, IRC, etc. Even could be made the program that supports such feature too.
- deleted 5y ago[deleted]
- anothernewdude 5y agoMicrosoft's poor Azure documentation is keeping me in business.
- morelisp 5y agoWell, it certainly will be if you hire his graduates.
- hintymad 5y agoThere are two ways to overcome this misfortune. One, work on product features. Yes we'd still integrate poorly documented APIs, but we'd innovate on solving new and touch problems, and our technical challenges come from managing complexities and building new concepts. Two, go deep and focus on productivity. Yes we'd still integrate poorly documented APIs, but along the way we identify opportunities to build something that is meaningful and makes our lives better. Our technical challenges come from getting unusual insights to optimize something seemingly impossible.
- lr 5y agoThis is why I still use LDAP. And I spend my days at work happy.
- larve 5y agoI don't resonate with this at all, and think this attitude is toxic. To take the engineering analogy: this is like reducing civil engineering to "90% of building a bridge is making shapes out of cement of varying quality". How you use cement of varying quality to build a bridge up to spec, that's the engineering part, and it's hard. Cement will always be of varying quality, because it's not unobtainium that you can mold into any bridge you want without effort. Noawadays, I am in the business of writing CRUD e-commerce applications, using PHP, wordpress and jquery of all things. In the past 20 years I've worked on anything from bare-metal embedded to building search engines and doing "decent scale" data engineering. I'm basically dividing software engineering in two broad categories. One is what I call "copying bytes". You get some business relevant input, and as a software engineer, your job is copy those input bytes to their business relevant destinations. For example, you receive an order from the customer (line items + shipping address), and you have to copy those line items and shipping address to the fulfillment department, recording a few log entries along the way. You might calculate some tax or apply a coupon along the way, slightly modifying those bytes, forward a few tracking metrics here and there, but roughly, you are just wrapping these bytes making up the order in fancy ways. APIs for "copying bytes" are one of: * "if you call readBytes, I will return someBytes", * "if you call writeBytes, I will store some bytes somewhere" * "please tell me when someone calls writes some bytes" * "hey, someone just wrote some bytes" The other is what I call "inventing bytes", or "transforming bytes in really complicated ways". This is the realm of funky algorithms, data science, mathematics, complicated stuff. This is taking an input 3D model and outputting some CAM instructions. This is taking a search string and computing features to quickly locate relevant information in indexes. The coding part of this style of software engineering is the tiniest part of the actual undertaking. Often, the hard thinking is done by people at universities or research labs. One could wonder if they're software engineers at all, and not "3D model transformation engineers", or "search index data structure engineers", as often their code needs to be rewritten by someone who knows how to do it pragmatically (say, port from the ugliest hacky matlab to java). The APIs here are weird, and frankly I don't really ever deal with them, so can't say too much as to their quality. Chances are, you fall into the "copying bytes" category. Which bytes to copy, when, where, what log entries to add along the way, how to scale the copying, how to ensure the copied bytes are transactionally aligned with other copied bytes, that stuff is complicated. But it's turtles all the way down, one copyBytes API upon another, until you hit the hardware, and boy are the hardware APIs to copy bytes utterly nonsense. You might think it's inane. You learned how to balance trees, and how to schedule processes, and how to do some differential curve geometry, how to write compilers and how design database engines, how to implement distributed consensus, and why don't you get to write all those beautiful algorithms as part of your day job? Thing is, clever people have already done so, and often they made a business out of it so that you can pay them for their clever thinking, hoping that they were indeed more clever than you, and save you time. If you are not in the business of thinking clever ideas or making pretty things like video games, but in the business of making money, you will probably just write code to copy bytes, using code from other people that also copies bytes, until those bytes finally reach the rare unicorn that makes better bytes out of them. (here's a secret: if you do adtech or e-commerce or banking or embedded software, that actually doesn't happen. The business value of your software is to copy bytes to the right destination at the right time). People want to transfer data for all kind of different reasons, in different ways, with different constraints. That means that even the best written API documentation will have to be generic enough to cover what was intended, and with some chance, what people are using it for. Most probably, it won't fit your use case to a T, and you call it "poorly documented" when really it means that people have different problems than yours. I actually think API documentation is insanely better than it ever was. All the APIs I had to deal with lately come with copypasta for 15 languages, a snappy searchable index, a plethora of tutorials and articles and videos. But here's why I find my job copying bytes with subpar APIs singularly fulfilling: Copying bytes is not easy, especially in today's distributed world. Doing it well and reliably is a tough problem. Figuring out how to do it reliably, securely and at scale requires a lot of thought. Figuring out which data to transfer to whom needs business understanding, knowledge of people and systems thinking. It requires planning a project, managing teams, dealing with changing requirements. It requires planning for maintenance, keeping a codebase healthy, mentoring other engineers, aligning with stakeholders, dealing with legacy code. The APIs I ultimately use are the least of my problems. Some are nice, some are terrible, some are really problematic, but they don't have much to do with the "engineering" part of my job. My engineering is drawing pictures with boxes and arrows. The boxes are other people's systems, the arrows are other people's APIs. What arrows I draw and where they point to and what the little bubbles along them contain, and which boxes I chose to use, that's what I do as an engineer. As a concrete example of a hard problem, fully implemented by glueing together shoddy APIs: fulfilling an order on an e-commerce store. * seamlessly show payment process and checkout status to the customer using a slick UX in their browser (APIs: some shoddy REST, some janky web socket or whatever, some hacked react and some dubious CSS. Ultimately just shoveling the status code from the payment processor to the user's browser, potentially adding some curly braces and escaping a string or two.) * log the transaction and provide metrics and analytics to business and ops (some inane pixel data, some putrid tracking JS, some shoddy SQL, maybe some janky JSON going to a SQS queue) * process the payment information, order information and display it in the customers account section / backend CRM / fedex (I'm not even going to list how subpar the APIs are here) * transform the shipping address into a proper shipping label with barcode (have fun with zebra and ZPL) * dispatches the line items to the required fulfillment centers, provides picking information to the warehouse workers, with tracking, metrics, inventory handling, ... Designing something like this so that it works correctly, efficiently, with visibility, reliability, scalability, and fulfills business needs, and sparks joy for every human involved. That's engineering.
- sandruso 5y agoWhen I saw this I thought about what S. Jobs said once: something like “We think about problem and work out the tech for it backward”. If the best solution is to use some APIs then go for it. The focus should be on the problem. We like tech we are using but these are just tools like any others.
- sjg007 5y agoProgramming is an onion, it has many layers of abstraction layered on top of each other. You can work at any level you want but it helps to understand how the onion peels. In many senses there are programs that write programs and in that regard all of our jobs are automatable. The higher levels are closer to "no codification" than the lower ones at this time. Remember when we had web developers writing HTML? Now it is JavaScript. As you learn to code together APIs, use frameworks, take the time to peel the onion and learn how it all works.
- deleted 5y ago[deleted]
- genousti 5y agoNode js recreated the dll hell in the browser
- BeFlatXIII 5y ago100% this. It's entirely impedance-matching rather than building anything worthwhile for 90% of the day.
- intesar 5y agoI agree 90%.
- stjohnswarts 5y agoCurrently working on "the code is the documentation" project and it's a fucking slog.
- galaxyLogic 5y agoThis seems so Twitter. No link to a resource that would even mention a study that says it is 90%, rather than say 10% . Ok it might be that 90% of YOUR work is integrating poorly documented APIs, then why don't you say so. Or say "I think it may be the case that ...". This bothers me because I've seen similar Twitter posts being discussed here where the Twitter post is clearly the writer's opinion but presented as a numerical fact. Is it that people who communicate over Twitter think their opinion is the truth therefore no need to present any (links to) evidence? Does Twitter make people dumb, or just attract dumb people :-)
- rsanek 5y agoIs it being presented as a numerical fact? I think it's actually fairly clear that this is just the author's opinion and is not meant to be taken literally. In general things I find that signify this include using very round numbers, mentioning it as the title in a longer piece that is clearly just opinion, and not including wording like "research has shown" or "we know that." That said, it would be great to get some actual data on this. Seems kind of hard to measure though -- while my personal projects do indeed have a good amount of API integration work, I'd find it hard to exactly say what percentage. And how are we even defining an API? Are we including eg a language's standard library?
- rsanek 5y agoIMO this whole thread hinges on how one chooses to define an 'API.' You can certainly make it true if you define it broadly enough. 0. Drag-and-drop / visual programming tools as APIs 1. Libraries/frameworks as APIs 2. Standard libraries / built-in abstractions of a programming language as APIs 3. Programming languages themselves as APIs 4. IDEs/editors as human-machine APIs etc.
- kumarvvr 5y agoDRY has a cost. And that cost is things like this,and things like rigid frameworks, rigid libraries, etc.
- legerdemain 5y ago...And Austen Allred's Lambda School (or however they rebranded to escape the terrible publicity) will 100% teach you how to do that and nothing else, for $30,000.