7 ms·
API Discovery: Can we do better?
- detaro 10y agoSemi-related question: is there a list of things that actually use Schema.org, outside of Google Search for some things?
- Meic 10y agoSchema.org is a joint effort between Google, Microsoft, Yahoo! and Yandex (all search engines) so that is going to be the primary use-case. But their FAQ section says other non-search-engine organisations may join later.
- virmundi 10y agoMany recipe websites use it. Allrecipes.com has it buried in the tags. One interesting thing about it is the inconsistent application of fields. Units of measures are sometimes cracked out into their own fields. Other sides clump them together. To fix this, we need to use a bit of NLP.
- asbjornu 10y agoJSON-LD[1], a way to express hyperlinks (linked data) in JSON documents, strongly encourages the use of shared vocabularies instead of every API being their own snowflake with their own private vocabulary every developer need to invest a lot of time into understanding. Schema.org is one such vocabulary. Not having to read documentation to understand whether the "table" property in the JSON document means "coffee table" or "tabular data" is going to save developers worldwide umpteen man-years. Still, most API developers continue creating snowflakes with proprietary designs, protocols and vocabularies. JSON-LD is a large step in the right direction by making it easy to share vocabularies. Hydra[2] takes the next logical step by standardizing the protocol as well, so no matter what your API does, it has one standardized way to express it. [1]: http://json-ld.org/ http://json-ld.org/ [2]: http://www.hydra-cg.com/ http://www.hydra-cg.com/
- oneweekwonder 10y agoI wonder if the author knows about Swagger specs; They also started the OpenAPI initiative[0]. I now have wrapped a couple of API's in swagger and using swagger-ui to create beautiful interactive api docs for users. Moving forward I'm going to wrap most of our legacy systems around a swagger spec'ed API and discontinue any alternative access to these systems as time goes by. [0]: http://swagger.io/introducing-the-open-api-initiative/ http://swagger.io/introducing-the-open-api-initiative/
- IvanGoncharov 10y agoHi, I'm the author, and I definitely know about Swagger/OpenAPI. By coincidence, I'm maintainer of the collection of 250+ Swagger specs for public APIs: https://github.com/APIs-guru/openapi-directory https://github.com/APIs-guru/openapi-directory But I have learned the hard truth over last two years: API catalogs aren't scalable solutions for API discovery. That's why I'm pushing this. P.S. it is pretty easy to generate Schema.org type based on Swagger/OpenAPI spec.
- alemhnan 10y agoWould you mind to share a link for that? I am very interested as I work on different projects with swagger apis.
- romanhotsiy 10y agoHere is the main site: https://apis.guru https://apis.guru And here is github repo: https://github.com/APIs-guru/openapi-directory https://github.com/APIs-guru/openapi-directory
- alemhnan 10y agothanks a lot!
- oneweekwonder 10y ago> I'm maintainer of the collection of 250+ Swagger specs for public APIs: https://github.com/APIs-guru/openapi-directory https://github.com/APIs-guru/openapi-directory Ah cool, thanks for the link and effort. Will have a look at it as time allow. > It is pretty easy to generate Schema.org type based on Swagger/OpenAPI spec. Do you have any example or gist of a snipped of code to accomplish this? > API catalogs aren't scalable solutions for API discovery. That's why I'm pushing this. But does the API catalog not enable auto discovery? Agreeing you maintaining a list is not scalable.
- BoppreH 10y agoWhat about https://schema.org/APIReference https://schema.org/APIReference ? Doesn't seem to be widely used, but exists.
- sapeien 10y agoI think HTTP-based APIs are closer to a service than an library with a public interface, even though the term API can be used interchangeably for both: https://schema.org/Service https://schema.org/Service
- ehnto 10y agoI put together a side project some time ago that allows you to retrieve and display arbitrary API endpoints, to create a dashboard of realtime data. One of the bigger challenges was finding appropriate APIs to use. The idea being that it had to make sense as a realtime datapoint. Current temperature in California is one example. Part of the solution I tried for the end user was to add a searchable database of APIs, but I never found many that had that kind of realtime datapoint. The majority of use cases for APIs seems to be CRUD apps, authentication, unchanging datasets, or really specific once off data transforms. I feel like that's a bummer, as I really love the idea of being able to build up a dashboard of random data points about the world that interest me. The number of astronauts currently in space, the temperature somewhere I like to visit, the traffic density on specific roads. All retrievable individually from various apps, but not in one scifi intelligence center style place. For anyone interested, the site was https://apiblocks.com https://apiblocks.com
- z4chj 10y agoThe best part about this site is the API database link in which you can find a Ron Swanson quotes API. Thank you for that entertainment
- sapeien 10y agoThis looks really great for discoverability that works within the web platform already, without the need for inventing a new format [0]. However, I'm not sure if this goes far enough. There exist some formats for describing what APIs can do in natural language, though I'm not convinced that they're really that useful [1][2]. [0] http://apisjson.org http://apisjson.org [1] http://alps.io http://alps.io [2] http://restdesc.org http://restdesc.org
- welder 10y ago> Then you automatically assume it’s a REST API using JSON payloads. Who’s doing anything else nowadays? I'm seeing people using GraphQL[1] more these days, which also solves the problem of API schema discovery because it's one endpoint with built-in introspection[2]. [1] https://graph.cool/ https://graph.cool/ [2] http://graphql.org/learn/introspection/ http://graphql.org/learn/introspection/
- IvanGoncharov 10y agoIt was a joke to make article easy to read. > people using GraphQL[1] more these days I fully agree. That's why we also maintain a list of GraphQL APIs: https://github.com/APIs-guru/graphql-apis https://github.com/APIs-guru/graphql-apis > solves the problem of API schema discovery because it's one endpoint with built-in introspection You're absolutely right. We plan to do a few interesting projects around GraphQL. Subscribe to our blog to not miss announcements :)
- dmd 10y agoHere in "enterprise" people are still building SOAP apps, sadly.
- orliesaurus 10y agoI am pretty sure there are a few API marketplaces out there where its easy to see pricing and code all at once, without having to browse around like a mad person, just like amazon has done it for shopping, with reviews, ratings and q&a.
- deleted 10y ago[deleted]
- ewittern 10y agoWe are working on a IBM "catalog" at IBM Research called API Harmony: https://apiharmony-open.mybluemix.net/ https://apiharmony-open.mybluemix.net/ One of our goals is to mine information about APIs, rather than relying on user input (hence the "catalog"). We still have a long way to go, though. In Web API land, many things follow a power-law distribution: for a few well-known APIs, a lot of information can be found (https://apiharmony-open.mybluemix.net/apis/instagram https://apiharmony-open.mybluemix.net/apis/instagram). For many other APIs, information is sparse and hard to mine. Adding Schema.org (or similar) annotations would help. But it may be as hard to convince providers to do this, as it is to convince them to write OpenAPI specifications (or similar).
- Senji 10y agoHaving a problem with industry acceptance? Throw the government at it. Either mandate or give tax breaks for following standards that the govt decrees you must meet. When you can't use the government make a conglomerate of large companies with key hold on industries.
- miguelrochefort 10y agoThis is stupid. The solution is the semantic web.
- sctb 10y agoThis doesn't count as a civil and substantive comment on HN, which we've asked you for multiple times. Please stop or we have to ban the account.
- romanhotsiy 10y agoThe funniest part here is that the proposed solution actually IS the semantic WEB. Schema.org is based on semantic web technologies. Check out wikipedia example: https://en.wikipedia.org/wiki/Semantic_Web#Example https://en.wikipedia.org/wiki/Semantic_Web#Example and note they use Schema.org as vocabulary
- angersock 10y ago> But why does searching for APIs and restaurants differ so much? Because--oh I don't know--APIs aren't restaurants? Is this really a problem, or is this a tool to help lazy blub developers be even lazier and blubbier? If you care enough about an API to worry about what things it does via the docs--and why aren't you using a client lib anyways?--then you care too much to be satisfied with a simple sketch of the API via Google search results.
- psadauskas 10y agoIt sounds like what you're describing is very similar to Hydra[1], which builds upon JSON-LD[2], which builds upon schema.org. JSON-LD provides a vocabulary (schema) for linking various documents together, so you can for example link a blog post document directly to its author. Hydra then provides a vocabulary that lets you describe how to fetch that author document, what fields are required to create a new author object, and how to update the blog post to instead link to that new author. In this case, both "author" and "blog post" can be their respective schema.org documents. [1]: http://www.hydra-cg.com/ http://www.hydra-cg.com/ [2]: http://json-ld.org/ http://json-ld.org/
- deleted 10y ago[deleted]
- asbjornu 10y agoMy thought was Hydra when I read the suggestion as well. As Ivan Goncharov mentions JSON-LD in his blog post, I find it a perfect match.
- allthingsapi 10y agoWe discussed a similar problem with regard to the availability of swaggers and providers willing to do so https://news.ycombinator.com/item?id=11583377 https://news.ycombinator.com/item?id=11583377
- zackmorris 10y agoWhile a lot of smart people are looking at this, I wanted to ask: does anyone know of any viable projects that build semantic data from existing websites? I have experience with Open Graph http://ogp.me/ http://ogp.me/ and oEmbed http://oembed.com/ http://oembed.com/ but something about humans having to manually categorize a site has never sat right with me. We should be able to grok structure from sites by how other sites refer to them, similar to what Google does. This may be related to open source search, I don't know.
- tomc1985 10y agoI find it strange that people regard APIs as their own thing. I have never "searched the web" for "email APIs", in fact that whole sentence just sounds amateurish. Similarly what is all this tooling he refers to? WTF is an "API console"? Is that not curl? One time, while interviewing a candidate, I picked a random buzzword on his resume (which happened to be API) and asked him to talk about it for a bit. He immediately launched into a mini-rant about all these crazy details with APIs that seemed to be foregone conclusions. Do people actually agonize over whether the API speaks XML or JSON? Or fret over the lack of a console? Are we talking about the same thing? You have an endpoint, you send it a request, you get back some data. WOW SO HARD!
- dozzie 10y agoIt actually can be so hard. People who call these services that mainly expose ad-hoc defined RPC interface "APIs" tend to overcomplicate their code. You get plethora of endpoint URLs, GET parameters, methods, HTTP headers, and queries encoded in JSON or XML or GraphQL or whatever sent in request body. It Ain't No Simple Thing[tm] with this many dimensions.
- tomc1985 10y agoMaybe, but that's a problem with the design of the program, and the contents of that response. An API is only the button board, not its layout or the colors of each button. That's up to the program designer.
- psadauskas 10y agoIt is pretty hard. One API, fine, but at several points in my career, I've had to write an "API Integration" layer for an app. For example, send notifications to Slack or Hipchat or GitHub, or poll one of a half-dozen endpoints for new info, or post some data to a half-dozen others. Usually each service provides a client package for whatever language you're using, but sometimes its from a third-party, and they're always in different levels of maintained-ness. In Ruby, there hasn't been a single overwhelming HTTP client, just several that passed in and out of popularity, so depending on when the initial release of the gems you're using were, you'll end up with 3-6 different HTTP client implementations, each with their own way to mock requests for your tests. On top of the client libraries, you have to spend a not-insignificant amount of time reading the service's documentation, to see how they prefer you do auth, what methods to call on the library, what the payload needs to look like, what response codes to expect, what exceptions may be raised... It is a non-trivial amount of work for each additional service. And then you get to stay on top of API updates for the dozen services you integrate with, and hope they have an email list or RSS feed so that you have some warning before they completely break your integration. The promise of schema.org (and JSONLD, and Hydra, as I mentioned in another comment), is that if we could all just stop NIH every damn API we write because "those look too complicated, how hard can it be to just return some JSON?", then people like me, who need to integrate with several APIs, wouldn't have to waste so much time on figuring out how your special snowflake actually works.
- ksavenkov 10y agoGoogle supports around seven Schema.org entity types (https://developers.google.com/search/docs/guides/mark-up-content https://developers.google.com/search/docs/guides/mark-up-con...). I think they look at the user segments and people looking for APIs is not their main audience. Especially nowadays when AI and data science is much cooler than just software engineering ;-)
- m3h 10y agoInteresting thoughts. I've always liked if we could have something like an `api.txt` file on a website, just like we have a `sitemap` file. The `api.txt` can simply be a text-based listing of links to API descriptions for all APIs exposed by that site publicly. Most API description formats already allow for the API name, category, endpoints, etc. so no need to reinvent the wheel there. A good idea would be to add the `<link rel="api" href="api.txt" />` tag on the webpage too. This would make discovery and cataloging much easier. Some API cataloging sites have come up with interesting ideas. For example, there is a service https://sdks.io/ https://sdks.io/ that can automatically generate SDKs right out of the API descriptions that they have crawled. It is powered by https://apimatic.io/ https://apimatic.io/
- psadauskas 10y agoYour thoughts on "/api.txt" are pretty similar to how a Hydra API can be discovered: https://www.hydra-cg.com/spec/latest/core/#discovering-a-hydra-powered-web-api https://www.hydra-cg.com/spec/latest/core/#discovering-a-hyd... Essentially, there can be a Link HTTP header to the "Entry-Point" document for the API, which further links to the vocabulary that describes the resources understood by the system and the actions you can perform on them. Link: <http://api.example.com/doc/>; rel="http://www.w3.org/ns/hydra/core#apiDocumentation" I recommend watching some of the talk videos listed on the hydra web page: http://www.hydra-cg.com/ http://www.hydra-cg.com/