12 ms·
TypeSpec: A new language for API-centric development
- leetrout 2y agoI love the idea of tools like this. I have looked at Buf and Fern and am curious to try this as well.
- mdaniel 2y agosome prior discussion, just not from the main url: https://news.ycombinator.com/item?id=39833321 https://news.ycombinator.com/item?id=39833321
- Splizard 2y agoWhy create a new language, rather than use an established programming language like Go where you can actually write an implementation too?
- seivan 2y ago[dead]
- skybrian 2y agoSchemas that support multiple languages are useful when you actually use more than one language. This is more common in large organizations and between organizations. But it might also happen if you have code on multiple platforms, for example for mobile apps.
- bterlson 2y agoMoreover, compiling an IDL to N languages is substantially easier than compiling implementation code across N languages, especially when generating idiomatic code is a requirement. A language purpose-built for this task is going to produce better results while having substantially lower complexity. (My $0.02 as someone who works on TypeSpec)
- jimbobimbo 2y agoSorry, could you elaborate? If I'm creating an API using, say, ASP.NET Core or Go, I can generate OpenAPI spec out of actual implementation. How this "IDL" fits into the workflow? Is this another output in addition to OpenAPI spec?
- BillyTheKing 2y agoIn my (limited) experience so far with TypeSpec - it really shines in an API first approach, so you define your API before you implement it, but not so much the other way around.
- bterlson 2y agoTypeSpec is designed primarily as an API first tool as opposed to being an output. In the context of ASP.NET and HTTP/REST APIs, our goal is that you can write your spec and generate much of the service implementation and clients. From this same source of truth you could also emit clients or service implementations in other API definition formats like OpenAPI, schemas like JSON Schema, and other things besides.
- seanmcdirmid 2y agoI would have expected a bit more than type specifications, maybe some behavior specifications also? Something like Daan’s type states. But I get why we are still splitting hairs over data types.
- waern 2y agoI've been working on an API spec language where state changes can be modeled with linear logic: https://apilog.net/docs/why/ https://apilog.net/docs/why/ It doesn't have "schemas" yet though. Which may seem odd given they are a crucial part of this type of languages. :-) But it is because I am experimenting with different designs on that front.
- neonsunset 2y agoBecause Go is a bad language, and imposing the requirement to use it on the teams is absurd.
- thesuperbigfrog 2y agoLooks like the Typescript version of WSDL: https://en.wikipedia.org/wiki/Web_Services_Description_Language https://en.wikipedia.org/wiki/Web_Services_Description_Langu... Perhaps it will last longer than WSDL did?
- dexwiz 2y agoI still use WSDLs, or rather the platform I work on does. Maybe not popular for new tech but they are still alive. Hate me, but I’d rather have generated xml than generate yaml.
- paulddraper 2y agoYou may already know this but: 1. A more exact analogy would be WSDL+SOAP. 2. WSDL and SOAP are defined in XML, and SOAP describes XML. 3. The popularity of these technologies followed the popularity (both rise and decline) of XML generally. 4. TypeSpec describes JSON and protobuf, and will likely also lose popularity if those formats do.
- vbezhenar 2y agoWSDL issue was that it was designed by a committee of several huge companies. So it was inconsistent and bloated. Same could be said about many XML-related standards. Nowadays big companies rarely work together and prefer to throw their own solutions to the market, hoping to capture it. That results in a higher quality approaches, because it's developed by a single team and focused on a single goal, rather than trying to please 10 vendors with their own agendas.
- tannhaeuser 2y ago> SOAP was designed as an object-access protocol and released as XML-RPC in June 1998 as part of Frontier 5.1 by Dave Winer, Don Box, Bob Atkinson, and Mohsen Al-Ghosein for Microsoft, where Atkinson and Al-Ghosein were working. The specification was not made available until it was submitted to IETF 13 September 1999. [1] WSDL 1.0's list of editors reads [2]: > Erik Christensen, Microsoft; Francisco Curbera, IBM; Greg Meredith, Microsoft; Sanjiva Weerawarana, IBM IOW, TypeScript is by the same company as SOAP and WSDL. > Nowadays big companies rarely work together [...] That results in a higher quality approaches [Citation needed] [1]: <https://en.wikipedia.org/wiki/SOAP https://en.wikipedia.org/wiki/SOAP> [2]: <http://xml.coverpages.org/wsdl20000929.html http://xml.coverpages.org/wsdl20000929.html>
- verdverm 2y agoBespoke languages are a hard sell and incur significant extra effort on the team building this. 1. People don't want to learn bespoke languages. 2. You have to build all the ecosystem tools for a language (compiler, docs, language-server, IDE integrations, dependency management) Similar endeavors are WaspLang and DarkLang, which I have yet to see in the wild or (meaningfully) on HN. Better to use an existing language and focus on the value add. I personally built something with similar value add (source of truth -> all the things). I've been through some of these pain points myself. https://github.com/hofstadter-io/hof https://github.com/hofstadter-io/hof The idea is CUE + text/template = <all the things> (not limited to APIs)
- cjonas 2y agoNew languages are at an even bigger disadvantage now with the rise of generative AI programming. A fancy new framework that is objectively better may actually be less productive because AI haven't been trained on it
- verdverm 2y agoThere are some AI companies focusing on chatbots for developer projects that do better than using a general purpose LLM. I think this will become more common and not really a barrier
- cjonas 2y agoAll the game changing AI programming I've seen is coming from LLM... Regardless, there will always be a lack of training context for newer languages or frameworks.
- BillyTheKing 2y agoIt's this or writing openapi ymls... Even for people who know yml picking this up to define and write basic openapi definitions is much simpler than writing an openapi doc from hand, which is really painful
- 2y ago
- favflam 2y agoWhat is wrong with protobufs and grpc?
- _ZeD_ 2y agoBoth of them. Both of them are wrong.
- vundercind 2y agoWhen one starts gluing together a lot of data pipelines full of JSON trash and from all kinds of systems with incompatible data types (whether or not “lol what’s an integer?” JSON is involved), one quickly comes to appreciate why things like Protobuf exist and look the way they do.
- ramesh31 2y ago> When one starts gluing together a lot of data pipelines full of JSON trash and from all kinds of systems with incompatible data types (whether or not “lol what’s an integer?” JSON is involved), one quickly comes to appreciate why things like Protobuf exist and look the way they do. And then one proceeds to spend days trying to mash that JSON mess into a protobuff and debugging segfaults, rather than just getting the job done with HTTP.
- vundercind 2y agoAt least it’s confined to the edges.
- deleted 2y ago[deleted]
- frankrobert 2y agoThe protobuf v3 spec doesn't support required fields anymore and also mandates field ordering. In my opinion, both of these are deal breakers for generating types and typed clients for your FE environment (or any consuming application). Your entire schema would be Partial<T> on every field. It defeats the purpose of the type safety to me
- deleted 2y ago[deleted]
- dexwiz 2y agoSo this is coming from Microsoft. I assume it’s going to be their answer to graphql. If the project is dogfooded internally, the tools may actually be half decent, compared to whatever an open source consortium cobbles together. Not sold yet, but might gain more traction.
- bterlson 2y ago(I work on the team) I wouldn't say that TypeSpec is like GraphQL, so it would be hard for TypeSpec to become that on its own. GraphQL has a lot of opinions that are required in order to build a concrete application (protocols, error handling, query semantics, etc.), whereas TypeSpec at its core is just an unopinionated DSL for describing APIs. Bindings for particular protocols are added via libraries, and a GraphQL library is something we have long considered. So in short, if Microsoft invented a set of opinions that solved similar scenarios to GraphQL, it might use TypeSpec as the API description language in that context, but it wouldn't be fair to equate those opinions with TypeSpec itself.
- nfw2 2y agoThe graphql spec does include a DSL to describe the api though, so this is similar to that specific piece. The DSL powers a lot of what people like about grapqhql, like auto-generating a client sdk with type safety. This library does seem to cover a subset of the graphql benefits that aren’t baked into REST by default.
- bterlson 2y agoYup, similar to that specific piece, and I definitely agree that GraphQL's DSL shows how much the DX of the description language itself matters, and how codegen is a productivity multiplier. I think gRPC also demonstrates this. You can think of TypeSpec as an attempt to bring these benefits to any protocol, schema vocabulary, or programming language.
- sa-code 2y agoWould you say it's an alternative to Open API?
- BillyTheKing 2y agoI've been using it for my latest API - I was looking for a tool that allowed me to describe APIs similarly to GraphQL and in a design-first sorta way. All these openapi editors just felt crazy clunky and made data-relationship within the API non-obvious. TypeSpec is a great tool, really helped me out here - was exactly what I was looking for!
- aleksiy123 2y agoHas validations thats awesome. I have a project in mind and was looking for something like this. Closest I found was CueLang. Now just need to find the time...
- dbrower 2y agoWill it translate to yaml for toolchains that want that? I'd be delighted to have a high-level IDL that gave the same sort of thing that CORBA IDL gave us 25 years ago -- schema and stub generation for multiple languages.
- bterlson 2y agoThe OpenAPI and Json Schema emitters can produce yaml.
- mnahkies 2y agoI added support for typespec as an input specification to my openapi 3 based code generator a couple of days ago. They provide an API to convert to openapi documents so it was pretty painless (https://github.com/mnahkies/openapi-code-generator/pull/158 https://github.com/mnahkies/openapi-code-generator/pull/158) My focus is on doing both client sdk and server stub generation, though only typescript so far - will hopefully add other languages eventually, when I'm satisfied with the completeness of the typescript templates.
- joelwilsson 2y agoLooks like a competitor/alternative to Smithy, https://smithy.io/2.0/index.html https://smithy.io/2.0/index.html. Since at least one person from the TypeSpec team is here, do you have any thoughts on how they compare?
- jen20 2y agoThis was my thought too - since smithy is already out there and used in a similar domain, it would be useful to have a comparison. “Doesn’t have Kotlin and Gradle all over the show” seems like a significant advantage in favour of TypeSpec.
- martinn 2y agoCould someone clarify what's the use case of a tool like this please. Is this something that helps if you, say, are building a new API and will need to create both the server implementation as well as the client implementations in multiple languages? And so, it can automatically do all of that for you based on an API spec? Or is it something different. Funnily enough, I developed a Python library recently that allows you to build API clients in a way that very closely resembles the TypeSpec example. But I'm pretty sure they are very different things.
- wg0 2y agoIf you are in a situation where you have a backend and you want to expose an API and then you would eventually want a client, you would need format specs as the starting point where server and clients are generated from that one source. At the moment, OpenAPI with YAML is the only way to go but you can't easily split the spec into separate files as you would do any program with packages, modules and what not. There are third party tools[0] which are archived and the libraries they depend upon are up for adoption. In that space, either you can use something like cue language 1] or something like TypeSpec which is purpose built for this so yet, this seems like a great tool although I have not tried it yet myself. [0]. https://github.com/APIDevTools/swagger-cli https://github.com/APIDevTools/swagger-cli [1]. https://cuelang.org/ https://cuelang.org/ EDIT: formating
- mnahkies 2y agoWhilst it's not as expressive/flexible as typespec, and in my experience it's not always well supported by tooling, you can do $ref's across files in openapi specifications. Eg: https://github.com/mnahkies/openapi-code-generator/blob/main/integration-tests-definitions%2Ftodo-lists.yaml#L43-L46 https://github.com/mnahkies/openapi-code-generator/blob/main...
- monkfish328 2y agoSo just to clarify, how would one go about auto-generating a stub handler for these route definitions?
- stapert 2y agoLooks interesting, but what would be the advantage of this over just writting an openAPI specification? It's more concise, but currently this would require you to increase your toolchain to go from TypeSpec to openAPI to generating code. Any plans to add code generatio to this project?
- usrusr 2y agoFeels like cheating, next to the yaml of openapi anything will look good. And that's all while I'm still considering openapi one of the best things that have happened. But I've also been kind of holding my breath for typescript making it's breakthrough as a schema language. More specifically its surprisingly big non-imperative, non-functional subset, obviously. And at first glance this seems to be exactly this, "what if we removed javascript from typescript so that only typing for JSON remains and added some endpoint metadata description to make it a viable successor to openapi and wsdl.
- deleted 2y ago[deleted]
- vbezhenar 2y agoTypeScript type system is very advanced. It won't be possible to generate corresponding bindings for all popular languages, while keeping them idiomatic. I'd prefer API language to be very simple and straightforward.
- madeofpalk 2y agoOpenAPI's 'type system' is surprisingly advanced also, supporting explicitly discriminated unions and other things like that, which doesn't model well into all other languages.
- brabel 2y agoYep and that comes from JSON Schema: https://json-schema.org/ https://json-schema.org/ I believe recent versions of OpenAPI are "compatible" with JSON Schema (at least they "wanted to be" last I checked as I was implementing some schema converters). Even TypeScript is not enough to represent all of JSON Schema! But it gets close (perhaps if you remove validation rules and stuff like that it's a full match). But even something like Java can represent most of it pretty well, specially since sealed interfaces were added. I know because I've done it :).
- usrusr 2y ago
- vbezhenar 2y agoI failed to find an answer to the main question: what output languages are supported. The only way is to emit OpenAPI and then using one of their terrible generators?
- JoyrexJ9 2y agoYou can create your own emitters, there's info in the docs on how to do so. My team built a custom TypeSpec emitter to output a SDK and set of libraries
- s4i 2y agoI wonder if they have plans to introduce any default emitters as part of the package? Nothing about that in the docs that I could find. [1] [1] https://typespec.io/docs/extending-typespec/emitters https://typespec.io/docs/extending-typespec/emitters
- bterlson 2y agoThere are a few emitters in our standard library - OpenAPI 3.0, JSON Schema 2020-12, and Protobuf. REST client and service emitters for a few languages are coming online now and should be ready in the next couple months.
- marioguerra 2y agoI'm the TypeSpec PM at Microsoft and I'd like to learn more about your use case and experience building a custom emitter. Are you willing to chat about it? If so, what's the best way for me to reach you?
- xiaolin 2y agoHow about ConnectRPC + grpc-gateway?
- pjmlp 2y ago> At Microsoft, we believe in the value of using our own products, a practice often referred to as "dogfooding". One would think, unfortunely that is not how it looks like in the zoo of native Windows desktop development, or Xamarin/MAUI adoption in their mobile apps.
- Savageman 2y agoIt must be a new policy, or maybe only for this product (which makes good marketing). I've used Microsoft Graph to manage emails, and I'd be very surprised if they use if for Outlook...
- gpderetta 2y agodogfooding has been a thing at MS probably since its inception. Of course it doesn't mean it is always practiced fully.
- eddythompson80 2y agoEven during Windows 95 it was an old established policy https://devblogs.microsoft.com/oldnewthing/20051116-12/?p=33323 https://devblogs.microsoft.com/oldnewthing/20051116-12/?p=33...
- madeofpalk 2y agoWindows is dog fooding _all_ the native app development options simultaneously.
- jayd16 2y agoDogfooding doesn't mean write everything from scratch as soon as something new comes up.
- DanielHB 2y agoWould be nice if you could just import those typespec files in typescript (and other languages?) and get automatic typescript types from them. Codegen is annoying and error prone.
- cachehit 2y agoI generally prefer code generation. I don't see why it is "error prone" any more than anything else is. In fact, it means that errors aren't concealed in two levels of abstraction which can make them much harder to debug. Also, with generated code, you can add a build step to work around short comings or bugs in the generator where as you otherwise have no choice to live with it. The "annoying" part I do get -- it's obviously nicer to have instant feedback than being forced to rebuild things -- so for things where you iterate a lot, that does weigh in the other direction.
- AntonCTO 2y ago+1. It even looks very similar to TypeScript. Why not use TypeScript as a description of APIs in the first place? Get TypeScript types and even generate OpenAPI schema on the fly to serve it at `/openapi`?
- darylteo 2y agoExactly - this is already a solved problem. https://docs.nestjs.com/openapi/introduction https://docs.nestjs.com/openapi/introduction
- BillyTheKing 2y agoyes, the whole world runs on nestjs
- darylteo 2y agohttps://github.com/domaindrivendev/Swashbuckle.AspNetCore https://github.com/domaindrivendev/Swashbuckle.AspNetCore https://github.com/RicoSuter/NSwag https://github.com/RicoSuter/NSwag There is no need to be facetious, solutions like these exist for many platforms and ecosystems out there. I can't possibly entertain all of them. By all means, if such similar solution does not exist for your platform, then TypeSpec could indeed be a solution for you. With best regards.
- deleted 2y ago[deleted]
- darylteo 2y agoSpec <-- api code is far superior to Spec --> api code, imo. Feels like it's going backwards - there's really no reason why it has to be a .tsp, instead of a .ts with actual api code. It's even using @annotations. In fact the annotations i see in the screenshot (@route, @query, @path) are practically the same in NestJS. I feel that we should be focusing on enhancing that paradigm instead. In fact I already have a working POC of NestJS -> OpenAPI -> Client libraries so I see no place for this. The spec itself is simply a vehicle for code generation, and serves little purpose otherwise and I'd be happy to be rid of it.
- catlifeonmars 2y agoWhat if you need to generate two server implementations (possibly in different languages) that adhere to the same specification? You don’t always go server -> spec -> client.
- darylteo 2y agoThat seems like a rare use case. Surely we optimise for the general use case - you start working on a new server powered platform, and write it once in your stack of choice, and release libraries for clients (mobile apps, integrations, web apps). Even so, a e2e test suite would surely serve far more utility over a spec that simply stubs out endpoints with no functionality.
- BillyTheKing 2y agofor internally consumed APIs I somewhat agree, but if you wanna expose APIs to external developers it usually does pay off to spend a bit of time on the API design itself, what APIs would you expect to see if you were the user of the your own service - and this is where tools like this really help imo - you could even preview your API in some openapi doc tool like readme or similar, in which case it's like previewing your product before releasing it
- vundercind 2y agoIt is not at all uncommon to want a spec for an api that’s out of your control. Or to want to define shared data structures at the transport layer which may have “owners” in heterogenous languages. Spec languages are (well, can be—so very many are terrible) very nice in those cases, which are not at all rare. They may well be more common than the “we’re writing a new single-language monolith that is free to control the shape of the data it slings” use case.
- schnable 2y agoDoes TypeSpec work well with asynchronous and event-driven APIs?
- domoritz 2y agoThis looks interesting but I already have TypeScript types for my APIs so I developed https://github.com/vega/ts-json-schema-generator https://github.com/vega/ts-json-schema-generator which lets me generate JSON schema from the sources directly. Yes, it does have some oddities because the two languages have slightly different feature sets but it’s been working well for us for a few years. If I didn’t have TypeScript or a smaller API surface I’d be okay with typing again I would look at TypeSpec though. It definitely beats writing JSON schema by hand.
- H12 2y agoSeems like this exists in a similar problem domain as pkl? https://pkl-lang.org/index.html https://pkl-lang.org/index.html
- ActionHank 2y agoYeah, but this one is better because new. I might be wrong, but I suspect that the crazy hype-driven-development has started to move on from frontend to backend.
- blowski 2y agoThe backend has definitely suffered from "crazy hype-driven development" for the last 30 years. Perl, Python, Ruby, Java, PHP, Scala, Clojure, Go, Rust have all had their brief moment as the silver bullet. Not to mention ops tooling - Vagrant, Docker, Kubernetes, Puppet, Chef, Ansible. I wasn't alive in the 1970s, but I'm guessing that those who were would say it was just as faddish then as well.
- vaylian 2y agoNot quite. pkl is a language that is mostly designed for parsing and serialization of data. TypeSpec is a language that is designed to describe APIs and the structure of data that those APIs take. You can actually combine the two technologies as follows: 1. Read a .pkl file from disk and generate (for example) a Person struct with a first, last name and an age value. 2. Let's say that according to some TypeSpec, the HTTP endpoint /greet accepts a POST request with a JSON payload containing a first and a last name. You convert your Person struct into a JSON literal (and drop the age field in the process) and send it to the HTTP endpoint 3. You should receive a greeting from the HTTP endpoint as a response. The TypeSpec may define a greeting as a structure that contains the fields "message" and "language". 4. You can then use pkl to write that greeting structure to disk. Sidenote: pkl also supports methods[1] but in almost all use cases you only use pkl to define which fields your data has. TypeSpec cares most about the methods/endpoints. Of course, you still need to define the shape of the arguments to these methods and that is where you have a bit of overlap between these two technologies. I can imagine that you would generate pkl definitons from TypeSpec definitions. [1] https://pkl-lang.org/main/current/language-reference/index.html#methods https://pkl-lang.org/main/current/language-reference/index.h...
- mbrock 2y agothe number of times this document says "not just ... but ..." and "revolutionize" is so obviously GPT-4
- kkukshtel 2y agoI wish any of Typespec, Cue, Pkl, Dhall, etc. would just implement their core functionality in C with an ABI for other language bindings. Needing to use whatever dep they decided to operate with as part of adoption of the language itself is a big ask. I want to try out your config language, I don't want/need all of node to make this happen.
- verdverm 2y agoCUE is building out language bindings for various languages right now Since CUE is written in Go, you can output a .so that is then used like your C based desire, if I understand you correctly
- duped 2y agoHow does this compare to MIDL?
- hoppersftw 2y agolol, about 5 years too late. I've been developing REST APIs with Ballerina for some time now and there's a multitude of quality of life improvements for me. Look at this https://ballerina.io/learn/write-a-restful-api-with-ballerina/ https://ballerina.io/learn/write-a-restful-api-with-ballerin... I guess it's NIH for Microsoft. Oh well, at least it's TypeScript.
- danhudlow 2y agoThe toy example with an API definition that includes zero semantic documentation doesn’t give me a lot of confidence that TypeSpec helps author API definitions that are actually good. It’s easy to create a concise language if all you want to generate is boilerplate.
- gslin 2y agohttps://xkcd.com/927/ https://xkcd.com/927/