12 ms·

Write OpenAPI with TypeSpec

118 points by bterlson 3y ago

tyayers 3y ago
This is typical Microsoft - overly complicated solution in search of a problem. Yaml and Json are beloved for a reason - they're simple and effective.
- karmajunkie 3y ago
  I think "beloved" is a bit of a stretch... tolerated perhaps, for their simplicity and versatility, in that you can make them work for most kinds of data transfer and configuration if you don't mind shoehorning your needs into that spare box. But beloved? hardly...
- eyelidlessness 3y ago
  Just because you don’t have the problem it’s solving doesn’t mean the problem doesn’t exist.
- dz0707 3y ago
  Having written quite a bit of open API specs, I don't agree with you. Json is hard to read, yaml has own quirks, especially when you try to spilt it into parts. Amazon also tries to invent own language for describing apis, so I guess they are not happy with open API too. Anyway, without ability to generate code from spec, there is not much use from it. Code gen/nswagger/open API generator and others produce terrible code, at least for java/c#/typescript(there's 4.1k open issues for open API generator), using custom generators for codegen make problem less painful, but that is additional burden, I'm looking for better alternative, would be very interesting to see what they will do with code generation.
  - bterlson 3y ago
    Codegen is coming online as we speak. We do codegen from TypeSpec in Azure across multiple languages, and the results are pretty great. We're moving that over to the TypeSpec project so everyone can generate code. Obviously my opinion is biased, but I think the results are significantly better than what you find elsewhere in the ecosystem.
- alecthomas 3y ago
  I would be absolutely stunned if anyone, ever, has described OpenAPI as "beloved". It does its job, but that's about the best you can say about it.
- beders 3y ago
  Yaml is beloved when it's maybe two pages of little indented config. It becomes a nightmare once people try to emulate a DSL with it.
dylmye 3y ago
Writing OpenAPI by hand sucks, this looks fantastic. The only reason I prefer generators from code comments is because the structure and the code itself live in the same place, so it's harder to forget.
buzziebee 3y ago
Looks interesting. Can it handle asyncapi specs too? We use boats (npm) to define our openapi and asyncapi specs which uses file based structure to separate definitions for models, paths, params etc into separate files which makes it much more maintainable. Native refs make it super easy to reuse definitions, and you can write custom helpers in js to abstract things like the Page definition in that example. Having a shared templating language for openapi / asyncapi specs which is ergonomic and can be used by more than just JS devs is a great idea. I'll keep an eye on this project.
- bterlson 3y ago
  Hey, author here. Right now we don't have any streaming or eventing support, but I am working on it as we speak. My first goal is to support describing SSE and JSONL HTTP streaming endpoints, but I want to work toward an AsyncAPI emitter as a peer for our OpenAPI emitter.
- Alupis 3y ago
  My biggest frustration when working with AsyncAPI is how different it's bundling logic is versus OpenAPI tooling, like Redocly. They're both based on JSON Schema, but AsyncAPI's bundling logic seems to fall down a lot when facing `$ref` in strange places (even though the actual spec allows), such as ref'ing an entire channel or operation. Looking at boats, it looks a lot like regular open/async api spec documents, with a few features sprinkled in. How great is the transpiled output? Does it pass standard validators provided by OpenApi and AsyncApi? Does it support AsyncApi 3.x?
LunaSea 3y ago
I wonder why this isn't written in Typescript directly so as to handle documentation, validation and type definitions in one go. Especially if you want to reuse type definitions of objects elsewhere in your code. An API request or response usually do not contain the exact model attributes. Only the public facing ones which are then often filtered down some more, depending on the user making the request (and their roles, groups, permissions).
- eyelidlessness 3y ago
  > I wonder why this isn't written in Typescript directly so as to handle documentation, validation and type definitions in one go. I suspect they want TypeSpec to be more runtime agnostic (as also tends to be the preference with TypeScript). Specifically to support arbitrary validation/serde. Having written a fully integrated generalized solution (runtime-defined schema -> parser, types, documentation, serializer), by far my biggest regret the first time around is coupling it so closely to a particular schema runtime. Partly this is because I have bigger dreams for designing a schema system purpose built for the task (I had previously piggybacked on io-ts, which worked but it took considerable non-essential effort and complexity); partly it’s because there can be significant tradeoffs between different runtime solutions in terms of performance and ergonomics.
- bterlson 3y ago
  Our first attempt at a TypeSpec-like thing was in fact a TypeScript dialect! It works great for simple cases, but at the limit it falls over for a number of reasons. HTTP needs a lot of metadata that isn't found in TypeScript, things like HTTP verbs, query/path params, headers, routes, etc. API patterns also need their own metadata, like for pagination you need to know various bits of metadata that describe how to paginate an endpoint. Also many things in the TypeScript type system and stdlib don't apply in the context of API descriptions, so we needed a subset. So after going down this road for a while, we found we were in a place where the syntax was actually very complex, with metadata spread between deeply nested generic type instantiations and JSDoc comments, and didn't feel at all like TypeScript. This didn't align with our goals of having something simple and easy to learn for all devs. So I wrote the first implementation of a custom language in a day and folks really liked the direction. 4 years later, here we are!
DanHulton 3y ago
Wow, I was _just_ thinking that it would be excellent to have something like this the other day. I really prefer spec-first development, since it gives other developers an opportunity to review the API _before_ the changes start being implemented, and trying to do this from a codegen-based OpenAPI implementation frequently leads to the PR being "broken", since you've only changed the signature of all your methods and not the implementation yet. But changing a big OpenAPI yaml file is really hard to easily review, and breaking it up into included files only helps a little, honestly. We can compile the new spec and host the HTML in a temporary location to make it easier to view what the new spec will look like, but once you've done that, you're no longer looking at the diff of what's changed. TypeSpec looks terse enough to be easy to review (and to write!) which really looks like it'll help with that. I'll have to mess around with it in some personal projects of mine with reasonably-complicated specs and see if there's no obvious speedbumps first, but I'm hoping not, because I'd love to start using this in all the projects I'm contributing to!
emi2k01 3y ago
This is cool! I made a similar project to this one when we switched from GraphQl to regular REST at $work but decided to drop it since I didn't have the bandwidth to work on it. I see editor support for VSCode, but is it backed by an LSP or is it VSCode only?
- bterlson 3y ago
  It's LSP, and we also have a VS extension in the marketplace.
skrebbel 3y ago
Question for the author, why not fully adopt TypeScript? Ie have TypeSpec be a well-defined, limited subset of TypeScript? At first glance, it seems to me that a model maps to an interface, an op is a function, and an enum is a string literal union. I understand that you can’t express everything that way, eg annotations, but you could get pretty far with putting those in JSDoc comments, no? I could see a big practical benefit to TS code being able to read an API spec directly without another conversion step, so I’m curious why you chose not to go with that.
- bterlson 3y ago
  I answered in more detail in another comment, but we actually tried this first, and couldn't make it work for the kinds of complex APIs we deal with in Azure. But, it worked great for simple APIs, and I hope someone releases such a tool at some point.
mdasen 3y ago
As someone who has used JAX-RS (Java) and ASP.NET, APIs are basically created with these kinds of annotations right in the language. @GET public Character getCharacter(@PathParam("id") int id) { return db.getCharacter(id); } That's very similar to TypeSpec's op getCharacter(@path id: safeint): Character; Java and C# classes already have the type information that you'd be getting from a TypeSpec: // TypeSpec model Character { name: string; id: safeint; status: "Alive" | "Dead"; class: Class; } enum Class { warrior; wizard; } // C# public class Character { public string Name { get; set; } public int Id { get; set; } public Status Status { get; set; } public CharacterClass CharacterClass { get; set; } } public enum Status { Alive, Dead } public enum CharacterClass { Warrior, Wizard } But with C#/Java you don't need to write the spec and make sure that it stays in sync with what your endpoints are actually doing. TypeSpec looks a lot better than writing YAML by hand, but I'd still rather something that was tied to the actual code that will be executing so that things are never out of sync or wrong.
- lijok 3y ago
  This is the code-first vs schema-first debate. Both have their pros and cons. Personally a die hard advocate of schema-first.
  - Petermarcu 3y ago
    Agreed. If the schema can also generate the service routes, models, serialization, etc, and you just have to maintain the business logic in the service, you get the spec matches the service benefits that way as well. Best of both worlds? People using gRpc seem to be 100% fine with schema first and generating the service stubs. One of the key things with having the spec is that you can actually describe a lot more than you can with the various attributes and comments in the code. Especially things that are not as much service concerns but potentially client concerns or documentation concerns. You can also encapsulate reusable API patterns so you know different operations are following the same pattern. The author didn't go into all the details but there are lots of ways in TypeSpec to separate the concerns of different consumers of the spec. There is a lot of opportunity and creative thinking that can be done here.
k8svet 3y ago
Cool, it certainly feels like there's some unrealized better way to describe APIs -- but the part that's more interesting for me is the codegen story? Or some way to validate the service I implemented is actually conformant? I guess for now it translates to OpenAPI, but the footnote implies that TypeSpec-driven codegen could be better. Maybe more is coming soon?
- Petermarcu 3y ago
  The author has an example repo demonstrating some of the early support for codegen. https://github.com/bterlson/typespec-todo https://github.com/bterlson/typespec-todo Server side codegen is also one the horizon and is a key way to keep the spec and service in sync. So many possibilities here!
lijok 3y ago
What are people's thoughts on this vs AWS's Smithy? We've been looking to buy into Smithy heavily as we no longer have the appetite to deal with OpenAPI and it's horrendous ecosystem.
- hocuspocus 3y ago
  I would pick Smithy if I had to go spec-first. I currently tolerate OpenAPI because I'm using a robust code-first approach (Tapir in Scala) on the server side, but I otherwise agree, the ecosystem seems brittle.
  - Petermarcu 3y ago
    TypeSpec and Smithy live in a similar space here. You can generate OpenAPI specs from them, or you can generate assets directly from them. OpenAPI is one of the many possible outputs from TypeSpec. This is an example the author created of TypeSpec being used to generate clients in many languages as well as the OpenAPI spec. https://github.com/bterlson/typespec-todo https://github.com/bterlson/typespec-todo
Stoids 3y ago
I like the idea, especially the TS-like syntax around enums and union types. I've always preferred the SDL for GraphQL vs writing OpenAPI for similar reasons. Most APIs I've run into in my career would benefit from modeling their API responses as ADTs versus the usual approach of overloading 4 different union members into a giant spare object. I echo the sentiment others have brought up, which is the trade-offs of a code-driven schema vs schema-driven code. At work we use Pydantic and FastAPI to generate the OpenAPI contract, but there's some cruft and care needed around exposing those underlying Pydantic models through the API documentation. It's been easy to create schemas that have compatibility problems when run through other code generators. I know there are projects such as connexction[1] which attempt to inverse this, but I don't have much experience with it. In the GraphQL space it seems that code-first approaches are becoming more favored, though there's a different level of complexity needed to create a "typesafe" GraphQL server (eg. model mismatches between root query resolvers and field resolvers). [1] https://github.com/spec-first/connexion https://github.com/spec-first/connexion
henry_pulver 3y ago
Great idea - spend far too long reading & writing OpenAPI! Particularly anyOf, allOf and oneOf (especially when nested) lead to really confusing nested specifications in OpenAPI. Really like how TypeSpec handles unions & intersections. Playground is great for getting a feel for it fast too
mooreds 3y ago
As someone who wrote a custom dsl to Open API spec script (in Ruby) I'm really interested to see if type spec would be a better output. Our spec is > 12k lines and we haven't even included everything that needs to be defined. Does anyone have experience layering this on to an existing API (as opposed to defining the API and then building the code)?
- bterlson 3y ago
  Extensive experience inside Azure. In general, supporting existing APIs is harder, predominantly because it might not sufficient to produce a semantically identical OpenAPI when downstream tools are sensitive to e.g. whether something is a ref or not, the order of properties in the document, whether something uses `const` or an `enum` with a single member, etc. If you have more specific questions I'm happy to consult!
  - mooreds 3y ago
    Haha, thanks for the reply! We're spending some time this quarter working on our open API spec and have engaged some experts, but if we need more help, I know who to reach out to!
ramilefu 3y ago
I've been toying with the design of something similar here and there over the past couple of months. It never made it past the pen-and-paper stage though. Pleasantly surprised to find this today and will have to try it out for a work project. My original motivation was the lack of OpenAPI definitions across projects, despite everyone agreeing their existence is a "best practice". As you suggest, developers just truly hate writing them. Unfortunately, even the "generate it from code" approach tends to be complicated and often feels duct-taped together.
zexodus 3y ago
TypeSpec is great, but if you're working with Rust and you're about to write a new project that will require an OpenApi spec sooner or later, I'd like to recommend a web framework that has spec generation baked in: https://github.com/poem-web/poem https://github.com/poem-web/poem (see poem_openapi) All you need to do is derive a trait on your response structs and in return you get an almost perfectly generated spec. Unions, objects, enums are first class citizens. Also, if you're from coming from PHP, the controllers feel very much like symfony controllers. P.s. Please do recommend an ORM that would feel closer to doctrine. I miss doctrine.
mmcclure 3y ago
We're also spec-first at my co (pretty mature, large API surface area), but maintaining the OpenAPI spec in YAML is extremely annoying and tedious. This checks a lot of boxes in terms of being interesting, but what does the path from an existing YAML spec to TypeSpec look like? In an ideal world we could just ingest our OpenAPI spec, get the TypeSpec version out, then be able to clean up/iterate from there. I didn't see anything in the docs, but I would love for someone to tell me this exists. Re: the post itself, this was much more compelling than TypeSpec's website imo. At the very least it feels like some of these examples, such as the enum bit, should be ported there. Also interesting that there isn't a single mention of languages like Cue[1]. [1] https://cuelang.org/ https://cuelang.org/
- bterlson 3y ago
  It does not exist, but it will be worked on fairly soon. You can track progress on GitHub [1]. It will be as you suggest - a one time conversion, after which you can iterate. We have this workflow for inside Azure, but the converter is Azure-specific (e.g. converts to TypeSpec that uses our extensive Azure-specific component library). It will take a bit to generalize. [1]: https://github.com/microsoft/typespec/issues/3038 https://github.com/microsoft/typespec/issues/3038
dajonker 3y ago
We have been using this for a couple of months now and it works really well. It's so much better than writing OpenAPI by hand, and I never ever will do that again after working with TypeSpec. Thank you!
patrick-fitz 3y ago
This looks great, is there a converter to convert an OpenAPI yml file to TypeSpec?
- bterlson 3y ago
  Not yet, but we're working on it. Should be out in a few weeks.
tholm 3y ago
As someone who prefers schema-first design this looks awesome. Having to hand cut OpenAPI specs is gross and quickly becomes unwieldy.
btown 3y ago
Ooh, magick.css ( https://css.winterveil.net/ https://css.winterveil.net/ ) spotted in the wild! Discussion on magick.css from 5 days ago: https://news.ycombinator.com/item?id=39793513 https://news.ycombinator.com/item?id=39793513
- bterlson 3y ago
  It's a beauty! I had to get on the train early before it's everywhere.
dadadad100 3y ago
I don’t see a way to specify the security scheme. Am I missing something?
- bterlson 3y ago
  This docs page will hopefully help: https://typespec.io/docs/libraries/http/authentication https://typespec.io/docs/libraries/http/authentication
- jaxelr 3y ago
  You can, see here: https://typespec.io/docs/libraries/http/authentication#available-security-schemes https://typespec.io/docs/libraries/http/authentication#avail...
esafak 3y ago
I found the blog's soft font hard to read. Fortunately my browser's reader mode fixed it.
lacrosse_tannin 3y ago
I specified an existing api in this, and I think it turned out pretty good. It exports openapi and json-schema from the same spec. Looking forward to having more linters for specs and stuff like that.
nonethewiser 3y ago
I find it kind of discouraging that we now need a language to define an API that compiles into OpenAPI but honestly I hate writing OpenAPI specs (especially compared to gql). So despite thinking it’s kinda dumb, Im all ears. Halfway through the article and it seems compelling so far.
FourOnTheFloor 3y ago
Targeting OpenAPI 3.0 is a bad idea. It wasn't until 3.1 that it became truly useful, if only because it fixed one glaring blunder: Pre-3.1, you inexplicably couldn't provide a description along with a $ref. This defeats one of the major purposes of OpenAPI, which is... documenting your API. Example: If you define a structure called Rectangle and use it all over the place, you can't say what this rectangle means or that rectangle means when the structure is used in your API. How was that ever deemed acceptable? Widespread use of common structures (which one would certainly expect in a well-designed API) resulted in an API that couldn't be documented in OAS. The OpenAPI code-generation landscape is truly abysmal. Not only do almost none of the tools support version 3.1 (which has been current for years), but the generators themselves are of widely varying (but mostly shitty) quality and scattered across a couple of apparent major repos. The documentation is likewise a confusing morass of conflicting and incomplete information from several sources. I flailed away trying to fix one generator for weeks, or write a new one. But I couldn't even find a concise list of the data structures offered to the template engine after an OpenAPI doc is ingested by OpenAPIGenerator. Isn't that a fundamental part of creating a new generator? So it raises the question of whether OpenAPI is salvageable, or whether this new DSL is better... from design to code generation. Given the trashy state of OpenAPI code generators, why bother generating OAS at all? Just move on and write some competent generators using this new language.
- bterlson 3y ago
  For what it's worth, we support 3.0 because as you note the ecosystem doesn't support 3.1 broadly yet. I'm personally interested to see if 3.1 becomes prevalent before 4.0 is released. Maybe the ecosystem will just skip 3.1? We generate OAS because it's useful for many folks, including for us in Azure. But like you we didn't have very good luck getting high quality codegen from OpenAPI. Our latest client codegen tech doesn't use OpenAPI, we generate code directly from the TypeSpec, which offers a number of advantages that result in higher quality. Probably at topic for a blog of its own! Anyway, you might be happy to know that we are working on bringing our emitters into the TypeSpec project so anyone can use it with their TypeSpecs.
  - FourOnTheFloor 3y ago
dheerajvs 3y ago
How does TypeSpec compare with Bru[0], the DSL used by Bruno? Why would I choose one over the other? [0] https://docs.usebruno.com/bru-lang-overview.html https://docs.usebruno.com/bru-lang-overview.html
- nonethewiser 3y ago
  Bru is a DSL for actual requests (plus other things like js). This actually made me think of Bru too but I don’t think there’s really any overlap. Compare Bru to OpenAPI or json schema. Its just kinda non sensical.
kkukshtel 3y ago
This project seems really interesting, I'm surprised this is the first time I'm hearing about it. Like another commenter here I'm also surprised to see no mention of Cue (or now Apple's Pkl). Is this meant to be something similar to those? Also given it's a MS project, would love for there to be some drop in NuGet package for C# use. All these data description languages are really interesting but the fact none of them have a C ABI or only work in the language of choice seems to limit how useful they can be outside of their original context.
drewda 3y ago
When I first saw TypeSpec, I was interested in it as a way to have a single-source-of-truth for both OpenAPI and GraphQL schemas... but unfortunately though the readme mentions GraphQL, the team at Microsoft didn't actually get around to adding support to output GraphQL schemas.

118 points by bterlson 3y ago

This is typical Microsoft - overly complicated solution in search of a problem. Yaml and Json are beloved for a reason - they're simple and effective.

karmajunkie 3y ago
I think "beloved" is a bit of a stretch... tolerated perhaps, for their simplicity and versatility, in that you can make them work for most kinds of data transfer and configuration if you don't mind shoehorning your needs into that spare box. But beloved? hardly...
eyelidlessness 3y ago
Just because you don’t have the problem it’s solving doesn’t mean the problem doesn’t exist.
dz0707 3y ago
Having written quite a bit of open API specs, I don't agree with you. Json is hard to read, yaml has own quirks, especially when you try to spilt it into parts. Amazon also tries to invent own language for describing apis, so I guess they are not happy with open API too. Anyway, without ability to generate code from spec, there is not much use from it. Code gen/nswagger/open API generator and others produce terrible code, at least for java/c#/typescript(there's 4.1k open issues for open API generator), using custom generators for codegen make problem less painful, but that is additional burden, I'm looking for better alternative, would be very interesting to see what they will do with code generation.
- bterlson 3y ago
  Codegen is coming online as we speak. We do codegen from TypeSpec in Azure across multiple languages, and the results are pretty great. We're moving that over to the TypeSpec project so everyone can generate code. Obviously my opinion is biased, but I think the results are significantly better than what you find elsewhere in the ecosystem.
alecthomas 3y ago
I would be absolutely stunned if anyone, ever, has described OpenAPI as "beloved". It does its job, but that's about the best you can say about it.
beders 3y ago
Yaml is beloved when it's maybe two pages of little indented config. It becomes a nightmare once people try to emulate a DSL with it.

dylmye 3y ago

Writing OpenAPI by hand sucks, this looks fantastic. The only reason I prefer generators from code comments is because the structure and the code itself live in the same place, so it's harder to forget.

buzziebee 3y ago

Looks interesting. Can it handle asyncapi specs too? We use boats (npm) to define our openapi and asyncapi specs which uses file based structure to separate definitions for models, paths, params etc into separate files which makes it much more maintainable. Native refs make it super easy to reuse definitions, and you can write custom helpers in js to abstract things like the Page definition in that example. Having a shared templating language for openapi / asyncapi specs which is ergonomic and can be used by more than just JS devs is a great idea. I'll keep an eye on this project.

bterlson 3y ago
Hey, author here. Right now we don't have any streaming or eventing support, but I am working on it as we speak. My first goal is to support describing SSE and JSONL HTTP streaming endpoints, but I want to work toward an AsyncAPI emitter as a peer for our OpenAPI emitter.
Alupis 3y ago
My biggest frustration when working with AsyncAPI is how different it's bundling logic is versus OpenAPI tooling, like Redocly. They're both based on JSON Schema, but AsyncAPI's bundling logic seems to fall down a lot when facing `$ref` in strange places (even though the actual spec allows), such as ref'ing an entire channel or operation. Looking at boats, it looks a lot like regular open/async api spec documents, with a few features sprinkled in. How great is the transpiled output? Does it pass standard validators provided by OpenApi and AsyncApi? Does it support AsyncApi 3.x?

LunaSea 3y ago

I wonder why this isn't written in Typescript directly so as to handle documentation, validation and type definitions in one go. Especially if you want to reuse type definitions of objects elsewhere in your code. An API request or response usually do not contain the exact model attributes. Only the public facing ones which are then often filtered down some more, depending on the user making the request (and their roles, groups, permissions).

eyelidlessness 3y ago
> I wonder why this isn't written in Typescript directly so as to handle documentation, validation and type definitions in one go. I suspect they want TypeSpec to be more runtime agnostic (as also tends to be the preference with TypeScript). Specifically to support arbitrary validation/serde. Having written a fully integrated generalized solution (runtime-defined schema -> parser, types, documentation, serializer), by far my biggest regret the first time around is coupling it so closely to a particular schema runtime. Partly this is because I have bigger dreams for designing a schema system purpose built for the task (I had previously piggybacked on io-ts, which worked but it took considerable non-essential effort and complexity); partly it’s because there can be significant tradeoffs between different runtime solutions in terms of performance and ergonomics.
bterlson 3y ago
Our first attempt at a TypeSpec-like thing was in fact a TypeScript dialect! It works great for simple cases, but at the limit it falls over for a number of reasons. HTTP needs a lot of metadata that isn't found in TypeScript, things like HTTP verbs, query/path params, headers, routes, etc. API patterns also need their own metadata, like for pagination you need to know various bits of metadata that describe how to paginate an endpoint. Also many things in the TypeScript type system and stdlib don't apply in the context of API descriptions, so we needed a subset. So after going down this road for a while, we found we were in a place where the syntax was actually very complex, with metadata spread between deeply nested generic type instantiations and JSDoc comments, and didn't feel at all like TypeScript. This didn't align with our goals of having something simple and easy to learn for all devs. So I wrote the first implementation of a custom language in a day and folks really liked the direction. 4 years later, here we are!

DanHulton 3y ago

Wow, I was _just_ thinking that it would be excellent to have something like this the other day. I really prefer spec-first development, since it gives other developers an opportunity to review the API _before_ the changes start being implemented, and trying to do this from a codegen-based OpenAPI implementation frequently leads to the PR being "broken", since you've only changed the signature of all your methods and not the implementation yet. But changing a big OpenAPI yaml file is really hard to easily review, and breaking it up into included files only helps a little, honestly. We can compile the new spec and host the HTML in a temporary location to make it easier to view what the new spec will look like, but once you've done that, you're no longer looking at the diff of what's changed. TypeSpec looks terse enough to be easy to review (and to write!) which really looks like it'll help with that. I'll have to mess around with it in some personal projects of mine with reasonably-complicated specs and see if there's no obvious speedbumps first, but I'm hoping not, because I'd love to start using this in all the projects I'm contributing to!

emi2k01 3y ago

This is cool! I made a similar project to this one when we switched from GraphQl to regular REST at $work but decided to drop it since I didn't have the bandwidth to work on it. I see editor support for VSCode, but is it backed by an LSP or is it VSCode only?

bterlson 3y ago
It's LSP, and we also have a VS extension in the marketplace.

skrebbel 3y ago

Question for the author, why not fully adopt TypeScript? Ie have TypeSpec be a well-defined, limited subset of TypeScript? At first glance, it seems to me that a model maps to an interface, an op is a function, and an enum is a string literal union. I understand that you can’t express everything that way, eg annotations, but you could get pretty far with putting those in JSDoc comments, no? I could see a big practical benefit to TS code being able to read an API spec directly without another conversion step, so I’m curious why you chose not to go with that.

bterlson 3y ago
I answered in more detail in another comment, but we actually tried this first, and couldn't make it work for the kinds of complex APIs we deal with in Azure. But, it worked great for simple APIs, and I hope someone releases such a tool at some point.

mdasen 3y ago

As someone who has used JAX-RS (Java) and ASP.NET, APIs are basically created with these kinds of annotations right in the language. @GET public Character getCharacter(@PathParam("id") int id) { return db.getCharacter(id); } That's very similar to TypeSpec's op getCharacter(@path id: safeint): Character; Java and C# classes already have the type information that you'd be getting from a TypeSpec: // TypeSpec model Character { name: string; id: safeint; status: "Alive" | "Dead"; class: Class; } enum Class { warrior; wizard; } // C# public class Character { public string Name { get; set; } public int Id { get; set; } public Status Status { get; set; } public CharacterClass CharacterClass { get; set; } } public enum Status { Alive, Dead } public enum CharacterClass { Warrior, Wizard } But with C#/Java you don't need to write the spec and make sure that it stays in sync with what your endpoints are actually doing. TypeSpec looks a lot better than writing YAML by hand, but I'd still rather something that was tied to the actual code that will be executing so that things are never out of sync or wrong.

lijok 3y ago
This is the code-first vs schema-first debate. Both have their pros and cons. Personally a die hard advocate of schema-first.
- Petermarcu 3y ago
  Agreed. If the schema can also generate the service routes, models, serialization, etc, and you just have to maintain the business logic in the service, you get the spec matches the service benefits that way as well. Best of both worlds? People using gRpc seem to be 100% fine with schema first and generating the service stubs. One of the key things with having the spec is that you can actually describe a lot more than you can with the various attributes and comments in the code. Especially things that are not as much service concerns but potentially client concerns or documentation concerns. You can also encapsulate reusable API patterns so you know different operations are following the same pattern. The author didn't go into all the details but there are lots of ways in TypeSpec to separate the concerns of different consumers of the spec. There is a lot of opportunity and creative thinking that can be done here.

k8svet 3y ago

Cool, it certainly feels like there's some unrealized better way to describe APIs -- but the part that's more interesting for me is the codegen story? Or some way to validate the service I implemented is actually conformant? I guess for now it translates to OpenAPI, but the footnote implies that TypeSpec-driven codegen could be better. Maybe more is coming soon?

Petermarcu 3y ago
The author has an example repo demonstrating some of the early support for codegen. https://github.com/bterlson/typespec-todo https://github.com/bterlson/typespec-todo Server side codegen is also one the horizon and is a key way to keep the spec and service in sync. So many possibilities here!

lijok 3y ago

What are people's thoughts on this vs AWS's Smithy? We've been looking to buy into Smithy heavily as we no longer have the appetite to deal with OpenAPI and it's horrendous ecosystem.

hocuspocus 3y ago
I would pick Smithy if I had to go spec-first. I currently tolerate OpenAPI because I'm using a robust code-first approach (Tapir in Scala) on the server side, but I otherwise agree, the ecosystem seems brittle.
- Petermarcu 3y ago
  TypeSpec and Smithy live in a similar space here. You can generate OpenAPI specs from them, or you can generate assets directly from them. OpenAPI is one of the many possible outputs from TypeSpec. This is an example the author created of TypeSpec being used to generate clients in many languages as well as the OpenAPI spec. https://github.com/bterlson/typespec-todo https://github.com/bterlson/typespec-todo

Stoids 3y ago

I like the idea, especially the TS-like syntax around enums and union types. I've always preferred the SDL for GraphQL vs writing OpenAPI for similar reasons. Most APIs I've run into in my career would benefit from modeling their API responses as ADTs versus the usual approach of overloading 4 different union members into a giant spare object. I echo the sentiment others have brought up, which is the trade-offs of a code-driven schema vs schema-driven code. At work we use Pydantic and FastAPI to generate the OpenAPI contract, but there's some cruft and care needed around exposing those underlying Pydantic models through the API documentation. It's been easy to create schemas that have compatibility problems when run through other code generators. I know there are projects such as connexction[1] which attempt to inverse this, but I don't have much experience with it. In the GraphQL space it seems that code-first approaches are becoming more favored, though there's a different level of complexity needed to create a "typesafe" GraphQL server (eg. model mismatches between root query resolvers and field resolvers). [1] https://github.com/spec-first/connexion https://github.com/spec-first/connexion

henry_pulver 3y ago

Great idea - spend far too long reading & writing OpenAPI! Particularly anyOf, allOf and oneOf (especially when nested) lead to really confusing nested specifications in OpenAPI. Really like how TypeSpec handles unions & intersections. Playground is great for getting a feel for it fast too

mooreds 3y ago

As someone who wrote a custom dsl to Open API spec script (in Ruby) I'm really interested to see if type spec would be a better output. Our spec is > 12k lines and we haven't even included everything that needs to be defined. Does anyone have experience layering this on to an existing API (as opposed to defining the API and then building the code)?

bterlson 3y ago
Extensive experience inside Azure. In general, supporting existing APIs is harder, predominantly because it might not sufficient to produce a semantically identical OpenAPI when downstream tools are sensitive to e.g. whether something is a ref or not, the order of properties in the document, whether something uses `const` or an `enum` with a single member, etc. If you have more specific questions I'm happy to consult!
- mooreds 3y ago
  Haha, thanks for the reply! We're spending some time this quarter working on our open API spec and have engaged some experts, but if we need more help, I know who to reach out to!

ramilefu 3y ago

I've been toying with the design of something similar here and there over the past couple of months. It never made it past the pen-and-paper stage though. Pleasantly surprised to find this today and will have to try it out for a work project. My original motivation was the lack of OpenAPI definitions across projects, despite everyone agreeing their existence is a "best practice". As you suggest, developers just truly hate writing them. Unfortunately, even the "generate it from code" approach tends to be complicated and often feels duct-taped together.

zexodus 3y ago

TypeSpec is great, but if you're working with Rust and you're about to write a new project that will require an OpenApi spec sooner or later, I'd like to recommend a web framework that has spec generation baked in: https://github.com/poem-web/poem https://github.com/poem-web/poem (see poem_openapi) All you need to do is derive a trait on your response structs and in return you get an almost perfectly generated spec. Unions, objects, enums are first class citizens. Also, if you're from coming from PHP, the controllers feel very much like symfony controllers. P.s. Please do recommend an ORM that would feel closer to doctrine. I miss doctrine.

mmcclure 3y ago

We're also spec-first at my co (pretty mature, large API surface area), but maintaining the OpenAPI spec in YAML is extremely annoying and tedious. This checks a lot of boxes in terms of being interesting, but what does the path from an existing YAML spec to TypeSpec look like? In an ideal world we could just ingest our OpenAPI spec, get the TypeSpec version out, then be able to clean up/iterate from there. I didn't see anything in the docs, but I would love for someone to tell me this exists. Re: the post itself, this was much more compelling than TypeSpec's website imo. At the very least it feels like some of these examples, such as the enum bit, should be ported there. Also interesting that there isn't a single mention of languages like Cue[1]. [1] https://cuelang.org/ https://cuelang.org/

bterlson 3y ago
It does not exist, but it will be worked on fairly soon. You can track progress on GitHub [1]. It will be as you suggest - a one time conversion, after which you can iterate. We have this workflow for inside Azure, but the converter is Azure-specific (e.g. converts to TypeSpec that uses our extensive Azure-specific component library). It will take a bit to generalize. [1]: https://github.com/microsoft/typespec/issues/3038 https://github.com/microsoft/typespec/issues/3038

dajonker 3y ago

We have been using this for a couple of months now and it works really well. It's so much better than writing OpenAPI by hand, and I never ever will do that again after working with TypeSpec. Thank you!

patrick-fitz 3y ago

This looks great, is there a converter to convert an OpenAPI yml file to TypeSpec?

bterlson 3y ago
Not yet, but we're working on it. Should be out in a few weeks.

tholm 3y ago

As someone who prefers schema-first design this looks awesome. Having to hand cut OpenAPI specs is gross and quickly becomes unwieldy.

btown 3y ago

Ooh, magick.css ( https://css.winterveil.net/ https://css.winterveil.net/ ) spotted in the wild! Discussion on magick.css from 5 days ago: https://news.ycombinator.com/item?id=39793513 https://news.ycombinator.com/item?id=39793513

bterlson 3y ago
It's a beauty! I had to get on the train early before it's everywhere.

dadadad100 3y ago

I don’t see a way to specify the security scheme. Am I missing something?

bterlson 3y ago
This docs page will hopefully help: https://typespec.io/docs/libraries/http/authentication https://typespec.io/docs/libraries/http/authentication
jaxelr 3y ago
You can, see here: https://typespec.io/docs/libraries/http/authentication#available-security-schemes https://typespec.io/docs/libraries/http/authentication#avail...

esafak 3y ago

I found the blog's soft font hard to read. Fortunately my browser's reader mode fixed it.

lacrosse_tannin 3y ago

I specified an existing api in this, and I think it turned out pretty good. It exports openapi and json-schema from the same spec. Looking forward to having more linters for specs and stuff like that.

nonethewiser 3y ago

I find it kind of discouraging that we now need a language to define an API that compiles into OpenAPI but honestly I hate writing OpenAPI specs (especially compared to gql). So despite thinking it’s kinda dumb, Im all ears. Halfway through the article and it seems compelling so far.

FourOnTheFloor 3y ago

Targeting OpenAPI 3.0 is a bad idea. It wasn't until 3.1 that it became truly useful, if only because it fixed one glaring blunder: Pre-3.1, you inexplicably couldn't provide a description along with a $ref. This defeats one of the major purposes of OpenAPI, which is... documenting your API. Example: If you define a structure called Rectangle and use it all over the place, you can't say what this rectangle means or that rectangle means when the structure is used in your API. How was that ever deemed acceptable? Widespread use of common structures (which one would certainly expect in a well-designed API) resulted in an API that couldn't be documented in OAS. The OpenAPI code-generation landscape is truly abysmal. Not only do almost none of the tools support version 3.1 (which has been current for years), but the generators themselves are of widely varying (but mostly shitty) quality and scattered across a couple of apparent major repos. The documentation is likewise a confusing morass of conflicting and incomplete information from several sources. I flailed away trying to fix one generator for weeks, or write a new one. But I couldn't even find a concise list of the data structures offered to the template engine after an OpenAPI doc is ingested by OpenAPIGenerator. Isn't that a fundamental part of creating a new generator? So it raises the question of whether OpenAPI is salvageable, or whether this new DSL is better... from design to code generation. Given the trashy state of OpenAPI code generators, why bother generating OAS at all? Just move on and write some competent generators using this new language.

bterlson 3y ago
For what it's worth, we support 3.0 because as you note the ecosystem doesn't support 3.1 broadly yet. I'm personally interested to see if 3.1 becomes prevalent before 4.0 is released. Maybe the ecosystem will just skip 3.1? We generate OAS because it's useful for many folks, including for us in Azure. But like you we didn't have very good luck getting high quality codegen from OpenAPI. Our latest client codegen tech doesn't use OpenAPI, we generate code directly from the TypeSpec, which offers a number of advantages that result in higher quality. Probably at topic for a blog of its own! Anyway, you might be happy to know that we are working on bringing our emitters into the TypeSpec project so anyone can use it with their TypeSpecs.
- FourOnTheFloor 3y ago

dheerajvs 3y ago

How does TypeSpec compare with Bru[0], the DSL used by Bruno? Why would I choose one over the other? [0] https://docs.usebruno.com/bru-lang-overview.html https://docs.usebruno.com/bru-lang-overview.html

nonethewiser 3y ago
Bru is a DSL for actual requests (plus other things like js). This actually made me think of Bru too but I don’t think there’s really any overlap. Compare Bru to OpenAPI or json schema. Its just kinda non sensical.

kkukshtel 3y ago

This project seems really interesting, I'm surprised this is the first time I'm hearing about it. Like another commenter here I'm also surprised to see no mention of Cue (or now Apple's Pkl). Is this meant to be something similar to those? Also given it's a MS project, would love for there to be some drop in NuGet package for C# use. All these data description languages are really interesting but the fact none of them have a C ABI or only work in the language of choice seems to limit how useful they can be outside of their original context.

drewda 3y ago

When I first saw TypeSpec, I was interested in it as a way to have a single-source-of-truth for both OpenAPI and GraphQL schemas... but unfortunately though the readme mentions GraphQL, the team at Microsoft didn't actually get around to adding support to output GraphQL schemas.