19 ms·
Show HN: A SQL database implemented purely in TypeScript type annotations
- CGamesPlay 6y agoPretty interesting, but it doesn't seem to be possible to use the resulting values? Except as type declarations of course. Also, the playground link either fails (there are several errors) or doesn't produce any output.
- phpnode 6y agohover over the type annotations to see results in the playground, that's the output :) The errors aren't really errors, just warnings about unused types
- amitport 6y agoyeah Typescript is compile-time-only type system, you can only use it as a type. still cool though
- ricksharp 6y agoBut if you declare a string literal as const, then the string runtime value will match the string type. So it would be possible to have a runtime result (that executes an actual db query for example) that is paired with the type result.
- james-mcelwain 6y agoNo, TypeScript is compile time only. There's no runtime type info.
- whatshisface 6y agoThe parent comment is saying that if you write the string twice, you can get the compiler to check #1 and the runtime to use #2.
- ricksharp 6y agoNot exactly (there is no duplication), a string constant in typescript is it's own type: const sql = 'SELECT ...'; type SQL = typeof sql; // type SQL = "SELECT ..."
- fbn79 6y agoMaybe useful to build dynamic-types based on configuration files I think.
- Tarean 6y agoThe implementation isn't even that horrible. Ignoring the integer list. That workaround is still hillarious to me. Anyway, it seems both impressive that someone did this and that it actually works. Weird how typescript has the nicest typelevel programming I know, even nicer than some dependent ones. It probably is the only language one where autocompletion properly works with computed types?
- phpnode 6y ago> The implementation isn't even that horrible You're making me blush! The list of integers could be replaced by using tuples and reading their lengths to do basic arithmetic. Might run into recursion limits though.
- jedimastert 6y ago> The list of integers could be replaced by using tuples and reading their lengths to do basic arithmetic. Is that anything like church encoding? https://www.wikiwand.com/en/Church_encoding#/Church_numerals https://www.wikiwand.com/en/Church_encoding#/Church_numerals
- jitl 6y agoNot really. In the Typescript type system tuple types are array types where the length is encoded in the type system. Tuple types can be “added” by concatenation to produce a new tuple type: type two = [0, 0] type three = [0, 0, 0] type five = [...two, ...three] type TupleToNumber<T extends any[]> = T[“length”] const num: TupleToNumber<five> = 5
- amitport 6y agoThis is a great demonstration of typescript's flexibility
- chrismorgan 6y agoI’m impressed/fascinated/horrified by the way it parses a string into a type with a variant of template literal syntax—though it looks like the parsing is probably very fragile. I’m curious why that functionality was added to the language. The end result is actually pretty approachable.
- phpnode 6y agoit's a bit fragile only due to my laziness, it could be done properly if this were a serious project.
- 3np 6y agoHow would it be done if done properly?
- phpnode 6y agosame mechanism, just deal with more edge cases. e.g. whitespace isn't really trimmed correctly at the moment, it doesn't handle escaping quotes in strings and so on. I did think about trying to match postgres's grammar but it's a lot of work and probably implementing a PEG first would make it easier.
- nicoburns 6y ago> I’m curious why that functionality was added to the language. I think it's because there are a whole bunch of APIs in javascript where there is a function like `.get("foo")` that internally calls a function called `getFoo` or similar. And this functionality allows such functions to be accurately typed. It basically allows strong typing for stringly typed functions (which are pretty common in dynamic languages)
- ianhorn 6y agoStringly typed functions frustrate me to no end. Tooling for programming gets better every year, yet stringly typed code is so resistant to any of those improvements. Pycharm at some point had to add SQL tools in python strings for gods sake. That kind of weird deep special casing is everywhere with strings drive behavior. Why use a class (inspectable) when a dictionary, a function, and an if statement (opaque) will do?
- cjdell 6y agoI had no idea you could compose string literal types AND infer variables using interpolation syntax in the context of conditional types. I am salivating at the potential use cases for this...
- ricksharp 6y agoRight! That is insane - I had no idea it was possible to infer part of a string literal. My mind is exploding at the possibilities.
- cuddlecake 6y agoIt's a new feature from TypeScript 4.1 (this version is yet in beta) https://devblogs.microsoft.com/typescript/announcing-typescript-4-1-beta/#template-literal-types https://devblogs.microsoft.com/typescript/announcing-typescr...
- cjdell 6y agoLots of new toys to play with in this release! They've also explained each new feature really well...
- dixon395 6y agoTypescript was created to bring static types in dynamic language, like JS. Now they made the static types more dynamic. What's next?)
- ricksharp 6y agoNice! I built something similar - A set of scripts that generate SQL statements from typescript types to wrap a PostgresSql db. It’s more a hybrid data structure where only relationships and certain fields become columns (which can be indexed and searched) and most of the other data becomes a Json column. I wasn’t completely satisfied with the final results, but it does at least allow me to add additional data fields without having to migrate the db structure and to use Postgres sql to filter and page data. I think the goal of being able to define a true database in typescript is great and I look forward to trying your code. It’s also pretty cool if you are able to extract types from a string using the type system - that’s next level typescript wizardry.
- ricksharp 6y agoThinking about this more. I think this project has some promising practical uses when combined with code generation from the typescript ast (which is what my project does). Since the type system can parse the SQL strings, then it becomes possible to include those strings as a source for the ast. Then the code generator can use the typescript parser to parse all the types and the sql. From that it can generate all the required code to create the runtime queries. I don’t like ORMs because they make it very difficult to write complex queries (and my own attempt did not solve that very well - I still had to manually type any complex queries). However, if this could essentially allow writing a complex query in raw sql and have it type-checked at code-time, it becomes easier to use. Then the code generator would convert that into a type-safe runtime library which solves 2 problems: 1. The poor editor performance of the type system inferring so many nested types. 2. Actually having a runtime that executes the queries. The outstanding problem would be migrations... Anyway, this is certainly a thought provoking project.
- thrashh 6y agoI find the ORM in SQLAlchemy pretty cool — I’ve really written complex queries as if I had witten it by hand but then finally mapped them into the ORM at the end with like a line of code. However it was a learning curve to figure out how to do it at first, but I didn’t have to drop down to raw SQL
- 6y ago
- ralusek 6y agoI'm viewing this on my phone, so maybe I'm missing something, but isn't this getting all of its typing information from you explicitly declaring a result set `as const`, and then subsequently doing `typeof` that const value? How does that help in real world queries, where the result sets are dynamic? Also, how does it help beyond just naturally declaring a result set as const without the use of this library?
- phpnode 6y agooh let's be clear there's no sensible real world use cases of this thing. it's a demonstration of what TS's type system can do
- chrismorgan 6y agoArt is all about making interesting things that serve no functional purpose, though often they have useful byproducts. This is a work of art.
- nikki93 6y agoOne use case of something similar is generating a regular expression matcher at compile time for example. This is what the C++ library https://github.com/hanickadot/compile-time-regular-expressions https://github.com/hanickadot/compile-time-regular-expressio... does. There are a lot of uses of compile-time code. One I have been interested in is generating bindings to other languages or generating serialization code, or even generating bytecodes for VMs with specific bytecodes for accessing your native data structures (instead of adding a level of indirection).
- tobyhinloopen 6y agoThis project allows you to query (transform) type definitions as data to create new type definitions using SQL. Dynamically transforming types could be useful to generate complex types based on other complex types. This project takes it to the extreme by adding an SQL interface to it
- 6y ago
- esamatti 6y agoOther uses for the template literal types: JSON parser https://twitter.com/buildsghost/status/1301976526603206657 https://twitter.com/buildsghost/status/1301976526603206657 This one is more realistic: Typed route params https://twitter.com/danvdk/status/1301707026507198464 https://twitter.com/danvdk/status/1301707026507198464
- dandare 6y agoCould someone ELI5? Actually, I wrote a lot of code in TypeScript and I know my SQL but I have absolutely no idea what is going on here.
- alex_duf 6y agoI'm not entirely sure, but I suspect it's one of these programming "joke" where you take a concept and bring it far enough that it becomes ridiculous. So I'm guessing there's no real practical use case for this, and is a nice flex from the author. It would be nice if that were made dead clear in the Readme, something like "Why? Because it's fun"
- gfody 6y agostrong typing for arbitrary sql queries is actually really nice and this is much better than stuff like sequelize or typeorm which seem sadistic in comparison
- nsonha 6y agogenerating types from a swagger spec? Graphql? Or any kind of schema? Look up fsharp type providers.
- alex_duf 6y agoAs far as I understand (and I might be wrong so please correct me), this can't be used to generate anything. Of course all the technologies you've listed are useful and brilliant but this isn't what I'm seeing. It's more of an experiment of thoughts to generate types that are defined by the result of the SQL query, bending a compiler to a new strange behaviour that happens to be SQL like.
- nsonha 6y agoIt definitely can parse anything, the code that is required to make such a parser might be a bit unreadable but look up somewhere in this thread there is a twitter link to a github project of a generic json parser. Typescript 4.1 isn't event released yet, once it is there will be a dozen of projects like this. If you think this is useless because it doesn't generate any code then I'm assuming you're not familiar with typescript because type level libraries isn't supposed to do do that anyway. There are already libraries that generate open api client and graphql client but they generate type definition via an extra build step which libraries like this can eliminate. And this typescript feature was made to parse strings so no this is not "bending the compiler"
- cjdell 6y agoTypeScript is becoming such a compelling language due to its insanely advanced type system (that allows for projects like this) that I now want to use it everywhere. I want it to become the next Python. I know Deno is supposed to be first class TypeScript but under the hood it's still a JavaScript runtime with all the baggage that comes with that. AssemblyScript is extremely interesting but last time I played with it I concluded it wasn't yet ready for anything serious, or is that no longer the case? A first class TypeScript runtime with no JS overhead would be a dream come true. I just hope it happens someday, or that other type systems get these amazing features.
- Wilem82 6y agoIs this satire? What do you think is advanced about TS' type system and why? Duck-typing that creates a minefield instead of providing correctness? The unknown type that does the same? TS is a step forward from JS, but JS's bar is so infamously low that making something better isn't a big achievement, especially compared to other languages with normal type systems.
- kingdomcome50 6y agoI'm honestly curious. What other languages have a type system the would allow for this project? That is, (a subset of) SQL as a type. And assuming these languages exist, are they as ergonomic to the developer as TS while instrumenting the above?
- bhouston 6y agoIs typescripts type system becoming like c++ template meta programming?
- bebe3000 6y agoThis whole magic can be explained in one sentence. `infer` is introduced to TypeScript's templating. The rest, I mean this sql parser is just the result of this new "tool".
- tobyhinloopen 6y agoThis is terrible, I love it
- aleksjess 6y ago> a database implemented purely IN TS Ok, we might have went a bit too far
- soheilpro 6y agoThis is insane! Just curios, are there real-world use cases for stuff like this?
- williamdclt 6y agoI could imagine working on very complex static JSON. You want to type-check selections and transformations on this JSON. It's a very niche use-case, this is definitely more of an impressive showcase of the new Template Literal Types feature of TS :)
- ricksharp 6y agoYou can import a json document directly into typescript and the entire document is a readonly constant type of the entire json object literal. I've used this with a template JSON document that has an instance of all possible types in the runtime json. Sometimes you might have to loosen the type slightly, but it works great for a quick start.
- greencore 6y agoYou can embed any language interpreter into TypeScript and have that language syntaticaly checked by TS compiler.
- flowerlad 6y agoThis shows how complex TypeSript has become. It may be fun to write complicated Type annotations such as this, but imagine having to read and maintain the code!
- smt88 6y agoFrom years of real-world experience, it's rare to run into types that are too complex to easily understand. The syntax is pretty easy to read, and the compiler and language service make it easy to navigate types, even if you haven't looked at the source for them yet.
- alangpierce 6y agoFWIW I find that TypeScript types can sometimes be overwhelming or hard to decipher, especially some third-party library type definitions. The connect function from react-redux is one example: https://github.com/DefinitelyTyped/DefinitelyTyped/blob/d07d3eefaa7647833dd1446808a8eccd018f3ec9/types/react-redux/v6/index.d.ts#L153 https://github.com/DefinitelyTyped/DefinitelyTyped/blob/d07d... Since types can get really complex, I feel a need to be conscious of avoiding complexity when writing types myself, unless the value is enough to justify a hard-to-read type.
- freeqaz 6y agoThis is the complexity introduced when applying static types to a vanilla JS codebase, imo. It's easy to get crazy with types in vanilla JS. Adding Typescript forces you to resolve the different types explicitly and it can get hairy. What this really demonstrates to me is the difference between starting with TS vs migrating to it. Code that has to think about the types the first time it is being built is going to be better structured, on average, than purely dynamic code.
- eyelidlessness 6y agoMany complex/difficult to understand TS types exist precisely because the type system was too limited to express certain real world use cases without workarounds. As the type system takes on that complexity, and removes more limitations, it actually makes writing simpler types for those cases much easier.
- meijer 6y agoThis feels a bit like C#'s "Linq to Objects". Could actually be useful!
- bartq 6y agoThat's pretty neat. What I really want is this: function fn(query:string){ const stuff = // do some stuff with `query` variable return createType(stuff); } type MyType = FromJS<fn("SELECT id, name AS nom FROM things WHERE active = true")>; This way we could compute types on the fly with JavaScript instead of creating monstrous types in TypeScript types system.
- Justsignedup 6y agowouldn't that be neat... However, there are Java frameworks that basically generate a type system from your database so whatever columns you query ends up being static types that can be determined at compile time. https://www.jooq.org/ https://www.jooq.org/ Java is really quite cool. I wonder if we'll ever get a typescript version of jOOQ
- NieDzejkob 6y agoSimilar libraries are also available for Rust, see diesel [1] or sqlx [2]. [1]: https://diesel.rs/ https://diesel.rs/ [2]: https://crates.io/crates/sqlx https://crates.io/crates/sqlx
- bartq 6y agoI think we eventually will be able to evaluate JS in context of types and return whatever we want even asynchronously. Now we have some combinators given by TS authors which allow us to compose some limited set of computations. I don't know how Java does it, but I hope TS will do it better :-)
- nsonha 6y agoif we are talking about code gen then it exists in every language, nothing extraordinary here. In nodejs, there is prisma for example. This typescript thing generates static type on the fly, not by generating code.
- nikki93 6y agoInterestingly this is something you can do in Zig with its `@Type`. Zig's compiler can eval (some) Zig code at compile time, this code can produce new types, and you can mention those types anywhere types are normally mentioned.
- alangpierce 6y agoGreat project, really expands what feels possible with the new improvements in TS 4.1. I wonder if this strategy could also be used to have strongly-typed GraphQL queries without the need to have a type generation step that you run separately. Currently when using GraphQL with Apollo, it's great that you get types corresponding to all queries, but having to run the type generator and maintain types in __generated folders is definitely an extra hassle.
- gmac 6y agoI'm interested to see if I can use these new template string type features to add more/better type checking and inference to my TS Postgres library[1]. My suspicion is that the TS recursion depth limit of 50 might limit this rather seriously. [1] https://jawj.github.io/zapatos/ https://jawj.github.io/zapatos/
- g5becks 6y agoLove Zapatos. Hope there’s a way to get this working!
- deleted 6y ago[deleted]
- bionhoward 6y agoJavaScript is such a cool language, and I really love it, especially the FP subcommunity, but the AI community for Python is way bigger, I wish there were more options for neural nets, differentiable programming, and data frame type things in JS. The interoperability with WASM is hugely valuable, and so is the ability to easily publish results on the web. It feels like it’s way easier to make an AI system in python, but then you have to figure out how to monetize it, while if you use JS, you can build a web platform or service and monetize much easier. Thoughts?
- ozorOzora 6y agoI think these are two different languages for two different jobs. I can see javascript slowly expanding from the front-end into the back-end, mostly thanks to projects like TypeScript that appeal to the back-end crowds, but also because of the whole serverless hype which IMO is justified. Now there still is a strong "cultural divide" between the software and the data-science communities. And when you see how strong the innertia for python's projects is.. Well I guess we won't see this changing anytime soon.
- yowlingcat 6y agoI think you're talking about two different systems and two different problems. First, there's the core service code, the AI system in Python you're talking about. Then, there's the web platform or service that you monetize. But there's no reason why you wouldn't just use both. Write the core AI code in Python because that's what's practical to do. Write the app code in JS because that's what's practical to do. Sell the product to customers that want to use it and probably don't care what's underneath.
- mewpmewp2 6y agoIn my opinion it would be great to have a single language for developer productivity. Backend and frontend can be already written fully in TypeScript and you can share interfaces between frontend and backend so everything is connected and business logic is not repeated. Same thing could be done for machine learning. If backend changes frontend has to change as well or compiler will yell. Same for machine learning input and output.
- primitivesuave 6y agoIs this just storing data in memory and querying it with SQL? I use DynamoDB Data Mapper in production (https://github.com/awslabs/dynamodb-data-mapper-js https://github.com/awslabs/dynamodb-data-mapper-js) - would be cool to see something like this for SQL. Effectively define a class with TS decorators to automatically generate tables and columns.
- yonixw 6y agohttps://typeorm.io/ https://typeorm.io/ support: > MySQL / MariaDB / Postgres / CockroachDB / SQLite / Microsoft SQL Server / Oracle / SAP Hana / sql.js
- UhDev 6y agoI think that TypeScript is nice but I would never touch a database running on javascript.
- svieira 6y agoOh, this isn't running on JavaScript - this is running on TypeScript's _typer_! ... which is implemented in JavaScript ... er ... don't know if that changes your desires to use this in PRODUCTION.
- kamilafsar 6y agoVery interesting project. If you want a deep dive into TypeScript generics, this project's code is truly a must-read! This project really shows how powerful TypeScript has become over the years. Shameless plug: we're also working on project in the same domain. It actually uses the TypeScript compiler API to generate/update your SQL schema's and a typesafe (mongo like) API on top PostgreSQL. It's still very early stage, but if you're interested: https://samen.io https://samen.io
- thelazydogsback 6y agoI love Typescript -- I just wish there were an alternate runtime (.Net core CLR/DLR or other) that offered shared-memory green and/or native threading for true support for embarrassingly parallel but not vectorized workloads. FWICT, I don't think Deno changes things here much. Typescript on the BEAM would be interesting to me as well.
- eyelidlessness 6y agoI would absolutely love to work with the TypeScript type system on BEAM, if it didn't leak as much of Erlang particulars as other BEAM languages like Elixir do.
- g5becks 6y agoI think I’d prefer compilation to Go.
- kwindla 6y agoRelated, C++ template metaprogramming. Very useful, quite complicated to wrap your head around initially (at least it was for me), and a somewhat accidental language feature: https://softwareengineering.stackexchange.com/questions/125314/in-c-were-sfinae-and-metaprogramming-intentional-or-just-a-byproduct-of-templ https://softwareengineering.stackexchange.com/questions/1253...
- gitgud 6y agoWow, I hope the typescript compiler is ready for this kind of dynamic complexity... type Protocol = 'http' | 'https'; type DomainExtension = 'com' | 'org' | 'net'; type DomainName = 'example' | 'google'; type Domain = `${DomainName}.${DomainExtension}`; type Path = string; type ApiEndpoint = `${Protocol}://${Domain}/${Path}`; This would seemingly get out of hand very quickly...
- ruffrey 6y agoI work on a codebase with a lot of this type of code. So far the compiler is up for it. The only time it gets slow is using the Zod language for runtime validation.
- z3t4 6y agoPersonally I find RegExp more readable: SELECT(.)FROM(.)WHERE?(.)ORDER BY?(.)LIMIT?(.*)
- crazypython 6y agoDLang can sort an array at compile-time using a normal function: (CTFE) void main() { import std.algorithm, std.conv, std.stdio; "Starting program".writeln; // Sort a constant declaration at Compile-Time enum a = [ 3, 1, 2, 4, 0 ]; static immutable b = sort(a); // Print the result _during_ compilation pragma(msg, text("Finished compilation: ", b)); } It can also reflect at compile time on user-defined annotations, which can store arbitrary metadata on any symbol, to generate arbitrary code at compile time. They serve a similar role to procedural macros from Rust, but they're easier to use than the earlier. Thanks to mixin templates, you can inject arbitrary symbols, making D's UDAs just as flexible as Python's decorators.
- blitz_skull 6y agoGenuinely curious: is this actually useful or just a cool thing that is useful simply for the fact that it can be done?