3 ms·

Show HN: GETlang – A Query Language for the Web

7 points by mattfysh 2y ago

hi hn, today I'm open sourcing a new SQL-like query language that's built for the web. it has dedicated syntax for request, parsers, selectors, and javascript snippets... it was built on nodejs with the incredible moo lexer and nearley parser. if you're a fan of regular expressions, do I have some code to show you! the website is mostly just a few examples and a playground where you can write & run (& share!) your own queries. there's also an introductory blog post where i try but ultimately fail to justify why this should be its own language and not a library/framework. enjoy!

breck 2y ago
Interesting. I was about to shrug it off and then saw your had a web playground and so gave it a shot. Very well executed. More examples would be nice. For example, I was easily able to get it to extract titles, but then I was curious--how would I get titles and links, combine them, et cetera. https://getlang.dev/query?c3cNUcgoKSkottLXTypKTc6uLM3LLCnWS87P5eJKrSgpSkwuUajmUlAoySzJSeVSUCjILy4ptlKwtVPQK04uys/JCQFL1HIBAA https://getlang.dev/query?c3cNUcgoKSkottLXTypKTc6uLM3LLCnWS8... One high level thought: There are 2 kinds of behaviors in this world: 1. the good, trustworthy behavior of people wanting to share data with you. The better long term solution for people with this habit, IMO, is to start moving to what comes after the web (a new language to replace HTML, where data is pristine - https://breckyunits.com/wws.html https://breckyunits.com/wws.html). 2. bad behavior of trying to restrict/hide/exploit/hoard data. IMO AIs/LLMs are the counter to this. I am not sure if there is a big enough market to thread the needle with your thing, but if there is, you've certainly demonstrated you are the person to build it. Nicely done.
- mattfysh 2y ago
  thanks for the feedback breck, I'm glad to hear you were able to get the playground running! the examples do need a bit more polish for clarity & to point out each of the language features they're demonstrating. there are 7 examples there which go through 90% of features, although I'm not sure if it's obvious you can scroll to the right to see more tabs? in the case of extracting data from lists -- first, select a list of the common parent (using the `=>` expanding list drill you had used), and then use a regular drill (->) into each item to select data from elements in the list, e.g. GET https://breckyunits.com extract { title posts: => .scrollSnippetContainer -> { title: h1 url: a -> @link date?: .scrollDateline } } https://getlang.dev/query?NY1RCsIwEET/9xR7AVv8DVgFFQ+gF4hxoaHrJiRTUKR3lwb9nDe8mcv5xiOQq+v7e5EwvWeLqF1ITyJ5ofgA/hAzIlSIOaeK6ng3cFdDSapXizkLjsngo0nhzdCEn+J43LY0F3Xs1/ag0abGHh6yd/+lk4dotPVloYW+ https://getlang.dev/query?NY1RCsIwEET/9xR7AVv8DVgFFQ+gF4hxoa... agree fully with your classification of open/closed data. this project was born from the desire to make data from the second cohort more open & accessible, in the hopes of progressing what the web can be, and putting power back to into the hands of users in terms of how/when they interact with web data. once the documentation is done, it would be great to see whether I can train a model to produce getlang queries through natural language, e.g. you provide the AI with a URL and a list of data points to retrieve; and it generates queries & tests them for completeness and accuracy before committing them to shared library of queries. thanks again for taking a look, let me know if you need a hand with writing queries or if you have any followup thoughts! matt.
  - breck 2y ago
    > once the documentation is done, it would be great to see whether I can train a model to produce getlang queries through natural language, e.g. you provide the AI with a URL and a list of data points to retrieve; and it generates queries & tests them for completeness and accuracy before committing them to shared library of queries. Ah great idea. You should be able to try this right now as well with modern context sizes. Looking forward to see how that develops!

7 points by mattfysh 2y ago

breck 2y ago

Interesting. I was about to shrug it off and then saw your had a web playground and so gave it a shot. Very well executed. More examples would be nice. For example, I was easily able to get it to extract titles, but then I was curious--how would I get titles and links, combine them, et cetera. https://getlang.dev/query?c3cNUcgoKSkottLXTypKTc6uLM3LLCnWS87P5eJKrSgpSkwuUajmUlAoySzJSeVSUCjILy4ptlKwtVPQK04uys/JCQFL1HIBAA https://getlang.dev/query?c3cNUcgoKSkottLXTypKTc6uLM3LLCnWS8... One high level thought: There are 2 kinds of behaviors in this world: 1. the good, trustworthy behavior of people wanting to share data with you. The better long term solution for people with this habit, IMO, is to start moving to what comes after the web (a new language to replace HTML, where data is pristine - https://breckyunits.com/wws.html https://breckyunits.com/wws.html). 2. bad behavior of trying to restrict/hide/exploit/hoard data. IMO AIs/LLMs are the counter to this. I am not sure if there is a big enough market to thread the needle with your thing, but if there is, you've certainly demonstrated you are the person to build it. Nicely done.

mattfysh 2y ago
thanks for the feedback breck, I'm glad to hear you were able to get the playground running! the examples do need a bit more polish for clarity & to point out each of the language features they're demonstrating. there are 7 examples there which go through 90% of features, although I'm not sure if it's obvious you can scroll to the right to see more tabs? in the case of extracting data from lists -- first, select a list of the common parent (using the `=>` expanding list drill you had used), and then use a regular drill (->) into each item to select data from elements in the list, e.g. GET https://breckyunits.com extract { title posts: => .scrollSnippetContainer -> { title: h1 url: a -> @link date?: .scrollDateline } } https://getlang.dev/query?NY1RCsIwEET/9xR7AVv8DVgFFQ+gF4hxoaHrJiRTUKR3lwb9nDe8mcv5xiOQq+v7e5EwvWeLqF1ITyJ5ofgA/hAzIlSIOaeK6ng3cFdDSapXizkLjsngo0nhzdCEn+J43LY0F3Xs1/ag0abGHh6yd/+lk4dotPVloYW+ https://getlang.dev/query?NY1RCsIwEET/9xR7AVv8DVgFFQ+gF4hxoa... agree fully with your classification of open/closed data. this project was born from the desire to make data from the second cohort more open & accessible, in the hopes of progressing what the web can be, and putting power back to into the hands of users in terms of how/when they interact with web data. once the documentation is done, it would be great to see whether I can train a model to produce getlang queries through natural language, e.g. you provide the AI with a URL and a list of data points to retrieve; and it generates queries & tests them for completeness and accuracy before committing them to shared library of queries. thanks again for taking a look, let me know if you need a hand with writing queries or if you have any followup thoughts! matt.
- breck 2y ago
  > once the documentation is done, it would be great to see whether I can train a model to produce getlang queries through natural language, e.g. you provide the AI with a URL and a list of data points to retrieve; and it generates queries & tests them for completeness and accuracy before committing them to shared library of queries. Ah great idea. You should be able to try this right now as well with modern context sizes. Looking forward to see how that develops!