4 ms·

Show HN: Every open source tool from the "What's HN working on" thread

169 points by themanmaran 2y ago

Jun8 2y ago
This is awesome, thanks for doing this! Did you use Omni for the sentiment analysis? Now, for the analysis of the Who’s Hiring posts.
mjcurl 2y ago
FYI some of the GitHub links return 404.
SeanAnderson 2y ago
Nice, this is cool :) Thanks for sharing
dang 2y ago
Recent and related: Ask HN: What are you working on (August 2024)? - https://news.ycombinator.com/item?id=41342017 https://news.ycombinator.com/item?id=41342017 - Aug 2024 (1365 comments)
- Rendello 2y ago
  Also, from today: Ask HN: Who's building an AI-free product? - https://news.ycombinator.com/item?id=41375100 https://news.ycombinator.com/item?id=41375100
cperciva 2y ago
Not sure how your code is identifying "open source", but you missed including FreeBSD on the list.
- themanmaran 2y ago
  It got tagged as open source, you can see it in the csv [1]. But the original comment didn't have a link and table I shared was filtered to just posts with links to the repo. [1] https://github.com/getomni-ai/datasets/blob/main/hn_projects_dataset.csv#L434 https://github.com/getomni-ai/datasets/blob/main/hn_projects... [2] https://news.ycombinator.com/item?id=41342592 https://news.ycombinator.com/item?id=41342592
maxbond 2y ago
Cool stuff! You might consider an alternative to screenshots for tables. There are a few problems with this strategy. Most significantly, it's inaccessible to the visually impaired - they're hearing your description of the data without getting to read the data itself, which must be frustrating. Problems like this can feel hypothetical and abstract, but HN is a large enough audience that there are probably real people having this experience. It also can't be copy pasted, it can't be plotted, it doesn't respect the settings of my browser like font size, etc. When I see some of these breakdowns, my first thought is, "hmm, I'd like to copy paste this into a spreadsheet and make a bar chart." (Indeed, that's when I noticed it was a screenshot.) You might consider copy pasting from DataGrip into your editor and formatting it to be a markdown list or something. It probably would take about the same amount of time as taking a screenshot and putting it into your markdown. Thanks again for sharing, this was an interesting breakdown.
- themanmaran 2y ago
  You know I agonized over this for a bit before giving up and adding the screenshots. Originally I was trying to create some bar charts as well but decided to just post and edit later. Planning on a better writeup later and will incorporate this feedback.
  - maxbond 2y ago
    For sure. Done is better than perfect and iteration is a great strategy. I wouldn't have made this criticism if I'd realized you agonized over it.
legohorizons 2y ago
This is awesome! Is this using Omni? Would love to see this for more topics, like products people are working on
- themanmaran 2y ago
  Yup! And this was pretty quick to pull together. I'll definitely be publishing some more of these type datasets.
deleted 2y ago
[deleted]
fuzzythinker 2y ago
Some Show HNs are linked to sites/blogs with open source links. Next iteration of this can include those.
- themanmaran 2y ago
  Oh yea. I wanted to do some recursive web scraping to pull more info. hn_comment => demo_link => github as well as hn_comment => github => demo_link One of these weekends I'll pull the last year of this thread and get a full dataset.
simonw 2y ago
Since this is a CSV on GitHub: https://github.com/getomni-ai/datasets/blob/main/hn_projects_dataset.csv https://github.com/getomni-ai/datasets/blob/main/hn_projects... You can open it in Datasette Lite (WebAssembly in your browser) to run SQLite SQL against it like this: https://lite.datasette.io/?csv=https://github.com/getomni-ai/datasets/blob/main/hn_projects_dataset.csv https://lite.datasette.io/?csv=https://github.com/getomni-ai... Here's a SQL query showing just the open source projects focused on AI, for example: https://lite.datasette.io/?csv=https://github.com/getomni-ai/datasets/blob/main/hn_projects_dataset.csv#/data?sql=select+one_liner%2C+github_link%2C+comment%2C+hn_user+from+hn_projects_dataset+where+%22is_open_source%22+%3D+%3Ap0+and+%22project_industry%22+%3D+%3Ap1+order+by+rowid+limit+101&p0=true&p1=ARTIFICIAL_INTELLIGENCE https://lite.datasette.io/?csv=https://github.com/getomni-ai...
- pdyc 2y ago
  how is this opening it? i am getting cors errors on my tool, are you doing some server side processing?
  - pdyc 2y ago
    answering my own question. github raw content does not have cors problem. You can construct the raw content link from the normal github content link to download using fetch.
- samstave 2y ago
  THIS IS FN DOPE. --- I need more examples of datasette! Thanks for showing this.
  - simonw 2y ago
    Turns out I've shared a whole bunch of examples like this in comments here in the past! https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=lite.datasette.io&sort=byPopularity&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
david927 2y ago
Great idea! Thanks for doing this!
adriamaker 2y ago
Hey! Cool!
yoavm 2y ago
why is the description for https://github.com/madprops/goldie https://github.com/madprops/goldie is "Firefox Vertical Tabs Powerful Python Chat Client Nim Text Finder" ?
- brazzledazzle 2y ago
  It got those from this comment and transposed the descriptions: https://news.ycombinator.com/item?id=41347438 https://news.ycombinator.com/item?id=41347438
mpduda 2y ago
Pretty cool, nice to have an easy to read list. One issue I noticed with my project [1] is that the description is for the wrong thing. This is probably partially my fault, since I wrote about two projects in one comment [2]. The music production app is closed source, but the UI library I maintain for it is open source. Seems like that might have thrown off your AI tool. [1] https://github.com/Pulselyre/UpbeatUI https://github.com/Pulselyre/UpbeatUI [2] https://news.ycombinator.com/item?id=41342977 https://news.ycombinator.com/item?id=41342977
- potatoball 2y ago
  I noticed this was the case for the project laudspeaker as well. The description says "Techno Thriller Panopticon Explores Encryption and Espionage" but the project is for customer engagement and onboarding.
  - themanmaran 2y ago
    Oh yea the titles are a little off, since they're pulling off of the original comment and not the repo itself. The Laudspeaker description came from: "Apart from working on my start up, and open source project https://github.com/laudspeaker/laudspeaker https://github.com/laudspeaker/laudspeaker (open source firebase cloud messaging) I've been writing a short sci fi story, sort of like a techno thriller inspired by michael crichton, called Panopticon." https://news.ycombinator.com/item?id=41348190 https://news.ycombinator.com/item?id=41348190
  - abe94 2y ago
    good catch - yes I added two projects in one comment! laudspeaker (open source firebase cloud messaging/customer engagment) - https://github.com/laudspeaker/laudspeaker https://github.com/laudspeaker/laudspeaker panopticon (short story on encryption and espionage) - https://docs.google.com/document/d/1VRI4X5fCUpwurUDvKmvzJpT7G50wIq6Ac0wa4uY1Avo/edit?usp=sharing https://docs.google.com/document/d/1VRI4X5fCUpwurUDvKmvzJpT7...
denysvitali 2y ago
I don't think it's a good idea to give a DB connection string to everyone, even if it's read only
- themanmaran 2y ago
  Funny enough, looking at the db logs, no one queried it. I think simonw made it so much easier with Datasette that no one bothered. But I did spin up a single use free tier supabase db for this usecase. Spent a lot of time worrying some hacker would find an exploit.
password4321 2y ago
As submitted 2 days ago with an extensive sales pitch but may be useful. A self second chance queue submission, perhaps! https://news.ycombinator.com/item?id=41351839 https://news.ycombinator.com/item?id=41351839
sweeter 2y ago
This is my favorite part about software
kixpanganiban 2y ago
This is amazing! I've been meaning to do something similar for all the Show HN threads, granted it's a much bigger set, but I haven't had the chance to.

169 points by themanmaran 2y ago

Jun8 2y ago

This is awesome, thanks for doing this! Did you use Omni for the sentiment analysis? Now, for the analysis of the Who’s Hiring posts.

mjcurl 2y ago

FYI some of the GitHub links return 404.

SeanAnderson 2y ago

Nice, this is cool :) Thanks for sharing

dang 2y ago

Recent and related: Ask HN: What are you working on (August 2024)? - https://news.ycombinator.com/item?id=41342017 https://news.ycombinator.com/item?id=41342017 - Aug 2024 (1365 comments)

Rendello 2y ago
Also, from today: Ask HN: Who's building an AI-free product? - https://news.ycombinator.com/item?id=41375100 https://news.ycombinator.com/item?id=41375100

cperciva 2y ago

Not sure how your code is identifying "open source", but you missed including FreeBSD on the list.

themanmaran 2y ago
It got tagged as open source, you can see it in the csv [1]. But the original comment didn't have a link and table I shared was filtered to just posts with links to the repo. [1] https://github.com/getomni-ai/datasets/blob/main/hn_projects_dataset.csv#L434 https://github.com/getomni-ai/datasets/blob/main/hn_projects... [2] https://news.ycombinator.com/item?id=41342592 https://news.ycombinator.com/item?id=41342592

maxbond 2y ago

Cool stuff! You might consider an alternative to screenshots for tables. There are a few problems with this strategy. Most significantly, it's inaccessible to the visually impaired - they're hearing your description of the data without getting to read the data itself, which must be frustrating. Problems like this can feel hypothetical and abstract, but HN is a large enough audience that there are probably real people having this experience. It also can't be copy pasted, it can't be plotted, it doesn't respect the settings of my browser like font size, etc. When I see some of these breakdowns, my first thought is, "hmm, I'd like to copy paste this into a spreadsheet and make a bar chart." (Indeed, that's when I noticed it was a screenshot.) You might consider copy pasting from DataGrip into your editor and formatting it to be a markdown list or something. It probably would take about the same amount of time as taking a screenshot and putting it into your markdown. Thanks again for sharing, this was an interesting breakdown.

themanmaran 2y ago
You know I agonized over this for a bit before giving up and adding the screenshots. Originally I was trying to create some bar charts as well but decided to just post and edit later. Planning on a better writeup later and will incorporate this feedback.
- maxbond 2y ago
  For sure. Done is better than perfect and iteration is a great strategy. I wouldn't have made this criticism if I'd realized you agonized over it.

legohorizons 2y ago

This is awesome! Is this using Omni? Would love to see this for more topics, like products people are working on

themanmaran 2y ago
Yup! And this was pretty quick to pull together. I'll definitely be publishing some more of these type datasets.

deleted 2y ago

[deleted]

fuzzythinker 2y ago

Some Show HNs are linked to sites/blogs with open source links. Next iteration of this can include those.

themanmaran 2y ago
Oh yea. I wanted to do some recursive web scraping to pull more info. hn_comment => demo_link => github as well as hn_comment => github => demo_link One of these weekends I'll pull the last year of this thread and get a full dataset.

simonw 2y ago

Since this is a CSV on GitHub: https://github.com/getomni-ai/datasets/blob/main/hn_projects_dataset.csv https://github.com/getomni-ai/datasets/blob/main/hn_projects... You can open it in Datasette Lite (WebAssembly in your browser) to run SQLite SQL against it like this: https://lite.datasette.io/?csv=https://github.com/getomni-ai/datasets/blob/main/hn_projects_dataset.csv https://lite.datasette.io/?csv=https://github.com/getomni-ai... Here's a SQL query showing just the open source projects focused on AI, for example: https://lite.datasette.io/?csv=https://github.com/getomni-ai/datasets/blob/main/hn_projects_dataset.csv#/data?sql=select+one_liner%2C+github_link%2C+comment%2C+hn_user+from+hn_projects_dataset+where+%22is_open_source%22+%3D+%3Ap0+and+%22project_industry%22+%3D+%3Ap1+order+by+rowid+limit+101&p0=true&p1=ARTIFICIAL_INTELLIGENCE https://lite.datasette.io/?csv=https://github.com/getomni-ai...

pdyc 2y ago
how is this opening it? i am getting cors errors on my tool, are you doing some server side processing?
- pdyc 2y ago
  answering my own question. github raw content does not have cors problem. You can construct the raw content link from the normal github content link to download using fetch.
samstave 2y ago
THIS IS FN DOPE. --- I need more examples of datasette! Thanks for showing this.
- simonw 2y ago
  Turns out I've shared a whole bunch of examples like this in comments here in the past! https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=lite.datasette.io&sort=byPopularity&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

david927 2y ago

Great idea! Thanks for doing this!

adriamaker 2y ago

Hey! Cool!

yoavm 2y ago

why is the description for https://github.com/madprops/goldie https://github.com/madprops/goldie is "Firefox Vertical Tabs Powerful Python Chat Client Nim Text Finder" ?

brazzledazzle 2y ago
It got those from this comment and transposed the descriptions: https://news.ycombinator.com/item?id=41347438 https://news.ycombinator.com/item?id=41347438

mpduda 2y ago

Pretty cool, nice to have an easy to read list. One issue I noticed with my project [1] is that the description is for the wrong thing. This is probably partially my fault, since I wrote about two projects in one comment [2]. The music production app is closed source, but the UI library I maintain for it is open source. Seems like that might have thrown off your AI tool. [1] https://github.com/Pulselyre/UpbeatUI https://github.com/Pulselyre/UpbeatUI [2] https://news.ycombinator.com/item?id=41342977 https://news.ycombinator.com/item?id=41342977

potatoball 2y ago
I noticed this was the case for the project laudspeaker as well. The description says "Techno Thriller Panopticon Explores Encryption and Espionage" but the project is for customer engagement and onboarding.
- themanmaran 2y ago
  Oh yea the titles are a little off, since they're pulling off of the original comment and not the repo itself. The Laudspeaker description came from: "Apart from working on my start up, and open source project https://github.com/laudspeaker/laudspeaker https://github.com/laudspeaker/laudspeaker (open source firebase cloud messaging) I've been writing a short sci fi story, sort of like a techno thriller inspired by michael crichton, called Panopticon." https://news.ycombinator.com/item?id=41348190 https://news.ycombinator.com/item?id=41348190
- abe94 2y ago
  good catch - yes I added two projects in one comment! laudspeaker (open source firebase cloud messaging/customer engagment) - https://github.com/laudspeaker/laudspeaker https://github.com/laudspeaker/laudspeaker panopticon (short story on encryption and espionage) - https://docs.google.com/document/d/1VRI4X5fCUpwurUDvKmvzJpT7G50wIq6Ac0wa4uY1Avo/edit?usp=sharing https://docs.google.com/document/d/1VRI4X5fCUpwurUDvKmvzJpT7...

denysvitali 2y ago

I don't think it's a good idea to give a DB connection string to everyone, even if it's read only

themanmaran 2y ago
Funny enough, looking at the db logs, no one queried it. I think simonw made it so much easier with Datasette that no one bothered. But I did spin up a single use free tier supabase db for this usecase. Spent a lot of time worrying some hacker would find an exploit.

password4321 2y ago

As submitted 2 days ago with an extensive sales pitch but may be useful. A self second chance queue submission, perhaps! https://news.ycombinator.com/item?id=41351839 https://news.ycombinator.com/item?id=41351839

sweeter 2y ago

This is my favorite part about software

kixpanganiban 2y ago

This is amazing! I've been meaning to do something similar for all the Show HN threads, granted it's a much bigger set, but I haven't had the chance to.