4 ms·

It is indeed an entire website built in a couple of hours with chatGPT or any other similar LLM. That it stands where it is rn on the home page is quite worryi

It is indeed an entire website built in a couple of hours with chatGPT or any other similar LLM. That it stands where it is rn on the home page is quite worrying.

behindai 2y ago
I believe that probably in the future that would be possible. However, today, nope. The website itself is a system of crawling, parsing, analysis, ML and LLM—months of work.
- PaulHoule 2y ago
  It is not months of work. I wrote an HN crawler that got all the posts in less than two hours of coding. (There is the tricky question of polling schedules though: if you want to get every story the minute it comes out it will not have any votes or comments and you'll have to come back later if you want to get representative numbers. I think discussions close in 2 weeks so a double scan is good enough for some purposes but not the up-to-the-minute analysis they claim to be doing) The rest of the analysis is all straightforward except for the topic classification: If I was running on all cylinders for nights and weekend hacking I'd expect to spend a week or so on gathering training data for that (maybe suck down category assignments from Tildes) and probably a week coding up a classifier w/ a BERT-family embedding and classifiers from scikit-learn. You could attempt the same with prompt engineering in a day or two of work and get something expensive to run working pretty quickly that will probably not perform as well as my classifier. (Their classifier gives the empty string as a category which is a bad sign) If you want to improve on it you need the same training set I do.
  - behindai 2y ago
    You have enough interesting points to do your project. Would love to see the results! Cheers!
PaulHoule 2y ago
A lot of it is wrong too. It gets my post total for the last 24 hours low by a factor of 5 or so. If it took a snapshot over the weekend it might have been that low because my submission queue bottomed out but if it is live it is wrong. The list of posts by category has the empty string as a category. Any kind of votes/submission, comments/submission, or similar ratio is going to be driven by luck in a 24 hour period. They should fix the bugs and try it again.

It is indeed an entire website built in a couple of hours with chatGPT or any other similar LLM. That it stands where it is rn on the home page is quite worrying.

behindai 2y ago

I believe that probably in the future that would be possible. However, today, nope. The website itself is a system of crawling, parsing, analysis, ML and LLM—months of work.

PaulHoule 2y ago
It is not months of work. I wrote an HN crawler that got all the posts in less than two hours of coding. (There is the tricky question of polling schedules though: if you want to get every story the minute it comes out it will not have any votes or comments and you'll have to come back later if you want to get representative numbers. I think discussions close in 2 weeks so a double scan is good enough for some purposes but not the up-to-the-minute analysis they claim to be doing) The rest of the analysis is all straightforward except for the topic classification: If I was running on all cylinders for nights and weekend hacking I'd expect to spend a week or so on gathering training data for that (maybe suck down category assignments from Tildes) and probably a week coding up a classifier w/ a BERT-family embedding and classifiers from scikit-learn. You could attempt the same with prompt engineering in a day or two of work and get something expensive to run working pretty quickly that will probably not perform as well as my classifier. (Their classifier gives the empty string as a category which is a bad sign) If you want to improve on it you need the same training set I do.
- behindai 2y ago
  You have enough interesting points to do your project. Would love to see the results! Cheers!

PaulHoule 2y ago

A lot of it is wrong too. It gets my post total for the last 24 hours low by a factor of 5 or so. If it took a snapshot over the weekend it might have been that low because my submission queue bottomed out but if it is live it is wrong. The list of posts by category has the empty string as a category. Any kind of votes/submission, comments/submission, or similar ratio is going to be driven by luck in a 24 hour period. They should fix the bugs and try it again.