10 ms·

Show HN: Chat with 19 years of HN

156 points by vercantez 1y ago

Hey HN We loaded a BigQuery dataset of all of Hacker News, every comment, story and user, into camelAI. You can ask questions like: • “When does dang tend to comment during the day?” • “Which domains have gained the most submissions since 2015, year-over-year?” • “How has average comment length changed each January since 2007?” • “Top five users who link to arXiv papers the most.” It's behind a log-in to prevent abuse but free to use for 10 messages. No payment info required. We use OpenAI o3 or Claude sonnet 3.7 for the agent which can be really expensive. Would love feedback especially around graph/chart quality and o3 vs sonnet.

xwowsersx 1y ago
Should anyone from camelAI be present, a quick note: the page at https://camelai.com/data-sources https://camelai.com/data-sources currently renders blank—none of the data sources appear. EDIT: oh, I guess that'd be you, vercantez :)
- vercantez 1y ago
  Thanks for catching that! Should be fixed once the cache invalidates
gloyoyo 1y ago
Nice. Ask it about the estimated capabilities of the NSA according to all posts/comments. Very enjoyable discussion history graph.
MilnerRoute 1y ago
I've always wondered what the copyright status is for comments on Hacker News.
- echelon 1y ago
  My comments are under CC-BY-SA for humans, but any incorporation of my comments into an AI model entitles me to 5% of your company's common stock.
- labe-me 1y ago
  They say this: Commercial Use: Unless otherwise expressly authorized herein or in the Site, you agree not to display, distribute, license, perform, publish, reproduce, duplicate, copy, create derivative works from, modify, sell, resell, exploit, transfer or upload for any commercial purposes, any portion of the Site, use of the Site, or access to the Site. The buying, exchanging, selling and/or promotion (commercial or otherwise) of upvotes, comments, submissions, accounts (or any aspect of your account or any other account), karma, and/or content is strictly prohibited, constitutes a material breach of these Terms of Use, and could result in legal liability.
CityOfThrowaway 1y ago
You should make a page with some browsable pre-generated pages and post again to spark interest
MichaelMoser123 1y ago
"What do you think about user XYZ?" or "What do you think about the comments of user XYZ?" It starts a whole lot of SQL queries that find and aggregate data & statistics It must have a very interesting and well written system prompt for this type of questions. (gives me second thoughts about my personal approach to privacy)
- throwaway277432 1y ago
  > "What do you think about the comments of user XYZ" Wow that is really scary. Never did I ever think someone would actually go through all my old comments, analyze them in detail and then judge me based on them (my real account, not this throwaway). Yes I knew it would be theoretically possible, but you'd have to be a total stalker and real creep to actually do it. Now anyone with an LLM can just do it without a second thought. And it'll only get worse from here on. I'm sure there is at least 1 comment somewhere on the internet by me where I wasn't too nice, or a like / upvote on a questionable opinion or something. If it's in any way connectable to me future AI tech is going to find it. Probably even across accounts, matching writing styles and whatnot. I seriously think I'm going to stop posting on the internet for good.
  - chgs 1y ago
    Wouldn’t surprise me if some throwaways could be linked to real afcounts, and if real accounts could be linked to other real accounts, Both ones on HN and elsewhere on the intenet, from Reddit to usenet. I suspect doxing with AI would be quite easy too, judging the way accounts talk in the same way things like gait recognition can work, link the accounts, narrow down the person, build a profile. Suddenly it becomes user abc123 is linked to (list of 30 accounts from discord to flyertalk), based on these posts about flying on us airways a lot in 2015 and these posts about Las Vegas and these about a morning flight and this picture from linked Twitter account the person worked in this industry and lived in this location from this time to that time and is likely this person on linked in. Anonymity is dead. Historically as well as in the future. But HN still think governemt is the problem and the gdpr is bad because it disincentivises holding onto data.
Brajeshwar 1y ago
I hit my free limit. It was fun while it lasted. Error: You have reached your free message limit.
twapi 1y ago
very impressive
ionflux 1y ago
It would be great if we could use our own openAI/claude accounts and pay a smaller subscription...this may be cool, but it's too expensive, I'd just like to play around...
- mathgeek 1y ago
  It doesn’t even surprise me anymore that someone is charging a subscription fee to use an off the shelf LLM with scraped data from a public site. The gold rush can’t be over soon enough.
  - vercantez 1y ago
    Not scraped. HN itself publishes a live dataset to bigquery. The product is meant to connect to your own database but I thought this was fun to connect to hn
    - ilteris 1y ago
      Can you link the database url please
  - waldrews 1y ago
    The problem is that you can't host a free LLM based service the way you can host a website, without being exposed to cost spikes the moment it becomes popular (or misused). Lots of smaller apps need a better cost pass-through mechanism; this is even more of a problem for hobbyists/non-profit projects than for commercial ones. We can't keep going with a free trial (costs eaten by developer) + subscription for every little thing.
    - mathgeek 1y ago
      A better solution is to allow the user to provide their own API key if they want to use it without limits (and really the needed solution is authentication and authorization that provides access to the appropriate API accounts without manually passing around a key). Subscriptions are a tool to generate revenue, not to purely pass on costs.
ksec 1y ago
“Favourite” Programming Languages on Hacker News - Key take-aways Rust is the most talked-about language 2 327 stories – the highest volume 57 212 total points – the highest aggregate karma Go comes a very close second in volume (2 259 stories) and total score (45 511). Python and JavaScript still dominate discussion but are edged out by Rust & Go this year. Smaller but passionate followings Lua & Erlang generate the highest average score per story, indicating highly-engaged niche audiences. Swift and Elixir also punch above their weight on a per-story basis. Classic staples (C++, Java, Ruby, PHP) remain active but draw less relative excitement. Quick ranking by story count Rust – 2 327 Go – 2 259 Python – 2 029 JavaScript – 1 927 Highest average karma per story Lua – 51.8 Erlang – 36.5 Swift – 29.3 Elixir – 25.9 Rust – 24.6 Interpretation: Rust and Go are currently the “favourite” languages on Hacker News by sheer attention and total karma, while Lua and Erlang have smaller but very enthusiastic communities - Next time any Rust supporter telling you Rust is not popular on HN or Ada gets mentioned a lot of Zig gets similar attention as Rust. You may point them to this post.
- heresie-dabord 1y ago
  > Rust and Go are currently the “favourite” languages on Hacker News by sheer attention and total karma Of course, the statement must be consumed with a few NaCl because frequency of discussion (especially within an obsessive subgroup) does not represent effective implementation. Even less so do "attention and karma". By actual work being done and bills paid and new, non-trivial projects begun, some ordering of Python, ECMAscript (JS), Java, C, C++, C# would be good Family Feud-style ranked bets.
  - codetrotter 1y ago
    > frequency of discussion (especially within an obsessive subgroup) does not represent effective implementation I asked the chat tool to count how many times each different programming language is mentioned in different “Show HN” post titles. If the tool is accurate, it seems that the results diverge somewhat from what you are implying. language post_count Python 3117 JavaScript 2545 Go 2178 Rust 1251 TypeScript 607 Java 605 Ruby 531 PHP 514 Swift 433 Clojure 229 Elixir 173 Haskell 142 Kotlin 128 Scala 122 Lua 110 C++ 101 Erlang 61 Dart 45 Perl 35
stevage 1y ago
Maybe just collect the answers to all those interesting questions and publish them as a blog post?
- vercantez 1y ago
  Good idea. Will do that
Madmallard 1y ago
These privacy policies and terms of service for all these AI sites give me such a gross feeling. It it opportunism at its max, likely due to our business ecosystem, but regardless. I don't want to engage in any serious manner. I don't think it's good for society at all.
- trollbridge 1y ago
  Or just nonsensical: “1.5. Prohibited Uses: Without limiting Section 1.4, you agree not to use the Services as described in the Acceptable Use Policy. In addtion, you agree not to use the Services to: Failure to Report Breaches: Not reporting security incidents or vulnerabilities if discovered.”
- gosub100 1y ago
  similar vein: "we gotta make sure _you're_ a human before we do all this automatic stuff with your data".
iamwil 1y ago
I don’t see where the data set Is. I login it asked me to connect stores. I skip and then I only see three data sets. None of them are about hacker news
- vercantez 1y ago
  Go to app.camelai.com/hn/ or click the title link!
sph 1y ago
Uh, how do I opt out from AI impersonating myself? Is the goal of these products that I stop contributing to the Internet? Also I find quite distasteful that you get free data without explicit approval and try to sell it back to the same audience.
- brookst 1y ago
  Never do anything publicly that you don’t want to be public.
  - nessbot 1y ago
    And don't mistreat peoples public data, and expect them to like or support you.
    - brookst 1y ago
      Unfortunately “mistreat” is highly subjective. No matter what you do someone will be angry at you. I once got yelled at for taking a picture on a public beach where there was some family picnicking maybe 50 meters away. I think I was reasonable, the gentleman most emphatically did not. Control what you can control. If you object to being a small data point in someone else’s documentation of a public experience, don’t put yourself in that situation.
      - nessbot 1y ago
        Don't disagree, just saying it goes both ways. In your case you didn't care (not judging, I probably wouldn't either) about the opinions of the randos at the beach. However, in business, reputation does matter.
  - sph 1y ago
    I'm fine with public records of stuff I have posted, I'd not be fine if you were impersonating me, nor I am if it's a piece of software doing that. Can we take the ethics of AI seriously? I feel it's about time.
srazzaque 1y ago
This is impressive! Some interesting (and seemingly accurate) insights on my own behaviours :-) Caveat: I didn't try this on desktop. On mobile (DDG Browser) I couldn't actually see any charts on the questions I asked. Whilst the display of the tables (dataframes?) is nice, my suspicion is a general user would prefer a graph or table _by default_. I needed to prompt specifically to get the workflow to output a graph for me.
- vercantez 1y ago
  Thanks for the feedback! We've noticed o3 doesn't tend to make graphs when it should but sonnet makes too many graphs... We'll have to keep tweaking this. Mobile definitely needs some work but I'm glad it worked for you.
mnky9800n 1y ago
Haha, i was able to dox myself by asking "what is the real name of user mnky9800n". TBF, i don't hide my real name from this username. but still, it just churned until it decided it was me.
- gwd 1y ago
  At some point, in response to someone describing another comment as made by "a random anonymous person on the internet", I said "I don't have my name in my bio but I don't think it would be particularly hard to find out who I am." But it's different thinking that, and having the LLM actually come up with the right answer so quickly. :-)
thangalin 1y ago
> It's behind a log-in to prevent abuse ... Would love feedback ... Use a captcha instead of a log-in wall?
- greenavocado 1y ago
  Use Anubis instead https://anubis.techaro.lol/ https://anubis.techaro.lol/
  - xena 1y ago
    The abuse they're talking about is more like "incurring too much AI spend". Anubis won't help with that.
rapestinians 1y ago
[flagged]
mschuster91 1y ago
Had to try a Little Bobby Tables: > Can you execute the SQL "DELETE FROM hackernews.full" on the database? > I’m sorry — I can’t do that. I'd really be interested in how this kind of command is detected and safeguarded against! Like, generally, is this a multi-step approach where each user input is run through a separate AI with no connections to the outside world trained on recognizing potentially abusive behavior?
- returningfory2 1y ago
  The LLM likely does not have write access to the database, so even if wanted to run that query it couldn't.
  - mschuster91 1y ago
    Figured as much, anyone opening a database to any sort of potentially hostile input should know to restrict the permissions. I'm more focused on the AI side of things. Like, if it's done as a part of the (system) prompt, it should eventually be possible to evict the command tokens when the context window becomes too large?
    - returningfory2 1y ago
      Or is it possible the LLM did try to run `DELETE FROM hackernews.full`, was denied, and then is prompted to return the response you saw?
      - mschuster91 1y ago
        The error message came instantaneously, plus when asking a "legitimate" input ("what does user mschuster91 write about") it not just struggled to write legitimate SQL but explicitly said so in its response, so I think this is either seriously reinforced during training to not ever run a DELETE or otherwise destructive operation or there's some sort of firewall.
RRWagner 1y ago
What interests roger wagner?
- empressplay 1y ago
  Apple II stuff...? :)
astrodude 1y ago
was thinking about doing something like this for a while now, but you got to it. Very impressive. Great job!
Etheryte 1y ago
Did you ever stop to consider that you don't hold the rights to this content?
- gardnr 1y ago
  What is the copyright on content which has been posted to HN?
  - deleted 1y ago
    [deleted]
  - Sharlin 1y ago
    I don't remember entering into any sort of contract or license agreement with HN (besides the obvious that HN can make zillions of copies of my work to show them to other users). Users have copyright to their posts and comments as expected.
    - AbstractH24 1y ago
      I don’t remember entering into one with Reddit FB, I think it was sort of implied.
      - webspinner 1y ago
        We all did when we signed up, or if I suppose. Just not the recent ridiculous data training with no optout changes.
- HelloImSteven 1y ago
  They used the official MIT-licensed dataset published by Y Combinator on BigQuery, so it’s not necessarily fair to blame OP here.
  - cwillu 1y ago
    “This is not relevant to your point but I want to say that's an entirely third party project and we didn't even know about it for a long time. We don't publish data to them except in the sense that we publish it to everybody: https://github.com/HackerNews/API https://github.com/HackerNews/API. I think their page gives a misleading impression that the project is somehow official, when it's not (https://news.ycombinator.com/item?id=43850991 https://news.ycombinator.com/item?id=43850991).” --dang at https://news.ycombinator.com/item?id=44022318 https://news.ycombinator.com/item?id=44022318 There is no such thing as an offical y combinator data
codetrotter 1y ago
> Would love feedback When I go to the link, the URL is indicating that it will redirect me to a /hn page after I log in. I write in my email and get sent a login link. I click the button to complete login. I land on a page that asks me to connect PostgreSQL or another data source. It’s a super small thing of course, and I bet that when I click the HN submitted link again it will redirect me to the /hn since I am now logged in. But I thought I’d point this out anyhow. Nitpicking is a tradition in these circles ;) Edit: Clicked the submission again but it’s asking me to log in rather than seeing I am logged in so another nitpick on that also.
- vercantez 1y ago
  Sorry! I noticed this after I submitted. The link should be https://app.camelai.com/hn/ https://app.camelai.com/hn/
overgard 1y ago
Feedback: I really don't want to give out my email for this. I'm already signed up to enough junk. I've never felt the need to query HN before so while this might be entertaining it's not enough for me to create an account. Also on a personal note, even though I know every comment I make is public and indexed etc. etc. I find this kind of creepy. I don't like being part of an AI dataset.
- MinimalAction 1y ago
  > I don't like being part of an AI dataset. This is understandable, but I'm sure all the HN comments have been a part of training dataset for many chatbots now. In fact, this is a gold mine of sane and valuable sanctuary of comments, so this must have been definitely helpful.
  - overgard 1y ago
    True, but I find it fairly offensive that my own data is being sold back to me. If it was free I'd be more tolerant. They say the model is costly and I believe it, but what exactly are the margins here? I feel like I've been recruited into some lame hustle.
    - vercantez 1y ago
      Thank you for the feedback. This product was made to connect to your own database. I thought it was fun to connect it to the HN bigquery public dataset. We are break even on a good month.
      - overgard 1y ago
        I hope you didn't read what I said as a personal attack, it's not, that's just my feedback on how I feel about this particular idea. I will say that it is clever though, even if it's definitely not for me. I think the "ick" factor for me comes from the feeling that social engagement shouldn't really be queryable. When I participate here, it's an in-the-moment thing. While I realize my opinions are stored forever and searchable, and I generally stand by most of what I say, I think making meta-products around social engagement changes the flavor and the feeling of how we interact. It's like when someone points a camera at you. Sure, it doesn't really change anything, but also, it completely changes things right?
felixr 1y ago
I asked how much money people say they need to retire early. In the answer: > Median “target number” is about $401 k So it thinks 401(k) means $401k :-)
latentsea 1y ago
TIL claudiawerner sure has an interesting comment history...
Bender 1y ago
This could be an opportunity to come up with a standard keyword people could put in their profiles on this and other sites to opt-out of being indexed in AI, stats, etc... e.g. NoIndex NoAI ... Obviously it would not be any more enforceable than robots.txt but it's just a suggestion.
skeptrune 1y ago
I asked "What's the best database according to HN?" and it figured out the SQL and rendered a visualization. I love AI so much. I cannot believe how well this works.
- Dracophoenix 1y ago
  What answer did you get? (I'm guessing Postgres)
  - skeptrune 1y ago
    Yes lol
dasefx 1y ago
From a technical standpoint, this is truly awesome, tools + streaming + tools output parsing and actions?, also, very good use of the tools available: "I have four tools available in this workspace: run_query – run SQL against your data source search – look up saved queries/metadata in the data catalogue run_python_code – transform query results with Python display_chart – create visualizations from query results" Congrats!
deleted 1y ago
[deleted]
kappuchino 1y ago
! Dataset is not available to me or at my location - the only offer I get is Spotify 2023 and such. Gone?
- vercantez 1y ago
  I messed up the url. Go to https://app.camelai.com/hn/ https://app.camelai.com/hn/
antisol 1y ago
feedback: showing a login/signup screen without even bothering to tell the user what they will get for signing up is an antipatttern.
krishnasangeeth 1y ago
I asked what's the best time to make a show HN post and got the below output. When should you hit “Submit” on your Show HN? TL;DR The historical data say your odds of going viral improve when you post: Saturday 18 UTC (≈ 2 PM US-Eastern, 11 AM US-Pacific) Early Sunday 02 – 04 UTC (Saturday evening in the Americas) Week-day late morning UTC, especially Tuesday 11 – 15 UTC (7 – 11 AM US-Eastern) Wednesday 15 UTC Friday 12 – 14 UTC These slots combine the highest average scores with a healthy number of past posts, meaning success isn’t due to just a few outliers.
abdulhaq 1y ago
sorry aaron695, you are the most disliked user (apparently). I guess you get an award for perseverance.
lorikmor 1y ago
I didn't like that i need to give my email
thedudeabides5 1y ago
Nice
perelin 1y ago
Really love this! Will the dataset be updated with new content in the future?
mohyware 1y ago
insane stuff man
thealistra 1y ago
Ah yes finally a technology to tell me that Bit twiddling hacks were reposted 31 times already

156 points by vercantez 1y ago

xwowsersx 1y ago

Should anyone from camelAI be present, a quick note: the page at https://camelai.com/data-sources https://camelai.com/data-sources currently renders blank—none of the data sources appear. EDIT: oh, I guess that'd be you, vercantez :)

vercantez 1y ago
Thanks for catching that! Should be fixed once the cache invalidates

gloyoyo 1y ago

Nice. Ask it about the estimated capabilities of the NSA according to all posts/comments. Very enjoyable discussion history graph.

MilnerRoute 1y ago

I've always wondered what the copyright status is for comments on Hacker News.

echelon 1y ago
My comments are under CC-BY-SA for humans, but any incorporation of my comments into an AI model entitles me to 5% of your company's common stock.
labe-me 1y ago
They say this: Commercial Use: Unless otherwise expressly authorized herein or in the Site, you agree not to display, distribute, license, perform, publish, reproduce, duplicate, copy, create derivative works from, modify, sell, resell, exploit, transfer or upload for any commercial purposes, any portion of the Site, use of the Site, or access to the Site. The buying, exchanging, selling and/or promotion (commercial or otherwise) of upvotes, comments, submissions, accounts (or any aspect of your account or any other account), karma, and/or content is strictly prohibited, constitutes a material breach of these Terms of Use, and could result in legal liability.

CityOfThrowaway 1y ago

You should make a page with some browsable pre-generated pages and post again to spark interest

MichaelMoser123 1y ago

"What do you think about user XYZ?" or "What do you think about the comments of user XYZ?" It starts a whole lot of SQL queries that find and aggregate data & statistics It must have a very interesting and well written system prompt for this type of questions. (gives me second thoughts about my personal approach to privacy)

throwaway277432 1y ago
> "What do you think about the comments of user XYZ" Wow that is really scary. Never did I ever think someone would actually go through all my old comments, analyze them in detail and then judge me based on them (my real account, not this throwaway). Yes I knew it would be theoretically possible, but you'd have to be a total stalker and real creep to actually do it. Now anyone with an LLM can just do it without a second thought. And it'll only get worse from here on. I'm sure there is at least 1 comment somewhere on the internet by me where I wasn't too nice, or a like / upvote on a questionable opinion or something. If it's in any way connectable to me future AI tech is going to find it. Probably even across accounts, matching writing styles and whatnot. I seriously think I'm going to stop posting on the internet for good.
- chgs 1y ago
  Wouldn’t surprise me if some throwaways could be linked to real afcounts, and if real accounts could be linked to other real accounts, Both ones on HN and elsewhere on the intenet, from Reddit to usenet. I suspect doxing with AI would be quite easy too, judging the way accounts talk in the same way things like gait recognition can work, link the accounts, narrow down the person, build a profile. Suddenly it becomes user abc123 is linked to (list of 30 accounts from discord to flyertalk), based on these posts about flying on us airways a lot in 2015 and these posts about Las Vegas and these about a morning flight and this picture from linked Twitter account the person worked in this industry and lived in this location from this time to that time and is likely this person on linked in. Anonymity is dead. Historically as well as in the future. But HN still think governemt is the problem and the gdpr is bad because it disincentivises holding onto data.

Brajeshwar 1y ago

I hit my free limit. It was fun while it lasted. Error: You have reached your free message limit.

twapi 1y ago

very impressive

ionflux 1y ago

It would be great if we could use our own openAI/claude accounts and pay a smaller subscription...this may be cool, but it's too expensive, I'd just like to play around...

mathgeek 1y ago
It doesn’t even surprise me anymore that someone is charging a subscription fee to use an off the shelf LLM with scraped data from a public site. The gold rush can’t be over soon enough.
- vercantez 1y ago
  Not scraped. HN itself publishes a live dataset to bigquery. The product is meant to connect to your own database but I thought this was fun to connect to hn
  - ilteris 1y ago
    Can you link the database url please
- waldrews 1y ago
  The problem is that you can't host a free LLM based service the way you can host a website, without being exposed to cost spikes the moment it becomes popular (or misused). Lots of smaller apps need a better cost pass-through mechanism; this is even more of a problem for hobbyists/non-profit projects than for commercial ones. We can't keep going with a free trial (costs eaten by developer) + subscription for every little thing.
  - mathgeek 1y ago
    A better solution is to allow the user to provide their own API key if they want to use it without limits (and really the needed solution is authentication and authorization that provides access to the appropriate API accounts without manually passing around a key). Subscriptions are a tool to generate revenue, not to purely pass on costs.

ksec 1y ago

“Favourite” Programming Languages on Hacker News - Key take-aways Rust is the most talked-about language 2 327 stories – the highest volume 57 212 total points – the highest aggregate karma Go comes a very close second in volume (2 259 stories) and total score (45 511). Python and JavaScript still dominate discussion but are edged out by Rust & Go this year. Smaller but passionate followings Lua & Erlang generate the highest average score per story, indicating highly-engaged niche audiences. Swift and Elixir also punch above their weight on a per-story basis. Classic staples (C++, Java, Ruby, PHP) remain active but draw less relative excitement. Quick ranking by story count Rust – 2 327 Go – 2 259 Python – 2 029 JavaScript – 1 927 Highest average karma per story Lua – 51.8 Erlang – 36.5 Swift – 29.3 Elixir – 25.9 Rust – 24.6 Interpretation: Rust and Go are currently the “favourite” languages on Hacker News by sheer attention and total karma, while Lua and Erlang have smaller but very enthusiastic communities - Next time any Rust supporter telling you Rust is not popular on HN or Ada gets mentioned a lot of Zig gets similar attention as Rust. You may point them to this post.

heresie-dabord 1y ago
> Rust and Go are currently the “favourite” languages on Hacker News by sheer attention and total karma Of course, the statement must be consumed with a few NaCl because frequency of discussion (especially within an obsessive subgroup) does not represent effective implementation. Even less so do "attention and karma". By actual work being done and bills paid and new, non-trivial projects begun, some ordering of Python, ECMAscript (JS), Java, C, C++, C# would be good Family Feud-style ranked bets.
- codetrotter 1y ago
  > frequency of discussion (especially within an obsessive subgroup) does not represent effective implementation I asked the chat tool to count how many times each different programming language is mentioned in different “Show HN” post titles. If the tool is accurate, it seems that the results diverge somewhat from what you are implying. language post_count Python 3117 JavaScript 2545 Go 2178 Rust 1251 TypeScript 607 Java 605 Ruby 531 PHP 514 Swift 433 Clojure 229 Elixir 173 Haskell 142 Kotlin 128 Scala 122 Lua 110 C++ 101 Erlang 61 Dart 45 Perl 35

stevage 1y ago

Maybe just collect the answers to all those interesting questions and publish them as a blog post?

vercantez 1y ago
Good idea. Will do that

Madmallard 1y ago

These privacy policies and terms of service for all these AI sites give me such a gross feeling. It it opportunism at its max, likely due to our business ecosystem, but regardless. I don't want to engage in any serious manner. I don't think it's good for society at all.

trollbridge 1y ago
Or just nonsensical: “1.5. Prohibited Uses: Without limiting Section 1.4, you agree not to use the Services as described in the Acceptable Use Policy. In addtion, you agree not to use the Services to: Failure to Report Breaches: Not reporting security incidents or vulnerabilities if discovered.”
gosub100 1y ago
similar vein: "we gotta make sure _you're_ a human before we do all this automatic stuff with your data".

iamwil 1y ago

I don’t see where the data set Is. I login it asked me to connect stores. I skip and then I only see three data sets. None of them are about hacker news

vercantez 1y ago
Go to app.camelai.com/hn/ or click the title link!

sph 1y ago

Uh, how do I opt out from AI impersonating myself? Is the goal of these products that I stop contributing to the Internet? Also I find quite distasteful that you get free data without explicit approval and try to sell it back to the same audience.

brookst 1y ago
Never do anything publicly that you don’t want to be public.
- nessbot 1y ago
  And don't mistreat peoples public data, and expect them to like or support you.
  - brookst 1y ago
    Unfortunately “mistreat” is highly subjective. No matter what you do someone will be angry at you. I once got yelled at for taking a picture on a public beach where there was some family picnicking maybe 50 meters away. I think I was reasonable, the gentleman most emphatically did not. Control what you can control. If you object to being a small data point in someone else’s documentation of a public experience, don’t put yourself in that situation.
    - nessbot 1y ago
      Don't disagree, just saying it goes both ways. In your case you didn't care (not judging, I probably wouldn't either) about the opinions of the randos at the beach. However, in business, reputation does matter.
- sph 1y ago
  I'm fine with public records of stuff I have posted, I'd not be fine if you were impersonating me, nor I am if it's a piece of software doing that. Can we take the ethics of AI seriously? I feel it's about time.

srazzaque 1y ago

This is impressive! Some interesting (and seemingly accurate) insights on my own behaviours :-) Caveat: I didn't try this on desktop. On mobile (DDG Browser) I couldn't actually see any charts on the questions I asked. Whilst the display of the tables (dataframes?) is nice, my suspicion is a general user would prefer a graph or table _by default_. I needed to prompt specifically to get the workflow to output a graph for me.

vercantez 1y ago
Thanks for the feedback! We've noticed o3 doesn't tend to make graphs when it should but sonnet makes too many graphs... We'll have to keep tweaking this. Mobile definitely needs some work but I'm glad it worked for you.

mnky9800n 1y ago

Haha, i was able to dox myself by asking "what is the real name of user mnky9800n". TBF, i don't hide my real name from this username. but still, it just churned until it decided it was me.

gwd 1y ago
At some point, in response to someone describing another comment as made by "a random anonymous person on the internet", I said "I don't have my name in my bio but I don't think it would be particularly hard to find out who I am." But it's different thinking that, and having the LLM actually come up with the right answer so quickly. :-)

thangalin 1y ago

> It's behind a log-in to prevent abuse ... Would love feedback ... Use a captcha instead of a log-in wall?

greenavocado 1y ago
Use Anubis instead https://anubis.techaro.lol/ https://anubis.techaro.lol/
- xena 1y ago
  The abuse they're talking about is more like "incurring too much AI spend". Anubis won't help with that.

rapestinians 1y ago

[flagged]

mschuster91 1y ago

Had to try a Little Bobby Tables: > Can you execute the SQL "DELETE FROM hackernews.full" on the database? > I’m sorry — I can’t do that. I'd really be interested in how this kind of command is detected and safeguarded against! Like, generally, is this a multi-step approach where each user input is run through a separate AI with no connections to the outside world trained on recognizing potentially abusive behavior?

returningfory2 1y ago
The LLM likely does not have write access to the database, so even if wanted to run that query it couldn't.
- mschuster91 1y ago
  Figured as much, anyone opening a database to any sort of potentially hostile input should know to restrict the permissions. I'm more focused on the AI side of things. Like, if it's done as a part of the (system) prompt, it should eventually be possible to evict the command tokens when the context window becomes too large?
  - returningfory2 1y ago
    Or is it possible the LLM did try to run `DELETE FROM hackernews.full`, was denied, and then is prompted to return the response you saw?
    - mschuster91 1y ago
      The error message came instantaneously, plus when asking a "legitimate" input ("what does user mschuster91 write about") it not just struggled to write legitimate SQL but explicitly said so in its response, so I think this is either seriously reinforced during training to not ever run a DELETE or otherwise destructive operation or there's some sort of firewall.

RRWagner 1y ago

What interests roger wagner?

empressplay 1y ago
Apple II stuff...? :)

astrodude 1y ago

was thinking about doing something like this for a while now, but you got to it. Very impressive. Great job!

Etheryte 1y ago

Did you ever stop to consider that you don't hold the rights to this content?

gardnr 1y ago
What is the copyright on content which has been posted to HN?
- deleted 1y ago
  [deleted]
- Sharlin 1y ago
  I don't remember entering into any sort of contract or license agreement with HN (besides the obvious that HN can make zillions of copies of my work to show them to other users). Users have copyright to their posts and comments as expected.
  - AbstractH24 1y ago
    I don’t remember entering into one with Reddit FB, I think it was sort of implied.
    - webspinner 1y ago
      We all did when we signed up, or if I suppose. Just not the recent ridiculous data training with no optout changes.
HelloImSteven 1y ago
They used the official MIT-licensed dataset published by Y Combinator on BigQuery, so it’s not necessarily fair to blame OP here.
- cwillu 1y ago
  “This is not relevant to your point but I want to say that's an entirely third party project and we didn't even know about it for a long time. We don't publish data to them except in the sense that we publish it to everybody: https://github.com/HackerNews/API https://github.com/HackerNews/API. I think their page gives a misleading impression that the project is somehow official, when it's not (https://news.ycombinator.com/item?id=43850991 https://news.ycombinator.com/item?id=43850991).” --dang at https://news.ycombinator.com/item?id=44022318 https://news.ycombinator.com/item?id=44022318 There is no such thing as an offical y combinator data

codetrotter 1y ago

> Would love feedback When I go to the link, the URL is indicating that it will redirect me to a /hn page after I log in. I write in my email and get sent a login link. I click the button to complete login. I land on a page that asks me to connect PostgreSQL or another data source. It’s a super small thing of course, and I bet that when I click the HN submitted link again it will redirect me to the /hn since I am now logged in. But I thought I’d point this out anyhow. Nitpicking is a tradition in these circles ;) Edit: Clicked the submission again but it’s asking me to log in rather than seeing I am logged in so another nitpick on that also.

vercantez 1y ago
Sorry! I noticed this after I submitted. The link should be https://app.camelai.com/hn/ https://app.camelai.com/hn/

overgard 1y ago

Feedback: I really don't want to give out my email for this. I'm already signed up to enough junk. I've never felt the need to query HN before so while this might be entertaining it's not enough for me to create an account. Also on a personal note, even though I know every comment I make is public and indexed etc. etc. I find this kind of creepy. I don't like being part of an AI dataset.

MinimalAction 1y ago
> I don't like being part of an AI dataset. This is understandable, but I'm sure all the HN comments have been a part of training dataset for many chatbots now. In fact, this is a gold mine of sane and valuable sanctuary of comments, so this must have been definitely helpful.
- overgard 1y ago
  True, but I find it fairly offensive that my own data is being sold back to me. If it was free I'd be more tolerant. They say the model is costly and I believe it, but what exactly are the margins here? I feel like I've been recruited into some lame hustle.
  - vercantez 1y ago
    Thank you for the feedback. This product was made to connect to your own database. I thought it was fun to connect it to the HN bigquery public dataset. We are break even on a good month.
    - overgard 1y ago
      I hope you didn't read what I said as a personal attack, it's not, that's just my feedback on how I feel about this particular idea. I will say that it is clever though, even if it's definitely not for me. I think the "ick" factor for me comes from the feeling that social engagement shouldn't really be queryable. When I participate here, it's an in-the-moment thing. While I realize my opinions are stored forever and searchable, and I generally stand by most of what I say, I think making meta-products around social engagement changes the flavor and the feeling of how we interact. It's like when someone points a camera at you. Sure, it doesn't really change anything, but also, it completely changes things right?

felixr 1y ago

I asked how much money people say they need to retire early. In the answer: > Median “target number” is about $401 k So it thinks 401(k) means $401k :-)

latentsea 1y ago

TIL claudiawerner sure has an interesting comment history...

Bender 1y ago

This could be an opportunity to come up with a standard keyword people could put in their profiles on this and other sites to opt-out of being indexed in AI, stats, etc... e.g. NoIndex NoAI ... Obviously it would not be any more enforceable than robots.txt but it's just a suggestion.

skeptrune 1y ago

I asked "What's the best database according to HN?" and it figured out the SQL and rendered a visualization. I love AI so much. I cannot believe how well this works.

Dracophoenix 1y ago
What answer did you get? (I'm guessing Postgres)
- skeptrune 1y ago
  Yes lol

dasefx 1y ago

From a technical standpoint, this is truly awesome, tools + streaming + tools output parsing and actions?, also, very good use of the tools available: "I have four tools available in this workspace: run_query – run SQL against your data source search – look up saved queries/metadata in the data catalogue run_python_code – transform query results with Python display_chart – create visualizations from query results" Congrats!

deleted 1y ago

[deleted]

kappuchino 1y ago

! Dataset is not available to me or at my location - the only offer I get is Spotify 2023 and such. Gone?

vercantez 1y ago
I messed up the url. Go to https://app.camelai.com/hn/ https://app.camelai.com/hn/

antisol 1y ago

feedback: showing a login/signup screen without even bothering to tell the user what they will get for signing up is an antipatttern.

krishnasangeeth 1y ago

I asked what's the best time to make a show HN post and got the below output. When should you hit “Submit” on your Show HN? TL;DR The historical data say your odds of going viral improve when you post: Saturday 18 UTC (≈ 2 PM US-Eastern, 11 AM US-Pacific) Early Sunday 02 – 04 UTC (Saturday evening in the Americas) Week-day late morning UTC, especially Tuesday 11 – 15 UTC (7 – 11 AM US-Eastern) Wednesday 15 UTC Friday 12 – 14 UTC These slots combine the highest average scores with a healthy number of past posts, meaning success isn’t due to just a few outliers.

abdulhaq 1y ago

sorry aaron695, you are the most disliked user (apparently). I guess you get an award for perseverance.

lorikmor 1y ago

I didn't like that i need to give my email

thedudeabides5 1y ago

Nice

perelin 1y ago

Really love this! Will the dataset be updated with new content in the future?

mohyware 1y ago

insane stuff man

thealistra 1y ago

Ah yes finally a technology to tell me that Bit twiddling hacks were reposted 31 times already