3 ms·

For some users I searched, the most similar user is a throwaway account, which is somewhat eye-opening and unnerving.

by magnio 5y ago

notRobot 5y ago
I was just about to point that out! I tried a friend's account and the second match was a throwaway that I know for a fact is theirs.
- gk1 5y ago
  That’s a really interesting and unintended use case…
  - sohei 5y ago
    How did you gather the comment histories? Would you mind sharing a copy?
    - gk1 5y ago
      See description at the bottom. We used the Hacker News API to pull data into BigQuery. From there we ran them through an embedding model and indexed the embeddings in Pinecone. The actual similarity search is done with Pinecone. (https://www.pinecone.io https://www.pinecone.io)
    - busymom0 5y ago
      Using Google BigQuery is one way. This comment might be of use: https://news.ycombinator.com/item?id=25075318 https://news.ycombinator.com/item?id=25075318 > A reminder that BigQuery (as used in the query in this link) is the best way to play with Hacker News data; don't scrape HN data manually! The `bigquery-public-data.hacker_news.full` table appears to be up to date with the most recent HN data as well (table last updated today). However, I'm not 100% sure the query is correct for unilaterally getting all links, as running the query on the full dataset returns the same results as running it from 2006-2015. And I value my sanity enough to not fuss around with the regex.
- JimDabell 5y ago
  Two of my matches were <username> and <username>2.
Darvokis 5y ago
Huh, checked some of my older accounts and none of them matched each other. So I must be doing something right.
briefcomment 5y ago
Wow, this works seriously good at some level then.
aasasd 5y ago
In fact, considering that unknown third-parties freely gather such similarity scores and correlate accounts, across different sites—by now it's a given that one's alt accounts have to adhere to different stylistic choices. Innit?

For some users I searched, the most similar user is a throwaway account, which is somewhat eye-opening and unnerving.

by magnio 5y ago

For some users I searched, the most similar user is a throwaway account, which is somewhat eye-opening and unnerving.

I was just about to point that out! I tried a friend's account and the second match was a throwaway that I know for a fact is theirs.

gk1 5y ago
That’s a really interesting and unintended use case…
- sohei 5y ago
  How did you gather the comment histories? Would you mind sharing a copy?
  - gk1 5y ago
    See description at the bottom. We used the Hacker News API to pull data into BigQuery. From there we ran them through an embedding model and indexed the embeddings in Pinecone. The actual similarity search is done with Pinecone. (https://www.pinecone.io https://www.pinecone.io)
  - busymom0 5y ago
    Using Google BigQuery is one way. This comment might be of use: https://news.ycombinator.com/item?id=25075318 https://news.ycombinator.com/item?id=25075318 > A reminder that BigQuery (as used in the query in this link) is the best way to play with Hacker News data; don't scrape HN data manually! The `bigquery-public-data.hacker_news.full` table appears to be up to date with the most recent HN data as well (table last updated today). However, I'm not 100% sure the query is correct for unilaterally getting all links, as running the query on the full dataset returns the same results as running it from 2006-2015. And I value my sanity enough to not fuss around with the regex.
JimDabell 5y ago
Two of my matches were <username> and <username>2.

Darvokis 5y ago

Huh, checked some of my older accounts and none of them matched each other. So I must be doing something right.

briefcomment 5y ago

Wow, this works seriously good at some level then.

aasasd 5y ago

In fact, considering that unknown third-parties freely gather such similarity scores and correlate accounts, across different sites—by now it's a given that one's alt accounts have to adhere to different stylistic choices. Innit?