4 ms·

Seems like it would lock in a user to cohere proprietary model in order to embed new search terms to the same space? Would love to see similar but embedded wit

• minimaxir 4y ago
  Unfortunately yes, as the corresponding model is not also open-source.
• jayalammar 4y ago
  There's a lot you can do with the vectors themselves without needing to embed any more text (e.g., clustering, exploration, visualization after dimensionality reduction...etc). Here's a previous embeddings exploration of top HN posts: https://txt.cohere.com/combing-for-insight-in-10-000-hacker-news-posts-with-text-clustering/ https://txt.cohere.com/combing-for-insight-in-10-000-hacker-... A lot of that code can be used here as well. If you want to query for a search term, you can use a trial API key which is free to use for prototyping. The embedding model itself is not open source, though. [co-author of the post here]
  • minimaxir 4y ago
    If that's the intent, IMO the release dataset should have more metadata (e.g. paragraph heading, article taxonomy)
    • jayalammar 4y ago
      How would you add that data? As new columns you mean? Or add the paragraph headings to the text of the paragraphs before embedding them?
      • minimaxir 4y ago
        New columns. For the headings, I mean the Wikipedia section headings (which isn't always a paragraphs, my mistake). In both cases the data can be used like to classify/visualize Show HNs in your linked post.