3 ms·
Sould embeddings similarity be used to create a global copyright system?
To me it seems like similarity embeddings would offer a way to compare 2 pieces of content of the same kind(not sure multi-model would work)
I don't know if this would work just kind of brainstorming.
- PaulHoule 4y agoI am impressed. Years ago i was not happy with the results i got clustering documents, recently i built something that used LDA that was not half-bad at making ‘big’ clusters (say sports or reinforcement learning) but it took me half an hour to build a really good clustering system based on the embeddings from sbert.net and KMeans from scikit-learn. DBSCAN, also in scikit-learn only clusters emeddings that are close together so it does a good job of dealing with the problem that one news event results in several articles. I have not yet incorporated it into my RSS reader but i plan to. So really the water is fine and i am amazed at how easy this stuff is compared to five years ago.
- bediger4000 4y agoI think this is a good idea. If used universally, we could accurately economically incentivize content creators, or their successors in interest, so they'll create more copyrighted content. We all win.
- sharemywin 4y agoI asked ChatGPT about using an embedding vector in a DHT: Yes, an embedding vector could be used in a Distributed Hash Table (DHT), depending on the specific application and requirements. In a DHT, nodes store and retrieve data based on keys, which are typically strings or numerical values. However, if the data being stored or retrieved is represented as a high-dimensional vector (such as an embedding vector used in natural language processing or computer vision), the key can be derived from the vector itself using a hash function. For example, in a text search engine application, document embeddings could be stored in a DHT where each node stores a subset of the vectors. When a query is submitted, the query embedding can be hashed to identify the set of nodes that are likely to contain the most relevant documents. The nodes can then return the document embeddings stored within their subset, which can be used to rank and retrieve the most relevant documents. The specific hash function used to derive keys from embeddings will depend on the requirements of the application, including factors such as the size of the embedding vectors, the number of nodes in the network, and the desired balance between load distribution and query efficiency.
- sharemywin 4y agoAlthough I was thinking of the embeddings as the keys.
- deleted 4y ago[deleted]