3 ms·
Interesting breakdown of the dataset. Are you guys manually assigning labels to the clusters after the fact or is this using some kind of LLM to create a cluste
by calebheinzman 2y ago
Interesting breakdown of the dataset. Are you guys manually assigning labels to the clusters after the fact or is this using some kind of LLM to create a cluster name?
- josh-sematic 2y agoEverything is automatic and non-manually curated; we basically just uploaded the data without doing anything dataset-specific as far as the clusters are concerned. First we create an embedding of the rows, then we run projection & clustering on it. The first clusters we generate are narrow. After we generate the first round of clusters we label them with an LLM, and then cluster those to create the more generalized clusters. Have an LLM label those, and then we're done.