2 ms·
The failure mode I see here, which seems quite likely, is that attribution would almost always list tens of thousands of source files, or more. They doesn’t see
by johnfn 4y ago
The failure mode I see here, which seems quite likely, is that attribution would almost always list tens of thousands of source files, or more. They doesn’t seem particularly useful or meaningful.
- dahart 4y agoYeah, correct, I would expect the same thing to happen if you list all attributions. That’s why I mentioned a criteria for filtering by number of tokens in a row for a given attribution, and one possible method for doing the filtering via largest common subsets. While the very long list would be impractical (but not infeasible), sorting and filtering the list by how much each source contributes would make it easier to manage. Pruning all the entries that are below a contribution threshold would almost certainly shrink it to a very small fraction of the total number of attributions. We don’t need to list all the attributions that contribute to only one token, all we need to know is which attributions ended up with 5 or 50 or 500 tokens in a row, right? Likely not many.