3 ms·
If you want to process 100M documents, you want to use the most performant and fastest option - which a 7b param model isn't going to be it. Additionally, enti
by chaxor 3y ago
If you want to process 100M documents, you want to use the most performant and fastest option - which a 7b param model isn't going to be it.
Additionally, entity linking is an extremely common task, for which LLMs fail at pretty miserably (certainly if the dictionary is custom/private. Additional work must be done to somehow (!? Many options here ?!) perform EL.
So, in the end, making a silver corpus from an LLM may be an option for NER to train a much much smaller algorithm.
But EL is _still_ not a 'plug and play' problem, and can actually be pretty difficult to do "well" (using the modern techniques of MHS, etc).