3 ms·

The #1 thing for me right now is determinism and traceability. I am in a "serious" business domain and non-determinism is a big no-no. We have to be able to jus

The #1 thing for me right now is determinism and traceability. I am in a "serious" business domain and non-determinism is a big no-no. We have to be able to justify everything in a traditional sense at the end of the day. Explaining to a regulator that we declined a customer as a consequence of a vast, unthinkable sea of weights and biases is not going to fly. For each predicted output token, I want to know exactly which source document(s) were utilized including indices from those documents and relevant statistics.

fogx 3y ago
> For each predicted output token, I want to know exactly which source document(s) were utilized including indices from those documents and relevant statistics. You don't do this for any other kind of decision or tool, why do you need it for LLMs? i think in truth you need a source that convinces you (or the regulator) that your choice is acceptable, so that you can pass off the responsibility. If the LLM were to give you an answer backed by relevant (cited) source documents of regulations and a good explanation it would make no difference compared to a human worker doing the same. -> this is already possible