4 ms·
I like your question, and I cannot answer it. But I have a benchmark: I can write a Markov chain "language model" in around 10-20 lines of Python, with zero ex
by klyrs 3y ago
I like your question, and I cannot answer it. But I have a benchmark: I can write a Markov chain "language model" in around 10-20 lines of Python, with zero external libraries -- with tokenization and "training" on a text file, and generating novel output. I wrote it in several minutes and didn't bother to save it.
I'm curious how much time & code it would take to implement this LLM stuff at a similar level of quality and performance.
- soraki_soladead 3y agoFLOPs by perplexity by samples is an interesting way to compare this family of models.
- fennecbutt 3y agoGenerally LLM architectures are pretty low code, I thought (not written one myself). Then all of the complexity comes with the training/weight data.