3 ms·

Isn't it more the four bit quantization than the choice of C++ as an orchestrator that's the win? It's not as if in either the C++ or the Python case that high

Isn't it more the four bit quantization than the choice of C++ as an orchestrator that's the win? It's not as if in either the C++ or the Python case that high level code is actually doing the matrix multiplications. That basically the whole AI revolution is powered by CPython of all things (not even PyPy) is the 100 megaton nuke that should end language warring forever. That the first AGI will likely be running under a VM so inefficient that it refcounts even integers is God laughing in the face of all the people who've spent the past decades arguing that this language or that language is "faster". Amdahl was right: only inner loops matter.

minimaxir 4y ago
> That basically the whole AI revolution is powered by CPython of all things (not even PyPy) is the 100 megaton nuke that should end language warring forever. And a lot of new AI tooling such as tokenization has been developed for Python using Rust (pyo3)
- Const-me 4y ago
  The original llama uses google's tooling for that, written in C++ https://github.com/google/sentencepiece https://github.com/google/sentencepiece
seydor 4y ago
Pretty sure the first AGI will design its own VM
- yreg 4y ago
  That would be the second one.
- machiaweliczny 4y ago
  I am pretty sure that first AGI will escape a VM if we won't be careful
cyber_kinetist 4y ago
That was because the current trend in AI (deep learning) mainly just requires batched matrix computations, which makes an efficient implementation fairly language-agnostic (provided that only the internal kernels are written in high-performance C/C++ or CUDA). The same thing doesn’t hold for other computationally intensive tasks like ray tracing or physics simulation, where computation cannot be expressed in huge batched matrix operations but rather millions of highly variable small-sized computations. If mainstream AI moves beyond just deep learning and start to encompass other approaches and disciplines (for example, differentiable simulators/renderers), suddenly the language (and the compiler) will be much more important than before.

minimaxir 4y ago

> That basically the whole AI revolution is powered by CPython of all things (not even PyPy) is the 100 megaton nuke that should end language warring forever. And a lot of new AI tooling such as tokenization has been developed for Python using Rust (pyo3)

Const-me 4y ago
The original llama uses google's tooling for that, written in C++ https://github.com/google/sentencepiece https://github.com/google/sentencepiece

seydor 4y ago

Pretty sure the first AGI will design its own VM

yreg 4y ago
That would be the second one.
machiaweliczny 4y ago
I am pretty sure that first AGI will escape a VM if we won't be careful

cyber_kinetist 4y ago

That was because the current trend in AI (deep learning) mainly just requires batched matrix computations, which makes an efficient implementation fairly language-agnostic (provided that only the internal kernels are written in high-performance C/C++ or CUDA). The same thing doesn’t hold for other computationally intensive tasks like ray tracing or physics simulation, where computation cannot be expressed in huge batched matrix operations but rather millions of highly variable small-sized computations. If mainstream AI moves beyond just deep learning and start to encompass other approaches and disciplines (for example, differentiable simulators/renderers), suddenly the language (and the compiler) will be much more important than before.