3 ms·

LLMs are about 0.5% Python. The hot path is all C/C++, or native code running on the GPU. Python is almost just a configuration language to connect it all toget

by et1337 4y ago

LLMs are about 0.5% Python. The hot path is all C/C++, or native code running on the GPU. Python is almost just a configuration language to connect it all together.

ModernMech 4y ago
What I'm saying is that the completions are Python code, which is then run through a Python interpreter. I keep reading stories about how this is the future of development, meaning the number of transistors to get 1+1 executing on a computer is astronomical today. Used to be that you could program a computer with everything in your head. Then you had to use the Internet, a global network of information, as a resource because it got so complicated. Now it's so bad that you also need an artificial intelligence that can interpret the information on the Internet. All so we can, what... serve more ads? Invade more privacy? Wage more wars? Earn more profits? What are we even doing with this technology?
- bornfreddy 4y ago
  > What I'm saying is that the completions are Python code, which is then run through a Python interpreter. Even that is incorrect, or, to be precise, correct that it is indeed Python - however it is that part of Python which is actually written in C/C++ and highly optimized. There is a reason why Python is used for data science, ML and similar. It is both convenient to use and fast, thanks to numpy and other libraries. I realize that the point of your post is something else entirely, but no need to pick on Python. It's actually quite performant in areas where this is needed.
  - ModernMech 4y ago
    This is only true if Python is acting as a useful glue between things. The only reason it's a useful glue is because it's ostensibly easier to write. If we're writing the Python code with the LLM, then Python is obsolete. A better thing to do would be to just emit a binary executable that directly calls whatever machine code. But no, here we are with the worst of all worlds. Maybe that day will come, but the long-term, industry-wide trend is more silicon needed to accomplish roughly the same end goal. We're playing the same games, writing the same excel docs, browsing the same webpages, but now it takes more energy, more silicon, more transistors... call me skeptical we're going to add all this LLM tech on top of the already towering tech stack, and it will result in a reduction of needed silicon because we've simplified the whole thing considerably. > It is both convenient to use and fast, thanks to numpy and other libraries. I realize that the point of your post is something else entirely, but no need to pick on Python. My point though, was that the fast and convenient is only interesting when it comes together as a package. Other languages are faster, even granting that some Python packages are written in native code. But if the new interface to Python is an LLM, then "Python" as a language doesn't have a place anymore; it'll be more convenient to emit code in other languages.

by et1337 4y ago

LLMs are about 0.5% Python. The hot path is all C/C++, or native code running on the GPU. Python is almost just a configuration language to connect it all together.

ModernMech 4y ago

What I'm saying is that the completions are Python code, which is then run through a Python interpreter. I keep reading stories about how this is the future of development, meaning the number of transistors to get 1+1 executing on a computer is astronomical today. Used to be that you could program a computer with everything in your head. Then you had to use the Internet, a global network of information, as a resource because it got so complicated. Now it's so bad that you also need an artificial intelligence that can interpret the information on the Internet. All so we can, what... serve more ads? Invade more privacy? Wage more wars? Earn more profits? What are we even doing with this technology?

bornfreddy 4y ago
> What I'm saying is that the completions are Python code, which is then run through a Python interpreter. Even that is incorrect, or, to be precise, correct that it is indeed Python - however it is that part of Python which is actually written in C/C++ and highly optimized. There is a reason why Python is used for data science, ML and similar. It is both convenient to use and fast, thanks to numpy and other libraries. I realize that the point of your post is something else entirely, but no need to pick on Python. It's actually quite performant in areas where this is needed.
- ModernMech 4y ago
  This is only true if Python is acting as a useful glue between things. The only reason it's a useful glue is because it's ostensibly easier to write. If we're writing the Python code with the LLM, then Python is obsolete. A better thing to do would be to just emit a binary executable that directly calls whatever machine code. But no, here we are with the worst of all worlds. Maybe that day will come, but the long-term, industry-wide trend is more silicon needed to accomplish roughly the same end goal. We're playing the same games, writing the same excel docs, browsing the same webpages, but now it takes more energy, more silicon, more transistors... call me skeptical we're going to add all this LLM tech on top of the already towering tech stack, and it will result in a reduction of needed silicon because we've simplified the whole thing considerably. > It is both convenient to use and fast, thanks to numpy and other libraries. I realize that the point of your post is something else entirely, but no need to pick on Python. My point though, was that the fast and convenient is only interesting when it comes together as a package. Other languages are faster, even granting that some Python packages are written in native code. But if the new interface to Python is an LLM, then "Python" as a language doesn't have a place anymore; it'll be more convenient to emit code in other languages.