3 ms·

Right? What’s more, we are now using warehouses of GPUs to sort out LLM completions, a lot of which is Python code, the slowest of languages. I mean, it seems

Right? What’s more, we are now using warehouses of GPUs to sort out LLM completions, a lot of which is Python code, the slowest of languages. I mean, it seems like we’ve finally admitted even Python is too hard, so we should instead ask Deep Mind to spend a millennia computing the answers to our questions. And then when it tells us “42”, we need an even bigger computer to understand it, because we’ve outsourced understanding to the computers. We are just coming off the highs of using GPUs to generate mountains of useless money that was used to effect massive scams. But even all that is still going on. And free market capitalism is supposed to result in efficient resource allocation? What are we even doing, trying to accelerate the heat death of the universe? Entropy not increasing fast enough for us? Like… maybe we wouldn’t need so much silicon if we were using it a little better. Might as well just store everything in linked lists. Who even cares about data structures anymore?

et1337 4y ago
LLMs are about 0.5% Python. The hot path is all C/C++, or native code running on the GPU. Python is almost just a configuration language to connect it all together.
- ModernMech 4y ago
  What I'm saying is that the completions are Python code, which is then run through a Python interpreter. I keep reading stories about how this is the future of development, meaning the number of transistors to get 1+1 executing on a computer is astronomical today. Used to be that you could program a computer with everything in your head. Then you had to use the Internet, a global network of information, as a resource because it got so complicated. Now it's so bad that you also need an artificial intelligence that can interpret the information on the Internet. All so we can, what... serve more ads? Invade more privacy? Wage more wars? Earn more profits? What are we even doing with this technology?
  - bornfreddy 4y ago
    > What I'm saying is that the completions are Python code, which is then run through a Python interpreter. Even that is incorrect, or, to be precise, correct that it is indeed Python - however it is that part of Python which is actually written in C/C++ and highly optimized. There is a reason why Python is used for data science, ML and similar. It is both convenient to use and fast, thanks to numpy and other libraries. I realize that the point of your post is something else entirely, but no need to pick on Python. It's actually quite performant in areas where this is needed.
    - ModernMech 4y ago
      This is only true if Python is acting as a useful glue between things. The only reason it's a useful glue is because it's ostensibly easier to write. If we're writing the Python code with the LLM, then Python is obsolete. A better thing to do would be to just emit a binary executable that directly calls whatever machine code. But no, here we are with the worst of all worlds. Maybe that day will come, but the long-term, industry-wide trend is more silicon needed to accomplish roughly the same end goal. We're playing the same games, writing the same excel docs, browsing the same webpages, but now it takes more energy, more silicon, more transistors... call me skeptical we're going to add all this LLM tech on top of the already towering tech stack, and it will result in a reduction of needed silicon because we've simplified the whole thing considerably. > It is both convenient to use and fast, thanks to numpy and other libraries. I realize that the point of your post is something else entirely, but no need to pick on Python. My point though, was that the fast and convenient is only interesting when it comes together as a package. Other languages are faster, even granting that some Python packages are written in native code. But if the new interface to Python is an LLM, then "Python" as a language doesn't have a place anymore; it'll be more convenient to emit code in other languages.

et1337 4y ago

LLMs are about 0.5% Python. The hot path is all C/C++, or native code running on the GPU. Python is almost just a configuration language to connect it all together.

ModernMech 4y ago
What I'm saying is that the completions are Python code, which is then run through a Python interpreter. I keep reading stories about how this is the future of development, meaning the number of transistors to get 1+1 executing on a computer is astronomical today. Used to be that you could program a computer with everything in your head. Then you had to use the Internet, a global network of information, as a resource because it got so complicated. Now it's so bad that you also need an artificial intelligence that can interpret the information on the Internet. All so we can, what... serve more ads? Invade more privacy? Wage more wars? Earn more profits? What are we even doing with this technology?
- bornfreddy 4y ago
  > What I'm saying is that the completions are Python code, which is then run through a Python interpreter. Even that is incorrect, or, to be precise, correct that it is indeed Python - however it is that part of Python which is actually written in C/C++ and highly optimized. There is a reason why Python is used for data science, ML and similar. It is both convenient to use and fast, thanks to numpy and other libraries. I realize that the point of your post is something else entirely, but no need to pick on Python. It's actually quite performant in areas where this is needed.
  - ModernMech 4y ago
    This is only true if Python is acting as a useful glue between things. The only reason it's a useful glue is because it's ostensibly easier to write. If we're writing the Python code with the LLM, then Python is obsolete. A better thing to do would be to just emit a binary executable that directly calls whatever machine code. But no, here we are with the worst of all worlds. Maybe that day will come, but the long-term, industry-wide trend is more silicon needed to accomplish roughly the same end goal. We're playing the same games, writing the same excel docs, browsing the same webpages, but now it takes more energy, more silicon, more transistors... call me skeptical we're going to add all this LLM tech on top of the already towering tech stack, and it will result in a reduction of needed silicon because we've simplified the whole thing considerably. > It is both convenient to use and fast, thanks to numpy and other libraries. I realize that the point of your post is something else entirely, but no need to pick on Python. My point though, was that the fast and convenient is only interesting when it comes together as a package. Other languages are faster, even granting that some Python packages are written in native code. But if the new interface to Python is an LLM, then "Python" as a language doesn't have a place anymore; it'll be more convenient to emit code in other languages.