6 ms·

Funny to see this now, when AI is done by using Python to orchestrate numeric kernels written in C++ or Fortran. GOFAI lost, big time, and the key to success in

Funny to see this now, when AI is done by using Python to orchestrate numeric kernels written in C++ or Fortran. GOFAI lost, big time, and the key to success in AI is the ability to perform matrix multiplications as fast as machinely possible -- which means the central data structure you should be using is the array or vector, to ensure data locality so you don't blow up your cache. Lisp is, almost, the opposite of a language you would choose to do AI in today. It may be a more powerful language for other applications, but that will hardly matter because the productivity gains from using AI will dwarf those from programming language choice.

mhh__ 4y ago
Python at scale is really just build a graph, doesn't really matter what language you do that in other than history
varjag 4y ago
Common Lisp has certainly better array and vector support than Python so this can't be it.
- bitwize 4y ago
  That's why the numeric kernels, where all the speed-sensitive array and vector processing is done, are written in C++ or Fortran.
- gcr 4y ago
  on the minimal chance you're not being sarcastic, could you say more about this?
  - varjag 4y ago
    Common Lisp has has native support for vectors and arrays. They can be adjustable or fixed size, displaced to other arrays (kind of a "view" with a different geometry), specialized on elements of particular type or generic. For adjustable vectors you can specify their allocation growth increments for push operations. Or pre-allocate adjustable arrays but specify a lower initial size. Arrays can be accessed in row-major order mapping them into vectors etc etc. Foreign function interface of different implementations typically provide a way to access C allocated memory areas as Lisp arrays.
    - mistrial9 4y ago
      numpy changed the ecosystem dramatically.. the combination of engineering-efficient matrix math, plus the plug-and-play, low barriers to entry broad ecosystem, tipped the scales and it never went back. Millions of coders literally are playing and implementing using these python stack tools.
      - varjag 4y ago
        The efficient part in numpy is certainly not the Python code.
        vkazanov 4y ago
        Yes but what difference does it make? The interface is python. Besides, libraries are much less rigid than language features so maintainers can and do iterate on these much faster. Saying this as a big Lisp/scheme/emacs fan, btw.
        varjag 4y ago
        What I was getting at is the reason why Python is popular and CL isn't is not technical but social.
      - pharmakom 4y ago
        But this is fundamentally a native code library and we can bind it to any language that supports this. Nothing unique to Python here.
        mistrial9 4y ago
        except, evolution (and market share) are built in things that really exist now. So .. millions of users, I would emphasize that part today.
  - User23 4y ago
    Why would you think it’s sarcasm? Modern Common Lisp implementations not only generate well optimized code, they make it simple to check what’s really happening with DISASSEMBLE.
    - gcr 4y ago
      i am genuinely curious, because I think CL could do some amazing things in the symbolic algebra/compiler space here. Tensorflow's AutoGrad and AutoGraph (and I think JAX maybe) had to do terrible things to the python bytecode interpreter to compile methods, and Theano only worked by overhauling the entire number tower and adding its own control flow. If CL could become a frontend for XLA perhaps, then maybe we can go full circle and turn Torch back into the lisp project it originally started out as!
freilanzer 4y ago
Not really, it just lacks the ecosystem. Rapid feedback cycles, prototyping, introspection, performance, etc. would actually make it the ideal language. If CL 2.0 came out today it would be ideal for ML/DL.
- pharmakom 4y ago
  I have vague hopes that a Clojure implementation liberated from the JVM can be this language.
gcr 4y ago
Practitioners make distinctions between machine learning (statistical analysis + optimization) and artificial intelligence (symbolic reasoning) for important reasons. For starters, machine learning systems based on matrix multiplies and accumulation are fundamentally unexplainable in human terms; for others, they're applied to different problems (path planning and game-playing started as AI problems, and haven't seen statistical optimization applied until very recently)
User23 4y ago
Out of curiosity have you ever done any nontrivial development in a production quality Lisp using appropriate tooling like Emacs/SLIME or one of the proprietary IDEs? It’s exactly the programming environment that would be optimal for an interactive coding dialog with a LLM.
pharmakom 4y ago
Nothing stopping a lisp language from manipulating native data structures just like we do in Python. I think that Python is actually kinda bad as a language for writing logic and building abstractions. We’re still using it for AI because that’s a small part of building these systems and inertia trumps everything.

mhh__ 4y ago

Python at scale is really just build a graph, doesn't really matter what language you do that in other than history

varjag 4y ago

Common Lisp has certainly better array and vector support than Python so this can't be it.

bitwize 4y ago
That's why the numeric kernels, where all the speed-sensitive array and vector processing is done, are written in C++ or Fortran.
gcr 4y ago
on the minimal chance you're not being sarcastic, could you say more about this?
- varjag 4y ago
  Common Lisp has has native support for vectors and arrays. They can be adjustable or fixed size, displaced to other arrays (kind of a "view" with a different geometry), specialized on elements of particular type or generic. For adjustable vectors you can specify their allocation growth increments for push operations. Or pre-allocate adjustable arrays but specify a lower initial size. Arrays can be accessed in row-major order mapping them into vectors etc etc. Foreign function interface of different implementations typically provide a way to access C allocated memory areas as Lisp arrays.
  - mistrial9 4y ago
    numpy changed the ecosystem dramatically.. the combination of engineering-efficient matrix math, plus the plug-and-play, low barriers to entry broad ecosystem, tipped the scales and it never went back. Millions of coders literally are playing and implementing using these python stack tools.
    - varjag 4y ago
      The efficient part in numpy is certainly not the Python code.
      - vkazanov 4y ago
        Yes but what difference does it make? The interface is python. Besides, libraries are much less rigid than language features so maintainers can and do iterate on these much faster. Saying this as a big Lisp/scheme/emacs fan, btw.
        varjag 4y ago
        What I was getting at is the reason why Python is popular and CL isn't is not technical but social.
    - pharmakom 4y ago
      But this is fundamentally a native code library and we can bind it to any language that supports this. Nothing unique to Python here.
      - mistrial9 4y ago
        except, evolution (and market share) are built in things that really exist now. So .. millions of users, I would emphasize that part today.
- User23 4y ago
  Why would you think it’s sarcasm? Modern Common Lisp implementations not only generate well optimized code, they make it simple to check what’s really happening with DISASSEMBLE.
  - gcr 4y ago
    i am genuinely curious, because I think CL could do some amazing things in the symbolic algebra/compiler space here. Tensorflow's AutoGrad and AutoGraph (and I think JAX maybe) had to do terrible things to the python bytecode interpreter to compile methods, and Theano only worked by overhauling the entire number tower and adding its own control flow. If CL could become a frontend for XLA perhaps, then maybe we can go full circle and turn Torch back into the lisp project it originally started out as!

freilanzer 4y ago

Not really, it just lacks the ecosystem. Rapid feedback cycles, prototyping, introspection, performance, etc. would actually make it the ideal language. If CL 2.0 came out today it would be ideal for ML/DL.

pharmakom 4y ago
I have vague hopes that a Clojure implementation liberated from the JVM can be this language.

gcr 4y ago

Practitioners make distinctions between machine learning (statistical analysis + optimization) and artificial intelligence (symbolic reasoning) for important reasons. For starters, machine learning systems based on matrix multiplies and accumulation are fundamentally unexplainable in human terms; for others, they're applied to different problems (path planning and game-playing started as AI problems, and haven't seen statistical optimization applied until very recently)

User23 4y ago

Out of curiosity have you ever done any nontrivial development in a production quality Lisp using appropriate tooling like Emacs/SLIME or one of the proprietary IDEs? It’s exactly the programming environment that would be optimal for an interactive coding dialog with a LLM.

pharmakom 4y ago

Nothing stopping a lisp language from manipulating native data structures just like we do in Python. I think that Python is actually kinda bad as a language for writing logic and building abstractions. We’re still using it for AI because that’s a small part of building these systems and inertia trumps everything.