4 ms·

Missing: write static python and transpile to rust pyO3 which is at the top of the ladder. Some nuance: try transpiling to a garbage collected rust like langua

Missing: write static python and transpile to rust pyO3 which is at the top of the ladder. Some nuance: try transpiling to a garbage collected rust like language with fast compilation until you have millions of users. Also use a combination of neural and deterministic methods to transpile depending on the complexity.

tda 7mo ago
One thing with python is that usually I will use one of the many c based libraries to get reasonable speed and well thought out abstractions from the start. I architect around numpy, scipy, shapely, pandas/polars or whatever. So my code runs at reasonable speed from the start. But transpiling to rust then effectively means a complete redesign of the code, data structures, algorithms etc. And I have seen the AI tools really struggle to get it right, as my intent gets lost somewhere. So what I do now (since Claude Code) is write really bare bones (and slow) pure python implementation (like I used to do for numba, pypy or cython ready code), with minimal dependencies. Then I use the REPL, notebooks and nice plotting tools to get a real understanding of the problem space and the intricacies of my algorithm/problem at hand. When done, I let Claude add tests and I ask it to transpile to equivalent Rust and boom! a flawless 1000x speed upgrade in a minutes. The great thing is I don't need to do the mental gymnastics to vectorize code in a write only mode like I've had to do since my Matlab days. Instead I can write simple to read for loops that follow my intent much better, and result in much more legible code. So refreshing! And with pyO3 i can still expose the Rust lib to python, and continue to use Python for glue and plotting
- adsharma 7mo ago
  Cython and all the libs you mention use the c-api, which is the #1 thing python needs to lose to be competitive. I wish someone writes a stdlib without using it. My attempt from a few months ago in a repo under the py2many org.
  - gf000 7mo ago
    > Cython and all the libs you mention use the c-api, which is the #1 thing python needs to lose to be competitive. Quite hard to lose the #1 reason people use the language for.
    - adsharma 7mo ago
      The #1 reason people use it is because it allows you to focus on the problem you're solving rather than the syntax, memory management or some other aspect. Most people don't even know what C-API is or why it slows things down. Compatibility is made into a bigger deal than it is. That's the COBOL argument. I wish the python community focused more on why openclaw and opencode are getting written in typescript, not python. Why aren't agents more efficient at translating python code into shippable end user binaries using fast interpreted -> compiled agentic loops and attempt memory safety only for binaries/libraries with a large distribution.
zahlman 7mo ago
> a garbage collected rust like language with fast compilation I don't know what languages you might have in mind. "Rust-like" in what sense?
- pjmlp 7mo ago
  Probably OCaml, Standard ML, Haskell, MLton, F#, Scala,.... If going to complain about some of those being slow, remeber that they have various options between interpreter, bytecode, REPL, JIT and AOT.
  - zahlman 7mo ago
    You know, I really should try out F# some time. I always preferred C# over Java, and I have some Scala experience (that wasn't overall very pleasant, but it was fun to use an FP language).
- adsharma 7mo ago
  It's not a popular thing to say on social media. V-lang is the one I'm tinkering with. It's like rust in terms of pattern matching as an expression, sum types, ?T instead of exceptions. Like golang, it has shorter compile times. I try to keep my argument abstract (that you need to lower python to something intermediate before rust) for that reason.
  - adsharma 7mo ago
    Here is a python AST parser written in V. It's targeting a dialect that's mostly compatible with a static subset of python3, but will break compatibility where necessary. In this case pattern matching, possibly elsewhere. https://github.com/py2many/v-ast https://github.com/py2many/v-ast
    - zahlman 7mo ago
      Never heard of this language, but it looks interesting. Very modern, certainly. One thing that stood out to me is that there's apparently the ability to write a bare `for` loop...? Is that just equivalent to while (true) in other languages?

tda 7mo ago

One thing with python is that usually I will use one of the many c based libraries to get reasonable speed and well thought out abstractions from the start. I architect around numpy, scipy, shapely, pandas/polars or whatever. So my code runs at reasonable speed from the start. But transpiling to rust then effectively means a complete redesign of the code, data structures, algorithms etc. And I have seen the AI tools really struggle to get it right, as my intent gets lost somewhere. So what I do now (since Claude Code) is write really bare bones (and slow) pure python implementation (like I used to do for numba, pypy or cython ready code), with minimal dependencies. Then I use the REPL, notebooks and nice plotting tools to get a real understanding of the problem space and the intricacies of my algorithm/problem at hand. When done, I let Claude add tests and I ask it to transpile to equivalent Rust and boom! a flawless 1000x speed upgrade in a minutes. The great thing is I don't need to do the mental gymnastics to vectorize code in a write only mode like I've had to do since my Matlab days. Instead I can write simple to read for loops that follow my intent much better, and result in much more legible code. So refreshing! And with pyO3 i can still expose the Rust lib to python, and continue to use Python for glue and plotting

adsharma 7mo ago
Cython and all the libs you mention use the c-api, which is the #1 thing python needs to lose to be competitive. I wish someone writes a stdlib without using it. My attempt from a few months ago in a repo under the py2many org.
- gf000 7mo ago
  > Cython and all the libs you mention use the c-api, which is the #1 thing python needs to lose to be competitive. Quite hard to lose the #1 reason people use the language for.
  - adsharma 7mo ago
    The #1 reason people use it is because it allows you to focus on the problem you're solving rather than the syntax, memory management or some other aspect. Most people don't even know what C-API is or why it slows things down. Compatibility is made into a bigger deal than it is. That's the COBOL argument. I wish the python community focused more on why openclaw and opencode are getting written in typescript, not python. Why aren't agents more efficient at translating python code into shippable end user binaries using fast interpreted -> compiled agentic loops and attempt memory safety only for binaries/libraries with a large distribution.

zahlman 7mo ago

> a garbage collected rust like language with fast compilation I don't know what languages you might have in mind. "Rust-like" in what sense?

pjmlp 7mo ago
Probably OCaml, Standard ML, Haskell, MLton, F#, Scala,.... If going to complain about some of those being slow, remeber that they have various options between interpreter, bytecode, REPL, JIT and AOT.
- zahlman 7mo ago
  You know, I really should try out F# some time. I always preferred C# over Java, and I have some Scala experience (that wasn't overall very pleasant, but it was fun to use an FP language).
adsharma 7mo ago
It's not a popular thing to say on social media. V-lang is the one I'm tinkering with. It's like rust in terms of pattern matching as an expression, sum types, ?T instead of exceptions. Like golang, it has shorter compile times. I try to keep my argument abstract (that you need to lower python to something intermediate before rust) for that reason.
- adsharma 7mo ago
  Here is a python AST parser written in V. It's targeting a dialect that's mostly compatible with a static subset of python3, but will break compatibility where necessary. In this case pattern matching, possibly elsewhere. https://github.com/py2many/v-ast https://github.com/py2many/v-ast
  - zahlman 7mo ago
    Never heard of this language, but it looks interesting. Very modern, certainly. One thing that stood out to me is that there's apparently the ability to write a bare `for` loop...? Is that just equivalent to while (true) in other languages?