6 ms·

It's weird, but the pain of moving to a whole new language seems less than the pain of using one of these language subsets. I guess it might be because of "loss

by ajb 11mo ago

It's weird, but the pain of moving to a whole new language seems less than the pain of using one of these language subsets. I guess it might be because of "loss aversion": we dislike losing something we have more the value we put on gaining it. In a new language each newly added feature is a gain, but in a language subset you're always encountering things you would have in the full language. So no matter how good the rational case is, there is a bias against them. I can't think of a successful one actually - anyone?

getnormality 11mo ago
The author seems to speak to that here: > SPy does something different: on one hand, it removes the dynamic features which make Python "slow", but on the other hand it introduces new features which make it possible to implement and use the same pythonic patterns which we like. The author seems unusually focused on explaining his new way, which may help it feel more like a new language with a coherent philosophy and unique benefits in its own right.
- jhbadger 11mo ago
  Not unlike how Crystal isn't really "compiled Ruby" but rather its own language that just looks and feels similar to Ruby.
pansa2 11mo ago
> in a language subset you're always encountering things you would have in the full language Exactly. I once pitched the idea of a Python subset (with a different focus, not performance like SPy), and almost every reaction was "will it support <favourite Python feature / favourite library>". For example, a new language can build its own solution for array math, or maybe that's not something its users need. OTOH many consider a Python subset to be unacceptable if it doesn't specifically support NumPy. In the end I came to agree with https://pointersgonewild.com/2024/04/20/the-alternative-implementation-problem/ https://pointersgonewild.com/2024/04/20/the-alternative-impl...: > positioning your project as an alternative implementation of something is a losing proposition > don't go trying to create a subset of Python [...] Do your own thing. That way, you can evolve your system at your own pace and in your own direction, without being chained by expectations that your language should have to match the performance, feature set, or library ecosystem of another implementation.
- HanClinto 11mo ago
  I can see the wisdom in this. As a counter-example, it feels as though Typescript has managed to (largely?) succeed as a subset of Javascript, so maybe the "do your own thing" isn't entirely a lost-cause -- maybe it's just really really difficult?
  - dec0dedab0de 11mo ago
    Typescript is a superset of javascript
    - HanClinto 11mo ago
      Aaah, you are correct -- I had that backwards. All Javascript is indeed valid Typescript, but not all Typescript is valid Javascript. Appreciated
- semi-extrinsic 11mo ago
  I think Python is a bit special in this case, as many people tend to use it more like duct tape for connecting different libraries (which often contain compiled and optimized code) rather than a language for doing stuff bottom-up.
netbioserror 11mo ago
Throwing my hands up and moving to Nim was downright easy next to the excessive effort I put into trying out Nuitka, Numba, and PyInstaller for my use case. If you want static compilation, use a language and libraries built with that assumption as a ground rule. The herculean effort of building a half-compatible compiler for a dynamic language seems like a fool's errand, and would be a fun curiosity if so many people hadn't already tried it, especially with Python.
- PaulHoule 11mo ago
  It's so common for something like this to struggle for decades before succeeding. For instance people have been trying to make a memory safe C for a long time, just recently we got https://fil-c.org/ https://fil-c.org/ which has high compatibility and relatively good performance for that kind of thing. The strength of Python is that so many people are trying things with it.
- almostgotcaught 11mo ago
  all of this is well and good if you completely forget that there are billions of lines of Python in prod right now. so your grand epiphany is basically on the level of "let's rewrite it in Rust". i'll let you hold your breath until that rewrite is done (and in the meantime i'll explore workarounds).
  - rangerelf 11mo ago
    I kind of object to this take. Nobody's talking about porting billions of lines of code, for all we know it's just for personal projects, or a learning experience. This kind of replies is like killing an idea before it's even started, smells like the sunk cost fallacy. OTOH I do understand the weight of a currently existing corpus in production, evidence is the ton of COBOL code still running. But still, your reply kind of sucks.
    - almostgotcaught 11mo ago
      > Nobody's talking about porting billions of lines of code, for all we know it's just for personal projects, or a learning experience. am i the only person in the room that can read tone? please tell me what is the force of this statement in what i've responded: > If you want static compilation, use a language and libraries built with that assumption as a ground rule. is this an imperative only for hobbyists? not sure. > This kind of replies is like killing an idea before it's even started, smells like the sunk cost fallacy. there is no idea - that's exactly my whole point. tear it down and build it again is not an idea.
dec0dedab0de 11mo ago
I would say that Rpython is successful. However, it's primary goal is to be whatever the PyPy developers need to write PyPy. I totally agree about subsets though, I would much rather have a superset like Cython, but that has it's own challenges for compatibility if you wanted to have a "pure python" version of the same library. Which is why I really like that Cython is using typehints now.
- ajb 11mo ago
  Yeah Rpython escapes this as most people don't write it directly. Micropython is another example. No-one expects micropython to support numpy. They're just happy to get away from C. But where you can use the full python, you wouldn't use micropython.
PaulHoule 11mo ago
One of the standard architectures for complex applications is to put together a scripting language and an systems programming language. You can either look at the scripting language as primary with the systems language used for performance or hardware interfacing or you can look at the systems language being primary and the scripting being used around the edges. The model of having a closely related scripting and systems language would be an optimization of this and SPy seems like an answer. [1] AAA games that have a scripting engine on top of C/C++, a finite element solver that lets you set up a problem with Lua and solve it with FORTRAN, etc.
- thomasmg 11mo ago
  Yes, there are quite many real-world cases of this architecure. But, wouldn't it be better it the same language (more or less) can be used for both? I don't think such a language exists currently, but I think it would be a nice goal.
  - sophrosyne42 11mo ago
    Oxcaml, of course.
    - thomasmg 11mo ago
      Interesting, I was not aware of OxCaml. (If this is what you mean.) It does seem to tick a few boxes actually. For my taste, the syntax is not as concise / clean as Python, and for me, it is "too functional". It shares with Python (and many high-level languages) the tracing garbage collection, but maybe that is the price to pay for an easy-to-use memory safe language. In my view, a "better" language would be a simple language as concise as Python, but fully typed (via type inference); memory safe, but without the need of tracing GC. I think memory management should be a mix of Swift and Rust (that is, a mix of reference counting and single ownership with borrowing, where need for speed).
  - stonemetal12 11mo ago
    What would be the point of the second language if it was basically the same language as the first? In general you want a solid engineering language with a focus on correctness, and a second "scripting" language focusing on quick development. Second if the languages are very similar but not the same it seems like you would see "confusion" errors where people accidently use lang 1 in lang 2's context or vice versa.

by ajb 11mo ago

getnormality 11mo ago

The author seems to speak to that here: > SPy does something different: on one hand, it removes the dynamic features which make Python "slow", but on the other hand it introduces new features which make it possible to implement and use the same pythonic patterns which we like. The author seems unusually focused on explaining his new way, which may help it feel more like a new language with a coherent philosophy and unique benefits in its own right.

jhbadger 11mo ago
Not unlike how Crystal isn't really "compiled Ruby" but rather its own language that just looks and feels similar to Ruby.

pansa2 11mo ago

> in a language subset you're always encountering things you would have in the full language Exactly. I once pitched the idea of a Python subset (with a different focus, not performance like SPy), and almost every reaction was "will it support <favourite Python feature / favourite library>". For example, a new language can build its own solution for array math, or maybe that's not something its users need. OTOH many consider a Python subset to be unacceptable if it doesn't specifically support NumPy. In the end I came to agree with https://pointersgonewild.com/2024/04/20/the-alternative-implementation-problem/ https://pointersgonewild.com/2024/04/20/the-alternative-impl...: > positioning your project as an alternative implementation of something is a losing proposition > don't go trying to create a subset of Python [...] Do your own thing. That way, you can evolve your system at your own pace and in your own direction, without being chained by expectations that your language should have to match the performance, feature set, or library ecosystem of another implementation.

HanClinto 11mo ago
I can see the wisdom in this. As a counter-example, it feels as though Typescript has managed to (largely?) succeed as a subset of Javascript, so maybe the "do your own thing" isn't entirely a lost-cause -- maybe it's just really really difficult?
- dec0dedab0de 11mo ago
  Typescript is a superset of javascript
  - HanClinto 11mo ago
    Aaah, you are correct -- I had that backwards. All Javascript is indeed valid Typescript, but not all Typescript is valid Javascript. Appreciated
semi-extrinsic 11mo ago
I think Python is a bit special in this case, as many people tend to use it more like duct tape for connecting different libraries (which often contain compiled and optimized code) rather than a language for doing stuff bottom-up.

netbioserror 11mo ago

Throwing my hands up and moving to Nim was downright easy next to the excessive effort I put into trying out Nuitka, Numba, and PyInstaller for my use case. If you want static compilation, use a language and libraries built with that assumption as a ground rule. The herculean effort of building a half-compatible compiler for a dynamic language seems like a fool's errand, and would be a fun curiosity if so many people hadn't already tried it, especially with Python.

PaulHoule 11mo ago
It's so common for something like this to struggle for decades before succeeding. For instance people have been trying to make a memory safe C for a long time, just recently we got https://fil-c.org/ https://fil-c.org/ which has high compatibility and relatively good performance for that kind of thing. The strength of Python is that so many people are trying things with it.
almostgotcaught 11mo ago
all of this is well and good if you completely forget that there are billions of lines of Python in prod right now. so your grand epiphany is basically on the level of "let's rewrite it in Rust". i'll let you hold your breath until that rewrite is done (and in the meantime i'll explore workarounds).
- rangerelf 11mo ago
  I kind of object to this take. Nobody's talking about porting billions of lines of code, for all we know it's just for personal projects, or a learning experience. This kind of replies is like killing an idea before it's even started, smells like the sunk cost fallacy. OTOH I do understand the weight of a currently existing corpus in production, evidence is the ton of COBOL code still running. But still, your reply kind of sucks.
  - almostgotcaught 11mo ago
    > Nobody's talking about porting billions of lines of code, for all we know it's just for personal projects, or a learning experience. am i the only person in the room that can read tone? please tell me what is the force of this statement in what i've responded: > If you want static compilation, use a language and libraries built with that assumption as a ground rule. is this an imperative only for hobbyists? not sure. > This kind of replies is like killing an idea before it's even started, smells like the sunk cost fallacy. there is no idea - that's exactly my whole point. tear it down and build it again is not an idea.

dec0dedab0de 11mo ago

I would say that Rpython is successful. However, it's primary goal is to be whatever the PyPy developers need to write PyPy. I totally agree about subsets though, I would much rather have a superset like Cython, but that has it's own challenges for compatibility if you wanted to have a "pure python" version of the same library. Which is why I really like that Cython is using typehints now.

ajb 11mo ago
Yeah Rpython escapes this as most people don't write it directly. Micropython is another example. No-one expects micropython to support numpy. They're just happy to get away from C. But where you can use the full python, you wouldn't use micropython.

PaulHoule 11mo ago

One of the standard architectures for complex applications is to put together a scripting language and an systems programming language. You can either look at the scripting language as primary with the systems language used for performance or hardware interfacing or you can look at the systems language being primary and the scripting being used around the edges. The model of having a closely related scripting and systems language would be an optimization of this and SPy seems like an answer. [1] AAA games that have a scripting engine on top of C/C++, a finite element solver that lets you set up a problem with Lua and solve it with FORTRAN, etc.

thomasmg 11mo ago
Yes, there are quite many real-world cases of this architecure. But, wouldn't it be better it the same language (more or less) can be used for both? I don't think such a language exists currently, but I think it would be a nice goal.
- sophrosyne42 11mo ago
  Oxcaml, of course.
  - thomasmg 11mo ago
    Interesting, I was not aware of OxCaml. (If this is what you mean.) It does seem to tick a few boxes actually. For my taste, the syntax is not as concise / clean as Python, and for me, it is "too functional". It shares with Python (and many high-level languages) the tracing garbage collection, but maybe that is the price to pay for an easy-to-use memory safe language. In my view, a "better" language would be a simple language as concise as Python, but fully typed (via type inference); memory safe, but without the need of tracing GC. I think memory management should be a mix of Swift and Rust (that is, a mix of reference counting and single ownership with borrowing, where need for speed).
- stonemetal12 11mo ago
  What would be the point of the second language if it was basically the same language as the first? In general you want a solid engineering language with a focus on correctness, and a second "scripting" language focusing on quick development. Second if the languages are very similar but not the same it seems like you would see "confusion" errors where people accidently use lang 1 in lang 2's context or vice versa.