7 ms·

SPy: An interpreter and compiler for a fast statically typed variant of Python

• jmpeax 1y ago
  I can't view the site on my mobile without accepting cookies.
  • alexaholic 1y ago
    Specifically Google Analytics cookies, but I found you can uncheck the box.
    • pred_ 1y ago
      It's still pretty confusing: Uunchecking the box doesn't seem to do much (is it actually unchecked when you click it? There's still a checkmark); you still have to click Accept to see the text; what are you accepting? In any case, pre-checked boxes are not valid consent under GDPR (“Planet49”).
  • austinjp 1y ago
    No cookie notice at all for me using Firefox on Android with the "I Still Don't Care About Cookies" extension.
• wodenokoto 1y ago
  I like the idea of a compiled language that takes the look and ethos of Python (or at least the "looks like pseudocode, but runs"-ethos) I don't think the article gives much of an impression on how SPy is on that front.
  • summarity 1y ago
    You can have that today with Nim.
    • Imustaskforhelp 1y ago
      Nim feels like a really amazing language. There were some minor things that I wanted to do with it. Like trying to solve a codeforces question just out of mere curiosity to build something on top of it. I felt like although it was similar to python. You can't underestimate the python's standard library features which I felt lacking. I am not sure if these were skill issues. Yes these are similar languages but I would still say that I really welcome a language like SPy too. The funny thing is that I ended up architecting a really complicated solution to a simple problem in nim and I was proud of it and then I asked chatgpt thinking no way there can be anything simpler for it in nim and I found something that worked in 7-10 or 12* lines and my jaw dropped lol. Maybe chatgpt could be decent to learn nim imo or reading some nim books for sure but the packages environment etc. felt really brittle as well. I think that there are good features of both nim and SPy and I welcome both personally.
      • summarity 1y ago
        GPT is amazing at Nim. Ive used it to find a subtle bug in a macro that’s hundreds of lines of code.
    • odie5533 1y ago
      There don't seem to be great web frameworks like Flask, Django, or FastAPI for Nim.
      • cb321 1y ago
        "Great" smells very subjective. I went to https://forum.nim-lang.org/ https://forum.nim-lang.org/ . Put "flask" in the search box. Second hit (https://forum.nim-lang.org/search?q=flask https://forum.nim-lang.org/search?q=flask) is this: https://forum.nim-lang.org/t/11032 https://forum.nim-lang.org/t/11032 . That mentions not one but 2 projects (https://github.com/HapticX/happyx https://github.com/HapticX/happyx & https://github.com/planety/prologue https://github.com/planety/prologue). If either/both are not "great enough" in some particulars you want, why not raise a github issue? (Or even better look into adding said particulars yourself? This is really the main way Python grew its ecosystem.)
  • walterlw 1y ago
    I believe that Python is as popular and widely used as it is because it's old enough to have an expansive ecosystem of libraries. It's easy enough to implement one in pure Python and possible to optimize it later (Pydantic is a great recent-ish example, switching to a Rust core for 2.0). That same combination of Python + (choose a compiled language) makes it quite difficult for any new language to tap into the main strength of Python.
    • rangerelf 1y ago
      It's not just its age, it's how easy it is (was?) to jump in and start writing useful code that could be revisited later on and be able to read it and understand it again. All of these efforts to turn it into another Typescript are going to, in the end, kill the ease of use it has always had.
    • ReflectedImage 1y ago
      Python is popular because of it's expansive ecosystem of libraries, which only exist because the language is duck typed. If it was statically typed, the expansive ecosystem of libraries wouldn't exist. There is a factor of 3x difference in dev speed between the two typing systems.
  • greener_grass 1y ago
    This is what F# provides. F# has a similar whitespace syntax to Python, but is statically typed and can be compiled AoT. Bubble sort Python: mylist = [64, 34, 25, 12, 22, 11, 90, 5] n = len(mylist) for i in range(n-1): for j in range(n-i-1): if mylist[j] > mylist[j+1]: mylist[j], mylist[j+1] = mylist[j+1], mylist[j] print(mylist) Bubble sort F#: let mylist = ResizeArray [ 64; 34; 25; 12; 22; 11; 90; 5 ] let n = Seq.length mylist for i = 0 to n - 2 do for j = 0 to n - i - 2 do if mylist[j] > mylist[j + 1] then let temp = mylist[j] mylist[j] <- mylist[j + 1] mylist[j + 1] <- temp printfn "%A" mylist
    • deleted 1y ago
      [deleted]
    • stOneskull 1y ago
      Nim: var mylist = [64, 34, 25, 12, 22, 11, 90, 5] let n = mylist.len for i in 0..n-2: for j in 0..n-i-2: if mylist[j] > mylist[j + 1]: swap(mylist[j], mylist[j + 1]) echo mylist
• srean 1y ago
  If you want different parts of your code to be a statically typed Python lookalike Cython is a mature option
  • leobuskin 1y ago
    Yes, it's mature, but you (and your potential audience) basically need to learn a new language, a lot of quirks and "weird" (I'd even say counter-intuitive) nuances, and it's also significantly less readable in comparison with strict and typed Python. Even its modern syntax doesn't click immediately (also performance wise the new syntax somehow is a bit slower in my tests)
    • srean 1y ago
      I am by no means a Cython fanboy but I think you are exaggerating the syntactic differences and readability differences. Apart from type annotation they are very minor, well worth the speed benefits and type-error benefits. Given that we are discussing it in the context of SPy, SPy is not fully compatible with Python either, which is quite understandable and in my opinion a Good trade-off. The benchmarking features are great, interactivity with C libraries is great. One annoyance I have with Cython though is debuggability. But it's an annoyance, not a show-stopper. Have used in production without problems.
    • nylonstrung 1y ago
      It does seems very much like "worst of both worlds" in that these nuances make it harder to write than a language like Go while still being drastically less performant
• falcor84 1y ago
  This seems to be going for a somewhat similar goal to Mojo [0] - anyone here who used both and is willing to offer a comparison? [0] https://www.modular.com/mojo https://www.modular.com/mojo
  • miohtama 1y ago
    Based on my understanding, Mojo aims to make number crunch computation faster (GPU), while as SPy aims to make generic Python application logic faster. Very similar, but different sweet spots and use cases.
    • ivell 1y ago
      While GPU is a focus of Mojo, it is also planned to make it a general system programming language similar to C++ and Rust.
    • skavi 1y ago
      Your understanding of mojo is incomplete. Just visiting their website would have cleared that up.
      • falcor84 1y ago
        Taking that on a tangent, there's the seed of a very interesting sci-fi story in a world where visiting a website is sufficient to give people a complete understanding of something. EDIT: thinking about it some more, I would say that there isn't any real barrier with current technoly to having a relatively small on-device LLM downloading a LoRA file from an arbitrary site to get the AI equivalent of "I know Kung-Fu" - the real issue would be on the safety, security and trust aspects.
  • actionfromafar 1y ago
    Time for me to remind everyone of the Shedskin Python compiler. https://shedskin.github.io/ https://shedskin.github.io/
  • emmelaich 1y ago
• mpweiher 1y ago
  Looks very interesting! I remember chatting with one of the creators of PyPy (not the author of TFA) a number of years ago at HPI. He had just given a talk about how RPython was used in PyPy development, and I was fascinated. To me, it seemed completely obvious that RPython itself seemed like a really interesting standalone language, but he would have none of it. Whenever I suggested that RPython might have advantages over PyPy he insisted that PyPy was better and, more strangely, just as fast. Which was sort of puzzling, because the reason given for RPython was speed. When I then suggested that they could (after bootstrap) just use PyPy without the need for RPython, he insisted that PyPy was too slow for that to be feasible. The fact that both of these statements could not really be true at the same time did not register.
  • albertzeyer 1y ago
    I have asked about using RPython as a generic standalone language before. I think the official statement is that is was never intended to become one, and it's really a very minimal subset of Python (so basically no existing Python code will run, it would require heavy refactoring or complete rewrite), and it's only specifically those features that they currently need, and it might also be a moving target, and they don't want to give certain guarantees on stability of the language etc. Once you consider that you anyway need to write very different kind of code for RPython, then maybe just using Nim or some other language is a better idea?
    • kombine 1y ago
      A general purpose language should be suitable to writing its own compiler. If it's to slow for that, what's the point?
      • Hasnep 1y ago
        A language can be suitable for writing a compiler, but if there is another language that's 10x faster that's also suitable, then you're losing out on a lot of compilation speed for no reason. Dog-fooding a language by writing a compiler in it can lead to the designers adding language features to make compiler development easier, even if they detract from the design of the language for the 99% of users who aren't writing a compiler.
• ksk23 1y ago
  Good level of detail (for me) to understand (some things).
• cardanome 1y ago
  Common Lisp also allows you to redefine everything at runtime but doesn't suffer from the same performance issues that Python has, does it? Doe anyone have insight into this?
  • pjmlp 1y ago
    Just like Smalltalk and SELF, also Lisp Machines and Interlisp-D. Usually comes down from a urban myth that Python is special and there was no other dynamic language before it came to be. The JIT research on those platforms is what gave us leading JIT capabilities on modern runtimes, OpenJDK HotSpot traces back to Smalltalk and StrongTalk, while V8 traces back to SELF. Especially in Smalltalk and SELF, you can change anything at any time across the whole image, and have the JIT pick up on that and re-optimize. Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available. A reason why JVM, CLR, V8, ART make use of handles and have marshaling layers not allowing such kind of liberties with native extensions.
    • gjvc 1y ago
      Great explanation. Five years ago I did the genealogical work to discover that StrongTalk begat HotSpot (by virtue of having some of the same authors) It was quite a joy to discover!
    • vidarh 1y ago
      > Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available. I'm doing a Ruby compiler (very, very slowly, though faster again now with Claude Code doing most of the heavy lifting), and the same issue with Ruby has made me seriously toy with the idea of one day embedding a C compiler so that the Ruby compiler can optimise across both (it'd still have to deal with linking to third party C code, of course, which is one reason I'm hesitating) as a simple not-so-optimizing C compiler is like a trivial toy compared to compiling Ruby where just the parser makes you want to claw your eyes out, but it'd at least widen the surface a bit.
• nickcw 1y ago
  This looks like a very interesting approach bringing comptime to a static version of python. This version of comptime can then be used to define new types in the same way Zig does it. I absolutely hate the terminology though red/blue redshifting etc. Why do blue functions disappear when redshifting? If you red shift blue then it goes down in frequency so you might get green or red. Perhaps my physics brain is just over thinking it! > The other fundamental concept in SPy is redshifting. > Each expression is given a color: > blue expressions are those which can safely be evaluated ahead of time, because they don't have side effects and all operands are statically known. > red expressions are those which needs to be evaluated at runtime. > During redshifting we eagerly evaluate all the blue parts of the code: it's a form of partial evaluation. This process plays very well with the freezing that we discussed above, because a lot of operations on frozen data become automatically blue: for example, if we statically know the type of an object, the logic to look up a method inside the frozen class hierarchy is a blue operation and it's optimized away, leaving just a direct call as a result. Please just rename it comptime then at least people who have learnt Zig will know what it means immediately. In FORTH these would have been called IMMEDIATE words. Namely functions which run at "compile" time rather than run time.
  • antocuni 1y ago
    hello, spy author here. > Why do blue functions disappear when redshifting? If you red shift blue then it goes down in frequency so you might get green or red. Perhaps my physics brain is just over thinking it! yes I think you are overthinking :). It's not meant to be accurate physics of course. The usage of colors to distinguish comptime vs runtim code comes from PyPy: in that context, we used "green" and "red", and initial versions of SPy used the same convention. Then someone pointed out that green/red is not colorblind friendly and so I changed it to blue. Having actual colors for the two phases is VERY useful for visualization: e.g. we already have a "spy --colorize" command which shows you which parts are blue and which are red. As for "redshifting": the AST "before" has a mixture of blue and red colors, while the AST "after" has only red nodes, thus the final AST is "more red" than the first one, that's why I chose that name.
• AmazingTurtle 1y ago
  > SPy is [...] a compiler > SPy is not a "compiler for Python" I think it's funny how it's confusing from the first paragraph
  • ForceBru 1y ago
    Reading the next sentence clears the confusion: > SPy is not a "compiler for Python". There are features of the Python language which will never be supported by SPy by design. Don't expect to compile Django or FastAPI with SPy.
    • AmazingTurtle 1y ago
      Yeah but then don't say that SPy is a (interpreter and) compiler in the first place? Just say it's a interpreter.
      • yorwba 1y ago
        It is a compiler. It is not a compiler for Python, because there are valid Python programs it can't compile and isn't intended to compile.
      • Hasnep 1y ago
        You can think of it like this: SPy is a compiler. SPy is not a compiler for OCaml. SPy is not a compiler for COBOL. SPy is not a compiler for Python.
  • amelius 1y ago
    To make it more confusing: SPy is not spyware (at least, I hope)
• sevensor 1y ago
  Neat idea! Author’s ideas about different subsets of Python are worth the price of admission. What you can express in the type system, what performs well under JIT, what’s basically same and reasonable, may not be precisely specified, but are still useful and distinct ideas.
• graemep 1y ago
  The problem with this is that the main value of Python is its ecosystem. SPy aims to be able to import Python libraries, but also not implement all Python features. If you are not 100% compatible how can you reliably import libraries? SPy seems most likely to be more likely to be appealing as a more Pythonic alternative to Cython rather than a Python replacement.
  • rurban 1y ago
    I did a similar project, a typed perl. cperl. I could import most the modules, and did add types to some of the important modules. Eg testing was 2x faster. I needed typing patches for about 10% for most CPAN packages. A type is a contract, not a hint!
    • setopt 1y ago
      > A type is a contract, not a hint! In Python it is a hint.
      • rurban 1y ago
        Exactly. That was their worst mistake ever
        • setopt 1y ago
          I agree. I use Beartype to get runtime type checks, but it shouldn’t be necessary. Some support for type checking, whether at byte compile time or runtime, should land upstream.
          • almostgotcaught 1y ago
            Never gonna happen - the fundamental premise of the language is duck typing.
            • setopt 1y ago
• codethief 1y ago
  Antonio Cuni gave a great talk about SPy at EuroPython 2024: https://ep2024.europython.eu/session/spy-static-python-lang-fast-as-c-pythonic-as-python/ https://ep2024.europython.eu/session/spy-static-python-lang-...
• ajb 1y ago
  It's weird, but the pain of moving to a whole new language seems less than the pain of using one of these language subsets. I guess it might be because of "loss aversion": we dislike losing something we have more the value we put on gaining it. In a new language each newly added feature is a gain, but in a language subset you're always encountering things you would have in the full language. So no matter how good the rational case is, there is a bias against them. I can't think of a successful one actually - anyone?
  • getnormality 1y ago
    The author seems to speak to that here: > SPy does something different: on one hand, it removes the dynamic features which make Python "slow", but on the other hand it introduces new features which make it possible to implement and use the same pythonic patterns which we like. The author seems unusually focused on explaining his new way, which may help it feel more like a new language with a coherent philosophy and unique benefits in its own right.
    • jhbadger 1y ago
      Not unlike how Crystal isn't really "compiled Ruby" but rather its own language that just looks and feels similar to Ruby.
  • pansa2 1y ago
    > in a language subset you're always encountering things you would have in the full language Exactly. I once pitched the idea of a Python subset (with a different focus, not performance like SPy), and almost every reaction was "will it support <favourite Python feature / favourite library>". For example, a new language can build its own solution for array math, or maybe that's not something its users need. OTOH many consider a Python subset to be unacceptable if it doesn't specifically support NumPy. In the end I came to agree with https://pointersgonewild.com/2024/04/20/the-alternative-implementation-problem/ https://pointersgonewild.com/2024/04/20/the-alternative-impl...: > positioning your project as an alternative implementation of something is a losing proposition > don't go trying to create a subset of Python [...] Do your own thing. That way, you can evolve your system at your own pace and in your own direction, without being chained by expectations that your language should have to match the performance, feature set, or library ecosystem of another implementation.
• IshKebab 1y ago
  Overall this looks very interesting - I always thought more could have been done with RPython, but there was never any documentation for it. I do have some nits though: > The following silly example happily passes mypy In fairness that's because Mypy is shit. Pyright catches this mistake. > As such, we need to treat Python type checkers more like linters than actual theorem provers Again I think this is true of Mypy but Pyright is much closer to a sound type checker. > redshifting This is just constant propagation isn't it? C compilers have been doing this for decades. I don't think we need a silly new term for it. Let's not cause another "tree shaking" situation. > So far, this is not different than usual constant folding, with the difference that it's guaranteed to happen. What makes it more powerful is the ability to mark some functions as @blue. That's not different. It's just constant folding with C++'s `consteval`. And @blue is an absolutely abysmal name. It would be much clearer if @blue were changed to @consteval or @comptime (either's good I think), and you just call it "compile time evaluation" or "constant propagation" instead of "redshifting".
• toast_x 1y ago
  i thought this was super cool and informative thank you so much for writing it. i work a lot in python and tried to build a runtime type checker for it that basically acted as automatic assertions on all stated types at function boundaries, and it worked really well to bring all TypeErrors in my code to the surface on first run. but, my knowledge is woefully short on compilers and languages. this definitely gave me a great frame of understanding, so thanks again for writing it
• adsharma 1y ago
  ++spy ethos and ideas But why limit to an interpreter? Translate to other excellent compiled languages and benefit from the optimization work there. Giving up on C-API and the dynamic parts of python that 1% of the people use is a good trade-off. In the age of cursor and windsurf it's not hard to auto replace incompatible code with something that works in the static-py ecosystem. Would love to participate in an effort to standardize such a subset.
  • nusl 1y ago
    There is a compiler detailed in the page on the link; > 3. We have a compiler for deployment and performance. The interpreter and the compiler are guaranteed to produce the exact same results at runtime.
    • adsharma 1y ago
      Where is it? Would love to compare the approach to py2many.
      • intalentive 1y ago
        https://github.com/spylang/spy https://github.com/spylang/spy
        • adsharma 1y ago
          Py2many would fail to compile dynamic code whereas spy runs it through the interpreter. Otherwise the internal structure looks similar. Py2many has been around for 10+ years under previous names and has significant test coverage.
  • rich_sasha 1y ago
    I'd imagine a lot of packages that you may want to use make deep use of some of these obscure features. So much magical "it just works" of Django is surely various kinds of deep introspection. Not sure an AI can fix it yet. It's not just adding type annotations.
• vidarh 1y ago
  I sympathise with the motivations for this, though I don't use Python much. I occasionally work on a toy Ruby compiler that started as a long blog series. More recently I've picked it up again with heavy AI use - I set Claude working on improving Rubyspec pass rates (which are atrocious). It's chugging along right now, actually. One of the things I've spent a lot of time thinking about are the ways to avoid a lot of the dynamic features of Ruby without affecting actual, real code much. There's a lot that can be done there - e.g. all of the research on Self and JS VMs is highly applicable. But I say "real code" because a lot of the "worst" dynamic features of Ruby (and Python) either doesn't appear in production code very often, or at all (there are still aspects of Ruby I have never seen used in real-life use despite having used Ruby for 20 years), or could be mitigated trivially, so I still believe you can do quite decently without a lot of the more complex optimisations. As an example (from Ruby): You can re-open the Integer class and override +: ruby -e 'class Integer; def +(other); 42; end; end; p 2 + 2' (don't do this in IRB; it crashes) But nobody does that. The problem with a lot of these features isn't that people use them, but that people might use them. That leaves two main avenues: We can do like the author of thise post and strip away the costly features that are rarely used. Or we can provide ways of promising not to use them through code or options. The first option is perfectly valid, but I quite like the second: In Ruby, it turns out a lot of the optimisation challenges goes away if you get an app to freeze the most important system classes after setup, because even most of the most horrific examples of Ruby monkeypatching tends to do most of it only during startup, and you then tend to get to a stable state where you can let applications opt in to additional optimisations just by calling "freeze" on a number of objects. Ruby programs will also do things like dynamically decide which files to load based on reading the directory, but if you compile an application, most of the time you want that to happen ahead of time with a few exceptions (e.g. plugins), and so similarly, if you freeze as many classes as possible at a given point, you can partially evaluate manipulation of the runtime up until that point, and treat it as mostly static afterward, and fall back to slow paths for anything you can't statically resolve the names of, and still end up with lots of optimisation potential for most of the low level code. I think a lot of the same would work for Python, and might bridge the gap between the categories of alternative implementations the author mentions with more predictability than relying on a JIT doing the right analysis. E.g. your compiler can eat least potentially guarantee under which circumstances it can statically determine that an Integer can inline the fast path if Integer is frozen so that you can in fact reason about the code.
• __mharrison__ 1y ago
  This is really cool and what I thought mojo would be. The subset of Python that is easy to use, read, and removes the dynamic nature that we don't use. Excited to see where this goes.
• ranger_danger 1y ago
  I'm still looking for a python-ish (at least the syntax) embeddable library for C that is easy to include in a project, a library that does not have its own build system with dozens of files... but I don't think one exists.
  • nylonstrung 1y ago
    RustPython seems like the fulfillment of this for Rust but unfortunately haven't seen an equivalent for C
• kgeist 1y ago
  Reminds me of Shed Skin, but no mention of it. I wonder how they compare. https://en.wikipedia.org/wiki/Shed_Skin https://en.wikipedia.org/wiki/Shed_Skin
• jszymborski 1y ago
  While obviously this would need CPython to SPy `import` support for it to displace CPython for me, it seems like you can `import` SPy to CPython, which makes it an attractive solution for implementing fast libraries where Rust is perhaps too heavy duty.
  • intalentive 1y ago
    Seems correct. See point.spy in the playground here: https://spylang.github.io/spy/ https://spylang.github.io/spy/ # - the unsafe module allows C-level direct memory access to pointers and unsafe arrays # - @struct maps directly to C structs # - most users will never have to deal with this directly: using the unsafe` module is the equivalent of writing C extensions or using Cython
• tonyplee 1y ago
  The feels like JS -> TS type of change on JS side, except that was done with transpiler (sp ? ).
• ltbarcly3 1y ago
  Statically typed. That's just Java with python syntax again, which the article tries to talk it's way out of but that's what it is.
  • Smaug123 1y ago
    This assertion is so vague as to be meaningless - say more? You're presumably not asserting that all statically typed languages are fundamentally Java, because that would be saying the patently false "C and Haskell are fundamentally Java". Are you specifically saying the article's hope that it's not "Java with Python syntax" is misplaced; if so, why?
    • ltbarcly3 1y ago
      Python, if you take away dynamic typing, is very Java like. No pointers, functions/methods always pass by reference, immutable strings, always garbage collected, etc. So if all you did was require static typing in Python, but changed nothing else, it will end up with a lot of java semantics (even if the syntax itself is different).
• ic_fly2 1y ago
  If in the end we can just have .spy on some files that have performance critical functions in them and the rest is just normal python, this could be down right amazing. We recently swapped out mypyc optimised module for a rust implementation to get a 2-6x speed up, and not having to do that would be great.
  • oofbey 1y ago
    +1 this. Also, when I read about the language features which make Python intrinsically slow, generally I think "I never use that." e.g. operator overloading meaning you need to do all these pointer dereferences just to add two numbers together. Yes, I get that pytorch and numpy rely critically on these. But it makes me wonder... Could you disable these language features on a module-by-module basis? That is, declare "in this sub-tree of the source code, there shalt be no monkey-patching, and no operator overloading" and therefore the compiler can do a better job. If anybody tries to do the disallowed stuff, then it's a RuntimeError. Would that work?
    • codethief 1y ago
      It probably would but it'd likely not be trivial. Because how do you recognize monkey-patching in a dynamically typed language? I can think of many edge cases here. That being said, you might be interested in https://docs.python.org/3/reference/datamodel.html#slots https://docs.python.org/3/reference/datamodel.html#slots
  • adsharma 1y ago
    See my other comment in the thread. I would argue that anything that uses arcane dynamic stuff in python should be renamed to .dpy and the vast majority of the commonly used constructs retain .py The issue in HN threads like this is that everyone is out to promote their own favorite language or their favorite python framework that uses dynamic stuff. The majoritarian and hacker-ethos of python don't always line up. Like Chris Lattner was saying on a recent podcast, he wrote much of Swift at home on nights/weekends over 18 months. We need someone like that do this for spy.
• intalentive 1y ago
  As a prospective user the first thing I look for is lists, list comprehension, list slices, string methods, zip, zip*, tuple destructuring, generators, dicts and sets. These data structures, their syntax, and associated methods are what make Python a joy to use. They are what make it possible to solve a puzzle in a one-liner. The examples in the playground are neat but I don’t see any of the Python bread and butter usage. I see C with Python syntax. I’d like to see Python syntax solving a problem in a pythonic way, albeit with type annotations. And then I’d like to see the compiled version run 30x-100x faster. Is that ultimately the promise of this project?
  • antocuni 1y ago
    yes (work in progress :))
    • intalentive 1y ago
      Great, thank you, I will watch with interest. The killer feature seems to be interoperability with existing CPython libraries. Best of luck!
• seg_lol 1y ago
  It is has been a hot minute and no one has mentioned ChocoPy, which is basically this project. https://chocopy.org/ https://chocopy.org/ The author invents so many terms, I don't want to uncharitably ascribe this to anything, but they should use existing terminology.
• bboynton97 1y ago
  imagine having a GDPR banner on your personal blog
• kilibe 1y ago
  How do you envision SPy's generics evolving to handle something like SQLAlchemy-style query builders, where dynamic introspection is key? Eager for the next posts on the type system and static dispatch—already forking the repo to tinker with that raytracer demo. Thanks for open-sourcing this under Anaconda's wing; it's the kind of ambitious reset Python needs.
• peter_d_sherman 1y ago
  >"During the years there have been many attempts to improve Python speed; generally they fall into two categories: Implement "full Python". To be able to support all dynamic features and be fast, they usually employ a Just In Time (JIT) compiler. Examples are PyPy, GraalPy, Pyston, and CPython's own JIT. Implement a "subset of Python" or "variant of Python", either as an Ahead of Time (AOT) or JIT compiler which is able to produce fast code. The usual approach here is to remove many (if not all) of the dynamic features which make Python hard to compile. Examples are RPython, Mypyc, Cython and Numba. The problem of "full Python" JIT compilers is that sometimes they work very well and produce huge speedups, other times they don't produce any speedup at all, or might even introduce slowdowns, or they might use too much memory, or they are slow to "warm up". The problem of the subset/variant approach is that by removing the dynamic features of Python, you end up with something which does not feel pythonic, and in which many typical and idiomatic Python patterns just don't work. You often end up with "Java with Python syntax" (nothing in particular against Java, but I hope it gives an idea of what I mean)." Isn't that interesting! You know what I'd love to see? A Python compiler that works on a function-by-function basis... That is, for each function, make the determination if a straight "clean" compilation of the function is possible -- that is, if it doesn't use special Python features that make compilation difficult/challenging/complex/impossible -- or have dependencies on other functions that do... So if you have a Python function that adds simple integer variables, let's say, then that can be converted to C or Assembler or VM or LLVM or QBE code in a straightforward way, but if a function uses objects and lambdas and higher-level/complex constructs in a way that makes compilation challenging, complex or impossible, then flag those functions (write them out in a log, so the programmer could optionally simplify them, etc.) -- those will continue to be interpreted like they always are... Yes, some or all of that infrastructure might already exist... I'm just thinking aloud... :-) >"The first problem is that Python is extremely dynamic. I'm not talking only about dynamic typing, but also about the fact that in a given Python process, the "world" is a moving target and "everything can change at any time" " (If we had a future language that supported both interpreted and compiled functions, then the interpreter should have the concept of "locked" functions and objects -- that is, once the compiler has compiled something, it's now "locked", and changes (if we want to keep it dynamic) to it mean invoking the compiler again post-change, at runtime... it would be messy, but there could be a hybrid dynamic interpreted yet JIT compiled language... but it would require runtime interaction between the interpreter and compiler... which could be messy, to say the least... and yes, some of that infrastructure might already exist...)
• rcarmo 1y ago
  …with WASM as a target. Hmmm. I would prefer it output C, or something I can link to other native code.
• nylonstrung 1y ago
  This is really promising and I hope it succeeds. It does seem like the most well conceived attempt at this yet and I think the network effects of Python (especially with LLM codegen) are so strong that something like this has to gain adoption So far the options like AOT python simply don't present enough performance improvement relative to the complexity they introduce