7 ms·

Lambdascript – A new pure functional language built on top of Python

116 points by baruchel 10y ago

edraferi 10y ago
Interesting idea. Given that you can use arbitrary Python expressions as functions, but that could pollute the functional purity, what stylistic restrictions to you expect users to self-impose?
- pmontra 10y ago
  About style, why perpetrate those double underscored __atrocities__? I'm sure there is a better design lurking around in other languages waiting to be copied. I also don't appreciate _variables. But kudos for adding tail recursion to a pythonic language. At last.
jerf 10y ago
You don't make it clear why you're posting this. If it's just for comment, that's some impressive work. The compiler is fairly short and is a good demo of how you can do this sort of thing surprisingly quickly if you know what you are doing. If you intend this as something a lot of people will use, then I'd express very serious concerns about building this on top of Python. Python is one of the slowest languages there is [1]. Putting a mismatching paradigm on top of it is going to cost you some sort of multiplicative slowdown, which could very easily be in the 5x range and could even be in the 20x-50x range. Expect another penalty in memory consumption for what is again already a bit of a memory-using language. It's not long before you've got a language on your hands that is 200-500x slower than C, and perhaps more to the point, 100x slower than Haskell itself. This will make trying to convince anyone else to use it an uphill battle, because while performance isn't everything (Python is perfectly suitable for many uses, of course), insufficient performance is a critical blocker for using it for some project, and performance this bad will be "insufficient performance" for a lot of projects. Let me be clear that I am NOT pointing this out as a "criticism" of the project. If you are doing this to learn or share some fun stuff, this is irrelevant to you, and let me re-emphasize that you're off to what looks like a good start! I mean this in the spirit of warning you about what may be your biggest problem, depending on your unspecified goals, so you can make good engineering decisions about what to look out for and what to tackle. One possible solution you could consider is rather than targeting Python AST, which I think your code is doing?, you might consider targeting the bytecode directly. While you're still going to experience a certain amount of paradigm mismatch, you might find this cuts perhaps a 2x out of the multiplicative slowdown stack, gives you more options that Python directly does, and especially in the case of PyPy, may give you the option of examining what bytecode constructs PyPy is better able to optimize and give you some wiggle room in what you emit that you can exploit. Even if this is just a learning project you may also find you learn useful things by doing this; "emitting bytecode" is a popular option nowadays, with things like LLVM. From emitting bytecode for Python you may even find it not to be too big a leap to emit native code with LLVM, too. A language that automatically works natively when code is implemented in pure lambdascript but transparently interacts with Python would start turning some heads. It's not hard to imagine potentially getting some speedups over Python. Though that's a tougher bar to clear; you've got a typesystem to think about at that point. The code is currently so small and agile it's a good time to start looking at these issues. [1]: I no longer believe "languages don't have speeds, only implementations do". After many years of trying to speed up the many dynamic scripting languages and generally hitting a wall around 10x-20x slower than C across the board, I now consider the burden of proof that languages do not at least set lower bounds on performance to be on someone who can produce a truly fast-as-C dynamic language interpreter/compiler. (Obligatory LuaJIT callout as the closest to date.)
- deleted 10y ago
  [deleted]
- vog 10y ago
  > You don't make it clear why you're posting this. It seems the submitter is also the author of that project, so I guess this is just a forgotten "Show HN:" prefix in the title.
- BuckRogers 10y ago
  Don't forget Google's Grumpy release. Grumpy= Python 2 on the Go runtime.
  - chrishacken 10y ago
    What's your point? You really want to recompile code twice just to run it?
    - jerf 10y ago
      Presumably that instead of writing a language that targets a slow language and pays a multiplicative slowdown on top of that, you could write something that targets a relatively fast language that Python is able to run in also, and pay the multiplicative price against that instead. Go is not the fastest language and I expect it to have roughly a 2x slowdown over C indefinitely, but that still puts it in the "compiled language" camp rather than the "interpreted language" camp, which tends to run 10x-40x slowdown.
      - chrishacken 10y ago
        I may have misinterpreted your comment, but I think that was my point. He could have built Lambdascript directly on top of Go or some other marginally fast language instead of building on Python.
    - BuckRogers 10y ago
      Almost everything is recompiled at least twice just to run it. Your comment is pretty ridiculous unless you had another point than the one I'm getting. What do you think executes on your CPU when you run CPython.
guitarbill 10y ago
Does anybody know a widely used and open source project using literate programming? (Edit: please include only programs expressed "as a web of ideas" that Knuth mentions - apparently otherwise it's not literate programming, just loads of inline documentation?) From what I've seen, literate programming is great in theory and worse than useless in practice. It's so far away from reality. I have had the misfortune of working on a FunnelWeb codebase, and it was one of the most incomprehensible codebases I've ever seen. Preprocessing the code is an awful step in any language. Somehow, the "not documentation" still goes out of date. There are several other problems, explained at [0] better than I can in a comment. Specifically about this project: While I enjoy writing blogs in markdown because of its constraints, reStructuredText (reST) is more powerful/expressive (by design). Because you can easily express semantics/relationships, the resulting links in the generated documentation make the "web" of source code easier to navigate - something Knuth was trying to do with literate programming. [0] http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming
- mzs 10y ago
  https://www.ctan.org/pkg/web https://www.ctan.org/pkg/web
- bpicolo 10y ago
  Literate programming, to me, feels like commenting on steroids. But that's not necessarily a good thing. A big philosophy right now, and one that I share, is that your goal should be to make code that can be easily understood without comments through expressive code. Literate programming goes in the opposite direction, and I don't think that's necessarily good, though it has it's use cases. I think data science with Jupyter notebooks are a great display of when it's good (and that's likely the best / most used form of literate programming out there), because in data science a lot of what you need to justify is the "why" a calculation is done the way it's done. Pretty graphs inline are just a bonus. For ordinary code though? Comments are a fallback for complicated bits, but I feel like they shouldn't be the norm. I want people to write code that doesn't need hand holding just to understand.
  - 10y ago
baldfat 10y ago
Maybe I am wrong but I feel like the strength of Python is writing Python and the arguable weakness of Python is it's speed and concurrency. Many of these Python built languages seem to flip it around and build on the weaknesses of Python and not build on Python's strengths.
- sevensor 10y ago
  As a Python programmer, the appeal of the Haskell family seems to be correctness -- I've come to think of Python as a "surprisingly typed" language. This particular language looks like it has Haskelly syntax without the type system. I'd be most interested in the latter; I'd be willing to trade off performance in some places for stronger ability to reason about types. I envision a pure, strongly typed language with strict evaluation, that looks like Python to the rest of my code base.
  - nkantar 10y ago
    Have you tried Python's type hints with Mypy, and so you have any thoughts on it?
    - sevensor 10y ago
      I haven't yet -- type annotations in Python 3 still don't seem to have stabilized. Although I admit I haven't looked too deeply into it, I know a bunch of new stuff landed in 3.6. The appeal of a library that adds a functional layer on top of Python is that it partitions "new stuff" from "old stuff". It seems easier to tear out and reimplement if we screw it up, versus type annotations diffusing out into the rest of our codebase.
    - chubot 10y ago
      FWIW I tried it on a relatively new 10K line project and gave up, because it doesn't play well with metaprogramming. I consider metaprogramming to be part of Python, so mypy and statically typed Python are really different languages with less power IMO. And of course the mypy author knew that -- that's why he gave it a different name. http://www.oilshell.org/blog/2016/11/30.html http://www.oilshell.org/blog/2016/11/30.html update here: http://www.oilshell.org/blog/2017/01/06.html http://www.oilshell.org/blog/2017/01/06.html I'm not claiming my experience generalizes... if you have a lot of Java-like code you can probably use mypy. Bizarrely, some people do actually have a lot of Java-like Python code.
xyproto 10y ago
Can this be used together with Python for Go? (Grumpy: https://github.com/google/grumpy https://github.com/google/grumpy)
- foxhop 10y ago
  No because: - Lambdascript runs on Python3.x - Grumpy runs on Python2.7
  - xyproto 10y ago
    Can't be hard to port to python 2x, though.
lotsoflumens 10y ago
Maybe 2017 will be the year of "functional fatigue"? I can only hope I guess ...
- nine_k 10y ago
  Once FP becomes mainstream, and all best parts of the approach are incorporated in the next great approach, then...
  - lotsoflumens 10y ago
    It looks like a tail-recursive process without termination unfortunately ... I think there are some much better ideas surfacing soon.
    - nine_k 10y ago
      A tail-recursive ("looped") process without an explicit termination is a typical server, terminated by killing the process (sorry, the impure outside world is sometimes cruel).
  - white-flame 10y ago
    Unfortunately, "going mainstream" usually seems to involve selecting and running with only the worst parts of any approach. :-P
    - nine_k 10y ago
      Cannot agree. E.g. the best parts of OOP were the ideas of data encapsulation, of modularity, of interfaces separate from implementations. These live on, whether you use method dispatch and inheritance or not.
35bge57dtjku 10y ago
Any day now that literate programming is going to hit its hockey stick curve.
nine_k 10y ago
A propos, http://coconut-lang.org/ http://coconut-lang.org/ shows a different approach, easier to integrate with existing Python codebases.
- brightsize 10y ago
  My experience is that it's a joy to use and its lambda and pipe syntax cleans up my code nicely. Coconut's TCO is handy though pattern-matching is the killer feature for me. It compiles to a Python 2,3,and PyPy-compatible code. The language puts a premium on interop with Python, so much so that Python code IS Coconut code. There's a vim plugin that yields syntax highlighting but no IDE plugins yet that I'm aware of yet.

116 points by baruchel 10y ago

Interesting idea. Given that you can use arbitrary Python expressions as functions, but that could pollute the functional purity, what stylistic restrictions to you expect users to self-impose?

pmontra 10y ago
About style, why perpetrate those double underscored __atrocities__? I'm sure there is a better design lurking around in other languages waiting to be copied. I also don't appreciate _variables. But kudos for adding tail recursion to a pythonic language. At last.

jerf 10y ago

You don't make it clear why you're posting this. If it's just for comment, that's some impressive work. The compiler is fairly short and is a good demo of how you can do this sort of thing surprisingly quickly if you know what you are doing. If you intend this as something a lot of people will use, then I'd express very serious concerns about building this on top of Python. Python is one of the slowest languages there is [1]. Putting a mismatching paradigm on top of it is going to cost you some sort of multiplicative slowdown, which could very easily be in the 5x range and could even be in the 20x-50x range. Expect another penalty in memory consumption for what is again already a bit of a memory-using language. It's not long before you've got a language on your hands that is 200-500x slower than C, and perhaps more to the point, 100x slower than Haskell itself. This will make trying to convince anyone else to use it an uphill battle, because while performance isn't everything (Python is perfectly suitable for many uses, of course), insufficient performance is a critical blocker for using it for some project, and performance this bad will be "insufficient performance" for a lot of projects. Let me be clear that I am NOT pointing this out as a "criticism" of the project. If you are doing this to learn or share some fun stuff, this is irrelevant to you, and let me re-emphasize that you're off to what looks like a good start! I mean this in the spirit of warning you about what may be your biggest problem, depending on your unspecified goals, so you can make good engineering decisions about what to look out for and what to tackle. One possible solution you could consider is rather than targeting Python AST, which I think your code is doing?, you might consider targeting the bytecode directly. While you're still going to experience a certain amount of paradigm mismatch, you might find this cuts perhaps a 2x out of the multiplicative slowdown stack, gives you more options that Python directly does, and especially in the case of PyPy, may give you the option of examining what bytecode constructs PyPy is better able to optimize and give you some wiggle room in what you emit that you can exploit. Even if this is just a learning project you may also find you learn useful things by doing this; "emitting bytecode" is a popular option nowadays, with things like LLVM. From emitting bytecode for Python you may even find it not to be too big a leap to emit native code with LLVM, too. A language that automatically works natively when code is implemented in pure lambdascript but transparently interacts with Python would start turning some heads. It's not hard to imagine potentially getting some speedups over Python. Though that's a tougher bar to clear; you've got a typesystem to think about at that point. The code is currently so small and agile it's a good time to start looking at these issues. [1]: I no longer believe "languages don't have speeds, only implementations do". After many years of trying to speed up the many dynamic scripting languages and generally hitting a wall around 10x-20x slower than C across the board, I now consider the burden of proof that languages do not at least set lower bounds on performance to be on someone who can produce a truly fast-as-C dynamic language interpreter/compiler. (Obligatory LuaJIT callout as the closest to date.)

deleted 10y ago
[deleted]
vog 10y ago
> You don't make it clear why you're posting this. It seems the submitter is also the author of that project, so I guess this is just a forgotten "Show HN:" prefix in the title.
BuckRogers 10y ago
Don't forget Google's Grumpy release. Grumpy= Python 2 on the Go runtime.
- chrishacken 10y ago
  What's your point? You really want to recompile code twice just to run it?
  - jerf 10y ago
    Presumably that instead of writing a language that targets a slow language and pays a multiplicative slowdown on top of that, you could write something that targets a relatively fast language that Python is able to run in also, and pay the multiplicative price against that instead. Go is not the fastest language and I expect it to have roughly a 2x slowdown over C indefinitely, but that still puts it in the "compiled language" camp rather than the "interpreted language" camp, which tends to run 10x-40x slowdown.
    - chrishacken 10y ago
      I may have misinterpreted your comment, but I think that was my point. He could have built Lambdascript directly on top of Go or some other marginally fast language instead of building on Python.
  - BuckRogers 10y ago
    Almost everything is recompiled at least twice just to run it. Your comment is pretty ridiculous unless you had another point than the one I'm getting. What do you think executes on your CPU when you run CPython.

guitarbill 10y ago

Does anybody know a widely used and open source project using literate programming? (Edit: please include only programs expressed "as a web of ideas" that Knuth mentions - apparently otherwise it's not literate programming, just loads of inline documentation?) From what I've seen, literate programming is great in theory and worse than useless in practice. It's so far away from reality. I have had the misfortune of working on a FunnelWeb codebase, and it was one of the most incomprehensible codebases I've ever seen. Preprocessing the code is an awful step in any language. Somehow, the "not documentation" still goes out of date. There are several other problems, explained at [0] better than I can in a comment. Specifically about this project: While I enjoy writing blogs in markdown because of its constraints, reStructuredText (reST) is more powerful/expressive (by design). Because you can easily express semantics/relationships, the resulting links in the generated documentation make the "web" of source code easier to navigate - something Knuth was trying to do with literate programming. [0] http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming

mzs 10y ago
https://www.ctan.org/pkg/web https://www.ctan.org/pkg/web
bpicolo 10y ago
Literate programming, to me, feels like commenting on steroids. But that's not necessarily a good thing. A big philosophy right now, and one that I share, is that your goal should be to make code that can be easily understood without comments through expressive code. Literate programming goes in the opposite direction, and I don't think that's necessarily good, though it has it's use cases. I think data science with Jupyter notebooks are a great display of when it's good (and that's likely the best / most used form of literate programming out there), because in data science a lot of what you need to justify is the "why" a calculation is done the way it's done. Pretty graphs inline are just a bonus. For ordinary code though? Comments are a fallback for complicated bits, but I feel like they shouldn't be the norm. I want people to write code that doesn't need hand holding just to understand.
- 10y ago

baldfat 10y ago

Maybe I am wrong but I feel like the strength of Python is writing Python and the arguable weakness of Python is it's speed and concurrency. Many of these Python built languages seem to flip it around and build on the weaknesses of Python and not build on Python's strengths.

sevensor 10y ago
As a Python programmer, the appeal of the Haskell family seems to be correctness -- I've come to think of Python as a "surprisingly typed" language. This particular language looks like it has Haskelly syntax without the type system. I'd be most interested in the latter; I'd be willing to trade off performance in some places for stronger ability to reason about types. I envision a pure, strongly typed language with strict evaluation, that looks like Python to the rest of my code base.
- nkantar 10y ago
  Have you tried Python's type hints with Mypy, and so you have any thoughts on it?
  - sevensor 10y ago
    I haven't yet -- type annotations in Python 3 still don't seem to have stabilized. Although I admit I haven't looked too deeply into it, I know a bunch of new stuff landed in 3.6. The appeal of a library that adds a functional layer on top of Python is that it partitions "new stuff" from "old stuff". It seems easier to tear out and reimplement if we screw it up, versus type annotations diffusing out into the rest of our codebase.
  - chubot 10y ago
    FWIW I tried it on a relatively new 10K line project and gave up, because it doesn't play well with metaprogramming. I consider metaprogramming to be part of Python, so mypy and statically typed Python are really different languages with less power IMO. And of course the mypy author knew that -- that's why he gave it a different name. http://www.oilshell.org/blog/2016/11/30.html http://www.oilshell.org/blog/2016/11/30.html update here: http://www.oilshell.org/blog/2017/01/06.html http://www.oilshell.org/blog/2017/01/06.html I'm not claiming my experience generalizes... if you have a lot of Java-like code you can probably use mypy. Bizarrely, some people do actually have a lot of Java-like Python code.

xyproto 10y ago

Can this be used together with Python for Go? (Grumpy: https://github.com/google/grumpy https://github.com/google/grumpy)

foxhop 10y ago
No because: - Lambdascript runs on Python3.x - Grumpy runs on Python2.7
- xyproto 10y ago
  Can't be hard to port to python 2x, though.

lotsoflumens 10y ago

Maybe 2017 will be the year of "functional fatigue"? I can only hope I guess ...

nine_k 10y ago
Once FP becomes mainstream, and all best parts of the approach are incorporated in the next great approach, then...
- lotsoflumens 10y ago
  It looks like a tail-recursive process without termination unfortunately ... I think there are some much better ideas surfacing soon.
  - nine_k 10y ago
    A tail-recursive ("looped") process without an explicit termination is a typical server, terminated by killing the process (sorry, the impure outside world is sometimes cruel).
- white-flame 10y ago
  Unfortunately, "going mainstream" usually seems to involve selecting and running with only the worst parts of any approach. :-P
  - nine_k 10y ago
    Cannot agree. E.g. the best parts of OOP were the ideas of data encapsulation, of modularity, of interfaces separate from implementations. These live on, whether you use method dispatch and inheritance or not.

35bge57dtjku 10y ago

Any day now that literate programming is going to hit its hockey stick curve.

nine_k 10y ago

A propos, http://coconut-lang.org/ http://coconut-lang.org/ shows a different approach, easier to integrate with existing Python codebases.

brightsize 10y ago
My experience is that it's a joy to use and its lambda and pipe syntax cleans up my code nicely. Coconut's TCO is handy though pattern-matching is the killer feature for me. It compiles to a Python 2,3,and PyPy-compatible code. The language puts a premium on interop with Python, so much so that Python code IS Coconut code. There's a vim plugin that yields syntax highlighting but no IDE plugins yet that I'm aware of yet.