7 ms·

Scopes – Retargetable programming language and infrastructure

122 points by homarp 8y ago

_bxg1 8y ago
What distinguishes it from Rust and Swift?
- panic 8y ago
  There are lots of differences, but a big one is its lack of a strong distinction between compile time and run time. From "Scopes for C/C++ Users" (https://scopes.readthedocs.io/en/latest/cpp_users.html https://scopes.readthedocs.io/en/latest/cpp_users.html): > C/C++ requires programs to be built and linked before they can be executed. For this, a build system is typically employed. > In contrast, Scopes’ primary execution mode is live, which means that the compiler remains on-line while the program is running, permitting to compile additional functions on demand, and to use other services provided by the Scopes runtime library.
  - DJBunnies 8y ago
    Oh, cool, skynet has come online.
  - seanmcdirmid 8y ago
    Is that really a good use of the word live? Sounds like this is just hot loading, not even hot reloading.
    - manch23 8y ago
      Sounds more like a JIT to me.
  - stevedonovan 8y ago
    Very much how Lisp does it. Interpret, and then compile-on-the-fly, since you are inside the compiler.
  - ahaferburg 8y ago
    Note the word primary. > However, Scopes can also build object files compatible with GCC and Clang at runtime using compile-object, which makes it both suitable for classic offline compilation and as foundation for a build system.
CharlesMerriam2 8y ago
The bar for a new programming language is higher than ten years ago. Introducing a language should now include "What are the cool new things in this language?" The short list where this shines, e.g., "Go - compiles really fast and no classes". "Rust - deep safety and a novel macro system". "Python - Batteries included and easy to write." A language also needs to include some design philosophies. Sometimes its simple: "DRY"; sometimes its the tradeoffs ordered in Zen of Python. You need to say something. So, what's this new language do again?
- throwawayjava 8y ago
  From the website: "A performant systems programming language that feels like a scripting language."
- mehrdadn 8y ago
  > Introducing a language should now include "What are the cool new things in this language?" From their page: - Aims to combine the expressivity of Scheme with the convenience of Python, Lua or Javascript and the performance of C. - Compile-time resource management using view propagation, an annotation-free variation of borrow checking. - Support for live code execution as well as offline compilation of object files. - Statically typed but fully inferred type system via forward propagation. Supports closures as zero-cost abstraction. etc. I don't know how tough the crowd is nowadays but I was already sold when I read the 1st and 2nd items above.
  - jerf 8y ago
    With no direct disrespect intended towards these authors in particular, I'll believe "expressivity of Scheme with the convenience of Python, Lua or Javascript and the performance of C" when I see it. Goals aren't results. I've seen that goal a lot. Given how often I've seen it, and how I'm yet to see a result that matches it, I'm coming around to the assumption that that set of goals is probably just not possible. (I suspect Python, Javascript, etc. are fundamentally, in their very core, based on some operations and conveniences that are just fundamentally going to be slow on modern processors. The only way around it is to drop some of the conveniences. That I have seen be successful.) On the "goals aren't results" front, the second one is also one that I'd want to see how it plays out a bit more before getting too excited; it's also possible that it may not be possible at it currently exists in the author's heads.
mehrdadn 8y ago
Has anybody managed to figure out how to use compile-object to produce executables?
- mehrdadn 8y ago
  Update: I found a demo in their source code in testing/test_object.sc which you can apparently compile to a nice and tiny ~16 KiB executable via: scopes test_object.sc && clang -O3 -s -o test.exe test.o
  - jacereda 8y ago
    On Mac it's a bit less (8k) and size reports: text data bss dec hex filename 172 24 44 240 f0
giornogiovanna 8y ago
The introduction mentions "view propagation", which does borrow checking without annotations. Is this documented anywhere? Also, I wish more languages had homepages like the Go one[0]. It takes a lot of reading to get to the point where you can see some non-trivial code[1], which is frustrating. [0]: https://golang.org https://golang.org [1]: https://scopes.readthedocs.io/en/latest/tutorial.html#a-mild-breeze-of-programming https://scopes.readthedocs.io/en/latest/tutorial.html#a-mild...
- Rusky 8y ago
  There's no technical reason the Rust borrow checker couldn't run without annotations either- that's a design choice, similar to the way Rust requires function types even though it could infer them. It makes inter-function contracts clearer, which improves error messages, and avoid the need for whole-program analysis. I couldn't find anything more about "view propagation" either but I strongly suspect it's no more powerful. Probably less and/or buggier at this point, given its relative maturity.
  - giornogiovanna 8y ago
    But how would you distinguish between struct A<'a, 'b>(&'a u8, &'b u8); and struct B<'a>(&'a u8, &'a u8); without lifetime annotations?
    - sanxiyn 8y ago
      You can always assume the former(all lifetimes are different). It is strictly more general.
      - giornogiovanna 8y ago
        Thanks, so lifetimes can definitely be inferred for safe code (although just like type inference, they can lead to leaky abstractions). But aren't lifetime annotations necessary for unsafe code? How could you cast a pointer into a struct into a raw pointer and back, while preserving the lifetime, without them?
mehrdadn 8y ago
This looks awesome. Sadly when I try to run it on Windows I get: error: execution engine failed: The file was not recognized as a valid object file Anybody have any idea what's up?
- deleted 8y ago
  [deleted]
- mehrdadn 8y ago
  Update: It seems this happens if the directory that contains the bin\ directory is named "Scopes".
protomyth 8y ago
Is there some downloadable documentation?
- adamfeldman 8y ago
  https://buildmedia.readthedocs.org/media/pdf/scopes/latest/scopes.pdf https://buildmedia.readthedocs.org/media/pdf/scopes/latest/s... Accessible through this page via the hamburger menu > click “Read the docs”: https://scopes.readthedocs.io/en/latest/ https://scopes.readthedocs.io/en/latest/
buovjaga 8y ago
Devlog for Scopes and the game it was created for (NOWHERE alien life simulator): https://lritterdevlog.hashbase.io/ https://lritterdevlog.hashbase.io/
anaguib 8y ago
For some reason I read it as regrettable programming language and infrastructure
- manifestsilence 8y ago
  Haha, I read it as rechargeable. I hoped that meant that it would somehow inject new functionality to avoid the ravages of time and not eventually be looked at like Cobol is.
jonathanstrange 8y ago
Out of curiosity, the docs state that closures are supported as zero-cost abstractions. That seems to imply that the language can store environments at zero cost. How is that possible?
- jlg23 8y ago
  The author referenced[0] this paper: http://compilers.cs.uni-saarland.de/papers/lkh15_cgo.pdf http://compilers.cs.uni-saarland.de/papers/lkh15_cgo.pdf [0] https://lritterdevlog.hashbase.io/#toc8 https://lritterdevlog.hashbase.io/#toc8
- sanxiyn 8y ago
  It does not imply that. Closures are usually implemented with environments, but this is unnecessary. You can compile so that everything that would be in the environment is passed as arguments. It seems to me Scopes is implemented this way.
Confusion 8y ago
What I would be very interested in seeing in an introduction to a new language is an explanation of why the authors think other languages that intend to support a similar combination of features have failed to achieve that or have failed to get traction. These kinds of introductions always read like "we're going to combine these and these and these awesome features from all your favorite languages" and well, what's not to like? Wouldn't everyone want this? So I can't help but wonder: why didn't anyone previously succeed at creating such a language? What are you going to do differently?
- sanxiyn 8y ago
  This answer will not satisfy you, but I can answer this as someone who worked on design and implementation of multiple programming languages: Q. why didn't anyone previously succeed? A. Nobody tried. There are a LOT of decisions that go into design and implementation of programming languages. Due to combinatorial explosion, vast majority of combination was never tried. This is still true even after you group "similar" combinations. But usual standard of similarity used by many people is "How is Rust different from Go?", which is too laughable to answer. To me, Scopes seems different enough that the author should not feel any need to answer questions like yours.
  - nickpsecurity 8y ago
    I'll add to your answer that a lot of them aren't production-worthy either. That might be the packaging, lacking a standard library, not integrating with whatever is dominant in that space, no docs on basic usage, and so on. This happens with verification tools in CompSci, too. So, they need to get a combination of features to whatever is the minimum of usability the target audience expects. I also push for it to integrate well with legacy libraries with no overhead. Scope author is doing that for C libraries. As a side effect, it's a big boost on productivity since people can always lean on whatever libraries are already there.
    - dTal 8y ago
      I was gonna say - I already have a language with the expressivity of Scheme, the convenience of Python, and the performance of C - it's called "Chez Scheme". Why isn't it popular? Because A) the performant implementation was proprietary until relatively recently, and B) it's less "batteries included" and more "that drawer in your house full of batteries, most of which are dead". There's a great language in there, but it lacks the tooling and social structures for popularity.

122 points by homarp 8y ago

_bxg1 8y ago

What distinguishes it from Rust and Swift?

panic 8y ago
There are lots of differences, but a big one is its lack of a strong distinction between compile time and run time. From "Scopes for C/C++ Users" (https://scopes.readthedocs.io/en/latest/cpp_users.html https://scopes.readthedocs.io/en/latest/cpp_users.html): > C/C++ requires programs to be built and linked before they can be executed. For this, a build system is typically employed. > In contrast, Scopes’ primary execution mode is live, which means that the compiler remains on-line while the program is running, permitting to compile additional functions on demand, and to use other services provided by the Scopes runtime library.
- DJBunnies 8y ago
  Oh, cool, skynet has come online.
- seanmcdirmid 8y ago
  Is that really a good use of the word live? Sounds like this is just hot loading, not even hot reloading.
  - manch23 8y ago
    Sounds more like a JIT to me.
- stevedonovan 8y ago
  Very much how Lisp does it. Interpret, and then compile-on-the-fly, since you are inside the compiler.
- ahaferburg 8y ago
  Note the word primary. > However, Scopes can also build object files compatible with GCC and Clang at runtime using compile-object, which makes it both suitable for classic offline compilation and as foundation for a build system.

CharlesMerriam2 8y ago

The bar for a new programming language is higher than ten years ago. Introducing a language should now include "What are the cool new things in this language?" The short list where this shines, e.g., "Go - compiles really fast and no classes". "Rust - deep safety and a novel macro system". "Python - Batteries included and easy to write." A language also needs to include some design philosophies. Sometimes its simple: "DRY"; sometimes its the tradeoffs ordered in Zen of Python. You need to say something. So, what's this new language do again?

throwawayjava 8y ago
From the website: "A performant systems programming language that feels like a scripting language."
mehrdadn 8y ago
> Introducing a language should now include "What are the cool new things in this language?" From their page: - Aims to combine the expressivity of Scheme with the convenience of Python, Lua or Javascript and the performance of C. - Compile-time resource management using view propagation, an annotation-free variation of borrow checking. - Support for live code execution as well as offline compilation of object files. - Statically typed but fully inferred type system via forward propagation. Supports closures as zero-cost abstraction. etc. I don't know how tough the crowd is nowadays but I was already sold when I read the 1st and 2nd items above.
- jerf 8y ago
  With no direct disrespect intended towards these authors in particular, I'll believe "expressivity of Scheme with the convenience of Python, Lua or Javascript and the performance of C" when I see it. Goals aren't results. I've seen that goal a lot. Given how often I've seen it, and how I'm yet to see a result that matches it, I'm coming around to the assumption that that set of goals is probably just not possible. (I suspect Python, Javascript, etc. are fundamentally, in their very core, based on some operations and conveniences that are just fundamentally going to be slow on modern processors. The only way around it is to drop some of the conveniences. That I have seen be successful.) On the "goals aren't results" front, the second one is also one that I'd want to see how it plays out a bit more before getting too excited; it's also possible that it may not be possible at it currently exists in the author's heads.

mehrdadn 8y ago

Has anybody managed to figure out how to use compile-object to produce executables?

mehrdadn 8y ago
Update: I found a demo in their source code in testing/test_object.sc which you can apparently compile to a nice and tiny ~16 KiB executable via: scopes test_object.sc && clang -O3 -s -o test.exe test.o
- jacereda 8y ago
  On Mac it's a bit less (8k) and size reports: text data bss dec hex filename 172 24 44 240 f0

giornogiovanna 8y ago

The introduction mentions "view propagation", which does borrow checking without annotations. Is this documented anywhere? Also, I wish more languages had homepages like the Go one[0]. It takes a lot of reading to get to the point where you can see some non-trivial code[1], which is frustrating. [0]: https://golang.org https://golang.org [1]: https://scopes.readthedocs.io/en/latest/tutorial.html#a-mild-breeze-of-programming https://scopes.readthedocs.io/en/latest/tutorial.html#a-mild...

Rusky 8y ago
There's no technical reason the Rust borrow checker couldn't run without annotations either- that's a design choice, similar to the way Rust requires function types even though it could infer them. It makes inter-function contracts clearer, which improves error messages, and avoid the need for whole-program analysis. I couldn't find anything more about "view propagation" either but I strongly suspect it's no more powerful. Probably less and/or buggier at this point, given its relative maturity.
- giornogiovanna 8y ago
  But how would you distinguish between struct A<'a, 'b>(&'a u8, &'b u8); and struct B<'a>(&'a u8, &'a u8); without lifetime annotations?
  - sanxiyn 8y ago
    You can always assume the former(all lifetimes are different). It is strictly more general.
    - giornogiovanna 8y ago
      Thanks, so lifetimes can definitely be inferred for safe code (although just like type inference, they can lead to leaky abstractions). But aren't lifetime annotations necessary for unsafe code? How could you cast a pointer into a struct into a raw pointer and back, while preserving the lifetime, without them?

mehrdadn 8y ago

This looks awesome. Sadly when I try to run it on Windows I get: error: execution engine failed: The file was not recognized as a valid object file Anybody have any idea what's up?

deleted 8y ago
[deleted]
mehrdadn 8y ago
Update: It seems this happens if the directory that contains the bin\ directory is named "Scopes".

protomyth 8y ago

Is there some downloadable documentation?

adamfeldman 8y ago
https://buildmedia.readthedocs.org/media/pdf/scopes/latest/scopes.pdf https://buildmedia.readthedocs.org/media/pdf/scopes/latest/s... Accessible through this page via the hamburger menu > click “Read the docs”: https://scopes.readthedocs.io/en/latest/ https://scopes.readthedocs.io/en/latest/

buovjaga 8y ago

Devlog for Scopes and the game it was created for (NOWHERE alien life simulator): https://lritterdevlog.hashbase.io/ https://lritterdevlog.hashbase.io/

anaguib 8y ago

For some reason I read it as regrettable programming language and infrastructure

manifestsilence 8y ago
Haha, I read it as rechargeable. I hoped that meant that it would somehow inject new functionality to avoid the ravages of time and not eventually be looked at like Cobol is.

jonathanstrange 8y ago

Out of curiosity, the docs state that closures are supported as zero-cost abstractions. That seems to imply that the language can store environments at zero cost. How is that possible?

jlg23 8y ago
The author referenced[0] this paper: http://compilers.cs.uni-saarland.de/papers/lkh15_cgo.pdf http://compilers.cs.uni-saarland.de/papers/lkh15_cgo.pdf [0] https://lritterdevlog.hashbase.io/#toc8 https://lritterdevlog.hashbase.io/#toc8
sanxiyn 8y ago
It does not imply that. Closures are usually implemented with environments, but this is unnecessary. You can compile so that everything that would be in the environment is passed as arguments. It seems to me Scopes is implemented this way.

Confusion 8y ago

What I would be very interested in seeing in an introduction to a new language is an explanation of why the authors think other languages that intend to support a similar combination of features have failed to achieve that or have failed to get traction. These kinds of introductions always read like "we're going to combine these and these and these awesome features from all your favorite languages" and well, what's not to like? Wouldn't everyone want this? So I can't help but wonder: why didn't anyone previously succeed at creating such a language? What are you going to do differently?

sanxiyn 8y ago
This answer will not satisfy you, but I can answer this as someone who worked on design and implementation of multiple programming languages: Q. why didn't anyone previously succeed? A. Nobody tried. There are a LOT of decisions that go into design and implementation of programming languages. Due to combinatorial explosion, vast majority of combination was never tried. This is still true even after you group "similar" combinations. But usual standard of similarity used by many people is "How is Rust different from Go?", which is too laughable to answer. To me, Scopes seems different enough that the author should not feel any need to answer questions like yours.
- nickpsecurity 8y ago
  I'll add to your answer that a lot of them aren't production-worthy either. That might be the packaging, lacking a standard library, not integrating with whatever is dominant in that space, no docs on basic usage, and so on. This happens with verification tools in CompSci, too. So, they need to get a combination of features to whatever is the minimum of usability the target audience expects. I also push for it to integrate well with legacy libraries with no overhead. Scope author is doing that for C libraries. As a side effect, it's a big boost on productivity since people can always lean on whatever libraries are already there.
  - dTal 8y ago
    I was gonna say - I already have a language with the expressivity of Scheme, the convenience of Python, and the performance of C - it's called "Chez Scheme". Why isn't it popular? Because A) the performant implementation was proprietary until relatively recently, and B) it's less "batteries included" and more "that drawer in your house full of batteries, most of which are dead". There's a great language in there, but it lacks the tooling and social structures for popularity.