16 ms·

The Spiral Language

329 points by Nelkins 8y ago

dangoljames 8y ago
Forth reborn?
- whitexn--g28h 8y ago
  It really reminds me of forth. The only forth program I’ve interacted with is the BIOS prompt for the olpc laptop which used forth to bootstrap the cpu from the gpu.
  - jacobush 8y ago
    When he said shallow call stack, I immediately thought of forth
blt 8y ago
show a code sample before your whole manifesto
- winter_blue 8y ago
  I actually prefer to read the philosophy of a language, before diving deeper into the language itself. It gives you an idea of the author's motivations for creating the language. I feel like a code sample would make it too easy to judge its syntax, and not tell you enough/much about its semantics.
  - agumonkey 8y ago
    The 'sample' thing is a result of the era, where people wanted to streamline language diffusion. I think it became a bit too much of a reflex for people to expect the same first piece of information about a language. I too need more than a sample and prefer a larger explanation.
    - zwkrt 8y ago
      Especially considering the syntax of this language is kind of strange. Seeing a hello world without context would put me off I think.
      - s-shellfish 8y ago
        I don't think it aligns with the philosophy of the language either, to essential chunk ideas into large discrete blocks of particular categorical intents. It seems like the intent is to distribute programming concepts on a finer granularity than most languages tend to retain flexibility for, as structure builds.
- PeCaN 8y ago
  Frankly, to me the specifics of the syntax are not nearly as important as what the language does. This: > Abstraction by heap allocation is a dead end. It works moderately well on the current generation of computers where CPU is still the dominant driver of computation. > It cannot work for devices like GPUs and the rest coming down the line. tells me way more about the language (and gets me much more interested) than a "hello world" would. I think syntax is by far the easiest part of a new language to learn, so it's ultimately not that important. EDIT: Regardless, it turns out it actually gives code samples like a page or two down.
  - garmaine 8y ago
    I don't trust that it actually achieves those goals without seeing code. Show an illustrative example of what you are trying to do and how it is accomplished, then wax philosophic.
    - coldtea 8y ago
      What he's trying to do is not conveyed in example code. That said, he gives plenty of examples.
  - ghayes 8y ago
    I just want to mention that code samples are about more than syntax. E.g. a code sample of Elixir might spawn concurrent processes, or a code sample of Idris might show type-safe vector concatenation.
  - DEADBEEFC0FFEE 8y ago
    I am not a developer beyond hobby scripts and such, and I find these threads and comments like this extremely interesting. Other people's domains are frequently fascinating.
- plaguuuuuu 8y ago
  there are heaps of samples on that page.
  - garmaine 8y ago
    Which you have to scroll past pages of manifesto to find, and all the examples seem to be basic illustrations of syntax. Where are the examples showing how this approach to a language actually results in superior code bases or more efficient programs or whatever?
    - azeirah 8y ago
      This is a one person project. This clearly took a lot of time and effort. You can afford it to look at the TOC and get scrolled directly to various examples.
      - garmaine 8y ago
        I didn’t find a single example that demonstrates what I was looking for. I gave up.
        zaphar 8y ago
        You are clearly not the target audience then.
      - deleted 8y ago
        [deleted]
    - coldtea 8y ago
      >Which you have to scroll past pages of manifesto to find Well, you are also free to skip the language altogether.
- blt 8y ago
  Yes I agree that the language semantics and the philosophical motivation are more important. But there is a reason academic papers have abstracts. The core ideas should be summarized crisply at the beginning of a technical document. A few sentences and a code sample are the equivalent of an abstract for programming languages.
  - abecedarius 8y ago
    A related-work section would really help, too.
    - abstractcontrol 8y ago
      I do not think there is a language with capabilities similar to Spiral. If it existed it would have saved me a lot of time. The closest you could get would would be [Scala LMS](https://scala-lms.github.io/ https://scala-lms.github.io/) and the work done on staging such as in [context of Ocaml](http://ocamllabs.io/iocamljs/staging2018.html http://ocamllabs.io/iocamljs/staging2018.html). Personally I think macro-style staging in statically typed languages is an atrocity and want nothing to do with it. I kind of like the type driven version as is done in LMS, but compared to Spiral the LMS is less expressive. A lot of work is being done on macros/partial evaluation/staging in Scala and I assume it will continue to be done as the authors are aware of the need for this in a language - without it a functional language would be dog slow, but my impression is that they are having significant difficulties making it fit with Scala's much more complex type system. One consequence of that is that they seemingly rewrite their macro system every few years. This time they will surely get it right despite being at it for decade(s). The reason why what Spiral has works so well is because its type system is so simple, so staging can be completely intervowen with how it does type inference. My view based on seeing the last 50 years of language design is that staging is really difficult to bolt onto an already established design. It tends to degenerate into having two languages - the core language and a scripting language on top of it. C++'s template system is the worst example of this. Another example which actually embraces this dichotomy would be [Terra](http://terralang.org/ http://terralang.org/) which could be described as bolting Lua on top of C. I rather dislike this sort of design. A language needs to be designed around staging from the ground up and it needs to be accepted that in the absence of magic, the programmer will have to mold his style to fit with the new paradigm. If a language is to have staging it should be everywhere and at all times much like regular type inference is in F# and other ML styled languages.
- DennisP 8y ago
  It does even better, it has a table of contents right at the top, so you can skip directly to whatever you want. "Intro" and "Design Philosophy" sound like manifestos, so pass over those and click on "Tutorials."
moomin 8y ago
Well, this is pretty interesting. I’ve been wondering when someone would finally put optimisation semantics into the code.
- weberc2 8y ago
  I think it's more interesting that they're in the type system. C has had inline pragmas for decades now, and one might also consider memory allocation semantics to be optimization semantics. Or more generally, one might consider any low-level semantics to be optimization semantics. The neat thing about Spiral isn't that it has 'optimization semantics', but rather that they integrate neatly into the type system.
  - moomin 8y ago
    “Inline”, though, is a request, like a hint. It doesn’t _enforce_ it. And this is my problem with most optimisations: you can’t guarantee they’ve happened. Putting them in the type system is definitely the right way of doing this. constexpr, however, does seem to be enforced.
akkartik 8y ago
My ears pricked up at "The culprit for this is the heap allocation by default dogma.." https://mastodon.social/@akkartik/100357841646414633 https://mastodon.social/@akkartik/100357841646414633 (Mastodon doesn't have in-page anchors, but search for 'memory allocator'.)
- chrismorgan 8y ago
  > document.querySelectorAll('[id]').length 0 That caught me by surprise. Especially when it has its class entry-center to indicate “this is the actively focused message”, to not load to that point in the page or at least have an ID to jump to there seems surprising. (A pure CSS solution for centering it on the page with an anchor: make the .entry-center `position: relative`, and give it a new child <div id=jump> with CSS `position: absolute; top: calc(50% - 50vh)`. The downside is that if the entire .entry-center is taller than the viewport, it will still be centered on the page, rather than sitting at the top. I suspect that defect could be solved if you used two nodes instead of just one, combining margin and padding, since padding can’t be negative.)
PeCaN 8y ago
In case anyone is wondering about why this project puts so much importance on things being inlineable, GPUs often don't support function calls or have an extremely shallow return stack. In this way it reminds me of http://futhark-lang.org/ http://futhark-lang.org/ somewhat, though Futhark is explicitly targeted at producing extremely efficient GPU code.
- ygra 8y ago
  As also described by the fourth to sixth paragraphs in the readme: Abstraction by heap allocation is a dead end. It works moderately well on the current generation of computers where CPU is still the dominant driver of computation. It cannot work for devices like GPUs and the rest coming down the line. Many of the most important computational devices of the future won't support heap allocation so an alternative is needed to draw out their full power. It is of absolute importance that a language for that task have excellent control over inlining. Inlining therefore must comes as guarantee in the language and be a part of the type system. Inlining is a trade-off that expresses the exchange of memory for computation. It should be the default instead of heap allocating. I guess whoever is wondering about the why hasn't read the introduction.
  - imtringued 8y ago
    There are multiple perspectives to this. Without heap allocation during kernel execution GPUs might not even become the dominant driver of computation in the first place.
  - xiaq 8y ago
    Function calls use stacks. It is not the same thing as heap allocation.
    - crististm 8y ago
      Function calls don't necessarily need to use stacks. There is nothing preventing frame allocations on heap.
eigenspace 8y ago
> There is no need to consider how tensors are made in a language made for numeric computation like Julia. Does the author know anything about julia or how cleanly it allows one to implement things like tensors?
- abstractcontrol 8y ago
  When I looked at Julia last time I do not think it had the capability to partially apply tensors. In Spiral indexing into a 3d tensor would give you back a 2d tensor. It might be possible to do this with function in Julia, but that would depend on its inlining capabilities and unlike Spiral it does not provide guarantees with regards to that. Its type system is definitely less powerful than Spiral's - some of the things you could express naturally in Spiral would require macros in Julia. All in all, the language strikes me as a better Matlab which I think it succeeds at, but Spiral is made to be a better F#. Some of the tradeoffs I had to make in language design means that I could not succeed at this across the board. The fact that it has a more powerful type system makes it take an immediate hit in terms of ergonomics, and the language is less pleasant to code in than F# because the IDE is not providing immediate and constant feedback. On the other hand for systems programming, I can't imagine what C (and even C++) could possibly do better than it except maybe compile times. Despite that, Spiral is also nothing ilke Rust - it does not try to check that pointers are safe. It is rather a extremely expressive static language that has no type annotations anywhere and looks like a dynamically typed functional language.
  - StefanKarpinski 8y ago
    > When I looked at Julia last time I do not think it had the capability to partially apply tensors. In Spiral indexing into a 3d tensor would give you back a 2d tensor. It might be possible to do this with function in Julia, but that would depend on its inlining capabilities and unlike Spiral it does not provide guarantees with regards to that. That's exactly how slicing a tensor works in Julia: julia> T = copy(reshape(1:2*3*4, 2, 3, 4)); julia> summary(T) "2×3×4 Array{Int64,3}" julia> T[:,2,:] 2×4 Array{Int64,2}: 3 9 15 21 4 10 16 22 Inlining seems irrelevant since the slicing is done eagerly. Unless you're talking about lazy slicing, in which case you could do `f(T) = T[:,2,:]` and your inlining/function comment makes more sense. If you use StaticArrays.jl (the closest thing in Julia to Spiral's tensors) then the machine code for this operation on a 2×3×4 Float64 tensor is just: vmovups 16(%rsi), %xmm0 vmovups 64(%rsi), %xmm1 vmovups 112(%rsi), %xmm2 vmovups 160(%rsi), %xmm3 vmovups %xmm0, (%rdi) vmovups %xmm1, 16(%rdi) vmovups %xmm2, 32(%rdi) vmovups %xmm3, 48(%rdi) movq %rdi, %rax retq > Its type system is definitely less powerful than Spiral's - some of the things you could express naturally in Spiral would require macros in Julia. The macro comment bit is a bit perplexing since macros are about syntax and are about as unrelated to the type system as it's possible to get. I'm unaware of any way that the presence of absence of macros in a language can affect the expressiveness of its type system. I'd be very curious to hear what can be expressed in Sprial's type system that cannot be in Julia's. Julia's type system is generally more expressive than state-of-the-art dependent static type systems since it punts on type checking entirely. This allows it to have, among other things, parametric types where the parameters can be other types, integers, floats, symbols and tuples of same (recursively).Parametric types even support fancy features like using an earlier type parameters to constrain later type parameters, e.g. `X{T<:Number, S<:Vector{<:T}}` — i.e. `T` is a possibly abstract number type and `S` is a vector whose element type is a subtype of `T`. The only major type system feature I'm aware of Julia lacking (aside from the obvious omission of static type checking) is intersection types. There's been some thought of adding those in order to support multiple inheritance among other things, but it hasn't proven necessary yet.
whatshisface 8y ago
Will there be any chance of a SPIRV backend becoming avalible any time soon?
- abstractcontrol 8y ago
  Author here. The language was made for Cuda programming and until something comes along to displace it, I have no plans of moving from it. That having said, the language is quite small and it would be possible to create a new backend in about 500 lines of code so if you really want it, you could probably do it yourself. I'd help you integrate it into the language. I know nothing about SPIRV at the moment though.
kgbier 8y ago
The commit messages across the entire project are... alarmingly insightful. There's some detailed stream of consciousness stuff in there.
- stochastic_monk 8y ago
  I’m quite happy to see random projections getting some love, but I hope more people start using Choromanski et al.’s 2016 Structured Orthogonal Random Features, which has provably higher accuracy while reducing runtime to linearithmic and memory to linear (or constant) from quadratic for each. I’ve verified this experimentally in my implementation here [0]. As a shameless plug, it’s quite fast, is written in C++, and comes with Python bindings for both kernel projections and orthogonal JL transforms. [0]: https://github.com/dnbaker/frp https://github.com/dnbaker/frp
  - wild_preference 8y ago
    How is this relevant to the upstream comment?
    - jpwagner 8y ago
      https://github.com/mrakgr/The-Spiral-Language/commit/d75e30403987df156e9f0de271abae5447e4e9b6 https://github.com/mrakgr/The-Spiral-Language/commit/d75e304...
  - abstractcontrol 8y ago
    The reason why I am using random projections in the latest test is because I am testing an algorithm that iteratively calculates the inverse Cholesky factor of the covariance matrix and am testing it on Mnist images. The cov matrices made from raw Mnist images are non-invertible, but projecting them to a much smaller dimension allows me to actually test the algorithm on non-synthetic data. I do not actually need more than I have, but I'll keep your link in mind if I ever need random projections though.
    - 8y ago
ghotli 8y ago
Probably the most mind bending idea I've had the pleasure of being exposed to via wayward link clicking in quite some time. I didn't burn through the entire thing but I went pretty far and the syntax primitives and the inlining by default type guarantees seem like a natural extension of language design in the history of programming languages. My apologies for getting a bit philosophical here at the end, but this kind of treatment is among my favorite things to encounter having had the privilege of being born at the cusp of the information age. During the first 50 years of the internet we get to casually encounter foundational new ways of expressing software with increasingly more guarantees and expressability from the languages themselves and reap the benefits of the plumbing and infrastructure that inevitably arises from such. Rust's compile time memory tracking borrow system comes to mind as another such set of compiler based guarantees that we have now that we just didn't have when I was a kid. tldr: Exciting times to be alive and clicking links on the internet.
- ConcernedCoder 8y ago
  I'm with you, this should be my entry in: [Ask HN: 2018 Summer Reading List?](https://news.ycombinator.com/item?id=17513576 https://news.ycombinator.com/item?id=17513576)
brianberns 8y ago
I think it's worth pointing out that the Spiral compiler seems to be written in F#, which is itself a terrific functional programming language. (And it looks like the Spiral compiler actually emits F#, which is then compiled?) The author states that Spiral "was made for the sake of making a deep learning library which was too difficult to do in F# itself." I wonder what he thinks about DeepML (http://www.deepml.net/ http://www.deepml.net/), which is a deep learning library written in F#.
- abstractcontrol 8y ago
  Before I started work on Spiral I spent way too long, like a year working on a ML library before throwing in the towel. I think I skimmed through the source of DeepML over a year ago and my impression was that the author was struggling against the limitations of F#'s type system. There were a bunch of places where he was doing the equivalent of telling the compiler to fuck itself by downcasting to `System.Object`. The way these projects go is that you start of with an AD library and then when the type system and the language starts getting in the way you either do a lot, and I mean a LOT of engineering with inferior tools or you make something better. The 'make something better' is always picked and can take various forms. Generally, people would not make a decision to work on a language for a ML library for almost a year before they write down the first line for it. They instead take the middle road of starting with an AD library and gradually extending it. First they realize that they cannot really express tensors properly with the confines of the language so they make everything symbolic, then they build engines to run such symbolic AST, then they realize that they need JITs and other optimizers to make everything run fast. Tensorflow and PyTorch are currently at this stage. There are more stages after this. Of course the task of working with ASTs is what a compiler does. I think it is a great pity that Tensorflow and PyTorch are written in C++ under the hood. C++ is probably the worst language I can imagine for working on compilers, so I applaud the authors of DeepML for picking F# instead. Had the TF team picked statically typed functional language for this they could have cut the size of TF by over 10x and saved themselves over a million lines of code. [ML](https://en.wikipedia.org/wiki/Standard_ML https://en.wikipedia.org/wiki/Standard_ML) derivatives like it were made for that sort of thing and I cannot imagine doing Spiral in a non-ML styled language.
Ono-Sendai 8y ago
If you're not allowed function calls on a GPU, why not just inline everything? Why introduce some funky keywords?
- setzer22 8y ago
  You're allowed some function calls. It just has a very short call stack.
dejawu 8y ago
As someone not particularly familiar with F#, could someone go into what the author means by "first-class types"? Do they mean that data types themselves can be treated as values (first-class citizens)?
- abstractcontrol 8y ago
  Indeed. But F# cannot actually do this, and neither can even dependently typed languages to the degree Spiral can. In Spiral types act much like values meaning you can bind them to variables and you can reflect on them without cost like with values, though if you tried something like `(type 1) + 2` you'd just get a error during the last phase of compilation. It is rather useful and without it, type inference would be impossible in Spiral.
rurban 8y ago
inl add a b = a + b inl f = add 1 f 2, f 3 is just awesome
- skrebbel 8y ago
  That's just normal partial function application, right? Or am I missing something special?
  - rurban 8y ago
    Yes, but I adore the simple syntax. inl add a b = a + b is basically a macro #define add(a,b) a+b inl f = add 1 is a macro #define f add(1) f 2, f 3 but e.g. inl f = add c would create a closure to capture c. inl would guarantee that the capturing is lexical, not dynamic. There's no need for the comma operator in args, comma is to create lists.
    - rurban 8y ago
      oops: inl f(x) = add(1,x) of course
- TurboHaskal 8y ago
  It blew my mind when I saw it on K. I just wasn't expecting it. add:{x+y} f:add 1 f 2 3
  - catwell 8y ago
    Similarly, Haskell would be: add = (+) f = add 1 f 2 This kind of syntax is great for small code samples or to play in a CLI, but in larger programs more traditional (verbose) syntax works better for me. YMMV.
    - moomin 8y ago
      The syntax, though, is part of a more important point: all functions have one input and one output. This _massively_ simplifies other things. Also, and this isn’t really an FP point, if you’ve got a lot of positional parameters you’re already Doing It Wrong.
slifin 8y ago
I wish there was a language that had the syntax of ReasonML, built on top of F# because the ecosystem of F# is much better It's very easy to get lost in syntax when transferring from C like languages like JavaScript to F#, some concessions (like Reason has) to semi colons and braces might help those programmers (myself included) to feel at home
- weberc2 8y ago
  This. I really like functional programming, and FP proponents keep telling me syntax isn't a big deal, but I'm constantly stubbing my toe on it. I guess my deferred-gratification mechanism isn't that strong, because I get bored quickly of battling syntax when I could do something interesting in an imperative language now.
ConcernedCoder 8y ago
I think the real meat, is here: https://github.com/mrakgr/The-Spiral-Language#5-tensors-and-structural-reflection https://github.com/mrakgr/The-Spiral-Language#5-tensors-and-...
- pimlottc 8y ago
  That link doesn’t work on mobile since the readme is abbreviated. Here’s a working one: https://github.com/mrakgr/The-Spiral-Language/blob/master/readme.md#5-tensors-and-structural-reflection https://github.com/mrakgr/The-Spiral-Language/blob/master/re...
arminiusreturns 8y ago
I'll admit this caused the first non-gaming related slight twinge of nostalgia for the windows ecosystem I've had in a long time since I went gnu/linux only. Good on the author.
weathererp 8y ago
It is fascinating how much a single person can achieve. Since the author seems to be aware of this thread, Marko, do you mind telling a little bit about your background? This project does not seem like something that was done on the side, and also not something that a non-expert in programming languages just decides to have a go at.
childintime 8y ago
Isn't this language a VHDL, or what a VHDL should/could be? If so, please, someone present the language from this perspective, preferably with an example.

The Spiral Language

329 points by Nelkins 8y ago

dangoljames 8y ago

Forth reborn?

whitexn--g28h 8y ago
It really reminds me of forth. The only forth program I’ve interacted with is the BIOS prompt for the olpc laptop which used forth to bootstrap the cpu from the gpu.
- jacobush 8y ago
  When he said shallow call stack, I immediately thought of forth

blt 8y ago

show a code sample before your whole manifesto

winter_blue 8y ago
I actually prefer to read the philosophy of a language, before diving deeper into the language itself. It gives you an idea of the author's motivations for creating the language. I feel like a code sample would make it too easy to judge its syntax, and not tell you enough/much about its semantics.
- agumonkey 8y ago
  The 'sample' thing is a result of the era, where people wanted to streamline language diffusion. I think it became a bit too much of a reflex for people to expect the same first piece of information about a language. I too need more than a sample and prefer a larger explanation.
  - zwkrt 8y ago
    Especially considering the syntax of this language is kind of strange. Seeing a hello world without context would put me off I think.
    - s-shellfish 8y ago
      I don't think it aligns with the philosophy of the language either, to essential chunk ideas into large discrete blocks of particular categorical intents. It seems like the intent is to distribute programming concepts on a finer granularity than most languages tend to retain flexibility for, as structure builds.
PeCaN 8y ago
Frankly, to me the specifics of the syntax are not nearly as important as what the language does. This: > Abstraction by heap allocation is a dead end. It works moderately well on the current generation of computers where CPU is still the dominant driver of computation. > It cannot work for devices like GPUs and the rest coming down the line. tells me way more about the language (and gets me much more interested) than a "hello world" would. I think syntax is by far the easiest part of a new language to learn, so it's ultimately not that important. EDIT: Regardless, it turns out it actually gives code samples like a page or two down.
- garmaine 8y ago
  I don't trust that it actually achieves those goals without seeing code. Show an illustrative example of what you are trying to do and how it is accomplished, then wax philosophic.
  - coldtea 8y ago
    What he's trying to do is not conveyed in example code. That said, he gives plenty of examples.
- ghayes 8y ago
  I just want to mention that code samples are about more than syntax. E.g. a code sample of Elixir might spawn concurrent processes, or a code sample of Idris might show type-safe vector concatenation.
- DEADBEEFC0FFEE 8y ago
  I am not a developer beyond hobby scripts and such, and I find these threads and comments like this extremely interesting. Other people's domains are frequently fascinating.
plaguuuuuu 8y ago
there are heaps of samples on that page.
- garmaine 8y ago
  Which you have to scroll past pages of manifesto to find, and all the examples seem to be basic illustrations of syntax. Where are the examples showing how this approach to a language actually results in superior code bases or more efficient programs or whatever?
  - azeirah 8y ago
    This is a one person project. This clearly took a lot of time and effort. You can afford it to look at the TOC and get scrolled directly to various examples.
    - garmaine 8y ago
      I didn’t find a single example that demonstrates what I was looking for. I gave up.
      - zaphar 8y ago
        You are clearly not the target audience then.
    - deleted 8y ago
      [deleted]
  - coldtea 8y ago
    >Which you have to scroll past pages of manifesto to find Well, you are also free to skip the language altogether.
blt 8y ago
Yes I agree that the language semantics and the philosophical motivation are more important. But there is a reason academic papers have abstracts. The core ideas should be summarized crisply at the beginning of a technical document. A few sentences and a code sample are the equivalent of an abstract for programming languages.
- abecedarius 8y ago
  A related-work section would really help, too.
  - abstractcontrol 8y ago
    I do not think there is a language with capabilities similar to Spiral. If it existed it would have saved me a lot of time. The closest you could get would would be [Scala LMS](https://scala-lms.github.io/ https://scala-lms.github.io/) and the work done on staging such as in [context of Ocaml](http://ocamllabs.io/iocamljs/staging2018.html http://ocamllabs.io/iocamljs/staging2018.html). Personally I think macro-style staging in statically typed languages is an atrocity and want nothing to do with it. I kind of like the type driven version as is done in LMS, but compared to Spiral the LMS is less expressive. A lot of work is being done on macros/partial evaluation/staging in Scala and I assume it will continue to be done as the authors are aware of the need for this in a language - without it a functional language would be dog slow, but my impression is that they are having significant difficulties making it fit with Scala's much more complex type system. One consequence of that is that they seemingly rewrite their macro system every few years. This time they will surely get it right despite being at it for decade(s). The reason why what Spiral has works so well is because its type system is so simple, so staging can be completely intervowen with how it does type inference. My view based on seeing the last 50 years of language design is that staging is really difficult to bolt onto an already established design. It tends to degenerate into having two languages - the core language and a scripting language on top of it. C++'s template system is the worst example of this. Another example which actually embraces this dichotomy would be [Terra](http://terralang.org/ http://terralang.org/) which could be described as bolting Lua on top of C. I rather dislike this sort of design. A language needs to be designed around staging from the ground up and it needs to be accepted that in the absence of magic, the programmer will have to mold his style to fit with the new paradigm. If a language is to have staging it should be everywhere and at all times much like regular type inference is in F# and other ML styled languages.
DennisP 8y ago
It does even better, it has a table of contents right at the top, so you can skip directly to whatever you want. "Intro" and "Design Philosophy" sound like manifestos, so pass over those and click on "Tutorials."

moomin 8y ago

Well, this is pretty interesting. I’ve been wondering when someone would finally put optimisation semantics into the code.

weberc2 8y ago
I think it's more interesting that they're in the type system. C has had inline pragmas for decades now, and one might also consider memory allocation semantics to be optimization semantics. Or more generally, one might consider any low-level semantics to be optimization semantics. The neat thing about Spiral isn't that it has 'optimization semantics', but rather that they integrate neatly into the type system.
- moomin 8y ago
  “Inline”, though, is a request, like a hint. It doesn’t _enforce_ it. And this is my problem with most optimisations: you can’t guarantee they’ve happened. Putting them in the type system is definitely the right way of doing this. constexpr, however, does seem to be enforced.

akkartik 8y ago

My ears pricked up at "The culprit for this is the heap allocation by default dogma.." https://mastodon.social/@akkartik/100357841646414633 https://mastodon.social/@akkartik/100357841646414633 (Mastodon doesn't have in-page anchors, but search for 'memory allocator'.)

chrismorgan 8y ago
> document.querySelectorAll('[id]').length 0 That caught me by surprise. Especially when it has its class entry-center to indicate “this is the actively focused message”, to not load to that point in the page or at least have an ID to jump to there seems surprising. (A pure CSS solution for centering it on the page with an anchor: make the .entry-center `position: relative`, and give it a new child <div id=jump> with CSS `position: absolute; top: calc(50% - 50vh)`. The downside is that if the entire .entry-center is taller than the viewport, it will still be centered on the page, rather than sitting at the top. I suspect that defect could be solved if you used two nodes instead of just one, combining margin and padding, since padding can’t be negative.)

PeCaN 8y ago

In case anyone is wondering about why this project puts so much importance on things being inlineable, GPUs often don't support function calls or have an extremely shallow return stack. In this way it reminds me of http://futhark-lang.org/ http://futhark-lang.org/ somewhat, though Futhark is explicitly targeted at producing extremely efficient GPU code.

ygra 8y ago
As also described by the fourth to sixth paragraphs in the readme: Abstraction by heap allocation is a dead end. It works moderately well on the current generation of computers where CPU is still the dominant driver of computation. It cannot work for devices like GPUs and the rest coming down the line. Many of the most important computational devices of the future won't support heap allocation so an alternative is needed to draw out their full power. It is of absolute importance that a language for that task have excellent control over inlining. Inlining therefore must comes as guarantee in the language and be a part of the type system. Inlining is a trade-off that expresses the exchange of memory for computation. It should be the default instead of heap allocating. I guess whoever is wondering about the why hasn't read the introduction.
- imtringued 8y ago
  There are multiple perspectives to this. Without heap allocation during kernel execution GPUs might not even become the dominant driver of computation in the first place.
- xiaq 8y ago
  Function calls use stacks. It is not the same thing as heap allocation.
  - crististm 8y ago
    Function calls don't necessarily need to use stacks. There is nothing preventing frame allocations on heap.

eigenspace 8y ago

> There is no need to consider how tensors are made in a language made for numeric computation like Julia. Does the author know anything about julia or how cleanly it allows one to implement things like tensors?

abstractcontrol 8y ago
When I looked at Julia last time I do not think it had the capability to partially apply tensors. In Spiral indexing into a 3d tensor would give you back a 2d tensor. It might be possible to do this with function in Julia, but that would depend on its inlining capabilities and unlike Spiral it does not provide guarantees with regards to that. Its type system is definitely less powerful than Spiral's - some of the things you could express naturally in Spiral would require macros in Julia. All in all, the language strikes me as a better Matlab which I think it succeeds at, but Spiral is made to be a better F#. Some of the tradeoffs I had to make in language design means that I could not succeed at this across the board. The fact that it has a more powerful type system makes it take an immediate hit in terms of ergonomics, and the language is less pleasant to code in than F# because the IDE is not providing immediate and constant feedback. On the other hand for systems programming, I can't imagine what C (and even C++) could possibly do better than it except maybe compile times. Despite that, Spiral is also nothing ilke Rust - it does not try to check that pointers are safe. It is rather a extremely expressive static language that has no type annotations anywhere and looks like a dynamically typed functional language.
- StefanKarpinski 8y ago
  > When I looked at Julia last time I do not think it had the capability to partially apply tensors. In Spiral indexing into a 3d tensor would give you back a 2d tensor. It might be possible to do this with function in Julia, but that would depend on its inlining capabilities and unlike Spiral it does not provide guarantees with regards to that. That's exactly how slicing a tensor works in Julia: julia> T = copy(reshape(1:2*3*4, 2, 3, 4)); julia> summary(T) "2×3×4 Array{Int64,3}" julia> T[:,2,:] 2×4 Array{Int64,2}: 3 9 15 21 4 10 16 22 Inlining seems irrelevant since the slicing is done eagerly. Unless you're talking about lazy slicing, in which case you could do `f(T) = T[:,2,:]` and your inlining/function comment makes more sense. If you use StaticArrays.jl (the closest thing in Julia to Spiral's tensors) then the machine code for this operation on a 2×3×4 Float64 tensor is just: vmovups 16(%rsi), %xmm0 vmovups 64(%rsi), %xmm1 vmovups 112(%rsi), %xmm2 vmovups 160(%rsi), %xmm3 vmovups %xmm0, (%rdi) vmovups %xmm1, 16(%rdi) vmovups %xmm2, 32(%rdi) vmovups %xmm3, 48(%rdi) movq %rdi, %rax retq > Its type system is definitely less powerful than Spiral's - some of the things you could express naturally in Spiral would require macros in Julia. The macro comment bit is a bit perplexing since macros are about syntax and are about as unrelated to the type system as it's possible to get. I'm unaware of any way that the presence of absence of macros in a language can affect the expressiveness of its type system. I'd be very curious to hear what can be expressed in Sprial's type system that cannot be in Julia's. Julia's type system is generally more expressive than state-of-the-art dependent static type systems since it punts on type checking entirely. This allows it to have, among other things, parametric types where the parameters can be other types, integers, floats, symbols and tuples of same (recursively).Parametric types even support fancy features like using an earlier type parameters to constrain later type parameters, e.g. `X{T<:Number, S<:Vector{<:T}}` — i.e. `T` is a possibly abstract number type and `S` is a vector whose element type is a subtype of `T`. The only major type system feature I'm aware of Julia lacking (aside from the obvious omission of static type checking) is intersection types. There's been some thought of adding those in order to support multiple inheritance among other things, but it hasn't proven necessary yet.

whatshisface 8y ago

Will there be any chance of a SPIRV backend becoming avalible any time soon?

abstractcontrol 8y ago
Author here. The language was made for Cuda programming and until something comes along to displace it, I have no plans of moving from it. That having said, the language is quite small and it would be possible to create a new backend in about 500 lines of code so if you really want it, you could probably do it yourself. I'd help you integrate it into the language. I know nothing about SPIRV at the moment though.

kgbier 8y ago

The commit messages across the entire project are... alarmingly insightful. There's some detailed stream of consciousness stuff in there.

stochastic_monk 8y ago
I’m quite happy to see random projections getting some love, but I hope more people start using Choromanski et al.’s 2016 Structured Orthogonal Random Features, which has provably higher accuracy while reducing runtime to linearithmic and memory to linear (or constant) from quadratic for each. I’ve verified this experimentally in my implementation here [0]. As a shameless plug, it’s quite fast, is written in C++, and comes with Python bindings for both kernel projections and orthogonal JL transforms. [0]: https://github.com/dnbaker/frp https://github.com/dnbaker/frp
- wild_preference 8y ago
  How is this relevant to the upstream comment?
  - jpwagner 8y ago
    https://github.com/mrakgr/The-Spiral-Language/commit/d75e30403987df156e9f0de271abae5447e4e9b6 https://github.com/mrakgr/The-Spiral-Language/commit/d75e304...
- abstractcontrol 8y ago
  The reason why I am using random projections in the latest test is because I am testing an algorithm that iteratively calculates the inverse Cholesky factor of the covariance matrix and am testing it on Mnist images. The cov matrices made from raw Mnist images are non-invertible, but projecting them to a much smaller dimension allows me to actually test the algorithm on non-synthetic data. I do not actually need more than I have, but I'll keep your link in mind if I ever need random projections though.
  - 8y ago

ghotli 8y ago

Probably the most mind bending idea I've had the pleasure of being exposed to via wayward link clicking in quite some time. I didn't burn through the entire thing but I went pretty far and the syntax primitives and the inlining by default type guarantees seem like a natural extension of language design in the history of programming languages. My apologies for getting a bit philosophical here at the end, but this kind of treatment is among my favorite things to encounter having had the privilege of being born at the cusp of the information age. During the first 50 years of the internet we get to casually encounter foundational new ways of expressing software with increasingly more guarantees and expressability from the languages themselves and reap the benefits of the plumbing and infrastructure that inevitably arises from such. Rust's compile time memory tracking borrow system comes to mind as another such set of compiler based guarantees that we have now that we just didn't have when I was a kid. tldr: Exciting times to be alive and clicking links on the internet.

ConcernedCoder 8y ago
I'm with you, this should be my entry in: [Ask HN: 2018 Summer Reading List?](https://news.ycombinator.com/item?id=17513576 https://news.ycombinator.com/item?id=17513576)

brianberns 8y ago

I think it's worth pointing out that the Spiral compiler seems to be written in F#, which is itself a terrific functional programming language. (And it looks like the Spiral compiler actually emits F#, which is then compiled?) The author states that Spiral "was made for the sake of making a deep learning library which was too difficult to do in F# itself." I wonder what he thinks about DeepML (http://www.deepml.net/ http://www.deepml.net/), which is a deep learning library written in F#.

abstractcontrol 8y ago
Before I started work on Spiral I spent way too long, like a year working on a ML library before throwing in the towel. I think I skimmed through the source of DeepML over a year ago and my impression was that the author was struggling against the limitations of F#'s type system. There were a bunch of places where he was doing the equivalent of telling the compiler to fuck itself by downcasting to `System.Object`. The way these projects go is that you start of with an AD library and then when the type system and the language starts getting in the way you either do a lot, and I mean a LOT of engineering with inferior tools or you make something better. The 'make something better' is always picked and can take various forms. Generally, people would not make a decision to work on a language for a ML library for almost a year before they write down the first line for it. They instead take the middle road of starting with an AD library and gradually extending it. First they realize that they cannot really express tensors properly with the confines of the language so they make everything symbolic, then they build engines to run such symbolic AST, then they realize that they need JITs and other optimizers to make everything run fast. Tensorflow and PyTorch are currently at this stage. There are more stages after this. Of course the task of working with ASTs is what a compiler does. I think it is a great pity that Tensorflow and PyTorch are written in C++ under the hood. C++ is probably the worst language I can imagine for working on compilers, so I applaud the authors of DeepML for picking F# instead. Had the TF team picked statically typed functional language for this they could have cut the size of TF by over 10x and saved themselves over a million lines of code. [ML](https://en.wikipedia.org/wiki/Standard_ML https://en.wikipedia.org/wiki/Standard_ML) derivatives like it were made for that sort of thing and I cannot imagine doing Spiral in a non-ML styled language.

Ono-Sendai 8y ago

If you're not allowed function calls on a GPU, why not just inline everything? Why introduce some funky keywords?

setzer22 8y ago
You're allowed some function calls. It just has a very short call stack.

dejawu 8y ago

As someone not particularly familiar with F#, could someone go into what the author means by "first-class types"? Do they mean that data types themselves can be treated as values (first-class citizens)?

abstractcontrol 8y ago
Indeed. But F# cannot actually do this, and neither can even dependently typed languages to the degree Spiral can. In Spiral types act much like values meaning you can bind them to variables and you can reflect on them without cost like with values, though if you tried something like `(type 1) + 2` you'd just get a error during the last phase of compilation. It is rather useful and without it, type inference would be impossible in Spiral.

rurban 8y ago

inl add a b = a + b inl f = add 1 f 2, f 3 is just awesome

skrebbel 8y ago
That's just normal partial function application, right? Or am I missing something special?
- rurban 8y ago
  Yes, but I adore the simple syntax. inl add a b = a + b is basically a macro #define add(a,b) a+b inl f = add 1 is a macro #define f add(1) f 2, f 3 but e.g. inl f = add c would create a closure to capture c. inl would guarantee that the capturing is lexical, not dynamic. There's no need for the comma operator in args, comma is to create lists.
  - rurban 8y ago
    oops: inl f(x) = add(1,x) of course
TurboHaskal 8y ago
It blew my mind when I saw it on K. I just wasn't expecting it. add:{x+y} f:add 1 f 2 3
- catwell 8y ago
  Similarly, Haskell would be: add = (+) f = add 1 f 2 This kind of syntax is great for small code samples or to play in a CLI, but in larger programs more traditional (verbose) syntax works better for me. YMMV.
  - moomin 8y ago
    The syntax, though, is part of a more important point: all functions have one input and one output. This _massively_ simplifies other things. Also, and this isn’t really an FP point, if you’ve got a lot of positional parameters you’re already Doing It Wrong.

slifin 8y ago

I wish there was a language that had the syntax of ReasonML, built on top of F# because the ecosystem of F# is much better It's very easy to get lost in syntax when transferring from C like languages like JavaScript to F#, some concessions (like Reason has) to semi colons and braces might help those programmers (myself included) to feel at home

weberc2 8y ago
This. I really like functional programming, and FP proponents keep telling me syntax isn't a big deal, but I'm constantly stubbing my toe on it. I guess my deferred-gratification mechanism isn't that strong, because I get bored quickly of battling syntax when I could do something interesting in an imperative language now.

ConcernedCoder 8y ago

I think the real meat, is here: https://github.com/mrakgr/The-Spiral-Language#5-tensors-and-structural-reflection https://github.com/mrakgr/The-Spiral-Language#5-tensors-and-...

pimlottc 8y ago
That link doesn’t work on mobile since the readme is abbreviated. Here’s a working one: https://github.com/mrakgr/The-Spiral-Language/blob/master/readme.md#5-tensors-and-structural-reflection https://github.com/mrakgr/The-Spiral-Language/blob/master/re...

arminiusreturns 8y ago

I'll admit this caused the first non-gaming related slight twinge of nostalgia for the windows ecosystem I've had in a long time since I went gnu/linux only. Good on the author.

weathererp 8y ago

It is fascinating how much a single person can achieve. Since the author seems to be aware of this thread, Marko, do you mind telling a little bit about your background? This project does not seem like something that was done on the side, and also not something that a non-expert in programming languages just decides to have a go at.

childintime 8y ago

Isn't this language a VHDL, or what a VHDL should/could be? If so, please, someone present the language from this perspective, preferably with an example.