3 ms·

A few months ago. First time to plot is noticeably better than it was a few versions ago, but still extremely slow compared to Python. The article gives a bench

by saiojd 6y ago

A few months ago. First time to plot is noticeably better than it was a few versions ago, but still extremely slow compared to Python. The article gives a benchmark of 9 seconds. I mean, come on. The main problem I had was simply that what any time I needed to modify a struct field, or anytime my program crashed, or any time the buggy IDE extension crashed, I needed to recompile everything. I also haven't found anyone particularly interested in basic things like interfaces, despite the language supporting type hierarchies (why can't we enforce contracts for types? The whole language is built around overloading...) Overall this is very frustrating as the language is excellent is many regards, in particular multiple dispatch and the compilation model are just great. The "just ahed of time" compilation is one of those obvious-in-hindsight ideas IMO, better than full interpretation or full compilation for nearly all use cases, if only it could be cached between interpreter sessions or if you didn't need to restart all the time...

Sukera 6y ago
I know that there are plans to cache even more, but other than that I can basically only recommend to put distinct projects into proper projects (with a module and Project.toml). That will already cache precompiled code for that module, even between sessions. Having things in a script won't have that benefit. For experimenting with struct layouts, I've found that NamedTuples (https://docs.julialang.org/en/v1/base/base/#Core.NamedTuple https://docs.julialang.org/en/v1/base/base/#Core.NamedTuple) are amazing for prototyping, since they can be accessed via A.b just like structs but don't have the limitation of being const global. The dynamism and flexibility combined with the compilation model is basically what leads down this path of recompilation, unfortunately. Since importing packages may change behaviour/invalidate some compiled method (that's what the SnoopCompile stuff in the article was about), it's nontrivial to just begin caching things left and right. You'd end up with an exponential explosion in the number of methods to cache, wasting huge amounts of disk space. That's not to say that there aren't more things that could be done, just that it's hard to do so.
- saiojd 6y ago
  I've read a bit on type invalidation and I know it's a hard problem (in fact it's hard for me to even wrap my head around it, lol). Still, it's unfortunate. One thing I would like to know is if the difficulties with invalidation are a symptom of the dynamic semantics, or of the compilation model. Namedtuples are cool, but I'm not sure I understand the tradeoffs between using them and using structs. Can I just replace all structs in my project with named tuples, without having a performance hit?
  - Sukera 6y ago
    Note that I didn't suggest replacing structs with NamedTuples entirely - only during prototyping, while you're figuring out what you want your struct to look like. Structs most definitely will be faster.
    - oxinabox 6y ago
      > Structs most definitely will be faster. I am not 100% sure this is true. Structs will definately look cleaner in the code. Not sure they will be faster though.
      - saiojd 6y ago
        Do you know what the differences end up being when it comes to compilation? (not a rhetorical question - I'd like to know)
        oxinabox 6y ago
        Run `@code_typed` and `@code_llvm` and find out?
    - saiojd 6y ago
      I mean, I could, it's just that its pretty hard to know in advance which structs I will have to modify... Most of the time I only need to do minor edits like add 1 field. By that point I need to recompile anyway if I am to switch to NamedTuples...
DNF2 6y ago
No one interested in interfaces? My impression is that interfaces is very commonly discussed, and is one of the most anticipated features in the language, though it may not come until v2.0.
- saiojd 6y ago
  OK, I was being hyperbolic. It's just that the interest is low compared to other features, as most people involved in the project are used to dynamic languages and don't feel the need. It's been in discussion for a long time, with no action so far. From what I've seen, the general attitude is a bit dismissive of the utility of static verification ("its different in our language because X" type attitude)
  - Certhas 6y ago
    My impression with this is that it's mostly coming from a few academics that never work on code they haven't written themselves, and that have no understanding of use-cases other than their own...
dTal 6y ago
> The "just ahed of time" compilation is one of those obvious-in-hindsight ideas IMO It's not new. One of the most widely used Lisp environments, SBCL, works this way. So does Chez Scheme, and therefore now Racket.
- saiojd 6y ago
  That's interesting, I didn't know Racket worked like that. What I meant wasn't so much that its a new idea, rather that it's a good one.
Certhas 6y ago
I feel your pain on interfaces. As it stands, Julia simply doesn't encourage carefully thinking/documenting about what assumptions your code makes and just banging things together, hoping they work. Good luck dealing with any obscure MethodErros that result if they don't. It still, at the end of the day, is a mostly academic language. So mostly small projects with very few people working on them. No need to architect bigger solutions/patterns, etc...
- BadInformatics 6y ago
  I'm not sure that's a fair characterization. Core team members have expressed serious interest in getting more static verification, interfaces and other type goodness into the language, but if you try to force the issue it turns into a Python 2->3 problem. Not to mention that there are few if any examples of how to fit type checking alongside multiple dispatch (e.g. C# punts with dynamic). I personally find static types indispensable when working with a large codebase, but to say people don't care about this in the Julia community is just not correct.
  - Certhas 6y ago
    One thing is the Core Team Members and their long term plans, which I'm not privy to. The other is the impression generated by interacting with others in the community online, where people disagree with the very premise that there is a problem. I also see your point that not too much is known in this design space, but I also think that's why it would be good for the community to step up and experiment with this more. Figure out what works. I had plans to do that last summer but life intervened so I am stuck commentating from the sidelines. :P
    - adgjlsfhk1 6y ago
      Note that if you ask on slack, you often will get answers from the core team members. They're pretty open. The main place where plans aren't the most clear is when there are features that everyone knows would be good, but aren't on the top of any of the the main people's to-do list. The JuliaLang repo has over 1000 people who have contributed, so a lot of the time, a new feature is just the result of a community member making a random PR. Stack trace improvements for example, started as a package, got ported to Base, and got improved by a bunch of people contributing to design decisions.

by saiojd 6y ago

Sukera 6y ago

I know that there are plans to cache even more, but other than that I can basically only recommend to put distinct projects into proper projects (with a module and Project.toml). That will already cache precompiled code for that module, even between sessions. Having things in a script won't have that benefit. For experimenting with struct layouts, I've found that NamedTuples (https://docs.julialang.org/en/v1/base/base/#Core.NamedTuple https://docs.julialang.org/en/v1/base/base/#Core.NamedTuple) are amazing for prototyping, since they can be accessed via A.b just like structs but don't have the limitation of being const global. The dynamism and flexibility combined with the compilation model is basically what leads down this path of recompilation, unfortunately. Since importing packages may change behaviour/invalidate some compiled method (that's what the SnoopCompile stuff in the article was about), it's nontrivial to just begin caching things left and right. You'd end up with an exponential explosion in the number of methods to cache, wasting huge amounts of disk space. That's not to say that there aren't more things that could be done, just that it's hard to do so.

saiojd 6y ago
I've read a bit on type invalidation and I know it's a hard problem (in fact it's hard for me to even wrap my head around it, lol). Still, it's unfortunate. One thing I would like to know is if the difficulties with invalidation are a symptom of the dynamic semantics, or of the compilation model. Namedtuples are cool, but I'm not sure I understand the tradeoffs between using them and using structs. Can I just replace all structs in my project with named tuples, without having a performance hit?
- Sukera 6y ago
  Note that I didn't suggest replacing structs with NamedTuples entirely - only during prototyping, while you're figuring out what you want your struct to look like. Structs most definitely will be faster.
  - oxinabox 6y ago
    > Structs most definitely will be faster. I am not 100% sure this is true. Structs will definately look cleaner in the code. Not sure they will be faster though.
    - saiojd 6y ago
      Do you know what the differences end up being when it comes to compilation? (not a rhetorical question - I'd like to know)
      - oxinabox 6y ago
        Run `@code_typed` and `@code_llvm` and find out?
  - saiojd 6y ago
    I mean, I could, it's just that its pretty hard to know in advance which structs I will have to modify... Most of the time I only need to do minor edits like add 1 field. By that point I need to recompile anyway if I am to switch to NamedTuples...

DNF2 6y ago

No one interested in interfaces? My impression is that interfaces is very commonly discussed, and is one of the most anticipated features in the language, though it may not come until v2.0.

saiojd 6y ago
OK, I was being hyperbolic. It's just that the interest is low compared to other features, as most people involved in the project are used to dynamic languages and don't feel the need. It's been in discussion for a long time, with no action so far. From what I've seen, the general attitude is a bit dismissive of the utility of static verification ("its different in our language because X" type attitude)
- Certhas 6y ago
  My impression with this is that it's mostly coming from a few academics that never work on code they haven't written themselves, and that have no understanding of use-cases other than their own...

dTal 6y ago

> The "just ahed of time" compilation is one of those obvious-in-hindsight ideas IMO It's not new. One of the most widely used Lisp environments, SBCL, works this way. So does Chez Scheme, and therefore now Racket.

saiojd 6y ago
That's interesting, I didn't know Racket worked like that. What I meant wasn't so much that its a new idea, rather that it's a good one.

Certhas 6y ago

I feel your pain on interfaces. As it stands, Julia simply doesn't encourage carefully thinking/documenting about what assumptions your code makes and just banging things together, hoping they work. Good luck dealing with any obscure MethodErros that result if they don't. It still, at the end of the day, is a mostly academic language. So mostly small projects with very few people working on them. No need to architect bigger solutions/patterns, etc...

BadInformatics 6y ago
I'm not sure that's a fair characterization. Core team members have expressed serious interest in getting more static verification, interfaces and other type goodness into the language, but if you try to force the issue it turns into a Python 2->3 problem. Not to mention that there are few if any examples of how to fit type checking alongside multiple dispatch (e.g. C# punts with dynamic). I personally find static types indispensable when working with a large codebase, but to say people don't care about this in the Julia community is just not correct.
- Certhas 6y ago
  One thing is the Core Team Members and their long term plans, which I'm not privy to. The other is the impression generated by interacting with others in the community online, where people disagree with the very premise that there is a problem. I also see your point that not too much is known in this design space, but I also think that's why it would be good for the community to step up and experiment with this more. Figure out what works. I had plans to do that last summer but life intervened so I am stuck commentating from the sidelines. :P
  - adgjlsfhk1 6y ago
    Note that if you ask on slack, you often will get answers from the core team members. They're pretty open. The main place where plans aren't the most clear is when there are features that everyone knows would be good, but aren't on the top of any of the the main people's to-do list. The JuliaLang repo has over 1000 people who have contributed, so a lot of the time, a new feature is just the result of a community member making a random PR. Stack trace improvements for example, started as a package, got ported to Base, and got improved by a bunch of people contributing to design decisions.