4 ms·

Also, a reminder that JS engines do regularly have to contend with enormous JSON and do not generally display pathological behavior for large literals. I'm not

by titzer 3y ago

Also, a reminder that JS engines do regularly have to contend with enormous JSON and do not generally display pathological behavior for large literals. I'm not sure who posted that comment, but if they are on the Swift team and blaming users, that should stop. Code like that is out there, and it's the job of tooling to handle it gracefully.

refulgentis 3y ago
It wasn’t a Swift team member, Swift community has a quick response for these cases because it’s been around forever and is unique enough of a “simple” case that it, ex. ends up at #1 on HN. My understanding of Swift’s teams thoughts, though a few years out of date, is it’s very very hard to get the Swift type inference engine to handle a huge blob, without type hinting, or with literals that could be multiple types.
- deleted 3y ago
  [deleted]
- earthnail 3y ago
  Three years ago I tried putting constants for a matrix (I think it was 1024x80 or sth like that) into a strongly typed array of floats. Still took the compiler 30mins to compile it. In the end I split the matrix up in rows, with each row being one variable, and a final variable to concatenate the rows. No typing hint whatsoever helped. Big wtf moment.
  - refulgentis 3y ago
    Yeah this one's real strange to me. I feel like it used to be as simple as the [int] definition people suggest in the thread. I don't miss Xcode too much. IIRC I built some custom script using arcanda dropped by Apple employees to output per-function build times. It was great! I knew exactly what functions were slow. But now I am old and lazy lol.
- euiq 3y ago
  Note that someone who seems to manage a Swift team at Apple left a more encouraging response later: <https://forums.swift.org/t/why-is-swift-so-slow-timeout-in-compiling-this-code/61382/5 https://forums.swift.org/t/why-is-swift-so-slow-timeout-in-c...>
turnsout 3y ago
Right, but unlike Swift, the type system in JS is not doing computationally expensive type checking for each element. This is the equivalent of creating 1M DOM trees in JS and expecting it to be responsive. Rather than trying to force Swift to create fixed dimension arrays, the uncomfortable part about Swift for control freaks is that you should just make the array dynamic, and trust that the compiler is smart. It's not blaming the user, but the compiler team has to balance functionality ("Why can't the type system give me a better error?") and speed. There's no way they can handle every pathological edge case gracefully.
- titzer 3y ago
  > computationally expensive type checking So, an hour's worth of compilation time = 3600 seconds, divided by a million elements, that's 3.6 ms, or about 10 million instructions per element. Just what in the heck is this compiler doing with 10 million instructions on an integer literal? Like, it's never seen integer literals before? And they're all integers. Every element. It's not a "pathological edge case". The whole thread is absurd. People are speaking up and blaming programmers, the language, the type system, when this is so obviously the compiler's fault.
  - stephc_int13 3y ago
    We have absurdly fast hardware these days. I can barely believe that a single thread, copy pasted quicksort can sort 1M elements in less than 50ms on my computer. Fast compilers such as TCC should be frequently used as a reality check for compiler teams. Of course the constraints are not the same but still...
  - turnsout 3y ago
    I have no idea what's going on under the hood, but I'm sure it's not a linear time process, and all it takes is one of those elements to be "109.1" to flip 1M elements from Ints to Doubles. Yes, they could special-case this (and arrays of Doubles, and Strings), but where do you draw the line? That's now extra code to maintain in the compiler just to support a behavior that should be discouraged. If you need to read in 1M integers, put them in a file and read them in.
jitl 3y ago
> JS engines do regularly have to contend with enormous JSON and do not generally display pathological behavior for large literals Actually it’s common advice to replace very large literals with `JSON.parse(“…”)` because it’s faster according to Chrome engineers [1]. At Notion we did so for our emoji unicode tables for a noticeable time-to-interactive improvement over the large literal. [1]: https://youtu.be/ff4fgQxPaO0 https://youtu.be/ff4fgQxPaO0
- derefr 3y ago
  Right, but that's a linear speedup (because the JSON parser is less complex than the JS parser), not a change from superlinear to linear parse time.

by titzer 3y ago

refulgentis 3y ago

It wasn’t a Swift team member, Swift community has a quick response for these cases because it’s been around forever and is unique enough of a “simple” case that it, ex. ends up at #1 on HN. My understanding of Swift’s teams thoughts, though a few years out of date, is it’s very very hard to get the Swift type inference engine to handle a huge blob, without type hinting, or with literals that could be multiple types.

deleted 3y ago
[deleted]
earthnail 3y ago
Three years ago I tried putting constants for a matrix (I think it was 1024x80 or sth like that) into a strongly typed array of floats. Still took the compiler 30mins to compile it. In the end I split the matrix up in rows, with each row being one variable, and a final variable to concatenate the rows. No typing hint whatsoever helped. Big wtf moment.
- refulgentis 3y ago
  Yeah this one's real strange to me. I feel like it used to be as simple as the [int] definition people suggest in the thread. I don't miss Xcode too much. IIRC I built some custom script using arcanda dropped by Apple employees to output per-function build times. It was great! I knew exactly what functions were slow. But now I am old and lazy lol.
euiq 3y ago
Note that someone who seems to manage a Swift team at Apple left a more encouraging response later: <https://forums.swift.org/t/why-is-swift-so-slow-timeout-in-compiling-this-code/61382/5 https://forums.swift.org/t/why-is-swift-so-slow-timeout-in-c...>

turnsout 3y ago

Right, but unlike Swift, the type system in JS is not doing computationally expensive type checking for each element. This is the equivalent of creating 1M DOM trees in JS and expecting it to be responsive. Rather than trying to force Swift to create fixed dimension arrays, the uncomfortable part about Swift for control freaks is that you should just make the array dynamic, and trust that the compiler is smart. It's not blaming the user, but the compiler team has to balance functionality ("Why can't the type system give me a better error?") and speed. There's no way they can handle every pathological edge case gracefully.

titzer 3y ago
> computationally expensive type checking So, an hour's worth of compilation time = 3600 seconds, divided by a million elements, that's 3.6 ms, or about 10 million instructions per element. Just what in the heck is this compiler doing with 10 million instructions on an integer literal? Like, it's never seen integer literals before? And they're all integers. Every element. It's not a "pathological edge case". The whole thread is absurd. People are speaking up and blaming programmers, the language, the type system, when this is so obviously the compiler's fault.
- stephc_int13 3y ago
  We have absurdly fast hardware these days. I can barely believe that a single thread, copy pasted quicksort can sort 1M elements in less than 50ms on my computer. Fast compilers such as TCC should be frequently used as a reality check for compiler teams. Of course the constraints are not the same but still...
- turnsout 3y ago
  I have no idea what's going on under the hood, but I'm sure it's not a linear time process, and all it takes is one of those elements to be "109.1" to flip 1M elements from Ints to Doubles. Yes, they could special-case this (and arrays of Doubles, and Strings), but where do you draw the line? That's now extra code to maintain in the compiler just to support a behavior that should be discouraged. If you need to read in 1M integers, put them in a file and read them in.

jitl 3y ago

> JS engines do regularly have to contend with enormous JSON and do not generally display pathological behavior for large literals Actually it’s common advice to replace very large literals with `JSON.parse(“…”)` because it’s faster according to Chrome engineers [1]. At Notion we did so for our emoji unicode tables for a noticeable time-to-interactive improvement over the large literal. [1]: https://youtu.be/ff4fgQxPaO0 https://youtu.be/ff4fgQxPaO0

derefr 3y ago
Right, but that's a linear speedup (because the JSON parser is less complex than the JS parser), not a change from superlinear to linear parse time.