cb321

77 ms·

811.
by cb321 6y ago
While it's true in 2004 there was still a Numeric/numarray schism, at the time I was using Pyrex (which soon evolved into Cython) to have a gradually strongly typed Python-like thing cross-compatible with the rest of the ecosystem
812.
by cb321 6y ago
@burntsushi's usage context is a highly multi-threaded single process doing many mmap()/munmap()s in parallel. There "being a good citizen" may well mean "not being so hard on your own self", but where the defi
813.
by cb321 6y ago
Playing with madvise MADV_SEQUENTIAL might also add some insight. The author of the original article here did mention different access patterns, but did not use madvise in any of her published code.
814.
by cb321 6y ago
Discussion of this exact issue over at https://lobste.rs/s/h5dfjo/why_mmap_is_faster_than_system_ca...
815.
by cb321 6y ago
Well, I got 93 ms on the same Tale Of Two Cities file, 2x slower than dga's version and 18x slower than my last 5.0 ms Nim version (done to properly avoid hanging forever if someone does a "mknod foo.txt p") mentioned elsewhe
816.
by cb321 6y ago
Beyond the above couple ideas, the next steps to make it faster would be to get rid of almost all dynamic memory allocation by having a table of byte-offset,len,count triples (or maybe add in a hashcode for a quadruple) and rolling your own
817.
by cb321 6y ago
Well, then you may also appreciate changing my Nim program from the `continue` to the final output `for` to: let mf = mopen(path) if mf == nil: stderr.write "wc: \"",path,"\" missing/irregular\n
818.
by cb321 6y ago
So, actually, this entire arc of TOCTTOU is little bit off track or at least incomplete. If someone replaces an ordinary file with a FIFO/named pipe "foo.txt" during the traversal (or simply if your recursive file tree walk
819.
by cb321 6y ago
Cool. And, yeah..mmap()ing the whole file would also auto-reject non-regular files at the OS layer fixing that TOCTTOU issue..I almost did that, but I'm sure there are various other tricks, too. I mostly thought Nim deserved to be see
820.
by cb321 6y ago
I kind of suspected that might be the reason..but you do still have a scalability problem for very large vocabularies. :-) I just did a profile and saw about 15% in strcmp in the hot cache run, but sure if it's not in RAM then IO is li
821.
by cb321 6y ago
(You could also just save the len and use that as a comparison prefix, but the hash value is less likely to yield a false positive.)
822.
by cb321 6y ago
Cool. You could probably simultaneously speed it up and simplify it by changing from linked list collision to open addressing with linear probes (but you may need a larger TSZ). Besides being more cache friendly for larger files, at the e
823.
by cb321 6y ago
Just trying to help, not criticize. As to correctness, if you change the tokenization to: while (fgets(buf, 1024, fp)) { char *c, word[1024], *w = word; for (c = buf; *c; c++) { *c = tolower(*c); if (*c >= '
824.
by cb321 6y ago
You have/had a bug in your code. The realloc() needs to multiply its second argument csz by sizeof *counts or sizeof(struct wordcount).
825.
by cb321 6y ago
FWIW, the Nim version is (no deps besides stdlib): import os, strutils, tables, heapqueue iterator topByVal*[K, V](c: Table[K, V], n = 10, min = V.low): (K, V) = var q = initHeapQueue[(V, K)]() for key, val in c: if val
826.
by cb321 6y ago
It's a fair point that sometimes software is "just done". Combining the dependency graph with version freshness may yield even more informative retention/attrition metrics. { EDIT: as in "either updated in N months
827.
by cb321 6y ago
I meant "character" as in "nature" not as in the prog.lang "character type"..I just realized that in context this might be confusing. Lol.
828.
by cb321 6y ago
I agree that sheer package count is a weak metric especially for very hyped up "can I put it on my resume" prog.langs, and in the context of "maturity" (as opposed to "popularity"). I also agree that programme
829.
by cb321 6y ago
Minor correction Re: "young age" - Nim dates back to 2006 with a first public release in 2008 and is older than Go and Go is older than Kotlin.
830.
by cb321 6y ago
It depends how good/low level a programmer you are, but it's really not that hard to spin up your own impl. I did an (incomplete) Python multiprocessing like dealio in like 100 lines of code ( https://github.com/c-
831.
by cb321 6y ago
Not that @ddragon exactly said you could not, but you can also use Nim for data science, such as https://mratsim.github.io/Arraymancer/ or https://github.com/Vindaar/nimhdf5 or many others and the
832.
by cb321 6y ago
Nim also has an emit pragma where you can just inline C code (or code for the Javascript backend or C++ backend, etc.). So, if there is some poorly optimized (for whatever reason) hot inner loop you can fix it right there, though you start
833.
by cb321 6y ago
There is a link in another of my comments here and because Nim is pretty open architecture you can roll your own basis for what you want (more than most languages) such as: https://github.com/mratsim/weave
834.
by cb321 6y ago
Unmentioned so far is the NimScript mode `nim e`..For a portable sort of alternative to Bash scripting ("sort of" from the presently much longer start-up time...).
835.
by cb321 6y ago
Also INim at the Nim-2020 conference for a REPL: https://www.youtube.com/watch?v=Qa_9vut4TzQ&list=PLxLdEZg8DR... In short, I think at least for the linked to table you mentioned, Nim does check all the boxes.
836.
by cb321 6y ago
https://news.ycombinator.com/item?id=9640384
837.
by cb321 6y ago
Go code runs much more slowly than Nim code in my experience. Nim realizes better ergonomics than Python in some ways (UFCS, command syntax, user-defined operators) and as good performance & lightweightness as C/Rust. I'm
838.
by cb321 6y ago
Rebol/Red is more like Python/Cython - able to be fast but falling back to an interpreter for dynamic things. As such there are probably more "performance footguns" (though any such thing is ultimately subjective based
839.
by cb321 6y ago
I have never seen anyone say that Rust was easier to pick up than Nim...(Famous last words, I know!)
840.
by cb321 6y ago
The default tracing Nim GC is capable of quarter-millisecond pause times while the new ARC/ORC stuff is more like single-digit microseconds: https://forum.nim-lang.org/t/5734

More ›