4 ms·
Is Go fast enough for large computations?
Let's say I basically want to generate a series of random numbers billions of times.
Would Go be a good choice for this? I keep trying to learn Rust, but it's very difficult for me.
- pantantrant 3y agoconst int randomnumber = 4; // It was picked with a dice roll.
- justanorherhack 3y agoThis is why I hn
- jjgreen 3y agoman od
- c0balt 3y ago> Let's say I basically want to generate a series of random numbers billions of times. That's a bit vague, if you only need pseudo random numbers consider a xorshift based approach, e.g., https://pkg.go.dev/github.com/db47h/rand64/xorshift https://pkg.go.dev/github.com/db47h/rand64/xorshift. Xorshift is fast and simple. > Would Go be a good choice for this? I keep trying to learn Rust, but it's very difficult for me. If you need fast but don't want to write Rust or C(++), Go is a good choice for this. It will likely also have a good amount of examples and existing code available for most common problems.
- speedgoose 3y agoYou will most likely be IO bounded.
- cpach 3y agoI don’t follow. Why do you think so?
- speedgoose 3y agoUnless you write your numbers to /dev/null, your program will wait a bit for the storage, the network, or the next program.
- machinekob 3y agoIf you are not doing AI with custom Go code or low lvl close to hardware programming -> No. Every other instance -> Yes.
- thrwwycbr 3y agoI mean, it's very hard to do and you have to build the GPU shader/processing pipeline yourself, but it's not impossible. [1] Gorgonia implemented most of the necessary libraries for tensor math for their alphago implementation. [1] https://github.com/gorgonia/agogo https://github.com/gorgonia/agogo
- bjourne 3y agoLet's do some napkin math. A modern cpu runs at 4 GHz so it runs 4 billion cycles per second. Generating a random number takes, if we are being very pessimistic, 100 cycles. So we can generate 40 million random numbers per second. There's some overhead for seeding the random number generator too, but it's amortized over the large number of random numbers generated. You want a billion number so it takes 25 seconds. If Go is ten times slower (it isn't) than good C code, the job still takes only 250 seconds.
- 999900000999 3y agoSo would you argue it's not worth it at the end of the day to switch to a faster language. I have a pretty quick solution in Go, but it's not scaling well. Original solution was in Python, which is much slower.
- bjourne 3y agoYeah. Unless your Go code truly suck you won't see that much improvement.
- holden_nelson 3y agoLink a gist of your code
- sethammons 3y agowe've replaced production python and perl systems with Go. It is faster because the concurrency story and speed of Go is faster. On average, replacing services at scale (as in, billions of networked requests per day), we saw that Go reduced our fleet footprint by 10-20x. In the best case, we went from 70+ nodes running Twisted Python to 1 node running Go, so we bumped it to 3 nodes in case it failed.
- 999900000999 3y agoWas it hard to convince your team to switch to Go ? What other languages did you consider? Do you see a positive future for Golang devs? Rust appears to of picked up a lot of its momentum. I vastly prefer Golang.
- logicalmonster 3y agoIf the problem you're trying to solve is literally generating randomness, you might look into custom hardware. I think some companies have devices to measure subtle variations in the output of tiny laser beams, as one example of how pure randomness can be quickly and continuously found.
- cpach 3y agoIs that really faster than the kernel’s CSPRNG?
- deleted 3y ago[deleted]