3 ms·

Makes me wonder about a distributed compilation. If everyone had a monster workstation but not everyone compiles at the same time, a theoretical networked compi

Makes me wonder about a distributed compilation. If everyone had a monster workstation but not everyone compiles at the same time, a theoretical networked compiler (which may already exist as I haven't really checked) could spread the files out among available workstations and bring things back together near the end. As the largest issue is the throwing away of duplicate work, I'd see it as a kind of reverse binary tree: machines working on files that depend on others talk together, then when finished send the condensed work up the chain (and signal their availability for the next workload chunk or phase) until everything collapses down back to the original machine.

fazzone 9y ago
Isn't this what IncrediBuild does on Windows?
- btschaegg 9y ago
  It is. Although, as far as I'm informed (having last used it in 2013 or so), even with that approach, you'll likely have problems linking all the output in the end if your project is large enough. Of course, thats another separate problem to begin with. I still remember dabbling with D and vibe.d and replacing the default GNU linker with ld.gold because over 90% of the build time was due to the linker...
ordu 9y ago
distcc: https://en.wikipedia.org/wiki/Distcc https://en.wikipedia.org/wiki/Distcc
moises_silva 9y ago
https://wiki.archlinux.org/index.php/Distcc https://wiki.archlinux.org/index.php/Distcc I used that 10+ years ago on Gentoo, and never saw anyone using it since. Don't know how often is used now days.
- rhaps0dy 9y ago
  I used it ~3 years ago on Funtoo; there's definitely still people that use it!
jdright 9y ago
The best solution today for distributed build is: http://www.fastbuild.org/docs/home.html http://www.fastbuild.org/docs/home.html It is faster than IncrediBuild, even faster than SN-DBS and has multi-platform support, the only problem is that it requires its own build script.
adrianratnapala 9y ago
Ok, I see distcc helping people with computer labs and server farms, but I thought mysterydip wanted to help the Gentoo community and other such. I.e. peer-to-peer build sharing. So what kind of cryptographic guarantees would you need for that? And if you can only verify the build results by trusting signatures from upon high, then what is the point? Perhaps those builds could be turned into work in a proof-of-work blockchain. Do compilers contain any hard-to-do, but easy to verify steps? Whole shelves full of useless PhDs thesis are just waiting to be written on this topic.

fazzone 9y ago

Isn't this what IncrediBuild does on Windows?

btschaegg 9y ago
It is. Although, as far as I'm informed (having last used it in 2013 or so), even with that approach, you'll likely have problems linking all the output in the end if your project is large enough. Of course, thats another separate problem to begin with. I still remember dabbling with D and vibe.d and replacing the default GNU linker with ld.gold because over 90% of the build time was due to the linker...

ordu 9y ago

distcc: https://en.wikipedia.org/wiki/Distcc https://en.wikipedia.org/wiki/Distcc

moises_silva 9y ago

https://wiki.archlinux.org/index.php/Distcc https://wiki.archlinux.org/index.php/Distcc I used that 10+ years ago on Gentoo, and never saw anyone using it since. Don't know how often is used now days.

rhaps0dy 9y ago
I used it ~3 years ago on Funtoo; there's definitely still people that use it!

jdright 9y ago

The best solution today for distributed build is: http://www.fastbuild.org/docs/home.html http://www.fastbuild.org/docs/home.html It is faster than IncrediBuild, even faster than SN-DBS and has multi-platform support, the only problem is that it requires its own build script.

adrianratnapala 9y ago

Ok, I see distcc helping people with computer labs and server farms, but I thought mysterydip wanted to help the Gentoo community and other such. I.e. peer-to-peer build sharing. So what kind of cryptographic guarantees would you need for that? And if you can only verify the build results by trusting signatures from upon high, then what is the point? Perhaps those builds could be turned into work in a proof-of-work blockchain. Do compilers contain any hard-to-do, but easy to verify steps? Whole shelves full of useless PhDs thesis are just waiting to be written on this topic.