3 ms·

Probably biased given that it's on the DuckDB site, but well-reasoned and referenced, and my gut agrees with the overall philosophy This feels like the kicker:

Probably biased given that it's on the DuckDB site, but well-reasoned and referenced, and my gut agrees with the overall philosophy This feels like the kicker: > In the cloud, you don’t need to pay extra for a “big iron” machine because you’re already running on one. You just need a bigger slice. Cloud vendors don’t charge proportionally more for a larger slice, so your cost per unit of compute doesn’t change if you’re working on a tiny instance or a giant one. It's obvious once you think about it: you aren't choosing between a bunch of small machines and one big machine, you may very well be choosing between a bunch of small slices of a big machine and one big slice of a big machine. The only difference would be in how your software sees it: as a complex distributed system, or as a single system (that can eg. share memory with itself instead of serializing and deserializing data over network sockets)

LeifCarrotson 3y ago
The reason this feels non-obvious is that people like to think that they're choosing a variable number of small slices of a big datacenter, scaling up and down hour-by-hour or minute-by-minute to get maximum efficiency. Really, though, you're generating enormous overhead while turning on and off small slices of a 128-core monster with a terabyte of RAM.
JohnMakin 3y ago
That's not the only difference - there are many more facets of reliability guarantees than the brief hand-waving this author does about it in the article.
- ryguyrg 3y ago
  (i work at motherduck) over my career i’ve seen countless cases where the complexity of a distributed system introduces as many (or more) reliability concerns as it solves. when everything works well, it can be beautiful. but when things go wrong, they can quickly become a cascading mess of hard-to-debug issues.
pottertheotter 3y ago
FYI, this is from the MotherDuck blog. MotherDuck and DuckDB have a partnership, but they are separate entities. > DuckDB Labs from its inception has received many investment offers from venture capitalists, but eventually decided not to take venture capital, and rather opted for a business model where it works with a few important partners. “We look forward to the continued growth and adoption that partnering with MotherDuck will bring. Thanks to the MotherDuck team, we are able to focus our efforts on independent innovation of the DuckDB core platform and growth of the open-source community.” says Mühleisen, who currently combines a tenured research position at CWI with the role of CEO at DuckDB Labs. https://www.cwi.nl/en/news/cwi-spin-off-duckdb-labs-partners-with-motherduck-who-raises-47-5-million-1/ https://www.cwi.nl/en/news/cwi-spin-off-duckdb-labs-partners... https://duckdblabs.com/news/2022/11/15/motherduck-partnership.html https://duckdblabs.com/news/2022/11/15/motherduck-partnershi... https://www.techtarget.com/searchdatamanagement/news/252527260/MotherDuck-raises-475M-for-open-source-DuckDB-database https://www.techtarget.com/searchdatamanagement/news/2525272...

LeifCarrotson 3y ago

The reason this feels non-obvious is that people like to think that they're choosing a variable number of small slices of a big datacenter, scaling up and down hour-by-hour or minute-by-minute to get maximum efficiency. Really, though, you're generating enormous overhead while turning on and off small slices of a 128-core monster with a terabyte of RAM.

JohnMakin 3y ago

That's not the only difference - there are many more facets of reliability guarantees than the brief hand-waving this author does about it in the article.

ryguyrg 3y ago
(i work at motherduck) over my career i’ve seen countless cases where the complexity of a distributed system introduces as many (or more) reliability concerns as it solves. when everything works well, it can be beautiful. but when things go wrong, they can quickly become a cascading mess of hard-to-debug issues.

pottertheotter 3y ago

FYI, this is from the MotherDuck blog. MotherDuck and DuckDB have a partnership, but they are separate entities. > DuckDB Labs from its inception has received many investment offers from venture capitalists, but eventually decided not to take venture capital, and rather opted for a business model where it works with a few important partners. “We look forward to the continued growth and adoption that partnering with MotherDuck will bring. Thanks to the MotherDuck team, we are able to focus our efforts on independent innovation of the DuckDB core platform and growth of the open-source community.” says Mühleisen, who currently combines a tenured research position at CWI with the role of CEO at DuckDB Labs. https://www.cwi.nl/en/news/cwi-spin-off-duckdb-labs-partners-with-motherduck-who-raises-47-5-million-1/ https://www.cwi.nl/en/news/cwi-spin-off-duckdb-labs-partners... https://duckdblabs.com/news/2022/11/15/motherduck-partnership.html https://duckdblabs.com/news/2022/11/15/motherduck-partnershi... https://www.techtarget.com/searchdatamanagement/news/252527260/MotherDuck-raises-475M-for-open-source-DuckDB-database https://www.techtarget.com/searchdatamanagement/news/2525272...