3 ms·

> For some types of software, we really do not want students doing it, for free or otherwise. There are whole classes of software, like database engines, that a

> For some types of software, we really do not want students doing it, for free or otherwise. There are whole classes of software, like database engines, that are non-obvious and require many years of real-world domain experience before it is plausible that someone will design a competent, scalable architecture and implementation. Of course as with all things the situation is more nuanced than this. Since you mention database engines we should keep in mind that without Stonebraker and 39 of his students[1] there would be no Postgres. Yet without incentives and many years of contributions from professionals (and students who would become professionals) we would not have PostgreSQL. A healthy system has a place for contributors of all levels of experience. 1- https://momjian.us/main/blogs/pgblog/2020.html#September_21_2020 https://momjian.us/main/blogs/pgblog/2020.html#September_21_...

jandrewrogers 5y ago
Database development has changed quite a lot since then. When I first started working on databases in the 1990s, two things were true that made it much easier for relatively inexperienced developers (like myself at the time) to produce a reasonably good result. First, the state-of-the-art implementations at the time were incredibly well-documented in an accessible way and the academic literature also reflected those designs. Second, the implementations were relatively simple and straightforward; you did not need esoteric systems knowledge and design theory to write code that was competitive with other implementations. The most complicated thing you had to worry about was lock structures and concurrency control. Not only were there relatively simple examples to copy and study, the gap between those examples and the state-of-the-art was pretty small. Neither of these is true today. The design of state-of-the-art implementations are often poorly documented; the computer science has evolved radically since the 1990s with significant gaps in the literature; competitive implementations require real expertise in silicon architecture and Linux kernel internals, you can't just write something obvious in C and expect a good result. And that's without even getting into difficult topics like distributed execution, parallel orchestration, scheduler design, etc that we didn't have to worry about back then. I'm not sure I'd be able to bootstrap the necessary expertise to build database engines today like I did back then.
- berkeley39 5y ago
  Regarding your points * First, state-of-the-art was accessible and well-documented with designs reflected in academic literature. * Second, implementations were relatively simple and straightforward. * Neither of these is true today. Again the situation is more nuanced than this. Before the web systems were expensive, resources were scarce and connections to research were critical. Now anyone with time and a laptop can watch youtube lectures, read recent research, clone a repo and join the party. Different things are hard now. Such is the way of competition. Hardware and economic trends move the state-of-the-art quickly. Ideas offering competitive advantage are rarely documented to the degree one may like. But they do find their way to entrepreneurial practitioners and students. Their time, energy, attention and motivation builds new and better systems and we should welcome their efforts when they advance open-source projects. > I'm not sure I'd be able to bootstrap the necessary expertise to build database engines today like I did back then. Don't sell yourself short :-)

jandrewrogers 5y ago

Database development has changed quite a lot since then. When I first started working on databases in the 1990s, two things were true that made it much easier for relatively inexperienced developers (like myself at the time) to produce a reasonably good result. First, the state-of-the-art implementations at the time were incredibly well-documented in an accessible way and the academic literature also reflected those designs. Second, the implementations were relatively simple and straightforward; you did not need esoteric systems knowledge and design theory to write code that was competitive with other implementations. The most complicated thing you had to worry about was lock structures and concurrency control. Not only were there relatively simple examples to copy and study, the gap between those examples and the state-of-the-art was pretty small. Neither of these is true today. The design of state-of-the-art implementations are often poorly documented; the computer science has evolved radically since the 1990s with significant gaps in the literature; competitive implementations require real expertise in silicon architecture and Linux kernel internals, you can't just write something obvious in C and expect a good result. And that's without even getting into difficult topics like distributed execution, parallel orchestration, scheduler design, etc that we didn't have to worry about back then. I'm not sure I'd be able to bootstrap the necessary expertise to build database engines today like I did back then.

berkeley39 5y ago
Regarding your points * First, state-of-the-art was accessible and well-documented with designs reflected in academic literature. * Second, implementations were relatively simple and straightforward. * Neither of these is true today. Again the situation is more nuanced than this. Before the web systems were expensive, resources were scarce and connections to research were critical. Now anyone with time and a laptop can watch youtube lectures, read recent research, clone a repo and join the party. Different things are hard now. Such is the way of competition. Hardware and economic trends move the state-of-the-art quickly. Ideas offering competitive advantage are rarely documented to the degree one may like. But they do find their way to entrepreneurial practitioners and students. Their time, energy, attention and motivation builds new and better systems and we should welcome their efforts when they advance open-source projects. > I'm not sure I'd be able to bootstrap the necessary expertise to build database engines today like I did back then. Don't sell yourself short :-)