3 ms·

> PjM/PM time to organize initial build This sounds what big companies or a disorganized company would need. For an efficient enough company, a project like th

> PjM/PM time to organize initial build This sounds what big companies or a disorganized company would need. For an efficient enough company, a project like this needs just one or two dedicated engineers. In fact, I can't imagine why this project needs a PM at all. The database is used by engineers and is built by engineers. Engineers should be their own PMs. It's like we need a PM for a programming language, but no, the compiler writer must be the language designer and must use the the language. Those who do not use a product or do not have in-depth knowledge in the the domain should not be the PM of the product.

vannevar 3y ago
>For an efficient enough company, a project like this needs just one or two dedicated engineers. Maybe for a research project or a hobby project, but not for a real, high performance database to be used in a business-critical application. FTA: "Databases are a nightmare to write, from Atomicity, Consistency, Isolation, and Durability (ACID) requirements to sharding to fault recovery to administration - everything is hard beyond belief." >Engineers should be their own PMs. For small projects, sure (your "one or two dedicated engineers"). But once you start tackling projects that require larger teams, or even teams of teams, you need someone to track and prioritize the work remaining and the work in progress (as well as the corresponding budgets for personnel, services, and other resources). Similar to the way a sole proprietor can do their own accounting, but a multi-million dollar business probably should have an accountant. As an aside, I wonder if this might be a use case for a bitmap db engine like Featurebase (https://www.featurebase.com/ https://www.featurebase.com/).
- delusional 3y ago
  > what we’ve built is just a cursor streaming a binary file feed with a very limited set of functionality - but then again, it’s the exact functionality we need and we didn’t lose any features. The trick is that they didn't need a database that provides "Atomicity, Consistency, Isolation, and Durability (ACID)". By only implementing what they need they were able to keep the project small. It's like people are scared of doing anything without making it into some huge multi hundred developer effort. They've written a super simple append only document store. It's not rocket science. It's not a general purpose arbitrary SQL database.
  - vannevar 3y ago
    If it's really true that they only needed a "cursor streaming binary file feed with a very limited set of functionality", then why claim in the headline they wrote their own database and begin the article with the observation that doing ACID databases is really hard? Maybe they really built only what they needed. If so, good on them, but ditch the clickbait title. On the other hand, a lot of people believe that they've built only what they need, only to discover in a week or a month or a year that they underestimated their own requirements. Then they gradually re-learn all the lessons that led to the development of real databases, to their eventual sorrow.
    - politician 3y ago
      OTOH, the Bitcask paper can be implemented in about a week for a write-ahead log that’ll give you pretty strong guarantees.
  - mbrumlow 3y ago
    This. Everything we are teaching people to do is centered around solving the problems with already built services, preferably from one of the big 3 cloud vendors. We are churning out “qualified” engineers who don’t know how anything works and we pay them to configure a few AWS services.
cortesoft 3y ago
> a project like this needs just one or two dedicated engineers. So that is at least 20k a month, for fairly cheap engineers.
- yosef123 3y ago
  I doubt he meant « two engineers full time », I suspect it’s more like a role of maintaining / improving when necessary.
- intelVISA 3y ago
  1-2 days work is 20k a month? This place hiring?? Even Google wanted me to work minimum 3 days a month...
vineyardmike 3y ago
> In fact, I can't imagine why this project needs a PM at all. The database is used by engineers and is built by engineers. Engineers should be their own PMs. What about when two different projects have two different requirements they need supported by the database. Which one is implemented first? What about if there is only engineering capacity to implement one? I don’t think a database is the place for “just send a PR for adding your required feature and ping the team that owns it” kind of development. It requires research, planning, architecture review, testing, etc. It’s not a hobby project, it’s a critical tool for the business.
- delusional 3y ago
  > Which one is implemented first? One of them. This is true whether you have a person named "PM" or not. It's just a matter of who picks. > What about if there is only engineering capacity to implement one? How does naming some guy "PM" solve the issue? The team just picks one of the features.

vannevar 3y ago

>For an efficient enough company, a project like this needs just one or two dedicated engineers. Maybe for a research project or a hobby project, but not for a real, high performance database to be used in a business-critical application. FTA: "Databases are a nightmare to write, from Atomicity, Consistency, Isolation, and Durability (ACID) requirements to sharding to fault recovery to administration - everything is hard beyond belief." >Engineers should be their own PMs. For small projects, sure (your "one or two dedicated engineers"). But once you start tackling projects that require larger teams, or even teams of teams, you need someone to track and prioritize the work remaining and the work in progress (as well as the corresponding budgets for personnel, services, and other resources). Similar to the way a sole proprietor can do their own accounting, but a multi-million dollar business probably should have an accountant. As an aside, I wonder if this might be a use case for a bitmap db engine like Featurebase (https://www.featurebase.com/ https://www.featurebase.com/).

delusional 3y ago
> what we’ve built is just a cursor streaming a binary file feed with a very limited set of functionality - but then again, it’s the exact functionality we need and we didn’t lose any features. The trick is that they didn't need a database that provides "Atomicity, Consistency, Isolation, and Durability (ACID)". By only implementing what they need they were able to keep the project small. It's like people are scared of doing anything without making it into some huge multi hundred developer effort. They've written a super simple append only document store. It's not rocket science. It's not a general purpose arbitrary SQL database.
- vannevar 3y ago
  If it's really true that they only needed a "cursor streaming binary file feed with a very limited set of functionality", then why claim in the headline they wrote their own database and begin the article with the observation that doing ACID databases is really hard? Maybe they really built only what they needed. If so, good on them, but ditch the clickbait title. On the other hand, a lot of people believe that they've built only what they need, only to discover in a week or a month or a year that they underestimated their own requirements. Then they gradually re-learn all the lessons that led to the development of real databases, to their eventual sorrow.
  - politician 3y ago
    OTOH, the Bitcask paper can be implemented in about a week for a write-ahead log that’ll give you pretty strong guarantees.
- mbrumlow 3y ago
  This. Everything we are teaching people to do is centered around solving the problems with already built services, preferably from one of the big 3 cloud vendors. We are churning out “qualified” engineers who don’t know how anything works and we pay them to configure a few AWS services.

cortesoft 3y ago

> a project like this needs just one or two dedicated engineers. So that is at least 20k a month, for fairly cheap engineers.

yosef123 3y ago
I doubt he meant « two engineers full time », I suspect it’s more like a role of maintaining / improving when necessary.
intelVISA 3y ago
1-2 days work is 20k a month? This place hiring?? Even Google wanted me to work minimum 3 days a month...

vineyardmike 3y ago

> In fact, I can't imagine why this project needs a PM at all. The database is used by engineers and is built by engineers. Engineers should be their own PMs. What about when two different projects have two different requirements they need supported by the database. Which one is implemented first? What about if there is only engineering capacity to implement one? I don’t think a database is the place for “just send a PR for adding your required feature and ping the team that owns it” kind of development. It requires research, planning, architecture review, testing, etc. It’s not a hobby project, it’s a critical tool for the business.

delusional 3y ago
> Which one is implemented first? One of them. This is true whether you have a person named "PM" or not. It's just a matter of who picks. > What about if there is only engineering capacity to implement one? How does naming some guy "PM" solve the issue? The team just picks one of the features.