34 ms·

> we’ve replaced our $10k/month Aurora instances with a $200/month Elastic Block Storage (EBS) volume. Without any intent to insult what you've done (because t

• donohoe 3y ago
  I came here to ask the same question. If this db requires 1 full-time developer then the cost would immediately be not worth it (assuming salary + benefits > $120k/yr) As you say, without details it’s hard to know if this was a good idea.
  • bilsbie 3y ago
    Shouldn’t we up our standard developer cost for inflation? That barely qualifies for the median mortgage in the US.
    • twbarber 3y ago
      I believe the 120k number was in reference to the OP’s Aurora spend.
    • Filligree 3y ago
      What makes you think a standard developer can afford a mortgage?
    • nightski 3y ago
      I find that highly unlikely. Maybe in specific markets but not US wide.
    • deleted 3y ago
      [deleted]
    • hibikir 3y ago
      The median home price is under 400K, so a 120k salary is not really stretched. Now, median in the Seattle metro, or in San Francisco, sure. But 120k in, say, St Louis is still going to get you an intermediate dev, no problem, and they can afford a house by themselves too. There are 4 bedroom houses in my neighborhood for 300K.
  • solatic 3y ago
    I actually disagree with you here. There are costs above and beyond the engineer's effect on the balance sheet. There's the partial salary of management to manage them, plus asking them to document their work and train others so that the database won't have a bus factor of 1. So in well-run engineering departments, there's no such thing as paying for a "single" engineering salary. You have teams; a team maintains the system and it has a pre-existing workload. A large part of the value of popular platforms is precisely that they are not bespoke. You can hire engineers with MySQL/Postgres experience. You cannot hire engineers who already have experience with your bespoke systems.
  • sophacles 3y ago
    They said 'instances' (plural), so that number should be at least $240k
• Spivak 3y ago
  I think for this kind of thing their needs are so simple and well-suited to a bespoke implementation that it probably paid for itself in less than 4 months. This doesn't seem like a db implementation that's going to need dedicated maintenance. They're operationally using a funny spelling of SQLite and I don't imagine anyone arguing that such a thing needs constant attention.
• ReflectedImage 3y ago
  Well presumably they need only 1/3 of a developer to do this and they intend to scale up 10x in the next 5 years. $60,000 per year in-house vs $1,200,000 per year aurora. No brainer really.
  • andai 3y ago
    Also worth mentioning that it's 150x faster.
  • cortesoft 3y ago
    Its $120,000 a year for aurora, not $1,200,000.
    • b112 3y ago
      What has changed though, is that we’ve replaced our $10k/month Aurora instances with a $200/month Elastic Block Storage (EBS) volume. Note 'instances' eg plural, versus a singular EBS. There is some ambiguity here, I'm not sure where the 10x came from, but it seems plausible.
      • swasheck 3y ago
        if they were able to replace aurora instances with a glorified kvp store, then they bought the wrong tool in the first place. i saved hundreds of dollars per month by switching from an audi a4 to riding a home-built bike for my 1.5 mile commute to work
        • b112 3y ago
          To be fair, one hardware mysql server for 5k, outperforms a dozen auroea instances. It really bugs me how everyone has drank the coolaide. Cloud is stupid expensive, but of course this is cloud vs cloud.
          • sgarland 3y ago
            RDS also outperforms Aurora. My 12 year old Dell R620s outperform both for certain types of queries (admittedly they have some fast NVMe over insanely fast Mellanox). SANs add latency, who knew? What did surprise me during testing is that Aurora seems to have added latency even if the entire dataset could easily fit into buffers, with zero disk reads required. If you want actually fast cloud DBaaS, the only way is with instances that have local NVMe for caching, like AWS ??gd.db instances.
• rdtsc 3y ago
  > The cost of 120,000/year for Aurora seems like it would be less than the cost of development/organization time for the custom database Only if they planned on hiring someone just to develop this new database and if they switch to Aurora they’d let them go immediately. If the said developer was already costing them $250k to maintain and develop the application and work on top of Aurora cost seems like a good way to save $100k/year.
  • organsnyder 3y ago
    There’s the opportunity cost of whatever else they could have been paying that developer to work on.
    • grogenaut 3y ago
      Agreed, a developer that can pull this off is pretty good, if maybe distracted by shiny objects, what could they do working on the actual product instead of this technological terror?
    • rdtsc 3y ago
      True. Also, to your point, one could argue that if that developer leaves, they'd have an easier time hiring anyone with Aurora experience as opposed to someone to learn and maintain the custom database. But at the same time, Aurora costs could also scale with usage. It may cost $120k one year, $180k next year, $500k the year after. If the database they have now is well designed after it's already built it may not need active development every year but adding a feature here and there. Also, switching back to Aurora could also be an opportunity cost "we should have written our own thing and could have saved millions ...".
    • addicted 3y ago
      Well considering the cost is lower than Aurora isn’t the opportunity cost in favor of the home built situation.
      • organsnyder 3y ago
        That’s only true if Aurora is the most valuable thing that developer could be working on.
• exe34 3y ago
  > PjM/PM What do you need them for?
  • thehappypm 3y ago
    To stop the engineers from spending too much time optimizing the database and to focus on customers
    • exe34 3y ago
      So now every change takes 10x as long because you need to explain it to somebody who's bonus depends on it not being done? I love how the solution to too much management is always more management.
• preommr 3y ago
  I feel like the word "database" is throwing people off because they're comparing it with something like MySql/Postgres, when this seems slightly more complex than a k/v store stored to a file, with some other indexing, where data integrity is a low priority. That shouldn't take too much time and should be fairly isolated on the tech side so little involvement from product/project managers.
  • hmottestad 3y ago
    A k/v store typically is really fast at looking up the value based on the key. So there are usually some pretty advanced indexes involved.
    • arandomusername 3y ago
      or a simple b-tree...
      • eatonphil 3y ago
        My simple btrees have had bugs in them. :) (Though to be fair, if I actually wanted to put this in prod it probably wouldn't take too long to fuzz it and fix the kinks.) https://github.com/eatonphil/btree-rs https://github.com/eatonphil/btree-rs
    • paulddraper 3y ago
      80/20
    • ozr 3y ago
      For a k/v store, a hash table will take you very, very far.
• samatman 3y ago
  I would imagine, as someone with no special insight into goings-on at Hivekit, that the answer is intended scale. They mention 13.5k simultaneous connections. The US has 4.2 million tractors alone, just the US, just tractors. If they get 10% of those tractors on the network that's a 30x to their data storage needs. So multiply that across the entire planet, and all the use cases they hope to serve. Investing time early on so that they can store 50x data-per-dollar is almost certainly time well spent.
  • kdazzle 3y ago
    Presumably those tractors wouldn't be connecting directly to the db though. Not sure why they dont just go the standard iot events route and store data in a data lake and propagate into an analytics db/warehouse from there. Add a layer to make recent events available immediately. S3 is relatively cheap.
• g9yuayon 3y ago
  > PjM/PM time to organize initial build This sounds what big companies or a disorganized company would need. For an efficient enough company, a project like this needs just one or two dedicated engineers. In fact, I can't imagine why this project needs a PM at all. The database is used by engineers and is built by engineers. Engineers should be their own PMs. It's like we need a PM for a programming language, but no, the compiler writer must be the language designer and must use the the language. Those who do not use a product or do not have in-depth knowledge in the the domain should not be the PM of the product.
  • vannevar 3y ago
    >For an efficient enough company, a project like this needs just one or two dedicated engineers. Maybe for a research project or a hobby project, but not for a real, high performance database to be used in a business-critical application. FTA: "Databases are a nightmare to write, from Atomicity, Consistency, Isolation, and Durability (ACID) requirements to sharding to fault recovery to administration - everything is hard beyond belief." >Engineers should be their own PMs. For small projects, sure (your "one or two dedicated engineers"). But once you start tackling projects that require larger teams, or even teams of teams, you need someone to track and prioritize the work remaining and the work in progress (as well as the corresponding budgets for personnel, services, and other resources). Similar to the way a sole proprietor can do their own accounting, but a multi-million dollar business probably should have an accountant. As an aside, I wonder if this might be a use case for a bitmap db engine like Featurebase (https://www.featurebase.com/ https://www.featurebase.com/).
    • delusional 3y ago
      > what we’ve built is just a cursor streaming a binary file feed with a very limited set of functionality - but then again, it’s the exact functionality we need and we didn’t lose any features. The trick is that they didn't need a database that provides "Atomicity, Consistency, Isolation, and Durability (ACID)". By only implementing what they need they were able to keep the project small. It's like people are scared of doing anything without making it into some huge multi hundred developer effort. They've written a super simple append only document store. It's not rocket science. It's not a general purpose arbitrary SQL database.
• deedasmi 3y ago
  Don’t forget this is a largely one time cost vs Aurora, which scales cost with usage. Also they said their current volume is around 13k/second. They’ve built the new platform for 30k/sec per node. This should last them a long time with minimal maintenance.
• kalleboo 3y ago
  Using Aurora is also not free in terms of cost of development. Developers need to be trained on it's implementation, features, constraints, performance implications, keeping up with API changes, etc
• mbrumlow 3y ago
  The problem is all those people you listed would still exist plus the 120k bill to Amazon. They may or may not be doing other things depending on the company size and state. You could drop the PM, engineers writing for engineers don’t need a PM. You will likely hire similarly costed engineers to maintain the database stack anyways. You basically hit all the talking points big cloud has brainwashed people into thinking into being true, but every day we see stories of a handful of engineers doing something we are told can’t be done and saving millions in cloud cost. It’s so painful to watch. Software Engineering became a thing because you could hire a engineer solve your problem, and big businesses stepped in and told you that your problem was something else and gone out of its way to stifle innovation by settling industry standards on how to do things that only guarantee you use cloud services. Any company that wants to own its destiny knows to stay away from lock-in.