37 ms·

The big issue I think we miss when people say "why are you using Dynamo, just use SQL" or "why are you using hadoop, a bash shell would be faster" or "why are y

by 013a 8y ago

The big issue I think we miss when people say "why are you using Dynamo, just use SQL" or "why are you using hadoop, a bash shell would be faster" or "why are you using containers and kubernetes, just host a raspberry pi in your closet": The former examples are all managed! That's amazing for scaling teams. (SQL can be managed with, say, RDS. Sure. But it's not the same level of managed as Dynamo (or Firebase or something like that). It still requires maintenance and tuning and upkeep. Maybe that's fine for you (remember: the point of this article was to tell you to THINK, not to just ignore any big tech products that come out). But don't discount the advantage of true serverless.) My goal is to be totally unable to SSH into everything that powers my app. I'm not saying that I want a stack where I don't have to. I'm saying that I literally cannot, even if I wanted to real bad. That's why serverless is the future; not because of the massive scale it enables, but because fuck maintenance, fuck operations, fuck worrying about buffer overflow bugs in OpenSSL, I'll pay Amazon $N/month to do that for me, all that matters is the product.

cheez 8y ago
Here I am still using SQLite (had to add an index yesterday!) Maybe one day I will grow up :(
- sli 8y ago
  SQLite is good quality software, and very heavily tested. There's nothing wrong with using it so long as you don't outright misuse it, and they are clear about when you shouldn't be using it.
  - snazz 8y ago
    I very rarely run into a situation where I need more than SQLite. The page on the subject is great: https://www.sqlite.org/whentouse.html https://www.sqlite.org/whentouse.html
    - collyw 8y ago
      I's not so good for multi user systems as I understand as it has a lock on the whole database when writing.
      - snazz 8y ago
        Queuing works most of the time, does it not?
        collyw 8y ago
        To be fair I have only really used SQLite for early development stages so haven't investigated workarounds in much detail.
        cheez 8y ago
        Queuing and retry works most of the time. Only time I truly get "database is locked" is when I open the shell while other processes are accessing the DB
        ngrilly 8y ago
        Where do you queue: in memory? on disk? in a dedicated queue server (Redis, Kafka, ActiveMQ, etc.)? in another SQLite database?
        cheez 8y ago
        So far, haven't needed queuing outside of a single thread in the application (I know, what happens if it crashes!) Hasn't happened in ~8 years. Won't cry if it does.
        ngrilly 8y ago
        I have an app that uses an in-memory queue too in one thread :-) I'm curious: why do you queue instead of directly writing to the database?
- ngrilly 8y ago
  How do you access remotely, from your workstation, to the SQLite database on your server, while it is still serving requests?
  - cheez 8y ago
    ssh user@server sqlite file://$PWD/db.sqlite?mode=ro
    - ngrilly 8y ago
      Yes, this will work of course :-) I was thinking of something more "graphical", similar to pgAdmin or SequelPro, when you need to navigate in a dataset.
      - Redoubts 8y ago
        There's https://sqlitebrowser.org https://sqlitebrowser.org, But I suppose you'll have to string up something if you want it to access a remote db.
        ngrilly 8y ago
        There is also https://www.phpliteadmin.org https://www.phpliteadmin.org, but I'm not a fan of the user interface.
      - cheez 8y ago
        I use https://sqlitebrowser.org/ https://sqlitebrowser.org/ if I need to but primarily Jupyter w/ Pandas to explore and understand the data and prototype new queries.
        ngrilly 8y ago
        Regarding SQLiteBrowser, I guess you have to download a copy of the database on your local workstation?
pfranz 8y ago
To your point, when I often see the "shell script" solution it's often missing basic things you'd want like version control or any kind of bread crumb trail (I find myself sshing in and "grep -r" trying to find things). That works if you're cobbling something together, but doesn't scale beyond 1 person. It's tough to get the correct level because you can ansible > docker > install an rpm or just change the file in place. Both have their place and "hacky" solutions can work just fine for many years.
beering 8y ago
For anyone who's not used these "managed" services before, I want to add that it's still a fuck ton of work. The work shifts from "keeping X server running" to "how do I begin to configure and tune this service". You will run into performance issues, config gotchas, voodoo tuning, and maintenance concerns with any of AWS's managed databases or k8s. > I'll pay Amazon $N/month to do that for me Until you pay Amazon $N/month to provide the service, and then another $M/month to a human to manage it for you.
- maxxxxx 8y ago
  "For anyone who's not used these "managed" services before, I want to add that it's still a fuck ton of work. The work shifts from "keeping X server running" to "how do I begin to configure and tune this service"." I have noticed that too. With some managed services you are trading a set of generally understood problems with a lot of quirky behavior of the service that's very hard to debug.
  - erikpukinskis 8y ago
    Yes, but I think very broadly speaking the quirky behavior is stuff you bump into, learn about, fix, and then can walk away from. The daily/monthly maintenance cycle on a self hosted SQL server is “generally understood” but you still have to wake up, check your security patches, and monitor your redeployments. You can do some of that in an automated fashion with public security updates for your containers and such. But if monitoring detects an anomaly, it’s YOU, not Heroku who gets paged. It’s a little like owning a house vs renting. Yes if you rent you have to work around the existing building, and getting small things fixed is a process. But if the pipes explode, you make a phone call and it’s someone else’s problem. You didn’t eliminate your workload, but you shrunk the domain you need to personally be on call for.
    - forty 8y ago
      It's someone else problem unless it prevents you from living here, in which case it's still your problem too. I think the analogy works quite well :)
KZeillmann 8y ago
I work on a team where our products are all in Fargate containers. I understand the appeal of serverless -- you never need to exec into the container, but half the time when we're debugging an issue in prod that we can't reproduce locally, we'll say, "wouldn't this be easier if we could just exec into the container and find out exactly what's going on?"
- worldsayshi 8y ago
  Not being able to ssh into a container sounds like a missing feature of that particular container solution? I would expect that I can ssh into a docker container hosted in a kubernetes cluster. Hmm, pretty sure I must've done this dozens of times.
- johnwyles 8y ago
  Removing SSH should be the goal though. If you follow the old Visible Ops book you also "Electrify the Fence" and introduce accountability, etc. If your goal is to see what a process is doing introduce tracing. If you need to add a "println()" then push that out as a change because the environment is changing from your altering of it. Because the tool doesn't exist yet that you need to SSH into a box doesn't mean it shouldn't - you have to instrument the tooling to prevent you from needing this adhoc ability. Admittedly it scares me still but ideally the end game is to never need to or have the ability to do so through a tool which has all the things you are looking for without allowing a human to be too human and miss a semi-colon.
  - worldsayshi 8y ago
    You need to be able to troubleshoot things in production though. Perhaps whenever a developer wants to troubleshoot the orchestrator could make a clone of the container. The clone continuously receives the same input as the original and gets to believe that it affects the back end in the same way. That way the developer can dissect the container without impacting real data.
  - xorcist 8y ago
    > "println()" then push that out as a change No, when you actually need to debug in production that's usually not what you want. Changing or restarting the software you are debugging might well make the behaviour you want to understand go away. > introduce tracing Yeah, well, that's basically "logging in". Just over less mature and likely less secure protocol than SSH. You don't need ptrace and tcpdump to debug software. It's just that it can shave a few weeks off your time when you need to reproduce something in the more tricky cases. These discussions tend to surface in the context of containers but that's all very irrelevant. You need to debug software isn't affected by the way you package it.
JustSomeNobody 8y ago
> That's why serverless is the future; Future of what exactly?
- revskill 8y ago
  Future of server ?
  - mrmuagi 8y ago
    Future of server is serverless? :P
    - ph0rque 8y ago
      Serverless is just somebody else's server.
      - anoncake 8y ago
        Serverless is the cloud?
      - jjeaff 8y ago
        No, "cloud" is somebody else's server. Serverless is somebody else's server that scales your services seamlessly (and usually opaguely).
        ph0rque 8y ago
        I was riffing off of the phrase, "The cloud is just someone else's computer".
    - paulddraper 8y ago
      Oh, I didn't realize we were doing trick questions. What's the safest way to go skiing? Don't ski.
- kingraoul3 8y ago
  The future of the pit they will dump money into.
dmix 8y ago
I'm not sure I follow, how does managed services help with choosing the right database technology for the problem set?
- lr4444lr 8y ago
  Yeah, I can't quite square what the OP is saying with how to deal with something like Amazon Dynamo's lack of large scale transactions. (Even basic ACID was only introduced late last year, if I read correctly.)
  - geofft 8y ago
    If you choose Dynamo, then you get to blame Dynamo for not having ACID when you lose data! This is an incredibly common fallacy in our industry. Remember, you are responsible for the appropriateness of the dependencies you choose. If you build your fancy product on a bad foundation, and the foundation crumbles, your product collapses with it and it's unprofessional to say "Well, I didn't pour the foundation, so I can't be blamed." Maybe not, but you decided to build on top of it.
- kingraoul3 8y ago
  He doesn't care about that. JUST THE PRODUCT. That's how amazing his fantasies are.
about_help 8y ago
Security and privacy yet again traded for convenience. We need strict data privacy laws and then I would have a little more trust in large centralized entities that would otherwise have every incentive to abuse people's data. Such centralized systems also become significant points of failure. What happens to your small business if one of the many services you rely on disappears or changes their API? "All that matters is the product" sounds very much like "all that matters is the bottom line" and we've seen the travesties that occur when profit is put above all else.
- entee 8y ago
  It's not clear that having your own datacenter is more secure than using AWS/GCP/Azure services. In both cases there are good monitoring solutions, however I'd say that cloud-based solutions have easier immediate access to most things because they just integrate with the provider's API whereas on prem you're installing agents and whatnot. Also having granular IAM for services and data is very helpful for security. You have a single source of truth for all your devs, and extremely granular permissions that can be changed in a central location. Contrast to building out all of that auditing and automation on your own. Granted, IAM permissions give us tons of headaches on the regular, but on balance I still think it's better when done well. If you're concerned about AWS/GCP/Azure looking at your private business's data, I think 1.) that's explicitly not allowed in the contracts you sign 2.) many services are HIPAA compliant, hence again by law they can't 3.) They'd for sure suffer massively both legally and through loss of business if they ever did that.
toast0 8y ago
> but because fuck maintenance, fuck operations, fuck worrying about buffer overflow bugs in OpenSSL, I'll pay Amazon $N/month to do that for me, all that matters is the product. That's nice in theory, but my experience with paying big companies for promises like that is I still end up having to debug their broken software; and it's a lot easier to do that if I can ssh into the box. I've got patches into OpenSSL that fixed DHE negotiation from certain versions of windows (especially windows mobile), that I really don't think any one's support team would have been able to fix for me / my users [1], unless I had some really detailed explanation -- at that point, I may as well be running it myself. [1] And as proof that nobody would fix it, I offer that nobody fixed this, even though there were public posts about it for a year before my patches went in; so some people had noticed it was broken.
- __jal 8y ago
  This. I've been informally tracking time spent on systems my team manages and managed tools we use. (I manage infra where I work.) There is very little difference. And workarounds and troubleshooting spent on someone else's system mean we only learn about some proprietary system. That's bad for institutional knowledge and flexibility, and for individual careers. Our needs don't mesh well with typical cloud offerings, so we don't use them for much. When we have, there has yet to be a cost savings - thus far they've always cost us more than we pay owning everything. I mean, I personally like not dealing with hardware or having to spend time in data centers. But I can't justify it from a cost or a service perspective.
  - pojzon 8y ago
    After amazon switched to ec2/sec kind of billing in 2017 i would say cloud is pretty good for prototyping and running CI. As to anything else i would say it depends.
- sethammons 8y ago
  absolutely. I am _right_now_ fighting ELB because we see traffic going to it but not coming out of it. If it were MY load balancer, I would just log in and tcpdump.
AnIdiotOnTheNet 8y ago
Yeah, "not my problem" sounds like a winning strategy, until you realize that it also means "nothing I can do about it" when things inevitably go wrong.
- wang_li 8y ago
  Not to mention that you're the lowest priority when something does go wrong.
msbarnett 8y ago
Enjoy the many (many, many) connection reset by peer, TCP idle timeouts, and 500 Internal Server Errors on Dynamo's end. Between that and the dogshit documentation, it's truly thrilling to be paying them a ridiculous amount of money for the privilege of having a managed service go undebuggably AWOL on the regular, instead of being able to resolve the issues locally.
- StreamBright 8y ago
  I can see my code handling all the exceptions correspond to the points you raised with exponential backoff. Which part of Dynamo's documentation is dogshit?
- dilyevsky 8y ago
  Yeah and if you’re on gcp you also get dogshit support whose main strategy is to sandbag you with endless (and useless) logs collection until you give up and move on or take too long so they can close the ticket.
tptacek 8y ago
Which Dynamo are you talking about? The Dynamo paper, or Dynamo, the AWS managed database product? Dynamo-the-product makes sense for all sorts of random tasks. Full-on implementations of Dynamo-the-paper (for instance: Cassandra) fall into the bucket of "understand the problem first" that the author is talking about.
mrmuagi 8y ago
What specific managed services do you use, if I may inquire?
milquetoastaf 8y ago
so what happens when they don't do what you want them to?
paganel 8y ago
> My goal is to be totally unable to SSH into everything that powers my app That sounds like a nightmare to me, and I'm not even a server guy, I'm a backend guy only managing my personal projects. > I'll pay Amazon $N/month to do that for me, all that matters is the product. I don't want to pay Amazon anything, I want to pay Linode/Digital Ocean 5 or 10 or 20 dollars per month and I can do the rest myself pretty well. My personal projects will never ever going to reach Google's scale and seeing that they don't bring me anything (they're not intended to) I'm not that eager to pay an order a magnitude more to a company like Amazon or Google in order to host those projects.
- staticassertion 8y ago
  As someone tasked with protecting infrastructure, devs not being able to SSH into production is a godsend.
  - scaryclam 8y ago
    There's a difference from devs not being able to ssh in due to access restrictions and devs not being able to ssh in because it's not a feature of the service. I actually agree that not having unilateral access to the production system is a good thing. But not being able to have anyone able to be granted access, even for a limited time, is neither productive or safe.
  - hackerpacker 8y ago
    so, what is to stop them from running code from the app?!? Who do you call when something breaks? Do you expect the dev to not be able to log in and investigate at 3am or whenever? I mean if you can't trust your dev team, or the procedures surrounding them, you are kinda screwed.
    - staticassertion 8y ago
      To be clear, the issue isn't about trusting developers in prod. It's about limiting access to prod as an avenue for attackers who will be looking to get in there and exfil sensitive data. Nothing to do with trust of devs.
StreamBright 8y ago
>> My goal is to be totally unable to SSH Great goal. I guess you never had any use case to login to a production server to investigate a business critical issue.
- jessaustin 8y ago
  It's more likely that parent has had to do that a lot of times, or perhaps has managed people who have. That would be a sign of organizational dysfunction, and fixing that is a good goal to have.
- not_kurt_godel 8y ago
  I have had to SSH into prod boxes many times to debug emergent issues. Now I'm using all serverless, and I get a happy fuzzy feeling whenever I think about how I never ever have to do that again.
  - teraflop 8y ago
    "The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at and repair." -- Douglas Adams
pojzon 8y ago
Issue is, this works only for very simple systems. With complexity comes maintenance and operations. You also cannot build anything innovative in case of heavy load systems because you cannot embbed your custom highly performant solutions (cloud provider simply does not support them ootb). Serverless sounds great for "some" kind of systems but in other cases its a complete missfire. Our job is to know which tool to use.
geofft 8y ago
A SQL database, bash, and a Raspberry Pi in your closet all are managed too. They're built on top of software that's been battle-tested over decades to keep itself running. MySQL/Postgres/whatever will manage running multiple queries for you and keeping the data consistent as well as Dynamo will. bash will manage running multiple processes for you as well as Hadoop will. A Raspberry Pi with cron will manage starting a service for you every time it powers on as well as Lambda will. The reasons to use Dynamo/Hadoop/Lambda are when you've got different problems from the ones that MySQL/bash/cron solve. If you don't believe that this counts as "managed," then for all your words at the end, you believe at the end of the day that making a solution more complex so that failures are expected and humans are required is superior to doing a simple and all-in-code thing. For all that you claim to not like operations, you think that a system requiring operational work makes it better. You are living the philosophy of the sysadmins who run everything out of .bash_history and Perl in their homedir and think "infrastructure as code" is a fad.
- eloisant 8y ago
  With this definition unless you write your own database system, everything is managed. That makes it pretty useless to qualify anything as managed. The Raspberry Pi is managed by you, as opposed to managed by a different company / different people so you can focus on your business problem.
apoorvgarg 8y ago
Before we make a technology choice, we should be clear what those choices are. SQL is a query language and DynamoDB is a database. "NoSQL" technologies can be fronted with SQL interfaces. Relational databases can vary a lot too in their ACID compliance ("I" being the component with maximum variance). The choice of technology should first be based on the problem and not whether it is serverless. Choosing DynamoDB when your application demands strict serializability would be totally wrong. I hope more people think about the semantic specifications of their applications (especially under concurrency).
BoorishBears 8y ago
To me this comment is not a response to the article. The article isn’t asking why use Dynamo vs SQL in the context of the traditional NoSQL vs SQL way It’s why Dynamo vs QLDB. Or why Cloud Firestone vs Cloud Spanner. Or Firehose + some lambda functions vs EMR It’s wholly separate of the managed angle and focuses on the actual complexity of the products involved and the trade offs therein. In the surface the two sound similar, but there’s much more nuance to this article's point (take the example of Dynamo, the author isn’t opposed to it for being NoSQL, they’re opposed to it for low level technical design choices that are a poor fit for the problem their client had)
gambler 8y ago
>SQL can be managed with, say, RDS. Sure. But it's not the same level of managed as Dynamo (or Firebase or something like that). It's managed at every shared hosting in the world.
- atombender 8y ago
  Not at the same level, hardly anywhere. Replication, auto-scaling, failover and point-in-time backups are typically not managed for you. AWS and GCP have decent solutions now, but they're not nearly as hands-off in terms of management.
OrgNet 8y ago
If you don't care about cost and don't care about someone else having all of your data, that is probably a fine route.
plufz 8y ago
If the main reason for your switch from sql to dynamo is that it is managed why not use aws aurora, aurora serverless, or any other more managed sql solution? Now when many sql databases support json I don’t quite understand why you would use a documentdb for anything else then for actual unstructured data, or key value pairs that needs specific performance. I used mongodb for a while now I’m 100% back in sql. Document dbs really increased my love for sql. But I’m also long from being a nosql expert and I never have to deal with data above a TB.
mattrp 8y ago
N gets to be a big number my friend. I love aws but I can’t get over the number of startups I’ve seen with $100k MRR and $35k MRC allocated to AWS.
Justsignedup 8y ago
It is important to understand the following: Depending on the problem, choose the right tools. For example so far everything I've seen about React and GraphQL tells me that maybe GQL isn't necessarily the best solution for a 5-person team, but a 100 person team may have a hard time living without it. Kubernates / Docker is significant work. And when we had 4 developers making that effort was wasteful and we literally didn't have the time. Now at 20 we're much more staffed and the things Kubernates / Docker solves is useful. Meanwhile we have a single PostgreSQL server. No nosql caching layer. We're at a point where we aren't sure how far we gotta go before we hit limits of a PSQL server, but at least 3-4 years before we hit it. Point is look at the tools. See where these tools win it for you. Don't just blindly pick because it can scale super well. Pick it because it gives you things you need and a simplicity you need for the current and near-term context, and have a long-term strategy how to move away from it when it is no longer useful.
theonemind 8y ago
If everyone did this, in the limiting case, no one has the ability to investigate and repair actual bugs until the point where the application can only crash every 10 ms and does nothing but get continually restarted.
semi-extrinsic 8y ago
> My goal is to be totally unable to SSH into everything that powers my app. If the 0day in your familiar pastures dwindles, despair not! Rather, bestir yourself to where programmers are led astray from the sacred Assembly, neither understanding what their programming languages compile to, nor asking to see how their data is stored or transmitted in the true bits of the wire. For those who follow their computation through the layers shall gain 0day and pwn, and those who say “we trust in our APIs, in our proofs, and in our memory models and need not burden ourselves with confusing engineering detail that has no value anyhow” shall surely provide an abundance of 0day and pwnage sufficient for all of us. Thus preacheth Pastor M. Laphroaig.
tigershark 8y ago
Serverless is not a panacea, actually most of the times is true the opposite. You will spend much more time dealing with configuration, deployments, managing cold starts, orchestrating everything. Sure, for some use cases is perfect, but I can tell you almost certainly that the future won’t be on AWS lambda &co.
- collyw 8y ago
  Its moving complexity from the application layer to the deployment layer.
  - guru4consulting 8y ago
    which is awesome. Application complexity is harder to solve than deployment/infrastructure complexity. And more often the cloud providers would solve the deployment complexities over time.
debaserab2 8y ago
But these services do break, and when they do break, often times you don't even get a useful error or log file to grep. Sometimes I just really wish I could SSH into a box to figure out what the hell is going on. You're gaining an expertise, but instead of it being with the underlying technology, it's with the management services for that technology. I think the premise of your argument is correct that understanding these services is simpler than the technology itself, but I seem to inevitably find myself in some edge use case of the service that is poorly documented or does not work as described, and trolling stack overflow or AWS developer forums is the only solution other than trial and (no) error. Let me tell you: it's not fun when you're entrenched into a proprietary service that is now outdated and kept only for legacy reasons, and the version you are on doesn't allow you to do things like update your programming language to a more recent version. Now you get to move to another managed service and lose all that expert knowledge you had with the old one.
goatlover 8y ago
> My goal is to be totally unable to SSH into everything that powers my app. I'm not saying that I want a stack where I don't have to. I'm saying that I literally cannot, even if I wanted to real bad. So you want technology that prevents you from doing something you might want to do, real badly?
Juliate 8y ago
If you go server less because fuck accountability, shouldn't the extension be jobless because someone else will provide the service you did? In that way, cloud/serverless migration is really the whole worldwide IT workforce progressively willingly migrating IT (design, development, operations & applications) out of companies, to just a few big providers. Just like energy. And who gets to decide destinies in the world? Those who control, have and provide energy. What a time to be alive. End of XIX century was the rush to oil-based energy to control the world. End of XX was the rush to data-based capture and control.
mrblues 8y ago
I recommend heroku and heroku postgres
deleted 8y ago
[deleted]
nukeop 8y ago
Using Dynamo is the worst thing we've done for productivity and costs in our product. It introduces onboarding overhead, unnecessary implementation time overhead, costs a lot, and is always a bottleneck in every operation. Using DAX requires paying for VPC which already costs more than a server that could power many small to mid-sized products. Resizing table throughputs requires lots of manual work to keep it cost effective, because the default settings are designed to nickel and dime you to death. If we just used a simple SQL or MongoDB server attached as a container we'd save literally months of development time and thousands of dollars per month.
- expliced 8y ago
  This sounds beyond painful. We have similar problems at work with expensive system that I believe we could reimplement with traditional technology, but we don't.
sureaboutthis 8y ago
I see that HN now steps into the gutter of reddit-like language and can damn all professional decorum.
ivanhoe 8y ago
> My goal is to be totally unable to SSH into everything that powers my app. In reality that means that when something doesn't work as expected, instead of logging in and solving the problem, you'll be spending your time writing tickets and fighting through the tiers of support and their templated replies, waiting for them to eventually escalate your issue to some actual engineer. Wish you a lot of luck with that, I prefer being able to fix my own stuff (or at least give it a try first).
yowlingcat 8y ago
Well, if all you want is managed, per request hosting of SQL, use Aurora serverless! It's a game changer. As for Dynamo vs SQL -- you really cannot substitute one for the other. They're different tools for different jobs. Dynamo is extremely low latency and great for that purpose, while SQL is great for being a store of record, and if you use it as long as you can get away with it for as many things as possible, it's usually not a bad idea.
mcguire 8y ago
... Fuck debuggablity. Fuck performance. Fuck understanding what is actually going on.... (Sure, managed solutions can satisfy all of those. But not without understanding a few orders of magnitude more complexity than a simpler solution, and you'll never get there if your attitude is "I'll pay Amazon to do that for me." Amazon doesn't care whether you succeed.)
- empthought 8y ago
  > Amazon doesn't care whether you succeed. Wait, what? Why wouldn't they? Successful companies will spend money with them; defunct companies can't.
oarabbus_ 8y ago
>"why are you using Dynamo, just use SQL" On the other hand, I find it frustrating how many developers either don't know basic SQL or try to get cute and use the flat side of an axe to smack down nails instead of just using the well-known, industry-standard beauty of a hammer that SQL.

by 013a 8y ago

cheez 8y ago

Here I am still using SQLite (had to add an index yesterday!) Maybe one day I will grow up :(

sli 8y ago
SQLite is good quality software, and very heavily tested. There's nothing wrong with using it so long as you don't outright misuse it, and they are clear about when you shouldn't be using it.
- snazz 8y ago
  I very rarely run into a situation where I need more than SQLite. The page on the subject is great: https://www.sqlite.org/whentouse.html https://www.sqlite.org/whentouse.html
  - collyw 8y ago
    I's not so good for multi user systems as I understand as it has a lock on the whole database when writing.
    - snazz 8y ago
      Queuing works most of the time, does it not?
      - collyw 8y ago
        To be fair I have only really used SQLite for early development stages so haven't investigated workarounds in much detail.
      - cheez 8y ago
        Queuing and retry works most of the time. Only time I truly get "database is locked" is when I open the shell while other processes are accessing the DB
        ngrilly 8y ago
        Where do you queue: in memory? on disk? in a dedicated queue server (Redis, Kafka, ActiveMQ, etc.)? in another SQLite database?
        cheez 8y ago
        So far, haven't needed queuing outside of a single thread in the application (I know, what happens if it crashes!) Hasn't happened in ~8 years. Won't cry if it does.
        ngrilly 8y ago
        I have an app that uses an in-memory queue too in one thread :-) I'm curious: why do you queue instead of directly writing to the database?
ngrilly 8y ago
How do you access remotely, from your workstation, to the SQLite database on your server, while it is still serving requests?
- cheez 8y ago
  ssh user@server sqlite file://$PWD/db.sqlite?mode=ro
  - ngrilly 8y ago
    Yes, this will work of course :-) I was thinking of something more "graphical", similar to pgAdmin or SequelPro, when you need to navigate in a dataset.
    - Redoubts 8y ago
      There's https://sqlitebrowser.org https://sqlitebrowser.org, But I suppose you'll have to string up something if you want it to access a remote db.
      - ngrilly 8y ago
        There is also https://www.phpliteadmin.org https://www.phpliteadmin.org, but I'm not a fan of the user interface.
    - cheez 8y ago
      I use https://sqlitebrowser.org/ https://sqlitebrowser.org/ if I need to but primarily Jupyter w/ Pandas to explore and understand the data and prototype new queries.
      - ngrilly 8y ago
        Regarding SQLiteBrowser, I guess you have to download a copy of the database on your local workstation?

pfranz 8y ago

To your point, when I often see the "shell script" solution it's often missing basic things you'd want like version control or any kind of bread crumb trail (I find myself sshing in and "grep -r" trying to find things). That works if you're cobbling something together, but doesn't scale beyond 1 person. It's tough to get the correct level because you can ansible > docker > install an rpm or just change the file in place. Both have their place and "hacky" solutions can work just fine for many years.

beering 8y ago

For anyone who's not used these "managed" services before, I want to add that it's still a fuck ton of work. The work shifts from "keeping X server running" to "how do I begin to configure and tune this service". You will run into performance issues, config gotchas, voodoo tuning, and maintenance concerns with any of AWS's managed databases or k8s. > I'll pay Amazon $N/month to do that for me Until you pay Amazon $N/month to provide the service, and then another $M/month to a human to manage it for you.

maxxxxx 8y ago
"For anyone who's not used these "managed" services before, I want to add that it's still a fuck ton of work. The work shifts from "keeping X server running" to "how do I begin to configure and tune this service"." I have noticed that too. With some managed services you are trading a set of generally understood problems with a lot of quirky behavior of the service that's very hard to debug.
- erikpukinskis 8y ago
  Yes, but I think very broadly speaking the quirky behavior is stuff you bump into, learn about, fix, and then can walk away from. The daily/monthly maintenance cycle on a self hosted SQL server is “generally understood” but you still have to wake up, check your security patches, and monitor your redeployments. You can do some of that in an automated fashion with public security updates for your containers and such. But if monitoring detects an anomaly, it’s YOU, not Heroku who gets paged. It’s a little like owning a house vs renting. Yes if you rent you have to work around the existing building, and getting small things fixed is a process. But if the pipes explode, you make a phone call and it’s someone else’s problem. You didn’t eliminate your workload, but you shrunk the domain you need to personally be on call for.
  - forty 8y ago
    It's someone else problem unless it prevents you from living here, in which case it's still your problem too. I think the analogy works quite well :)

KZeillmann 8y ago

I work on a team where our products are all in Fargate containers. I understand the appeal of serverless -- you never need to exec into the container, but half the time when we're debugging an issue in prod that we can't reproduce locally, we'll say, "wouldn't this be easier if we could just exec into the container and find out exactly what's going on?"

worldsayshi 8y ago
Not being able to ssh into a container sounds like a missing feature of that particular container solution? I would expect that I can ssh into a docker container hosted in a kubernetes cluster. Hmm, pretty sure I must've done this dozens of times.
johnwyles 8y ago
Removing SSH should be the goal though. If you follow the old Visible Ops book you also "Electrify the Fence" and introduce accountability, etc. If your goal is to see what a process is doing introduce tracing. If you need to add a "println()" then push that out as a change because the environment is changing from your altering of it. Because the tool doesn't exist yet that you need to SSH into a box doesn't mean it shouldn't - you have to instrument the tooling to prevent you from needing this adhoc ability. Admittedly it scares me still but ideally the end game is to never need to or have the ability to do so through a tool which has all the things you are looking for without allowing a human to be too human and miss a semi-colon.
- worldsayshi 8y ago
  You need to be able to troubleshoot things in production though. Perhaps whenever a developer wants to troubleshoot the orchestrator could make a clone of the container. The clone continuously receives the same input as the original and gets to believe that it affects the back end in the same way. That way the developer can dissect the container without impacting real data.
- xorcist 8y ago
  > "println()" then push that out as a change No, when you actually need to debug in production that's usually not what you want. Changing or restarting the software you are debugging might well make the behaviour you want to understand go away. > introduce tracing Yeah, well, that's basically "logging in". Just over less mature and likely less secure protocol than SSH. You don't need ptrace and tcpdump to debug software. It's just that it can shave a few weeks off your time when you need to reproduce something in the more tricky cases. These discussions tend to surface in the context of containers but that's all very irrelevant. You need to debug software isn't affected by the way you package it.

JustSomeNobody 8y ago

> That's why serverless is the future; Future of what exactly?

revskill 8y ago
Future of server ?
- mrmuagi 8y ago
  Future of server is serverless? :P
  - ph0rque 8y ago
    Serverless is just somebody else's server.
    - anoncake 8y ago
      Serverless is the cloud?
    - jjeaff 8y ago
      No, "cloud" is somebody else's server. Serverless is somebody else's server that scales your services seamlessly (and usually opaguely).
      - ph0rque 8y ago
        I was riffing off of the phrase, "The cloud is just someone else's computer".
  - paulddraper 8y ago
    Oh, I didn't realize we were doing trick questions. What's the safest way to go skiing? Don't ski.
kingraoul3 8y ago
The future of the pit they will dump money into.

dmix 8y ago

I'm not sure I follow, how does managed services help with choosing the right database technology for the problem set?

lr4444lr 8y ago
Yeah, I can't quite square what the OP is saying with how to deal with something like Amazon Dynamo's lack of large scale transactions. (Even basic ACID was only introduced late last year, if I read correctly.)
- geofft 8y ago
  If you choose Dynamo, then you get to blame Dynamo for not having ACID when you lose data! This is an incredibly common fallacy in our industry. Remember, you are responsible for the appropriateness of the dependencies you choose. If you build your fancy product on a bad foundation, and the foundation crumbles, your product collapses with it and it's unprofessional to say "Well, I didn't pour the foundation, so I can't be blamed." Maybe not, but you decided to build on top of it.
kingraoul3 8y ago
He doesn't care about that. JUST THE PRODUCT. That's how amazing his fantasies are.

about_help 8y ago

Security and privacy yet again traded for convenience. We need strict data privacy laws and then I would have a little more trust in large centralized entities that would otherwise have every incentive to abuse people's data. Such centralized systems also become significant points of failure. What happens to your small business if one of the many services you rely on disappears or changes their API? "All that matters is the product" sounds very much like "all that matters is the bottom line" and we've seen the travesties that occur when profit is put above all else.

entee 8y ago
It's not clear that having your own datacenter is more secure than using AWS/GCP/Azure services. In both cases there are good monitoring solutions, however I'd say that cloud-based solutions have easier immediate access to most things because they just integrate with the provider's API whereas on prem you're installing agents and whatnot. Also having granular IAM for services and data is very helpful for security. You have a single source of truth for all your devs, and extremely granular permissions that can be changed in a central location. Contrast to building out all of that auditing and automation on your own. Granted, IAM permissions give us tons of headaches on the regular, but on balance I still think it's better when done well. If you're concerned about AWS/GCP/Azure looking at your private business's data, I think 1.) that's explicitly not allowed in the contracts you sign 2.) many services are HIPAA compliant, hence again by law they can't 3.) They'd for sure suffer massively both legally and through loss of business if they ever did that.

toast0 8y ago

> but because fuck maintenance, fuck operations, fuck worrying about buffer overflow bugs in OpenSSL, I'll pay Amazon $N/month to do that for me, all that matters is the product. That's nice in theory, but my experience with paying big companies for promises like that is I still end up having to debug their broken software; and it's a lot easier to do that if I can ssh into the box. I've got patches into OpenSSL that fixed DHE negotiation from certain versions of windows (especially windows mobile), that I really don't think any one's support team would have been able to fix for me / my users [1], unless I had some really detailed explanation -- at that point, I may as well be running it myself. [1] And as proof that nobody would fix it, I offer that nobody fixed this, even though there were public posts about it for a year before my patches went in; so some people had noticed it was broken.

__jal 8y ago
This. I've been informally tracking time spent on systems my team manages and managed tools we use. (I manage infra where I work.) There is very little difference. And workarounds and troubleshooting spent on someone else's system mean we only learn about some proprietary system. That's bad for institutional knowledge and flexibility, and for individual careers. Our needs don't mesh well with typical cloud offerings, so we don't use them for much. When we have, there has yet to be a cost savings - thus far they've always cost us more than we pay owning everything. I mean, I personally like not dealing with hardware or having to spend time in data centers. But I can't justify it from a cost or a service perspective.
- pojzon 8y ago
  After amazon switched to ec2/sec kind of billing in 2017 i would say cloud is pretty good for prototyping and running CI. As to anything else i would say it depends.
sethammons 8y ago
absolutely. I am _right_now_ fighting ELB because we see traffic going to it but not coming out of it. If it were MY load balancer, I would just log in and tcpdump.

AnIdiotOnTheNet 8y ago

Yeah, "not my problem" sounds like a winning strategy, until you realize that it also means "nothing I can do about it" when things inevitably go wrong.

wang_li 8y ago
Not to mention that you're the lowest priority when something does go wrong.

msbarnett 8y ago

Enjoy the many (many, many) connection reset by peer, TCP idle timeouts, and 500 Internal Server Errors on Dynamo's end. Between that and the dogshit documentation, it's truly thrilling to be paying them a ridiculous amount of money for the privilege of having a managed service go undebuggably AWOL on the regular, instead of being able to resolve the issues locally.

StreamBright 8y ago
I can see my code handling all the exceptions correspond to the points you raised with exponential backoff. Which part of Dynamo's documentation is dogshit?
dilyevsky 8y ago
Yeah and if you’re on gcp you also get dogshit support whose main strategy is to sandbag you with endless (and useless) logs collection until you give up and move on or take too long so they can close the ticket.

tptacek 8y ago

Which Dynamo are you talking about? The Dynamo paper, or Dynamo, the AWS managed database product? Dynamo-the-product makes sense for all sorts of random tasks. Full-on implementations of Dynamo-the-paper (for instance: Cassandra) fall into the bucket of "understand the problem first" that the author is talking about.

mrmuagi 8y ago

What specific managed services do you use, if I may inquire?

milquetoastaf 8y ago

so what happens when they don't do what you want them to?

paganel 8y ago

> My goal is to be totally unable to SSH into everything that powers my app That sounds like a nightmare to me, and I'm not even a server guy, I'm a backend guy only managing my personal projects. > I'll pay Amazon $N/month to do that for me, all that matters is the product. I don't want to pay Amazon anything, I want to pay Linode/Digital Ocean 5 or 10 or 20 dollars per month and I can do the rest myself pretty well. My personal projects will never ever going to reach Google's scale and seeing that they don't bring me anything (they're not intended to) I'm not that eager to pay an order a magnitude more to a company like Amazon or Google in order to host those projects.

staticassertion 8y ago
As someone tasked with protecting infrastructure, devs not being able to SSH into production is a godsend.
- scaryclam 8y ago
  There's a difference from devs not being able to ssh in due to access restrictions and devs not being able to ssh in because it's not a feature of the service. I actually agree that not having unilateral access to the production system is a good thing. But not being able to have anyone able to be granted access, even for a limited time, is neither productive or safe.
- hackerpacker 8y ago
  so, what is to stop them from running code from the app?!? Who do you call when something breaks? Do you expect the dev to not be able to log in and investigate at 3am or whenever? I mean if you can't trust your dev team, or the procedures surrounding them, you are kinda screwed.
  - staticassertion 8y ago
    To be clear, the issue isn't about trusting developers in prod. It's about limiting access to prod as an avenue for attackers who will be looking to get in there and exfil sensitive data. Nothing to do with trust of devs.

StreamBright 8y ago

>> My goal is to be totally unable to SSH Great goal. I guess you never had any use case to login to a production server to investigate a business critical issue.

jessaustin 8y ago
It's more likely that parent has had to do that a lot of times, or perhaps has managed people who have. That would be a sign of organizational dysfunction, and fixing that is a good goal to have.
not_kurt_godel 8y ago
I have had to SSH into prod boxes many times to debug emergent issues. Now I'm using all serverless, and I get a happy fuzzy feeling whenever I think about how I never ever have to do that again.
- teraflop 8y ago
  "The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at and repair." -- Douglas Adams

pojzon 8y ago

Issue is, this works only for very simple systems. With complexity comes maintenance and operations. You also cannot build anything innovative in case of heavy load systems because you cannot embbed your custom highly performant solutions (cloud provider simply does not support them ootb). Serverless sounds great for "some" kind of systems but in other cases its a complete missfire. Our job is to know which tool to use.

geofft 8y ago

A SQL database, bash, and a Raspberry Pi in your closet all are managed too. They're built on top of software that's been battle-tested over decades to keep itself running. MySQL/Postgres/whatever will manage running multiple queries for you and keeping the data consistent as well as Dynamo will. bash will manage running multiple processes for you as well as Hadoop will. A Raspberry Pi with cron will manage starting a service for you every time it powers on as well as Lambda will. The reasons to use Dynamo/Hadoop/Lambda are when you've got different problems from the ones that MySQL/bash/cron solve. If you don't believe that this counts as "managed," then for all your words at the end, you believe at the end of the day that making a solution more complex so that failures are expected and humans are required is superior to doing a simple and all-in-code thing. For all that you claim to not like operations, you think that a system requiring operational work makes it better. You are living the philosophy of the sysadmins who run everything out of .bash_history and Perl in their homedir and think "infrastructure as code" is a fad.

eloisant 8y ago
With this definition unless you write your own database system, everything is managed. That makes it pretty useless to qualify anything as managed. The Raspberry Pi is managed by you, as opposed to managed by a different company / different people so you can focus on your business problem.

apoorvgarg 8y ago

Before we make a technology choice, we should be clear what those choices are. SQL is a query language and DynamoDB is a database. "NoSQL" technologies can be fronted with SQL interfaces. Relational databases can vary a lot too in their ACID compliance ("I" being the component with maximum variance). The choice of technology should first be based on the problem and not whether it is serverless. Choosing DynamoDB when your application demands strict serializability would be totally wrong. I hope more people think about the semantic specifications of their applications (especially under concurrency).

BoorishBears 8y ago

To me this comment is not a response to the article. The article isn’t asking why use Dynamo vs SQL in the context of the traditional NoSQL vs SQL way It’s why Dynamo vs QLDB. Or why Cloud Firestone vs Cloud Spanner. Or Firehose + some lambda functions vs EMR It’s wholly separate of the managed angle and focuses on the actual complexity of the products involved and the trade offs therein. In the surface the two sound similar, but there’s much more nuance to this article's point (take the example of Dynamo, the author isn’t opposed to it for being NoSQL, they’re opposed to it for low level technical design choices that are a poor fit for the problem their client had)

gambler 8y ago

>SQL can be managed with, say, RDS. Sure. But it's not the same level of managed as Dynamo (or Firebase or something like that). It's managed at every shared hosting in the world.

atombender 8y ago
Not at the same level, hardly anywhere. Replication, auto-scaling, failover and point-in-time backups are typically not managed for you. AWS and GCP have decent solutions now, but they're not nearly as hands-off in terms of management.

OrgNet 8y ago

If you don't care about cost and don't care about someone else having all of your data, that is probably a fine route.

plufz 8y ago

If the main reason for your switch from sql to dynamo is that it is managed why not use aws aurora, aurora serverless, or any other more managed sql solution? Now when many sql databases support json I don’t quite understand why you would use a documentdb for anything else then for actual unstructured data, or key value pairs that needs specific performance. I used mongodb for a while now I’m 100% back in sql. Document dbs really increased my love for sql. But I’m also long from being a nosql expert and I never have to deal with data above a TB.

mattrp 8y ago

N gets to be a big number my friend. I love aws but I can’t get over the number of startups I’ve seen with $100k MRR and $35k MRC allocated to AWS.

Justsignedup 8y ago

It is important to understand the following: Depending on the problem, choose the right tools. For example so far everything I've seen about React and GraphQL tells me that maybe GQL isn't necessarily the best solution for a 5-person team, but a 100 person team may have a hard time living without it. Kubernates / Docker is significant work. And when we had 4 developers making that effort was wasteful and we literally didn't have the time. Now at 20 we're much more staffed and the things Kubernates / Docker solves is useful. Meanwhile we have a single PostgreSQL server. No nosql caching layer. We're at a point where we aren't sure how far we gotta go before we hit limits of a PSQL server, but at least 3-4 years before we hit it. Point is look at the tools. See where these tools win it for you. Don't just blindly pick because it can scale super well. Pick it because it gives you things you need and a simplicity you need for the current and near-term context, and have a long-term strategy how to move away from it when it is no longer useful.

theonemind 8y ago

If everyone did this, in the limiting case, no one has the ability to investigate and repair actual bugs until the point where the application can only crash every 10 ms and does nothing but get continually restarted.

semi-extrinsic 8y ago

> My goal is to be totally unable to SSH into everything that powers my app. If the 0day in your familiar pastures dwindles, despair not! Rather, bestir yourself to where programmers are led astray from the sacred Assembly, neither understanding what their programming languages compile to, nor asking to see how their data is stored or transmitted in the true bits of the wire. For those who follow their computation through the layers shall gain 0day and pwn, and those who say “we trust in our APIs, in our proofs, and in our memory models and need not burden ourselves with confusing engineering detail that has no value anyhow” shall surely provide an abundance of 0day and pwnage sufficient for all of us. Thus preacheth Pastor M. Laphroaig.

tigershark 8y ago

Serverless is not a panacea, actually most of the times is true the opposite. You will spend much more time dealing with configuration, deployments, managing cold starts, orchestrating everything. Sure, for some use cases is perfect, but I can tell you almost certainly that the future won’t be on AWS lambda &co.

collyw 8y ago
Its moving complexity from the application layer to the deployment layer.
- guru4consulting 8y ago
  which is awesome. Application complexity is harder to solve than deployment/infrastructure complexity. And more often the cloud providers would solve the deployment complexities over time.

debaserab2 8y ago

But these services do break, and when they do break, often times you don't even get a useful error or log file to grep. Sometimes I just really wish I could SSH into a box to figure out what the hell is going on. You're gaining an expertise, but instead of it being with the underlying technology, it's with the management services for that technology. I think the premise of your argument is correct that understanding these services is simpler than the technology itself, but I seem to inevitably find myself in some edge use case of the service that is poorly documented or does not work as described, and trolling stack overflow or AWS developer forums is the only solution other than trial and (no) error. Let me tell you: it's not fun when you're entrenched into a proprietary service that is now outdated and kept only for legacy reasons, and the version you are on doesn't allow you to do things like update your programming language to a more recent version. Now you get to move to another managed service and lose all that expert knowledge you had with the old one.

goatlover 8y ago

> My goal is to be totally unable to SSH into everything that powers my app. I'm not saying that I want a stack where I don't have to. I'm saying that I literally cannot, even if I wanted to real bad. So you want technology that prevents you from doing something you might want to do, real badly?

Juliate 8y ago

If you go server less because fuck accountability, shouldn't the extension be jobless because someone else will provide the service you did? In that way, cloud/serverless migration is really the whole worldwide IT workforce progressively willingly migrating IT (design, development, operations & applications) out of companies, to just a few big providers. Just like energy. And who gets to decide destinies in the world? Those who control, have and provide energy. What a time to be alive. End of XIX century was the rush to oil-based energy to control the world. End of XX was the rush to data-based capture and control.

mrblues 8y ago

I recommend heroku and heroku postgres

deleted 8y ago

[deleted]

nukeop 8y ago

Using Dynamo is the worst thing we've done for productivity and costs in our product. It introduces onboarding overhead, unnecessary implementation time overhead, costs a lot, and is always a bottleneck in every operation. Using DAX requires paying for VPC which already costs more than a server that could power many small to mid-sized products. Resizing table throughputs requires lots of manual work to keep it cost effective, because the default settings are designed to nickel and dime you to death. If we just used a simple SQL or MongoDB server attached as a container we'd save literally months of development time and thousands of dollars per month.

expliced 8y ago
This sounds beyond painful. We have similar problems at work with expensive system that I believe we could reimplement with traditional technology, but we don't.

sureaboutthis 8y ago

I see that HN now steps into the gutter of reddit-like language and can damn all professional decorum.

ivanhoe 8y ago

> My goal is to be totally unable to SSH into everything that powers my app. In reality that means that when something doesn't work as expected, instead of logging in and solving the problem, you'll be spending your time writing tickets and fighting through the tiers of support and their templated replies, waiting for them to eventually escalate your issue to some actual engineer. Wish you a lot of luck with that, I prefer being able to fix my own stuff (or at least give it a try first).

yowlingcat 8y ago

Well, if all you want is managed, per request hosting of SQL, use Aurora serverless! It's a game changer. As for Dynamo vs SQL -- you really cannot substitute one for the other. They're different tools for different jobs. Dynamo is extremely low latency and great for that purpose, while SQL is great for being a store of record, and if you use it as long as you can get away with it for as many things as possible, it's usually not a bad idea.

mcguire 8y ago

... Fuck debuggablity. Fuck performance. Fuck understanding what is actually going on.... (Sure, managed solutions can satisfy all of those. But not without understanding a few orders of magnitude more complexity than a simpler solution, and you'll never get there if your attitude is "I'll pay Amazon to do that for me." Amazon doesn't care whether you succeed.)

empthought 8y ago
> Amazon doesn't care whether you succeed. Wait, what? Why wouldn't they? Successful companies will spend money with them; defunct companies can't.

oarabbus_ 8y ago

>"why are you using Dynamo, just use SQL" On the other hand, I find it frustrating how many developers either don't know basic SQL or try to get cute and use the flat side of an axe to smack down nails instead of just using the well-known, industry-standard beauty of a hammer that SQL.