6 ms·
> Both postgres and redis are used with the out of the box settings Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting softw
by ezekiel68 1y ago
> Both postgres and redis are used with the out of the box settings
Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously.
Some will say that in a container-orchestrated environment, tuning goes out the window since "you never know" where the orchestrator will schedule the service but this is bogus. If you've got time to write a basic deployment config for the service on the orchestrator, you've also got time to at least size the memory usage configs for PostgreSQL and/or Redis. It's just that simple.
This is the kind of thing that is "hard and tedious" for only about five minutes of LLM query or web search time and then you don't need to revisit it again (unless you decide to change the orchestrator deployment config to give the service more/less resources). It doesn't invite controversy to right-size your persistence services, especially if you are going to publish the results.
- wewewedxfgdf 1y agoFully agree. Postgres is a power tool usable for many many use cases - if you want performance it must be tuned. If you judge Postgres without tuning it - that's not Postgres being slow, that's the developer being naive.
- gopalv 1y ago> If you judge Postgres without tuning it - that's not Postgres being slow, that's the developer being naive. Didn't OP end by picking Postgres anyway? It's the right answer even for a naive developer, perhaps even more so for a naive one. At the end of the post it even says >> Having an interface for your cache so you can easily switch out the underlying store is definitely something I’ll keep doing
- lelanthran 1y agoHe concluded postgresql to be fast enough, so what's the problem? IOW, he judged it fast enough.
- high_na_euv 1y agoDisagree, majority of software is running on defaults, it makes sense to compare them this way
- vidarh 1y agoOn one hand I agree with you, but on the other hand defaults matter because I regularly see systems with the default config and no attempt to tune. Benchmarking the defaults and benchmarking a tuned setup will measure very different things, but both of them matter.
- matt-p 1y agoIME very very few people tune the underlying host. Orgs like uber, google or whatever do but outside of that few people know what they're really doing/cares that much. Easier to "increase EC2 size" or whatever.
- kijin 1y agoDefaults have all sorts of assumptions built into them. So if you compare different programs with their respective defaults, you are actually comparing the assumptions that the developers of those programs have in mind. For example, if you keep adding data to a Redis server under default config, it will eat up all of your RAM and suddenly stop working. Postgres won't do the same, because its default buffer size is quite small by modern standards. It will happily accept INSERTs until you run out of disk, albeit more slowly as your index size grows. The two programs behave differently because Redis was conceived as an in-memory database with optional persistence, whereas Postgres puts persistence first. When you use either of them with their default config, you are trusting that the developers' assumptions will match your expectations. If not, you're in for a nasty surprise.
- vidarh 1y agoYes, all of this is fine but none of it address my point: Enough people use the default settings that benchmarking the default settings is very relevant. It often isn't a good thing to rely on the defaults, but it's nevertheless the case that many do. (Yes, it is also relevant to benchmark tuned versions, as I also pointed out, my argument was against the claim that it is somehow unfair not to tune)
- IanCal 1y agoI disagree. They found that Postgres, without tuning, was easily fast enough on low level hardware and would come with the benefit of not deploying another service. Additionally tuning it isn’t really relevant. If the defaults are fine for a use case then unless I want to tune it for personal interest it’s either a poor use of my fun time or a poor use of my clients funds.
- lemagedurage 1y ago"If we don't need performance, we don't need caches" feels like a great broader takeaway here.
- motorest 1y ago> "If we don't need performance, we don't need caches" feels like a great broader takeaway here. I don't think this holds true. Caches are used for reasons other than performance. For example, caches are used in some scenarios for stampede protection to mitigate DoS attacks. Also, the impact of caches on performance is sometimes negative. With distributed caching, each match and put require a network request. Even when those calls don't leave a data center, they do cost far more than just reading a variable from memory. I already had the displeasure of stumbling upon a few scenarios where cache was prescribed in a cargo cult way and without any data backing up the assertion, and when we took a look at traces it was evident that the bottleneck was actually the cache itself.
- Timshel 1y ago> for only about five minutes of LLM query or web search I think I have more trust in the PG defaults that in the output of a LLM or copy pasting some configuration I might not really understand ...
- huflungdung 1y ago[dead]
- rollcat 1y agoIt's crazy how wildly inaccurate "top-of-the-list" LLMs are for straightforward yet slightly nuanced inquiries. I've asked ChatGPT to summarize Go build constraints, especially in the context of CPU microarchitectures (e.g. mapping "amd64.v2" to GOARCH=amd64 GOAMD64=v2). It repeatedly smashed its head on GORISCV64, claiming all sorts of nonsense such as v1, v2; then G, IMAFD, Zicsr; only arriving at rva20u64 et al under hand-holding. Similar nonsense for GOARM64 and GOWASM. It was all right there in e.g. the docs for [cmd/go]. This is the future of computer engineering. Brace yourselves.
- yomismoaqui 1y agoIf you are going to ask ChatGPT some specific tidbit it's better to force it to search on the web. Remember, an LLM is a JPG of all the text of the internet.
- dgfitz 1y agoWait, what? Isn't that the whole point, to ask it specific tidbits of information? Are we to ask it large, generic pontifications and claim success when we get large, generic pontifications back? The narrative around these things changes weekly.
- wredcoll 1y agoI mean, like most tools they work when they work and don't when they fail. Sometimes I can use an llm to find a specific datum and sometimes I use google and sometimes I use bing. You might think of it as a cache, worth checking first for speed reasons. The big downside is not that they sometimes fail, its that they give zero indication when they do.
- oulipo2 1y agoPerhaps, but in this case this shows at least that even non-tuned Postgres can be used as a fast cache for many real-world use-cases
- deleted 1y ago[deleted]
- conradfr 1y agoBut why doesn't Postgres tuned itself based on the system is running on, at least the basics based on available RAM & cores?
- simonw 1y agoI've not tried it myself but I believe that's what pgtune does: https://github.com/gregs1104/pgtune https://github.com/gregs1104/pgtune
- otikik 1y ago> Ugh. > if you want to be taken seriously For someone so enthusiastic about giving feedback you don't seem to have invested a lot of effort into figuring out how to give it effectively. Your done and demeanor diminish the value of your comment.
- ezekiel68 1y agoThanks for the feedback. I'll definitely think about it.
- deleted 1y ago[deleted]
- GuinansEyebrows 1y ago> This is the kind of thing that is "hard and tedious" for only about five minutes of LLM query or web search time not even! if you don't need to go super deep with tablespace configs or advanced replication right away, pgtune will get you to a pretty good spot in the time it takes to fill out a form. https://pgtune.leopard.in.ua/ https://pgtune.leopard.in.ua/ https://github.com/le0pard/pgtune https://github.com/le0pard/pgtune
- ezekiel68 1y agoGlad you mentioned these resources. I was soft-pedaling it to avoid the "Look, a gatekeeper!" brigade.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- rich_sasha 1y agoIsn't tuning before hitting constraints a premature optimisation? The approach of not spending time on tuning settings before you have to seems sane. And TFA shows you that in this world Postgres is close enough to Redis.
- aprdm 1y agoYep. I worked in a famous-big-company that had a 15 years old service that was dogslow, systemd restarts would take multiple hours. Everyone was talking about C++ optimizations, mutex everywhere etc - which was in fact a problem. However.. I seemed to be the first person to actually try to debug what the database was doing, and it was going to disk all the time with a very small cache.. weird.. I see the MySQL settings on a 1TB ram machine and they were... out-of-the-box settings. With small adjustments I improved the performance of this core system an order of magnitude.
- icedchai 1y agoAt one startup, all I did was increase the innodb buffer pool size. They were using default settings.