phamilton

8 ms·

61.
by phamilton 2y ago
To handle 25k qps with Rails, assuming 500ms per request, you need 12k Ruby processes. That probably means you run on 200 boxes with 64 CPUs each. Maybe you get a little bit of oversubscribing of processes to CPUs due to I/O. Conservat
62.
by phamilton 2y ago
Beyond application performance, I've also found a non-trivial drag from the complexity of scaling Rails horizontally. Databases need connection pools, which must be external. Processes vs threads ratio need to be tuned and retuned and
63.
by phamilton 2y ago
The language is incredibly flexible and allows for "DSLs" that are just ruby libraries. A simple example: `3.days.ago` is a very commonly used idiom in Rails projects. Under the hood, it extends the base Number class with `def day
64.
by phamilton 2y ago
http://www.supersimplestorageservice.com/ exists and calls itself S4. It's a decent gag and the immediately came to mind when I heard S2 vs S3.
65.
by phamilton 2y ago
I introduced my kids (13 and 11) to the sofa problem last week as we installed this cabinet organizer from IKEA: https://www.ikea.com/us/en/p/utrusta-corner-base-cab-pull-ou... After showing them a youtube vi
66.
by phamilton 2y ago
An interesting project for niche types: https://github.com/rick-de-water/nonany It's not perfect but does allow some flexibility.
67.
by phamilton 2y ago
Asserts are convenient, but every time I encounter them I ask "Can the compiler prove this can't happen?" In the example of "min(ages) > 0", making age a NonZero type renders the assert unnecessary. Rust even has
68.
by phamilton 2y ago
I'd add that cloud services are dramatically cheaper when your application is able to tolerate preemption. Spot pricing in AWS is around $0.015/vcpu/hr. That still going to be more expensive than a dedicated server, but it&#x
69.
by phamilton 2y ago
Thanks for sharing. It matches my experience. After many years of a lean team serving high scale traffic (> 1 million monthly active users per engineer), most abstractions between customer and data seem to turn into performance liabiliti
70.
by phamilton 2y ago
Has anyone ever actually moved a mature application from one database to another and found that the code around calling the DB was a major painpoint? I'm all for the unit testing argument, but in an active SaaS business I've never
71.
by phamilton 2y ago
Yes. Running multiple multi-billion row Postgres DBs on AWS Aurora. The number of rows isn't that consequential honestly. Btrees are very fast at navigating deep hierarchies. It's the volume and complexity of traffic that matters.
72.
by phamilton 2y ago
A testament to the quality of Anandtech is that in 2011 I started a job at Micron on their SSD team and the first thing they said was to go read some articles on anandtech about how SSDs work. They covered slc vs mlc, trim, etc in better de
73.
by phamilton 2y ago
Postgres + Rust is one of the most exciting intersections of tech I've seen in a while. There's external tooling like his project, but postgres extensions in Rust are exciting. Full extensions via pgrx have been cool to see, but p
74.
by phamilton 2y ago
Very erlang/otp. Joe Armstrong used to rant to anyone who would listen that we used databases too often. If data was important, multiple nodes probably need a copy of it. If multiple nodes need a copy, you probably have plenty of durab
75.
by phamilton 2y ago
I've been out of the space for about a decade, so I can't say how things have continued to evolve, but the only consistent antidote to fraud has been a trusted third party. I spent a lot of my time back then pulling in metrics fro
76.
by phamilton 2y ago
As a college student trying to figure out where I wanted to go next (industry vs academia) I had an epiphany that there was intrinsic value in doing things. Even things that were neither novel (academia), profitable (industry), nor impactfu
77.
by phamilton 2y ago
When flash started to fade for games, many developers moved over to adtech where their skills were still valued. Flash continued for a few years to power ads as well as shims around other videos that would collect data and even trigger seco
78.
by phamilton 2y ago
One of the final nails was the infamous Chrome 45 aka the Chromepocalypse in the video ad world. Chrome 45, in the name of performance, defaulted to only loading flash from 3rd party domains after a "click to load". This was bad f
79.
by phamilton 2y ago
> indexes often don't help. An "index on expression" should perform the same regardless of the input column types. All that matters is the output of the expression. Were you just indexing the whole jsonb column or were you
80.
by phamilton 2y ago
A few things: Redis Data Tiering - Redis Enterprise and AWS Elasticache for Redis support data tiering (using SSD for 80% of the dataset and moving things in and out). On AWS, a cache.r6gd.4xlarge with 100GB of memory can handle 500GB of da
81.
by phamilton 3y ago
I mostly have used AWS Aurora there, which is significantly better than vanilla MySQL or Postgres and both are similar enough. In Aurora, Postgres has Aurora Limitless (in preview) which looks pretty fantastic. As far as running yourself, P
82.
by phamilton 3y ago
Some things that are nice in Postgres and missing in MySQL: - Create constraints as NOT VALID and later VALIDATE them. This allows you to create them without expensive locks. - explain (analyze, buffers). I miss this so much. - Row level se
83.
by phamilton 3y ago
The goal isn't to enforce a certain query plan. The goal is to execute queries efficiently. The planner/optimizer is good at its job. In general, better than I am. If I can inform it about a particular shape of data, all queries a
84.
by phamilton 3y ago
`set local` allows tuning query planner settings scoped to a transaction, so you don't have to do it globally.
85.
by phamilton 3y ago
All the knobs for query planning and cost estimation can be scoped to the transaction with `set local <key>=<value>`.
86.
by phamilton 3y ago
That's valuable, but cranking statistics up to 10000 and/or creating custom statistics can go a very long way to helping the planner understand the dataset more fully.
87.
by phamilton 3y ago
I like it as an emergency measure, but I often see them used when there's a shallow understanding of operating the db. Before using a hint or rewriting a query to force a specific plan, I try and push the team to do these things: 1. Ru
88.
by phamilton 3y ago
https://www.pgmustard.com/ is a great tool. There are some free alternatives for the actual tool, but their documentation is pretty great. For example, https://www.pgmustard.com/docs/explain/heap-f
89.
by phamilton 3y ago
FWIW, I've had to track down licensing and proof of IP assignment for fundraising rounds too.
90.
by phamilton 3y ago
> Not sure how flying a plane while it's essentially doing nothing is helpful to anyone. When we train engineers for oncall, we take a look at incidents they saw while shadowing. Just X days shadowing doesn't really cut it, the

More ›