slotrans

10 ms·

91.
by slotrans 3y ago
Basically correct. If you have less than tens-of-terabytes the big exciting stuff just isn't needed and isn't particularly helpful. Focus on understanding your data and building a useful model. Use Postgres if you can. Supplement
92.
by slotrans 3y ago
See also this piece from 2017, of the same title and commenting on the same book https://samzdat.com/2017/06/28/without-belief-in-a-god-but-n... This is part of a series of articles called "Uruk Machines
93.
by slotrans 3y ago
> ...do we rememember why we stopped developing web pages that way? Because we needed an API for our mobile apps, and once we had that, SPAs seemed convenient. It was definitely, absolutely NOT because the page-centric way of building si
94.
by slotrans 3y ago
Come after who? And how?
95.
by slotrans 3y ago
I wasn't familiar with StarRocks so thanks for calling attention to it. It appears to make very different tradeoffs in a number of areas so that makes it a potential useful alternative. In particular transactional DML will make it much
96.
by slotrans 3y ago
Impressive that this works. It's so small! Also: absolute endorsement for anything and everything James Hoffmann
97.
by slotrans 3y ago
Yeah ClickHouse is definitely the way to go here. Its ability to serve queries with low latency and high concurrency is in an entirely different league from Snowflake, Redshift, BigQuery, etc.
98.
by slotrans 3y ago
Just gonna keep linking this til the heat death of the universe: https://mcfunley.com/whom-the-gods-would-destroy-they-first-... Real-time analytics are worse than useless . At best they are a distracting resource sink, at
99.
by slotrans 3y ago
Yes. Just write SQL. SQL is good. Never use an (active record) ORM. Ever. For any purpose. They are a disastrous idea that should be un-invented. I have this discussion with a lot of people who claim they cannot imagine working without an O
100.
by slotrans 3y ago
It's not a DBT alternative if it's not SQL-oriented.
101.
by slotrans 3y ago
I assume these storie-lets are true but the timeline is wrong, it was much more than 10 years ago. I went to an Adrian Cockroft talk in 2012: they had thousands of engineers and hundreds of services at that time, Asgard already existed and
102.
by slotrans 3y ago
quit redesigning it and spend the effort making it use MB of RAM instead of GB if BitchX and mIRC can do it, Slack can do it, but they have to try
103.
by slotrans 3y ago
> In our experience, Kafka is one of the most polarizing technologies in the data space. Some people hate it, some people swear by it, but almost every technology company uses it. (emphasis added) Surely that's false? Or, I mean,
104.
by slotrans 3y ago
Because a series of print/log statements can illustrate the evolution of state over time, where a debugger only shows you "now".
105.
by slotrans 3y ago
no kidding: opening Google Meet in Firefox on Windows instantly crashes it, every window. As I routinely keep 300+ tabs open, this makes me sad. since FF is my main browser I have to take care to manually open Meet links in Chrome
106.
by slotrans 3y ago
Jenkins is honestly Really Good as a scheduling system. The best, frankly. If you can combine it with a library- or tool-style DAG system (vs. server-style like Airflow), like make or Luigi or NextFlow or even Step Functions, that is a gr
107.
by slotrans 3y ago
The Prefect guys wrote this https://www.prefect.io/guide/blog/why-not-airflow it's something but far from comprehensive. There's not a lot of writing about this. Folks seem content to fight Airflow'
108.
by slotrans 3y ago
It's trivially easy to beat Snowflake on latency because its latency is truly awful. It often takes 1-4 SECONDS end-to-end to run a query that touches just a few thousand (not million!) rows. In theory this is fine for OLAP but when
109.
by slotrans 3y ago
We were paying over $1M/yr for it inclusive of support. One real production instance (with Veritas Cluster), one reporting instance, 3 dev/QA instances. This was circa 2012, when the company had roughly $300M/yr in revenue. I
110.
by slotrans 3y ago
It's on my list for sure. The hard requirement of running on K8s makes me sad in advance, though.
111.
by slotrans 3y ago
> I like SQL - there, I said it!! SQL is the only, only tech thing I love. Other stuff might be fine tools to accomplish various goals but SQL is Good and Beautiful and True.
112.
by slotrans 3y ago
Cookie banners are absolutely the fault of the EU. Users have always been in control of whether they accept cookies. There have been settings in your browser since (at least) Netscape 3.0. It's only because of dumb EU laws that cooki
113.
by slotrans 3y ago
Yeah, it's this. Having worked for a number of companies implementing these measures, there's no malicious intent, they are rolling their eyes the whole time. It's just a box they need to tick. Everyone wishes it would just g
114.
by slotrans 3y ago
OLTP databases. I've used Oracle, MySQL, and Postgres in anger (in that order, historically), plus DynamoDB and MongoDB if you want to count those. I have to admit Oracle has some technical superiority but the pricing is insane. MySQL
115.
by slotrans 3y ago
$0.084/kWh in Colorado (in summer, lower in winter)
116.
by slotrans 3y ago
The first time I hit bugs trying to run a JRE on Alpine because of musl, I threw it away and never looked back. I understand there are musl builds of Java these days but I don't care, the whole thing is just an unnecessary headache.
117.
by slotrans 3y ago
Massive fan of this piece. I linked it in another comment yesterday. The place its truth has been most obvious to me is in analyzing subscription businesses. Your customers pay once a month, or once a year. Nothing, absolutely nothing you
118.
by slotrans 3y ago
No, he got it exactly right. It is just as relevant today as when it was written.
119.
by slotrans 3y ago
I've built a multi-armed bandit system which lived alongside our A/B system. 1. Product didn't have any idea how to interpret its behavior and therefore never made any decisions based on it 2. Experimentation != product desig
120.
by slotrans 3y ago
Hot takes: SQL is good. Great, in fact. Not just in what it's capable of doing, but in form . It has warts, yeah, but over the years I've been writing it I've realized most of them are there for a reason. To me it's mor

More ›