georgewfraser

12 ms·

91.
by georgewfraser 6y ago
Materialize has tackled the hardest problem in data warehousing, materialized views, which has never really been solved , and built a solution on a completely new architecture. This solution is useful by itself, but I'm also watching
92.
by georgewfraser 6y ago
The thing that has really surprised me is that public health officials are expressing so much confidence that masks work, despite the evidence being quite meh [1]. There have been many attempts to demonstrate that masks prevent transmission
93.
by georgewfraser 6y ago
Visual ETL is not as good of an idea as it seems at first. You end up putting a ton of business logic into the menus of these tools, and it’s not version controlled, and it’s not searchable. You’re better off doing this kind of work in SQL.
94.
by georgewfraser 6y ago
Welcome to HN Andy! As you are witnessing, people find that actually reading the links can be a real impediment to hot takes...
95.
by georgewfraser 6y ago
There’s F# language server, which is not as featureful as ionide but a lot faster.
96.
by georgewfraser 6y ago
The DKIM headers from the emails have been verified, so they are real: https://twitter.com/erratarob/status/1322007153415200768?s=2...
97.
by georgewfraser 6y ago
When I was in grad school I was a TA for a biochemistry class and the prof asked me to deliver the lecture on crystallography while he was on vaca. Part of the lecture was about how electron microscopy was easier but pretty inaccurate compa
98.
by georgewfraser 6y ago
It seems to me that under this scheme, if I make a single erroneous identify call, I will irreversibly merge two users. This is a surprising approach. Given that identify calls may occasionally be wrong, I would expect that identify(ano
99.
by georgewfraser 6y ago
Do I understand correctly that when someone calls identify(anonymous_id, new_canonical_id) you merge the entire set of anonymous ids associated with find(anonymous_id) with the set of anonymous ids associated with new_canonica
100.
by georgewfraser 6y ago
In order to grow into this valuation, snowflake will need to become the biggest data warehouse company ever. They have a great product and that will probably happen! But you need to be highly confident of the best case scenario happening to
101.
by georgewfraser 6y ago
For sure, but you're not going to fix that by making your own data lake using, for example, Parquet-on-S3. You're still going to pay the cost of compute when you analyze that data, and a well-optimized commercial database system i
102.
by georgewfraser 6y ago
I agree that no matter what you're going to store unstructured binary data, like audio files, in object storage. But that is perfectly compatible with storing structured data in a relational database. > they are great if the use cas
103.
by georgewfraser 6y ago
> Snowflake and BigQuery are quite expensive for big datasets. This is just flatly untrue---they are nearly the same cost/TB as object storage, and they store everything in compressed columnarized format, so they're about as ef
104.
by georgewfraser 6y ago
I am truly baffled by the desire to build new data lakes. I understand why Hadoop data lakes came to be: Teradata charged a lot for storage. But now there are database systems that use object storage under-the-hood: Snowflake, BigQuery, R
105.
by georgewfraser 6y ago
OK, my comment was over the top, apologies to Kleppmann if you’re still reading. He is a thought leader, and he’s the first introduction to distributed systems for a lot of people, and sometimes people stop with his work and don’t learn how
106.
by georgewfraser 6y ago
It’s a trivial observation that tells you nothing about the trade offs of building distributed systems.
107.
by georgewfraser 6y ago
Not that I know of. His lectures are amazing though.
108.
by georgewfraser 6y ago
YES!
109.
by georgewfraser 6y ago
Klepmann is a wannabe thought-leader, who repackages old ideas into pseudo-profound “insights”, designed to capture the attention of amateurs rather than move the state of the art forward. He’s the Nassim Taleb of distributed systems. If yo
110.
by georgewfraser 6y ago
The CAP theorem is so dumb and yet so influential. If you want to gain a more useful understanding of the trade offs of distributed systems, I highly highly recommend the “PACELC theorem” by Daniel Abadi: https://www.cs.umd.edu&#
111.
by georgewfraser 6y ago
Andy Pavlo talks about this in his class at CMU. You shouldn’t expect to get better performance by running a disk-optimized storage engine on memory, because you’re still paying all the overhead of locks and pages to work around the latency
112.
by georgewfraser 6y ago
Postgres foreign data wrappers is a weird choice of engine. Most queries to this service will be scans, in which case a column-oriented, vectorized, massively parallel engine like Presto will be 1000 times faster or so. Postgres’ underlying
113.
by georgewfraser 6y ago
I find it baffling that SoftBank isn’t simply taking ARM public. If ever there was a company that should be independent, it’s ARM. Is there a good explanation out there for why this isn’t the obvious choice?
114.
by georgewfraser 6y ago
I think people overestimate the importance of failures of the underlying cloud platform. One of the most surprising lessons of the last 5 years at my company has been how rarely single points of failure actually fail . A simple load-balanc
115.
by georgewfraser 6y ago
Yes
116.
Ask HN: No-code tool for creating API shims?
5 points by georgewfraser 6y ago|1 comments
117.
by georgewfraser 6y ago
It’s much faster for analytical queries, because it’s a column store rather than a row store.
118.
by georgewfraser 6y ago
ksql does not solve any of the hard consistency or contention problems you will face if you attempt to use Kafka as a datastore. Consider the simplest possible example: you write an "update event" to a topic and then read a ksql v
119.
by georgewfraser 6y ago
This notion of “stream-table duality” might be the most misleading, damaging idea floating around in software engineering today. Yes, you can turn a stream of events into a table of the present state. However, during that process you will e
120.
by georgewfraser 6y ago
I think we all know what happened here: Nassim Taleb took an IQ test, it didn’t show him to be the great genius he knows himself to be, and so the entire concept of IQ must be invalid.

More ›