hodgesrm

11 ms·

121.
by hodgesrm 1y ago
Right, that's the archive case. One way to handle that is to replicate the log records and let users figure it out in the analytic storage. That also allows time travel. Either way the data should be frozen or it makes the problem way
122.
by hodgesrm 1y ago
I don't really understand what problem replication of mutable transaction data to Iceberg or any data lake for that matter solves for most PostgreSQL users. Iceberg is optimized for fact data in very large tables with relatively rare c
123.
by hodgesrm 1y ago
Seems like this is going to be a permanent issue, no? Library level storage APIs are complex and often quite leaky. That's based on looking at the innards of MySQL and ClickHouse for a while. It seems quite possible that there will be
124.
by hodgesrm 1y ago
This Google article was nice as a high level overview of Iceberg V3. I wish that the V3 spec (and Iceberg specs in general) were more readable. For now the best approach seems to be read the Javadoc for the Iceberg Java API. [0] [0] https:
125.
by hodgesrm 1y ago
Thanks. That's my definition of "use freely." The argument about what's open source mostly boils down to whether you accept these terms.
126.
by hodgesrm 1y ago
> SaaS, meanwhile, is the least open and least free model of software distribution. Less free and less open than Oracle DBMS? That seems like a stretch.
127.
by hodgesrm 1y ago
> Even if people somehow get this to work like a real database, they will quickly become frustrated with the same features that the data lake avoids: access controls, locks, consistency, centralization, a schema, you name it. Are you say
128.
by hodgesrm 1y ago
> When corporations that actually have a market test and profit motive are funding the research, avenues that are unlikely to succeed will be cut off sooner... It's hard to see how your suggestion would work for fundamental advances
129.
by hodgesrm 1y ago
Sounds interesting! What kind of query latency do you see with this approach? Also, have you thought about caching? My team is working on a similar problem and we have caches for everything from contents of S3 list_objects_v2 calls to Parqu
130.
by hodgesrm 1y ago
As the Romans say, "there's no point arguing about taste." I find Gibson's stories hard to follow in some cases, but at the same time he is very inventive when it comes to language and mood. I find it worth the effort. T
131.
by hodgesrm 1y ago
> It was ridiculous. They seriously wanted to rewrite everything in Java, including office and web browsers. There's another perspective. Many people were looking for something like Java well before it was released: VM-based, port
132.
by hodgesrm 1y ago
One question that the article does not cover: compaction. Adding custom indexes means you have to have knowledge of the indexes to compact Parquet files, since you'll want to reindex each time compaction occurs. Otherwise the indexes
133.
by hodgesrm 1y ago
Not sure why this is downvoted. There’s now abundant evidence it’s happening.
134.
by hodgesrm 1y ago
That really should be a word: kind of a portmanteau that combines hire and raise up/promote.
135.
by hodgesrm 1y ago
This. I recall an interview with Gibson where he said that he hung out in watering holes frequented by IT folks. He spent a lot of time picking up the atmosphere and cadence of the language, which he imitated in his works.
136.
Air India Crash Report Confirms Fuel Was Cut Off Before Accident(wsj.com)
4 points by hodgesrm 1y ago|1 comments
137.
by hodgesrm 1y ago
> Local-first apps, on the other hand, have better privacy and security built in at the core. I love this article, but the section on security raised a lot of questions. What's the model for authorizing access to documents for colla
138.
by hodgesrm 1y ago
Here's how we do it for analytic systems: data and software. The software services are open source running on Kubernetes. If you don't like the vendor or the vendor goes away, the existing services keep running. You can also mai
139.
by hodgesrm 1y ago
Think like an engineer to solve the problem. You could start by adjusting the beer-to-lad ratio and see where that gets you.
140.
by hodgesrm 1y ago
> * Data portability guarantees: Vendors must spell out how data gets migrated out, and all formats must be either open or (at minimum) fully documented. This is not practical for data of any size. Prod migrations to a new database take
141.
by hodgesrm 1y ago
I'm glad you answered as my comment was not very complete. It left out mentioning that transactions should not be expensive on reads unless you are doing something wrong. In general MVCC--which PostgreSQL uses--does not have a lot of p
142.
by hodgesrm 1y ago
This analysis seems wrong for at least a couple of reasons. 1. Freight is easier to manage and has better economics on a dedicated network. The US freight network is extremely efficient as others have pointed out. Other networks, e.g., Germ
143.
by hodgesrm 1y ago
> Clickhouse has much less transactional guarantees ... Is this relevant? The benchmark is just reads.
144.
by hodgesrm 1y ago
> At this rate, we'll soon see YC startups raising money to maintain roads. That model has existed for hundreds of years. It's called a turnpike. There were hundreds of them in the US and UK. [0] Many still exist today. [0] ht
145.
by hodgesrm 1y ago
> It can require approval from the legal department and a high level official from the company to approve updating it and the verbiage used on the status page. Is that true in this case or are you speculating? My company runs a cloud pla
146.
by hodgesrm 1y ago
Montaigne's Essays are primarily philosophical reflections. He did write a very interesting travel diary recounting his voyage from France to visit the Pope in Rome. [0] That diary has a wealth of observations covering everything from
147.
by hodgesrm 1y ago
I suspect most people who read unabridged Pepys for any length of time end up feeling very ambivalent about him as a person. His diary contains the some of the deepest introspection of a human life--both the good and the bad--in the histo
148.
by hodgesrm 1y ago
> More seriously, it's a columnar data store, not a relational database. Could you explain why you don't think ClickHouse is relational? The storage is an implementation detail. It affects how fast queries run but not the query
149.
by hodgesrm 1y ago
One of the tradeoffs for ClickHouse versus databases like Snowflake is that you have to have some knowledge about the internals to use it effectively. For example, Snowflake completely hides partitioning but on the other hand it does not de
150.
by hodgesrm 1y ago
It's used in production by many thousands of companies at this point. The ClickHouse Inc numbers are just a fraction of the total users. p.s., It's also possible to break ClickHouse as you demonstrated. It used to be a lot easier.

More ›