zX41ZdbW

31 ms·

481.
by zX41ZdbW 3y ago
There are parts of C++ standard library that no one should ever use. The examples are: regex, iostreams, locale... My main concern - this can also become such a dead weight.
482.
by zX41ZdbW 3y ago
The best way I've found is - patching LLVM's libunwind to make it fully async-signal safe, and sending the stack trace to another thread for symbolization. This is implemented in ClickHouse.
483.
by zX41ZdbW 3y ago
There should not be such a distinction between in-memory and disk based databases. Being in-memory does not make OLAP databases faster. And typically, "in-memory" is a disadvantage and a limiting factor in OLAP database architectu
484.
by zX41ZdbW 3y ago
It's not a problem with SQL. For example, in ClickHouse, you can write FROM table SELECT ... But this is not a big deal.
485.
by zX41ZdbW 3y ago
This is a bold claim.
486.
by zX41ZdbW 3y ago
Just to note, ClickHouse is also dependency-free single binary, following https://notes.volution.ro/v1/2022/01/notes/fbf3f06c/ You can get it as https://clickhouse.com/ | sh
487.
by zX41ZdbW 3y ago
We bought the clickhouse.com domain two years ago, and it was listed in some adblockers due to abuse by the previous owner. We have cleaned up most of the block lists, but there is still a chance it is filtered by your ad-blocker.
488.
by zX41ZdbW 3y ago
It sounds like reimplementing ClickHouse, but worse. Sorry for pointing this out, but isn't it?
489.
by zX41ZdbW 3y ago
My favorite tool is clickhouse-local: https://clickhouse.com/blog/extracting-converting-querying-l... It is the most powerful (works with any formats, with remote datasets, and supports all the ClickHouse SQL) and the
490.
by zX41ZdbW 3y ago
As of today, it's almost pointless using Gorilla or DoubleDelta codecs for time series if the performance constraints make ZSTD an affordable choice. See https://clickhouse.com/blog/optimize-clickhouse-codecs-compr
491.
by zX41ZdbW 3y ago
Manually constructing machine code (as in the example) is not the best idea - it is error-prone, difficult to debug, and prevents testing with sanitizers. I'd not do it. Using LLVM for JIT is also not the best idea because LLVM is a co
492.
by zX41ZdbW 3y ago
ClickHouse if you need search + analytics and full SQL support: https://clickhouse.com/blog/vector-search-clickhouse-p2
493.
by zX41ZdbW 3y ago
Not much - the source code contains a brief description. See MergeTreeData.h, MergeTreeDataPartWide, MergeTreeDataPartCompact.
494.
by zX41ZdbW 3y ago
> I've already switched GMP back to use an outdated release tarball because of how annoyingly flakey their mercurial server is. We had a similar problem. Step one - don't allow submodules from outside of github.com: https:&#x2
495.
by zX41ZdbW 3y ago
How does it compare to VictoriaMetrics?
496.
by zX41ZdbW 3y ago
Yes, this is the idea. I set the threshold for approximate search with a margin so that on a few test queries, I get 100% recall, and then it is filtered by precise search. But I didn't evaluate it thoroughly. The threshold for Hamming
497.
by zX41ZdbW 3y ago
You should compare to ClickHouse: https://github.com/ClickHouse/ClickHouse/ See this tutorial for local data processing: https://clickhouse.com/blog/extracting-converting-querying-l... I&#x27
498.
by zX41ZdbW 3y ago
Most of the improvements in the article, including unrestricted aliases, were added in ClickHouse and subsequently influenced DuckDB. They still have to implement many usability and language improvements from ClickHouse.
499.
by zX41ZdbW 3y ago
Moreover, making as many as possible people to learn database engineering and production C++ experience - is one of my goals with ClickHouse.
500.
by zX41ZdbW 3y ago
I'm doing a similar thing - invite every student who is interested, without interviewing or skill tests: https://github.com/ClickHouse/ClickHouse/issues/42194 It works if you target for ~10% outcome if y
501.
by zX41ZdbW 3y ago
According to my estimation, a new database engine is born every week - mostly key-value and document databases. Only a small subset of them survive after one year. According to a guess by Stonebreaker, a DBMS takes around 7 years to become
502.
by zX41ZdbW 3y ago
The article leaves a bitter taste of unnecessary complexity. Data engineering should not be hard. For example, you can load the GitHub Archive to ClickHouse, and it will be accessible with interactive real-time queries: https://g
503.
by zX41ZdbW 3y ago
Here is an example of how to obtain a list of the top six million domains from Tranco and analyze their content with ClickHouse: https://github.com/ClickHouse/ClickHouse/issues/18842
504.
by zX41ZdbW 3y ago
It should take less than one second on CPU. For example, see my comment above: https://news.ycombinator.com/item?id=36341754
505.
by zX41ZdbW 3y ago
It is inefficient to build (unless on GPU), but good for search. Moreover, the search naturally benefits from data locality after ordering - this allows using MergeTree tables in ClickHouse. See https://news.ycombinator.com/
506.
by zX41ZdbW 3y ago
I've recently presented the approach with exhaustive search and heap, then iteratively adding simple indices, using ClickHouse. It works pretty well despite the overall usage simplicity. Video: https://www.youtube.com/w
507.
by zX41ZdbW 3y ago
I've also did an experiment by generating and searching embeddings for all the comments on HN. Here is the walkthrough: https://www.youtube.com/watch?v=hGRNcftpqAk
508.
by zX41ZdbW 3y ago
It is only around 5 GB in ClickHouse. Details: https://github.com/ClickHouse/ClickHouse/issues/29693
509.
by zX41ZdbW 3y ago
> Looking for a Sponsor to Host the Database PubliclyPermalink > In the meantime, it’d be great if anyone can query the database. I tried to host a public database and real-time query interface online, but couldn’t afford the bill for
510.
by zX41ZdbW 3y ago
Here is an example of analyzing 20+ billion Reddit comments in ClickHouse: https://clickhouse.com/docs/en/getting-started/example-datas...

More ›