mytherin

7 ms·

1.
by mytherin 2y ago
I agree that the blog post seems to hint at the fact that this functionality is fully baked in in certain places - we've adjusted the blog post to be more explicit on the fact that this is an extension. We have collaborated with Mother
2.
by mytherin 2y ago
One of the DuckDB maintainers here. To clarify - the UI is not built into the DuckDB release. It is an extension that is downloaded and installed like any other extension. This extension happens to be developed by MotherDuck. We collaborate
3.
by mytherin 3y ago
You can query the metadata of a Parquet file using DuckDB's parquet_schema and parquet_metadata functions [1] [1] https://duckdb.org/docs/data/parquet/metadata
4.
by mytherin 4y ago
Thanks for trying it out! Could you perhaps open an issue [1] or share the file with us so we could investigate the problem? [1] https://github.com/duckdb/duckdb/issues
5.
by mytherin 4y ago
Perhaps have a look at this article [1] [1] https://www.vantage.sh/blog/querying-aws-cost-data-duckdb
6.
by mytherin 4y ago
Feature were stripped from Google Image Search because of a lawsuit [1]. Yandex, Bing and others can offer a better image search because they are not big/important enough to sue in the US search engine market. If Google were to be deth
7.
by mytherin 4y ago
We (the DuckDB team) are very happy working together with MotherDuck in a close partnership [1]. [1] https://duckdblabs.com/news/2022/11/15/motherduck-partnershi...
8.
by mytherin 4y ago
Again, you are still conflating tables in a relational database with variables in Typescript. While I can understand the confusion - given that SQL makes it so natural to interact with tables - they are not equivalent. SQL as a language is
9.
by mytherin 4y ago
> Same goes for SQL. Given the previous program, it is impossible to determine what the type of v is until you are ready to execute the SELECT statement. If the CREATE TABLE statement fails, v will not be the VARCHAR you think it is. If
10.
by mytherin 4y ago
Types are static within the context of a query, not necessarily within the context of the lifetime of a database. Types are determined and propagated at query compile time . In a query, the value "v" refers to a VARCHAR column (a
11.
by mytherin 4y ago
Most implementations of SQL are not dynamically typed - they are statically typed. There is an explicit compilation phase (`PREPARE`) that compiles the entire plan and handles any type errors. For example - this query throws a type error wh
12.
by mytherin 4y ago
C/C++ code can certainly be compiled down to WASM, but you cannot interface with the operating system as you would in a normal C/C++ program. To get around that restriction postgres-wasm ships an entire Linux distribution that i
13.
by mytherin 4y ago
Because DuckDB uses ACID [1] data is loaded in an all-or-nothing manner. As the load was interrupted due to the system running out of memory, the table is expected to be empty. [1] https://en.wikipedia.org/wiki/ACID
14.
by mytherin 4y ago
CREATE INDEX currently has the restriction that the index must fit in memory [1]. As the data is already sorted, creating an index is not necessary anyway. The min/max indexes created automatically by the system are sufficient to compl
15.
by mytherin 4y ago
It is possible to JIT compile Python just fine. There are projects like PyPy that have been doing this for a long time [1]. The reason these alternative projects never take off is because many of Python's most used libraries are writte
16.
by mytherin 4y ago
As a Dutch person hearing people talk about mandatory helmet laws for bikes sounds very strange. Nobody in the Netherlands bikes with a helmet, and it is incredibly safe to bike here (around 10X fewer fatalities per KM ridden than in the US
17.
by mytherin 4y ago
> I’ve never been in a car accident that required a seat belt either. We should remove them all by your logic. Having a seat belt in a car is far less of an inconvenience than carrying around a helmet. If mandatory helmet laws cause peop
18.
by mytherin 4y ago
While I can (unfortunately) see this happening - it seems incredibly futile and pointless to try and fight such an arms race. When AI-supported writing becomes the norm having students write essays without AI-assistance will be like trying
19.
by mytherin 4y ago
Perhaps I am missing something in the spec - but trying this in various compilers, it seems that you *can* assign structs holding arrays to one another, but you *cannot* assign arrays themselves. This compiles: struct BigStruct {
20.
by mytherin 4y ago
If your employer expects you to work 50 or 60 hours while only paying you for 40 hours then that is indeed also illegal.
21.
by mytherin 4y ago
If you are charging two companies for 40 hours per week of work, while only working 20 hours per week for each of them, then that is fraud by any reasonable definition of the word.
22.
by mytherin 4y ago
You may also want to have a look at the DuckDB command line client [1]. The shell itself is based on the SQLite client, and DuckDB can be used to natively query CSV and Parquet files. Using extensions, DuckDB can also query SQLite and Postg
23.
by mytherin 4y ago
The hardest part about optimizing Python is the fact that there are so many packages out there relying on every single documented and undocumented aspect of the C/C++ interface, which is incredibly broad and directly tied to the inte
24.
by mytherin 4y ago
We have a PR in the works that is adding support for sum types to DuckDB [1]. Not quite fully dynamic, but perhaps still useful :) [1] https://github.com/duckdb/duckdb/pull/4966
25.
by mytherin 4y ago
Postgres uses a similar custom implementation of try/catch [1]. [1] https://github.com/postgres/postgres/blob/master/src/include...
26.
by mytherin 4y ago
If anything, it might make use-after frees more painful. Now the address sanitizer won't help you find them.
27.
by mytherin 4y ago
You are misinterpreting it indeed. The scenario I am describing is roughly the following: Suppose we allocate 100K buffers that all have an equal size, and our memory usage is now 10GB. After that point we free 20K buffers, but allocate 10K
28.
by mytherin 4y ago
We are allocating and freeing buffers repeatedly. Despite freeing more buffers than we allocate, memory usage might still increase because of internal fragmentation in the allocator. Essentially, fragmentation might create "unused&quot
29.
by mytherin 4y ago
The particular way in which the data is loaded into DuckDB and the particular machine configuration on which it is run triggers a problem in DuckDB related to memory management. Essentially the standard Linux memory allocator does not like
30.
by mytherin 4y ago
That specific SQLite test suite (one of four [1]) has loads of generated SQL queries, and a long tail of more varied hand-written tests. That 95% of the test suite will be mostly generated queries that follow the same basic pattern of joins

More ›