captaintobs

7 ms·

31.
by captaintobs 3y ago
Although I have a degree in mathematics, I was never very good at deriving proofs.
32.
18xx.Games: open-source platform to play hardcore economic train games(github.com)
2 points by captaintobs 3y ago|0 comments
33.
Pure-Python implementation of the MySQL server wire protocol(github.com)
2 points by captaintobs 3y ago|0 comments
34.
by captaintobs 3y ago
Google had every opportunity to capitalize on the architectural innovation and failed tremendously.
35.
SQLMesh: Virtual Data Environments(medium.com)
3 points by captaintobs 3y ago|0 comments
36.
by captaintobs 3y ago
I've always preferred using notebooks (Jupyter Labs) for data. It preserves history and I can bounce around and execute arbitrarily complex scripts.
37.
by captaintobs 3y ago
It's such a funny story that Blockbuster had a chance to buy Netflix for $50 million.
38.
Stop wasting time and money rebuilding your tables unnecessarily(tobikodata.com)
9 points by captaintobs 3y ago|0 comments
39.
by captaintobs 4y ago
How does this compare to Flink and Spark? Is it performant for batch processing as well?
40.
by captaintobs 4y ago
It's all a facade.
41.
by captaintobs 4y ago
Nice and simple! I had a similar design on the iPhone back in the day where you could play head to head on the same phone. Great work!
42.
by captaintobs 4y ago
It's just not gonna happen unless the government does something.
43.
by captaintobs 4y ago
i was looking to do yc but decided against it because of the remote aspect
44.
by captaintobs 4y ago
ah i see
45.
by captaintobs 4y ago
why did you choose polars over duckdb?
46.
by captaintobs 4y ago
Even if you don't have much data, SQLMesh will save you time over dbt. Iteration is an important part to developing data pipelines, and fully refreshing your warehouse every time is wasteful and can slow you down. Trying out SQLMesh is
47.
Show HN: SQLMesh – The Future of DataOps(github.com)
29 points by captaintobs 4y ago|2 comments
48.
by captaintobs 4y ago
Nice work! As a software / data engineer, I totally agree. I always default to SQL and then only write Spark code when needed.
49.
by captaintobs 4y ago
I've run SQLGlot on the browser using Pyodide or something before. Since it has no deps, it didn't have any issues.
50.
by captaintobs 4y ago
thanks!
51.
by captaintobs 4y ago
Cool, sorry that SQLGlot didn't satisfy your needs. But if you file issues, we fix them quickly.
52.
by captaintobs 4y ago
I believe one of the main benefits of it being in Python is that it's easy to follow and jump in.
53.
by captaintobs 4y ago
Haha, well they are two very different projects with different goals :)
54.
Writing a Python SQL engine from scratch(github.com)
112 points by captaintobs 4y ago|15 comments
55.
by captaintobs 4y ago
it can do everything that can be done with the query and schema. it doesn't yet do indices or physical statistics
56.
by captaintobs 4y ago
recursion! https://github.com/tobymao/sqlglot/tree/main/sqlglot/optimiz...
57.
by captaintobs 4y ago
I'm the author of SQLGlot. Optimizing SQL is quite common and means things like projection / predicate push downs and logical simplification. SQL can be optimized relatively easily compared to a programming language because it is
58.
by captaintobs 4y ago
i made one as well called SAQ! https://github.com/tobymao/saq Looks like you use BLPOP, but there could be race conditions or lost jobs if your workers crash, check out BLMOVE and RPOPLPUSH.
59.
by captaintobs 4y ago
For this use case, it’s intentional, since only the outcome is important to us. That’s kind of the draw of a declarative language, you ask what you want and don’t have too much control over how that’s done (the engine should optimize that a
60.
by captaintobs 4y ago
Yes. SQLGlot is very customizable and you can pretty much override everything. It kind of needs to be flexible because even common sql dialects vary greatly. Here's an example of how we use SQLGlot to output raw Python code directly fr

More ›