broner

4 ms·

1.
by broner 1y ago
Nope, just reading from S3. Check this out: https://duckdb.org/2024/07/09/memory-management.html
2.
by broner 1y ago
Works fine for me on TB+ datasets. Maybe you were doing in-memory rather than persistent database and running out of RAM? https://duckdb.org/docs/stable/clients/cli/overview.html#in-...
3.
by broner 1y ago
You can use DuckDB in ipython to solve the globbing issue. Then you don't have to worry about OOMs with geopandas.
4.
by broner 2y ago
My workload is 100% read. Querying zstd parquet on s3 standard. Neither clickhouse nor duckdb has a great s3 driver, which is why smart people like https://www.boilingdata.com/ wrote their own. I compared a handful of queri
5.
by broner 2y ago
I'm doing this with clickhouse querying parquet files on S3 from an EC2 instance in the same region as the S3 bucket (yes DuckDB pretty similar). S3 time to first byte within AWS is 50ms and I get close to saturating an big EC2 instanc
6.
by broner 2y ago
Switch your blind faith to Go (s5cmd) and get ~1200 MB/s upload speed! zfs send tank/pgdata@snapshot | pbzip2 > mybackup.zfs.bz2 s5cmd cp mybackup.zfs.bz2 s3://mygooglebucket/ https://github.com/p
7.
by broner 2y ago
For $1,500 per TB per month? No thanks.
8.
by broner 2y ago
Should be a tax advantage to the non-grind income. $920k with 20% fed + 5% state capital gains tax?
9.
by broner 2y ago
Could you use this to create 100s of on-demand read replicas and get 100s of GB/s out of S3? Seems like a nice way to cheaply and quickly do batch jobs.
10.
by broner 3y ago
This looks pretty neat if you're ok moving to AWS https://www.boilingdata.com/
11.
by broner 3y ago
Reminds me of this previous HN submission Blotato: https://news.ycombinator.com/item?id=38206235
12.
by broner 4y ago
Glitch? Oh they're with Fastly now! https://www.tiktok.com/@benedicttown/video/70983764223009620...
13.
How will climate change affect your home?(fastcompany.com)
1 points by broner 6y ago|1 comments
14.
by broner 6y ago
As the climate rapidly changes and gets more dangerous, it’s good to know what you’re in store for. This site gives you a picture. https://climatecheck.com
15.
by broner 6y ago
I switched from Overcast to Pocketcasts. Everything I liked about Overcast plus playlists I can figure out how to use.
16.
by broner 13y ago
I'd be happy to help :) What age guys are you looking for?