wenc

54 ms·

721.
by wenc 3y ago
They would probably ship a specialized model trained provide specific answers on device and offload to remote for more general answers. Probably not a full Llama or Falcon. Their latest keyboard autocomplete for instance is a GPT-2 model.
722.
by wenc 3y ago
Processing is adequate but RAM is not. The 15 Max Pro only has 8GB RAM. You need 32 GB to run any decent sized LLMs. (You can quantize a smaller model but the quality of the output will be noticeably bad) I have an M2 Pro with 16gb ram and
723.
by wenc 3y ago
How?
724.
by wenc 3y ago
Projects that use Meson https://mesonbuild.com/Users.html Includes Gimp, Gtk+, nautilus, Postgres, qemu, Wayland…
725.
by wenc 3y ago
What an amazing read. Now I know why my pip installs are failing in 3.12 but we now have a brighter future ahead. Also while I love Python it’s helpful to understand why Python packaging is a (manageable) mess. It’s because of non standardi
726.
by wenc 3y ago
LLMs do not have general intelligence, but it does an awfully good job doing something human thought typically does -- synthesizing ideas. It's able to combine X and Y to produce Z. LLMs cannot do first-principles reasoning well (unles
727.
by wenc 3y ago
It's not doing a full table scan of the entire source table 64 times -- it's only scanning the id column (DuckDB + Parquet is a columnar store) and retrieving the corresponding rows from other columns. It's pretty fast.
728.
by wenc 3y ago
Breaking it up. Instead of running a query for the entire year, run it month by month and stitch the final results back together. Or if you have a unique string ID, calculate an integer hash using hash(ID) % 50 to get 50 chunks which you ca
729.
by wenc 3y ago
Makes sense — every S3 call is high latency so the fewer you make the better. Prefetching would help reduce the number of those high latency calls which databases naturally make. We often think of S3 as a file system but it isn’t one — it d
730.
by wenc 3y ago
No I got you. Like I said S3 is a high throughput high latency storage. When you fetch the S3 object to disk, that’s a high throughput operation and S3 excels at that. Once on disk DuckDB can operate at low latency. If you run DuckDB end to
731.
by wenc 3y ago
DuckDB interoperates with polars dataframes easily. I see DuckDB as a SQL engine for dataframes. Any DuckDB result is easily converted to Pandas (by appending .df()) or Polars (by appending .pl()). The conversion to polars is instantaneous
732.
by wenc 3y ago
No you can definitely use globs in DuckDB. And no you don’t have to know the exact parquet file. You would treat the Hive partitioned data as a single dataset and DuckDB will scan it automatically. (Partition elimination, predicate pushdown
733.
by wenc 3y ago
Not since 0.9.0. There’s been a lot of work on out of core stuff especially with big joins. It’s in the release notes. Also if you set a temp folder it will spill to disk (not set by default). I used to have to chunk my data to avoid OOMs b
734.
by wenc 3y ago
You definitely pay a performance penalty on S3 (S3 is high throughput but high latency storage) so not optimal for (any) database use cases. Local disk will always be faster if you can swing that. It’s not a DuckDB specific issue (although
735.
by wenc 3y ago
S3 and SageMaker. Same region.
736.
by wenc 3y ago
Fantastic job by the DuckDB team. I’ve been using it for the past year to query 100s of GBs of Parquet files with complex analytic queries involving multiple levels of aggregations, joins and window functions and it all works and works fast
737.
by wenc 3y ago
> “I don’t turn right on red” - because, at least in Oregon, it’s entirely optional. I thought it was entirely optional everywhere, and prohibited in NYC and Montreal. A red does generally mean red, so no one can fault you for not turni
738.
by wenc 3y ago
Yes, I was a Google Reader user for a long time. When it shut down, I never went back to RSS.
739.
by wenc 3y ago
I open tons of tabs but they rarely stay open. I have a folder called "Daily" that I middle click each day -- they contain news sites (Google News/HN/Reddit etc.) From those I middle-click articles/comments I want t
740.
by wenc 3y ago
Audrey Tang (formerly Autrijus Tang), Taiwan's minister of digital affairs, was a prolific perl hacker back in the day with many contributions on CPAN. https://en.wikipedia.org/wiki/Audrey_Tang
741.
by wenc 3y ago
I've learned to take anecdotes as data points but also recognize that in a large enough company, there is a huge variance. A large company is like a country. Different people will have different experiences. Take America for instance.
742.
by wenc 3y ago
This is a brilliant and simple solution. If it's a static barcode, a photo is all that is needed. There's little advantage to having it in the wallet. I store all my IDs in a Photo Album on my phone.
743.
by wenc 3y ago
If you've used the competition (ChatGPT, Claude, even Perplexity free version), you will find that Bard is not good in terms of quality of answers. (I'm forced to use Bard at work. The UI is responsive I'll give you that, but
744.
by wenc 3y ago
One issue is because there are so few CR cameras out there, most people would sign their photos in Lightroom. So the provenance starts from software rather than capture device. So you can sign a fake photo that you modified and its provenan
745.
by wenc 3y ago
"Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why
746.
by wenc 3y ago
This is very nice. Replaces reduce() for me. Now to upgrade to Python 3.8! (stock Python on my cloud machine is still 3.7)
747.
by wenc 3y ago
I doubt it would have succeeded. The latent associations with the unreliable and bloated Windows CE was too strong which had a halo effect — the vibe was that Microsoft tried to shoehorn Windows into mobile devices where it doesn’t really w
748.
by wenc 3y ago
For me it was a tossup between the Herman Miller Aeron ($450 refurb) and Steelcase Leap 2 ($250 refurb). I picked the Leap 2 and never looked back. Fabric seats, more adjustability, very sturdy and cheaper refurb prices (I think at full pri
749.
by wenc 3y ago
I’m advocating for seeing the big picture. It’s not that I don’t care about programming, but I try to care for it in proportion to achieving impact, which I feel is the more important thing (most seniors and higher would agree). I did give
750.
by wenc 3y ago
I think everybody agrees and is not pretending otherwise. But my thing is, I think the problem could be solved in a different way in Python where we can use performant libraries to get an answer in a reasonable time. The author says the nai

More ›