zackangelo

11 ms·

91.
by zackangelo 2y ago
I wonder if the prevalence of coding LLMs will bring renewed attention to code-based modeling systems like OpenSCAD. It seems much easier to get them to generate code vs. translating into GUI interactions or modifying some other internal st
92.
by zackangelo 2y ago
Is this tied to a specific framework like pytorch or an inference server like vLLM? Our inference stack is built using candle in Rust, how hard would it be to integrate?
93.
by zackangelo 2y ago
L40S has 48GB of RAM, curious how they're able to run Llama 3.1 70B on it. The weights alone would exceed this. Maybe they mean quantized/fp8? I just had to implement GPU clustering in my inference stack to support Llama 3.1 70b,
94.
by zackangelo 2y ago
Any strings attached? Do you know if they’ll do it pre-funding?
95.
by zackangelo 2y ago
New tokenizer has a much larger vocabulary (200k)[0]. [0] https://github.com/openai/tiktoken/commit/9d01e5670ff50eb74c...
96.
by zackangelo 2y ago
Yes we’re running our own inference server because we do nonstandard things in the decoder loop (think logit masking, other types of sampling, etc).
97.
by zackangelo 2y ago
Would like to take a moment to recommend fly.io for GPU workloads. I've been building a prototype using them for the last couple of weeks and it's been great to use. I didn't have to jump through any hoops or apply for any qu
98.
by zackangelo 2y ago
The table at the bottom says they initialized the 65K version from "LLaMA-3 7B"? (Assuming the 7B is a typo and they meant 8B.) And each successive version with a larger window was initialized on the previous smaller one (65K -&gt
99.
by zackangelo 3y ago
If anyone is interested in a Rust-based on-disk KV store, I've come across sled[0] a few times, seems interesting. The author's also built a lot of other cool concurrency primitives for Rust as well. [0] https://githu
100.
by zackangelo 3y ago
I like Berkeley Mono [0]. [0] https://berkeleygraphics.com/typefaces/berkeley-mono
101.
by zackangelo 3y ago
He's referring to Robert Lustig, in case anyone else was curious.
102.
by zackangelo 3y ago
That is satire.
103.
by zackangelo 3y ago
Prescription drugs are dispensed by a pharmacist for a reason. In the US, they are legally required to report any unusually large, frequent or suspicious orders to the government. Many pharmacies were found to have neglected that responsibi
104.
by zackangelo 3y ago
Yes but Send + Sync is required everywhere regardless.
105.
by zackangelo 3y ago
My first introduction to this subject was “Let’s Build a Compiler” by Jack Crenshaw. Even though it’s quite old now, I still highly recommend if you’re looking for a gentle, no-frills introduction: https://compilers.iecc.com&#x2F
106.
by zackangelo 3y ago
Don't miss the YouTube video with Lupe Fiasco [0]. The cynic in me says that this might be all marketing; I would be curious to know if he's actually using LLMs regularly to write songs. So many people think AI will entirely repla
107.
by zackangelo 3y ago
He’s not trying or claiming to improve attention. He’s trying to reduce outliers to improve the ability to quantize the parameters.
108.
by zackangelo 3y ago
I have a similar experience when using rust-analyzer and copilot in vscode.
109.
by zackangelo 3y ago
I think the read-only use case you mention is a compelling one. I’m working on a developer product that makes it easy to prematerialize data in the way you describe. It works by consuming data out of upstream APIs and then publishes a new,
110.
by zackangelo 3y ago
The Eight mattress will measure sleep stages without you having to wear anything.
111.
by zackangelo 3y ago
Every time stories like this get posted, the first question is typically "when will they convert these buildings to residential?". And, like clockwork, there's always someone there to point out that residential conversion is
112.
by zackangelo 3y ago
They’ve since reduced PFAS levels by half but still not great: https://www.consumerreports.org/bottled-water/topo-chico-cut...
113.
by zackangelo 3y ago
This also brought back memories for me of writing Win32 apps. If you're looking for a good book, checkout Programming Windows by Charles Petzold. It really demystified a lot of things for me and I bet it's still relevant if you wa
114.
by zackangelo 3y ago
"runs WASM" made me wonder: is transpiling a ROM to WASM an effective strategy for emulating something like a GameBoy? You'd still have many issues to solve, not least of which mapping the hardware onto a runtime layer, but y
115.
by zackangelo 4y ago
If you're doing memory-intensive tasks that are clearly bounded (e.g., handling an http request), I wonder if it's worth looking at something like an arena allocator [0] that can free the entire task's memory at once. [0] ht
116.
by zackangelo 4y ago
I stopped wearing my Apple Watch while surfing because this feature would trigger almost every time. It’s made worse by the fact that the touch screen is inoperable when the phone senses that it’s wet, so it’s difficult to cancel.
117.
by zackangelo 4y ago
Most JVMs take advantage of a thread local "bump" allocator[1] as well to avoid having to cross JVM or kernel boundaries to allocate memory, which can result in huge speedups for memory-intensive use cases. [1] https://
118.
by zackangelo 4y ago
An often overlooked option for profiling Rust is Apple's Instruments.app. It's amazing and usually the first thing I reach for when I need a profiler on Mac OS X.
119.
Tiny Glade: castle doodling game in Rust(store.steampowered.com)
2 points by zackangelo 4y ago|0 comments
120.
by zackangelo 4y ago
Surprised this is the case because I use their Rust v8 bindings on an ARM laptop almost every day. Have you tried building it from source?

More ›