liuliu

26 ms·

361.
by liuliu 3y ago
What's the best way to store around 300TiB data cost-effectively? Assuming would have access from time to time to around 25% of that data (monthly) through 1Gbps connection. On-prem seems to be the best, but carrying around 30 disks is
362.
by liuliu 3y ago
Yes. But it also needs a very different org structure to support that. Their internal infra from what I heard is dated (monolithic PHP binary deployment, no federated authorization management etc.). It is doable (FAIR's org structure w
363.
by liuliu 3y ago
34B should be able to run on 24GiB consumer graphics card, or 32GiB Mac (M1 / M2 chips) with quantization (5~6bit) (and 7B should be able to run on your smart toaster).
364.
by liuliu 3y ago
Yeah, I am talking about ~10% performance differences for a specific model (I believe it is when benchmarking vanilla seq2seq model (from AIAYN paper) with small batch size and no gradient accumulation). NVLink is about ~$100, so for these
365.
by liuliu 3y ago
It is model-dependent. I've seen that (NVLink benefits) when comparing against PCIe-3 connection, with small batch size, no gradient accumulation. Once you have larger batch size and gradient accumulation, DDP won't be improved by
366.
by liuliu 3y ago
I am thinking about this. I think our distinction between IO-bounded tasks v.s. compute-bounded tasks helped design in some ways. However, as time goes by, the code you are dealing with can bounce between these two modes, and that is where
367.
by liuliu 3y ago
Yeah, you need 100gbe minimal. 10gbe is too little (PCIe bandwidth can be a bottleneck, and that is already clocked around 100GbE (16GB)). BTW: echo to the author, PSU and in the U.S. (120v) is a major issue why I am limiting to 4-GPUs. Als
368.
by liuliu 3y ago
Definitely interesting. It seems that I accidentally implemented that in Draw Things.
369.
Metal FlashAttention: Accelerating Image Generation in the Apple Ecosystem(engineering.drawthings.ai)
2 points by liuliu 3y ago|0 comments
370.
by liuliu 3y ago
I am not arguing against having an oracle. Just saying in the static type-system (like the ones in Rust / C++ / Swift) would be too frequent to invoke to inhibit development. I think having a linter pass or similar is great. Much
371.
by liuliu 3y ago
Because that's not my workflow looks like? * Fire up Jupyter notebook, create some PyTorch nn.Module, validate the output is expected. * Built out the model, validate the output is expected. * Check the data shape, do a eval() run with
372.
by liuliu 3y ago
After a few years, I am coming to an opposite opinion on this. Yeah, shape checking is nice and all and you should be able to run them before-hand (might be a linter pass). But inside type-system while most people have LSP running in the ba
373.
by liuliu 3y ago
Don't see any ill-will here. They changed license fairly early without soliciting years of contributions from others (like, only a few months old and probably a handful of contributions from public from what I can skim). They don'
374.
by liuliu 3y ago
I overspoke. FID is a fine metrics to observe the training progress of your own model. And it correlates well with some coherency issues of generative models. But for cross model comparisons, especially for models that generally do well und
375.
by liuliu 3y ago
Will be more info upon release. SDXL v0.9 performs generally the same as SD v1 / v2 on the same resolution. But because you tend to run it at larger resolution, you might feel it slower.
376.
by liuliu 3y ago
Yeah, Draw Things. It will be submitted as soon as SDXL v1.0 weights available. Quantized model should run on iPhones (4GiB / 6GiB models), but we haven't done that yet. So no, these are just typical FP16 weights on iPad.
377.
by liuliu 3y ago
SDXL 0.9 runs on iPad Pro 8GiB just fine.
378.
by liuliu 3y ago
SDXL 0.9 should be the state-of-the-art image generation model (in the open). It generates at 1024x1024 large resolution, with high coherency and good selection of styles out of box. It also has reasonable text-understanding comparing to ot
379.
by liuliu 3y ago
And this is a superficial difference carried from old days when we need to do deployment and deployment-specific optimizations. With LoRA / QLoRA, my bet is that edge training capabilities are as important in the next decade. I don&#x2
380.
by liuliu 3y ago
Ajax probably just a acronym of Apple Jax given that their training is done on TPUs.
381.
by liuliu 3y ago
Sorta of? Latent diffusion models operate in a compressed latent space, which is just a richer / learnable representation than DCT.
382.
by liuliu 3y ago
Yes, computes are carried out in FP16 (so there is no compute efficiency gains, might be latency reductions due to memory-bandwidth saving). These savings are not realized yet because no custom kernels introduced yet.
383.
by liuliu 3y ago
Draw Things recently released a 8-bit quantized SD model that has comparable output as the FP16. It does use k-means based LUT and separate weights into blocks to minimize quantization errors.
384.
by liuliu 3y ago
(Unfortunately, it is a 1-hour long video and I don't have time to watch it over). The problem with 2 is often that the database is too far away from the view, hence, what you prioritize to sync might not be the most important thing th
385.
by liuliu 3y ago
The logic is twisted and strange, the justification sums up to this: talking past projects like talking to a PhD student about his research paper, it is all they does and doesn't put them in enough "foreign situation" to see
386.
by liuliu 3y ago
When you are hiring in volume, you are hiring for additive value, not for transformative value (nor multiplicative value). For additive value, conformance is essential. Most of the semi-successful companies don't need to be "the i
387.
by liuliu 3y ago
I don't write Rust. But here is what you said and what the author said don't conflict with each other, and it has been on my mind for a while. People who write similar code, or work on things for decades usually don't really
388.
by liuliu 3y ago
Many ways, for one: https://magicfusion.github.io/
389.
by liuliu 3y ago
It reflects the defensible moat in our industry (SaaS offerings in general) is no longer technology.
390.
by liuliu 3y ago
I don't know. Nowadays a top-of-the-spec Linux workstation without any cost on top (i.e. bare-components) is around ~10k. 5995WX in the graph wccf provided alone costs ~5k.

More ›