liuliu

12 ms·

181.
by liuliu 1y ago
It really is not obvious. These launches are asynchronous, and data movement / computation is overlapped properly through CUDA APIs. Even per-kernel launch cost is reduced with the cudagraph introduction. CUDA programming model relies
182.
by liuliu 1y ago
The Qwen 8B number, if verified, is very impressive. Much more practical than the previous megakernel one. That's being said, these one-persisted kernel on each SM reminds me Larrabee, and now wondering what the world will be if we jus
183.
BF16 and Image Generation Models(engineering.drawthings.ai)
2 points by liuliu 1y ago|0 comments
184.
by liuliu 1y ago
Both uses cublas under the hood. So I think it is similar for prefilling (of course, this framework is too early and don't have FP16 / BF16 support for GEMM it seems). Hand-roll gemv is faster for token generation hence llama.cpp
185.
by liuliu 1y ago
Congrats on finding a bug! However, the keyword here is training / inference divergence. Unfortunately, nobody is going to spend multi-million to retrain a model, so our reimplementation needs to be bug-to-bug correct to use the traine
186.
by liuliu 1y ago
If you are interested in this: Flux reference implementation is very minimalistic: https://github.com/black-forest-labs/flux/tree/main/src/flux The minRF project is very easy to start with training
187.
by liuliu 1y ago
Your leadership on continuing investing in core technologies in Facebook were as fruitful as it could ever being. GraphQL, PyTorch, React to name a few cannot happen without.
188.
by liuliu 1y ago
Yeah. It is great. So apparently separating spatial / temporal attention works if you are careful and train with large enough dataset too!
189.
by liuliu 1y ago
Or AWS, and AWS managed services v.s. other managed services on top of AWS.
190.
by liuliu 1y ago
Diffusion based. There is no point to move to auto-regressive if you are not also training a multimodality LLM, which these companies are not doing that.
191.
by liuliu 1y ago
Seems implementation is straightforward (very similar to everyone else, HiDream-E1, ICEdit, DreamO etc.), the magic is on data curation (which details are lightly shared).
192.
by liuliu 1y ago
Thanks. It is a bit vague to me too. If you need to load 5B per token generation any way, what's that different from selective offloading technique where some MLP weights offloaded to fast storage and loaded during each token generatio
193.
by liuliu 1y ago
The particularly integration pain point to me is about network access, that prohibits several banal tasks to be offloaded to codex: 1. Cannot git fetch and sync with upstream, fixing any integration bugs; 2. Cannot pull in new library as d
194.
by liuliu 1y ago
This blog post is miles better than MCP spec, which yes, described what you should do but doesn't really differentiate from what's beyond JSON-RPC + Auth. I think that's the point though. It is really just a RPC layer for LLM
195.
by liuliu 1y ago
Hi! Draw Things should be able to add support in the next 2 weeks after we get video feature a little bit more polished out with existing video models (Wan 2.1, Hunyuan etc).
196.
by liuliu 1y ago
CSV is standardized in RFC 4180 (well, as standardized as most of what we considered internet "standard"). Otherwise agree, if you don't do escaping (a.k.a. "quoting", the same thing for CSV), you are not implementi
197.
by liuliu 1y ago
Yeah, I use the full model which is slightly better at some of these prompts.
198.
by liuliu 1y ago
Do you mind to share which HiDream-I1 model you are using? I am getting better results with these prompts from mine implementation inside Draw Things.
199.
by liuliu 1y ago
That let you think if we can rewind the time, maybe we should just allocate one more bit for half precision (6 exp, 9 mantissa) and not doing this bfloat16 thing.
200.
by liuliu 1y ago
Of course corporations will have a lot of different bets. Most of them will not pan out but they will try. Meta will not be able to produce a chip that can run GenAI workload in the next 2 years. Microsoft is doing a side-quest, and they ha
201.
by liuliu 1y ago
Only one of the 4 companies you mention is successful at this. And it will remain that way. Chinese CSPs are the only ones can develop their own hardware / software for AI / HPC.
202.
by liuliu 2y ago
That's not going to be true after your facilities are built.
203.
by liuliu 2y ago
Not blocking, just annoyance: https://github.com/swift-server/swift-backtrace/issues/72
204.
by liuliu 2y ago
If you stick with x86_64 land and > Swift 6.0, there is nothing infeasible about it.
205.
by liuliu 2y ago
I found reasoning models are much more faithful at text related tasks too (i.e. 1. translating long key-value pairs (i.e. Localizable.strings), 2. long transcript fixing and verification; 3. look at csv / tabular data and fix) probably
206.
by liuliu 2y ago
I think, we fundamentally lack a mechanism to enforce secure / privacy aware APIs without resorting to trusted inner-circle type of things. I am already not comfortable with Apple picking winners (such as giving Zoom special entitlemen
207.
by liuliu 2y ago
I would listen to people who used the previous frameworks about the deficiencies and pain points, not people who just casually browse the documentation about their high-flying ideas why these have deficiencies and pain points. One group has
208.
by liuliu 2y ago
Of course I trust people who working on L2 chains to tell me how to scale Bitcoin and people who working on cryptography to walk me through the ETH PoS algorithms. You cannot lead to truth by learning from people who don't know. People
209.
by liuliu 2y ago
I think all these articles begging the question: what's author's credential to claim these things. Be careful about consuming information from chatters, not doers. There is only knowledge from doing, not from pondering.
210.
by liuliu 2y ago
Maybe they don't view him as a warlord but ideologically aligned friend. Israel is a conservative Jewish state which is more ideologically aligned with current administration than a Islamic state.

More ›