liuliu

23 ms·

331.
by liuliu 3y ago
64.
332.
by liuliu 3y ago
Very similar sentiment when AppStore built. Everyone tries to avoid host their business on someone else's platform. Hence FB tried to do H5 with their app (so it is open-standard (a.k.a. web) based, people launches their own mobile pho
333.
by liuliu 3y ago
Might use activations rather than weights in the title then. Weights are commonly done, activations are not.
334.
by liuliu 3y ago
There are plenty of models to try with Draw Things app. You can try SDXL on it to see what's the quality looks like. The speed comparison here: https://engineering.drawthings.ai/integrating-metal-flashatt...
335.
by liuliu 3y ago
PyTorch checkpoint is slow to load.
336.
by liuliu 3y ago
The implementation is not even optimized for Macs. LCM is just very easy to be fast (batch size = 1 and only 2 to 8 steps, depending on what kind of headline you are trying to make).
337.
by liuliu 3y ago
I meant Swift. Golang compiles fast enough to be comparable with interpreted languages coding patterns. Swift by being a compiled language, also have awkwardness here and there when it comes to type-checking where TypeScript is much more re
338.
by liuliu 3y ago
It is a compiled language. The first time I tried Go (~10 years ago now), the core dump is even more cryptic. It is a OK language if you don't want to worry every memory allocations and wants a simpler / effective language to work
339.
by liuliu 3y ago
You can generate low-res image faster / first and selectively generate hi-res ones for longer time.
340.
by liuliu 3y ago
It is a lie that we tell people "your compute is offloaded to the cloud" until now. User-centric heavy compute in the past 20 years simply don't happen in "the cloud". Monetization and engagement tricks yes (ads bid
341.
by liuliu 3y ago
Just want to call out this is fresh from OpenAI that actually talks about data preparation, their models (to some extents) and some negative learnings. Still not as complete as what's from their peers (Stability AI, Meta, adept.ai etc)
342.
by liuliu 3y ago
Very cool. A few notes: 1. MPS backend uses MPSGraph exclusively, might hit some performance ceilings limited by MPSGraph. s4nnc moved more and more ops from MPSGraph to Metal directly to have better control on both allocation and some erra
343.
by liuliu 3y ago
Yes. But also the U.S. sees no way to influence the faction of China to the ways of the U.S. influence traditionally worked in China (unlike how China can influence the U.S. politics). So the cooperation game quickly becomes a game of one b
344.
by liuliu 3y ago
Thanks for the kind words! We are in the early days of what these software can do, doing ads and other things would not be productive for figuring out what this should look like :)
345.
Show HN: Draw Things Democratizes Local Large Model Fine-Tuning on iPad and Mac(engineering.drawthings.ai)
3 points by liuliu 3y ago|2 comments
346.
by liuliu 3y ago
Draw Things added CUDA compatibility seed mode allows you to match NVDIA card generated images on Mac.
347.
by liuliu 3y ago
This seems to be a reasonable approach for Go, but you did need to carry a lot in your containerized environment (Go tends to have very lean container, and this approach requires a fat container with CUDA, PyTorch, Python etc).
348.
by liuliu 3y ago
This is a reasonable approach for systems that allowed to load binaries (either the running artifact is a binary or semi-binary (WASM executable) or it allows to load .so / .dll from user-provided places). It basically runs with the pr
349.
by liuliu 3y ago
If SQLite is faster, the problem is the zip library you use. SQLite has a major draw back (and yes, I love SQLite and built a lot of things around it over the years): the blob you get from the DB cannot be mmap and you have to copy it to so
350.
by liuliu 3y ago
Maybe simpler? When open the DB, change it to WAL mode, turn off the automatic checkpoint https://www.sqlite.org/pragma.html#pragma_wal_autocheckpoint When user saves, you just checkpointing the file, merging it back into t
351.
by liuliu 3y ago
It should be about 6 images per minutes (for SD v1 at 512x512) and 1 image per minute on SDXL for the said hardware (M1 Max): https://engineering.drawthings.ai/integrating-metal-flashatt...
352.
by liuliu 3y ago
Yeah, but I was under the impression that for the same prompt, implementations are already share the KV cache. This area is so new so these obvious ideas might not get implemented as widely as I thought.
353.
by liuliu 3y ago
Not only batching. It works by serving different requests as long as they share the prefix. This basically enables KV cache reuse when there is a prefix matching (from my shallow understanding of how KV cache works). I failed to see how thi
354.
by liuliu 3y ago
Oh, by "boxing object", I meant to create the in-memory object that can be used by JavaScript runtime, preferably native objects. For example, in JSC case, something like https://developer.apple.com/documentation&#
355.
by liuliu 3y ago
Speculative decoding have higher batch size.
356.
by liuliu 3y ago
The Mojo code in this post cannot be run in Python. So in this case, you wrote code for Mojo language, not Python.
357.
by liuliu 3y ago
With multi-thread and some tricks, csv parsing should be able to reach SSD speed (~2GiB/s read): https://liuliu.me/eyes/loading-csv-file-at-the-speed-limit-o... At that point, I think the speed is limited by boxin
358.
by liuliu 3y ago
Only recently. mmap file and directly use in Metal kernels is actually not supported until iOS 16 / macOS 13. Also, there are limited optimization opportunities around that and the recommended way seems still to use the specific Metal
359.
by liuliu 3y ago
mmap weights directly from a file seems to be new (I think). Need to check notes to remember whether you can already do that with some cuda* API.
360.
by liuliu 3y ago
I haven't figured out how. But apparently, there is a logic hole people can exploit to insert their own content into anyone's "Following" tab without "Sponsored" or "Ads" tag (and without anyone in yo

More ›