lhl

27 ms·

1. 211.▲
   by lhl 3y ago

   In my post I link to my issue where I outline what I needed to do from a clean mamba env that might help. Pytorch nightly (I use for cuda-12) doesn't work w Python 3.12, but if you stick w 3.11 or 3.10 you should be ok. Rest was just w
2. 212.▲
   by lhl 3y ago

   Just a followup for those interested, inference implementation notes and comparison clip between StyleTTS2, TTS VITS, and XTTS: https://fediverse.randomfoo.net/notice/AaOgprU715gcT5GrZ2
3. 213.▲
   by lhl 3y ago

   Ha awesome! I just checked the repo literally before I posted and it was still empty, thanks for the heads up, will give it a spin now.
4. 214.▲
   by lhl 3y ago

   For getting started, I'd recommend https://huggingface.co/autotrain - you should be able to find docs on the CLI which can let you do your first training run w/ a single command (which includes automatically pulli
5. 215.▲
   by lhl 3y ago

   The rule of thumb is that inference speed halves with every doubling of parameter size (and obviously a doubling of memory size). You can check out real world performances on devices here: https://llm.mlc.ai/
6. 216.▲
   by lhl 3y ago

   For neutral sounding very fast/efficient voices, I find Coqui TTS VITS models to be very good. For slower, more expressive voice or voice cloning I think the Coqui TTS XTTS is good (or you can look at the mrq/tortoise-tts). I&#x27
7. 217.▲
   by lhl 3y ago

   Use the code/model included here: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 Change the initial device line from "cuda" to "cpu" and it'll run. (Edit: just a note, use the m
8. 218.▲
   by lhl 3y ago

   While the llama2-34b base model hasn't been released, CodeLlama2 is effectively a fine-tuned version of 34b and there are some people working with that. As Ollama uses a llama.cpp fork on the backend, I'd expect its memory usage t
9. 219.▲
   by lhl 3y ago

   One of your problems might be that gfx1032 is not supported by AMD's ROCm packages, which has a laughably short list of supported hardware: https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h...
10. 220.▲
    by lhl 3y ago

    Hamel Husain hasn't done testing vs llama.cpp, but this still might be of interest (includes mlc which is roughly in line w/ llama.cpp batch=1 perf): https://hamel.dev/notes/llm/inference/03_inferenc
11. 221.▲
    by lhl 3y ago

    The numbers are always changing, but from my testing, they're close enough that it doesn't really matter. My most recent benchmarks: https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYp... I&#x27
12. 222.▲
    by lhl 3y ago

    I put together a list of OpenAI API compatibility layers for local LLMs recently: https://llm-tracker.info/books/llms/page/openai-api-compatib... Some like c0sogi/llama-api are pretty neat because they s
13. 223.▲
    by lhl 3y ago

    The UM790 is pretty quiet under normal use, but has about a +7dB wooshing when maxed out (not too annoying, but definitely noticeable - it's a bit quieter than my Framework laptop under stress testing). Notebookcheck has a full review
14. 224.▲
    by lhl 3y ago

    I'm not sure where you're getting your pricing from but in the US a new 4090 is at least $1600, a 7900 XTX is about $1000 (a 20GB 7900 XT is $750-800). Used 3090s are usually about $700. If you're just dipping your toe in I&#
15. 225.▲
    by lhl 3y ago

    Makes sense to me, although if you're using it professionally as an individual, then I would also seriously consider a last-gen A6000 (about $4K) - despite advances in quants, I'd say 48GB is a better minimum bar atm especially if
16. 226.▲
    by lhl 3y ago

    While I haven't given ExLLamaV2 a try yet since my 7900XT is on a Win10 dedicated gaming machine, I recently sat down and figured out getting llama.cpp w/ ROCm running on it: https://llm-tracker.info/books/how
17. 227.▲
    by lhl 3y ago

    I think OmniQuant is notable because it shifts the bend of the curve to 3-bit. While < 3-bit still ramps up, it's notable in that it's usable and doesn't go asymptotic: https://github.com/OpenGVLab/Omn
18. 228.▲
    by lhl 3y ago

    The big problem is that apparently if you have a phone number linked to your account, it can be used to reset your password even with TOTP 2FA enabled which to me, is bonkers: https://twitter.com/TimBeiko/status/
19. 229.▲
    by lhl 3y ago

    Neat to see a new implementation, although I'll note that for those looking for a drop-in replacement for the whisper library , I believe that both https://github.com/guillaumekln/faster-whisper and https:/
20. 230.▲
    by lhl 3y ago

    Ryzen 6000 (Rembrandt) USB4 support was generally underwhelming. Some of it may have been FP7 limitations, but some of it may have been lack of AMD board support (eg, the ZenBooks were supposed to have USB4 support, don't have a dGPU s
21. 231.▲
    by lhl 3y ago

    Hmm, are you sure that's what the suggestion is because the first line of the Verdict says: "If you want an amazing 1,000 nits (IPS) LCD — go for the HP EliteBook 865 G10, Ryzen"... My comment directly talks about why the Sur
22. 232.▲
    by lhl 3y ago

    One note, while the author recommends the 1000-nit HP Sureview display, those SureView panels have significant downsides. A short summary from a Notebookcheck review: https://www.notebookcheck.net/HP-EliteBook-865-G9-Laptop-
23. 233.▲
    by lhl 3y ago

    The wiki looks like a much better organized starting point! I wonder if that could simply be linked after the intro in the BlinkDL repo until search rankings catch up: "For those looking to get started using or testing out RWKV models,
24. 234.▲
    by lhl 3y ago

    Presumably what you're missing is that it's that IshKebab probably doesn't work in AI/ML at all (no links in his profile, but you can judge his post history yourself). Anyone can have voice opinion, but that doesn't
25. 235.▲
    by lhl 3y ago

    Glad to hear that you're focused on improving tooling, as you point out, as there is definitely a gap (that maybe I was too circumspect about it originally, but to be clear, difference wasn't in RKWV's favor). Some of it ma
26. 236.▲
    by lhl 3y ago

    I've been catching up on the papers, and the mechanics of RWKV are interesting, but I'm more of a hands on guy and I played a bunch with RWKV earlier this year when its quality was comparable with the early LLaMA models of similar
27. 237.▲
    by lhl 3y ago

    I spent a few minutes exploring a similar. Using the EFF wordlist (7776 words designed to be unambiguous)[1] you should be able to get down to about 100m^2 blocks (good enough for SAR), the BIP39 word list (2048) is a bit too resolution, bu
28. 238.▲
    by lhl 3y ago

    Here's an English blog that looks like it focuses a lot on Suica/Japanese contactless payments goes a bit more in depth both on the Visa issue and the FeliCa/NFC-F shortage: https://atadistance.net/2023/0
29. 239.▲
    by lhl 3y ago

    The Japanese IC systems all use FeliCa, which not all devices support. I believe Garmin only has it enabled (unsure if this is licensing or hardware) in their East Asian models (Hong Kong's Octopus cards also FeliCa). * https:/&#
30. 240.▲
    by lhl 3y ago

    Sadly due to semiconductor shortage, physical Suica cards are very hard to come by atm: https://www.timeout.com/tokyo/news/sale-of-pasmo-and-suica-c... FWIW I've been using Suica on a Japanese Garmin smart