lhl

38 ms·

361.
by lhl 4y ago
6800 is RDNA2, not RDNA3. The latter is still waiting for ROCm support 4 months post-launch: https://github.com/RadeonOpenCompute/ROCm/issues/1813
362.
by lhl 4y ago
For a general guide, I recommend: https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni... There's a subreddit r/LocalLLaMA that seems like the most active community focused on self-hosting L
363.
by lhl 4y ago
> Or, OpenAI added a calculator function. I just asked my local LLaMA 30Bq4 "What is 23214 + 34243?" and it gave me 57457 so there's probably some innate math abilities in larger LLMs. That being said, I'm much more
364.
by lhl 4y ago
I think different technologies are going to appeal to different audiences and users. I've been pretty blown away by generative AI, both stuff like Stable Diffusion and now, these ultra-powerful LLMs (although I didn't much about i
365.
by lhl 4y ago
You don't need to just take one source. OpenAI authored their own paper [1] on the economic impacts of just LLMs: "Our findings reveal that around 80% of the U.S. workforce could have at least 10% of their work tasks affected by t
366.
by lhl 4y ago
From my evals on nat.dev I found claude instant to give great responses and yes, avg 3-4X faster than 3.5, but one big difference atm is that anyone can sign up and get access to gpt-3.5-turbo right now , but claude is still gated behind a
367.
by lhl 4y ago
BLOOM's training corpus ROOTS did make some efforts at removing PII https://arxiv.org/pdf/2303.03915.pdf btw, but AFAICT that was not at the behest of the French government.
368.
by lhl 4y ago
> Right now your only real option is OpenAI (or maybe Anthropic) > The only reason we have LLAMA is because Meta threw us a bone IMO, this is pretty inaccurate, you can look at my other post in the thread to see how many other recent
369.
by lhl 4y ago
The ones I saw mentioned so far were Flan, Cerebras, GPT-J, and RWKV. Not yet mentioned: * Pythia https://github.com/EleutherAI/pythia * GLM-130B https://github.com/THUDM/GLM-130B - see also ChatG
370.
by lhl 4y ago
It's worth pointing out that size isn't everything. From Meta's benchmarking [1] LLaMA 33B outperforms GPT-3 175B, Gopher 280B, Chinchilla 70B and even matches PaLM 540B on a bunch of common evals. Those interested in doing m
371.
by lhl 4y ago
I'm pretty confident that the landscape is going to look very different by the end of the year, as there are so many people poking around now. I think that significantly smaller models will definitely be good enough for specialized tas
372.
by lhl 4y ago
If you (or anyone else is) interested in the topic, I'd highly recommend giving some of these a look: Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, et al. “Sparks of Artifici
373.
by lhl 4y ago
I was playing around w/ a lot of these models as well, and was surprised by how bad LLaMA performed vs it's benchmark scores [1][2][3]. However, recently @tyfon mentioned he had great success w/ LLaMA and shared his prompt [4
374.
by lhl 4y ago
You have to request access/waitlist for the alpha features, but I believe they're bucketing roll-outs somewhat randomly. Eg, I have browsing but I don't have plugins or the code interpreter version. Also, I still don't h
375.
by lhl 4y ago
If you have access to 4 (you are a ChatGPT Plus subscriber) you will have a pull-down to choose that model. One detail is that 3.5 has a green icon and 4 has a black icon. I see a lot of people online posting screenshots of their experience
376.
by lhl 4y ago
Ah thanks a lot, I tried out the llama.cpp examples before the k-shot chat prompts, this is definitely much better! I have a 5950X as well, but sadly, token generation is a bit too slow for me now. (I've had turbo turned off for effici
377.
by lhl 4y ago
The dangers of AGI I don't think are very related with consciousness or sentience (maybe the saddest outcome if it killed us all and weren't at all). I'm still catching up on my reading, but the existential risk of AGI as I u
378.
by lhl 4y ago
Sounds like you've had some more success w/ w/ raw LLaMA - would def be interested in how you're prompting it. BTW, for those interested (looks like the markdown rendering is a bit messed up) but here are some notes I&#x
379.
by lhl 4y ago
I'm in the process of testing the various self-hosted LLMs. I have an M2 MBA laptop and a 5950X w/ 64GB RAM and an RTX 4090 (24GB VRAM). I've used ChatGPT 3.5 and 4 quite a bit, and have done a bunch of comparisons w/ na
380.
by lhl 4y ago
The default loader doesn't seem to let you load quantized models but if you use something like https://github.com/oobabooga/text-generation-webui you can 1) use the model with `--load-in-8bit` which halves the mem
381.
by lhl 4y ago
I'm always surprised on r/linuxhardware (another good resource) the number of people that ask questions without doing basic searches online. I'd definitely second the Arch and Gentoo Wikis as excellent community maintained re
382.
by lhl 4y ago
One nice thing about Lenovo's business line is they fully support LVFS for easy firmware updates via Linux. Here's their system updates for the X1 Carbon g10 for example: https://fwupd.org/lvfs/devices/co
383.
by lhl 4y ago
Again, I agree that hardware compatibility issues suck... Personally, I think life's too short to live with glitches like that - ASMedia chips are used in a lot of PCIe USB cards these days as well, but I've had good luck w/
384.
by lhl 4y ago
While having hardware compatibility issues suck, I'm not sure what your problems have to do with AMD's mobile APUs, which have their USB2/3/4 controllers implemented on-die (block diagram: https://cdn.videocar
385.
by lhl 4y ago
I wrote this a while back comparing Intel 12th Gen to AMD 6000, and power/perf should apply even more to the 13th Gen vs 7040 (Zen4 + RDNA3): The reason that people want AMD CPUs is simple - they’re much better than what Intel is offer
386.
by lhl 4y ago
I saw this being posted, but that's not very definitive and it's not clear if the traffic is an attack or not: https://www.reddit.com/r/ChatGPT/comments/11wqscn/china_is_c...
387.
by lhl 4y ago
Idle power consumption is good, but sadly as soon as you get into near-idle/light usage (eg, listening to Spotify in the background, or even running a bash script echoing to your terminal, the power jumps a lot (I place the blame mainl
388.
by lhl 4y ago
Framework is in fact are working on the firmware to improve battery life, primarily with the Intel TB4 (Burnside Bridge) retimers. Here's the work to reduce power consumption w/ the DP expansion card: https://community.
389.
by lhl 4y ago
I've been running Linux laptops for years now and have been enjoying using (well, mostly poking around) with a Framework laptop, however, recently I've been doing a lot of long-haul travel and it turns out that the battery life on
390.
by lhl 4y ago
> No There's no point to having further factual discussion if you aren't going to read the referenced papers. From the 2018 study I referenced you can see that DXA-positive and BMI-positive correlation was 2,668/2,778 (Tab

More ›