eurekin

31 ms·

481.
by eurekin 3y ago
Can you share more about the usage? Was it really useful, in what cases? Would you recommend it over gpt-4 or codepilot?
482.
by eurekin 3y ago
theBloke cannot rest :)
483.
by eurekin 3y ago
Phew, now I will bump session time to a week, since shorter ones were bothering me a lot. Thank you! :D
484.
by eurekin 3y ago
Just checked on a m1 w 32GB ram. 7b and 13b work as a charm. 70b doesn't fit, even the 2 bit version
485.
by eurekin 3y ago
How did it perform? Was it well received by members?
486.
by eurekin 3y ago
I'd look for llama2 thebloke 70b GGML on hugginface. Update, this might work: https://huggingface.co/TheBloke/Llama-2-70B-GGML Example command mentions the 4bit variant: ./main -m llama-2-70b.ggmlv3.q4_0.bin
487.
by eurekin 3y ago
This is very interesting and actually in the usable realm, for some use cases
488.
by eurekin 3y ago
Interesting. Where can I read more about that?
489.
by eurekin 3y ago
About the RAM. I doubt they wanted to do that, since basic gpu function is to render a frame in as little ms as possible. Currently VRAM is latency optimized on consumer gpus and all memory chips are an inch away from the gpu. Light only tr
490.
by eurekin 3y ago
Whoa, imagine you get a good base LLM model and save all conversations with it. Run a batch process every night to fine tune a LORA on convo dataset. If I ever came across such a chat bot I'd probably freak out as to why it remembers t
491.
by eurekin 3y ago
How many tokens a second?
492.
by eurekin 3y ago
I don't use NVLink
493.
by eurekin 3y ago
Hi, so what are you using the models for?
494.
by eurekin 3y ago
Not really? I just got those cards in separate PCI slots and the Exllama_hf handles spreading the load internally. No NVLink bridge in particular. I use the "20,22" memory split so that the display card has some room for the frame
495.
by eurekin 3y ago
Yes, it runs totally fine. I ran it in Oobabooga/text generation web ui. Nice thing about it is that it autodownloads all necessary gpu binaries on it's own and creates a isolated conda env. I asked same questions on the official
496.
by eurekin 3y ago
TBH, this is what most "open LLM" community is researching right now. Most LORAs or custom models are 7B and sometimes 13B quantized. They fit into 24 GB vram nicely
497.
by eurekin 3y ago
Please explain more. I toyed around with few initial Llama derived ones and they didn't seem very useful. I was interested most using them as some for of a coding assitant. Which models work ok for you and for what usages?
498.
by eurekin 3y ago
I use it only for sketching out some programming ideas in python. It takes a lot of back and forth to get anything useful and a lot of times, I have to workaround: > However, these approaches would be a lot more complex and beyond the sc
499.
by eurekin 3y ago
I did some initial tests and found that whisper.ai gets transcripts a loot better. Might improve punctuation. Also, when asked to summarize, even Vicuna removed all portions of irrelevant talking (like to the crowd on a meeting) and jokes.
500.
by eurekin 3y ago
So much good information is presented now as videos... I'm thinking of automating that. A service, which follows your watched yt videos feed and automatically saving a transcript. Maybe with some LLM summary and few screenshots - as si
501.
by eurekin 3y ago
Whoa, first time I'm reading about Llama Index... How does it work? Where can I learn more?
502.
by eurekin 3y ago
He could've gotten a switch
503.
by eurekin 3y ago
I'm not in any ways an expert in linux. I don't notice errors, but a lot of times I'm totally stumped how easy they jump to radical conclusions: "we're not using because single detail ", "it's just b
504.
by eurekin 3y ago
What's the story with regards to high dynamic range and tone mapping in raytracers in general? I only know of Filmic Blender to advertise this as a feature
505.
by eurekin 3y ago
That figures are really impressive. A single node is still useful for finetuning smaller models. Apple could move the needle in this market a bit
506.
by eurekin 3y ago
I'm still waiting intently for debugging tools to be able to create * object * diagrams of an running app. Debuggers are excellent, but I keep forgetting, if the object in question is @Hdj637 or @HU83NS
507.
by eurekin 3y ago
Seconded. I were playing around for my native language (Polish) and the large models actually blew me away. For example, it handled "przescreenować" spelling correctly, which is an english word with a polish prefix and a conjugate
508.
by eurekin 3y ago
That cheap ram is about 10x slower than the VRAM. Didn't see any actual figures for latency, but there must be a reason, why newer gpus have memory chips on both sides of the PCB, as close to the GPU as possible
509.
by eurekin 3y ago
CUDA works great in a WSL2 vm. Highly recommended. I found that just sound doesn't work (any soundcard operation isn't implemented in the WSL)
510.
by eurekin 3y ago
Interesting. Where are those tools deployed to? Is there any runtime needed on the client? How can I grab a go tool on a pristine machine?

More ›