fzzzy

16 ms·

241.
by fzzzy 2y ago
Thanks a lot for llm, Simon. I've started to integrate it more and more in what I do as an abstraction layer over llm apis. By the way, do you know of any models that support streaming input tokens to reduce latency to output tokens?
242.
by fzzzy 2y ago
Well, when Llama 1 came out I signed up and downloaded it, and that led me to llama.cpp. I followed the instructions to quantize the model to fit in my graphics card. Then later when more models like llama2 and mixtral came out I would down
243.
by fzzzy 2y ago
First, you should do some experimentation to determine if fine tuning is even needed for you. gpt4 and claude opus have enough context window to ground the model with your knowledge base and one of the best of class hosted models is going t
244.
by fzzzy 2y ago
download llama.cpp convert the fine tuned model into gguf format. choose a number of quantization bits such that the final gguf will fit in your free ram + vram run the llama.cpp server binary. choose the -ngl number of graphics layers whic
245.
by fzzzy 3y ago
That usb a to usb a cable is beyond cursed.
246.
by fzzzy 3y ago
TheBloke's grant ran out.
247.
by fzzzy 3y ago
which two?
248.
by fzzzy 3y ago
Completely agree.
249.
by fzzzy 3y ago
There was a buffer overflow or some other exploit like that in llama.cpp and the gguf format. It has been fixed now, but it's definitely possible. Also weights distributed as python pickles can run arbitrary code.
250.
by fzzzy 3y ago
A 4 bit quant of the new one would still be about 70 gb, so yeah. Gonna need a lot more ram.
251.
by fzzzy 3y ago
LLMs are very bad at counting.
252.
by fzzzy 3y ago
for an easier way to import, download the module you want to import, put it next to the js file you will import it from, and use "import foo from './foo'". You might need to serve it from a local http server.
253.
by fzzzy 3y ago
I have never integrated notebooks into any of my workflows, but it sounds fun. Thanks for sharing, I'll check out your code.
254.
by fzzzy 3y ago
Why aren't you working on it any more? Sounds cool.
255.
by fzzzy 3y ago
That's not enough to get it to load in a browser. Which bundler will you use? Shark jumped.
256.
by fzzzy 3y ago
When I used Proton Glorious Eggroll for a while, it played my entire steam library better than windows. Also, pop os makes nvidia drivers automatic.
257.
by fzzzy 3y ago
It's funny that you say this, because the first thing I tried after ChatGPT came out (3.5-turbo was it?) was writing a haiku. It couldn't do it at all. Also, after 4 came out, it hallucinated an api that wasted a day for me. It&#x
258.
by fzzzy 3y ago
Thank you for correcting my misconception. I thought it classified them as common carriers.
259.
by fzzzy 3y ago
I'm very curious. Can you explain more?
260.
by fzzzy 3y ago
Sorry, I should have said this is my opinion. As my sibling mentioned, courts haven't tested it yet.
261.
by fzzzy 3y ago
If a platform wants section 230 protection, then it can't use a content display algorithm because that is editorial control. [edit] I stand corrected.
262.
by fzzzy 3y ago
The people holding the money bags who don't want the inefficiencies removed, will have their money bags removed by people who do. Not instantly and not perfectly though. Inefficiency can be very resistant to change as you are pointing
263.
by fzzzy 3y ago
Literacy isn't required if a computer can read any text. That is a huge amount of the brain that could be applied to something else.
264.
by fzzzy 3y ago
Isn't the gist that you should be able to trust your teacher?
265.
by fzzzy 3y ago
Every single object in smalltalk is free to have its own implementation of handling a message. In javascript, the runtime decides for the object how to locate the code and how to call it.
266.
by fzzzy 3y ago
I don't think he would think it is a bastardization. HTTP is 100% message passing. You have an object reference (url) and you pass a message to it with POST. State and behavior is encapsulated. In fact, one of the desireable properties
267.
by fzzzy 3y ago
It's incredibly easy to build hypermedia as the engine of application state sites. You generate html pages with state in them, and hyperlinks and forms that filter and change the state. Hateoas didn't invent anything. He literally
268.
by fzzzy 3y ago
How does that verify you were talking to a human?
269.
by fzzzy 3y ago
Let's hope so.
270.
by fzzzy 3y ago
It may become a tradition since weights are so large. Perhaps it started when the Llama torrent link leaked. Then, Mistral decided to release their weights using bittorrent.

More ›