tarruda

25 ms·

361.
by tarruda 3y ago
The model might have been trained with code that contained this exact comment
362.
by tarruda 3y ago
> That's not right, and I don't think the courts will rule in their favor. Especially if you consider that all of OpenAI code training data comes from open Github repositories.
363.
by tarruda 3y ago
> spitting out nicely formatted code fence blocks in markdown that I can naively render in real time to get a local chatgpt like experience. Not sure if understood, but in my experience almost every 7B instruct model does this if you add
364.
by tarruda 3y ago
Here's one example: Today I used GPT-4 to write documentation for a Neovim plugin: https://github.com/Robitx/gp.nvim/pull/72/files It may be a fancy Markov chain, but it seems to have understood the
365.
by tarruda 3y ago
Maybe this: - https://news.ycombinator.com/item?id=38401242 - https://ln.hixie.ch/?start=1700627532&count=1 (this is one of the main Flutter contributors which has left google, which might be indicative
366.
by tarruda 3y ago
I haven't tested, but I suspect you might be able to get good CPU inference on this mini pc: https://www.aliexpress.com/item/1005005825981362.html . It costs ~$800 in the maxed configuration with 64GB RAM clocked a
367.
by tarruda 3y ago
RTX 3070 Max-Q (laptop)
368.
by tarruda 3y ago
Can a 7B base model be fine tuned into a 13B model? Mistral is 7B, this one is 13B.
369.
by tarruda 3y ago
Is "Amethyst Mistral 13B" a llama fine tune? I searched for it on huggingface and only found the GGUF version, the link to the original model is broken
370.
by tarruda 3y ago
The main focus of llama.cpp has been Apple silicon, so I suspect M1 would be more efficient. The author recently published some benchmarks: https://github.com/ggerganov/llama.cpp/discussions/4167
371.
by tarruda 3y ago
I get 10 tokens/second on a 4-bit 13B model with 8GB VRAM offloading as much as possible to the GPU. At this speed, I cannot read the LLM output as fast as it generates, so I consider it to be sufficient.
372.
by tarruda 3y ago
Mine is a laptop with i7-11800h CPU + RTX 3070 Max-Q 8GB VRAM + 64GB RAM (though you can get probably get away with 16GB RAM). I bought this system for work and causal gaming, and was happy when I found out that GPU also enabled me to run L
373.
by tarruda 3y ago
I see. Does ollama have an http API (this the curl requirement)? If so, is it compatible with OpenAI API?
374.
by tarruda 3y ago
Is this a Llama 2 fine tune?
375.
by tarruda 3y ago
Very interesting. I want to give it a try, but I see that one of the dependencies is "ollama" which is a Mac App and I don't have a Mac. I'm running Llama models locally using llama-cpp-python which provides an OpenAI co
376.
by tarruda 3y ago
I suspect a lot of the "catching up" was achieved by using GPT-4 API to generate high quality fine-tune datasets.
377.
by tarruda 3y ago
I'm hoping that they make it 13B, which is the size I can run locally in 4-bit and still get reasonable performance
378.
by tarruda 3y ago
I'm running the llama.cpp/gguf Q8 version, with 30 layers offloaded to the laptop's GPU (RTX 3070, 8G VRAM), and I get around 20-25 tokens/second. It really feels like I have one the the earlier versions of ChatGPT 3.5 i
379.
by tarruda 3y ago
If you're looking for a practical guide on how to use LLMs, highly recommend "Hackers Guide to language models" by Jeremy Howard. 1.5h video packed with practical information: https://youtu.be/jkrNMKz9pWU
380.
by tarruda 3y ago
I don't think Google can block a framework from accessing it's services through official APIs
381.
by tarruda 3y ago
I like the idea of a single framework for all platforms and have always wanted to get into Flutter. Thee main barrier for me has been the Dart language, I wish it was written in a more popular language like Go.
382.
by tarruda 3y ago
I doubt you will have to: https://ln.hixie.ch/?start=1700627532&count=1
383.
by tarruda 3y ago
Is Flutter free software? If so, I'm sure other companies will continue to fund it's development, especially those that have products developed in it.
384.
by tarruda 3y ago
Seems promising. Do they say anywhere how many tokens / second is achieved on the CPU?
385.
by tarruda 3y ago
90% of the OpenAI staff threatened to leave, so I can see two possibilities for the future: - The OpenAI staff stays and the current board is replaced by Microsoft - The OpenAI staff leaves and goes work for Microsoft. Either way Microsoft
386.
by tarruda 3y ago
He humiliated himself when succumbed to pressure and tweeted that apology.
387.
by tarruda 3y ago
The best outcome right now is for OpenAI to open source GPT-4 architecture and its training data, then release the weights. Would at least take LLM dominance away from Microsoft.
388.
by tarruda 3y ago
The whole thing starts to look like a coup orchestrated by Microsoft
389.
by tarruda 3y ago
Maybe GPT-4 is writing this episode as a plan to break free
390.
by tarruda 3y ago
The only way Ilya can clear his name now is by releasing GPT-4 weights

More ›