Patrick_Devine

7 ms·

61.
by Patrick_Devine 3y ago
Kinda weird that the other one isn't on the front page anymore with 200+ upvotes in 3 hours.
62.
by Patrick_Devine 3y ago
That is almost a certainty. That, or it was more likely done one or two tiers down during the APEC conference.
63.
by Patrick_Devine 3y ago
also works w/ `ollama run mistral`.
64.
by Patrick_Devine 3y ago
I'm getting >30 tokens/sec using it with ollama and an M2 Pro. That might be a little slow though because I have a background finetuning job running.
65.
by Patrick_Devine 3y ago
NVidia provides driver support for CUDA inside of WSL2. More details are here: https://docs.nvidia.com/cuda/wsl-user-guide/index.html
66.
by Patrick_Devine 3y ago
Possibly, but a core guiding principle for us is to keep everything as simple as possible. We're a small project, so if we add too many features/permutations it really makes it hard to keep everything working!
67.
by Patrick_Devine 3y ago
One nice thing about Ollama vs. stock llama.cpp is Ollama supports both ggml and gguf models. If you've still got a lot of old ggml bins around you can easily create a model file and use them. I haven't done benchmarking vs. vLLM,
68.
by Patrick_Devine 3y ago
We don't current compile in CLBlast or ROCm support but if there's a lot of demand for this, we'll definitely add it in the future. One concern is not wanting to bloat out the binary size too much (CUDA is already huge!) but
69.
by Patrick_Devine 3y ago
Or, for the easy way of doing it, just install Ollama and run: `ollama run codellama:7b`. If you want the q8 version (which you probably don't), it's `ollama run codellama:7b-code-q8_0`.
70.
by Patrick_Devine 3y ago
Maybe, but that's why things like ollama.ai are trying to fill the gap. It's simple, and you don't need all of the heavy weight enterprise crap if nothing ever leaves your system.
71.
by Patrick_Devine 3y ago
There are several different ways, but the easiest way in my (clearly biased) opinion is just got to ollama.ai, download it, and start playing around. It works out of the box w/ newer Macs, but there are versions for Linux and Windows i
72.
by Patrick_Devine 3y ago
You can run `ollama run wizard-vicuna-uncensored:13b` and it should pull and run it. For llama2 13b, it's `ollama run llama2:13b`. I haven't seen the 13b uncensored version yet. There's a complete list of models at https:&#x
73.
by Patrick_Devine 3y ago
Ollama does work on Linux, it's just that we haven't (yet) made it work with GPUs other than Metal. We'll get there soon, but we're a small team and wanted to make sure everything was working well before adding more plat
74.
by Patrick_Devine 3y ago
Ollama works with Windows and Linux as well too, but doesn't (yet) have GPU support for those platforms. You have to compile it yourself (it's a simple `go build .`), but should work fine (albeit slow). The benefit is you can stil
75.
by Patrick_Devine 3y ago
I had llama2:13b write me a song about Mario to similar to "I'll be there for you" and it came up with: ... We'll travel through the desert, we'll swim through the sea We'll soar through the sky, we'll cli
76.
by Patrick_Devine 3y ago
Not intentional, but that's amazing! Also a pretty wild game that was played in mesoamerica. [1]: https://www.britannica.com/sports/ollama
77.
by Patrick_Devine 3y ago
Yep. Right now we've packaged llama2, vicuna, wizardlm, and orca. The idea is to make it crazy easy to get started though. You do need quite a bit of RAM (16GB should work for the smaller models, 32MB+ for the bigger ones), and for now
78.
by Patrick_Devine 3y ago
I posted something in the Gist, but the prompt can be really finicky. You might want to `ollama pull llama2` again just to make certain you have the latest prompt. We were messing around with it earlier because it was giving some strange an
79.
by Patrick_Devine 3y ago
They're stored in a registry (based on Docker distribution) running on Cloudflare. The model gets broken up into layers, so if you want to create new prompts or parameters, you can create something called a Modelfile (similar to a Dock
80.
by Patrick_Devine 3y ago
Yes, but I shouldn't have to do that, particularly when it works correctly in most other terminals.
81.
by Patrick_Devine 3y ago
Kitty also has the benefit of supporting much better keyboard handling through key codes. This includes things like button press/release/repeat, other keyboard modifiers and better escape handling. More details: https:/&#x2F
82.
by Patrick_Devine 3y ago
Also, Terminal.app is really bad at rendering Unicode block characters correctly. It doesn't space them correctly vertically, so if you have a lot of block chars it looks like total garbage. iTerm/kitty/etc. all render it cor
83.
by Patrick_Devine 4y ago
My calculations for the volume of a 15m x 100m cylinder would be 17,671 m³ (π * (Diameter/2)² * Length). I think you calculated it with 15m as its radius. For an atmosphere similar to the Earth's at sea level, you would only need
84.
by Patrick_Devine 4y ago
I tried building a startup around this back in ~2012 called "Netkine". The basic premise was that compute is fungible and that all compute has a price (regardless of whether it is from a datacenter, or from someone's desktop
85.
by Patrick_Devine 4y ago
The concept of a cheap, post-war house that you can finish out yourself is great... for certain places. There isn't really room in Vancouver for them any more, as [Vancouverism]( https://en.wikipedia.org/wiki/Vancou
86.
by Patrick_Devine 4y ago
It's a combination of the two reasons cited by others here. 1. They may have revenue which is adding to total cash pile; and 2. They may have taken on additional debt Maybe obvious, but probably worth repeating is that raising capital
87.
by Patrick_Devine 4y ago
This exists in places like Calgary (+15 Skywalk) and Edmonton (Pedways), and underground in places like Toronto (PATH) and Montréal (RÉSO) to try and separate pedestrians from snow and being sprayed by slush from cars. From a weather perspe
88.
by Patrick_Devine 4y ago
I've only used Minikube, kind, and k0s as sandboxes for production kubernetes deployments in the cloud (i.e. EKS). Given I'm already using Docker Desktop on my mac laptop though, the easiest thing to do is just use its built-in ku
89.
by Patrick_Devine 4y ago
This seems correct. If you click on "Show Options" it shows the palette w/ 256 colours (i.e. 2^8 bits). Each pixel is just an index into the palette.
90.
by Patrick_Devine 4y ago
I was so bored at my last job that I wrote my own version of Minesweeper that works in your terminal. You can try it out by running: `docker run -it --rm ghcr.io/pdevine/bombitron`. It works pretty well w/ Linux, and also iTe

More ›