brucethemoose2

11 ms·

91.
by brucethemoose2 3y ago
Its not that cost prohibitive! You can run GPT 3.5-replacing AIs (aka 34Bs) on a $1500 PC... If you can find a used 3090 now.
92.
by brucethemoose2 3y ago
Or more specifically, be able to look at everything all at once instead of in chunks.
93.
by brucethemoose2 3y ago
And Sodium! (For minecraft)
94.
by brucethemoose2 3y ago
We've been testing it in the local llm Discords, turns out its just a llama 7B finetune that can run on any old GPU (which is cool). https://huggingface.co/brucethemoose/LargeWorldModel_LWM-Tex... https:/&#x
95.
by brucethemoose2 3y ago
Its more of a tech demo since its llama 7B (a model that is, TBH, obsoleted by mistral 7B), and its dataset is not that great. We've had Yi 6B 200K context for some time, which is also quite good. The problem, of course, is hardware re
96.
by brucethemoose2 3y ago
Its llama 7B, so anything that runs that. You can quantize the cache and fit quite a bit on GPUs. At least 75k on my mere 24GB 3090, maybe 200K with a fancy quantization repo.
97.
by brucethemoose2 3y ago
They have at least some of the dataset uploaded: https://huggingface.co/LargeWorldModel And the model page specifically mention Books3
98.
by brucethemoose2 3y ago
How very reasonable. ...But the climate change problem is now social, and I think its largely a symptom of the information dysfunction that is getting worse across the globe.
99.
by brucethemoose2 3y ago
> the more technical users will leverage llama.cpp to run whatever models they are interested in. Llama.cpp is much slower, and does not have built-in RAG. TRT-LLM is a finicky deployment grade framework, and TBH having it packaged in
100.
by brucethemoose2 3y ago
Fooocus is the fastest way to try SDXL/SDXL turbo with good quality. ComfyUI is cool but very DIY. You don't get good results unless you wrap your head around all the augmentations and defaults. No idea if it will support cascade.
101.
by brucethemoose2 3y ago
What's more, they can presumably be swapped in and out like the SDXL base + refiner, right?
102.
by brucethemoose2 3y ago
Indeed. The unusual thing seems to be how two such objects ended up in such an orbit... But that doesn't seem to be something people would upvote. > Difficulty of explaining the current wide orbit The current separation between the
103.
by brucethemoose2 3y ago
1.9 solar masses is not too unusual.
104.
by brucethemoose2 3y ago
There's more ROCm compatibility than you'd think. I'm at a startup, and we'd love to be using MI300Xs. Out stack works with it, they are amazing, our wallet is open... But we can't! We simply can't find any.
105.
by brucethemoose2 3y ago
Google prices the TPUs pretty exorbitantly, actually. But they give a lot of TPU time away for research, which is nice. It seems Intel Gaudi 2 is priced in a sweeter spot, but I've never head of anyone but Intel using them.
106.
by brucethemoose2 3y ago
We are getting that (in early 2025?) With AMD Strix Halo. 40 CUs, 256 bit LPDDR5X, 16 CPU cores. Or so the rumors say.
107.
by brucethemoose2 3y ago
AWS/Azure prices are really awful TBH. There are already much better places to get GPUs.
108.
by brucethemoose2 3y ago
There is a lot of silicon consumer GPUs don't "need" for AI. But on the other hand the software stack is very mature and they are heavily amortized by the huge volume, so its kinda hard to argue with. In fact its so good that
109.
by brucethemoose2 3y ago
This question is confusing to me. Functionary is a series of function calling llama architecture finetuned models, not a new base model, if thats what you mean. Hence, it can be turned into a gguf like any other llama finetune. It just has
110.
by brucethemoose2 3y ago
Grammar has nothing to do with gguf, as it's logit processing (and that is being integrated into other backends as well).
111.
by brucethemoose2 3y ago
Also, Mass Effect Andromeda. Bethesda Game Studios is getting there. I don't know about indie games as much, but I know Distant Worlds 2 got bit by its exotic engine.
112.
by brucethemoose2 3y ago
On the contrary, it's not one-size-fits-all. It doesn't support everything other backends need (like certain quantization schemes). If it somehow does, then its not universally compatible and that's just going to create more
113.
by brucethemoose2 3y ago
Everyone uses that for openai compatible api requests. Its actually great. Most backends and frontends "just work" with each other because they all talk with openai (albeit with some caveats).
114.
by brucethemoose2 3y ago
Also, game dev is hard. Engine dev is really hard. There is a growing graveyard of games/devs suffering from bespoke engine woes instead of just using Unreal...
115.
by brucethemoose2 3y ago
Even Google is making a llama.cpp wrapper. Technically a wrapper of a wrapper, though llama-cpp-python is quite excellent... TBH I would just recommend using that and all its under-the-radar features (like grammar, and built in function cal
116.
by brucethemoose2 3y ago
Or trying to run large maps in sim games. Speaking as a 7800X3D owner, I don't think any CPU on the planet is sufficient for modded Rimworld yet.
117.
by brucethemoose2 3y ago
Actually... machine learning. One would think its all GPU. But you'd be suprised how single-core-speed limited parts of certain tasks are.
118.
by brucethemoose2 3y ago
"AI" GPUs like the 3090 and 3060 are starting to shoot up in price, unfortunately.
119.
by brucethemoose2 3y ago
I mean, it depends on the circumstances. In the US, where its kind of a possibility... I'm not sure. But I'd be pretty pissed at the parties I feel have an underlying responsibility for the conflict (like, trying to stay relevan
120.
by brucethemoose2 3y ago
> tone mapping is part of artistic creativity And its done incorrectly way too often. (I am sitting on no high horse here, I'm guilty of it as well).

More ›