4 ms·

Honestly if you can get yourself a video card with at least 12gb of RAM and I highly recommend running open source models locally. It doesn’t have to be the la

Honestly if you can get yourself a video card with at least 12gb of RAM and I highly recommend running open source models locally. It doesn’t have to be the latest fancy gaming card - I run an old 2016 Tesla P100 ($200AUD~) which is fine for 16B q4 models. They’re more configurable, there’s no subscription model, you contribute to them and meaningful ways, there is a privacy concern if you’re passing your own data and it’s perhaps strangely a more fulfilling experience. I’ve got mine hooked up to Home Assistant so I can automate things with Siri and shortcuts, I’m using it finding patterns and code and several decades of digital notes. If you’re not sure where to start, you can try some models out locally on your machine, using apps like https://lmstudio.ai https://lmstudio.ai or GPT4All with almost no ML knowledge. Then if you want to level up and customise things and run them on the server you can use localAI, Text generation web UI etc which provide openAI compatible APIs. A good 13B model with decent prompting feels pretty on par with chatGPT (3.5) for general tasks, but can be configured to be quite good at very specific things. https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui https://github.com/go-skynet/LocalAI https://github.com/go-skynet/LocalAI https://github.com/gitmylo/audio-webui https://github.com/gitmylo/audio-webui

ChatGTP 3y ago
A good 13B model with decent prompting feels pretty on par with chatGPT (3.5) for general tasks, but can be configured to be quite good at very specific things. What is a "general task" to you? People at work were using ChatGPT-4, langchain etc and I dont' really see them being any more special, useful, productive or available so I haven't been bothered to invest. In fact some of them seem to have slowed right down on their usuage if using it at all. I can see that it might be fun to play with generating stories or whatever but I mean, what am I missing from not investing in these things?
- smcleod 3y ago
  There’s plenty of resources out there as to why such tools might be useful but no one is forcing you to be interested or find value in them. Personally I think we’re at that point a bit like when the horse and carriage met the motor car and from a career / technical point of view I’d rather learn how an engine works than raise horses. Ok maybe a poor analogy but I’m sure you get the drift.

ChatGTP 3y ago

A good 13B model with decent prompting feels pretty on par with chatGPT (3.5) for general tasks, but can be configured to be quite good at very specific things. What is a "general task" to you? People at work were using ChatGPT-4, langchain etc and I dont' really see them being any more special, useful, productive or available so I haven't been bothered to invest. In fact some of them seem to have slowed right down on their usuage if using it at all. I can see that it might be fun to play with generating stories or whatever but I mean, what am I missing from not investing in these things?

smcleod 3y ago
There’s plenty of resources out there as to why such tools might be useful but no one is forcing you to be interested or find value in them. Personally I think we’re at that point a bit like when the horse and carriage met the motor car and from a career / technical point of view I’d rather learn how an engine works than raise horses. Ok maybe a poor analogy but I’m sure you get the drift.