5 ms·
It's not host your own AI at all! You need like 5 saas api-keys to operate this app. But sure, you can host this UI on your own.
by yonixw 3y ago
It's not host your own AI at all! You need like 5 saas api-keys to operate this app. But sure, you can host this UI on your own.
- causality0 3y agoIt makes me wonder how long it will be until open-source locally-run AI chatbots reach the GPT-4 level. Five years? Ten?
- swyx 3y agoits more a question of hardware progress than software probably. theres a minimum level for these behaviors to emerge
- crooked-v 3y agoThe real limiter is having enough GPU RAM to train and run a usefully large model. Everything else is just fiddly details.
- brucethemoose2 3y agoNot anymore. Llama.cpp and kobold.cpp run well with low vram and non Nvidia GPUs. Huggingface is full of finetunes now, and I believe a 33b model can be finetuned on a single 3090. Llama.cpp is developing some kind of training, but I have no idea what the requirements will be.
- causality0 3y agoTo me, GPT's real "secret sauce" is how it primes the model for interactive prompting instead of just text completion.
- ynniv 3y agoThere are "chat" and "instruct" variants of most models now.
- lettergram 3y agoIt depends on the tasks, I’d argue some of the open source alternatives are at the gpt-4 level already, particularly for code generation. That said, I suspect summarization, translation, etc will take time. I’d suspect under 1 year.
- worldsayshi 3y ago>I’d argue some of the open source alternatives are at the gpt-4 level already, particularly for code generation. Like which one?
- brucethemoose2 3y agoLarge context Chronos 33b (and some mixes) for roleplaying type chat. And there are some very new 65b finetunes I have not tried.
- vorpalhex 3y agoThe open source bot I played with last week (Wizard Vicuna uncensored) was 85% of gpt 3.5 on a VERY hard use case (fiction stories). Maybe a year before we are the level of stablediffusion?
- deleted 3y ago[deleted]
- eob 3y agoWhen it happens, I think it’ll happen on our phones first.
- koheripbal 3y agoGiven the fundamental hardware matrix operations, that seems unlikely, unless it's something very scaled down.
- layoric 3y agoIt doesn't seem to be a major barrier for most (it absolutely is for me). Is there enough of a want/market for tools that integrate with common OSS LLM APIs like oobabooga/Kobold/Novel/vLLM etc? One idea I've been toying with is a BYO model with support for these APIs, eg for IDE integration, brain storming etc. Seems like a good approach to me but how many people would actually bother standing up these LLM engines with APIs to use it?
- roseway4 3y agoWell, it is a round-up of their venture investments in the space ;-) If you're looking to self-host chat memory rather than go all in on Supabase, there's Zep: https://github.com/getzep/zep https://github.com/getzep/zep Full disclosure: I'm a co-author.
- kaliqt 3y agoSupabase is also self-hostable though.
- kiwicopple 3y agoAlso worth noting that they aren’t investors in supabase (but we’re very appreciative that they included us in this stack)
- roseway4 3y agoI stand corrected. Sometimes my cynicism gets the best of me ;-)
- deleted 3y ago[deleted]
- eob 3y agoI think we’re in platform reliance mode for quite a while. I think of all of these SaaS companies as different icons on an abstract AWS console that doesn’t exist yet. We wouldn’t bat an eye at using S3, EC2, and RDS as a host your own setup. The only difference here is that startups are moving faster than incumbents. FWIW that’s one reason why Steamship (disclaimer: I’m the founder) aggregates all AI services under a single API key and interface. It’s to deal with the insane glue-code hassle of running this stuff on your own.
- chillbill 3y agoThat’s fine but the title is wrong
- quickthrower2 3y agoAnd using S3, RDS and EC2 I would argue is hosting your own as you are using foundational components. Same if you run LLaMa on a rented A100.
- chillbill 3y agoNot the same at this point. Setting up properly secure AWS stuff is arguably more demanding than setting up your own box at home.