7 ms·
Worth remembering GPT3 was 10-20 times more expensive 2yrs ago. There's a super-moore's law type learning function going on here and I suspect in 1yr GPT4.5 wil
by obblekk 4y ago
Worth remembering GPT3 was 10-20 times more expensive 2yrs ago. There's a super-moore's law type learning function going on here and I suspect in 1yr GPT4.5 will be the same cost as GPT3.5 today.
It's insane... feels like iPhone 3G to iPhone 4 level quality improvement every year.
- Maxion 4y agoGPT-4 is a scary improvement over 3.5, especially for handling code. It will be the literal definition of awesome when these models get a large enough context space to hold a small-medium sized codebase. I've been playing around with it for an hour seeing what it can do to refactor some of the things we have with the most tech debt, and it is astounding how well it does with how little context I give it.
- speedgoose 4y agoThe 32k tokens version should do fairly well on such codebase. I don’t know if it’s the one used in ChatGPT.
- Maxion 4y agoI’m not even kidding when I say I just did 6h or so of work in around 45 minutes
- deleted 4y ago[deleted]
- worldsayshi 4y agoDo you let it refactor your code base? How do you feed the code to it? Just copy paste?
- Maxion 4y agoYup, it’s a bit of a learning curve how to get it to output the most useful stuff with the least amount of prompts. Figure out what your goal is, and then start by giving a wide context and at first it will give wrong answers due to lack of context. With each wrong answer give it some more context fro what you think the solution provide is missing the most. Eventually you get 100% working code, or something so close to it that you can easily and quickly finish it yourself. This is one strategy, but there are many that you can use to get it to reduce the burden of refactoring.
- intelVISA 4y agoI'm worried it's going to decimate the already wobbly jr market - who are the sr devs of tomorrow to wrangle the LLM output after it replaces the jr's job of coding all day?
- theappsecguy 4y agoCertainly it will.
- visarga 4y agoAll programmers that use AI just got a boost. I tried it and it is amazing, it writes complex code on demand like I never seen before. But competition also gets the boost. So now we have to compete with devs armed with GPT-4. Humans are still the differentiating factor, what company will use AI better?
- int_is_compress 4y agoThere are already some cool projects that help LLM go beyond the context window limitation and work with even larger codebases like https://github.com/jerryjliu/llama_index https://github.com/jerryjliu/llama_index and https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain.
- Der_Einzige 4y agoThe fundamental techniques that they use are highly lossey and are far inferior to ultra-long context length models where you can do it all in one prompt. Hate to break it to you and all the others.
- nico 4y agoWhere can someone find and try ultra-long context length models? Any links?
- intelVISA 4y agoThe longest one that is generally available is always going to be yourself :)
- thelittleone 4y agoMy context model is getting shorter and fuzzier.
- EntrePrescott 4y ago… but still the weights are increasing ;)
- hombre_fatal 4y ago> Hate to break it to you and all the others. Jeez. Their comment is quite obviously a complementary one in response to the limitation rather than a corrective one about the limitation.
- woeirua 4y agoThe only thing holding this back now is lack of enough context. That’s the big nut to crack. How do you hold enough information in memory at once?
- falcor84 4y agoGPT4 supports 32k tokens, which I guesstimate would be ~25k code tokens and perhaps ~2.5k lines of code. So already enough to work on a whole module of code at the same time. If you're using small microservices, it might already be good enough.
- IanCal 4y agoAlso if you were doing it personally you'd probably not take the whole codebase, but look at function signatures/docs as you get further away from the code you care about. While there's probably benefits in clever systems for summarising and iteratively working on the code, you can get away with just cramming a load of context in. I have been playing with GPT4 and it's good. It's easy to imagine a bunch of use cases where even the maximal cost of a few dollars ($1.80 ish for the context alone) for a single query is worth it. If it's good enough to save you time, it's very easy to cross the threshold where it's cheaper than a person.
- tmaly 4y agoI can't get it to run a single query today. Are you on the paying plan?
- behnamoh 4y agoThat's incorrect information. The original GPT3 price was about x3 times its current price, not 10-20 times. https://the-decoder.com/openai-cuts-prices-for-gpt-3-by-two-thirds/ https://the-decoder.com/openai-cuts-prices-for-gpt-3-by-two-...
- sebzim4500 4y agoI think he means it was 30x more than gpt-3.5 is today.
- stavros 4y agoYeah but that's also 30x times worse. ChatGPT is much worse-performing than davinci-003.
- avereveard 4y agoyeah, I used to be able to create valid units for WarStuff with original chatgpt, but now all the unit it creates are full of hallucinated traits and special rules that aren't in the source.
- stavros 4y agoIf you try davinci-003, it will definitely be able to do it.
- refulgentis 4y agoNo, something’s off, and I’m interested in finding out what
- stavros 4y agoI don't think so. Try to convince ChatGPT to reply with "perform action: email" whenever you send a message that looks like you want to send an email. It just refuses to do it consistently. It might work once or twice per conversation, but soon it'll be asking you for SMTP credentials. It's an 1.3B parameter model, we can't expect much from it. It's a wonder it's as good as it is.
- alfalfasprout 4y agoKeep in mind OpenAI is able to lose money on GPT*. Inference for the same predictive power will indeed improve (Facebook's recent model is a step in the right direction) but efficiency of running on given hardware likely won't follow moore's law for long.
- Raidion 4y agoAre there truely top tier ML optimized chips? I would imagine the hardware space still having room to grow if just because most of the brainpower having been focused on x86, etc.
- moffkalast 4y agoThere are analog neural net accelerators that can get around lots of existing limitations, but still have a few of their own that need to be figured out.
- pixl97 4y agoIf you believe Nvidia market speak, they took AI and used it to optimize their A100 chips when creating the H100 line, so we're already using artificial brainpower on the problem.