8 ms·
Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddin
by whytai 3y ago
Every day this video ages more and more poorly [1].
categories of startups that will be affected by these launches:
- vectorDB startups -> don't need embeddings anymore
- file processing startups -> don't need to process files anymore
- fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming
- cost reduction startups -> they literally lowered prices and increased rate limits
- structuring startups -> json mode and GPT4 turbo with better output matching
- vertical ai agent startups -> GPT marketplace
- anthropic/claude -> now GPT-turbo has 128k context window!
That being said, Sam Altman is an incredible founder for being able to have this close a watch on the market. Pretty much any "ai tooling" startup that was created in the past year was affected by this announcement.
For those asking: vectorDB, chunking, retrieval, and RAG are all implemented in a new stateful AI for you! No need to do it yourself anymore. [2]
Exciting times to be a developer!
[1] https://youtu.be/smHw9kEwcgM https://youtu.be/smHw9kEwcgM
[2] https://openai.com/blog/new-models-and-developer-products-announced-at-devday https://openai.com/blog/new-models-and-developer-products-an...
- Der_Einzige 3y agoStartups built around actual AI tools, like if one formed around automatic1111 or oogabooga, would be unaffected, but because so much VC money went to the wrong places in this space, a whole lot of people are about to be burned hard.
- throwaway-jim 3y agodamn hahaha it's oobabooga not oogabooga
- atleastoptimal 3y agoThere will be a lot of startups who rely on marketing aggressively to boomer-led companies who don't know what email is and hoping their assistant never types OpenAI into Google for them.
- deleted 3y ago[deleted]
- yawnxyz 3y agoi'm excited for the open source, local inferencing tech to catchup. The bar's been raised.
- morkalork 3y agoIf you want to be a start-up using AI, you have to be in another industry with access to data and a market that OpenAI/MS/Google can't or won't touch. Otherwise you end up eaten like above.
- ushakov 3y agoWe just launched our AI-based API-Testing tool (https://ai.stepci.com https://ai.stepci.com), despite having competitors like GitHub Co-Pilot. Why? Because they lack specificity. We're domain experts, we know how to prompt it correctly to get the best results for a given domain. The moat is having model do one task extremely well rather than do 100 things "alright"
- darkwater 3y agoSorry to be blunt but they can be totally right, if you do not succeed and have to shut down your startup.
- ushakov 3y agoIt certainly will be a fun experience. But our current belief is that LLMs are a commodity and the real value is in (application-specific) products built on top of them.
- Rastonbury 3y agoExactly, everyone is so pessimistic but for every AWS sku there is a billion dollar startup that leads that market.
- esafak 3y agoIf you just launched it is too soon to speak.
- ushakov 3y agoOf course! Today our assumption is that LLMs are commodities and our job is to get the most out of them for the type of problem we're solving (API Testing for us!)
- colordrops 3y agoI haven't been paying attention, why are embeddings not needed anymore?
- lazzlazzlazz 3y agoOP is incorrect. Embeddings are still needed since (1) context windows can't contain all data and (2) data memorization and continuous retraining is not yet viable.
- nextworddev 3y ago"yet"
- coding123 3y agoIt's also much slower. LLMs are generating text token at a time. That's not very good for search. Pre-search tokenization however, probably a good fit for LLMs.
- zwily 3y agoBut the common use case of using a vector DB to pull in augmentation appears to now be handled by the Assistants API. I haven't dug into the details yet but it appears you can upload files and the contents will be used (likely with some sort of vector searching happening behind the scenes).
- emadabdulrahim 3y agoI believe their API can be stateful now: https://openai.com/blog/new-models-and-developer-products-announced-at-devday#:~:text=A%20key%20change%20introduced%20by%20this%20API%20is%20persistent%20and%20infinitely%20long%20threads%2C%20which%20allow%20developers%20to%20hand%20off%20thread%20state%20management%20to%20OpenAI%20and%20work%20around%20context%20window%20constraints.%20With%20the%20Assistants%20API%2C%20you%20simply%20add%20each%20new%20message%20to%20an%20existing%20thread https://openai.com/blog/new-models-and-developer-products-an...
- sharemywin 3y ago
- lazzlazzlazz 3y agoEmbeddings are still important (context windows can't contain all data + memorization and continuous retraining is not yet viable), and vertical AI agent startups can still lead on UX.
- Finbarr 3y agoContext windows can't contain all data... yet.
- dragonwriter 3y agoSeparate embedding DBs are less important if you are working with OpenAI, since their Assistants API exists to (among other things) let you bring in additional data and let them worry about parsing it, storing it, and doing RAG with it. Its like "serverless", but for Vector DBs and RAG implementations instead of servers.
- ren_engineer 3y agodepends on how much developers are willing to embrace the risk of building everything on OpenAI and getting locked onto their platform. What's stopping OpenAI from cranking up the inference pricing once they choke out the competition? That combined with the expanded context length makes it seem like they are trying to lead developers towards just throwing everything into context without much thought, which could be painful down the road
- keithwhor 3y agoI suspect it is in OpenAI's interest to have their API as a loss leader for the foreseeable future, and keep margins slim once they've cornered the market. The playbook here isn't to lock in developers and jack up the API price, it's the marketplace play: attract developers, identify the highest-margin highest-volume vertical segments built atop the platform, then gobble them up with new software. They can then either act as a distributor and take a marketplace fee or go full Amazon and start competing in their own marketplace.
- stuckkeys 3y agoReminds me of that sales entrapment approach from cloud providers. “Here is your free $400, go do your thing” next thing you know you have build so much on there already that it is not worth the time and effort to try and allocate it regardless of the 2k bill increase -haha. Good times.
- klabb3 3y ago> depends on how much developers are willing to […] getting locked onto their platform. I mean.. the lock in risks have been known with every new technology since forever now, and not just the risk but the actual costs are very real. People still buy HP printers with InkDRM and companies willingly write petabytes of data into AWS that they can’t even afford to egress at current prices. To be clear, I despise this business practice more than most, but those of us who care are screaming into the void. People are surprisingly eager to walk into a leaking boat, as long as thousands of others are as well.
- ky0ung 3y ago
- baq 3y agoChecking hn and product hunt a few times a week gives you most of that awareness and I don’t need to remind you about the person behind hn ‘sama’ handle.
- bluecrab 3y agoVector DBs should never have existed in the first place. I feel sorry for the agent startups though.
- m3kw9 3y agoHow does this absolve vectordbs
- danielbln 3y agoIt doesn't, but semantic search is a lot less relevant if you can squeeze 350 pages of text into the context.
- quinncom 3y agoOpenAI charges for all those input tokens. If an app requires squeezing 350 pages of content in every request is going to cost more. Vector DB still relevant for cost and speed.
- gk1 3y agoBesides the cost factor, stuffing the context window can actually make the results worse. https://www.pinecone.io/blog/why-use-retrieval-instead-of-larger-context/ https://www.pinecone.io/blog/why-use-retrieval-instead-of-la...
- dragonwriter 3y agoIf you are using OpenAI, the new Assistants API looks like itnwill handle internally what you used to handle externally with a vector DB for RAG (and for some things, GPT-4-Turbo’s 128k context window will make it unnecessary entirely.) There are some other uses for Vector DBs than RAG for LLMs, and there are reasons people might use non-OpenAI LLMs with RAG, so there is still a role for VectorDBs, but it shrunk a lot with this.
- oezi 3y agoOpenAI is still way too expensive to run a corporate knowledge base on top
- echelon 3y agoWe don't want Open AI to win everything.
- blibble 3y agoHN is quite notorious for that Dropbox comment I suspect that video is going to end up more notorious, it's even funnier given it's the VCs themselves
- arcanemachiner 3y agoMore context, please. EDIT: I guess it's this: https://news.ycombinator.com/item?id=8863#9224 https://news.ycombinator.com/item?id=8863#9224
- blibble 3y agothat's the one
- thisgoesnowhere 3y agoI'm firmly in the camp that in a vacuum that comment looks dumb but the thread was actually great. Those were valid concerns at the time and the market for non technical file storage like they were building was non existant. Perfectly rational to be skeptical and Drew answered all his questions with well thought out responses.
- TeMPOraL 3y agoThe infamous comment itself made sense in context, too.
- bilsbie 3y agoWhy don’t you need embedding?
- monkeydust 3y agoYou might. Depends what your trying to do. For RAG seems like they can 'take care of it' but embeddings also offer powerful semantic search and retrieval ignoring LLMs.
- riku_iki 3y ago> - vectorDB startups -> don't need embeddings anymore they don't provide embedings, but storage and query engines for embeddings, so still very relevant > - file processing startups -> don't need to process files anymore curious what is that exactly?.. > - vertical ai agent startups -> GPT marketplace sure, those startups will be selling their agents on marketplace
- make3 3y agothey definitely do provide embeddings, https://openai.com/blog/new-models-and-developer-products-announced-at-devday https://openai.com/blog/new-models-and-developer-products-an... ctrl+f retrieval, "... won't need to ... compute or store embeddings"
- riku_iki 3y agoI mean embeddingsDB startups don't provide embeddings. They provide databases which allows to store and query computed embeddings (e.g. computed by ChatGPT), so they are complimentary services.
- taf2 3y agoYeah I still see a chat bot being able to look for related information in a database as useful. But I see it as just one of many tools a good chat experience will require. 128k context means for me there other applications to explore and larger tasks to accomplish with fewer api requests. Better chat history and context not getting lost
- visarga 3y agoIt's easy to host your query engine somewhere else and integrate it as a search function in chatGPT. Quite easy to switch providers of search.
- obmelvin 3y agoAs in, use an existing search and call it via 'function calling' as part of the assistants routine - rather than uploading documents to the assistant API?
- deleted 3y ago[deleted]
- larodi 3y agoWell, if said startups were visionaries, the could've known better the business they're entering. On the other hand - there are plenty of VC-inflated balloons, making lots of noise, that everyone would be happy to see go. If you mean these startups - well, farewell. There's plenty more to innovate, really, saying OpenAI killed startups it's like saying that PHP/Wordpress/NameIt killed small shops doing static HTML. or IBM killing the... typewriter companies. Well, as I said - they could've known better. Competition is not always to blame.
- karmasimida 3y agoTBH those are low-hanging fruits for OpenAI. Much of the value still being captured by OpenAI's own model. The sad thing is, GPT-4 is its own league in the whole LLM game, whatever those other startups are selling, it isn't competing with OpenAI.
- Yadayadaaaa 3y agoJust because something is great doesn't mean that others can't compete. Even a secondary good product can easily be successful due to a company having invested too much, not being aware of openai (ai progress in general), due to some magic integration, etc. If it would be only me, no one would buy azure or aws but just gcp.
- cityzen 3y agoWhere is the part about embeddings?
- teaearlgraycold 3y agoI’ve been keeping my eye on a YC startup for the last few months that I interviewed with this summer. They’ve been set back so many times. It looks like they’re just “ball chasing”. They started as a chatbot app before chatgpt launched. Then they were a RAG file processing app, then enterprise-hosted chat. I lost track of where they are now but they were certainly affected by this announcement. You know you’re doing the wrong thing if you dread the OpenAI keynotes. Pick a niche, stop riding on OpenAI’s coat tails.
- seydor 3y agomore startups should focus on foundation models, it's where the meat is. Ideally there won't be a need for any startup as the platform should be able to self-build whatever the customer wants.
- treprinum 3y agoThere is not much info about retrieval/RAG in their docs at the moment - did you find any example on how is the retrieval supposed to work and how to give it access to a DB?
- mvkel 3y agoProbably best not to make your company about features that a frontier AI company would have a high probability of adding in the next 6-12 months.
- deleted 3y ago[deleted]
- felixding 3y agoOfftopic. I find it's amusing that we not only have "chatGPT" but now also "vectorDB". Apple's influence is really strong.
- andrewjl 3y agoNone of those categories really fall under the second order category mentioned in the video. Using their analogy they all sound more like a mapping provider versus something like Uber.