4 ms·

In-context learning also seems like the best path to commercializing LLMs. I'm surprised that Microsoft is going the D2C route with ChatGPT rather than commerci

by elhudy 4y ago

In-context learning also seems like the best path to commercializing LLMs. I'm surprised that Microsoft is going the D2C route with ChatGPT rather than commercializing it in a B2B fashion. ...Or maybe that's coming? Imagine feeding an LLM a ton of disparate data sources and asking it questions about that data as a whole. What is a data engineer again, anyway?

tyingq 4y ago
Interesting, though I imagine that will often play out with the business person rephrasing the question or filtering out data until it spits out the answer they expected :) That data engineer can at least push back and have their own opinion.
zenlikethat 4y ago
D2C makes a lot of sense imo because at this point it's most valuable to obtain a huge corpus of (reasonably well labeled) training data (with a side helping of being really great marketing). Just think about all the programmers who are probably rolling their eyes and correcting ChatGPT's work for OpenAI, or at least providing some details. I know it's a reflex for me, not even deliberate, to at least type something in like "Ah I figured out that it was XYZ" and thumbs up or thumbs down. Microsoft will undoubtedly be rolling this all out in Docs, Excel, Teams etc in no time. A good reason to be skeptical about the flood of "GPT for your Notion!" startups.
- elhudy 4y ago
  That's a great point. As someone who isn't really a programmer I wouldn't have seen the value in that feedback until you pointed it out.
  - tripleo1 4y ago
    thumbs up
z3c0 4y ago
It's still not an extractive tool (it's generative), so it stating logical facts based on the data is not something one should bank on. It will likely never be capable of what you're suggesting, without pairing it with an actually-extractive model. The GPT-4 paper outright states that it should not be used for sensitive or high-stakes matters, which is what a lot of enterprises would consider their data pipelines.
- tripleo1 4y ago
  Wikipedia vs Brittanica
- kolinko 4y ago
  Plugins are extractive. Embeddings as well, to an extent.
kolinko 4y ago
It was released as a B2B, almost for a year - only playground and an API. It had limited impact, and only when they built chatgpt they gained significant traction. (I know, davinci3 was not instruct-optimised, but anyone could’ve built something close to a chat before them)

by elhudy 4y ago

tyingq 4y ago

Interesting, though I imagine that will often play out with the business person rephrasing the question or filtering out data until it spits out the answer they expected :) That data engineer can at least push back and have their own opinion.

zenlikethat 4y ago

D2C makes a lot of sense imo because at this point it's most valuable to obtain a huge corpus of (reasonably well labeled) training data (with a side helping of being really great marketing). Just think about all the programmers who are probably rolling their eyes and correcting ChatGPT's work for OpenAI, or at least providing some details. I know it's a reflex for me, not even deliberate, to at least type something in like "Ah I figured out that it was XYZ" and thumbs up or thumbs down. Microsoft will undoubtedly be rolling this all out in Docs, Excel, Teams etc in no time. A good reason to be skeptical about the flood of "GPT for your Notion!" startups.

elhudy 4y ago
That's a great point. As someone who isn't really a programmer I wouldn't have seen the value in that feedback until you pointed it out.
- tripleo1 4y ago
  thumbs up

z3c0 4y ago

It's still not an extractive tool (it's generative), so it stating logical facts based on the data is not something one should bank on. It will likely never be capable of what you're suggesting, without pairing it with an actually-extractive model. The GPT-4 paper outright states that it should not be used for sensitive or high-stakes matters, which is what a lot of enterprises would consider their data pipelines.

tripleo1 4y ago
Wikipedia vs Brittanica
kolinko 4y ago
Plugins are extractive. Embeddings as well, to an extent.

kolinko 4y ago

It was released as a B2B, almost for a year - only playground and an API. It had limited impact, and only when they built chatgpt they gained significant traction. (I know, davinci3 was not instruct-optimised, but anyone could’ve built something close to a chat before them)