5 ms·

Why not use a combination of open source and OpenAI models? GPT-3.5 is already beaten by Mixtral and Mistral-Medium. The first one you can host for free and the

Why not use a combination of open source and OpenAI models? GPT-3.5 is already beaten by Mixtral and Mistral-Medium. The first one you can host for free and the second has a darn cheap API while getting really close to GPT-4 performance.

weird-eye-issue 3y ago
"GPT-3.5 is already beaten by Mixtral and Mistral-Medium" Not really relevant when they will be using GPT-4, is it? Plus the powerful features from code interpreter is a night and day difference Not everything is worth saving a bit of money to get worse performance from something that has a worse network effect
- mepian 3y ago
  The article says GPT-3.5 though.
  - onethought 3y ago
    But that’s free. So what did they acquire then? Logins?
    - asabla 3y ago
      Probably organization accounts
    - behnamoh 3y ago
      Probably Enterprise contract to comply with privacy regulations especially regarding students in EU.
  - imutemyteam92 3y ago
    Why would anyone use 3.5 though... I guess organisations move slow. But atleast now they will have a culture and practices of working around it.
  - weird-eye-issue 3y ago
    Bold of you to think I read the article
- jojobas 3y ago
  It's not so much about money as it is about not using proprietary technology with an obscure set of biases.
  - weird-eye-issue 3y ago
    That sounds like all mainstream technology. Might as well get the kids used to it
- behnamoh 3y ago
  You can build a code interpreter pretty quickly. In fact I made one that would debug itself back then using gpt-3.
MattGaiser 3y ago
So neither is easily usable out of the box? That seems to be a significant barrier to adoption. Is a school supposed to set up infrastructure for all of that themselves?
- behnamoh 3y ago
  It's the government not the school. No school has 110k students.
SkyPuncher 3y ago
How do you do that for 110k kids? Who builds the UI? Who manages accounts?
- behnamoh 3y ago
  Tons of UI clones out there. And it shouldn't be difficult for a government to hire a web dev team to mange accounts.
speedgoose 3y ago
OpenAI has the GPUs, Oslo does not. The Mistral cloud offers are perhaps a bit too young to have been considered. And perhaps a bit more risky too.
m3kw9 3y ago
Mistral medium still trash vs gpt4
- behnamoh 3y ago
  Not vs gpt-3.5 which Oslo has acquired.
deleted 3y ago
[deleted]
a321neo 3y ago
Even the free GPT-3.5 is better at smaller European languages than Mixtral/Mistral-Medium. However I think it's a typo when the article says GPT-3.5. It doesn't make sense to "buy" GPT-3.5. They probably meant ChatGPT Plus which includes GPT-4 access (50 messages in a rolling 3 hour window).
- dimask 3y ago
  No, they bought a gpt3.5 license, with privacy related terms that are absent to the normal free version.
- behnamoh 3y ago
  It's not a typo, they bought vanilla chatgpt.
vitorgrs 3y ago
Did you tried to use open source models on languages that is not English? Tried asking about things that is not U.S, etc?
aaron695 3y ago
[dead]
Roark66 3y ago
There are certain "quality of life" issues with the open source ecosystem. I don't blame them for choosing (not at all) OpenAI. For one putting together a chatgpt like experience that supports reliably at least chat history and per user system prompts of different models, requires a chat client that doesn't exist (at least it didn't a couple of months ago when I was looking for one). The closest we have now is a vscode plugin written by one guy I had to modify to work with multiple models (but it is a pretty good OpenAI api client). Also, OpenAI API is a defacto standard for clients talking to AI chat bots. To set it up with Mixtral I had to put together a non trivial system of huggingface TGI server (code extended to support token bias, cfg and negative prompts), and a (very slightly modified) litellm proxy to translate OpenAI api to TGI API. These products were used from the latest github branch and all had various shortcomings that required coding to resolve. Now I can say I truly have an "OpenAI - like" chat experience. But one huge functionality is missing. Function calling. Although implementing it now that I already have context free grammars is not that difficult, it still requires time I haven't found yet. Compare this to just paying a fee every month and getting it all done for you. Still I believe it is very important people recreate what OpenAI offers locally using open source software. Why? Be it is clear AI's like chatgpt is essentially sold well below cost now to hook people up. In 5 years from now once no one will be able to maintain their productivity without it, (not at all)OpenAI will raise the prices 100x and everyone will pay begrudgingly. Then they will raise them 100x more and people will pay too... Unless there is a viable alternative. This is why I(and many like me) are working on having my own.
- vorticalbox 3y ago
  Have you looked at llamafile[0]? It can spin up a openAi compatible API for a while host of LLM. [0] https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile
Irishsteve 3y ago
I wouldn’t be surprised if the deal was somehow done via Microsoft and some contract they already have with the Norwegian government. This is how certain ms tech end up with the Irish government so quickly. Mistral , open source etc do not have a sales force. Shame really
- sigmoid10 3y ago
  Mistral models, in particular Mixtral 8x7, while free to download, are not "free" to run. Even if you have the necessary high end GPUs lying around unused (which is already highly unlikely to begin with), you still need to build and maintain a whole infrastructure around them. It would be extremely difficult to do this more cost effective per computed token than say OpenAI or any of the other big API providers.
liampulles 3y ago
If you are training future workers en-masse, it makes sense to train them to use the industry standard

weird-eye-issue 3y ago

"GPT-3.5 is already beaten by Mixtral and Mistral-Medium" Not really relevant when they will be using GPT-4, is it? Plus the powerful features from code interpreter is a night and day difference Not everything is worth saving a bit of money to get worse performance from something that has a worse network effect

mepian 3y ago
The article says GPT-3.5 though.
- onethought 3y ago
  But that’s free. So what did they acquire then? Logins?
  - asabla 3y ago
    Probably organization accounts
  - behnamoh 3y ago
    Probably Enterprise contract to comply with privacy regulations especially regarding students in EU.
- imutemyteam92 3y ago
  Why would anyone use 3.5 though... I guess organisations move slow. But atleast now they will have a culture and practices of working around it.
- weird-eye-issue 3y ago
  Bold of you to think I read the article
jojobas 3y ago
It's not so much about money as it is about not using proprietary technology with an obscure set of biases.
- weird-eye-issue 3y ago
  That sounds like all mainstream technology. Might as well get the kids used to it
behnamoh 3y ago
You can build a code interpreter pretty quickly. In fact I made one that would debug itself back then using gpt-3.

MattGaiser 3y ago

So neither is easily usable out of the box? That seems to be a significant barrier to adoption. Is a school supposed to set up infrastructure for all of that themselves?

behnamoh 3y ago
It's the government not the school. No school has 110k students.

SkyPuncher 3y ago

How do you do that for 110k kids? Who builds the UI? Who manages accounts?

behnamoh 3y ago
Tons of UI clones out there. And it shouldn't be difficult for a government to hire a web dev team to mange accounts.

speedgoose 3y ago

OpenAI has the GPUs, Oslo does not. The Mistral cloud offers are perhaps a bit too young to have been considered. And perhaps a bit more risky too.

m3kw9 3y ago

Mistral medium still trash vs gpt4

behnamoh 3y ago
Not vs gpt-3.5 which Oslo has acquired.

deleted 3y ago

[deleted]

a321neo 3y ago

Even the free GPT-3.5 is better at smaller European languages than Mixtral/Mistral-Medium. However I think it's a typo when the article says GPT-3.5. It doesn't make sense to "buy" GPT-3.5. They probably meant ChatGPT Plus which includes GPT-4 access (50 messages in a rolling 3 hour window).

dimask 3y ago
No, they bought a gpt3.5 license, with privacy related terms that are absent to the normal free version.
behnamoh 3y ago
It's not a typo, they bought vanilla chatgpt.

vitorgrs 3y ago

Did you tried to use open source models on languages that is not English? Tried asking about things that is not U.S, etc?

aaron695 3y ago

[dead]

Roark66 3y ago

There are certain "quality of life" issues with the open source ecosystem. I don't blame them for choosing (not at all) OpenAI. For one putting together a chatgpt like experience that supports reliably at least chat history and per user system prompts of different models, requires a chat client that doesn't exist (at least it didn't a couple of months ago when I was looking for one). The closest we have now is a vscode plugin written by one guy I had to modify to work with multiple models (but it is a pretty good OpenAI api client). Also, OpenAI API is a defacto standard for clients talking to AI chat bots. To set it up with Mixtral I had to put together a non trivial system of huggingface TGI server (code extended to support token bias, cfg and negative prompts), and a (very slightly modified) litellm proxy to translate OpenAI api to TGI API. These products were used from the latest github branch and all had various shortcomings that required coding to resolve. Now I can say I truly have an "OpenAI - like" chat experience. But one huge functionality is missing. Function calling. Although implementing it now that I already have context free grammars is not that difficult, it still requires time I haven't found yet. Compare this to just paying a fee every month and getting it all done for you. Still I believe it is very important people recreate what OpenAI offers locally using open source software. Why? Be it is clear AI's like chatgpt is essentially sold well below cost now to hook people up. In 5 years from now once no one will be able to maintain their productivity without it, (not at all)OpenAI will raise the prices 100x and everyone will pay begrudgingly. Then they will raise them 100x more and people will pay too... Unless there is a viable alternative. This is why I(and many like me) are working on having my own.

vorticalbox 3y ago
Have you looked at llamafile[0]? It can spin up a openAi compatible API for a while host of LLM. [0] https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile

Irishsteve 3y ago

I wouldn’t be surprised if the deal was somehow done via Microsoft and some contract they already have with the Norwegian government. This is how certain ms tech end up with the Irish government so quickly. Mistral , open source etc do not have a sales force. Shame really

sigmoid10 3y ago
Mistral models, in particular Mixtral 8x7, while free to download, are not "free" to run. Even if you have the necessary high end GPUs lying around unused (which is already highly unlikely to begin with), you still need to build and maintain a whole infrastructure around them. It would be extremely difficult to do this more cost effective per computed token than say OpenAI or any of the other big API providers.

liampulles 3y ago

If you are training future workers en-masse, it makes sense to train them to use the industry standard