4 ms·

I love their approach! Mistral seems to be what I long ago hoped OpenAI to be.

by Timon3 3y ago

unshavedyak 3y ago
Makes me wonder if i should be giving my money to them instead of OpenAI. Do they have a Chat Interface like ChatGPT? Without first signing up, it's looking like they only have endpoints?
- yieldcrv 3y ago
  LM Studio I use dolphin mixstral 8x7B way more than ChatGPT4 for the past several months if any of this sentence makes sense, I have a M1 with 64gb RAM and I use 5 bit quantizing with metal with 10,000 token context window. Its around 21 tokens/sec which is a little faster than the text speed that ChatGPT4 responds with
  - josephg 3y ago
    What is quality like compared to GPT4?
    - yieldcrv 3y ago
      for the kinds of discussions I have it is very close I often have discussions about political topics that I don’t have a complete history on, things that are hard to get a non-emotional non-accusative response from in a forum or in person, licensing questions about obscure professions, liability questions, roleplays without the preachiness about the topic, coding, brand ideas and naming Lots of stuff that I dont want in any cloud or sent online, but then it just became second nature to primarily use it. the hallucinations are heavier. like it will give you specific links that it made up, and never apologize like chatgpt will, it will say “no, thats a real link” so, much more of a bullshitter LM Studio makes it easy to change the temperament though with a variety of premade system prompts, or allowing custom ones very easily I mainly use ChatGPT4 for multimodal like audio conversations, figuring a DIY problem out by sending it a photo of what I’m looking at, having it render photos on how to use something - I was at a gym and had it look at every piece of equipment there and tell me what it was and show me how a human would use it (I notice this is at odds with another comment, I havent used ChatGPT3.5 in nearly a year, but my experience with ChatGPT4 on the aforementioned topics is similar to my output in mixtral)
    - moralestapia 3y ago
      I can answer that from experience (just built a couple Q&A style sites), although a bit subjectively. Compared with the GPTs, Mixtral (and derivatives) perform: * 5% of the time as good as GPT4 * 75% of the time on par with GPT3.5 * 20% of the time a bit worse than GPT3.5 (too chatty and hallucinates with ease, although one could argue a better prompt could improve things a lot) Advantages of Mixtral, for me: * Cost 10-100x cheaper * Faster completion time * More deterministic output, in the sense that if you run the same query several times you get the same answers back (but they could be wrong). GPT almost always gives me an answer of great quality BUT with a lot of variance across them; even with temperature set to zero. This is a PITA when you want some sort of predictable, structured output like a JSON object.
      - sjwhevvvvvsj 3y ago
        Another factor is they monkey around with GPT constantly. The GPT you get on Monday may be totally weird on Friday. Whereas for a local model it doesn’t change unless YOU change it.
- terhechte 3y ago
  They have an api and it’s quite cheap. Their model mistral medium is also very powerful. I use it instead of GPT 4 regularly
  - suslik 3y ago
    In my experience it is comparable with chatGPT 3.5 but is more expensive (I still use it cause I hate guardrails).
    - thierrydamiba 3y ago
      Does mistral not have any guardrails?
      - suslik 3y ago
        Mistral API has a flag in the json payload to remove guardrails.
      - sjwhevvvvvsj 3y ago
        Nope! It’s actually for adults to make their own choices and not some SV public relations firm nerfing it.
    - sjwhevvvvvsj 3y ago
      You can buy a good GPU and quickly come below cost for GPT APIs depending on your workload. I’m doing millions of tasks, so eating $8k on a custom build for ML up front is a long term cost savings. Not least of all that every other month there’s an even better model.
      - rmbyrro 3y ago
        You'd need high usage density to justify investing on a GPU just to run a 70b model locally.
      - suslik 3y ago
        I don't see myself doing millions of tasks on 70b+ models. Work pays for more gpt4 that I can chew; I can run up to 30b models on my mac with a decent speed, and I can use mistral APIs when I travel without powering my own home server. I can see how that buying nvidia GPUs would make sense for a heavy user... Also, I had a gaming addiction for years and try to stay away from anything that can be used for gaming.

I love their approach! Mistral seems to be what I long ago hoped OpenAI to be.

by Timon3 3y ago

I love their approach! Mistral seems to be what I long ago hoped OpenAI to be.

Makes me wonder if i should be giving my money to them instead of OpenAI. Do they have a Chat Interface like ChatGPT? Without first signing up, it's looking like they only have endpoints?

yieldcrv 3y ago
LM Studio I use dolphin mixstral 8x7B way more than ChatGPT4 for the past several months if any of this sentence makes sense, I have a M1 with 64gb RAM and I use 5 bit quantizing with metal with 10,000 token context window. Its around 21 tokens/sec which is a little faster than the text speed that ChatGPT4 responds with
- josephg 3y ago
  What is quality like compared to GPT4?
  - yieldcrv 3y ago
    for the kinds of discussions I have it is very close I often have discussions about political topics that I don’t have a complete history on, things that are hard to get a non-emotional non-accusative response from in a forum or in person, licensing questions about obscure professions, liability questions, roleplays without the preachiness about the topic, coding, brand ideas and naming Lots of stuff that I dont want in any cloud or sent online, but then it just became second nature to primarily use it. the hallucinations are heavier. like it will give you specific links that it made up, and never apologize like chatgpt will, it will say “no, thats a real link” so, much more of a bullshitter LM Studio makes it easy to change the temperament though with a variety of premade system prompts, or allowing custom ones very easily I mainly use ChatGPT4 for multimodal like audio conversations, figuring a DIY problem out by sending it a photo of what I’m looking at, having it render photos on how to use something - I was at a gym and had it look at every piece of equipment there and tell me what it was and show me how a human would use it (I notice this is at odds with another comment, I havent used ChatGPT3.5 in nearly a year, but my experience with ChatGPT4 on the aforementioned topics is similar to my output in mixtral)
  - moralestapia 3y ago
    I can answer that from experience (just built a couple Q&A style sites), although a bit subjectively. Compared with the GPTs, Mixtral (and derivatives) perform: * 5% of the time as good as GPT4 * 75% of the time on par with GPT3.5 * 20% of the time a bit worse than GPT3.5 (too chatty and hallucinates with ease, although one could argue a better prompt could improve things a lot) Advantages of Mixtral, for me: * Cost 10-100x cheaper * Faster completion time * More deterministic output, in the sense that if you run the same query several times you get the same answers back (but they could be wrong). GPT almost always gives me an answer of great quality BUT with a lot of variance across them; even with temperature set to zero. This is a PITA when you want some sort of predictable, structured output like a JSON object.
    - sjwhevvvvvsj 3y ago
      Another factor is they monkey around with GPT constantly. The GPT you get on Monday may be totally weird on Friday. Whereas for a local model it doesn’t change unless YOU change it.
terhechte 3y ago
They have an api and it’s quite cheap. Their model mistral medium is also very powerful. I use it instead of GPT 4 regularly
- suslik 3y ago
  In my experience it is comparable with chatGPT 3.5 but is more expensive (I still use it cause I hate guardrails).
  - thierrydamiba 3y ago
    Does mistral not have any guardrails?
    - suslik 3y ago
      Mistral API has a flag in the json payload to remove guardrails.
    - sjwhevvvvvsj 3y ago
      Nope! It’s actually for adults to make their own choices and not some SV public relations firm nerfing it.
  - sjwhevvvvvsj 3y ago
    You can buy a good GPU and quickly come below cost for GPT APIs depending on your workload. I’m doing millions of tasks, so eating $8k on a custom build for ML up front is a long term cost savings. Not least of all that every other month there’s an even better model.
    - rmbyrro 3y ago
      You'd need high usage density to justify investing on a GPU just to run a 70b model locally.
    - suslik 3y ago
      I don't see myself doing millions of tasks on 70b+ models. Work pays for more gpt4 that I can chew; I can run up to 30b models on my mac with a decent speed, and I can use mistral APIs when I travel without powering my own home server. I can see how that buying nvidia GPUs would make sense for a heavy user... Also, I had a gaming addiction for years and try to stay away from anything that can be used for gaming.