4 ms·

Mistral instruct doesn't have system prompt AFAIK. Also llama chat system prompt is very useless in my testing.

rrr_oh_man 3y ago
It seems to work well on GPT4All (macOS) with system prompts? Can you link to any doc why it shouldn't work?
refulgentis 3y ago
It does; and if it's LLaMa 2 7B Chat stock from Facebook, that was a little rushed imho, doesn't seem as baked in. (GPTs matters but it's _very_ bizarre who it thinks it's coming from)
nl 3y ago
Mistral Instruct does use a system prompt. You can see the raw format here: https://www.promptingguide.ai/models/mistral-7b#chat-template-for-mistral-7b-instruct https://www.promptingguide.ai/models/mistral-7b#chat-templat... and you can see how LllamaIndex uses it here (as an example): https://github.com/run-llama/llama_index/blob/1d861a9440cdc9e1b2335320a7a2bf26667b90b2/llama_index/llms/llama_utils.py#L27 https://github.com/run-llama/llama_index/blob/1d861a9440cdc9...
- sp332 3y ago
  So the system prompt is just part of the first prompt in a conversation? How is that different from not having a system prompt?
  - nl 3y ago
    No, system prompts are surrounded by [INST] [/INST]
    - sp332 3y ago
      No, every user input is surrounded by those tags. Scroll down from that link you posted and read the next two example prompts.
      - nl 3y ago
        No this isn't the case. Here's the example from https://www.promptingguide.ai/models/mistral-7b#chat-template-for-mistral-7b-instruct https://www.promptingguide.ai/models/mistral-7b#chat-templat... <s>[INST] What is your favorite condiment? [/INST] "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"</s> [INST] The right amount of what? [/INST] Note that the sentence starting "Well, I'm quite partial isn't inside the tag. This example comes from the official system card created by MistralAI available here: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 You can try it: ollama run Mistral "<s>[INST] What is your favorite condiment? [/INST] Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen</s> [INST] The right amount of what? [/INST]"
        sp332 3y ago
        That's the whole context with two user inputs in the INST tags, and one assistant output between/outside of the tags. They're just simulating the beginning of a conversation. You can see this very clearly in the JSON version in the next code block: messages = [ {"role": "user", "content": "What is your favourite condiment?"}, {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"}, {"role": "user", "content": "Do you have mayonnaise recipes?"} ]
        nl 3y ago
        Yes, but this whole block of text gets passed to the LLM on each call as the conversation history. The [INST] tags tell the LLM which parts were inputs (system instructions) as opposed to outputs.
- YetAnotherNick 3y ago
  Look into their official page: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 There is no system prompt. Unless Llamaindex or some other sources cite something from mistral, I am inclined to believe they just copied it from llama.

It seems to work well on GPT4All (macOS) with system prompts? Can you link to any doc why it shouldn't work?

It does; and if it's LLaMa 2 7B Chat stock from Facebook, that was a little rushed imho, doesn't seem as baked in. (GPTs matters but it's _very_ bizarre who it thinks it's coming from)

nl 3y ago

Mistral Instruct does use a system prompt. You can see the raw format here: https://www.promptingguide.ai/models/mistral-7b#chat-template-for-mistral-7b-instruct https://www.promptingguide.ai/models/mistral-7b#chat-templat... and you can see how LllamaIndex uses it here (as an example): https://github.com/run-llama/llama_index/blob/1d861a9440cdc9e1b2335320a7a2bf26667b90b2/llama_index/llms/llama_utils.py#L27 https://github.com/run-llama/llama_index/blob/1d861a9440cdc9...

sp332 3y ago
So the system prompt is just part of the first prompt in a conversation? How is that different from not having a system prompt?
- nl 3y ago
  No, system prompts are surrounded by [INST] [/INST]
  - sp332 3y ago
    No, every user input is surrounded by those tags. Scroll down from that link you posted and read the next two example prompts.
    - nl 3y ago
      No this isn't the case. Here's the example from https://www.promptingguide.ai/models/mistral-7b#chat-template-for-mistral-7b-instruct https://www.promptingguide.ai/models/mistral-7b#chat-templat... <s>[INST] What is your favorite condiment? [/INST] "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"</s> [INST] The right amount of what? [/INST] Note that the sentence starting "Well, I'm quite partial isn't inside the tag. This example comes from the official system card created by MistralAI available here: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 You can try it: ollama run Mistral "<s>[INST] What is your favorite condiment? [/INST] Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen</s> [INST] The right amount of what? [/INST]"
      - sp332 3y ago
        That's the whole context with two user inputs in the INST tags, and one assistant output between/outside of the tags. They're just simulating the beginning of a conversation. You can see this very clearly in the JSON version in the next code block: messages = [ {"role": "user", "content": "What is your favourite condiment?"}, {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"}, {"role": "user", "content": "Do you have mayonnaise recipes?"} ]
        nl 3y ago
        Yes, but this whole block of text gets passed to the LLM on each call as the conversation history. The [INST] tags tell the LLM which parts were inputs (system instructions) as opposed to outputs.
YetAnotherNick 3y ago
Look into their official page: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 There is no system prompt. Unless Llamaindex or some other sources cite something from mistral, I am inclined to believe they just copied it from llama.