3 ms·

We (LM Studio) found the bug with the 31B model and a fix will be going out hopefully tonight

• c0wb0yc0d3r 6mo ago
  I am not deep in this world. What does it mean when you (LM Studio) fixed a bug in a model Google released?
  • why_only_15 6mo ago
    I am in this world, but am not familiar with this specifically. My guess is that they found a bug with their implementation of the model using the weights Google released. These bugs are often difficult to track down because the only indication is that the model is worse with your implementation than with someone else's.
  • khimaros 6mo ago
    llama.cpp also fixed some chat template issues this afternoon. could be related.
  • airspresso 6mo ago
    There is a surprising amount of code needed in each of the inference frameworks (LM Studio, llama.cpp, etc) to support each new model release. For example to format the input in the right way using a chat template, to parse the output properly with the model-specific tokens the model provider decided to standardize on for their model, and more. This particular instance was a fix to the output parsing [1] in LM Studio, described like this: "Adds value type parsers that use <|\"|> as string delimiters instead of JSON's double quotes, and disables json-to-schema conversion for these types." [1]: https://github.com/ggml-org/llama.cpp/pull/21326/commits/a507ae5498257c6fd0835e85c78825298b2252ba https://github.com/ggml-org/llama.cpp/pull/21326/commits/a50... edit: formatting