3 ms·

isnt the active parameter count more relevant than the total? qwen is a dense model no?

• kzrdude 1mo ago
  For some tasks yes, and we don't know how many active parameters Luna is using..probably less than 27B
  • Alifatisk 1mo ago
    Is Luna MoE? Had no idea OpenAi went that direction
    • kzrdude 1mo ago
      We don’t know but I think we assume that it is MoE or some almost equivalent sparse activation technology.
      • Alifatisk 1mo ago
        Right, I guess the ”Open” in OpenAI means open for speculation.