3 ms·
Downloaded params.json - GeLU & 2D RoPE are used for the vision adapter. The vocab size also got larger - 131072 in size. Also Mistral's latest tokenizer PR sh
by danielhanchen 2y ago
Downloaded params.json - GeLU & 2D RoPE are used for the vision adapter. The vocab size also got larger - 131072 in size.
Also Mistral's latest tokenizer PR shows 3 extra new tokens (the image, the start & end).
The torrent is 24GB, and I guess the implementation will be up in HF in a few days!
Exciting times!
- generalizations 2y agoAt 24gb, I wonder how small the quantized models will go.