4 ms·

I like the idea. Here is my feedback. 1. Bug: checking "Only MoE models" leaves the list empty. 2. I'd like to see the number of active parameters for MoE mod

I like the idea. Here is my feedback. 1. Bug: checking "Only MoE models" leaves the list empty. 2. I'd like to see the number of active parameters for MoE models. You could make it a parenthetical in the parameters column. 3. Practical RAM/VRAM requirements would be valuable. For example, see this thread on K2 Horizon: https://old.reddit.com/r/LocalLLaMA/comments/1wg4a0u/k2_horizon_lineup_is_out_on_aa_and_once_again_aa/ https://old.reddit.com/r/LocalLLaMA/comments/1wg4a0u/k2_hori.... It is important information that isn't obvious from the model size. The next level of time and effort would be to benchmark the models yourself. I am interested in x86-64 CPU benchmarks, but that's probably niche and there will be more interest in benchmarks on a modest GPU. The most common amount of VRAM on Steam (https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam https://store.steampowered.com/hwsurvey/Steam-Hardware-Softw...) is 16 GB, followed closely by 8 GB.

nunodonato 19d ago
hey thanks a lot for the feedback! 1. works fine for me, are you sure you don't have any other filters active that might result in 0 models? 2. good idea, a few people have requested that. It becomes a little bit more difficult when models have engrams, but I will consider! 3. There are some tools to do that, I personally don't like them. I understand the convenience but I am staying away from that. There are many factors at play, not all models work the same way, even if they use the same VRAM. Not to speak of using quants and how each quant may affect a model differently
- networked 19d ago
  You're welcome! This is what I see when I check the MoE checkbox. The list at the top of the page has no models: https://paste.dbohdan.com/1nagxp2-s90bk/screenshot.png https://paste.dbohdan.com/1nagxp2-s90bk/screenshot.png.
  - nunodonato 18d ago
    ahh I missed that first table. thanks, fixed it :)

nunodonato 19d ago

hey thanks a lot for the feedback! 1. works fine for me, are you sure you don't have any other filters active that might result in 0 models? 2. good idea, a few people have requested that. It becomes a little bit more difficult when models have engrams, but I will consider! 3. There are some tools to do that, I personally don't like them. I understand the convenience but I am staying away from that. There are many factors at play, not all models work the same way, even if they use the same VRAM. Not to speak of using quants and how each quant may affect a model differently

networked 19d ago
You're welcome! This is what I see when I check the MoE checkbox. The list at the top of the page has no models: https://paste.dbohdan.com/1nagxp2-s90bk/screenshot.png https://paste.dbohdan.com/1nagxp2-s90bk/screenshot.png.
- nunodonato 18d ago
  ahh I missed that first table. thanks, fixed it :)