4 ms·
Context Window is 32k (present in the blog). I suppose number of Params, MOE setup are intentionally not revealed. Numbers on well known benchmarks and comparis
by ruckfool 2y ago
Context Window is 32k (present in the blog). I suppose number of Params, MOE setup are intentionally not revealed. Numbers on well known benchmarks and comparison with Google and Open AI is also published later in the blog.