Mixtral 8x22B
Mixtral 8x22B is a sparse Mixture-of-Experts (SMoE) model with 39B active parameters out of 141B, offering cost efficiency. It is fluent in multiple languages, has strong math/coding capabilities, and 64K tokens context window. Released under Apache 2.0 license.