Mixtral 8x7B v0.1

Mixtral 8x7B v0.1 is Mistral AI's open-weight base large language model built on a sparse Mixture-of-Experts (MoE) architecture, routing each token through 2 of 8 experts for roughly 46B total parameters but only ~12.9B active per token. It is a decoder-only text generation model released under the Apache 2.0 license. Source: supabase.