- Mixtral 8x7B exl2 is now supported natively in oobabooga!
Mixtral of experts | Mistral AI | Open source models Its like running a 70B model, and it competes with GPT3 5 and other 70B's It has fast inference (40 tokens a sec on a measly 3090, close to what il get on a 13B) Overall it can do code, RP, summarization, Q A and many more tasks that a 7B-13B would have hard time doing all of those,
- Setting ideal Mixtral-Instruct Settings : r LocalLLaMA - Reddit
But Mixtral, uh, breaks this 'rule' regularly? What Interestingly, unlike the near-total confidence tokens I was used to seeing in the past, it's possible for Mixtral to be so occasionally confident in the next token choice that it is the only positive probability assigned period The above token in question
- Tips for Mixtral 8x7B Instruct: Yea or Nay? : r SillyTavernAI - Reddit
Recently (two days ago), I started using Mixtral 8x7B (so not the ' Instruct') because I was looking for a model with a large context (tired of being limited to 4k or 16k) for uncensored roleplay, and I found that one
- ⬛ LLM Comparison Test: Mixtral-8x7B, Mistral . . . - Reddit
With Mixtral, it feels very natural, and errors are minimal I've used some of the models specifically finetuned for German, but they didn't feel much better quality-wise than standard Llama Mistral, they seemed to switch back to English less, but the quality didn't seem much higher
- Opinions on Mixtral 0. 1 8x7b and Mistral 0. 2 7b - Reddit
Mixtral 8x7b Q5_0 is better at understanding and following complex prompts and explain logic, while Mistral 0 2 7b Q8_0 is a bit better at writing text in general, like stories and hypothetical events (this could just be a preference on my part)
- Mistrsal officially announces Mixtral 8x7B : r LocalLLaMA - Reddit
Mixtral outperforms Llama 2 70B on most benchmarks with 6x faster inference It is the strongest open-weight model with a permissive license and the best model overall regarding cost performance trade-offs
- Mixtral 8x7B is a scaled-down GPT-4 : r LocalLLaMA - Reddit
The conclusion is that (probably) Mixtral 8x7B uses a very similar architecture to that of GPT-4, but scaled down:
- How to prime Mixtral 8x7B for NSFW : r SillyTavernAI - Reddit
Mixtral is IMHO the best model (as in open source) out there, and so far I haven't had the same quality with any of the finetunes of it The only problem is removing that partial alignment both for NSFW RP and also for creative writing, when you want to have a high-quality but sardonic style that is usually filtered out by the 'alignment'
|