Best Open Source Language Models for 8GB VRAM

Gemma 27B

• Supports various context lengths and produces good results.
• Doesn't support system prompts.
• Lack of truly uncensored model.
• User quote: "Gemma does not support system prompts and there is no truly uncensored model."

Yi-1.5-34B

• Fine-tunable model with versions suitable for 24GB cards.
• User suggests fine-tuning Yi-1.5-34B for specific tasks.
• User quote: "Yi-1.5-34B as a model that could be fine tuned and a quant version should fit on a 24GB card."

Dolphin 2.6 Mistral DPO

• Model suitable for various tasks like coding, storytelling, and general use.
• Easy to use for general tasks.
• User quote: "For a set and forget model that will do most that I throw at it, this is my model."

Nous-Hermes-2-SOLAR-10.7B

• Well-balanced model between 7B mistral models and MoE models.
• Good performance and results.
• User quote: "The performance and results with this model are excellent."

Mixtral

• Varied opinions on Mixtral's performance.
• Some users report poor results with Mixtral.
• Others prefer Mixtral, especially Mixtral-8x7B-Instruct version.
• User quote: "Mixtral always gave me poor results."

Bagel on Yi 34B Capybara

• Preferred model for general tasks.
• RPbird 34B is favored for role-playing.
• User quote: "My favorite is YI 34B Capybara for general tasks & RPbird 34b for role playing."

CodeLlama 34b

• Suitable for fine-tuning on a 24GB card.
• Enables faster fine-tuning with reduced memory usage.
• User quote: "CodeLlama 34b also fits now on ur 24GB card, since Unsloth reduces memory by 70% and makes finetuning 2x faster."

Pros and Cons:

Gemma 27B

• Pros: Good results, supports various context lengths.
• Cons: Lacks system prompts and truly uncensored model.

Yi-1.5-34B

• Pros: Fine-tunable, suitable for 24GB cards.
• Cons: None mentioned in comments.

Dolphin 2.6 Mistral DPO

• Pros: Versatile for multiple tasks.
• Cons: None mentioned in comments.

Nous-Hermes-2-SOLAR-10.7B

• Pros: Well-balanced model, good performance.
• Cons: None mentioned in comments.

Mixtral

• Pros: Varied models for different tasks.
• Cons: Some users report poor results.

Bagel on Yi 34B Capybara

• Pros: Suitable for general tasks.
• Cons: None mentioned in comments.

CodeLlama 34b

• Pros: Enables faster fine-tuning.
• Cons: None mentioned in comments.