Best Open Source LLM for 8GB VRAM in 2024

Dolphin 2.6 Mistral DPO

• Pros:
  • Good results for general use cases.
  • Set and forget model.
• Cons:
  • Might not be the best for specific tasks like coding or story telling.

YI 34B Capybara and RPbird 34B

• Pros:
  • Preferred for general tasks and role playing.
• Cons:
  • Performance might vary based on personal experience.

Mixtral

• Pros:
  • Can be faster with similar results to other models already used.
  • Can run models up to 70B.
• Cons:
  • Results might be poor for some users.

Nous-Hermes-2-SOLAR-10.7B

• Pros:
  • Balanced niche model with excellent performance and results.
• Cons:
  • Might not be suitable for everyone based on personal preferences.

OpenChat

• Pros:
  • Liked by some users.
• Cons:
  • Specific details on performance and suitability are not provided.

Laserxtral-GGUF

• Pros:
  • Recommended for use.
• Cons:
  • No specific drawbacks mentioned.

Overall Tips and Recommendations

• LLMs: Consider YI 34B, RPbird 34B, or Mixtral for better performance.
• RAM Upgrade: Upgrade to at least 32GB RAM or use CPU/RAM models like GGUF for 34B models.
• Usage Consideration: Assess your needs and usage case before choosing a specific LLM for optimization.

Additional Information

• Function Calling: Models like Command R or Functionary are recommended for function calling tasks.
• Knowledge Graph Extraction: Explore models like LLama3 and PHI3 for extracting knowledge graph data efficiently.
• Model Comparison: Utilize online platforms like Hugging Face for comparing different LLMs and finding the best fit for your requirements.

Disclaimer

• Some comments indicate variability in user experiences with different models and the importance of testing models for specific tasks.