Summary of Reddit Comments on "RTX 3090 for LLMs"

Comparison of RTX 3090 and RTX 4060 Ti for LLMs:

• RTX 3090 is almost 2X faster than RTX 4060 Ti for inference and training.
• RTX 4060 Ti seems to be severely bottlenecked by memory bandwidth compared to RTX 3090.
• The GTX 1080 Ti is slightly faster at inference than RTX 4060 Ti.
• The 3090 remains the best option for LLMs.

User Observations and Experiences:

• Some users find that RTX 4060 Ti is a better value for its price.
• Overclocking memory on the 4060 Ti can provide a 10-15% speed boost.
• A user was happy to have purchased a 3090 before prices surged.
• Some users express frustration with Nvidia's card limitations.

Additional Insights:

• LLMs may not be effectively run on personal computers due to their size and the need for quantized models.
• Consideration of PCIe configuration, power supply, and cooling is crucial for running multiple GPUs efficiently.
• In some cases, using two RTX 3060 GPUs can provide satisfactory results and balance price, performance, and power requirements.

Alternate Configuration Suggestions:

• Recommendations for specific CPU, motherboard, memory, storage, and power supply components for a build.
• Suggestions to use a miner rack instead of a desktop case for LLM builds.
• Advice on how to distribute layers to optimize GPU usage and performance.

Product Comparison Information:

• Fakespot Reviews analyzed AMD Ryzen 9 5900X with a trustworthy rating.
• A user shared a PCPartPicker list for a build featuring an Intel Core i9-13900K CPU and an EVGA RTX 3090.

Based on the Reddit comments, the RTX 3090 is generally favored over the RTX 4060 Ti for LLM tasks due to superior performance and efficiency. There are various considerations highlighted by users regarding hardware configurations, limitations of GPUs, and experiences with different setups for LLM tasks.