What gpu do you recommend for this setup?
I'm just looking to run Llama 3.1:8B something that can run 300 tokens in under 10 seconds. What's the cheapest GPU that can do this?
I'm just looking to run Llama 3.1:8B something that can run 300 tokens in under 10 seconds. What's the cheapest GPU that can do this?