Mac mini is good as a hardware I control so I know it is private / secure. But is it powerful enough for my use-cases? (Testing local LLMs on Mac mini)
Other options are Open model labs + rented GPU or Closed model labs API
- I can Rent GPUs instead of owning hardware as persistent (always loaded) or serverless (cold until triggered)
- Closed model chat interface often uses interactions to train. APIs often don’t
Better vs owning the hardware because I can scale up the infrastructure and cost as needed
Considerations:
- Who is renting the GPU - what are their privacy/security policies? Could be less secure than even API to closed models. I would need to redact Vault secrets before sending
- More complicated system overall - SSH/security layers
- Could destroy instance after every job for additional privacy/security
- API pricing is per token, GPU pricing is per time