Budget-safe GPUs
May 11, 2019 10:57 PM Subscribe
I want to start playing around with using GPUs, but I've heard scary warnings about being absolutely sure to properly shut off your session when you're done if you don't want to get a huge surprise bill. Are some GPU providers safer than others when it comes to preset budget or time limits or other safeguards? Is there any other advice for budget-safe GPU use for a nervous novice?
I don't understand this question. A GPU is part of a graphics card you put in your PC. If you're worried it's still consuming lots of electricity after you thought you were done running a GPU-based program, plug your computer into a Kill-A-Watt electricity usage monitor.
posted by Harvey Kilobit at 12:56 AM on May 12, 2019 [1 favorite]
posted by Harvey Kilobit at 12:56 AM on May 12, 2019 [1 favorite]
Response by poster: Sorry if I was unclear, I was asking about rental servers for machine-learning tasks. Some names I've run across for Jupyter are Salamander, SageMaker, Colab, Paperspace Gradient, and Crestle, and Azure and Google Compute Platform for "full servers."
posted by Umami Dearest at 1:20 AM on May 12, 2019
posted by Umami Dearest at 1:20 AM on May 12, 2019
Best answer: If you rent servers on AWS or GCP & forget to stop the rental altogether you’ll rack up a large bill in no time - these machine are expensive (much more expensive than ordinary AWS servers) on a per hour basis & that cost mounts - so your fear isn’t completely ungrounded.
Easiest thing to do is of course to double-check in your AWS (or GCP) dashboard that you’re no longer renting the machines after you’ve finished using them, but being human & prone to error you might want to also set up budget alerts on whichever cloud platform you’re using: On both GCP & AWS you can set budget limits & receive alerts when your usage crosses those limits. Hopefully give you some reassurance that you’re not incurring huge unforeseen bills.
posted by pharm at 4:54 AM on May 12, 2019 [4 favorites]
Easiest thing to do is of course to double-check in your AWS (or GCP) dashboard that you’re no longer renting the machines after you’ve finished using them, but being human & prone to error you might want to also set up budget alerts on whichever cloud platform you’re using: On both GCP & AWS you can set budget limits & receive alerts when your usage crosses those limits. Hopefully give you some reassurance that you’re not incurring huge unforeseen bills.
posted by pharm at 4:54 AM on May 12, 2019 [4 favorites]
Best answer: Yeah Azure, AWS, nor GCP are going to try to trick you into spending more money. You can trust their budget caps. (For that matter, AWS has a generous policy of refunding people who mistakenly used more than they intended.) I'm less certain about the others you mentioned for Jupyter, they sound like resellers? Again I'd trust that they have some sort of budget cap and will honor it. Beyond that, as pharm says it pays to double check things are off when you mean to have them off.
posted by Nelson at 10:49 AM on May 12, 2019 [1 favorite]
posted by Nelson at 10:49 AM on May 12, 2019 [1 favorite]
Best answer: Google Colab provides free access to GPUs according to the NVIDIA developer blog. It’s not as generic as renting a GPU server directly would be — I think it only provides a limited set of machine learning libraries — but it might be a good place to get started.
posted by a device for making your enemy change his mind at 12:21 PM on May 12, 2019 [2 favorites]
posted by a device for making your enemy change his mind at 12:21 PM on May 12, 2019 [2 favorites]
Best answer: Others have given solid advice but I would just make sure you are not putting the cart before the horse in terms of renting GPU time on a cloud service. The AI/ML developers I know generally work on their models for quite a long time on local development systems before moving up to (relatively expensive, but scalable) cloud hardware, and then only to train the model against a full-size dataset. This is time consuming and happens in the context of a lot of other cloud-based infrastructure (obviously you need the dataset, which necessitates a storage bucket, and then the trained model is generally transferred to some other environment for testing, etc.).
Anyway, I would suggest very mildly that you ensure you're not jumping into the deep end of the pool, and potentially spending lots of money on cloud services, when what you really need is a more modest development system capable of working on smaller datasets so you can develop the ML model and training processes.
Premature scaling is as bad as premature optimization, IMO.
posted by Kadin2048 at 5:56 PM on May 12, 2019 [3 favorites]
Anyway, I would suggest very mildly that you ensure you're not jumping into the deep end of the pool, and potentially spending lots of money on cloud services, when what you really need is a more modest development system capable of working on smaller datasets so you can develop the ML model and training processes.
Premature scaling is as bad as premature optimization, IMO.
posted by Kadin2048 at 5:56 PM on May 12, 2019 [3 favorites]
This thread is closed to new comments.
posted by Umami Dearest at 11:00 PM on May 11, 2019