What would you recommend for an always-on device for running cron jobs?
October 25, 2017 2:01 PM Subscribe
Needs: internet-enabled, single digit GB storage sufficient (would have a few sqlite databases), cheap (<$100), low power consumption, unix-y, easy to interface with my MacBook. Use for now: updating a few websites, Twitter bot-type stuff (shell, python, R).
I have an additional old MacBook without a battery or power cable/adapter. I could repurpose that. I might not need a battery, since none of the scripts are mission critical. On the other hand, a new adapter will cost $50 and other components likely on the verge of failing, too.
I don't want to pay monthly for some cloud-hosted service, like Linode.
There's a related AskMe from 4 years ago. Our use cases differ and lots of the links have died since it was posted, so I figure there might be some new shiny option.
Thank you!
I have an additional old MacBook without a battery or power cable/adapter. I could repurpose that. I might not need a battery, since none of the scripts are mission critical. On the other hand, a new adapter will cost $50 and other components likely on the verge of failing, too.
I don't want to pay monthly for some cloud-hosted service, like Linode.
There's a related AskMe from 4 years ago. Our use cases differ and lots of the links have died since it was posted, so I figure there might be some new shiny option.
Thank you!
If the cron jobs are rare (like every few minutes to hour) and short (say under a minute) then the modern way of doing this is with serverless, e.g. AWS Lambda. These instances run for the duration of a single script and you get charged only for the few seconds it runs.
Working in this mode can require changes to your code (or even a change of language), and the memory and side-effect constraints can make certain things difficult or impossible. But it's worth looking into.
I have a handful of Twitter bots that post this way, and it costs me less than a cent per month (which is less than I'd spend on electricity for an always-on low power device). Here's the github repo that shows how the code needs to be set up, and documents one way of deploying them.
posted by caek at 2:32 PM on October 25, 2017 [5 favorites]
Working in this mode can require changes to your code (or even a change of language), and the memory and side-effect constraints can make certain things difficult or impossible. But it's worth looking into.
I have a handful of Twitter bots that post this way, and it costs me less than a cent per month (which is less than I'd spend on electricity for an always-on low power device). Here's the github repo that shows how the code needs to be set up, and documents one way of deploying them.
posted by caek at 2:32 PM on October 25, 2017 [5 favorites]
I agree with caek, AWS Lambda is a nice solution here. On the other hand, an AWS ec2 t2.nano costs about 4 bucks a month, so it seems penny wise and pound foolish to refuse that option just because it's a monthly charge. It's likely to be more capable and reliable than something running in your house for $100, and it'll take two years for the ec2 costs to reach $100...
posted by primethyme at 2:42 PM on October 25, 2017
posted by primethyme at 2:42 PM on October 25, 2017
You could also do Google App Engine, which has built-in cron support so your instances wouldn't be running all the time if there's nothing to do, and has a pretty decent free amount for each project (you can also disable billing for a project so that it can only use the free resources.) It should only require a little code change depending on your usage.
posted by thewumpusisdead at 3:05 PM on October 25, 2017
posted by thewumpusisdead at 3:05 PM on October 25, 2017
Response by poster: Butting in: Would I not need to pair AWS Lambda with S3 (or similar) to store my databases? Kneejerk reaction to Google App Engine is that I'm not so interested in switching to their datastore.
posted by cichlid ceilidh at 4:22 PM on October 25, 2017
posted by cichlid ceilidh at 4:22 PM on October 25, 2017
Yeah, to persist data on Lambda you'll have to use S3 or one of their database services. It's not interactive or unix-y at all.
I'd go with a Pi 3 -- heck it'll be more fun. AFAIK it supports Sqlite (and also the database format is cross-platform, so you should be ok there)
posted by RobotVoodooPower at 6:01 PM on October 25, 2017
I'd go with a Pi 3 -- heck it'll be more fun. AFAIK it supports Sqlite (and also the database format is cross-platform, so you should be ok there)
posted by RobotVoodooPower at 6:01 PM on October 25, 2017
Yes, if you want to persist the output of a Lambda function (or read persisted input), you need to write it to a filesystem or database. In AWS world that means S3 for plain files, or something like Redshift or RDS. It gets complicated, but if you're optimizing for cost, it's probably the cheapest way of doing things for hobbyist-scale cron jobs.
If your data is simple, one possibility would be to use Lambda to read/write to something like the Google Sheets API.
posted by caek at 7:26 PM on October 25, 2017
If your data is simple, one possibility would be to use Lambda to read/write to something like the Google Sheets API.
posted by caek at 7:26 PM on October 25, 2017
You could use the free level of heroku. The scheduler add-on makes it easy to run jobs as frequently at every 10 minutes, and there may be other add-ons that give even finer grained control.
posted by Cogito at 8:07 PM on October 26, 2017
posted by Cogito at 8:07 PM on October 26, 2017
Response by poster: Heroku's not economical if I use it to store my DBs, which are larger than covered by their free tier; that would cost me $9/month. I'll likely go with something like the Raspberry Pi 3.
posted by cichlid ceilidh at 8:30 AM on October 27, 2017
posted by cichlid ceilidh at 8:30 AM on October 27, 2017
« Older What is wrong with my new coffee grinders?... | What's the key to a good Halloween party? Newer »
This thread is closed to new comments.
posted by mbrubeck at 2:08 PM on October 25, 2017