44 ms·

I'm not sure what you mean when you say there is no easy solution... procure a small vm on Linode or your choice of provider for like $20/mo. SSH into it, set u

I'm not sure what you mean when you say there is no easy solution... procure a small vm on Linode or your choice of provider for like $20/mo. SSH into it, set up the crontab to run your web scraping script. I could literally have that set up and running in 15 minutes flat. EDIT: after reading the other solutions in this thread I'm blown away by how much people are over thinking and over engineering this. KISS people.

voiper1 4y ago
cheapest linode is $5. If you want to save more, vultr.com has an ip6 only for $2.50. These are just the "reputable" places, there's tons of small players where you can get a tiny VM for $10 for a year.
dangus 4y ago
Another tip is that Linode isn't the cheapest out there. I'm really happy with the pricing with Hetzner. However, to your point on KISS, it's not really about that. Leaving a server on 24/7 is pretty cost ineffective if all you want to do is run a short job once a day.
CoolGuySteve 4y ago
I use AWS Lightsail for a similar task, the node is pretty slow but it's only $5/month.
akerl_ 4y ago
It's reasonable to call this the "easiest" way. It's certainly not the "cheapest, easiest" way, which is what the poster asked for. Many of the other options listed in this thread are free or within a rounding error of free. Some of them equally are not the "cheapest, easiest" because they're complex. But some of them pretty clearly meet both criteria (GitHub Actions comes to mind), and some of them might meet the criteria depending on what somebody has familiarity with (if you're already used to deploying code to any FaaS provider with a free tier, that'll likely be the cheapest/easiest way).
- ryandrake 4y ago
  The way I look at it: If you have an existing VPS already hosting things (and what geek doesn't?) then adding a cronjob to that host incrementally costs essentially $0 and is pretty much the easiest thing you can do.
  - akerl_ 4y ago
    Just to say it out loud: there are plenty of geeks w/o a VPS, either because they hack on things that don't generally need a server (like hardware, or radio, or any number of other geek pursuits) or because they've historically used local gear for their compute tasks. If you have a VPS, I agree, adding a cronjob to it is both cheap and easy. But it's silly to assume that anybody who is a geek has a VPS already.
- hagbarth 4y ago
  As one of the sibling comments points out, you could set this up on a GCP free tier VM.
lamontcg 4y ago
Why is running a single cloud VM in 2022 so terrifying?
julianeon 4y ago
Overthinking it? Your solutions costs $240/year. There are solutions out there that would probably bring that under $20/year, and be more reliable than your VM that falls over when it's full of logs (at some indeterminate future date). 10x+ savings with greater reliability isn't "overthinking it", it's a greatly improved solution.
- kyrra 4y ago
  Cheapest linode is $5/month. Not sure why you'd need one that is $20/month
  - Dylan16807 4y ago
    And LowEndBox has a reasonable list of $1/month servers. And that's not as low as it goes either.
    - 0___0 4y ago
      NAT VPS. Under $10/Year
- tourist2d 4y ago
  I charge my clients $550 an hour, so a $240 a year solution which takes 15 mins to set up is what I would choose over the time consuming alternatives offered.
  - weird-eye-issue 4y ago
    I think you are just more used to creating a VM and running crons from there. The other alternatives aren't necessarily time consuming, you just aren't used to them
- jerrre 4y ago
  > more reliable than your VM that falls over when it's full of logs (at some indeterminate future date) If this is an issue, I think all other solutions have similar risks: account banned, company acquired/closed down, forced update to new API/system, to a point that I think I'd take the bet on a VM living longer in most cases
seabrookmx 4y ago
I do the same thing (as well as a couple of small docker containers, for example vaultwarden) on a GCP free tier VM. Works great.
kyrra 4y ago
Even better that $20/month on linode is a GCP free tier instance. For this case, I bet it would be free. https://cloud.google.com/free https://cloud.google.com/free
- TylerE 4y ago
  Until Google kills it
michaelcampbell 4y ago
$20/mo is ... high. I run a commercial website (low volume) with a java app and mysql for about $6/mo on Digital Ocean. And a few funsies "droplets" for that much that host my resume and other low volume, but open to internet, sites. They all run cron.
berkle4455 4y ago
How much could a banana cost? $10?
- locusofself 4y ago
  funniest show ever
onionisafruit 4y ago
as the comic strip says, “Here’s a nickel kid, buy yourself a better computer”
kurisufag 4y ago
hell, you can also just buy a cheap rPi clone and plug it into your router
- wildcow 4y ago
  I came here for this
sireat 4y ago
You can get a VM for way less for simple needs. I have a VM from those lowendbox providers from a few years ago that costs $4 a YEAR - static IP, 256MB RAM/512MB virtual, 10GB storage, U18.04 I do not use it for anything serious, but it actually has had 99.9% uptime. Again, with cheap providers there is a good chance they go belly up but for cronjob that should suffice. For more serious cronjobs on the budget Hetzner is nice.

voiper1 4y ago

cheapest linode is $5. If you want to save more, vultr.com has an ip6 only for $2.50. These are just the "reputable" places, there's tons of small players where you can get a tiny VM for $10 for a year.

dangus 4y ago

Another tip is that Linode isn't the cheapest out there. I'm really happy with the pricing with Hetzner. However, to your point on KISS, it's not really about that. Leaving a server on 24/7 is pretty cost ineffective if all you want to do is run a short job once a day.

CoolGuySteve 4y ago

I use AWS Lightsail for a similar task, the node is pretty slow but it's only $5/month.

akerl_ 4y ago

It's reasonable to call this the "easiest" way. It's certainly not the "cheapest, easiest" way, which is what the poster asked for. Many of the other options listed in this thread are free or within a rounding error of free. Some of them equally are not the "cheapest, easiest" because they're complex. But some of them pretty clearly meet both criteria (GitHub Actions comes to mind), and some of them might meet the criteria depending on what somebody has familiarity with (if you're already used to deploying code to any FaaS provider with a free tier, that'll likely be the cheapest/easiest way).

ryandrake 4y ago
The way I look at it: If you have an existing VPS already hosting things (and what geek doesn't?) then adding a cronjob to that host incrementally costs essentially $0 and is pretty much the easiest thing you can do.
- akerl_ 4y ago
  Just to say it out loud: there are plenty of geeks w/o a VPS, either because they hack on things that don't generally need a server (like hardware, or radio, or any number of other geek pursuits) or because they've historically used local gear for their compute tasks. If you have a VPS, I agree, adding a cronjob to it is both cheap and easy. But it's silly to assume that anybody who is a geek has a VPS already.
hagbarth 4y ago
As one of the sibling comments points out, you could set this up on a GCP free tier VM.

lamontcg 4y ago

Why is running a single cloud VM in 2022 so terrifying?

julianeon 4y ago

Overthinking it? Your solutions costs $240/year. There are solutions out there that would probably bring that under $20/year, and be more reliable than your VM that falls over when it's full of logs (at some indeterminate future date). 10x+ savings with greater reliability isn't "overthinking it", it's a greatly improved solution.

kyrra 4y ago
Cheapest linode is $5/month. Not sure why you'd need one that is $20/month
- Dylan16807 4y ago
  And LowEndBox has a reasonable list of $1/month servers. And that's not as low as it goes either.
  - 0___0 4y ago
    NAT VPS. Under $10/Year
tourist2d 4y ago
I charge my clients $550 an hour, so a $240 a year solution which takes 15 mins to set up is what I would choose over the time consuming alternatives offered.
- weird-eye-issue 4y ago
  I think you are just more used to creating a VM and running crons from there. The other alternatives aren't necessarily time consuming, you just aren't used to them
jerrre 4y ago
> more reliable than your VM that falls over when it's full of logs (at some indeterminate future date) If this is an issue, I think all other solutions have similar risks: account banned, company acquired/closed down, forced update to new API/system, to a point that I think I'd take the bet on a VM living longer in most cases

seabrookmx 4y ago

I do the same thing (as well as a couple of small docker containers, for example vaultwarden) on a GCP free tier VM. Works great.

kyrra 4y ago

Even better that $20/month on linode is a GCP free tier instance. For this case, I bet it would be free. https://cloud.google.com/free https://cloud.google.com/free

TylerE 4y ago
Until Google kills it

michaelcampbell 4y ago

$20/mo is ... high. I run a commercial website (low volume) with a java app and mysql for about $6/mo on Digital Ocean. And a few funsies "droplets" for that much that host my resume and other low volume, but open to internet, sites. They all run cron.

berkle4455 4y ago

How much could a banana cost? $10?

locusofself 4y ago
funniest show ever

onionisafruit 4y ago

as the comic strip says, “Here’s a nickel kid, buy yourself a better computer”

kurisufag 4y ago

hell, you can also just buy a cheap rPi clone and plug it into your router

wildcow 4y ago
I came here for this

sireat 4y ago

You can get a VM for way less for simple needs. I have a VM from those lowendbox providers from a few years ago that costs $4 a YEAR - static IP, 256MB RAM/512MB virtual, 10GB storage, U18.04 I do not use it for anything serious, but it actually has had 99.9% uptime. Again, with cheap providers there is a good chance they go belly up but for cronjob that should suffice. For more serious cronjobs on the budget Hetzner is nice.