4 ms·

I just wish there was a single VM that wasn't SSD backed. I write web crawlers sometimes and I usually only have to parse the HTML / JS once into JSON or SQL, b

I just wish there was a single VM that wasn't SSD backed. I write web crawlers sometimes and I usually only have to parse the HTML / JS once into JSON or SQL, but I like to keep the data backed up just in case I need to re-parse it later, or to see the changes between crawls. It's completely impossible for me to even fit it on their $640 / month box. But I love Digital Ocean for everything else.

stadeschuldt 11y ago
Have you looked into the Kimsufi boxes? Sometimes they are available for 5 EUR/month. They come with a Atom cpu and 500 GB of storage.
icebraining 11y ago
That's why I like TransIP. Cheap SSD-backed VM + large network drive (up to 400TB).
shortstuffsushi 11y ago
Couldn't you potentially push that aspect elsewhere, into a data storage solution like S3 or something (not sure of the cost efficiency, not a huge cloud guy myself). Then you'd just be utilizing the DO server for processing. It would have the added benefit of allowing you to scale your crawling horizontally if you wanted, as all your servers could have access to the same set of data.
- icebraining 11y ago
  The traffic between datacenters would be significantly slower and more costly, though. Besides, S3 is not a replacement for a real drive.
  - shortstuffsushi 11y ago
    It would be slower than on disk for sure, so if that's a concern, it wouldn't be a good choice. If the poster is just storing off results, pushing it to S3, or even potentially a document database could be a logical choice. They said they're just using it for occasionally re-parsing at a later date, so immediacy doesn't seem like a huge concern. Just a thought, at any rate.
- deleted 11y ago
  [deleted]
- 3pt14159 11y ago
  Yeah I could, and I currently do - but I don't like to. I want the data on the same network for lots of reasons and I also don't like maintaining two sets of deployment systems. I'd prefer to just use Cloud66 + Digital Ocean and focus on writing code, not setting up and maintaining servers.
  - shortstuffsushi 11y ago
    Oh yeah, I totally understand that, was just offering an alternative as a non-ideal workaround. I have the same set up for a service I offer, primarily because (afaik) it's hard to beat S3 pricing.
sfilipov 11y ago
Vultr, who have offerings very similar to DO, have a class of VMs called "Storage Series"[1]. In my experience their uptime is similar to DO and their performance is better. Might be of interest to you: [1] https://www.vultr.com/pricing/ https://www.vultr.com/pricing/
vfuse 11y ago
Take a look at serverhub.com ssd cached vps, $5 per month for 500GB of diskspace.

stadeschuldt 11y ago

Have you looked into the Kimsufi boxes? Sometimes they are available for 5 EUR/month. They come with a Atom cpu and 500 GB of storage.

icebraining 11y ago

That's why I like TransIP. Cheap SSD-backed VM + large network drive (up to 400TB).

shortstuffsushi 11y ago

Couldn't you potentially push that aspect elsewhere, into a data storage solution like S3 or something (not sure of the cost efficiency, not a huge cloud guy myself). Then you'd just be utilizing the DO server for processing. It would have the added benefit of allowing you to scale your crawling horizontally if you wanted, as all your servers could have access to the same set of data.

icebraining 11y ago
The traffic between datacenters would be significantly slower and more costly, though. Besides, S3 is not a replacement for a real drive.
- shortstuffsushi 11y ago
  It would be slower than on disk for sure, so if that's a concern, it wouldn't be a good choice. If the poster is just storing off results, pushing it to S3, or even potentially a document database could be a logical choice. They said they're just using it for occasionally re-parsing at a later date, so immediacy doesn't seem like a huge concern. Just a thought, at any rate.
deleted 11y ago
[deleted]
3pt14159 11y ago
Yeah I could, and I currently do - but I don't like to. I want the data on the same network for lots of reasons and I also don't like maintaining two sets of deployment systems. I'd prefer to just use Cloud66 + Digital Ocean and focus on writing code, not setting up and maintaining servers.
- shortstuffsushi 11y ago
  Oh yeah, I totally understand that, was just offering an alternative as a non-ideal workaround. I have the same set up for a service I offer, primarily because (afaik) it's hard to beat S3 pricing.

sfilipov 11y ago

Vultr, who have offerings very similar to DO, have a class of VMs called "Storage Series"[1]. In my experience their uptime is similar to DO and their performance is better. Might be of interest to you: [1] https://www.vultr.com/pricing/ https://www.vultr.com/pricing/

vfuse 11y ago

Take a look at serverhub.com ssd cached vps, $5 per month for 500GB of diskspace.