4 ms·

GAE has become completely infeasible as a hosting solution for me (ThatHigh.com). My hosting cost increased by 90x, and I did not get nearly enough notice. I

GAE has become completely infeasible as a hosting solution for me (ThatHigh.com). My hosting cost increased by 90x, and I did not get nearly enough notice. I don't have the time or resources to move the site, so I'm forced to shut it down. It really, really sucks. Personally I'm more disappointed by the lack of notice (1 month is nowhere near enough time) than the actual increase. I totally understand the need to charge.

Travis 15y ago
Hey I shot you a tweet, I'd be interested in helping migrate the site. Email is in my HN profile, shoot me a message if you're interested.
StavrosK 15y ago
Did you turn multithreading on? It's helped on my sites somewhat.
yesbabyyes 15y ago
As a fan I urge you not to shut it down. Please consider selling it or something!
yahelc 15y ago
If you implement optimizations, you can really significantly curb the cost. I spent some time tuning SharedCount's API, which would have cost me $30-$50/day, and its now at about $1-$2/day. - Move to Python 2.7 and enable multithreading - Setup Cloudflare (this swallows about half of all my requests) - Increase minimum latency and reduce the maximum number of idle instances. (I have 5-8s and 1-2 set, respectively) - Setup the semi-undocumented Google edge cache (basically, just a Cache-Control: public, max-age=[seconds] header. - Take advantage of memcache. With this setup, I'm doing 3 million API calls per day at $2.
- cr4zy 15y ago
  Python 2.7 is experimental and requires migrating to the new datastore (high replication) which is a lot more expensive. So you might not want to do this if your costs are coming from datastore writes. Also, high replication queries can return stale results unless you use ancestor queries. Ancestor queries require putting entities in groups by giving them all the same parent (which can never be changed). Basically it's a very inflexible semaphore and kind of sucks IMO. Your suggestions in general are very good though. Thanks, I'm switching my DNS to CloudFlare now.
  - stickfigure 15y ago
    The HRD is the same price as the Master/Save datastore. In the old pricing regime it was more expensive, but now they are at parity. It's true that eventual consistency of queries on the HRD can be tricky to program around. On the other hand... your data is replicated to 3+ datacenters and failed over in realtime. Pretty rad.
  - yahelc 15y ago
    Ah, yes. My unusual advantage here is that SharedCount doesn't use the datastore, which I now realize is likely not the case with ThatHigh. (In fact, when I had to migrate my app from MS to HRD, it kept failing because I didn't have any datastore entities. The workaround was to just create a single entity)
- detst 15y ago
  Do you see much benefit to CloudFlare swallowing half of your requests when you're already using the GAE edge cache?
  - yahelc 15y ago
    As far as I can tell, the edge cache doesn't work predictably; it relies solely on standard caching at the Google infrastructure lebel. The Cloudflare caching appears to be more centralized and covers a higher percentage of hits. But, I haven't looked into the effect of relying solely on the edge cache and getting rid of Cloudflare. Can't think of any reason to try, though, since Cloudflare is free.
- adrianwaj 15y ago
  I checked out SharedCount, and was wondering why the front-end uses PHP.
  - yahelc 15y ago
    Inertia; I initially built the API in PHP, but switched it to AppEngine when ProgrammableWeb listed it and people actually started using it. The front end uses the API though, and doesn't get enough traffic to justify worrying about scale.
roel_v 15y ago
Not to be an asshole, I'm genuinely curious: what about that site makes it impossible to run on a 10$/month vps with plenty of capacity to spare for running mail and 5 other sites of the same complexity? From the dates on the posts it doesn't seem like you get much traffic.
- endlessvoid94 15y ago
  Nothing is impossible about it. It just takes time. It also means downtime, which I suppose is unavoidable at this point. Honestly it has more to do with how much time I want to spend on the site, how much it returns, and whether or not I should spend my nights and weekends transferring it to another host. The last time I backed up all the data from GAE it took 4 days to download all of it to a VPS. 4 days to download all of it. Migrating means 4 days of downtime, or alternatively some complex solution involving posting all new data to BOTH places while the migration takes place. That takes time and energy, and quite frankly I'd rather see someone with the resources do it right instead of trying to hack it.
  - matc 15y ago
    If you are up for SQL, the downtime issue is solvable with ChronicDB pain-free.

Travis 15y ago

Hey I shot you a tweet, I'd be interested in helping migrate the site. Email is in my HN profile, shoot me a message if you're interested.

StavrosK 15y ago

Did you turn multithreading on? It's helped on my sites somewhat.

yesbabyyes 15y ago

As a fan I urge you not to shut it down. Please consider selling it or something!

yahelc 15y ago

If you implement optimizations, you can really significantly curb the cost. I spent some time tuning SharedCount's API, which would have cost me $30-$50/day, and its now at about $1-$2/day. - Move to Python 2.7 and enable multithreading - Setup Cloudflare (this swallows about half of all my requests) - Increase minimum latency and reduce the maximum number of idle instances. (I have 5-8s and 1-2 set, respectively) - Setup the semi-undocumented Google edge cache (basically, just a Cache-Control: public, max-age=[seconds] header. - Take advantage of memcache. With this setup, I'm doing 3 million API calls per day at $2.

cr4zy 15y ago
Python 2.7 is experimental and requires migrating to the new datastore (high replication) which is a lot more expensive. So you might not want to do this if your costs are coming from datastore writes. Also, high replication queries can return stale results unless you use ancestor queries. Ancestor queries require putting entities in groups by giving them all the same parent (which can never be changed). Basically it's a very inflexible semaphore and kind of sucks IMO. Your suggestions in general are very good though. Thanks, I'm switching my DNS to CloudFlare now.
- stickfigure 15y ago
  The HRD is the same price as the Master/Save datastore. In the old pricing regime it was more expensive, but now they are at parity. It's true that eventual consistency of queries on the HRD can be tricky to program around. On the other hand... your data is replicated to 3+ datacenters and failed over in realtime. Pretty rad.
- yahelc 15y ago
  Ah, yes. My unusual advantage here is that SharedCount doesn't use the datastore, which I now realize is likely not the case with ThatHigh. (In fact, when I had to migrate my app from MS to HRD, it kept failing because I didn't have any datastore entities. The workaround was to just create a single entity)
detst 15y ago
Do you see much benefit to CloudFlare swallowing half of your requests when you're already using the GAE edge cache?
- yahelc 15y ago
  As far as I can tell, the edge cache doesn't work predictably; it relies solely on standard caching at the Google infrastructure lebel. The Cloudflare caching appears to be more centralized and covers a higher percentage of hits. But, I haven't looked into the effect of relying solely on the edge cache and getting rid of Cloudflare. Can't think of any reason to try, though, since Cloudflare is free.
adrianwaj 15y ago
I checked out SharedCount, and was wondering why the front-end uses PHP.
- yahelc 15y ago
  Inertia; I initially built the API in PHP, but switched it to AppEngine when ProgrammableWeb listed it and people actually started using it. The front end uses the API though, and doesn't get enough traffic to justify worrying about scale.

roel_v 15y ago

Not to be an asshole, I'm genuinely curious: what about that site makes it impossible to run on a 10$/month vps with plenty of capacity to spare for running mail and 5 other sites of the same complexity? From the dates on the posts it doesn't seem like you get much traffic.

endlessvoid94 15y ago
Nothing is impossible about it. It just takes time. It also means downtime, which I suppose is unavoidable at this point. Honestly it has more to do with how much time I want to spend on the site, how much it returns, and whether or not I should spend my nights and weekends transferring it to another host. The last time I backed up all the data from GAE it took 4 days to download all of it to a VPS. 4 days to download all of it. Migrating means 4 days of downtime, or alternatively some complex solution involving posting all new data to BOTH places while the migration takes place. That takes time and energy, and quite frankly I'd rather see someone with the resources do it right instead of trying to hack it.
- matc 15y ago
  If you are up for SQL, the downtime issue is solvable with ChronicDB pain-free.