4 ms·

The main lock-in issue to me is the datastore. If you are worried about lock-in, you could try to use vanilla Django as much as possible. I think it's likely

The main lock-in issue to me is the datastore. If you are worried about lock-in, you could try to use vanilla Django as much as possible. I think it's likely that an opensource datastore equivalent will appear, or at least something that's close enough like HBase or Hypertable.

jm4 18y ago
Yeah, that's what I was getting at. As soon as you need any persistent storage you're chained to BigTable. That leaves a lot of code that needs to be rewritten if you need to move to a new storage model. This isn't unlike the storage problems PHP had quite a few years ago. There was a separate module for every database that was supported and they lacked consistent naming conventions as well as consistent functionality. It was literally one set of functions you used to talk to MySQL and a completely different set of functions for PostgreSQL or SQL Server. This is still reflected in the documentation: http://www.php.net/manual/en/refs.database.php http://www.php.net/manual/en/refs.database.php For a long time there wasn't a common wrapper for all of these modules so if you started with MySQL and needed to move to Oracle or something you had to rewrite a bunch of code. I had to do that once and I never used PHP again. It seems that's where we're at with App Engine and BigTable. I'm sure there's going to eventually be something equivalent or some sort of abstraction, but I wouldn't want to be holding my breath.
- DocSavage 18y ago
  Isn't there already a decent abstraction, namely the Datastore API (http://code.google.com/appengine/docs/datastore/ http://code.google.com/appengine/docs/datastore/)? Current apps are written against that API and it's not a stretch to see an opensource BigTable on EC2 with a Datastore API. (See appdrop.com for an even simpler datastore replacement.)
  - jm4 18y ago
    Yes, that's BigTable specific. My point was that right now you have no option but to rewrite code if you need to move off App Engine and BigTable. How is it not a stretch to see an open source BigTable on EC2? As far as I can tell BigTable has, up to this point, given Google a competitive edge. They are merely letting us come along for a ride and Amazon is a direct competitor in this market. Why in the world would they allow it to run on EC2?
    - DocSavage 18y ago
      If you are coding to the Datastore API, you won't have to rewrite that code when moving to another provider who offers the Datastore API. AppDrop.com, for example, was a proof-of-concept that let you run apps on EC2 unchanged. He simply replaced the implementation below the datastore stubs. Google may not open-source BigTable code, but as I mentioned above, there are a number of open-source projects that provide similar features. Any of these can run on EC2 together with the open-source AppEngine SDK.

jm4 18y ago

Yeah, that's what I was getting at. As soon as you need any persistent storage you're chained to BigTable. That leaves a lot of code that needs to be rewritten if you need to move to a new storage model. This isn't unlike the storage problems PHP had quite a few years ago. There was a separate module for every database that was supported and they lacked consistent naming conventions as well as consistent functionality. It was literally one set of functions you used to talk to MySQL and a completely different set of functions for PostgreSQL or SQL Server. This is still reflected in the documentation: http://www.php.net/manual/en/refs.database.php http://www.php.net/manual/en/refs.database.php For a long time there wasn't a common wrapper for all of these modules so if you started with MySQL and needed to move to Oracle or something you had to rewrite a bunch of code. I had to do that once and I never used PHP again. It seems that's where we're at with App Engine and BigTable. I'm sure there's going to eventually be something equivalent or some sort of abstraction, but I wouldn't want to be holding my breath.

DocSavage 18y ago
Isn't there already a decent abstraction, namely the Datastore API (http://code.google.com/appengine/docs/datastore/ http://code.google.com/appengine/docs/datastore/)? Current apps are written against that API and it's not a stretch to see an opensource BigTable on EC2 with a Datastore API. (See appdrop.com for an even simpler datastore replacement.)
- jm4 18y ago
  Yes, that's BigTable specific. My point was that right now you have no option but to rewrite code if you need to move off App Engine and BigTable. How is it not a stretch to see an open source BigTable on EC2? As far as I can tell BigTable has, up to this point, given Google a competitive edge. They are merely letting us come along for a ride and Amazon is a direct competitor in this market. Why in the world would they allow it to run on EC2?
  - DocSavage 18y ago
    If you are coding to the Datastore API, you won't have to rewrite that code when moving to another provider who offers the Datastore API. AppDrop.com, for example, was a proof-of-concept that let you run apps on EC2 unchanged. He simply replaced the implementation below the datastore stubs. Google may not open-source BigTable code, but as I mentioned above, there are a number of open-source projects that provide similar features. Any of these can run on EC2 together with the open-source AppEngine SDK.