4 ms·

Amazon SimpleDB Goes Beta

44 points by blader 19y ago

Scalable database as a service. My previous employer Powerset was an early tester, good to see it finally out in the world.

raghus 19y ago
http://www.amazon.com/gp/browse.html?node=342335011 http://www.amazon.com/gp/browse.html?node=342335011
cperciva 19y ago
This is cute, but doesn't really do anything which isn't already possible with EC2 and S3. The most critical limitation of EC2 and S3, at least where databases are concerned, is the "eventual consistency" model: If you store data, you'll eventually be able to get it back -- but there are no guarantees as to when. When I heard rumours that Amazon was going to be launching a database service, I hoped that it would solve this problem; instead, it seems that Amazon has released the world's first "database" which not only transactions, but even lacks consistency!
- rontr 19y ago
  "This is cute, but doesn't really do anything which isn't already possible with EC2 and S3." EC2 doesn't give you durability, and S3 doesn't give you speed. SimpleDB gives you both.
  - wmf 19y ago
    It sounds like SimpleDB runs on EC2, meaning it can't do anything that you couldn't build yourself on EC2. If you replicate data across enough EC2 nodes you can get durability (at the cost of complexity and high MTTR).
charlie 19y ago
http://www.satine.org/archives/2007/12/13/amazon-simpledb/ http://www.satine.org/archives/2007/12/13/amazon-simpledb/
- sriram 19y ago
  I wrote up an overview post at http://www.sriramkrishnan.com/blog/2007/12/amazon-simpledb-technical-overview.html http://www.sriramkrishnan.com/blog/2007/12/amazon-simpledb-t... The eventual consistency part is going to be interesting to watch.
mwerty 19y ago
The last line: > There's one significant cost that SimpleDB users can avoid: keeping a database administrator on the payroll. cringe cringe cringe.
- bayareaguy 19y ago
  No, all it may do is change the DBA skill set a little. If your data wasn't important enough for a DBA then you wouldn't have had a DBA anyways and SimpleDB does nothing to change that. Conversely, if your data matters enough to you then there will be a DBA somewhere who knows the data well enough to ensure it survives. Who are you going to call when your application corrupts your data? Amazon probably won't lose your data on purpose but they can't prevent mistakes. However, this could be an opportunity for an startup - outsourced SimpleDB data administration.
  - mwerty 19y ago
    Good points and all but the cringing was for the cliched way the article ended.
inovica 19y ago
This is great news. We have just started using EC2 and S3 for a startup I've invested in and one of our concerns has been using mysql as it needs to be used on an EC2 instance. We've been looking at ways to ensure our data is safe, but this SimpleDB could provide the answer we've been looking for. Hope I get into the beta! Out of interest, anyone else out there using mysql on EC2 and how have you dealt with ensuring your data is safe?
DarrenStuart 19y ago
mmmm costs $1226.4 just to run without datacost being accounted for. I love the idea but I feel the pricing should of been like s3 I.E just paying for storage and bandwidth. I can't see how you would get out a key from the system when you add a new record...
- sadiq 19y ago
  That $0.14/hour is the amount of processing you do, not the hourly cost to use the service. $1226.4 is if you're hammering the data-base with intense queries 100% of the time (so it's always being utilised). In reality (though this depends on how efficient their database is) it'll likely be orders of magnitude smaller.
  - rms 19y ago
    So $.14/hour is for a full unit of processing for an hour? What's the smallest amount of processing you can get?
    - sadiq 19y ago
      "SimpleDB costs 14 cents per machine hour, with no minimum fee." I guess we won't know the costs in reality until the service is up. It pretty much all depends on how efficient their implementation is.
      - DarrenStuart 19y ago
        so if you get one hit to the db per hour then you are going to end up paying for it big time.
        sadiq 19y ago
        How exactly? If a query takes 1ms, you're going to pay for one sixty-thousanth of $0.14 . I fail to see what you're doing? Or am I missing something?
jsjenkins168 19y ago
I like the focus on simplicity, but I'm not excited about writing a custom framework to handle the new proprietary query language. I was hopeful a service like this would accept standard SQL but this is not the case. Eventual Consistency will require a complete re-thinking of how traditional web apps persist data. This is a big problem. I'm just hopeful that it will eventually be possible to guarantee that written data is immediately accessible. Maybe by briefly buffering it in an external location until the SimpleDB data is consistent? Yet more custom framework code to write..
- mechanical_fish 19y ago
  Don't forget the part where your brand-new custom framework needs to properly serialize and deserialize each value based on its type. Because according to this person: http://www.satine.org/archives/2007/12/13/amazon-simpledb/ http://www.satine.org/archives/2007/12/13/amazon-simpledb/ ...Amazon's magical indexing technology is to make all searches lexicographical. So you have to "zero-pad your integers, add positive offsets to negative integer sets, and convert dates into something like ISO 8601". Or something. But at least you don't have to write one of those fiendishly difficult CREATE TABLE statements! Ah, "simplicity"!
sanj 19y ago
No joins! I guess you have to take denormalization to heart!
- chrisconley 19y ago
  really! somewhat disappointing but I suppose there's a reason for this
chaostheory 19y ago
I was interested until I read this: "Query execution time is currently limited to 5 seconds."
DocSavage 19y ago
SimpleDB appears to be Amazon's Dynamo opened up as a web service. There's good technical information, including a paper, over here: http://allthingsdistributed.com/2007/10/amazons_dynamo.html http://allthingsdistributed.com/2007/10/amazons_dynamo.html
- codeslinger 19y ago
  ARRGH! Did no one actually read that paper? Its 1MB limit on data size and concentration on HA for writes makes it unsuitable for this purpose, just as it is unsuitable for S3 (a previous rumor purported by the commenters on this site). RTFP, people!
  - DocSavage 19y ago
    max 256 attribute-values per item. Max 1024 bytes per attribute value.

44 points by blader 19y ago

Scalable database as a service. My previous employer Powerset was an early tester, good to see it finally out in the world.

raghus 19y ago

http://www.amazon.com/gp/browse.html?node=342335011 http://www.amazon.com/gp/browse.html?node=342335011

cperciva 19y ago

This is cute, but doesn't really do anything which isn't already possible with EC2 and S3. The most critical limitation of EC2 and S3, at least where databases are concerned, is the "eventual consistency" model: If you store data, you'll eventually be able to get it back -- but there are no guarantees as to when. When I heard rumours that Amazon was going to be launching a database service, I hoped that it would solve this problem; instead, it seems that Amazon has released the world's first "database" which not only transactions, but even lacks consistency!

rontr 19y ago
"This is cute, but doesn't really do anything which isn't already possible with EC2 and S3." EC2 doesn't give you durability, and S3 doesn't give you speed. SimpleDB gives you both.
- wmf 19y ago
  It sounds like SimpleDB runs on EC2, meaning it can't do anything that you couldn't build yourself on EC2. If you replicate data across enough EC2 nodes you can get durability (at the cost of complexity and high MTTR).

charlie 19y ago

http://www.satine.org/archives/2007/12/13/amazon-simpledb/ http://www.satine.org/archives/2007/12/13/amazon-simpledb/

sriram 19y ago
I wrote up an overview post at http://www.sriramkrishnan.com/blog/2007/12/amazon-simpledb-technical-overview.html http://www.sriramkrishnan.com/blog/2007/12/amazon-simpledb-t... The eventual consistency part is going to be interesting to watch.

mwerty 19y ago

The last line: > There's one significant cost that SimpleDB users can avoid: keeping a database administrator on the payroll. cringe cringe cringe.

bayareaguy 19y ago
No, all it may do is change the DBA skill set a little. If your data wasn't important enough for a DBA then you wouldn't have had a DBA anyways and SimpleDB does nothing to change that. Conversely, if your data matters enough to you then there will be a DBA somewhere who knows the data well enough to ensure it survives. Who are you going to call when your application corrupts your data? Amazon probably won't lose your data on purpose but they can't prevent mistakes. However, this could be an opportunity for an startup - outsourced SimpleDB data administration.
- mwerty 19y ago
  Good points and all but the cringing was for the cliched way the article ended.

inovica 19y ago

This is great news. We have just started using EC2 and S3 for a startup I've invested in and one of our concerns has been using mysql as it needs to be used on an EC2 instance. We've been looking at ways to ensure our data is safe, but this SimpleDB could provide the answer we've been looking for. Hope I get into the beta! Out of interest, anyone else out there using mysql on EC2 and how have you dealt with ensuring your data is safe?

DarrenStuart 19y ago

mmmm costs $1226.4 just to run without datacost being accounted for. I love the idea but I feel the pricing should of been like s3 I.E just paying for storage and bandwidth. I can't see how you would get out a key from the system when you add a new record...

sadiq 19y ago
That $0.14/hour is the amount of processing you do, not the hourly cost to use the service. $1226.4 is if you're hammering the data-base with intense queries 100% of the time (so it's always being utilised). In reality (though this depends on how efficient their database is) it'll likely be orders of magnitude smaller.
- rms 19y ago
  So $.14/hour is for a full unit of processing for an hour? What's the smallest amount of processing you can get?
  - sadiq 19y ago
    "SimpleDB costs 14 cents per machine hour, with no minimum fee." I guess we won't know the costs in reality until the service is up. It pretty much all depends on how efficient their implementation is.
    - DarrenStuart 19y ago
      so if you get one hit to the db per hour then you are going to end up paying for it big time.
      - sadiq 19y ago
        How exactly? If a query takes 1ms, you're going to pay for one sixty-thousanth of $0.14 . I fail to see what you're doing? Or am I missing something?

jsjenkins168 19y ago

I like the focus on simplicity, but I'm not excited about writing a custom framework to handle the new proprietary query language. I was hopeful a service like this would accept standard SQL but this is not the case. Eventual Consistency will require a complete re-thinking of how traditional web apps persist data. This is a big problem. I'm just hopeful that it will eventually be possible to guarantee that written data is immediately accessible. Maybe by briefly buffering it in an external location until the SimpleDB data is consistent? Yet more custom framework code to write..

mechanical_fish 19y ago
Don't forget the part where your brand-new custom framework needs to properly serialize and deserialize each value based on its type. Because according to this person: http://www.satine.org/archives/2007/12/13/amazon-simpledb/ http://www.satine.org/archives/2007/12/13/amazon-simpledb/ ...Amazon's magical indexing technology is to make all searches lexicographical. So you have to "zero-pad your integers, add positive offsets to negative integer sets, and convert dates into something like ISO 8601". Or something. But at least you don't have to write one of those fiendishly difficult CREATE TABLE statements! Ah, "simplicity"!

sanj 19y ago

No joins! I guess you have to take denormalization to heart!

chrisconley 19y ago
really! somewhat disappointing but I suppose there's a reason for this

chaostheory 19y ago

I was interested until I read this: "Query execution time is currently limited to 5 seconds."

DocSavage 19y ago

SimpleDB appears to be Amazon's Dynamo opened up as a web service. There's good technical information, including a paper, over here: http://allthingsdistributed.com/2007/10/amazons_dynamo.html http://allthingsdistributed.com/2007/10/amazons_dynamo.html

codeslinger 19y ago
ARRGH! Did no one actually read that paper? Its 1MB limit on data size and concentration on HA for writes makes it unsuitable for this purpose, just as it is unsuitable for S3 (a previous rumor purported by the commenters on this site). RTFP, people!
- DocSavage 19y ago
  max 256 attribute-values per item. Max 1024 bytes per attribute value.