4 ms·

I appreciate the context, but this just adds so many more questions. How are you integrating hardware clocks into the Postgres transaction model? Is this just p

I appreciate the context, but this just adds so many more questions. How are you integrating hardware clocks into the Postgres transaction model? Is this just postgres wire compatible like Cockroach or Yugabyte, or have you modified Postgres MVCC implementation to not use the standard TID. Does this support all of postgres at the Transaction coordinator ? Super exciting announcement, and I am really looking forwards to learning more!

jitl 3y ago
They’re probably using it similar to how Spanner uses TrueTime: https://cloud.google.com/spanner/docs/true-time-external-consistency https://cloud.google.com/spanner/docs/true-time-external-con...
- dkhenry 3y ago
  but Spanner isn't Postgres compatible, they have their own transaction processing layer that is built for TrueTime. If this is using the Postgres frontend how are they incorporating TrueTime into the normal Postgres MVCC model that uses the monotonic xid. edit: More details make it seem like this _isn't_ Postgres on the frontend, its just mostly postgres compatible, which would explain how they got away from the xid based MVCC. So it seems like this is an entirely new distributed database, rather then a modification to existing postgres.
  - eightnoteight 3y ago
    I think they are internally using postgres only. aurora uses a custom storage process on host which internally supports the custom storage engine (s3) given the mentions of "log as a database", I believe that depending on the transaction the storage responds differently. like how mysql mvcc uses undolog to essentially rollback database internally so that transaction sees data consistency. they could be doing something similar i.e regardless of whatever postgres uses, if they can get a reference of transaction and its start time then they can use the custom storage engine to rollback the log and respond in that way
  - eightnoteight 3y ago
    remember it is aws, "leverage" is built into their DNA, I don't believe that they would go on the route of building a totally new SQL database like google did
- alex_127 3y ago
  https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-time-sync-service-microsecond-accurate-time/ https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-ti...
joelrwilliams1 3y ago
Here's the deep dive talk from re:Invent 2023 https://youtu.be/a9FfjuVJ9d8 https://youtu.be/a9FfjuVJ9d8

jitl 3y ago

They’re probably using it similar to how Spanner uses TrueTime: https://cloud.google.com/spanner/docs/true-time-external-consistency https://cloud.google.com/spanner/docs/true-time-external-con...

dkhenry 3y ago
but Spanner isn't Postgres compatible, they have their own transaction processing layer that is built for TrueTime. If this is using the Postgres frontend how are they incorporating TrueTime into the normal Postgres MVCC model that uses the monotonic xid. edit: More details make it seem like this _isn't_ Postgres on the frontend, its just mostly postgres compatible, which would explain how they got away from the xid based MVCC. So it seems like this is an entirely new distributed database, rather then a modification to existing postgres.
- eightnoteight 3y ago
  I think they are internally using postgres only. aurora uses a custom storage process on host which internally supports the custom storage engine (s3) given the mentions of "log as a database", I believe that depending on the transaction the storage responds differently. like how mysql mvcc uses undolog to essentially rollback database internally so that transaction sees data consistency. they could be doing something similar i.e regardless of whatever postgres uses, if they can get a reference of transaction and its start time then they can use the custom storage engine to rollback the log and respond in that way
- eightnoteight 3y ago
  remember it is aws, "leverage" is built into their DNA, I don't believe that they would go on the route of building a totally new SQL database like google did
alex_127 3y ago
https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-time-sync-service-microsecond-accurate-time/ https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-ti...

joelrwilliams1 3y ago

Here's the deep dive talk from re:Invent 2023 https://youtu.be/a9FfjuVJ9d8 https://youtu.be/a9FfjuVJ9d8