6 ms·

It would be nice if Amazon provided an API to access the data via SQL alongside the MongoDB API; I've seen quite a number of organizations migrate from mongo to

by erikig 8y ago

It would be nice if Amazon provided an API to access the data via SQL alongside the MongoDB API; I've seen quite a number of organizations migrate from mongo to Postgres once they get out of the rapid development phase. This would make that transition butter smooth.

ahachete 8y ago
That would make the internal representation used an "API" and thus won't be able to change it in the future. Apparently, they are using a 1:1 mapping between a collection and a table. Either by flattening the document or by using jsonb or equivalent. I'm not a big believer this is good for performance reasons, at least compared to a more normalized approach like the one we did for https://www.torodb.com https://www.torodb.com But they may change it in the future --if they don't expose the SQL API to their internal representation.
scarface74 8y ago
Anecdote: I led a C# project where we could seamlessly switch back and forth between Mongo and SQL Server without changing the underlying LINQ expressions. We sent the expressions to the Mongo driver and they got translated to MongoQuery we sent the expressions to Entity Framework and they got translated to Sql Server.
- deleted 8y ago
  [deleted]
- manigandham 8y ago
  C# is ahead of the game with LINQ, expression syntax, and the entire Rosyln platform. Passing an IQueryable<> around that can be interpreted and transformed for multiple backends is a incredibly productive. I wish more people knew about this, and .NET in general.
  - scarface74 8y ago
    And I’ve seen a few Java and Javascript libraries that purport to “implement LINQ” and they don’t get the power of LINQ is not the syntax, it’s that LINQ translates your code to expressions that can be parsed and translated to any back end query - it’s not just an ORM. I’ve seen a LINQ to REST API provider.
talawahdotnet 8y ago
I doubt that they actually built this on top of Postgres. They probably just integrated the WiredTiger[1] storage engine used by Mongo with their Aurora storage subsystem. I am however really hoping Amazon provides a MySQL 8.0 compatible version of Aurora with full support for its new hybrid SQL and Document Store interfaces[2] courtesy of the X DevAPI[3] and lightweight "serverless" friendly connections courtesy of the new X Protocol. That way your don't have to choose just one approach, and you can have your data in one place with high reliability and durability. My ultimate pipe dream would be that they also provided a redis compatible key/value interface that allows you to fetch simple values directly from the underlying innodb storage engine without going thru the SQL layer, similar to how the memcached plugin currently works[4] 1. https://github.com/wiredtiger/wiredtiger https://github.com/wiredtiger/wiredtiger 2. https://mysqlserverteam.com/mysql-8-0-announcing-ga-of-the-mysql-document-store/ https://mysqlserverteam.com/mysql-8-0-announcing-ga-of-the-m... 3. https://dev.mysql.com/doc/x-devapi-userguide/en/devapi-users-introduction.html https://dev.mysql.com/doc/x-devapi-userguide/en/devapi-users... 4. https://dev.mysql.com/doc/refman/8.0/en/innodb-memcached.html https://dev.mysql.com/doc/refman/8.0/en/innodb-memcached.htm...
- mrep 8y ago
  > I doubt that they actually built this on top of Postgres. Maybe not (but OP makes a lot of good points for why it is), but it is still based on the aurora limits, 64TB of size, 15 low latency read replicas in minutes, and presumably 1 write capacity which makes it a laughable nosql system since it cannot scale past 1 servers write capacity.
  - sorenbs 8y ago
    Are you aware that they are working on multi-master for Aurora? https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...
- UlfWendel 8y ago
  What's the motivation for a faster access path to InnoDB: performance? X DevAPI and X Protocol/X Plugin could team up and map K/V style access to the server internal InnoDB API instead of using a SQL service as it is currently done. They could try to do it "transparently" or let you set hints. Whatever is desired from an application standpoint.
StreamBright 8y ago
And there are organizations who can do rapid development in Postgres.

by erikig 8y ago

ahachete 8y ago

That would make the internal representation used an "API" and thus won't be able to change it in the future. Apparently, they are using a 1:1 mapping between a collection and a table. Either by flattening the document or by using jsonb or equivalent. I'm not a big believer this is good for performance reasons, at least compared to a more normalized approach like the one we did for https://www.torodb.com https://www.torodb.com But they may change it in the future --if they don't expose the SQL API to their internal representation.

scarface74 8y ago

Anecdote: I led a C# project where we could seamlessly switch back and forth between Mongo and SQL Server without changing the underlying LINQ expressions. We sent the expressions to the Mongo driver and they got translated to MongoQuery we sent the expressions to Entity Framework and they got translated to Sql Server.

deleted 8y ago
[deleted]
manigandham 8y ago
C# is ahead of the game with LINQ, expression syntax, and the entire Rosyln platform. Passing an IQueryable<> around that can be interpreted and transformed for multiple backends is a incredibly productive. I wish more people knew about this, and .NET in general.
- scarface74 8y ago
  And I’ve seen a few Java and Javascript libraries that purport to “implement LINQ” and they don’t get the power of LINQ is not the syntax, it’s that LINQ translates your code to expressions that can be parsed and translated to any back end query - it’s not just an ORM. I’ve seen a LINQ to REST API provider.

talawahdotnet 8y ago

I doubt that they actually built this on top of Postgres. They probably just integrated the WiredTiger[1] storage engine used by Mongo with their Aurora storage subsystem. I am however really hoping Amazon provides a MySQL 8.0 compatible version of Aurora with full support for its new hybrid SQL and Document Store interfaces[2] courtesy of the X DevAPI[3] and lightweight "serverless" friendly connections courtesy of the new X Protocol. That way your don't have to choose just one approach, and you can have your data in one place with high reliability and durability. My ultimate pipe dream would be that they also provided a redis compatible key/value interface that allows you to fetch simple values directly from the underlying innodb storage engine without going thru the SQL layer, similar to how the memcached plugin currently works[4] 1. https://github.com/wiredtiger/wiredtiger https://github.com/wiredtiger/wiredtiger 2. https://mysqlserverteam.com/mysql-8-0-announcing-ga-of-the-mysql-document-store/ https://mysqlserverteam.com/mysql-8-0-announcing-ga-of-the-m... 3. https://dev.mysql.com/doc/x-devapi-userguide/en/devapi-users-introduction.html https://dev.mysql.com/doc/x-devapi-userguide/en/devapi-users... 4. https://dev.mysql.com/doc/refman/8.0/en/innodb-memcached.html https://dev.mysql.com/doc/refman/8.0/en/innodb-memcached.htm...

mrep 8y ago
> I doubt that they actually built this on top of Postgres. Maybe not (but OP makes a lot of good points for why it is), but it is still based on the aurora limits, 64TB of size, 15 low latency read replicas in minutes, and presumably 1 write capacity which makes it a laughable nosql system since it cannot scale past 1 servers write capacity.
- sorenbs 8y ago
  Are you aware that they are working on multi-master for Aurora? https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...
UlfWendel 8y ago
What's the motivation for a faster access path to InnoDB: performance? X DevAPI and X Protocol/X Plugin could team up and map K/V style access to the server internal InnoDB API instead of using a SQL service as it is currently done. They could try to do it "transparently" or let you set hints. Whatever is desired from an application standpoint.

StreamBright 8y ago

And there are organizations who can do rapid development in Postgres.