6 ms·
Upsert Lands in PostgreSQL 9.5 – A First Look
- postila 11y agoYep, this is awesome. Go Postgres!
- sksixk 11y agogood news: upsert works, bad news: select doesn't work
- Tenhundfeld 11y agoIMHO, this qualifies as trolling without further explanation.
- bsg75 11y agoI will bite: SELECT does not work on what? As compared to?
- Keats 11y agoGetting Sorry, I cannot find /2015/05/08/upsert-lands-in-postgres-9.5/
- robbles 11y agoI've read several times now that the tradeoff for upsert taking so long was that the implementation is "safe". Is MySQL's / other DB's version of this "unsafe" somehow? Or was there a particular architecture issue specific to Postgres that made this so difficult to implement without bugs?
- spacemanmatt 11y agoNot especially more difficult than other architectures, but their quality bar is typically much higher. YMMV.
- luckycharms810 11y agoI think one of the quirks you can run in to with MySQL's implementation is that even the DB decides to do an update, you end up burning an auto incremented key. Curious to see what PG ends up doing here.
- pilif 11y ago> you end up burning an auto incremented key. Curious to see what PG ends up doing here I haven't tried, but Postgres always uses values in sequences in similar cases. The good news is that there are so many values in a 64 bit sequence that this really shouldn't matter. I much prefer some values in sequences to be not used over even the tiniest possibility of a value being used twice accidentally.
- anarazel 11y agoYou pretty much have to do so. The conflict could be on the column with the default value after all. And obviously you can't just rollback the sequence/autoincrement value after deciding to update because that'd either require locking the sequence for the duration (horrible for concurrency) or would pose problems with other sessions already having used further values.
- zimpenfish 11y agoIn the previous discussion, someone mentioned MSSQL (and T-SQL)'s issues with MERGE: http://www.mssqltips.com/sqlservertip/3074/use-caution-with-sql-servers-merge-statement/ http://www.mssqltips.com/sqlservertip/3074/use-caution-with-... And also this: http://www.depesz.com/2012/06/10/why-is-upsert-so-complicated/ http://www.depesz.com/2012/06/10/why-is-upsert-so-complicate...
- jandrewrogers 11y agoThe difficulty is supporting high concurrency AND correctness AND high performance with this feature. Most databases that have supported this in the past have sacrificed one of these; PostgreSQL requires an implementation with no impact to existing users and without weakening the guarantees people are used to. In most good databases you can compose an upsert transactionally from a set of independent operations. The problem is that this is slow because you are plumbing the database engine multiple times for an operation that could be executed in one shot in principle. A native upsert implementation provides a fast, single operation. In the case of PostgreSQL though, they had to support this in the context of high-performance, high-concurrency, and correctness which makes it tricky because it is relatively complex to implement under those constraints.
- ryanjshaw 11y agoThis implementation of upsert is especially nice because it is atomic [1]: <literal>ON CONFLICT DO UPDATE</literal> guarantees an atomic <command>INSERT</command> or <command>UPDATE</command> outcome - provided there is no independent error, one of those two outcomes is guaranteed, even under high concurrency. I don't know about Oracle or MySQL, but the SQL Server implementation of MERGE requires you to carefully consider which locks to use and it's very easy to shoot yourself in the foot with. [1] http://git.postgresql.org/gitweb/?p=postgresql.git;a=blobdiff;f=doc/src/sgml/ref/insert.sgml;h=c88d1b7b50a30dacf4c1809e73e84eacf2ca3611;hp=a3cccb9f7c79a5bac71ed35a96d171e9b9587041;hb=168d5805e4c08bed7b95d351bf097cff7c07dd65;hpb=2c8f4836db058d0715bc30a30655d646287ba509 http://git.postgresql.org/gitweb/?p=postgresql.git;a=blobdif...
- pilif 11y agoNow if only there was some syntax shorthand for "just take everything from this now conflicting INSERT and discard what's already there". Like SQLite's "INSERT OR REPLACE". Aside of the transactional benefits that upsert has, it's also convenient in that you don't have to update and then insert, saving a lot of typing in cases where you're not relying on an ORM and still have to work with bulk data where you really don't care about local changes (because there were none). The syntax for this is a bit inconvenient right now.
- mbreese 11y agoThe syntax looks inconvenient, but I like it. If you have a conflict, you might not want to just update everything, but only a subset of the fields. With this syntax (verbose as it is), you have the option of explicitly stating what you want to happen in the event of a conflict with no ambiguity. Perhaps there is a place for some shorthand like ON CONFLICT SET *=*. But aside from that, for a RDBMS like Postgres, I'd opt for explicit over implicit.
- tracker1 11y agoOne would think you would just need a list of fields... ON CONFLICT SET colA,colB,colC As to the gp, you don't want to update the "CREATED" field on an upsert.
- deleted 11y ago[deleted]
- jeffdavis 11y ago"DO UPDATE SET description=description;" I think that should be: "DO UPDATE SET description=excluded.description;" And doesn't it also need an inference clause or constraint specification for DO UPDATE? Please update example code to be self-contained, so that readers can copy/paste. The example itself -- upserting product descriptions from another data source -- is a great one though.
- craigkerstiens 11y agoHi, Jeff Will update as soon as I'm at a machine, but yes of course you're correct.
- ccleve 11y agoIs there a reason for yet another syntax for upserts? There's an ANSI SQL syntax: http://en.wikipedia.org/wiki/Merge_%28SQL%29 http://en.wikipedia.org/wiki/Merge_%28SQL%29 And MySQL has its own syntax: http://dev.mysql.com/doc/refman/5.6/en/insert-on-duplicate.html http://dev.mysql.com/doc/refman/5.6/en/insert-on-duplicate.h... This is a pain for those of us who are trying to maintain cross-database libraries. (https://github.com/dieselpoint/norm https://github.com/dieselpoint/norm)