5 ms·
The update isn't inserting anything. Either the row exists already, or it doesn't. Maybe I'm missing something?
by jessaustin 9y ago
The update isn't inserting anything. Either the row exists already, or it doesn't. Maybe I'm missing something?
- anarazel 9y ago> The update isn't inserting anything. Either the row exists already, or it doesn't. Maybe I'm missing something? Rows can exists without you being able to see them. Consider: S1: BEGIN; S2: BEGIN; S1: INSERT name = 'Alice' S2: UPDATE WHERE name = 'Alice' -> no row matches S2: INSERT name = 'Alice' WHERE NOT EXISTS (name = 'Alice') S2: <blocks, due to potential uniqueness violation> S1: COMMIT; S2: <resumes, raises uniqueness violation> Edit: formatting
- eclark 9y agoI haven't tested but I don't think that will happen on the default settings for postgres. I know for a fact that it won't on higher isolation levels. https://www.postgresql.org/docs/9.1/static/transaction-iso.html https://www.postgresql.org/docs/9.1/static/transaction-iso.h... > If the first updater commits, the second updater will ignore the row if the first updater deleted it, otherwise it will attempt to apply its operation to the updated version of the row. My understanding is that the update from S2 would be rerun applied. Specifically postgres calls out: > The search condition of the command (the WHERE clause) is re-evaluated to see if the updated version of the row still matches the search condition.
- anarazel 9y ago> I haven't tested but I don't think that will happen on the default settings for postgres. I know for a fact that it won't on higher isolation levels. Unfortunately you're wrong. I've now tested it, but I was also pretty confident before - I'm a postgres developer, and I worked on/comitted the PG upsert implementation ;) postgres[10287][1]=# CREATE TABLE data(key text unique); CREATE TABLE postgres[10287][1]=# BEGIN ISOLATION LEVEL SERIALIZABLE ; BEGIN postgres[10284][1]=# BEGIN ISOLATION LEVEL SERIALIZABLE ; BEGIN postgres[10287][1]*=# INSERT INTO data VALUES('alice'); INSERT 0 1 postgres[10284][1]*=# INSERT INTO data SELECT 'alice' WHERE NOT EXISTS(SELECT * FROM data WHERE key = 'alice'); postgres[10287][1]*=# COMMIT; COMMIT postgres[10284][1]*=# ERROR: 23505: duplicate key value violates unique constraint "data_key_key" DETAIL: Key (key)=(alice) already exists. SCHEMA NAME: public TABLE NAME: data CONSTRAINT NAME: data_key_key LOCATION: _bt_check_unique, nbtinsert.c:535 (the number in brackets in the prompt is the backend pid, allowing to differentiate the two sessions). > > If the first updater commits, the second updater will ignore the row if the first updater deleted it, otherwise it will attempt to apply its operation to the updated version of the row. > My understanding is that the update from S2 would be rerun applied. > Specifically postgres calls out: > > The search condition of the command (the WHERE clause) is re-evaluated to see if the updated version of the row still matches the search condition. Those comments are about row-level locks - they're not the problem here. What you get is a constraint violation due to the unique constraint.
- chacham15 9y agoI know very little about Postgres, but doesnt this appear to violate the isolation property of ACID?
- anarazel 9y agoNo. ACID doesn't guarantee that you can't have constraint violations triggered by a concurrent session. That'd make any sort of efficient constraint pretty much impossible. It guarantees that that error is caught despite the concurrency however.
- chacham15 9y agoAm I missing something? Wikipedia[1] says: "The isolation property ensures that the concurrent execution of transactions results in a system state that would be obtained if transactions were executed sequentially." Oh, could the exception here be caused by the raw insert (without the where clause)? If you retried your example where both inserts were: INSERT INTO data SELECT 'alice' WHERE NOT EXISTS(SELECT * FROM data WHERE key = 'alice'); would that still cause the constraint error? If so, isnt that not equivalent to running the two queries sequentially hence violating the isolation property? Have I misunderstood something? [1]https://en.wikipedia.org/wiki/ACID https://en.wikipedia.org/wiki/ACID
- deleted 9y ago[deleted]
- imtringued 9y agoThink of it like git branching. In the master branch there is no file "Alice". Two branches get created. One of them runs sed via find on all files with the name "Alice" and then it creates a file called "Alice" if find doesn't find any files called Alice. The second branch creates a file called "Alice". Both want to merge now but unfortunately there is a merge conflict. In this case automatic conflict resolution is impossible so one of the branches has to be discarded (transaction rollback). You can now "rerun" the branch and everything should work on the next merge. Conclusion: "update + insert where" is not equivalent to "upsert". The former can cause a transaction to fail and must be retried on failure. With "upsert" the database engine has enough information to handle the unique constraint.
- tscs37 9y agoUsually for the serializable isolation level you need to manually rerun the transaction. There are some DBs that do that for you like CockroachDB.
- aseipp 9y agoI believe the issue is a concurrent transaction (i.e. a separate one from the one you're currently in) may perform the INSERT and COMMIT that insert after your transaction's SELECT subquery returns nothing (thus allowing INSERT to proceed, by the WHERE clause), but before the actual INSERT itself is committed. This means there is a window between the sub-SELECT and the INSERT itself where a secondary transaction can insert something, causing both your original update to do nothing, and the insert to fail. Thus you've lost the write entirely.
- jessaustin 9y agoIn that case the other effort (I'm reluctant to call it a transaction) wins. Maybe it was later and should win, maybe it was earlier and shouldn't. It takes a lot more care than seen here to preserve a meaning of "earlier" and "later" that isn't vulnerable to this issue. The way the schema is designed, there can be only one "Alice". I'm trying and failing to imagine a way in which both this effort and the other effort can fail.
- aseipp 9y agoI think the particular point made by Andres (and elsewhere in this thread) is really not about what the schema for a phone book is like, and more that "Accurately replicating what UPSERT does without it is exceedingly hard and you will probably get it wrong or suboptimal in a number of scenarios you can't possibly imagine until they happen". Isolation levels will for example have impact on how transactions proceed and what results you see (REPEATABLE READ vs READ COMMITTED); the error handling (did your transaction actually fail from a concurrent INSERT succeeding, or did the update fail, or did something else happen entirely? UPSERT is often not a standalone operation in a transaction) must be carefully decided for each use-case or else you're likely to redo needless transactions or throw away good ones, and you have to work around incidental problems like latency using stored procedures to keep everything server-side, etc etc. It's just not the kind of thing you want to replicate a number of places at each use site in a number of tools. It's something only the database knows how to manage properly by having "global knowledge".