3 ms·
An aside: On https://scotrail.datasette.io/scotrail/random_apology https://scotrail.datasette.io/scotrail/random_apology I see heavy use of: order by rando
by brasic 4y ago
An aside: On https://scotrail.datasette.io/scotrail/random_apology https://scotrail.datasette.io/scotrail/random_apology I see heavy use of:
order by random() limit 1
I understand that in more or less all databases this is quite inefficient: it runs `random()` for all rows, sorts those random values and then chooses the row corresponding to the first one. In tables with millions of values this could be rather expensive.
Is there any generic way in SQL to cheaply choose an arbitrary row from a collection in a way that is not deterministic? [1] I’m thinking about something that has the same characteristics as Ruby’s `Array#sample`.
Postgres has TABLESAMPLE but that only works across an entire table so it’s not flexible enough to be useful in real world situations (ie where the collection to choose a random element from is already filtered by WHERE)
Or are there any databases that know how to recognize this `order by random()` idiom and skip the extra work? Kind of like how `count(*)` usually has optimized semantics to not actually look at the distinct values of each row.
[1] I realize that SELECT without an order clause is technically arbitrary but in practice it’s usually pretty deterministic.
- simonw 4y agoIn this case my database is 500KB in size so I'm not thinking about query performance at all. I've run into horrific performance problems with order by random() in larger databases. I actually suggested the SRANDMEMER feature that was added to Redis as a result of one of those projects! Details here: https://simonwillison.net/2009/Dec/20/crowdsourcing/ https://simonwillison.net/2009/Dec/20/crowdsourcing/