That's a good point, but I've just skimmed through the fantastically detailed entries in the Oxford English Dictionary, which seem to suggest that the word "ora

That's a good point, but I've just skimmed through the fantastically detailed entries in the Oxford English Dictionary, which seem to suggest that the word "orange" was used in English around 1400 and was used as a colour in 1532, while the word "ginger" was used in English around 1200 (or earlier) and was used as a colour in 1538. Obviously "earliest citation" is not the same thing as "first use" (though it's amazing how many people seem not to understand that) but it doesn't look as though "ginger" as a colour is much older than "orange" as a colour. On the other hand, the early 16th century is perhaps the earliest point for which we have masses of surviving texts and can therefore get a good idea of which colour names were in use. For earlier periods perhaps there's a good chance that some fairly ordinary words are missing from the record just through bad luck.