4 ms·
Isn't creating a sequence a bad idea in general, anyway? Aren't there a zillion ways to compromise things if you know that some field is a sequence?
by bsder 5y ago
Isn't creating a sequence a bad idea in general, anyway?
Aren't there a zillion ways to compromise things if you know that some field is a sequence?
- combatentropy 5y agoDo you mean enumeration, whereby an attacker starts at some ID and tries several in sequence? It has never been a problem for me and my applications. Just because you know a record exists, doesn't mean you can see it. For example, if you are authorized to view https://www.example.com/records/100 https://www.example.com/records/100, and you decide to try https://www.example.com/records/101 https://www.example.com/records/101, then the code will check to see if you're authorized to see record 101. If not, then you will get Unauthorized. I suppose there are situations where it's a problem if someone finds out that record 101 even exists, but not in any of my apps.
- bsder 5y ago> I suppose there are situations where it's a problem if someone finds out that record 101 even exists, but not in any of my apps. Well, it's things like say, an invoice number. I can buy something from you. And then 7 days later I buy something else from you. If the invoice numbers are in sequence, I just gained quite a bit of information about how fast you are selling things. That's the kind of information leak that sequences can create.
- wokkel 5y agoNice idea, but please check your local tax office. At least here in the Netherlands, you are required to use a monotically increasing series for invoice numbers. So I wouldn't do this if I were you.
- silon42 5y agoYou can't stop anyone from doing what he describes if you allow customers to see the invoice number. IMO, your sequential sequence number should be for tax audit purposes only.
- miken123 5y agoThat may be your opinion, but an invoice in at least the Netherlands needs an monotonically increasing number. And it needs to be on the invoice and your customer needs to be able to see it. [edit] It's even EU-wide, see article 226(2) of directive 2006/112/EC: > a sequential number, based on one or more series, which uniquely identifies the invoice;
- d_k_f 5y agoNothing stops you from using the "...or more series" part to generate numbers that are specific to the day, the hour or, if you feel like it, the minute. Today's first invoice could be 2021-07-16-001, the second one 2021-07-16-002, etc. If you really don't want people to be able to guess your invoice volume from numbers alone, there are various ways to do that while still being compliant to EU laws.
- zimpenfish 5y agoI think e.g. 2021-07-16-123 followed by 2021-07-17-001 wouldn't fall under "sequential number" though because, well, they're not sequential. The Italian authorities seem to see this the same way - https://vatdesk.eu/en/eu-vat-news/italy-mandatory-mentions-on-invoice/ https://vatdesk.eu/en/eu-vat-news/italy-mandatory-mentions-o... (Annoyingly the directive doesn't give a definition for "a sequential number" itself.)
- d_k_f 5y agoThey are sequential for all intents and purposes, they are simply using two separate series within the same number – you are perfectly able to determine the order of the invoices. I do agree that (as is so often the case) the directive is not specific enough to determine whether the Italian interpretation is correct or not, though I can provide some context from the German side as provided by the ministry of finance: "Eine lückenlose Abfolge der ausgestellten Rechnungsnummern ist nicht zwingend"[1] (~ "it is not required for the invoice numbers to be gapless"), which directly contradicts the Italians as far as I understand it. [1] https://www.bundesfinanzministerium.de/Content/DE/Downloads/BMF_Schreiben/Steuerarten/Umsatzsteuer/Umsatzsteuer-Anwendungserlass/Umsatzsteuer-Anwendungserlass-aktuell-Stand-2021-06-28.pdf?__blob=publicationFile&v=47 https://www.bundesfinanzministerium.de/Content/DE/Downloads/... - p. 522, 14.5 (10) 4
- mcraiha 5y agoMost places do not care about this. e.g. our local Subway restaurants print purchase number of the day to every receipt that customer gets. So you can see how many sales certain Subway restaurant has done in a single day by going to that restaurant before it closes and buying something.
- sonotathrowaway 5y agoIf you can find a way to take a representative sample of their stores, you can front-run their quarterly earnings report and make a guaranteed profit off them.
- joshribakoff 5y agoEven if you could approximate revenue, that hardly equates to profitability. Even if you could approximate profits, the market still behaves irrationally (in the short term), there are no guarantees
- jsight 5y agoThat is absolutely true, but an individual store is unlikely to care.
- TheCoelacanth 5y agoMost places don't care about security very much in general. It is a security leak, albeit usually a minor one.
- teddyh 5y agoYes; it’s often called the German tank problem: https://en.wikipedia.org/wiki/German_tank_problem https://en.wikipedia.org/wiki/German_tank_problem
- allknowingfrog 5y agoThanks for sharing this. I'm not saying this will definitely come up in my day job, but it's at least possible, and now I'll know what to call it. :)
- wayneftw 5y agoYou asked if it was a bad idea in general. It's not. Perhaps it's a bad idea in this one specific case but not in general.
- jacobsenscott 5y agoI've certainly seen this be a problem - even if you and every future programmer who touches your code gets the authorization check right every time over the years the app is live, and over the hundreds or thousands of endpoints it exposes, plenty of apps don't get it right every time. Using random id's is a nice and low effort additional layer of security there.
- worble 5y agoIf you're not doing your authorization properly, then no, basic ID obfuscation is not an extra layer of security. If anything, the fact that some people might gloss over the glaring security issues because "well its random ids so I never decided to check" is worse, an incremental id can at least be easily tested while developing to make sure, or a nice white hat hacker might notice and let you know.
- tonyarkles 5y agoDepending, very much, on that balance between usability and security. For the use case in the story, I'd way rather talk about (and write down) incidents 122 through 124 with my team, rather than incidents 549697a9-dd6a-4a90-a5f4-b2ff1a1d9289, d6150929-a692-414b-ba9e-ccbcf5e48a59, and 2756998d-035f-42bc-a97d-4135529e85d9.
- redis_mlc 5y ago> Isn't creating a sequence a bad idea in general, anyway? No: - sequences are very common. I recommend using them on every table for mgmt. and internal efficiency reasons. For example, with Innodb, if you don't have a numeric id as a PK, it will assign an invisible one for internal use anyway. Most third-party tools won't allow you to manage tables without numeric PK's. - in most large applications, most sequence ID's are only used internally - for public display (your concern) uuids or random numbers are possible Source: DBA.
- evanelias 5y agoI agree with your overall point, but wanted to clarify one topic. > with Innodb, if you don't have a numeric id as a PK, it will assign an invisible one for internal use anyway. There's no requirement that your PK be numeric with InnoDB. A monotonically increasing numeric ID will have the best performance, yes. But even a small-ish varchar PK may perform better than relying on InnoDB's internal invisible one, depending on the workload. InnoDB will only use an invisible numeric PK if you have no explicit PK defined, and you either have no UNIQUE KEYs at all either, or all of your UNIQUE KEYs have nullable columns. The column type of your PK is irrelevant though. The invisible numeric PK is terrible because it uses a system-wide lock (or at least it did prior to 8.0, not sure if this has been fixed). So if you're inserting at any real volume to multiple tables like this, performance suffers badly. Worse still, the lock it uses is the dict_sys mutex, which other code paths (e.g. DROP TABLE) also hit.
- bouke 5y agoObscurity isn’t a substitute for security. You still need to have proper authentication and authorization in place.
- SahAssar 5y agoIt can still leak information, like if a companies customers have sequential id's and your id is 590 then it is reasonable to assume that the company has had around 500-600 customers. It is usually not something to worry about, but in some cases you want to avoid leaking that info.
- Symbiote 5y agoThe German Tank Problem is one example of this. The Allies estimated how many tanks Germany was producing based on the serial numbers, and this gave a closer result compared to other intelligence sources. https://en.wikipedia.org/wiki/German_tank_problem https://en.wikipedia.org/wiki/German_tank_problem
- laurent92 5y agoOr Jira tickets. When you submit two support cases in a few days and the number increased by 20 while the company is still a startup, you are pretty sure to have no decent answer.
- combatentropy 5y agoSo is that a bug or a feature ;)
- peterejhamilton 5y agoHey, author here :) I don't think they're always a bad idea, but when talking about external IDs many folks would agree with you, in a few ways, actually. Relying on exposed primary IDs being ordered, and/or exposing them, is often a bit of a can of worms. A common case is that you leak information about your company because people can see how quickly you're growing. I've seen this in a few products I use and it's always interesting when you take an action a few days apart and can see how much volume they're doing. TIL: German Tank Problem, thanks @teddyh! I first came across it with a good story about someone buying Donuts/Coffee in a shop and using the receipt numbers to estimate yesterday's sales, can't find the link, though :( In this instance it's not our primary ID field - those are internal, and are long and random. They holds no meaning other than being a reference, so aren't used for sorting or exposed to the customer, and even if it was, it wouldn't mean anything or confer any information. The IDs I refer to the in the article are more like external references. References we _explicitly_ want to increment by 1 each time. For those referencing invoices as a parallel, it's a good equivalent. I'm not familiar with the details in the comments below, but if I created two invoices and they were referenced #1 and #33, I'd be quite confused (which is a version of what our customers felt/experienced here). IMO It's also often a good idea to use different external IDs to your internal ones in APIs too, and ideally have no meaning attached to them, either. That way users don't do things like assume "record 99" was created before "record 100", and you can also move data around and migrate things, so long as you honour those external references (i.e., you can change the type/format of your internal IDs at will).