3 ms·

I don't understand the first one: a shared database table’s auto-incrementing ID column exceeded the size that can be represented by the MySQL

by TekMol 6y ago

I don't understand the first one: a shared database table’s auto-incrementing ID column exceeded the size that can be represented by the MySQL Integer type Followed by: GitHub’s monitoring systems currently alert when tables hit 70% of the primary key size So why was there no alert?

romanhn 6y ago
The latter looks to be the solution put in place to deal with the former. That seems to be the pattern with all of these - first list the issue, then the last paragraph describes the remediation.
Doxin 6y ago
More importantly, why is their ID type small enough to where this is even a possibility?
- hvidgaard 6y ago
  The "default" is a 32 bit int, large enough for almost 4.3 billion records. That is enough for the vast majority of tables, and going to a 64 bit int has some performance implications, which at GitHubs scale is absolutely something they need to take into consideration, just as they should have with the table size.
- dx034 6y ago
  I've set up systems where I chose 32bit int because I could've never imagined to need more than 4 billion values. AT some points I did (related to deletes) but changing the datatype when you have >1bn records can be next to impossible for 24/7 operations, especially for primary keys.
rsa25519 6y ago
My interpretation is that "currently" is referring to right now, after that incident. I assume they meant to suggest that the monitoring system was put in place as a result of the incident. Or, I'm wrong. Maybe there wasn't an alert, or maybe the alert was ignored.
deleted 6y ago
[deleted]
mjw1007 6y ago
The report could certainly be clearer here. They don't unambigously say that the table with the primary key was the one rejecting the data, and they do say « We are now extending our test frameworks to include a linter in place for int / bigint foreign key mismatches. » So perhaps the primary key itself was 64 bit, but a column referring to that key in a different table was mistakenly 32 bit.
user5994461 6y ago
Sound like they are using 32 bits keys and reached the limits of 4 billion items. Had the same thing happen in my company earlier this year in our homemade chat system. Why didn't they get alerts? Because the alert was probably counting existing items in the table, not deleted elements that did increase the auto increment.

by TekMol 6y ago

romanhn 6y ago

The latter looks to be the solution put in place to deal with the former. That seems to be the pattern with all of these - first list the issue, then the last paragraph describes the remediation.

Doxin 6y ago

More importantly, why is their ID type small enough to where this is even a possibility?

hvidgaard 6y ago
The "default" is a 32 bit int, large enough for almost 4.3 billion records. That is enough for the vast majority of tables, and going to a 64 bit int has some performance implications, which at GitHubs scale is absolutely something they need to take into consideration, just as they should have with the table size.
dx034 6y ago
I've set up systems where I chose 32bit int because I could've never imagined to need more than 4 billion values. AT some points I did (related to deletes) but changing the datatype when you have >1bn records can be next to impossible for 24/7 operations, especially for primary keys.

rsa25519 6y ago

My interpretation is that "currently" is referring to right now, after that incident. I assume they meant to suggest that the monitoring system was put in place as a result of the incident. Or, I'm wrong. Maybe there wasn't an alert, or maybe the alert was ignored.

deleted 6y ago

[deleted]

mjw1007 6y ago

The report could certainly be clearer here. They don't unambigously say that the table with the primary key was the one rejecting the data, and they do say « We are now extending our test frameworks to include a linter in place for int / bigint foreign key mismatches. » So perhaps the primary key itself was 64 bit, but a column referring to that key in a different table was mistakenly 32 bit.

user5994461 6y ago

Sound like they are using 32 bits keys and reached the limits of 4 billion items. Had the same thing happen in my company earlier this year in our homemade chat system. Why didn't they get alerts? Because the alert was probably counting existing items in the table, not deleted elements that did increase the auto increment.