5 ms·

Do you mean Chaos-Mesh or TiDB? For Chaos-Mesh, it is not a suitable tool for deterministic unit test case (like inserting an overflow value and expect to get a

Do you mean Chaos-Mesh or TiDB? For Chaos-Mesh, it is not a suitable tool for deterministic unit test case (like inserting an overflow value and expect to get an error). For TiDB, it is easy to use `Alter Table` statement to enlarge the length of a integer column with zero impact on the online business.

capableweb 6y ago
Not sure if you're purposefully missing the question here. ithkuil is not asking if you can change the length of the integer column but rather if you somehow would have prevented the issue GitHub saw here with hitting the limit. So something like automatically updating the length, or giving the developers warnings, or alerts of some sort. My guess would be no, as I haven't seen this in the wild. And if the answer is indeed "no", your comment seems again to plug something that wouldn't actually solve the problem GitHub saw here.
ithkuil 6y ago
What capableweb said. Furthermore, I wonder how long does it take to enlarge a pk column using alter table on a database that has a significant fraction of 4 billion rows? EDIT: I'm not arguing that github would not benefit from TiDB; they might be genuinely better of with it on many fronts. I just wonder if it's fair to plug a product this way
- shenli3514 6y ago
  Thanks for the clarification. There is not automatically way to enlarge the length. And I think that is not the things should be done by the database itself. That's my own opinion. But I think it would be helpful to show a warning to DBA when the allocated value is close to the upper limit. For the following question, the answer is in seconds. To enlarge the column length, TiDB just need to modify the meta data. "how long does it take to enlarge a pk column using alter table on a database that has a significant fraction of 4 billion rows?"
  - capableweb 6y ago
    > we believe there is something we can do to help solve the database high availability and scalability problem > There is not automatically way to enlarge the length > I think that is not the things should be done by the database itself > would be helpful to show a warning to DBA So, in the end, TiDB does nothing to actually solve the problem they experienced but you still feel comfortable offering them your own solution? Sounds like you're a VP of Marketing, not a VP of Engineering. Leaves a very sour taste when spamming a forum like this. I hope your bosses don't read HN and sees your comments plugging TiDB everywhere where it's not relevant.
    - neilhan 6y ago
      well said, we will take note; We were just trying to help anything related to MySQL's scalability
  - ithkuil 6y ago
    > TiDB just need to modify the meta data. thanks, that's interesting. So existing data using the previous 32-bit encoded values happily coexist with the new data being written after the migration?
  - ithkuil 6y ago
    > There is not automatically way to enlarge the length. And I think that is not the things should be done by the database itself. Sorry; it's hard to balance terseness with clarity. What I meant is to ask whether a tool such as Chaos-Mesh could have helped the github operations team to be aware of such a failure mode by causing this kinds of problems early. I think that's the purpose of chaos-engineering right? Some things happen too infrequently for us to plan for them, until when they finally do, causing problems that are more costly to solve than the cost of the prevention. It's still unclear to me whether this particular failure mode is common enough to be part of what a chaos-engineering tool explores by default; after all, not all databases around contain billions of rows. If engineers have to explicitly enable such a test because they are afraid of what would happen if a PK overflows, well, then probably they would take other preventive measures so that the problem doesn't happen in the first place (either preventively upgrading the size of the counter, or at least put some monitoring). I think what happened here is that a solution was designed when the database was way smaller; the user base grew and grew and this particular failure mode was overlooked (because there are many many other things to care about!). (I write this because I'm genuinely curious about what other people think about this topic, not to bash on parent for a shameless plug; I mentioned that only as a side comment; I heard good things about TiDB, I wish you well)

capableweb 6y ago

Not sure if you're purposefully missing the question here. ithkuil is not asking if you can change the length of the integer column but rather if you somehow would have prevented the issue GitHub saw here with hitting the limit. So something like automatically updating the length, or giving the developers warnings, or alerts of some sort. My guess would be no, as I haven't seen this in the wild. And if the answer is indeed "no", your comment seems again to plug something that wouldn't actually solve the problem GitHub saw here.

ithkuil 6y ago

What capableweb said. Furthermore, I wonder how long does it take to enlarge a pk column using alter table on a database that has a significant fraction of 4 billion rows? EDIT: I'm not arguing that github would not benefit from TiDB; they might be genuinely better of with it on many fronts. I just wonder if it's fair to plug a product this way

shenli3514 6y ago
Thanks for the clarification. There is not automatically way to enlarge the length. And I think that is not the things should be done by the database itself. That's my own opinion. But I think it would be helpful to show a warning to DBA when the allocated value is close to the upper limit. For the following question, the answer is in seconds. To enlarge the column length, TiDB just need to modify the meta data. "how long does it take to enlarge a pk column using alter table on a database that has a significant fraction of 4 billion rows?"
- capableweb 6y ago
  > we believe there is something we can do to help solve the database high availability and scalability problem > There is not automatically way to enlarge the length > I think that is not the things should be done by the database itself > would be helpful to show a warning to DBA So, in the end, TiDB does nothing to actually solve the problem they experienced but you still feel comfortable offering them your own solution? Sounds like you're a VP of Marketing, not a VP of Engineering. Leaves a very sour taste when spamming a forum like this. I hope your bosses don't read HN and sees your comments plugging TiDB everywhere where it's not relevant.
  - neilhan 6y ago
    well said, we will take note; We were just trying to help anything related to MySQL's scalability
- ithkuil 6y ago
  > TiDB just need to modify the meta data. thanks, that's interesting. So existing data using the previous 32-bit encoded values happily coexist with the new data being written after the migration?
- ithkuil 6y ago
  > There is not automatically way to enlarge the length. And I think that is not the things should be done by the database itself. Sorry; it's hard to balance terseness with clarity. What I meant is to ask whether a tool such as Chaos-Mesh could have helped the github operations team to be aware of such a failure mode by causing this kinds of problems early. I think that's the purpose of chaos-engineering right? Some things happen too infrequently for us to plan for them, until when they finally do, causing problems that are more costly to solve than the cost of the prevention. It's still unclear to me whether this particular failure mode is common enough to be part of what a chaos-engineering tool explores by default; after all, not all databases around contain billions of rows. If engineers have to explicitly enable such a test because they are afraid of what would happen if a PK overflows, well, then probably they would take other preventive measures so that the problem doesn't happen in the first place (either preventively upgrading the size of the counter, or at least put some monitoring). I think what happened here is that a solution was designed when the database was way smaller; the user base grew and grew and this particular failure mode was overlooked (because there are many many other things to care about!). (I write this because I'm genuinely curious about what other people think about this topic, not to bash on parent for a shameless plug; I mentioned that only as a side comment; I heard good things about TiDB, I wish you well)