7 ms·

I think these are billion dollar mistakes only because they've allowed for trillions in improvements in the first place. People choose these system with loose

I think these are billion dollar mistakes only because they've allowed for trillions in improvements in the first place. People choose these system with loose conversions over and over again because it's an advantage up to a point. Of course it becomes a liability when expanded too much and used in now critical systems, but IMHO the tradeoff usually ends as a net positive. Which is why, over and over we build more of these systems, and they fare better in the market than the stricter ones.

jjk166 3y ago
> People choose these system with loose conversions over and over again because it's an advantage up to a point. The world is full of antipatterns that people implement with little, no, or flawed analysis. They do not fare better in the market, they propagate the same way entropy does.
dacryn 3y ago
it would be so easy to just implement an opt-out function, but Microsoft doesn't want to. There doesn't have to be a tradeoff if you make it optional.
- cedilla 3y ago
  An opt-out doesn't solve the problem, because one person in the chain of custody if an excel file who hasn't opted out would be enough to destroy the data.
  - acdha 3y ago
    Only if the option was implemented as a user preference which isn’t stored in the file. The right way to do this would be to add the flag as a file header, and give people a UI control to set it by default. That way you’d always see spreadsheets as the person who gave it to you did, and you would create new ones with your preferred behavior.
    - makeitdouble 3y ago
      Channeling my inner QA person, this breaks in many subtle and not so subtle ways: - people love to export as CSV. And I kinda understand them, csv looks simple enough, what could go wrong ? - other spreadsheet softs need to recognize the same field and act the same way. Google Sheets, LibreOffice, KingSoft's one etc. Not counting old office versions that aren't updated anymore, because the owner doesn't want to move into the cloud. - the flag needs to stay when moving sheets around, copy to another set, etc. That could still be better than nothing. But could still lead to catastrophic loses as people get used to it working most of the time, there's no perfect solution.
      - acdha 3y ago
        It might be easier to think about as the difference between perfection and practical improvements. Right now 100% of Excel users are at risk of silent data corruption. The proposed change would allow people to reliably prevent this category of corruption for all of the documents they create and over time that’d become a majority in their field. This would also offer the path for changing the default over time to safe by default. There will be more spreadsheets created in the future than exist now.
        makeitdouble 3y ago
        > Right now 100% of Excel users are at risk of silent data corruption. They already added an option in the settings last year to deal with this specific issue. Their solution isn't perfect either, they made the tradeoff to optimize for up to date excel users.
    - cedilla 3y ago
      Still the same problems. Unless you exclude people with older excel versions they can still ruin the file. And if the first person in the chain didn't make the change, who would even notice it? So we're back at requiring extreme vigilance without any assurance. Also keep in mind that the first person doesn't necessarily use excel at all. The file might start and end as a text file or TSV.
    - 3y ago
- makeitdouble 3y ago
  As usual, I think it's a lot more nuanced than that. An option was added last year [0] the way you propose it, but until then users already had the option to mark the columns as text. The same way we have to deal with any data where auto detection might be problematic: explicitely set a type, and potentially a format. But of course that's bothersome, sometime people forget to do it, sometimes they don't even know how to properly do it etc. And some of these people will still probably forget to set the option, and there are still edge cases where it can fail (macros are cited) [0] https://www.theverge.com/2023/10/21/23926585/microsoft-excel-misreading-dates-human-genes-conversion-fixed https://www.theverge.com/2023/10/21/23926585/microsoft-excel...
  - DonsDiscountGas 3y ago
    > but until then users already had the option to mark the columns as text This only works on Excel files though, which save a type for columns/cells. With CSV files, Excel would just auto-format them. I guess you could save everything as xls/xlsx, but I shouldn't have to use a vendors file format because they're threatening to corrupt my data if I don't.
    - SAI_Peregrinus 3y ago
      That's just CSV being unusable shit. There's always a schema for the types of the fields, but there's no way to communicate that schema, so CSV is inherently ambiguous. If your data includes fancy things like numbers you should avoid CSV.
      - WorldMaker 3y ago
        It's still a shame that Excel's JSON support is so good, but no business user trusts it because "you can't double click the files" and CSV remains the worst best thing you can double click. I don't want Excel to take over the file association for .json but I do sometimes wish Excel would just invent a file extension like .xljson that loads JSON so that at least you could trivially rename a JSON file to .xljson and get working double-click for your business users but a file format (with optional schema support even) that's a more modern and saner de facto standard than CSV. I think that one dumb trick would improve everyone's productivity a great deal.
- deleted 3y ago
  [deleted]
_visgean 3y ago
> because it's an advantage up to a point or they dont know any better.
darby_eight 3y ago
> I think these are billion dollar mistakes only because they've allowed for trillions in improvements in the first place. That's a hell of a claim to try and demonstrate when its initial advantage is far from obvious to begin with.
- maxcoder4 3y ago
  when I type "age" into a spreadsheet it is text. When I type "23" into a spreadsheet I can later use it as a number without converting to another type. People want and expect this behavior. What is your preferred solution to this problem?
  - darby_eight 3y ago
    > People want and expect this behavior. A) I have no clue where you got this from. It seems like a convenient assumption, but not one that's easily verified, especially when folks are introduced to the dangers of such an approach. B) Giving people control over how the data is interpreted outside of the default interpretation seems like a no-brainer. I don't really rely on default interpretation of anything, especially in the context of spreadsheets given how eager they are to interpret data in unexpected ways, but based on this thread it seems like it's rather difficult to turn off the date conversion behavior in excel. C) I was replying to a comment about implicit conversions, which is far larger than just the context of excel spreadsheets.
    - makeitdouble 3y ago
      Wouldn't your approach be 100% covered by current type implementation ? As a workflow, if every time you create a new spreadsheet you first define all the column types and constraints, you're in the world you're describing. Passing around that spreadsheet would also keep these settings as long as it's not converted and everyone's version is compatible. That means you get the best of both worlds: people who care about strict types can be as strict as they want, other people can benefit from auto conversions.
    - imtringued 3y ago

jjk166 3y ago

> People choose these system with loose conversions over and over again because it's an advantage up to a point. The world is full of antipatterns that people implement with little, no, or flawed analysis. They do not fare better in the market, they propagate the same way entropy does.

dacryn 3y ago

it would be so easy to just implement an opt-out function, but Microsoft doesn't want to. There doesn't have to be a tradeoff if you make it optional.

cedilla 3y ago
An opt-out doesn't solve the problem, because one person in the chain of custody if an excel file who hasn't opted out would be enough to destroy the data.
- acdha 3y ago
  Only if the option was implemented as a user preference which isn’t stored in the file. The right way to do this would be to add the flag as a file header, and give people a UI control to set it by default. That way you’d always see spreadsheets as the person who gave it to you did, and you would create new ones with your preferred behavior.
  - makeitdouble 3y ago
    Channeling my inner QA person, this breaks in many subtle and not so subtle ways: - people love to export as CSV. And I kinda understand them, csv looks simple enough, what could go wrong ? - other spreadsheet softs need to recognize the same field and act the same way. Google Sheets, LibreOffice, KingSoft's one etc. Not counting old office versions that aren't updated anymore, because the owner doesn't want to move into the cloud. - the flag needs to stay when moving sheets around, copy to another set, etc. That could still be better than nothing. But could still lead to catastrophic loses as people get used to it working most of the time, there's no perfect solution.
    - acdha 3y ago
      It might be easier to think about as the difference between perfection and practical improvements. Right now 100% of Excel users are at risk of silent data corruption. The proposed change would allow people to reliably prevent this category of corruption for all of the documents they create and over time that’d become a majority in their field. This would also offer the path for changing the default over time to safe by default. There will be more spreadsheets created in the future than exist now.
      - makeitdouble 3y ago
        > Right now 100% of Excel users are at risk of silent data corruption. They already added an option in the settings last year to deal with this specific issue. Their solution isn't perfect either, they made the tradeoff to optimize for up to date excel users.
  - cedilla 3y ago
    Still the same problems. Unless you exclude people with older excel versions they can still ruin the file. And if the first person in the chain didn't make the change, who would even notice it? So we're back at requiring extreme vigilance without any assurance. Also keep in mind that the first person doesn't necessarily use excel at all. The file might start and end as a text file or TSV.
  - 3y ago
makeitdouble 3y ago
As usual, I think it's a lot more nuanced than that. An option was added last year [0] the way you propose it, but until then users already had the option to mark the columns as text. The same way we have to deal with any data where auto detection might be problematic: explicitely set a type, and potentially a format. But of course that's bothersome, sometime people forget to do it, sometimes they don't even know how to properly do it etc. And some of these people will still probably forget to set the option, and there are still edge cases where it can fail (macros are cited) [0] https://www.theverge.com/2023/10/21/23926585/microsoft-excel-misreading-dates-human-genes-conversion-fixed https://www.theverge.com/2023/10/21/23926585/microsoft-excel...
- DonsDiscountGas 3y ago
  > but until then users already had the option to mark the columns as text This only works on Excel files though, which save a type for columns/cells. With CSV files, Excel would just auto-format them. I guess you could save everything as xls/xlsx, but I shouldn't have to use a vendors file format because they're threatening to corrupt my data if I don't.
  - SAI_Peregrinus 3y ago
    That's just CSV being unusable shit. There's always a schema for the types of the fields, but there's no way to communicate that schema, so CSV is inherently ambiguous. If your data includes fancy things like numbers you should avoid CSV.
    - WorldMaker 3y ago
      It's still a shame that Excel's JSON support is so good, but no business user trusts it because "you can't double click the files" and CSV remains the worst best thing you can double click. I don't want Excel to take over the file association for .json but I do sometimes wish Excel would just invent a file extension like .xljson that loads JSON so that at least you could trivially rename a JSON file to .xljson and get working double-click for your business users but a file format (with optional schema support even) that's a more modern and saner de facto standard than CSV. I think that one dumb trick would improve everyone's productivity a great deal.
deleted 3y ago
[deleted]

_visgean 3y ago

> because it's an advantage up to a point or they dont know any better.

darby_eight 3y ago

> I think these are billion dollar mistakes only because they've allowed for trillions in improvements in the first place. That's a hell of a claim to try and demonstrate when its initial advantage is far from obvious to begin with.

maxcoder4 3y ago
when I type "age" into a spreadsheet it is text. When I type "23" into a spreadsheet I can later use it as a number without converting to another type. People want and expect this behavior. What is your preferred solution to this problem?
- darby_eight 3y ago
  > People want and expect this behavior. A) I have no clue where you got this from. It seems like a convenient assumption, but not one that's easily verified, especially when folks are introduced to the dangers of such an approach. B) Giving people control over how the data is interpreted outside of the default interpretation seems like a no-brainer. I don't really rely on default interpretation of anything, especially in the context of spreadsheets given how eager they are to interpret data in unexpected ways, but based on this thread it seems like it's rather difficult to turn off the date conversion behavior in excel. C) I was replying to a comment about implicit conversions, which is far larger than just the context of excel spreadsheets.
  - makeitdouble 3y ago
    Wouldn't your approach be 100% covered by current type implementation ? As a workflow, if every time you create a new spreadsheet you first define all the column types and constraints, you're in the world you're describing. Passing around that spreadsheet would also keep these settings as long as it's not converted and everyone's version is compatible. That means you get the best of both worlds: people who care about strict types can be as strict as they want, other people can benefit from auto conversions.
  - imtringued 3y ago