4 ms·

Looks horrible to me. Why not a spreadsheet?

ajross 2y ago
Without bothering to try to enumerate minutiae, here's the One Thing that would control such a decision in any such "why not just" argument: Complicated formats can't be meaningfully version controlled.
- AlienRobot 2y ago
  Nor can a single file. If you're keeping records as files, I think you should just use a million files instead. 1. Easy to version control 2. You can sync on the cloud with less conflicts if each record is isolated from the rest 3. Sounds unique and cool
  - TZubiri 2y ago
    Single files can easily be versioned. Consider git, which uses blob based tracking, not file based tracking. It doesn't matter if you split file A into A and B, git tracks it just fine.
- zabzonk 2y ago
  > Complicated formats can't be meaningfully version controlled. Well, at the risk of simply saying "why not" - why not? Preferably with an example.
  - ajross 2y ago
    How can you diff the formulas in two spreadsheet templates?
    - jodrellblank 2y ago
      XLSX files are zipped XML. Unzip them, strip the formulae out of the XML, and diff them?
      - ajross 2y ago
        How do you "strip the formulae out of the XML" then? You sort of hide a lot of complexity there. The Excel file format isn't complicated because it's stored as a zip file, it's complicated because it's complicated. Again, the use case is straightforward. You have an already-solved accounting problem. But, say, last years accounting solution is giving different answers than the one you just ran. So what changed? With the script in the linked article, the answer is trivially findable via the git history (and in larger software searchable via tools like "git bisect", etc...). And the reason for this is that source code is intended to be read by human beings, often including things like "comments" and "style guidelines" and "literate programming" to help the process. None of that exists for ad hoc GUI tools, and the result is that you can't meaningfully develop them as software. That kind of tasks maps very poorly to "strip the formulae out of the XML, and diff them".
        jodrellblank 2y ago
        That's a mixed bag of claims all jumbled together. > How do you "strip the formulae out of the XML" then? Use any XML parser, or any pre-existing Excel library which can do that, e.g. the Import-Excel PowerShell module mentioned earlier: PS C:\test> $x = Open-ExcelPackage book1.xlsx PS C:\test> $x.Workbook.Worksheets[1].Cells["B3"].Formula SUM(B1:B2) > "trivially findable via the git history (and in larger software searchable via tools like "git bisect", etc...)". So put a pre-commit hook which dumps the spreadsheet as some kind of text file instead of a binary blob. > "And the reason for this is that source code is intended to be read by human beings" That's not "the reason" that's an unrelated popular saying. If it were true, reverse engineering, maintenance, refactoring, and porting between different languages would be easy. It isn't. Instead source code appears to be intended to be read by the compiler/interpreter the programmer is using, and if anyone else can make anything out of it, good luck to them. > "often including things like "comments" and "style guidelines" and "literate programming" to help the process". If you allow helpers external to the source code as part of the development process, that's good because it cuts off your incoming reply saying that a pre-commit hook writing the formulae from spreadsheet to text is too hard/too much work/unreasonable. > "None of that exists for ad hoc GUI tools" Ad-hoc gui tools aren't programmable. Notepad isn't programmable. Calculator isn't programmable. Excel isn't an ad-hoc tool, Excel is one of the most famous, most used, GUI tools on the planet with some of the largest ecosystem and community around it, and one of the most pluggable, scriptable, documented, standardised, systems going. > "and the result is that you can't meaningfully develop them as software". Humans wrote Microsoft Excel itself, wrote Windows search indexers for searching inside Excel documents, wrote SharePoint which can index and work with Excel content, wrote the Microsoft Graph API and M365 cloud which can integrate with Excel spreadsheets, wrote the OpenOffice/LibreOffice Excel importers/exporters, wrote the ImportExcel module and the DLLs it's based on, rewrote Excel in TypeScript for Office365. Claiming that humans can't meaningfully write code to work with ... weird formats? Excel? spreadsheets? Files that were once touched by a GUI? ... is such a throwing-hands-up-and-giving-up-without-trying take on things. People could if they wanted to. People running $16Bn departments with Excel sheets could if they took it seriously and invested an appropriate amount of money in doing so. It just occurred to me to google "diff excel spreadsheet" and look, there's one built-in: https://support.microsoft.com/en-us/office/compare-two-versions-of-a-workbook-by-using-spreadsheet-compare-0e1627fd-ce14-4c33-9ab1-8ea82c6a5a7e https://support.microsoft.com/en-us/office/compare-two-versi... And there's a SaaS product which does it: https://www.diffchecker.com/excel-compare/ https://www.diffchecker.com/excel-compare/ And another SaaS product: https://xlcompare.com/ https://xlcompare.com/ And another SaaS product: https://www.textcompare.org/excel/ https://www.textcompare.org/excel/ and more I didn't click on.
- TZubiri 2y ago
  OTOH Excel guy: Boom got it done in 5 minutes sips coffee what's next? 10x engineer: I have devised a new programming paradigm to tell me I need to spend less on coffee, wrote a blog post about it and missed a call from a client and a recruiter.
  - endofreach 2y ago
    But then, who'd invent excel? Maybe we need the compromise and let some people be the 10xcel engineer...
    - TZubiri 2y ago
      For sure, I just don't see it coming out of a place of "I need to analyze my credit card statement" Excel wasn't built out of a small personal project, it was a huge multi engineer project built on microsoft (built upon decades of spreadsheet software tradition and arguably centuries of accounting tradition), by a company that had a huge userbase that acted as stakeholders for managing the data and accounts of multi million dollar companies and small businesses alike. I think that complex contraptions arise more from complex necessities than overengineering a solution from a simple problem
      - ajross 2y ago
        The VisiCalc/Lotus erasure is very strong in this comment.
        TZubiri 2y ago
        Wouldn't they fit within the "decades of spreadsheet software tradition"?
      - bitwize 2y ago
        > Excel wasn't built out of a small personal project, But another program with a similar impact -- dBase, the precursor to Microsoft Access, was. Cecil Ratliff wrote it to manage football statistics so he could win his office football pool.
- jerryhomelab 2y ago
  I version control them by duplicating sheets

Without bothering to try to enumerate minutiae, here's the One Thing that would control such a decision in any such "why not just" argument: Complicated formats can't be meaningfully version controlled.

AlienRobot 2y ago
Nor can a single file. If you're keeping records as files, I think you should just use a million files instead. 1. Easy to version control 2. You can sync on the cloud with less conflicts if each record is isolated from the rest 3. Sounds unique and cool
- TZubiri 2y ago
  Single files can easily be versioned. Consider git, which uses blob based tracking, not file based tracking. It doesn't matter if you split file A into A and B, git tracks it just fine.
zabzonk 2y ago
> Complicated formats can't be meaningfully version controlled. Well, at the risk of simply saying "why not" - why not? Preferably with an example.
- ajross 2y ago
  How can you diff the formulas in two spreadsheet templates?
  - jodrellblank 2y ago
    XLSX files are zipped XML. Unzip them, strip the formulae out of the XML, and diff them?
    - ajross 2y ago
      How do you "strip the formulae out of the XML" then? You sort of hide a lot of complexity there. The Excel file format isn't complicated because it's stored as a zip file, it's complicated because it's complicated. Again, the use case is straightforward. You have an already-solved accounting problem. But, say, last years accounting solution is giving different answers than the one you just ran. So what changed? With the script in the linked article, the answer is trivially findable via the git history (and in larger software searchable via tools like "git bisect", etc...). And the reason for this is that source code is intended to be read by human beings, often including things like "comments" and "style guidelines" and "literate programming" to help the process. None of that exists for ad hoc GUI tools, and the result is that you can't meaningfully develop them as software. That kind of tasks maps very poorly to "strip the formulae out of the XML, and diff them".
      - jodrellblank 2y ago
        That's a mixed bag of claims all jumbled together. > How do you "strip the formulae out of the XML" then? Use any XML parser, or any pre-existing Excel library which can do that, e.g. the Import-Excel PowerShell module mentioned earlier: PS C:\test> $x = Open-ExcelPackage book1.xlsx PS C:\test> $x.Workbook.Worksheets[1].Cells["B3"].Formula SUM(B1:B2) > "trivially findable via the git history (and in larger software searchable via tools like "git bisect", etc...)". So put a pre-commit hook which dumps the spreadsheet as some kind of text file instead of a binary blob. > "And the reason for this is that source code is intended to be read by human beings" That's not "the reason" that's an unrelated popular saying. If it were true, reverse engineering, maintenance, refactoring, and porting between different languages would be easy. It isn't. Instead source code appears to be intended to be read by the compiler/interpreter the programmer is using, and if anyone else can make anything out of it, good luck to them. > "often including things like "comments" and "style guidelines" and "literate programming" to help the process". If you allow helpers external to the source code as part of the development process, that's good because it cuts off your incoming reply saying that a pre-commit hook writing the formulae from spreadsheet to text is too hard/too much work/unreasonable. > "None of that exists for ad hoc GUI tools" Ad-hoc gui tools aren't programmable. Notepad isn't programmable. Calculator isn't programmable. Excel isn't an ad-hoc tool, Excel is one of the most famous, most used, GUI tools on the planet with some of the largest ecosystem and community around it, and one of the most pluggable, scriptable, documented, standardised, systems going. > "and the result is that you can't meaningfully develop them as software". Humans wrote Microsoft Excel itself, wrote Windows search indexers for searching inside Excel documents, wrote SharePoint which can index and work with Excel content, wrote the Microsoft Graph API and M365 cloud which can integrate with Excel spreadsheets, wrote the OpenOffice/LibreOffice Excel importers/exporters, wrote the ImportExcel module and the DLLs it's based on, rewrote Excel in TypeScript for Office365. Claiming that humans can't meaningfully write code to work with ... weird formats? Excel? spreadsheets? Files that were once touched by a GUI? ... is such a throwing-hands-up-and-giving-up-without-trying take on things. People could if they wanted to. People running $16Bn departments with Excel sheets could if they took it seriously and invested an appropriate amount of money in doing so. It just occurred to me to google "diff excel spreadsheet" and look, there's one built-in: https://support.microsoft.com/en-us/office/compare-two-versions-of-a-workbook-by-using-spreadsheet-compare-0e1627fd-ce14-4c33-9ab1-8ea82c6a5a7e https://support.microsoft.com/en-us/office/compare-two-versi... And there's a SaaS product which does it: https://www.diffchecker.com/excel-compare/ https://www.diffchecker.com/excel-compare/ And another SaaS product: https://xlcompare.com/ https://xlcompare.com/ And another SaaS product: https://www.textcompare.org/excel/ https://www.textcompare.org/excel/ and more I didn't click on.
TZubiri 2y ago
OTOH Excel guy: Boom got it done in 5 minutes sips coffee what's next? 10x engineer: I have devised a new programming paradigm to tell me I need to spend less on coffee, wrote a blog post about it and missed a call from a client and a recruiter.
- endofreach 2y ago
  But then, who'd invent excel? Maybe we need the compromise and let some people be the 10xcel engineer...
  - TZubiri 2y ago
    For sure, I just don't see it coming out of a place of "I need to analyze my credit card statement" Excel wasn't built out of a small personal project, it was a huge multi engineer project built on microsoft (built upon decades of spreadsheet software tradition and arguably centuries of accounting tradition), by a company that had a huge userbase that acted as stakeholders for managing the data and accounts of multi million dollar companies and small businesses alike. I think that complex contraptions arise more from complex necessities than overengineering a solution from a simple problem
    - ajross 2y ago
      The VisiCalc/Lotus erasure is very strong in this comment.
      - TZubiri 2y ago
        Wouldn't they fit within the "decades of spreadsheet software tradition"?
    - bitwize 2y ago
      > Excel wasn't built out of a small personal project, But another program with a similar impact -- dBase, the precursor to Microsoft Access, was. Cecil Ratliff wrote it to manage football statistics so he could win his office football pool.
jerryhomelab 2y ago
I version control them by duplicating sheets