5 ms·
The program is the database is the interface
- zabzonk 2y agoLooks horrible to me. Why not a spreadsheet?
- ajross 2y agoWithout bothering to try to enumerate minutiae, here's the One Thing that would control such a decision in any such "why not just" argument: Complicated formats can't be meaningfully version controlled.
- AlienRobot 2y agoNor can a single file. If you're keeping records as files, I think you should just use a million files instead. 1. Easy to version control 2. You can sync on the cloud with less conflicts if each record is isolated from the rest 3. Sounds unique and cool
- TZubiri 2y agoSingle files can easily be versioned. Consider git, which uses blob based tracking, not file based tracking. It doesn't matter if you split file A into A and B, git tracks it just fine.
- zabzonk 2y ago> Complicated formats can't be meaningfully version controlled. Well, at the risk of simply saying "why not" - why not? Preferably with an example.
- ajross 2y agoHow can you diff the formulas in two spreadsheet templates?
- jodrellblank 2y agoXLSX files are zipped XML. Unzip them, strip the formulae out of the XML, and diff them?
- ajross 2y agoHow do you "strip the formulae out of the XML" then? You sort of hide a lot of complexity there. The Excel file format isn't complicated because it's stored as a zip file, it's complicated because it's complicated. Again, the use case is straightforward. You have an already-solved accounting problem. But, say, last years accounting solution is giving different answers than the one you just ran. So what changed? With the script in the linked article, the answer is trivially findable via the git history (and in larger software searchable via tools like "git bisect", etc...). And the reason for this is that source code is intended to be read by human beings, often including things like "comments" and "style guidelines" and "literate programming" to help the process. None of that exists for ad hoc GUI tools, and the result is that you can't meaningfully develop them as software. That kind of tasks maps very poorly to "strip the formulae out of the XML, and diff them".
- jodrellblank 2y agoThat's a mixed bag of claims all jumbled together. > How do you "strip the formulae out of the XML" then? Use any XML parser, or any pre-existing Excel library which can do that, e.g. the Import-Excel PowerShell module mentioned earlier: PS C:\test> $x = Open-ExcelPackage book1.xlsx PS C:\test> $x.Workbook.Worksheets[1].Cells["B3"].Formula SUM(B1:B2) > "trivially findable via the git history (and in larger software searchable via tools like "git bisect", etc...)". So put a pre-commit hook which dumps the spreadsheet as some kind of text file instead of a binary blob. > "And the reason for this is that source code is intended to be read by human beings" That's not "the reason" that's an unrelated popular saying. If it were true, reverse engineering, maintenance, refactoring, and porting between different languages would be easy. It isn't. Instead source code appears to be intended to be read by the compiler/interpreter the programmer is using, and if anyone else can make anything out of it, good luck to them. > "often including things like "comments" and "style guidelines" and "literate programming" to help the process". If you allow helpers external to the source code as part of the development process, that's good because it cuts off your incoming reply saying that a pre-commit hook writing the formulae from spreadsheet to text is too hard/too much work/unreasonable. > "None of that exists for ad hoc GUI tools" Ad-hoc gui tools aren't programmable. Notepad isn't programmable. Calculator isn't programmable. Excel isn't an ad-hoc tool, Excel is one of the most famous, most used, GUI tools on the planet with some of the largest ecosystem and community around it, and one of the most pluggable, scriptable, documented, standardised, systems going. > "and the result is that you can't meaningfully develop them as software". Humans wrote Microsoft Excel itself, wrote Windows search indexers for searching inside Excel documents, wrote SharePoint which can index and work with Excel content, wrote the Microsoft Graph API and M365 cloud which can integrate with Excel spreadsheets, wrote the OpenOffice/LibreOffice Excel importers/exporters, wrote the ImportExcel module and the DLLs it's based on, rewrote Excel in TypeScript for Office365. Claiming that humans can't meaningfully write code to work with ... weird formats? Excel? spreadsheets? Files that were once touched by a GUI? ... is such a throwing-hands-up-and-giving-up-without-trying take on things. People could if they wanted to. People running $16Bn departments with Excel sheets could if they took it seriously and invested an appropriate amount of money in doing so. It just occurred to me to google "diff excel spreadsheet" and look, there's one built-in: https://support.microsoft.com/en-us/office/compare-two-versions-of-a-workbook-by-using-spreadsheet-compare-0e1627fd-ce14-4c33-9ab1-8ea82c6a5a7e https://support.microsoft.com/en-us/office/compare-two-versi... And there's a SaaS product which does it: https://www.diffchecker.com/excel-compare/ https://www.diffchecker.com/excel-compare/ And another SaaS product: https://xlcompare.com/ https://xlcompare.com/ And another SaaS product: https://www.textcompare.org/excel/ https://www.textcompare.org/excel/ and more I didn't click on.
- TZubiri 2y agoOTOH Excel guy: Boom got it done in 5 minutes sips coffee what's next? 10x engineer: I have devised a new programming paradigm to tell me I need to spend less on coffee, wrote a blog post about it and missed a call from a client and a recruiter.
- endofreach 2y agoBut then, who'd invent excel? Maybe we need the compromise and let some people be the 10xcel engineer...
- TZubiri 2y agoFor sure, I just don't see it coming out of a place of "I need to analyze my credit card statement" Excel wasn't built out of a small personal project, it was a huge multi engineer project built on microsoft (built upon decades of spreadsheet software tradition and arguably centuries of accounting tradition), by a company that had a huge userbase that acted as stakeholders for managing the data and accounts of multi million dollar companies and small businesses alike. I think that complex contraptions arise more from complex necessities than overengineering a solution from a simple problem
- ajross 2y agoThe VisiCalc/Lotus erasure is very strong in this comment.
- TZubiri 2y agoWouldn't they fit within the "decades of spreadsheet software tradition"?
- bitwize 2y ago> Excel wasn't built out of a small personal project, But another program with a similar impact -- dBase, the precursor to Microsoft Access, was. Cecil Ratliff wrote it to manage football statistics so he could win his office football pool.
- jerryhomelab 2y agoI version control them by duplicating sheets
- throwaway173738 2y agoThey just re-invented Jupyter.
- dustingetz 2y ago(2023), i think - https://news.ycombinator.com/item?id=34761768 https://news.ycombinator.com/item?id=34761768
- Kwpolska 2y agoAnd it's still a "!!!DRAFT!!!" ending with a TODO that ended up on the front page?
- pianoben 2y agoI think they just invented Microsoft Access :)
- almosthere 2y agoeverything is trying to reinvent ms access these days, with a bit of RBAC on top. It was the original low code environment.
- hilti 2y agoHa! I see the resemblance, but they're actually opposites. Access adds visual layers between you and your data, while this LISP approach removes them. Access makes databases more approachable to non-programmers through GUIs, while this example makes programming more direct by collapsing all those abstraction layers into pure code. Same problems, completely different philosophies!
- pianoben 2y agoThe way I see it, the problem being addressed here is that there is structured data (for some definition of "structured"), and we are in search of a useful visual representation of that data in various aggregates. Lisp is a fantastic tool for accomplishing this, but to me, Access is tailor-made for exactly this problem (and a whole lot more besides, it's a beast). If one follows the trail of the original problem far enough, I think one is likely to end up with an Access-like solution regardless of implementation language.
- amelius 2y agoNice. In the same vein, I also like "the program is the config file".
- jim_lawless 2y agoThis reminds me of home computing in the late 70's when we used to keep our "database" info in DATA statements embedded in a given BASIC program. https://jimlawless.net/images/basic_data.png https://jimlawless.net/images/basic_data.png
- julesallen 2y agoGlad I'm not the only lunatic who would do something like this. At least until I got my paws on dBASE. I worked with somebody with your name in the early 90s on a Sequent/Dynix system, that wasn't you by chance was it?
- jim_lawless 2y agoNope. Not me.
- TZubiri 2y agoThese thoughts are a bit too scattered imo, maybe collect your thoughts a bit.
- JBiserkov 2y agoThis was posted on scattered-thoughts.net ;-) If this was a joke, I'm sorry for explaining it. If this was not a joke, I point you to the giant <h1>!!! DRAFT !!!</h1> as the first element of the <article>
- AlienRobot 2y agoAnd to think this could all have been solved in LibreOffice Calc if they spent similar amounts of time learning it...
- Hammershaft 2y agotbf learning clojure is a more valuable spend of time then learning LibreOffice Calc.
- hilti 2y agoWhat the author demonstrates here is a powerful principle that dates back to LISP's origins but remains revolutionary today: the collapse of artificial boundaries between program, data, and interface creates a more direct connection to the problem domain. This example elegantly shows how a few dozen lines of Clojure can replace an entire accounting application. The transactions live directly in the code, the categorization rules are simple pattern matchers, and the "interface" is just printed output of the transformed data. No SQL, no UI framework, no MVC architecture - yet it solves the actual problem perfectly. The power comes from removing indirection. In a conventional app, you have: - Data model (to represent the domain) - Storage layer (to persist the model) - Business logic (to manipulate the model) - UI (to visualize and interact) Each boundary introduces translation costs, impedance mismatches, and maintenance burden. In the LISP approach shown here, those boundaries disappear. The representation is the storage is the computation is the interface. And that direct connection to your problem is surprisingly empowering - it's why REPLs and notebooks have become so important for data work. Of course, there are tradeoffs. This works beautifully for personal tools and small-team scenarios. It doesn't scale to massive collaborative systems where you need rigid interfaces between components. But I suspect many of us are solving problems that don't actually need that complexity. I'm reminded of Greenspun's Tenth Rule: "Any sufficiently complicated program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp." The irony is that by embracing LISP principles directly, you can often avoid building those complicated programs in the first place. Rich Hickey's "Simple Made Easy" talk explores this distinction perfectly - what the industry calls "easy" (familiar tools and patterns) often creates accidental complexity. The approach shown here prioritizes simplicity over easiness, and the result speaks for itself.
- jahewson 2y agoOver time I’ve come to see LISP less as the natural collapse of artificial boundaries but the artificial collapse of natural ones. Where and how data is stored is a real concern, but where and how the program is stored isn’t. Security boundaries around data and executable code are of paramount importance. Data storage concerns don’t benefit from being mixed with programming language concerns but from computer and storage architecture concerns (eg column stores). In toy programs, such as this one, those concerns can all be discarded, so LISP is a good fit. But in serious systems it’s soon discovered that what is offered is in fact “simplistic made easy”. That’s not to say that traditional systems don’t suffer from all the ills Hickey diagnoses in them, but that we differ on what the cure is.
- meetkevin 2y ago[dead]
- w10-1 2y agoThat reminds me of a dream: 1. Write queries in datalog 2. Compose them like reusable functions 3. Stream that data between SQL db and dataframe columns and graph db 4. Both also supported with reusable composable function interfaces 5. With IDE/notebook feedback and content-assist including not just syntax but the referenced data model. 6. With distinct modes for exploring datasets and for generating code and tests for hardened pipelines. 7. With abstract accounting for operations of the db, transforms, and transfers. Data and language will always have ultra-specialized forms. I’m looking for a low-overhead way to explore solutions using different combinations of baseline paradigms before generating code for the one to productize.
- kaeland 2y agoThis might be what you're looking for... https://gtoolkit.com/ https://gtoolkit.com/
- bob1029 2y agoI am a huge proponent of ideas like using SQL to directly implement business logic. When you bring the logic and the data together under the same abstraction, things will begin to click that you didn't even know existed. Business logic expressed as SQL commands can be stored as yet more data within the same schema. This opens the door for reflection and other features you would ordinarily only think to look for in advanced backend languages.
- eitland 2y agoI sometimes deal with such code. I belive it works for you, and I can even say I think most projects have (a little) too few lines of embedded sql in them. But arguing for code that isn't checked out with the rest of the code when I clone the repo, isn't greppable, and radically departs from everything most people are used to, that isn't something I'd accept.
- tmountain 2y agoYou can greatly improve DX with something like this and get the best of both worlds. https://github.com/t1mmen/srtd https://github.com/t1mmen/srtd
- jamii 2y agoThis is an old prototype. I ended up making a language for it from scratch so that I could attach provenance metadata to values, making them directly editable even when far removed from their original source. https://www.scattered-thoughts.net/log/0027#preimp https://www.scattered-thoughts.net/log/0027#preimp https://x.com/sc13ts/status/1564759255198351360/video/1 https://x.com/sc13ts/status/1564759255198351360/video/1 I never wrote up most of that work. I still like the ideas though. Also if I had ever finished editing this I wouldn't have buried the lede quite so much.
- codr7 2y agoLisp makes a lovely data format imo, much nicer to work with than CSV/JSON/YAML/XML. https://github.com/codr7/whirlog https://github.com/codr7/whirlog
- nullpoint420 2y agoThe engineers yearn for org-mode
- pmkary 2y agoThe moment I saw the title I knew it has something to do with EVE. Those people were revolutionary and it is a shame EVE never made it, what a glorious masterpiece of an article.
- ZeroTalent 2y agoThe tests are the program
- moi2388 2y ago(defn text->tag [text] (first (for [[keyword tag] keyword->tag :when (clojure.string/includes? text keyword)] tag))) “Joe’s Coffee Hit” text->tag :error #object[TypeError TypeError: a.indexOf is not a function. (In 'a.indexOf(b)', 'a.indexOf' is undefined)] If you type "Joe's Coffee Hut" (including the "!) into the textbox above and hit the text->tag button, you'll see the result of running the text->tag function on that input. Looks great xD