4 ms·
Everybody's right. Excel is a powerful, flexible tool that also has almost no guard rails and all but begs people to make profound mistakes and huge messes. The
by jerf 10y ago
Everybody's right. Excel is a powerful, flexible tool that also has almost no guard rails and all but begs people to make profound mistakes and huge messes. There's too many people who sneer at spreadsheets when they should be using them, and there's too many people who use them when they shouldn't.
No contradictions.
It would be interesting to see if anyone could get some power Excel users together and construct a next-generation spreadsheet that encouraged better practices and worked to prevent huge messes. Spreadsheets are like SQL, where the initial release was so far ahead of its time that it managed to entrench itself into the very fabric of computing, even though it's long overdue for a reimagining.
- claytonjy 10y agoCould you say a bit more about what you'd like to see in a "reimagined" SQL? Are there any serious efforts to replace it? There was a comment thread around here a week or two ago where someone pointed out it's kind of insane SQL has stuck around so long, and no one could point to any worthy potential replacements.
- gregw134 10y agoVisual query tools like Tableau don't seem to be going away. I'd love to see an effective open-source alternative to Tableau that doesn't require scripting your own D3 website.
- theoh 10y agoButler Lampson makes the point (in a recent set of slides) that relations are a good base for DSLs: they have enough complexity to model graphs, functions, sets etc. That thought has made me wonder if logic programmming has something to offer in the design of new APIs.
- jerf 10y agoSQL in fact deviates from true relational theory, in which the "cells" of a table could themselves have additional structure rather than just being "a string" or "a number". Cells could also be truly absent. SQL's NULL, while something you can make your peace with, could use some tweaking with 21st century experience. SQL's syntax has acquired a lot of cruft over the years to deal with new features... in fact in that sense it reminds me of the evolution of OpenGL and the way it acquired extension after extension until finally it needed to be broken apart into Vulkan and CUDA pieces (to brutally summarize the situation to the point of inaccuracy; please try to see what I mean rather than pick nits with that). More controversially, I question the entire intent of making the core query language something that is putatively declarative, but then in practice often requires extensive engine-specific annotations to tell the engine how to actually do the query. (More on that https://news.ycombinator.com/item?id=3506345#3507281 https://news.ycombinator.com/item?id=3506345#3507281 ). I think RethinkDB's query language was much more imperative, because of the level of development resources they had, and I bet it actually worked out OK. However, even if I could not sell the development world on making SQL++/SQL-replacement non-declarative, we certainly could do a better job this time around of separating query strategy from query contents in some deliberate manner, rather than hacking crap up. Imagine if, for instance, you could feed the query optimizer a query, get back a query plan that was actually manipulable and executable, tweak that to your tastes, and then send it back to the DB, rather than working via hints and circumlocutions and hopes and dreams. It would also be nice if SQL were more composible. The serialized version of SQL is not practical to use string manipulations to combine two queries into a larger query. Many languages have libraries that permit this, but they're always second-class citizens. If I were redesigning SQL I'd want something that handled this more cleanly. I'd seriously consider something RethinkDB-esque in the sense that it didn't have an "english" serialization, but was purely symbolic, leaving it to language authors to figure out how to best represent it in the local language. Also, bear in mind that most if not all features I describe in this post exist in databases already. (Not sure about that last one.) What I'm saying is that SQL integrates poorly with all that, not that the features don't exist. Recursive queries and common table expressions also seem ripe for some serious rethinking. Plus I think for a long time SQL really kinda limited the sort of DBs that would be produced because if a feature integrated poorly with SQL, it was a lot less likely to come out. (In particular, structured cells took IMHO forever to come out. Possibly the massive market failure of "object databases" also scared DB developers off from that feature too, though. They aren't the same thing but may be closely enough related.)
- nradov 10y agoMost modern relational databases now allow the cells of a table to have additional structure through the use of SQL/XML. We can query into the contents of a cell using XQuery.
- nradov 10y agoThere's Tutorial D but it probably doesn't qualify as "serious". https://en.wikipedia.org/wiki/D_(data_language_specification) https://en.wikipedia.org/wiki/D_(data_language_specification...
- coldtea 10y ago>There was a comment thread around here a week or two ago where someone pointed out it's kind of insane SQL has stuck around so long, and no one could point to any worthy potential replacements SQL is based on relational algebra -- so it's the model with the best theoritical justification out there, even if the syntax could be improved. It's the other ad-hoc solutions that is crazy that they keep getting suggested. SQL/RDBMS were invented because we had those (key stores etc, tree dbs) and they were crap.
- willhslade 10y agoI'm an Excel power user. I think Microsoft is moving in the right direction, with the addition of Tables, PowerBI, PowerPivot and R in SQL Server. What I'd like to see in Excel is: Hotkey training built into Excel Python as an optional language along side VBA Proper Data Tables with Types and Indices, or even SQL in Excel. Regex Search over Columns PowerPivot use case training Web publishing of reports made stupid easy
- eon1 10y agoBoth Python and R can be used in Excel via several addons. Python: http://www.python-excel.org/ http://www.python-excel.org/ https://www.xlwings.org/ https://www.xlwings.org/ https://datanitro.com/ https://datanitro.com/ R: http://rcom.univie.ac.at/download.html http://rcom.univie.ac.at/download.html https://bert-toolkit.com/ https://bert-toolkit.com/ However, officially MS have chosen to move ahead with Javascript for add-ins and VBA-like automation: https://dev.office.com/docs/add-ins/develop/understanding-the-javascript-api-for-office https://dev.office.com/docs/add-ins/develop/understanding-th... http://rockthecode.io/blog/javascript-and-excel/ http://rockthecode.io/blog/javascript-and-excel/ As for SQL, you can use Data Connections from the GUI or ADO with JET/ACE in VBA to query Excel sheets, CSVs, etc as you please. Regex can be used with the VBScript.Regexp object, but it's a slightly funky (perl-like) syntax and not a great implementation.
- s_q_b 10y agoI fully concur. This is an excellent summary and suggestion for future progress. The barriers to moving beyond Excel can be overcome, but it will take some serious effort on many fronts. Both Excel and SQL embody genius concepts, but are such poor implementations that it is easy to conflate the cruft with the advantages.
- jlg23 10y agoSQL is not an implementation but a specification and thus cannot be compared to Excel, a very specific implementation of non-monotonic dataflow programming. Regarding your "stockholm syndrome" comment above: Someone in his car hears a PSA about "some guy wrong-way driving" on the very road he is on and thinks "one? hundreds!". Unless you can beef up your argumentation you are that guy.
- EdHominem 10y ago> Unless you can beef up your argumentation you are that guy. That's fallacious too. I can be right, even if my argument is incorrect or unconvincing. Warren Buffet and Nate Silver are both driving against traffic and both of them are righter than everyone else combined. > SQL [...] cannot be compared to Excel What Excel and SQL have in common is that they're both a first attempt at a solution to (different) problems, and they've been too successful to properly iterate on. That's why everyone uses some proprietary extensions to SQL and everyone extends Excel with VB or C#.
- Lio 10y ago> SQL is not an implementation but a specification For clarification, is the GP referring to Microsoft SQL Server when then they say 'SQL' or do they actually mean SQL? Microsoft's product naming convention is confusing IMHO.
- eon1 10y agoAlmost all of these are done or do-able, though. They're just not right there on the surface.
- lmm 10y agohttps://www.herculus.io/ https://www.herculus.io/ was doing the rounds a few days ago - the idea seems to be a spreadsheet with a type system.