6 ms·

In my experience, those that use Spreadsheets are not building full-fledge applications but instead more Jupyter/Notebook style "I ran these reports for my next

In my experience, those that use Spreadsheets are not building full-fledge applications but instead more Jupyter/Notebook style "I ran these reports for my next meeting" sorta deal. For that type of "MVP" or "I can throw it away and start over" deal, it seems like a _great_ tool.

yomly 6y ago
There's a real mixture: I essentially had a spreadsheet driven CRM tool that could Ping an internal data warehouse API Run data validation Generate email templates Send emails via outlook integration So some of that wasn't FP as it used VBA but all the validation and checks were done using vanilla functions and were therefore functional. But you're right, until I figured out how abstract and actually program, VBA script was bound to report generation before I started hitting unmaintainable complexity
KMag 6y ago
I wish there were a better bridge from spreadsheets to full-blown applications, though. I imagine some kind of tool where you have your prototype spreadsheet and click "Export to application", and a wizard walks you through naming your columns, which then get exported as named arrays/matrices in a declarative/functional dataflow language. You name your formulas, which become named functions. You mark some of the cells as inputs and others as outputs, which then kicks you over to a GUI wizard for laying out the end-user UI. Cells that aren't formulas, inputs, or outputs become named constants in your program. Maybe there's JSON REST, Protobuf gRPC, and Bloomberg market data API integration for inputs and outputs. I've spent the past 10 years maintaining and improving a finance-centric lazily-evaluated dataflow language with integrated distributed NoSQL database for a Fortune 500 financial services company. My understanding is that most of our competitors use Excel where we use our language/database. Some of our competitors have hired away some of our developers to explicitly write more modern clones of our system. One competitor's clone uses Python with function decorators that inspect the function's AST and transform it into a reactive dataflow language, though I imagine it's very frustrating to write something that looks like Python with very non-Pythonic evaluation semantics. I think starting with something like Haskell or Prolog would be less jarring. My wife does finance/accounting for another large multinational, and I see the appeal and power of both Excel and our system. I wish there were some easier way to migrate from spreadsheets to more scalable and maintainable systems. (And, I suppose, there's some utility in bidirectional migration.)
- tlarkworthy 6y ago
  https://observablehq.com/@tomlarkworthy/saas-tutorial https://observablehq.com/@tomlarkworthy/saas-tutorial
  - KMag 6y ago
    Absolutely beautiful! It's not immediately obvious to me how to import an Excel spreadsheet, but something like Observable or AirTable is where spreadsheets need to migrate. Lazily evaluated dataflow like Observable is the right paradigm.
    - tlarkworthy 6y ago
      wow it be crazy if you could embed a spreadsheet and trigger observable cells from embedded spreadsheet cells, giving you the opportunity to refactor out the logic from the spreadsheet slowly.
  - dmaevsky 6y ago
    This is NUTS!!! Exactly the example I was looking for! Mind if I re-implement all this on the Ellx platform (https://ellx.io https://ellx.io)?
    - tlarkworthy 6y ago
      No problem it's to help people
- dmaevsky 6y ago
  I think we should talk! :) From your description I know exactly where you work and which other Python based system you're talking about ) I have been in the industry for 20 years myself as a quant, mostly using Excel as a "glue" layer indeed, and I worked with the Python based "clone" you mentioned too. For the last two years I have been working on a platform to fix exactly the pain points you are talking about. Amazing how your post sounds almost like as if I wrote it myself before I started working on this project :) Please, take a look at https://ellx.io https://ellx.io I would be more than happy to discuss it in detail: dmitry + at + ellxoft.com
magicalhippo 6y ago
Guy at support uses Excel to generate SQL statements. Typically when we have a request like "oops, we accidentally entered wrong weights for these articles, please fix", where he gets an Excel sheet of article numbers and correct weights. But thanks to the flexible nature, toss in a little logic when needed for example, he can quickly churn out some non-trivial SQL to do the job. Of course may other ways to do this kinda job, but for him this is quite productive. He starts by writing the query he needs for doing one update, then just stuff it into Excel, replacing fixed values with cell references and copy formula down all rows and copies result back into interactive SQL editor.

yomly 6y ago

There's a real mixture: I essentially had a spreadsheet driven CRM tool that could Ping an internal data warehouse API Run data validation Generate email templates Send emails via outlook integration So some of that wasn't FP as it used VBA but all the validation and checks were done using vanilla functions and were therefore functional. But you're right, until I figured out how abstract and actually program, VBA script was bound to report generation before I started hitting unmaintainable complexity

KMag 6y ago

I wish there were a better bridge from spreadsheets to full-blown applications, though. I imagine some kind of tool where you have your prototype spreadsheet and click "Export to application", and a wizard walks you through naming your columns, which then get exported as named arrays/matrices in a declarative/functional dataflow language. You name your formulas, which become named functions. You mark some of the cells as inputs and others as outputs, which then kicks you over to a GUI wizard for laying out the end-user UI. Cells that aren't formulas, inputs, or outputs become named constants in your program. Maybe there's JSON REST, Protobuf gRPC, and Bloomberg market data API integration for inputs and outputs. I've spent the past 10 years maintaining and improving a finance-centric lazily-evaluated dataflow language with integrated distributed NoSQL database for a Fortune 500 financial services company. My understanding is that most of our competitors use Excel where we use our language/database. Some of our competitors have hired away some of our developers to explicitly write more modern clones of our system. One competitor's clone uses Python with function decorators that inspect the function's AST and transform it into a reactive dataflow language, though I imagine it's very frustrating to write something that looks like Python with very non-Pythonic evaluation semantics. I think starting with something like Haskell or Prolog would be less jarring. My wife does finance/accounting for another large multinational, and I see the appeal and power of both Excel and our system. I wish there were some easier way to migrate from spreadsheets to more scalable and maintainable systems. (And, I suppose, there's some utility in bidirectional migration.)

tlarkworthy 6y ago
https://observablehq.com/@tomlarkworthy/saas-tutorial https://observablehq.com/@tomlarkworthy/saas-tutorial
- KMag 6y ago
  Absolutely beautiful! It's not immediately obvious to me how to import an Excel spreadsheet, but something like Observable or AirTable is where spreadsheets need to migrate. Lazily evaluated dataflow like Observable is the right paradigm.
  - tlarkworthy 6y ago
    wow it be crazy if you could embed a spreadsheet and trigger observable cells from embedded spreadsheet cells, giving you the opportunity to refactor out the logic from the spreadsheet slowly.
- dmaevsky 6y ago
  This is NUTS!!! Exactly the example I was looking for! Mind if I re-implement all this on the Ellx platform (https://ellx.io https://ellx.io)?
  - tlarkworthy 6y ago
    No problem it's to help people
dmaevsky 6y ago
I think we should talk! :) From your description I know exactly where you work and which other Python based system you're talking about ) I have been in the industry for 20 years myself as a quant, mostly using Excel as a "glue" layer indeed, and I worked with the Python based "clone" you mentioned too. For the last two years I have been working on a platform to fix exactly the pain points you are talking about. Amazing how your post sounds almost like as if I wrote it myself before I started working on this project :) Please, take a look at https://ellx.io https://ellx.io I would be more than happy to discuss it in detail: dmitry + at + ellxoft.com

magicalhippo 6y ago

Guy at support uses Excel to generate SQL statements. Typically when we have a request like "oops, we accidentally entered wrong weights for these articles, please fix", where he gets an Excel sheet of article numbers and correct weights. But thanks to the flexible nature, toss in a little logic when needed for example, he can quickly churn out some non-trivial SQL to do the job. Of course may other ways to do this kinda job, but for him this is quite productive. He starts by writing the query he needs for doing one update, then just stuff it into Excel, replacing fixed values with cell references and copy formula down all rows and copies result back into interactive SQL editor.