10 ms·

I think about this a lot. There is a need for more data-driven applications to create graphics that are more “track able” than excel. I have a series of charts

I think about this a lot. There is a need for more data-driven applications to create graphics that are more “track able” than excel. I have a series of charts that need to be sent out to multiple people (internal and external) each week. Charts, logos, tables, etc could benefit from a pipeline but without needing a data engineer. Right now, we have a standard data download from Bloomberg or another data provider, macros to change the format, access to slice, back to excel where the charts are 75% standard but might need some tweaking. Bloomberg might not classify one company as healthcare but I want to include them so access keeps a list or companies I want. That same Bloomberg output could include companies some of my colleagues would classify as tech companies so relevant for multiple people. No way to see what was done last week or changes made to the chart / table output from one person to another besides new versions of the same excel file (but still no way to see the logs) Long winded description but hopefully helps describe how important excel is but there is still room to grow.

jayd16 5y ago
>No way to see what was done last week or changes made to the chart / table output from one person to another besides new versions of the same excel file (but still no way to see the logs) Why are multiple columns/pages with the various stages of the data insufficient?
jsmith99 5y ago
Excel includes Power Query which is a decent ETL pipeline tool.
wvenable 5y ago
Isn't this just a programming problem? Build an application that accepts the data, produces the result, and has a GUI for managing all exceptions? I've build many of these things.
- dragonwriter 5y ago
  > Isn't this just a programming problem? What seems to be described is a UI design problem. Once the UI is designed, it also needs programmed in some form, but that doesn't seem to be the problem, just a not-particularly-interesting task required in the implementation. Not all problems that require programming are programming problems. (In fact, most are not.)
  - wvenable 5y ago
    I meant "isn't this a problem that requires software development?" There are always complaints that Excel is a terrible solution to some very specific set of repeated tasks. Every few months, there's a posting about how all we need is a "Better Excel" that would perfectly solve everyone's totally unique 12-step problem. Such a tool is never going to exist. Thankfully we can build tools specifically for those problems when generic tools aren't good enough. Not every problem is worth a programming solution; there's only so much time and money and paying a bunch of people to cut and paste might just be the better option.
Closi 5y ago
It sounds like you are looking for Excel's PowerQuery functionality? You should not need to bring the data into Access, or write Macro's to do what you are doing anymore (Macro's were required in older versions of Excel, but not newer ones that have the data modelling capabilities). This effectively is a data pipeline built within Excel that can either be edited visually or in 'M-Code'.

jayd16 5y ago

>No way to see what was done last week or changes made to the chart / table output from one person to another besides new versions of the same excel file (but still no way to see the logs) Why are multiple columns/pages with the various stages of the data insufficient?

jsmith99 5y ago

Excel includes Power Query which is a decent ETL pipeline tool.

wvenable 5y ago

Isn't this just a programming problem? Build an application that accepts the data, produces the result, and has a GUI for managing all exceptions? I've build many of these things.

dragonwriter 5y ago
> Isn't this just a programming problem? What seems to be described is a UI design problem. Once the UI is designed, it also needs programmed in some form, but that doesn't seem to be the problem, just a not-particularly-interesting task required in the implementation. Not all problems that require programming are programming problems. (In fact, most are not.)
- wvenable 5y ago
  I meant "isn't this a problem that requires software development?" There are always complaints that Excel is a terrible solution to some very specific set of repeated tasks. Every few months, there's a posting about how all we need is a "Better Excel" that would perfectly solve everyone's totally unique 12-step problem. Such a tool is never going to exist. Thankfully we can build tools specifically for those problems when generic tools aren't good enough. Not every problem is worth a programming solution; there's only so much time and money and paying a bunch of people to cut and paste might just be the better option.

Closi 5y ago

It sounds like you are looking for Excel's PowerQuery functionality? You should not need to bring the data into Access, or write Macro's to do what you are doing anymore (Macro's were required in older versions of Excel, but not newer ones that have the data modelling capabilities). This effectively is a data pipeline built within Excel that can either be edited visually or in 'M-Code'.