4 ms·

If Microsoft releases a solution for viewing big data in Excel, does data have to become even bigger to be "big"? They're stuck endlessly chasing a carrot! :)

kreeben 6y ago
My thesis is MS will never go down this path, not because it isn't an excellent idea but because Excel has become inrefactorable.
- bitwize 6y ago
  That's okay, they'll just rewrite it -- largely in JavaScript -- and offer it as a service as part of Microsoft 365. It could then ingest datasets from anywhere in the cloud, even large SQL or MongoDB databases.
- dragonwriter 6y ago
  > My thesis is MS will never go down this path, not because it isn't an excellent idea but because Excel has become inrefactorable. They’ll just make an optional feature called PowerSomething that is integrated with the Excel UI but mostly a separate tool that interacts with the main Excel system without sharing its limits.
PeterisP 6y ago
Yes, sure - the "big data" issues are essentially about data above a certain point where very different techniques for processing it (e.g. cluster computing and algorithms to split and merge partial results of computing that's not trivially parallelizable) start to make sense or even become required. If technology progress grants a huge increase in available memory and computing power and algorithm efficiency, then that boundary moves upwards. There are early "big data" publications on datasets that decidedly shouldn't be treated as "big data" today because now it's entirely appropriate to process them with simple methods within the RAM on a decent server or in some cases even on my laptop.
rvba 6y ago
Excel can connect to relational databases since forever. Prehistoric versions had Microsoft Query and new ones have Power Query, which allow to build SQL. There are also various GUI addons that connect Excel to a database. For example SAP has one in their Business Warehouse (I think they try to change into something else). Also the "new" Power Query removes a lot of limits.

kreeben 6y ago

My thesis is MS will never go down this path, not because it isn't an excellent idea but because Excel has become inrefactorable.

bitwize 6y ago
That's okay, they'll just rewrite it -- largely in JavaScript -- and offer it as a service as part of Microsoft 365. It could then ingest datasets from anywhere in the cloud, even large SQL or MongoDB databases.
dragonwriter 6y ago
> My thesis is MS will never go down this path, not because it isn't an excellent idea but because Excel has become inrefactorable. They’ll just make an optional feature called PowerSomething that is integrated with the Excel UI but mostly a separate tool that interacts with the main Excel system without sharing its limits.

PeterisP 6y ago

Yes, sure - the "big data" issues are essentially about data above a certain point where very different techniques for processing it (e.g. cluster computing and algorithms to split and merge partial results of computing that's not trivially parallelizable) start to make sense or even become required. If technology progress grants a huge increase in available memory and computing power and algorithm efficiency, then that boundary moves upwards. There are early "big data" publications on datasets that decidedly shouldn't be treated as "big data" today because now it's entirely appropriate to process them with simple methods within the RAM on a decent server or in some cases even on my laptop.

rvba 6y ago

Excel can connect to relational databases since forever. Prehistoric versions had Microsoft Query and new ones have Power Query, which allow to build SQL. There are also various GUI addons that connect Excel to a database. For example SAP has one in their Business Warehouse (I think they try to change into something else). Also the "new" Power Query removes a lot of limits.