3 ms·
No way, not data science. How are you going to load in millions, possibly billions of individual numbers? With finance there's a lot of things that seem like g
by lordnacho 10y ago
No way, not data science. How are you going to load in millions, possibly billions of individual numbers?
With finance there's a lot of things that seem like good fits, but only if you keep them small. Something like a personal budget is fine, where everything fits on a screen.
- blahi 10y agoI have 2.4 billion data points loaded in Excel at the moment. Not that it matters. "Data science" is not something that magically kicks in after you go beyond some "big data" threshold. Excel is heavily used in managerial science type of positions and I can assure you those are rather heavy on the "data science" workflows.
- ramblenode 10y agoThe maximum number of rows in the most recent version is 1,048,576 [0]. I'm deducing then that your data is in wide format. What would happen if you wanted it in long format? It seems you would just be out of luck with Excel, which is not a problem in any other major statistics software. [0] https://support.office.com/en-us/article/Excel-specifications-and-limits-ca36e2dc-1f09-4620-b726-67c00b05040f https://support.office.com/en-us/article/Excel-specification...
- blahi 10y agoExcept that there is Power Pivot which is xVelocity - the same columnar store database use in SQL Server.
- goatlover 10y agoWhy on Earth would you use Excel for that many data points?
- blahi 10y agoQuik self-service OLAP. Blazing fast slice & dice and multidimensional expressions (cube formulas). I would need a corporate BI solution to get that and it won't be self-service. Nothing beats Excel + Power BI in reporting.
- dandermotj 10y agoThe "big data vs. small data" argument isn't the issue for data scientists. It's the total lack of decent statistical libraries, visualisation tools, programming capabilities and on and on. Don't think anything I've worked on would come close to hitting the excel memory ceiling, but without a doubt completing it in excel would be nigh on impossible.
- PeterisP 10y agoA simple excel pivotchart on 10k-100k financial entries often works better than the best BI / MIS packages. For most companies, all their data science fits in excel easily. Full transaction history with each individual item sold since founding the company? No problem. Every single visitor/page view on their website? I've seen such logs imported to excel to do some rough aggregation. Manufacturing data about each particular widget sub-part that ever went off your conveyor belt? Again, for many companies with hundreds of employees that would still fit in excel without issues. There are so many companies who speak about big data while their largest datasets can fit into RAM of a cheap laptop. That doesn't mean that data science and analysis is worthless to them, quite contrary.