5 ms·

I was a workshop at the google office in Toronto a while back, and people asked when Google sheets would be able to perform big data analysis. The presenter lo

I was a workshop at the google office in Toronto a while back, and people asked when Google sheets would be able to perform big data analysis. The presenter looked confused, and then the group asking the question clarified that their big data workflow involves loading millions and millions of records into excel and they would have trouble migrating to google cloud until google sheets could match excel in performance. Somewhat unrelated story, but it shows how well loved some software is, and how some users will take it to use cases you never imagined.

probablybroken 7y ago
That's really strange... I always found Excel to perform dreadfully on data sets with even just hundreds of thousands of rows - it was always easier to just load the data into a database, or manipulate it with a script.
- juped 7y ago
  That's because you don't know how to use Excel (this isn't a knock on you, no one ever tells people how to use Excel). Excel sheets have a hard row limit of around a million rows, but long before you approach that you will have moved your data into the "Data Model", which is a relational table engine stapled on to Excel, or to some other backend which you will query from Excel (often via the Data Model).
kragen 7y ago
> big data workflow involves loading millions and millions of records Dude, if it fits on your laptop disk, it's not big data. Here you're talking about “big data” that fits in RAM on my cellphone. Somebody needs to get over himself.
- untog 7y ago
  But if Google Sheets can't fit the data they are using, it is by definition "big data", even if it's not Big Data. I think this over-protectiveness of the definition doesn't really serve anyone.
  - kragen 7y ago
    In four lines you manage to be arrogant, insulting, patronizing, and intellectually sloppy (“by definition”?) I'm disappointed. I'd like to invite you to produce a higher level of discourse.
    - untog 7y ago
      ...you can't be serious. This is an incredibly over the top reaction to a very mild reply. Not to mention that telling me you're disappointed in me and inviting me to "produce a higher level of discourse" is far more arrogant and patronising than anything in my original statement. But anyway, this is very obviously off-topic. I don't think it serves the wider community to continue this conversation so I'll leave it here.
      - kragen 7y ago
        Well, I'm certainly guilty of the same offense!
  - sidlls 7y ago
    It serves everyone to have relatively stable meanings for words and phrases. "Relatively" because of course there is natural drift over time. But if today I write "big data" and it means "millions of rows in Excel" to Bob but "hundreds or thousands of terrabytes" to Jane, it's effectively useless to use in communication. Put another way, that Bob thinks he's dealing with "big data" doesn't mean he is in fact.
- sct202 7y ago
  Right, if it's only millions of records, why can't Google sheets accommodate it.
  - kragen 7y ago
    DHTML involves trading 99.9% or so of your computational power for greater convenience of programming.
- spamizbad 7y ago
  There's a company in my building that outfitted their team with $30K+ HP Z series workstations with 256GB of RAM... to run Microsoft Excel 2016.
  - kragen 7y ago
    That's more RAM than my cellphone has, but less disk than my laptop. It's getting close to being “big data.”

probablybroken 7y ago

That's really strange... I always found Excel to perform dreadfully on data sets with even just hundreds of thousands of rows - it was always easier to just load the data into a database, or manipulate it with a script.

juped 7y ago
That's because you don't know how to use Excel (this isn't a knock on you, no one ever tells people how to use Excel). Excel sheets have a hard row limit of around a million rows, but long before you approach that you will have moved your data into the "Data Model", which is a relational table engine stapled on to Excel, or to some other backend which you will query from Excel (often via the Data Model).

kragen 7y ago

> big data workflow involves loading millions and millions of records Dude, if it fits on your laptop disk, it's not big data. Here you're talking about “big data” that fits in RAM on my cellphone. Somebody needs to get over himself.

untog 7y ago
But if Google Sheets can't fit the data they are using, it is by definition "big data", even if it's not Big Data. I think this over-protectiveness of the definition doesn't really serve anyone.
- kragen 7y ago
  In four lines you manage to be arrogant, insulting, patronizing, and intellectually sloppy (“by definition”?) I'm disappointed. I'd like to invite you to produce a higher level of discourse.
  - untog 7y ago
    ...you can't be serious. This is an incredibly over the top reaction to a very mild reply. Not to mention that telling me you're disappointed in me and inviting me to "produce a higher level of discourse" is far more arrogant and patronising than anything in my original statement. But anyway, this is very obviously off-topic. I don't think it serves the wider community to continue this conversation so I'll leave it here.
    - kragen 7y ago
      Well, I'm certainly guilty of the same offense!
- sidlls 7y ago
  It serves everyone to have relatively stable meanings for words and phrases. "Relatively" because of course there is natural drift over time. But if today I write "big data" and it means "millions of rows in Excel" to Bob but "hundreds or thousands of terrabytes" to Jane, it's effectively useless to use in communication. Put another way, that Bob thinks he's dealing with "big data" doesn't mean he is in fact.
sct202 7y ago
Right, if it's only millions of records, why can't Google sheets accommodate it.
- kragen 7y ago
  DHTML involves trading 99.9% or so of your computational power for greater convenience of programming.
spamizbad 7y ago
There's a company in my building that outfitted their team with $30K+ HP Z series workstations with 256GB of RAM... to run Microsoft Excel 2016.
- kragen 7y ago
  That's more RAM than my cellphone has, but less disk than my laptop. It's getting close to being “big data.”