14 ms·
To people asking, "why do they use Excel?" that's like asking "why must we be subjected to gravity?" The whole world's data ultimately comes from or ends up in
by jordigh 6y ago
To people asking, "why do they use Excel?" that's like asking "why must we be subjected to gravity?"
The whole world's data ultimately comes from or ends up in an Excel spreadsheet. Sure, we might use other intermediate data storage methods, but in the end it's going to go into some scientist's or some politician's computer, and by golly it's gonna be in Excel. Trying to rally against Excel is like trying to rally against fundamental forces of nature.
This is just an example of that fundamental force winning.
- pratio 6y agoTrue in every sense. We shifted from google sheets to O365, there's just no way to go around excel. Its efficient and every one from a novice to expert can find their way around it.
- pmarreck 6y agoPray tell, why did you switch from Sheets?
- pratio 6y agoThe dataset was growing and sheets were slowing down. Excel installed locally on a machine with good hardware is a pleasure to work with. As the team grew more and more people were asking for office licenses and like i said, we have colleagues who've been using excel for ages and they like to have 2-3 files open on multiple screens and work between files using references. It's just not as fast as using excel locally. I'd like to add that we also had an issue with our clients using only excel and there were issues with calculations not working, not so often but with enough people making noise our management decided to get the subscription. I've been using sheets for years even as a student when i couldn't afford an office license.
- azalemeth 6y ago> " To people calling, "why do they use Excel?" that's like asking "why must we be subjected to gravity?"" I respectfully disagree with this. Excel is fundamentally not suited to analysing *omics data. It's often the default program affiliated with a .csv filetype on people's computers, but trying to get an entire field of scientific research to rewrite itself based on its glorified bugs is...wrong, in my opinion. If you see wrong things in the world, do you accept them as they are, or try -- however ineffectually -- to force change for the better? I for one bang the drum into the wind and try to get biochemists off it. I teach people to be very sceptical of excel in my stats courses, for example (aside from some showstopping bugs and downright dangerous defaults, its RNG is particularly crap).
- jordigh 6y agoExcel is not fundamentally suited, but it's fundamentally used. I wish you luck, but I long ago declared defeat against Excel. I can't prevent everyone from using it. I have come to accept that whatever Excel does is what I have to live with, because someone, somewhere out there hasn't heard the anti-Excel gospel yet.
- ppod 6y agoQuantitative social science (which maybe a lot of comp bio people might regard as a "softer" science) has really made huge strides in moving this kind of analysis to R. Of course one person can't change it, but those with high status in the field can make a difference by leading the way and setting standards.
- deleted 6y ago[deleted]
- take_a_breath 6y agoAt some point, you generate a dataframe with the data in R, save it as a csv, email it to someone, and they open it in.....Excel.
- ianai 6y agoThis is the exact sort of thing that indicates market-power being concentrated to a point requiring severe, acute regulatory action.
- arthurcolle 6y agoNot at all There is tremendous competition in the spreadsheet business, and anyone is free to use Julia, R, whatever they like.
- mcv 6y agoBut they don't, and that's a problem. It's an education problem: everybody automatically defaults to Excel, even when it's not the proper tool. People need to learn about better tools.
- meow1032 6y agoI don't disagree completely with this, but just want to point out that it's kind of a bad smell to have computational biologists who are - as someone in the article puts it - computationally illiterate. I have met lots of these types over the years, and usually their methods are kind of a gong show. If you can't properly sanitize your data inputs on your column headers, why should I trust that you've treated the rest of your data properly?
- kang 6y agohttps://news.ycombinator.com/item?id=24070913 https://news.ycombinator.com/item?id=24070913
- acidburnNSA 6y agoI have a strong feeling that, if people really put an effort into reading and replicating more papers, we would find that a lot of what's being published is simply meaningless. In grad school I had a subletting roommate for a while who was writing code to match some experimental data with a model. He showed me his model. It was quite literally making random combinations of various trigonometric functions, absolute value, logarithms, polynomials, exponents, etc. into equations that were like a whole page long and just wiggling them around. He was convinced that he was on a path to a revolution in understanding the functional form of his (biological) data, and I believe his research PI was onboard. I guess "overfitted" never made it into the curriculum.
- gameswithgo 6y agoisn’t the saying, 80% of everything is garbage?
- bryanrasmussen 6y agoSturgeon's Law https://en.wikipedia.org/wiki/Sturgeon%27s_law https://en.wikipedia.org/wiki/Sturgeon%27s_law "ninety percent of everything is crap."
- cproctor 6y ago
- agumonkey 6y agoAnd other fields are probably stuck with bad word processing papercuts.
- wittyreference 6y agoThis accords with my experience. When I worked in clinical trial design/analysis, no one did their analyses in excel, but for initial data formatting and clean-up? /everything/ went through excel, even if the final cleanup was a python script. You’re not going to have the same ease of eyeballing your data in SPSS or SAS or R.
- ekianjo 6y agoI'd wage for formatting and clean-up, it's a lot faster to proceed via R or Python (notebooks), and it can be reproduced a lot more easily.
- t-c-h 6y agoI understand that Excel allows for a quick glance, but I'm not sure how R does not offer the same visual cues? Assuming that the data originates from some csv/table structure, Excel requires some GUI clicking and R needs a read.csv() call. I would say R is a lot more useful for a quick check on whether all columns have one data structure, contain NaN/NA's, are of equal length, etc.
- donatj 6y agoSo much this. I work in educational software, and have a little write up about the troubles we’ve had with schools “fixing up” CSVs from their student information systems before sending them to us. The simple act of opening the CSV and hitting save loses data (Microsoft actually fixed some of the problem very recently). That said, they will use Excel, it’s not in my control. I received dozens of comments “Just use Open Office / Libre Office”. I am but a man, I cannot change the world. I forgive that which I cannot control. We work with tens of thousands of schools. Getting all of their administrators to install Open Office, and thousands of administrative assistants to remember to use it rather than Excel is simply an impossibility. - https://donatstudios.com/CSV-An-Encoding-Nightmare https://donatstudios.com/CSV-An-Encoding-Nightmare
- JoBrad 6y agoI’ve shifted to trying to get people to use the PowerBI data import function, at least. It offers the ability to import your data without managing it.
- BurningFrog 6y agoIf the gravity metaphor doesn't reach you, maybe this will: The world speaks English. It's an illogical and absurdly spelled language. Esperanto is miles ahead on all logical measures. Yet we will never switch, both because the cost of switching is enormous, and because there is no way to coordinate/organize such a move.
- ethanbond 6y agoAlso... Excel is great. What exactly is "ahead" of Excel? People on here really underestimate how great of a product Excel is because of its cultural affiliations.
- t-c-h 6y agoExcel is great, and generally really useful. But not so much in microbiology. Sure, if you have several dozen genes/proteins with some quantitative values and some conditional formatting to visually distinct certain properties, it's fine. But in microbiology you usually works with very large datasets that undergo a lot of calculations through a lengthy pipeline. Larger sets and more intense ML approaches even need HPC's. Inserting Excel into this pipeline would be catastrophic.
- ethanbond 6y agoOh yeah, I'm totally aware. My day job is designing and building software to construct/maintain exactly those pipelines. What I'm saying (which is not news to you) is that clearly Excel is excelling along some dimension that people really care about. Even if it is lacking along other dimensions, the goal ought to be figuring out what makes Excel so good, then figuring out how to reconcile that with the needs of e.g. a high scale genomics pipeline. But to make even an inch of progress against that, we need to get rid of this "haha Excel is for plebes" attitude that many in the software community adopt.
- jpindar 6y agoExcel excells at being already installed on most computers. That's it.
- gsich 6y agoExcel is quick and easy. From startup to import to calculation. You haven't imported a single line with Python or R.
- TallGuyShort 6y agoAlso, on the list of dumb things that make me roll my eyes at Microsoft, interpreting the string "MARCH1" as a date and trying to normalize date formats isn't exactly the dumbest.
- rorykoehler 6y agoExcel should belong to the commons at this stage
- rat_1234 6y agoThis calls to mind a quote from Cormac McCarthy's Blood Meridian: It makes no difference what men think of war, said the judge. War endures. As well ask men what they think of stone. War was always here. Before man was, war waited for him. The ultimate trade awaiting its ultimate practitioner. That is the way it was and will be. That way and not some other way. Just swap our war for Excel! "It makes no difference what men think of Excel, said the judge. Excel endures. As well ask men what they think of stone."
- throwawaylabor 6y agoThe hyperbole here is amazing. You're saying that a 40 year old computer program that runs exclusively on windows is as un-changable as a _law of nature_. The english comparison made in a peer comment is more apt, but wow. The tech industry is so self-centered we think that Scientists should change their fields because of our bugs.
- jordigh 6y agoExcel is older than many of us, so yeah, having it around for our whole lives does really make it seem like a law of nature. Sooner or later, all data ends up in Excel. I don't like it at all, but that's what happens. As for this being an Excel bug: many, perhaps most people don't think it is. That's why it's so permanent: almost everyone likes it.
- lifeisstillgood 6y agoNah, I plan on having HDF5 or Apache Arrow be the portable table of data and then build WASM based viewers - it's the excel killer but the war of attrition will be long !
- pmarreck 6y agoOK then. Use Excel. But at least know the basics of Excel. Such as simply right clicking the column header and picking a different format so that it will stop auto-interpreting the input.
- bromuro 6y ago> to people asking, "why do they use Excel?" I often read comments on HN starting with “to people asking” or “to the people wondering”... and introducing a new discussion, usually on another similar topic, basically hijacking the whole discussion.
- importantbrian 6y agoWhenever people ask me about getting into the field and if they should learn Python or R or Scala or whatever else, I always respond with you need to get really good with excel/Google Sheets and SQL. Then worry about everything else. I can't tell you how many times I've done an analysis I'm really proud of in something like R Markdown only to have the end user say that's great now can I get it in an excel? I just deliver everything in excel as a default now.
- goatlover 6y ago> good with excel/Google Sheets and SQL Those aren't very good for cleaning and wrangling data, even if the output for analysis ends up being a spreadsheet.
- importantbrian 6y agoAgreed that spreadsheets aren't great for that, but totally disagree about SQL. SQL is the best cleaning and wrangling data tool out there. There's a reason the trend in the industry is towards tools like DBT. There are certainly things for which you need to use Python or R, but for 95% of tasks some combination of SQL and a spreadsheet will get you where you need to go.
- guenthert 6y agoAh, just make the data file larger. No one is going to use a spreadsheet tool when there are millions of rows (I hope).
- deleted 6y ago[deleted]
- anameforauser 6y ago“Newton had a great time for a long time with his description [of gravity], and then at some point it was clear that that description was fraying at the edges, and then Einstein offered a more complete version,” As for flat spreadsheets, they serve their purpose just like any other tool. I don’t think I heard people complain about spreadsheets for dealing with flat data, the same way no one complains about hammers to deal with nails. If we’re trying to deal with interconnected and enterprise resources we don’t maintain those resources in Excel. We store them in Enterprise Resource Planning software or build relational databases (which are as old as gravity). Nothing is wrong with spreadsheets, they’re the sweetest way to deal with quick data until we use them for things they can’t do.