12 ms·
Human genes renamed to stop Microsoft Excel from misreading them as dates (2020)
- loufe 5y agoGood initiative, but there are hundreds of used languages, I hope they put a bit more effort into it than just testing the English dates cases. I caught one bad example, for any French speakers, in the article itself: and MARS tweaked to MARS1.
- joshvm 5y agoPlenty of gene names are partial/full backronyms that don't really bear any relation to their function. For example the R in MARCH stands for "RING Finger", RING means "Really Interesting New Gene" [0]; finger here is some substructure that I guess looks superficially like a finger on the protein. That's an astronomy-worthy acronym if ever there was one. You could probably avoid a lot of this if the naming conventions avoided using protected words, but then they're a lot less catchy and easy to remember. Also I don't think Excel has a list, but it might be useful to make one. [0] https://pubmed.ncbi.nlm.nih.gov/24928685/ https://pubmed.ncbi.nlm.nih.gov/24928685/
- Semaphor 5y agoAt the time (506 comments): https://news.ycombinator.com/item?id=24070385 https://news.ycombinator.com/item?id=24070385
- adamors 5y ago
- mdb31 5y agoPSA: if you use Excel as intended, you would just go File/New/Blank Worksheet, switch to the Data tab, then use the 'From Text/CSV' wizard to specify the data types for each column in your data source, neatly bypassing this entire issue. But yes, just double-clicking the CSV from Explorer, using the legacy 'open this as a sheet' functionality, experiencing data loss and then complaining about it (and the state of Excel, MSFT and The World in general) on social media is much, much more fun...
- hoseja 5y agoThe easy action should be safe by default.
- mdb31 5y agoYes, but without inventing a time machine to go back to 1985 or so (at which time the safe option did not exist, so better invent temporal code backporting as well), that's not a viable option. Any change to Excel's 'open a worksheet' logic would break so many workflows it's just not funny anymore. I'm not kidding if I say I suspect it would significantly impact several countries' GDP for a while. Even the (sometimes comically inadequate) heuristics that Excel uses to auto-determine field types can't be updated, for very similar reasons. Backwards-compatibility is... interesting...
- Linosaurus 5y ago> Any change to Excel's 'open a worksheet' logic would break so many workflows it's just not funny anymore. My preferred work flow would be to open the csv, change some formatting, then reinterpret the original data. It would require keeping two sets of data for each cell, but that seems to already be the case since F2-enter on each cell would accomplish exactly that. Last I tried.
- Ekaros 5y agoEasy action should be what majority expects. I wonder what is the ratio of people dealing with dates versus genes is... Probably very substantial on favour to those who deal with dates. So things just to work for them is likely better option.
- tidenly 5y agoIn Japan everyone passes around CSVs in shift-jis still for exactly this reason - the default opening action of Excel with CSVs. Its easy to say its people being dumb, but at this point I really wish excel just wasn't so confident in itself and actually asked you during the default open operation what you want to do.
- deleted 5y ago[deleted]
- jmchuster 5y agoSimilarly, if your last name is "Null", sometimes it's just easier to just change your last name.
- hamasho 5y agoFor reference [0]. It should be tough to have such a name. The most important systems like tax/insurance/airline booking are often the most unkind systems. If you're not familiar with computers, it's almost impossible to imagine the potential cause of problems is your name. [0] https://www.bbc.com/future/article/20160325-the-names-that-break-computer-systems https://www.bbc.com/future/article/20160325-the-names-that-b... > These unlucky people have names that break computers
- jmchuster 5y agoI had a PM who would get deleted out of the company directory every 6 months. She resolved the issue by getting married.
- jdrc 5y agoTo Nan
- hpcjoe 5y agoThis has got a "Bobby Tables"[1] vibe to it. While I personally do not like the concept of adjusting data to fit a tool, as others have pointed out, Excel, for better or worse, is ubiquitous. To the obvious detriment of genomics. And likely others. Any tool that seeks to replace it would need to be as easy to use, as flexible. And preferably fewer bugs. The problem is that the paradigm for spreadsheets works well for many projects, even if the implementations are crap. [1] https://www.explainxkcd.com/wiki/images/5/5f/exploits_of_a_mom.png https://www.explainxkcd.com/wiki/images/5/5f/exploits_of_a_m...
- hoosieree 5y agoDon't change it to "None", though. That's just a curse for your grandchildren to deal with when Python eventually becomes the new old legacy system. "Nil" is probably safe, because the lisp curse will actually work in your favor.
- imtringued 5y ago>Even then, a scientist might fix their data but export it as a CSV file without saving the formatting. Is this supposed to be a joke? Since when does excel support CSV files? Yes you can import and export CSV files but that is just there to check a box. That feature doesn't actually work. Just import and export .xlsx files in your applications directly. CSV is such a bad format because it's not even a standard, there is RFC4180 but most people have never heard of it. CSV is complicated enough that anyone who thinks they can implement it will get it wrong on their first attempt but simple enough that people believe they can implement it on their first attempt.
- rini17 5y agoOpening CSVs in Excel almost never worked for me. Good thing there's LibreOffice Calc with spiffy import dialog with all the csv options/variants where you can preview the result.
- joenathanone 5y agoExcel has exactly the same functionality.
- rini17 5y agoGood, since when? Last used it 10ish years ago.
- joenathanone 5y agoHere is the feature in Excel 2007 https://youtu.be/ML0VDMpWllM https://youtu.be/ML0VDMpWllM around 2:45, could have also been there in earlier versions too.
- prepend 5y agoI think the import wizard was added in office 2000.
- dwringer 5y ago
- nulbyte 5y ago> Microsoft Excel may be fleeting, but human genes will be around for as long as we are. It’s best to give them names that work. This is the crux of the issue. Yes, Excel could do better with support for CSV. (OpenOffice and LibreOffice habdle this better, for example.) But no, genetics is not some niche use-case for this better support. It's just one example of many. I do analytics of another sort for a bank, and we run into this problem all the time with those pesky 16-digit account numbers. But, while Microsoft might do well to make changes to address their poor implementation, someone somewhere will implement the next best thing and screw it up, too.
- nicoburns 5y agoI really wish there was a better interchange format for tabular data. Criteria: - Plain text file - Supports formatting, cell types, etc - Does not support full spreadsheet features like formulas
- olau 5y agoUTF-8 CSV with a few HTML tags for formatting? You could call it CHU. Now you just need to make Excel accept it.
- nicoburns 5y agoI'd like it to have support for things like "this is a <format> cell", where format is something like text, numeric, date, currency. I'm not sure how you'd specify that in HTML.
- ElectricalUnion 5y ago> text, numeric, date I think it's already defined? <input type="text">, <input type="number">, <input type="date"> > currency You probably need to use the microdata format https://schema.org/Offer https://schema.org/Offer
- mcv 5y agoI've been thinking about a project exactly to replace this sort of Excel misuse. The frontend should be about as Excel-like as possible, where you can just easily type (or import) as many rows and columns you like, but without the formulas and other real spreadsheet calculations. Instead, make it easy to add data types, validation for those types, foreign keys, etc, and store the whole thing in a database.
- mcv 5y agoThe real problem is of course using Excel to maintain data. That's not what it's for. That's what databases are for. Excel is for complex calculations over rows and columns. And of course it should be better at data validation; you should be able to tell it whether a certain column is a date or not, and Excel shouldn't be jumping to stupid conclusions about it, but it does, and that means you need to be careful and not use Excel for something like that. Excel misuse is sadly rampant and one of my main frustrations at my previous project. Excel is popular for this sort of misuse because when you open it, it presents you with an empty table and you can immediately start typing. This invites tabular data. But Excel inevitably fucks up because it lacks proper data types, data validation, or foreign keys, and loves making assumptions about what you mean. This makes it a terrible and harmful choice for any sort of serious data. I even proposed a new project for an Excel-like frontend backed by a database exactly for these sort of situations. Because I do get what attracts people to Excel for this sort of thing. And most people don't realise what a terrible choice it is.
- prepend 5y agoThere’s not really a better tool that I’ve found. Excel is ubiquitous and pretty easy to use. And there aren’t good “data browsers” that have as low a learning curve. I’ve been especially looking for a JSON browser/editor since excel doesn’t do that well and I’m unsuccessful trying to explain how to use basic text editors to people who can’t do basic functions like open files that aren’t associated with a program.
- tanbog10 5y ago"And there aren’t good “data browsers” that have as low a learning curve." If someone is trained to the point of working on genetic data at this level, should they not also have been trained to a reasonable level in domain appropriate software and tools?
- overthemoon 5y agoI'm continually shocked at how technically incompetent academics can be. Otherwise brilliant people with the most batshit technical workflows I've ever seen.
- wil421 5y agoI love it when excel decides a bunch of digits is a date or decides to remove leading 0000s from a very important number that needs 0000s. Or maybe it wants to make a bunch of digits into scientific notation. I would prefer if excel left my cells alone or just asked what I wanted to do.
- chipsa 5y agoPart of the problem, highlighted in the article: Users don't set the data type of the column off of "General". So it tries to figure out the data type on entry. Setting the column to use a specific type ("Text"), keeps it from trying to use heuristics to figure out that something is a date. Something more recent is the introduction into Excel of Power Query, which lets you import a CSV and apply arbitrary transformations (such as applying a type), before it hits the workbook, so if you need to pull in a CSV, you can do so, and it will always be imported the same way.
- deleted 5y ago[deleted]
- washadjeffmad 5y agoDoes anyone here use Excel in any capacity on production data in the biosciences? If so, did you go to college for the biosciences, or are you supplying work using a tool you learned under another discipline? I'm always fascinated to learn how Excel makes its way into unusual places.
- lofatdairy 5y agoNot in biotech, but have worked with people who use a tremendous amount of excel, especially in biology. A lot of the people still using Excel either came before bioinformatics libraries got really really good, had to basically teach themselves how to use a computer to deal with bio data, worked in a lab that had protocols in place for data that used Excel, and/or haven't gotten around to teaching themselves R. You have to keep in mind that Excel is extremely good at being a visual representation of data. R, Python (only really adopted in recent years), and Matlab (which I don't see too often in bio) are all excellent for manipulating the same data as Excel, but if you wanted to quickly scroll through your RNA sequencing counts, Excel provides a fast, (almost) out-of-the box way to get a "big picture" view of your data (I haven't met anyone doing bioinformatics in SAS or STATA, but I know people who have used those software in academia for other fields). Even if no one is directly manipulating the data in Excel, if you don't import the data correctly, or forget to _not_ save the data, you'll end up fucking it with Excel's auto-formatting. These subtleties lead to things as mentioned in the article, but aren't things that the ordinary person learns except through mistakes. Nobody is born knowing what tools to use or how to use them, but Excel is one of the first pieces of software that deal with data for a lot of people, and as such one of the first things they turn to when faced with a new problem.
- pelasaco 5y agoLooks more like the scientists are choosing the wrong tool to do their job. Sure you can use excel, to get a sense of the data, but for god sake, there are better tools for that - or even contribute to an existing open source one to do what you need. I'm quite sure scientists could get some grants to hire actual software engineers to come up with a better solution, based on the open source options existing there.
- hpcjoe 5y ago> I'm quite sure scientists could get some grants to hire actual software engineers to come up with a better solution Uh ... no. Grants rarely ever support SWE outside of a core application. No granting agency would support writing a new open source tool that effectively replicates what is available in market today. You are right in that genomic scientists chose poorly here. Excel isn't the right tool for the job, but there are very few options that could work, and the others required some assembly ... which they couldn't get money to fund. And the other potential solutions were not ubiquitous. For them, renaming genes is the easier solution than switching workflows to new (to be assembled) tooling. Remember, scientists are people too ... they will opt to take paths of lower resistance even when they are suboptimal.
- pelasaco 5y agoThe error is to assume that we have a tool available to do it, aka Excel. If they assume that there is no tool available, then a tool have to be found/developed.
- AviationAtom 5y agoIsn't this solved by putting a single quote, or something of the sort, in front of the data?
- teh_klev 5y agoYes, but if you're opening a CSV file, some folks don't "import" the CSV which would give you a chance to specify the data types for each column: https://news.ycombinator.com/item?id=30986868 https://news.ycombinator.com/item?id=30986868
- rngcntr 5y agoWho said Bill Gates wants to manipulate our genes? ;)
- adhesive_wombat 5y ago> Be conservative in what you send, be liberal in what you accept. This is why I think that's wrong: if you allow people to be sloppy with how they do things, they'll do it, and then make it part of their workflow, product, religion or whatever, and now everyone is stuck with it. Be absolutely explicit with what you accept and refuse to deal with crap. Then you will only ever have to maintain a simple validator and the code that deals with good data, rather than having to have an incredibly hairly validator that leaks into your logic at every level, followed by cementing your bugs into everyone's implementations.
- cjsawyer 5y agoThen your competitor accepts messy data and you go out of business.
- adhesive_wombat 5y agoThen file a bunch of bugs on their product relating to messy data in various pathological states and tie up their engineering for months. Game on!
- burntoutfire 5y agoReality adapting to software (instead of the other way around) seems to be more and more common. Just today, my local library sent out email informing users that, while they agree that they shouldn't charge late fees for non-work days, the software company they use doesn't want to introduce such feature into the software, so they have no way but to charge for those days.
- max599 5y agoExcel allows you to force a specific type during file import or directly on the worksheet with literally two clicks once the region is selected. You can also create templates if you want all your new documents to use a specific type. The scientists are not adapting to the software, they are adapting to the incompetence of people in their research groups who refuse to learn how to use their main work tool and/or don't want to do the extra 2 clicks it takes to select "text".
- hoosieree 5y ago"4 base pairs ought to be enough for anybody" - Bill Gates
- keithalewis 5y agoSuggested title: "Human genes renamed to stop people who don't know how to use Microsoft Excel from misreading them as dates."
- Shadonototra 5y agoModern stupidity at its finest, a point of no return was achieved
- Schroedingersat 5y agoJust don't use excel. Or anything with autocorrect and autoformat.
- nikonyrh 5y agoI work in the retail sector, GTINs / EANs in the exponential form aren't ideal but sadly fairly common.