6 ms·

Maybe my bug? I wrote most of the original Excel text-based file import code in 1984 (including the SLYK format, which was the main way we migrated files from

by mckoss 10y ago

Maybe my bug? I wrote most of the original Excel text-based file import code in 1984 (including the SLYK format, which was the main way we migrated files from Multiplan to Excel).

magoon 10y ago
32 years ago! What are the chances your code has survived that long in a product like Excel?
- eric_h 10y ago
  Admittedly this is for Excel 2003, so only ~20 years. I think there's a good chance that this bug was theirs ;)
  - puetzk 10y ago
    The bug is still there, and my software still has to work around it in my CSV emitter (force redundant quoting so the first byte will be '"' instead). Boo, I say!
pbreit 10y ago
Spreadsheets still suck at leaving my data alone. At least Excel has that ridiculous import facility where you can instruct it to leave certain columns alone. Google Sheets are surprisingly the worst in this department. It's pretty much impossible to prevent Google Sheets to convert 5/7 to a date and 0123 to a number (losing the leading 0 of course and rendering the data invalid). No, ' is not the answer.
boterock 10y ago
I'd like to know if you (or someone) knows who had that stupid idea of localizing csv files (for those who don't know, excel in spanish uses semicolon for separating values, because commas are reserved for decimals, i'd prefer some standard behavior rather than localized unstandard behavior)
newscracker 10y ago
A tangent here. Imagine if this had been an open source effort with good version control. You (or someone else) could've gone to that time and seen all the related changes, possibly some context and other aspects about this bug. :)
- btbuildem 10y ago
  Ha.. this one is from a land before time..
- dredmorbius 10y ago
  For 1986, that's fairly unlikely. At G+, Noah Friedman, who's part of the team who worked on the code, has inquired occasionally about the availability of some early Emacs code. Pre-1990, if I recall. I'm not sure he ever turned that up even as a standalone tarball, let alone from a revision control repository.
  - fanf2 10y ago
    There is BSD revision control history going back to the early 1980s, e.g. http://svnweb.freebsd.org/csrg/sys/sys/errno.h?view=log http://svnweb.freebsd.org/csrg/sys/sys/errno.h?view=log
    - dredmorbius 10y ago
      Impressive, thanks.
rjbwork 10y ago
This is one of the great things about HN, and indeed the internet at large. Someone posts a link to a little quirk of our world, and then someone is able to pipe up and say "hey that's me! I made that thing!" Even better when it's something like a software bug.
- batat 10y ago
  Comment proving HN superiority over other pathetic news websites.
  - MOARDONGZPLZ 10y ago
    This happens pretty often on many forums. It's like the Birthday Paradox. You can't expect some creator to chime on on some specifically-selected topic, but relatively often some creator will give their input on some random topic. In the same way, one cannot expect two people in a relatively small group to share birthday X, but can expect two people to share some birthday in the same group with high probability.
    - deleted 10y ago
      [deleted]
- drzaiusapelord 10y ago
  Yeah but for every superstar post like this there are thousands of equally upvoted yet ignorant comments that play on the sympathies/bias of HN readers for upvotes. I wish we had more of the former and less of the latter. I suspect no forum is ever safe from eternal September.
  - dredmorbius 10y ago
    What do you suggest as a means for achieving that end?
    - drzaiusapelord 10y ago
NamTaf 10y ago
This is why I like HN. Opening the comments and seeing "oh maybe this is due to something I did back in the 1980s when I wrote most of that, here's some inside info/explanation" about something as significant as Excel's file handling is something that will never grow old for me.
saynsedit 10y ago
This is why HN is cool. You see a Microsoft knowledgebase article documenting an obsolete enterprise software's bug and its workaround, then an anonymous account takes credit for it. This is what I come here for.
- deleted 10y ago
  [deleted]
hasenj 10y ago
Was it implemented before the generic CSV import was implemented? It seems like if parsing fails it should throw an exception and fallback to regular csv parsing. Am I missing something?
- mark-r 10y ago
  I suspect the parsing for CSV and the parsing for SYLK are completely different pieces of code, and the appropriate one gets called once a determination of the file format is made. This would be in an upper level where you wouldn't want to put format-specific behavior. The problem then is that the code determining the format doesn't dig far enough into the file to make sure it's valid before sending it to the parser.
- ygra 10y ago
  That's probably how it has been fixed. I cannot test right now, but since the article doesn't apply to newer versions of Office, I guess they are no longer susceptible to that.
  - Piskvorrr 10y ago
    Nope - just no longer admitting to it. Had to redo a bunch of exports because of this recently.
  - moobs 10y ago
    This is so funny, I was literally fixing this bug last night. Still applies to office 2013
    - cm2187 10y ago
      Another bug with CSV files is that opening a CSV file triggers a full recalculation, even if the calculations are set to manual. This is clearly a wrong behaviour.
  - deleted 10y ago
    [deleted]
jeffwass 10y ago
Hi mckoss, awesome to hear one of the main Excel architects is here on HN! Any interesting stories to share? I think we'd all be interested. Mind if I ask a question directly related to one of my comments a few days ago? I was lamenting the fact that the ASCII codes 29, 30, 31 (Group, Record, and File separators) never really became widely implemented, as these were specifically designed to delimit data. Ie, one could easily include commas, new lines/carriage returns, etc in data cells without clashing. But instead CSV seems to be the most common standard for tabular data. Were these ASCII codes ever considered for tabular files? Thanks.
- simonh 10y ago
  You couldn't conveniently type those codes on a keyboard and there was no standard way to display them. You'd need a special editor for such files, eliminating the main advantage of an interchangeable text based file format.
  - achr2 10y ago
    Part of your response is accurate as to 'why' dedicated separators never became mainstream - software compatibility - but frankly 99% of csv files were/are never meant to be hand edited, simply used as a neutral file. For the later usage, dedicated separators that do not require escaping or import settings would be major improvement.
    - dpark 10y ago
      For files not intended to be hand-edited, it doesn't really matter what delimiters you use. It's not that difficult to code up char-escaping for csv. At Yahoo, the typical delimiters for logs and whatnot were ctrl+a, ctrl+b, etc. It was slightly nicer than CSV, but only slightly. It was mostly nicer when manually inspecting files with columns that had embedded commas (that otherwise would have been escaped). The machines don't care, and for any interesting processing you'd often end up with escaping anyway.
    - sameerds 10y ago
      But csv files can also be manipulated in shell scripts, with the text being piped from one util to the next. Here it becomes important that the separators can be correctly typed and displayed.
JohnDoe365 10y ago
Kudos for standing up and saying "Hey, may be my fault"
ximeng 10y ago
"SYLK": https://en.wikipedia.org/wiki/SYmbolic_LinK_(SYLK) https://en.wikipedia.org/wiki/SYmbolic_LinK_(SYLK) not SLYK I assume
chris_wot 10y ago
Maybe this was written quite a bit later, but can you explain why the following would allow data to be returned in Excel 97? "If you move your mouse pointer continuously while the data is being returned to Microsoft Excel, the query may not fail. Do not stop moving the mouse until all the data has been returned to Microsoft Excel." https://support.microsoft.com/en-us/kb/168702 https://support.microsoft.com/en-us/kb/168702 I've always wanted to know why Method 2 works!
- pavlov 10y ago
  Some kind of weird event loop issue! Previous HN discussion on this topic goes fairly deep: https://news.ycombinator.com/item?id=7011228 https://news.ycombinator.com/item?id=7011228

by mckoss 10y ago

Maybe my bug? I wrote most of the original Excel text-based file import code in 1984 (including the SLYK format, which was the main way we migrated files from Multiplan to Excel).

magoon 10y ago

32 years ago! What are the chances your code has survived that long in a product like Excel?

eric_h 10y ago
Admittedly this is for Excel 2003, so only ~20 years. I think there's a good chance that this bug was theirs ;)
- puetzk 10y ago
  The bug is still there, and my software still has to work around it in my CSV emitter (force redundant quoting so the first byte will be '"' instead). Boo, I say!

pbreit 10y ago

Spreadsheets still suck at leaving my data alone. At least Excel has that ridiculous import facility where you can instruct it to leave certain columns alone. Google Sheets are surprisingly the worst in this department. It's pretty much impossible to prevent Google Sheets to convert 5/7 to a date and 0123 to a number (losing the leading 0 of course and rendering the data invalid). No, ' is not the answer.

boterock 10y ago

I'd like to know if you (or someone) knows who had that stupid idea of localizing csv files (for those who don't know, excel in spanish uses semicolon for separating values, because commas are reserved for decimals, i'd prefer some standard behavior rather than localized unstandard behavior)

newscracker 10y ago

A tangent here. Imagine if this had been an open source effort with good version control. You (or someone else) could've gone to that time and seen all the related changes, possibly some context and other aspects about this bug. :)

btbuildem 10y ago
Ha.. this one is from a land before time..
dredmorbius 10y ago
For 1986, that's fairly unlikely. At G+, Noah Friedman, who's part of the team who worked on the code, has inquired occasionally about the availability of some early Emacs code. Pre-1990, if I recall. I'm not sure he ever turned that up even as a standalone tarball, let alone from a revision control repository.
- fanf2 10y ago
  There is BSD revision control history going back to the early 1980s, e.g. http://svnweb.freebsd.org/csrg/sys/sys/errno.h?view=log http://svnweb.freebsd.org/csrg/sys/sys/errno.h?view=log
  - dredmorbius 10y ago
    Impressive, thanks.

rjbwork 10y ago

This is one of the great things about HN, and indeed the internet at large. Someone posts a link to a little quirk of our world, and then someone is able to pipe up and say "hey that's me! I made that thing!" Even better when it's something like a software bug.

batat 10y ago
Comment proving HN superiority over other pathetic news websites.
- MOARDONGZPLZ 10y ago
  This happens pretty often on many forums. It's like the Birthday Paradox. You can't expect some creator to chime on on some specifically-selected topic, but relatively often some creator will give their input on some random topic. In the same way, one cannot expect two people in a relatively small group to share birthday X, but can expect two people to share some birthday in the same group with high probability.
  - deleted 10y ago
    [deleted]
drzaiusapelord 10y ago
Yeah but for every superstar post like this there are thousands of equally upvoted yet ignorant comments that play on the sympathies/bias of HN readers for upvotes. I wish we had more of the former and less of the latter. I suspect no forum is ever safe from eternal September.
- dredmorbius 10y ago
  What do you suggest as a means for achieving that end?
  - drzaiusapelord 10y ago

NamTaf 10y ago

This is why I like HN. Opening the comments and seeing "oh maybe this is due to something I did back in the 1980s when I wrote most of that, here's some inside info/explanation" about something as significant as Excel's file handling is something that will never grow old for me.

saynsedit 10y ago

This is why HN is cool. You see a Microsoft knowledgebase article documenting an obsolete enterprise software's bug and its workaround, then an anonymous account takes credit for it. This is what I come here for.

deleted 10y ago
[deleted]

hasenj 10y ago

Was it implemented before the generic CSV import was implemented? It seems like if parsing fails it should throw an exception and fallback to regular csv parsing. Am I missing something?

mark-r 10y ago
I suspect the parsing for CSV and the parsing for SYLK are completely different pieces of code, and the appropriate one gets called once a determination of the file format is made. This would be in an upper level where you wouldn't want to put format-specific behavior. The problem then is that the code determining the format doesn't dig far enough into the file to make sure it's valid before sending it to the parser.
ygra 10y ago
That's probably how it has been fixed. I cannot test right now, but since the article doesn't apply to newer versions of Office, I guess they are no longer susceptible to that.
- Piskvorrr 10y ago
  Nope - just no longer admitting to it. Had to redo a bunch of exports because of this recently.
- moobs 10y ago
  This is so funny, I was literally fixing this bug last night. Still applies to office 2013
  - cm2187 10y ago
    Another bug with CSV files is that opening a CSV file triggers a full recalculation, even if the calculations are set to manual. This is clearly a wrong behaviour.
- deleted 10y ago
  [deleted]

jeffwass 10y ago

Hi mckoss, awesome to hear one of the main Excel architects is here on HN! Any interesting stories to share? I think we'd all be interested. Mind if I ask a question directly related to one of my comments a few days ago? I was lamenting the fact that the ASCII codes 29, 30, 31 (Group, Record, and File separators) never really became widely implemented, as these were specifically designed to delimit data. Ie, one could easily include commas, new lines/carriage returns, etc in data cells without clashing. But instead CSV seems to be the most common standard for tabular data. Were these ASCII codes ever considered for tabular files? Thanks.

simonh 10y ago
You couldn't conveniently type those codes on a keyboard and there was no standard way to display them. You'd need a special editor for such files, eliminating the main advantage of an interchangeable text based file format.
- achr2 10y ago
  Part of your response is accurate as to 'why' dedicated separators never became mainstream - software compatibility - but frankly 99% of csv files were/are never meant to be hand edited, simply used as a neutral file. For the later usage, dedicated separators that do not require escaping or import settings would be major improvement.
  - dpark 10y ago
    For files not intended to be hand-edited, it doesn't really matter what delimiters you use. It's not that difficult to code up char-escaping for csv. At Yahoo, the typical delimiters for logs and whatnot were ctrl+a, ctrl+b, etc. It was slightly nicer than CSV, but only slightly. It was mostly nicer when manually inspecting files with columns that had embedded commas (that otherwise would have been escaped). The machines don't care, and for any interesting processing you'd often end up with escaping anyway.
  - sameerds 10y ago
    But csv files can also be manipulated in shell scripts, with the text being piped from one util to the next. Here it becomes important that the separators can be correctly typed and displayed.

JohnDoe365 10y ago

Kudos for standing up and saying "Hey, may be my fault"

ximeng 10y ago

"SYLK": https://en.wikipedia.org/wiki/SYmbolic_LinK_(SYLK) https://en.wikipedia.org/wiki/SYmbolic_LinK_(SYLK) not SLYK I assume

chris_wot 10y ago

Maybe this was written quite a bit later, but can you explain why the following would allow data to be returned in Excel 97? "If you move your mouse pointer continuously while the data is being returned to Microsoft Excel, the query may not fail. Do not stop moving the mouse until all the data has been returned to Microsoft Excel." https://support.microsoft.com/en-us/kb/168702 https://support.microsoft.com/en-us/kb/168702 I've always wanted to know why Method 2 works!

pavlov 10y ago
Some kind of weird event loop issue! Previous HN discussion on this topic goes fairly deep: https://news.ycombinator.com/item?id=7011228 https://news.ycombinator.com/item?id=7011228