7 ms·

> the nofly csv is almost 80mb in size and contains over 1.56 million rows of data. this HAS to be the real deal (we later get confirmation that it is indeed a

by asn0 4y ago

> the nofly csv is almost 80mb in size and contains over 1.56 million rows of data. this HAS to be the real deal (we later get confirmation that it is indeed a copy of the nofly list from 2019).

netsharc 4y ago
CSV you say? Let's hope Bobby Droptable's cousin Bobby Double Quote Comma-Semicolon don't get in it...
- andreareina 4y ago
  1.5 million rows you say? Better hope nobody opens it in Excel and overwrites it…
  - cube00 4y ago
    Hopefully the first column isn't ID either[1] [1]: https://alunr.com/excel-csv-import-returns-an-sylk-file-format-error/ https://alunr.com/excel-csv-import-returns-an-sylk-file-form...
    - andreareina 4y ago
      OpenOffice can’t print on Tuesdays https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161/comments/28 https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...
      - bongobingo1 4y ago
        Software is hard...
- wrldos 4y ago
  Oh don’t. I spent several hours last week unfucking one of those.
  - tomcam 4y ago
    Do tell! If you’ve recovered ;)
    - wrldos 4y ago
      I have. All I'm saying is this plus someone who left 10 years ago who can't and shouldn't have written a CSV parser using regular expressions. Input row: A,B,Alice "Mallory" Bob,123
  - owl57 4y ago
    Thank you, I'm considering adopting this as job title, like "senior unfucker".
    - TheSpiceIsLife 4y ago
      Intermediate Fucker reporting for duty.
    - wrldos 4y ago
      Mine is principal dung roller.
      - pdpi 4y ago
        I’ve long described myself as half janitor, half firefighter.
        jks 4y ago
        I sometimes fill in my name as [object Object].
        cduzz 4y ago
        Data Plumber. Some planning, some building out new stuff (usually clean work), some repairing active geysers of partially processed data that's getting all over the place fouling up the works.
- dylan604 4y ago
  I just had one today that had the last char of a field as '\', which escaped the closing quote and then munged the column count from there.
  - Dylan16807 4y ago
    Isn't that your parser's problem? I don't think true CSV has any special characters other than comma, quote, and CRLF.
    - fy20 4y ago
      How do I name my child CRLF?
      - dylan604 4y ago
        earn Musk money, you can name your child WTF you want
        asdfghjkuytrf 4y ago
        i think you failed to parse the technical joke
        dylan604 4y ago
        no, i totally got the line ending joke. i'm not an ID10T
    - dylan604 4y ago
      I thought it strange too. I saw what the issue was, and just "fixed" it be correcting the data in the CSV. For the lulz, I guess I could have played with the parser's options on deciding what is needed to be escaped. However, the data would have still been incorrect as the '\' is definitely not part of the desired content, so ultimately it was better to correct the input. i would kind of rather the import die than having the potential foot gun of '\' in a field for later sabotage.

by asn0 4y ago

> the nofly csv is almost 80mb in size and contains over 1.56 million rows of data. this HAS to be the real deal (we later get confirmation that it is indeed a copy of the nofly list from 2019).

netsharc 4y ago

CSV you say? Let's hope Bobby Droptable's cousin Bobby Double Quote Comma-Semicolon don't get in it...

andreareina 4y ago
1.5 million rows you say? Better hope nobody opens it in Excel and overwrites it…
- cube00 4y ago
  Hopefully the first column isn't ID either[1] [1]: https://alunr.com/excel-csv-import-returns-an-sylk-file-format-error/ https://alunr.com/excel-csv-import-returns-an-sylk-file-form...
  - andreareina 4y ago
    OpenOffice can’t print on Tuesdays https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161/comments/28 https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...
    - bongobingo1 4y ago
      Software is hard...
wrldos 4y ago
Oh don’t. I spent several hours last week unfucking one of those.
- tomcam 4y ago
  Do tell! If you’ve recovered ;)
  - wrldos 4y ago
    I have. All I'm saying is this plus someone who left 10 years ago who can't and shouldn't have written a CSV parser using regular expressions. Input row: A,B,Alice "Mallory" Bob,123
- owl57 4y ago
  Thank you, I'm considering adopting this as job title, like "senior unfucker".
  - TheSpiceIsLife 4y ago
    Intermediate Fucker reporting for duty.
  - wrldos 4y ago
    Mine is principal dung roller.
    - pdpi 4y ago
      I’ve long described myself as half janitor, half firefighter.
      - jks 4y ago
        I sometimes fill in my name as [object Object].
      - cduzz 4y ago
        Data Plumber. Some planning, some building out new stuff (usually clean work), some repairing active geysers of partially processed data that's getting all over the place fouling up the works.
dylan604 4y ago
I just had one today that had the last char of a field as '\', which escaped the closing quote and then munged the column count from there.
- Dylan16807 4y ago
  Isn't that your parser's problem? I don't think true CSV has any special characters other than comma, quote, and CRLF.
  - fy20 4y ago
    How do I name my child CRLF?
    - dylan604 4y ago
      earn Musk money, you can name your child WTF you want
      - asdfghjkuytrf 4y ago
        i think you failed to parse the technical joke
        dylan604 4y ago
        no, i totally got the line ending joke. i'm not an ID10T
  - dylan604 4y ago
    I thought it strange too. I saw what the issue was, and just "fixed" it be correcting the data in the CSV. For the lulz, I guess I could have played with the parser's options on deciding what is needed to be escaped. However, the data would have still been incorrect as the '\' is definitely not part of the desired content, so ultimately it was better to correct the input. i would kind of rather the import die than having the potential foot gun of '\' in a field for later sabotage.