3 ms·
I have written a new database system that will convert CSV, JSON, and XML files into relational tables. On of the biggest challenges to CSV files is the lack o
by didgetmaster 2y ago
I have written a new database system that will convert CSV, JSON, and XML files into relational tables.
On of the biggest challenges to CSV files is the lack of data types on the header line that could help determine the schema for the table.
For example a file containing customer data might have a column for a Zip Code. Do you make the column type a number or a string? The first thousand rows might have just 5 digit numbers (e.g. 90210) but suddenly get to rows with the expanded format (e.g. 12345-1234) which can't be stored in an integer column.
- fsckboy 2y agocsv does not stop you from making the first line be column headers, with implied data types, you just have to comma separate them!
- didgetmaster 2y agoI realize that. But when reading a header you have to imply the data types which might be wrong. I always thought it would have been great if the first line read something like: name:STRING,address:STRING,zip code:INTEGER,ID:BIG_INT,...