3 ms·
Do you handle the cases where people nest delimited values inside cells? Like if a cell is itself a delimited set of codes? What about when two cells in a row
by BugsJustFindMe 5y ago
Do you handle the cases where people nest delimited values inside cells? Like if a cell is itself a delimited set of codes? What about when two cells in a row do that together and are assumed to be related? Like if a cell contains a delimited set of codes and another cell in the row contains a delimited set of descriptions for those codes? What about when people transpose or unmelt their data? Like if all the categories are down the side instead of across the top and identifiers are across the top and then they have a bunch of cells in the middle that are all pluses and minuses and blanks? Or if some arbitrary column itself contains novel header values and their existence implies a yes/positive value? (btw this is not finger wagging. please work on these!)
People always think that the hard part of automated data wrangling is things like delimiters and headers, but the real hard part of automated data wrangling is when people who aren't data wranglers themselves are making the data or when the rules or apparatus for filling cells are even the slightest bit flexible because people will always find a way to mess with you.