2 ms·

I'm afraid I don't quite get the data problem. Is there a toy example of the kind of data you are talking about? Not entirely clear on how the indexing would w

I'm afraid I don't quite get the data problem. Is there a toy example of the kind of data you are talking about? Not entirely clear on how the indexing would work to your benefit with parallel arrays. If there is a 1 to 1 mapping between datapoints to items in each || array, the memory consumption is the same as a struct, so I assume thats not it. It almost sounds like you are describing a (possibly non-numeric? non-homogenous?) sparse matrix, which are usually worked with in vastly different formats for construction vs manipulation vs iterating over.

woodman 11y ago
> Is there a toy example of the kind of data you are talking about? Not that I know of, sorry. > Not entirely clear on how the indexing would work to your benefit with parallel arrays. Lets say that we have data that would be represented in an array of structs like so: typedef struct { char* name; int age; } prsnT1; prsnT1* peopleAOS = calloc(100, sizeof(*peopleAOS)); You could also represent it as a struct of arrays: typedef struct { char** name; int* age; } prsnT2; prsnT2* peopleSOA = malloc(sizeof(*peopleSOA)); peopleSOA->name = calloc(100, sizeof(*peopleSOA->name)); peopleSOA->age = calloc(100, sizeof(*peopleSOA->age)); Now why would you want to use SOA over AOS (disregarding the AOS alignment padding overhead)? Locality and compression. When you want to search on age, you don't need to waste cache space stepping over name. Age can also be bin packed pretty tightly, so in reality it wouldn't be an int* - but a variably sized ADT. Ideally name wouldn't be char either, but another ADT. Transparent zlib compression isn't unheard of. > ... which are usually worked with in vastly different formats for construction vs manipulation vs iterating over. Yup, classic abstract data type - doing OO in C the hard way :)

woodman 11y ago

> Is there a toy example of the kind of data you are talking about? Not that I know of, sorry. > Not entirely clear on how the indexing would work to your benefit with parallel arrays. Lets say that we have data that would be represented in an array of structs like so: typedef struct { char* name; int age; } prsnT1; prsnT1* peopleAOS = calloc(100, sizeof(*peopleAOS)); You could also represent it as a struct of arrays: typedef struct { char** name; int* age; } prsnT2; prsnT2* peopleSOA = malloc(sizeof(*peopleSOA)); peopleSOA->name = calloc(100, sizeof(*peopleSOA->name)); peopleSOA->age = calloc(100, sizeof(*peopleSOA->age)); Now why would you want to use SOA over AOS (disregarding the AOS alignment padding overhead)? Locality and compression. When you want to search on age, you don't need to waste cache space stepping over name. Age can also be bin packed pretty tightly, so in reality it wouldn't be an int* - but a variably sized ADT. Ideally name wouldn't be char either, but another ADT. Transparent zlib compression isn't unheard of. > ... which are usually worked with in vastly different formats for construction vs manipulation vs iterating over. Yup, classic abstract data type - doing OO in C the hard way :)