just having a generic tool using SIMD instructions with controllable masking arguments in the pipeline would be awesome. Another example: When ingesting data

just having a generic tool using SIMD instructions with controllable masking arguments in the pipeline would be awesome. Another example: When ingesting data into postgre with copy from exceptions are bad...all entries in the copy from batch are discarded because of one faulting line. So either it have to be flawless or the data needs to also be stored on disk until commit is successfull. Datasets with invalid utf-8 are a pain because of that. One solution is to pipe all input to iconv −f UTF−8 −t UTF−8 -c That will drop invalid characters, but it eats alot of CPU as every character is parsed one by one. With a pipe processing stage using SIMD custom masks as a way to control the pipe flow I could: Select lines containing chars indicating we have a multi byte utf-8 sequence(that can possibly fail) and only pipe those to iconv, it would reduce overhead by 99%.