3 ms·
I think that is not done because it's hard/unclear how to handle patches that are not the same size. In text it doesn't matter because all tokens are mapped to
by rolisz 4y ago
I think that is not done because it's hard/unclear how to handle patches that are not the same size.
In text it doesn't matter because all tokens are mapped to an integer (eg. the -> 54, hacker-> 1356).
- knodi123 4y ago> I think that is not done because it's hard/unclear how to handle patches that are not the same size. Is that because of all the effort put into running the same algorithm in parallel on a ton of tiles and then piecing together the results?
- rolisz 4y agoI think that is one of the benefits of having an easily parallelizable model.
- jameshart 4y agoThe tiles would all be the same size - they’d just be sampled from the image at different resolutions.
- rolisz 4y agoThen you introduce a resizing component, which resizes differently for different parts of the image. And you still have to decide how to select the tile sizes: if it's heuristics, well, the field of ML has been moving away from that. If it's with another learnable component, maybe it's worth a shot, at the cost of a more complicated model, which probably has two stages, so the inference time is worse (and maybe even train time)