found https://utdemir.com/posts/ann-distributed-dataset.html https://utdemir.com/posts/ann-distributed-dataset.html , which makes me think that kind of distribu

found https://utdemir.com/posts/ann-distributed-dataset.html https://utdemir.com/posts/ann-distributed-dataset.html , which makes me think that kind of distributed approach could be workable for channable (the default existing backend spins up lambda workers and stores intermediate results in S3), but you'd need to write a different backend that farms out work to whatever worker machines you have available