3 ms·
Looks nice. How well does the distributed filesystem perform and what size data sets an it handle? How quickly can you ramp up 10, 100 or 1000 cores? Improve
by micro_cam 13y ago
Looks nice.
How well does the distributed filesystem perform and what size data sets an it handle?
How quickly can you ramp up 10, 100 or 1000 cores?
Improved performance in these areas are the big things that would get our group to adopt a new platform.
- tristanz 13y agoScaling up from 10, 100, to 1000 cores is fast (3s per engine in parallel). However, something like 1000 cores would currently require spinning up new instances (1-2 minutes) if deployed in the cloud. The distributed filesystem is meant for easily sharing code and medium sized data across containers. In the cloud, it is best to to use S3 directly for large data and local disks for high IO tasks. For on premise deployments, there are more options. I'd be interested to hear about your use case. Feel free to drop me a line at tristan@senseplatform.com