3 ms·
How fast is data import? Loading into RAM? (for example booting up a cluster for an existing imported database on AWS) Working with some datasets with 100s of
by paulsutter 9y ago
How fast is data import? Loading into RAM? (for example booting up a cluster for an existing imported database on AWS)
Working with some datasets with 100s of billions of short rows, curious to give it a try.
- nikita 9y agoWe have seen 1Gb/s streaming ingest into MemSQL. Also feel free to join https://memsql-public.slack.com https://memsql-public.slack.com where people help with developer edition.
- paulsutter 9y agoWith how many nodes? Does it scale linearly with nodes? (1Gb/s by itself doesn't help me estimate the scale of the project), Thanks!
- nikita 9y agoData ingest scales linearly assuming the source is scalable (S3, HDFS, or Kafka). There are other things that matter: how wide is the table, what data types, etc. We achieve 1GB/s on a 16 nodes cluster for some combination of the above. What is your target?