4 ms·
Ask HN: ML – how big is your training data and how do you store/manage it?
Big in terms of disk size.
Manage in terms of update, extract, share, collaborate.
- verdverm 7y agoBig by what measure? Videos, images, and text have very different profiles. What do you mean by manage? Sharing, splitting, cleaning, versioning, auth, providing for distributed training are all options here. Can you please elaborate one your Ask HN
- deleted 7y ago[deleted]
- mrmrcoleman 7y agoThanks, updated.
- PaulHoule 7y agoFor text classification: * unsupervised: 50 million to 1 billion characters * supervised: labels for 2,000-50,000 documents. Data stored in Arangodb, accessed through Python scripts that run in Jupyter notebooks (e.g. using GUI widgets)