akarve

11 ms·

31.
by akarve 7y ago
Not yet. But it's closer than one might think. Spaces has an S3-compatible API, and we have plans to use something like min.io to make Quilt work "all the blobs": GCP, Azure, Digital Ocean, etc. OSS contributions welcome :) W
32.
by akarve 7y ago
Can do. A lot of the magic happens in the es/indexer and search lambdas here: https://github.com/quiltdata/quilt/tree/master/lambdas . The short of what we do: we listen for bucket notifications in L
33.
by akarve 7y ago
Thanks. We were careful to publish as `quilt3` on PyPI, so there are no naming conflicts with the `quilt` patch manager. We are also "Quilt Data, Inc." officially. Our thesis is that blob storage is already "git for data&quot
34.
by akarve 7y ago
Just determining the shape of "git for data" has been a nontrivial exercise. We found that, if you do the naive translation, you get a "one size fits none," because data and code are fundamentally different. What would b
35.
Launch HN: Quilt (YC W16) – A versioned data portal for S3
177 points by akarve 7y ago|64 comments
36.
Quilt Data (YC W16) is hiring a cloud engineer(quilt-data.breezy.hr)
1 points by akarve 7y ago
37.
by akarve 8y ago
Quilt isn't doing the inference (the PyTorch model is). But, in any case, no. Super-resolution is more than blurring, it's pixel inference. https://arxiv.org/abs/1609.05158
38.
by akarve 8y ago
i hear you. on pypi the name is uncontested so, at least in the python eco-system, there is only one quilt. that said, for future revisions we'll try for a unique name because it can indeed be confusing, e.g. in the apt-get case.
39.
by akarve 8y ago
Interesting thoughts. Quilt has a ways to grow. You correctly point out that, in some cases, S3 is lighter weight. You'll see future versions of Quilt get lighter, and offer more S3-like "just store this" functionality. In it
40.
Reproducible machine learning with PyTorch and Quilt(blog.paperspace.com)
135 points by akarve 8y ago|26 comments
41.
by akarve 8y ago
Ironically, the same challenges are better solved in the world of code: Docker, GitHub, npm, etc. Some friends and I created Quilt to bring versioning and packaging to data: https://quiltdata.com/ . The interface is the fami
42.
Declarative data processing with pandas and pyarrow(blog.quiltdata.com)
2 points by akarve 9y ago|0 comments
43.
Quilt (Docker for data) is hiring a Tech Lead(quilt-data.breezy.hr)
1 points by akarve 9y ago
44.
Reproducible machine learning with Quilt and Jupyter(blog.dominodatalab.com)
4 points by akarve 9y ago|0 comments
45.
Quilt (Docker for Data) is hiring a Developer Advocate in data engineering
1 points by akarve 9y ago
46.
Quilt – Docker for Data (YC S17)
1 points by akarve 9y ago
47.
Quilt (YC S17) Is Hiring a Developer Advocate in Data Engineering(quilt-data.breezy.hr)
1 points by akarve 9y ago
48.
Quilt Data Is Hiring a Developer Advocate (YC S17)(quilt-data.breezy.hr)
1 points by akarve 9y ago
49.
by akarve 9y ago
Hi. Key differences from data dot world: https://news.ycombinator.com/item?id=14792143
50.
by akarve 9y ago
Yeah. Important differences: Quilt offers unlimited public storage, users do not need to log in to use public data, offers on-premise solution, handles serialization, works offline with local builds, etc. You'll see more differentiatio
51.
by akarve 9y ago
Yes. In progress and the first on-prem installs are up and running. Happy to chat further: aneesh at quilt data dot io.
52.
by akarve 9y ago
Stay tuned :)
53.
by akarve 9y ago
Where does it break down? Quilt can package and version directly from in-memory objects so if you are working in Python (more languages planned) you can package as you go and include any dependencies?
54.
by akarve 9y ago
The login allows you to push packages. If you only want to consume public packages: no login required.
55.
by akarve 9y ago
We need to clarify the EULA and terms on the website. The data belongs to the users and we want to keep it that way. Quilt is not "hosted-only". The whole point of the on-prem install is that customers can run Quilt on their own i
56.
by akarve 9y ago
I'd like to chat more about this. Quilt uses S3 and I think we could make the process of getting data into Databricks much simpler. Drop me a line if you'd like to discuss aneesh at quiltdata dot io.
57.
by akarve 9y ago
Namespaces are free. Public repos are free for unlimited data. Should we run a charity and also make private date free? :) We understand that cleaning/filtering/preparation are key to the analysis, and we plan to support those ope
58.
by akarve 9y ago
Frictonless is interesting but 1) doesn't handle serialization (which is essential for performance); 2) requires users to hand annotate schemas (we think schemas should be auto-generated whenever possible).
59.
by akarve 9y ago
Got it. If you'd like me to ping you once we have custom build hooks: aneesh at quiltdata dot io. Have you tried Luigi or Bonobo for data cleaning?
60.
by akarve 9y ago
Spot on. I get where you're coming from. The value of data is whatever the buyer and seller agree upon. I was mostly taking about the value/price of the service.

More ›