sighingnow

8 ms·

31.
Vineyard 0.2.7: Airflow, Dask, and better ML experience(github.com)
4 points by sighingnow 5y ago|1 comments
32.
by sighingnow 5y ago
We previously have seen many possible scenarios that vineyard could be helpful in https://news.ycombinator.com/item?id=25832831 After several months of development we are glad to announce our improvements that briging viney
33.
by sighingnow 5y ago
The GraphScope[1] team is pleased to announce the v0.6.0 release after 2 months of development work and 98 commits in the main repository. We would like to highlight the following major added features: 1. The implementation of lazy evaluati
34.
Show HN: GraphScope v0.6.0: lazy evaluation and GAIA(graphscope.io)
2 points by sighingnow 5y ago|1 comments
35.
Show HN: Vineyard v0.2.0: big-data applications optimization on Kubernetes(github.com)
1 points by sighingnow 5y ago|1 comments
36.
by sighingnow 5y ago
Vineyard, an open-source in-memory data manager, was previously announced at three months ago[1]. After three-months iteration vineyard release v2.0.2 finally comes out, with a more tight integration with kubernetes and enables orchestratio
37.
by sighingnow 6y ago
We don't have a benchmark between the analytical engine in GraphScope (aka. GAE) with GraphX/Giraph. But we do have evaluated the performance of the underlying engine of GAE (libgrape-lite) with LDBC Graph Analytics Benchmark and
38.
by sighingnow 6y ago
Thanks for you interests on GraphScope! We do have a concrete plan for k8s-less deployment and we already have an issue [1] to track that. That will be available before the end of March 2021. To simplify the environment setup process we wil
39.
by sighingnow 6y ago
The learning engine in GraphScope (aka. GIE) has a simiary programming interface and functionlity with euler. They both support graph neural network training, e.g., GraphSAGE, GCN, GAT, etc. However there are many differences under the hood
40.
by sighingnow 6y ago
GraphScope addresses on computations on extermely large graphs, e.g., the friendship networks of all facebook users, the hyperlink relations between all webpages around the world, and so on. Typically the graph may have billions of nodes an
41.
GraphScope: A One-Stop Large-Scale Graph Computing System(github.com)
66 points by sighingnow 6y ago|25 comments
42.
by sighingnow 6y ago
GraphScope is a unified distributed graph computing platform that provides a one-stop environment for performing diverse graph operations on a cluster of computers through a user-friendly Python interface. GraphScope makes multi-staged proc
43.
by sighingnow 6y ago
Thanks for liking this project! Your understanding is correct. If you want to give vineyard a try, you are welcome to drop me an email (sighingnow@gmail.com), I can help you set it up.
44.
by sighingnow 6y ago
Thanks for your interest on this project. Training multiple models using the same data is truely a typical use case for vineyard. We will release more integeration with machine learning engines (tensorflow and pytorch) in the next major rel
45.
by sighingnow 6y ago
Vineyard addresses on sharing "distributed abstracted immutable data" in memory, e.g., dask[1] generates a distributed dataframe (since the size of data cannot be fit into a single machine), and feed it to tensorflow as features f
46.
by sighingnow 6y ago
> Does distributed mean it’s replicated across machines? No, in vineyard "distributed" doesn't mean data replicated across machines, rather, big data partitioned across machines. We address cases where the data cannot be f
47.
by sighingnow 6y ago
Let's say someone want to use python first to create a few tensors with numpy, and then use another python package networkx to create a graph from the tensors just created and next to do some graph analysis with networkx. It is easy to
48.
Vineyard: An open-source in-memory data manager(github.com)
112 points by sighingnow 6y ago|11 comments
49.
by sighingnow 6y ago
Vineyard is a distributed in-memory data manager. Features: 1. distributed in-memory immutable data storage; 2. zero-copy data access through shared-memory; 3. out-of-the-box high-level data abstractions for commonly used data structures; 4