3 ms·
Indeed, the best support for large graphs in Python is found inside scipy.sparse
by enriquto 6y ago
Indeed, the best support for large graphs in Python is found inside scipy.sparse
- jim22k 6y agoAnyone who is serious about graph analysis in Python using linear algebra should check out GraphBLAS (graphblas.org). The inclusion of semirings makes things like SSSP and BFS extremely elegant on sparse adjacency matrices. The main implementation is SuiteSparse::GraphBLAS, a C library which has two Python bindings (search for grblas or pygraphblas). Disclosure: I'm the author of grblas. Both are available on conda-forge for easy installation. If you want to try it out without installing, here is a binder link: https://mybinder.org/v2/gh/metagraph-dev/grblas/HEAD?filepath=notebooks%2FIntro%20to%20GraphBLAS%20%2B%20SSSP%20example.ipynb https://mybinder.org/v2/gh/metagraph-dev/grblas/HEAD?filepat...
- VHRanger 6y agoThanks! What's the underlying data structure of the graph?
- lmeyerov 6y agoI've found sparse good for many-columns (10K+). However, for graphistry's graph workloads, we do most scipy-size graphs at much higher speeds via cudf + cugraph. Similar apis, more speed. Ex: Preprocess many cols with scikit sparse (genomes, many-featured fraud, ...) -> cudf (= gpu dataframes) for most of workload -> infer relationships with gpu UMAP then gpu k-nn -> graphistry for visual analysis. We're starting to experiment with bigger-than-memory (TB etc) scale via the gpu-enabled dask libs. I definitely recommend trying. If interested in adding visual analytics here, these'll be hitting our free + self-hosted layers soon, and drop a contact method if you'd like early access. Exciting times!