8 ms·
NetworkX – Network Analysis in Python
- 7thaccount 3y agoVery useful library. Representing large networks with graphs is super useful.
- lemonish97 3y agoused this throughout my ML on graphs grad course this sem, it’s a really neat library
- zubiaur 3y agoLove it. Using it for load balancing of substation transformers. The grid is a graph.
- isotropy 3y agoV. cool! It's off topic, but what do the algorithms look like in that task?
- ok_dad 3y agoInteresting you should say that, as I am trying to start a project where I need to make an electric grid graph, but I am not sure where to find the node/edge data for substations and transmission lines that include their specs and capacities. Is that stuff open source somewhere, like with the ISOs, or do you need to build it from scratch?
- gkoller 3y agoHave a look at IEC CIM (Common Information Model) - https://en.m.wikipedia.org/wiki/Common_Information_Model_(electricity) - https://zepben.bitbucket.io/cim/cim100/ - https://ontology.tno.nl/IEC_CIM/
- ok_dad 3y agoThanks!
- maliker 3y agoThat data is privately held by utilities, and the high voltage transmission infrastructure is highly confidential (CEII/NERC CIP). If you just need sample data, I'd recommend checking out the test data provided with power flow simulators like OpenDSS for distribution systems [1] or MATPOWER for generation + transmission [2]. The IEEE test systems are what are used in research, they have the component specs you're looking for, and are provided with those tools. [1] https://sourceforge.net/projects/electricdss/ https://sourceforge.net/projects/electricdss/ [2] https://matpower.org https://matpower.org
- rwlincoln 3y agoMATPOWER and OpenDSS test cases are also available in CSV format: https://github.com/casecsv https://github.com/casecsv https://github.com/cktcsv https://github.com/cktcsv
- ok_dad 3y agoThanks to you and the next commenter up, these synthetic data sets are perfect for my current use case.
- zubiaur 3y agoTake a look here: https://electricgrids.engr.tamu.edu/electric-grid-test-cases/ https://electricgrids.engr.tamu.edu/electric-grid-test-cases... I’m working on the distribution side, I take my graph data from the inputs for power flow: cyme, synergi etc
- jncfhnb 3y agoI have tried to talk to engineers about contingency analyses for what would happen if a unit went down, and they tend to have very wishy washy answers. Or giant tediously compiled reports that can model exactly one change. Any idea why that is?
- soundarana 3y agoNot in the field, but don't forget the grid is dynamic and has feedback loops, control algorithms and humans in the loop. It's closer to an unstable chaotic system which needs constant balancing and tweaking.
- jncfhnb 3y agoIs that true? Surely things regress to a stable mean most of the time of a few archetypes, no?
- diroussel 3y agoYes. It’s generally stable in most localities. “Instabilities” in this system are browns outs power failures and other events. There are stabilising features within most electricity grids, but they can only cope so much. In general forward planning is down so the amount of dynamic adjustment needed is within allowable range. But to be honest i don’t know how modern grids have adapted with many more micro generators than in the old days.
- scrlk 3y agoGood question. Are you talking about generation, transmission or distribution? From my experience as an electrical engineer working for a distribution network: * The traditional approach to network planning: take your edge cases (e.g. winter peak demand), and apply your engineering knowledge and intuition to manually study the most onerous outage conditions. * This will vary on where you are in the world, but networks tend to have a good amount of slack built in. * As networks become more complex, and the cost of computing has fallen, it's more feasible to automate contingency analysis (think about the number of different outage combinations for an N-2 scenario). FWIW, the internal tools that I work on makes use of networkx to determine contingency cases.
- cjdrake 3y agoThe "N" in HN stands for "news". Is there anything new about networkx to talk about?
- taosx 3y agoI believe it's really useful that sometimes things resurface on HN after a while. 1. Exposure of techniques, tools to people that are new to the field, or now the context is right. 2. People with experience share their insights and opinions (TIL: Igraph an/cugraph)
- tudorw 3y agoI'd add to that, the comments are fresh on resurfaced article, so if there is a new competitor or alternative it get's mentioned.
- mickeyp 3y agoGraph theory underpins nearly everything we do in software development and computer science. Networkx is an expansive -- though not the only -- package for Python that'll solve 90% of people's problems.
- simonw 3y agohttps://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html > On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.
- d0mine 3y agoThere may be lucky 10,000 https://xkcd.com/1053/ https://xkcd.com/1053/
- polotics 3y agoSo much "news" these days is just unadulterated crass clickbait, that a friendly reminder to revisit interesting subjects really does qualify as above average "news", yes. Sadly?
- Der_Einzige 3y agoIgraph and cugraph, and graph tool are far superior for a wide variety of reasons
- rdedev 3y agoI've used igraph. While it's much faster, for me at least, modifying the graph once it's constructed is harder compared to network. Haven't worked with cugraph though. As always use the right tool for the job
- gkoller 3y agoI would be interested in one that is properly type annotated. None of the options here (NetworkX, igraph, cugraph) are.
- westurner 3y agohttps://news.ycombinator.com/item?id=36922924 https://news.ycombinator.com/item?id=36922924 : > pytype (Google) [1], PyAnnotate (Dropbox) [2], and MonkeyType (Instagram) [3] all do dynamic / runtime PEP-484 type annotation type inference [4] to generate type annotations. Hypothesis generates tests from type annotations; and icontract and pycontracts do runtime type checking.
- westurner 3y ago/? "networkx" "igraph" "cugraph" site:github.com inurl:awesome https://www.google.com/search?q=%22networkx%22+%22igraph%22+%22cugraph%22+site%3Agithub.com+inurl%3Aawesome https://www.google.com/search?q=%22networkx%22+%22igraph%22+... : - https://github.com/johnhany/awesome-list#graph https://github.com/johnhany/awesome-list#graph lists a few Tensorflow and Pytorch + graphs applications CuGraph docs > List of Supported and Planned Algorithms: https://docs.rapids.ai/api/cugraph/stable/graph_support/algorithms/ https://docs.rapids.ai/api/cugraph/stable/graph_support/algo... https://github.com/rapidsai/cugraph#news https://github.com/rapidsai/cugraph#news : > NEW! nx-cugraph, a NetworkX backend that provides GPU acceleration to NetworkX with zero code change. : pip install nx-cugraph-cu11 --extra-index-url https://pypi.nvidia.com export NETWORKX_AUTOMATIC_BACKENDS=cugraph
- pushedx 3y agoHappy user of NetworkX since 2009!
- jonathaneunice 3y agoBeen a few years since I put NetworkX through its paces, but the several times I have tried it, found remarkably weak support for graph layout and display. NetworkX analytic routines may be strong, but attractively displaying graph-structured problems remarkably more interactive and attractive via d3.js, GraphViz, etc. At least for my problems, communicating graph structures, and having nodes and edges that represent different kinds of things…these are basic requirements, not optional frills.
- nick0garvey 3y agoI use NetworkX to build the graphs and Gephi to visualize them. No need to pick a single tool.
- mickeyp 3y agoLaying out a graph so it's "friendly to humans" is a seriously hard problem. I've built complex DAG workflow engines using networkx and its layout tools and they worked just fine. But, yeah, I guess it depends on what you need? Export to .dot -> open in your favourite viewer.
- standfest 3y agoI usually write small functions for postprocessing the proposed layouts from the default algorithms. So far this was always more than sufficient.
- dragonwriter 3y ago> Been a few years since I put NetworkX through its paces, but the several times I have tried it, found remarkably weak support for graph layout and display. Well, yeah, it is pretty open that it is the wrong tool for that job. Here's what the NetworkX documentation [0] says about its visualization support: NetworkX provides basic functionality for visualizing graphs, but its main goal is to enable graph analysis rather than perform graph visualization. In the future, graph visualization functionality may be removed from NetworkX or only available as an add-on package. Proper graph visualization is hard, and we highly recommend that people visualize their graphs with tools dedicated to that task. Notable examples of dedicated and fully-featured graph visualization tools are Cytoscape, Gephi, Graphviz and, for LaTeX typesetting, PGF/TikZ. To use these and other such tools, you should export your NetworkX graph into a format that can be read by those tools. For example, Cytoscape can read the GraphML format, and so, networkx.write_graphml(G, path) might be an appropriate choice. [0] https://networkx.org/documentation/latest/reference/drawing.html https://networkx.org/documentation/latest/reference/drawing....
- dang 3y agoRelated: NetworkX 3.0 - create, manipulate, and study complex networks in Python - https://news.ycombinator.com/item?id=34321135 https://news.ycombinator.com/item?id=34321135 - Jan 2023 (55 comments)
- rekoil 3y agoNetworkX has helped me with so many Advent of Code questions. Some where it was warranted, some where I made it work anyway because I like it!
- vlad_ungureanu 3y agoI found the documentation for networkx much better than the one from igraph[1] (at least the Python version). However, for community detection algorithms graph-tool[2] is better (it also uses a different class of models than the standard in literature) [1] https://igraph.org https://igraph.org [2] https://graph-tool.skewed.de https://graph-tool.skewed.de
- rpigab 3y agoI've used recently the networkx algorithm to find Hamiltonian cycles in a graph, in order to generate a Secret Santa with constraints (couples don't send gifts to each other, and people don't give to the same person as last year), it works great even though the problem is NP-complete, since my number of participants is very low. I've tried the same in Rust with petgraph which resembles networkx, but it doesn't have the algorithm for Hamiltonian built in and I couldn't wrap my head around the DFS/BFS visitor pattern, but I'll continue this some day.
- samsquire 3y agoIs this similar to graph colouring?
- rpigab 3y agoThey are distinct problems, because if you have a graph with a Hamiltonian cycle inside, you can add as many edges as you want, the cycle will always be there, but some N-colouring solutions might break. They are both NP-complete though.
- okasaki 3y agoIf you need more speed (but less features), check out networkit - https://networkit.github.io/ https://networkit.github.io/
- graphviz 3y agoSome algorithms in networkit, like approximate betweenness, are amazing.
- wslh 3y agoI recommend to complement with the recent Reddit thread in /r/Python [1]. [1] "What are the best libraries to work with graphs?" https://www.reddit.com/r/Python/comments/185xexg/what_are_the_best_libraries_to_work_with_graphs/ https://www.reddit.com/r/Python/comments/185xexg/what_are_th...
- liotier 3y agoNetworkX let me whip up a useful shortest path routing proof of concept from telco data in a few hours. I was impressed with myself, but all glory goes to NetworkX !
- codetrotter 3y agoSee also https://github.com/Qiskit/rustworkx https://github.com/Qiskit/rustworkx – a general purpose graph library for Python written in Rust to take advantage of the performance and safety that Rust provides. > Rustworkx was originally called retworkx and was created initially to be a replacement for qiskit's previous (and current) NetworkX usage (hence the original name). The project was originally started to build a faster directed graph to use as the underlying data structure for the DAG at the center of qiskit-terra's transpiler. However, since it's initial introduction the project has grown substantially and now covers all applications that need to work with graphs which includes Qiskit.
- deleted 3y ago[deleted]
- shrubbery 3y agoOn a related note, is there a popular option nowadays for a solid FOSS graph visualization library in the browser. Years back I used zoomcharts which was really good but had an expensive licence for on premise usage per client.
- tangue 3y agoBack in the days I’ve discovered NetworkX and Gephi in a Coursera course and was really surprised about how simple it managed to represent visually such a hard problem (I’ve never been able to find this course again it started with Erdos number that’s the only thing I remember)
- quibono 3y agoAh I was about to ask what course this was. If you ever find it again I'd love to hear what it was.
- BerislavLopac 3y agoNetworkX's documentation is the source of one of the most surreal but true sentences ever written (with just a slight alteration) [0]: A lobster is a tree that reduces to a caterpillar when pruning all leaves. [0] https://networkx.org/documentation/latest/reference/generated/networkx.generators.random_graphs.random_lobster.html https://networkx.org/documentation/latest/reference/generate...