10 ms·

Automate Your Network

157 points by hjuutilainen 3y ago

runjake 3y ago
The author states they have evolved from Ansible to pyATS[1], but pyATS is a Cisco project. With Cisco's poor code project and open source track record, I'm not sure how this is much of an improvement, and IMHO, it's arguably worse. For possible alternatives, check out NAPALM[2] and Nornir[3]. It's also worth checking out Python for Network Engineers[4]. 1. https://developer.cisco.com/docs/pyats/ https://developer.cisco.com/docs/pyats/ 2. https://napalm.readthedocs.io/en/latest/ https://napalm.readthedocs.io/en/latest/ 3. https://nornir.readthedocs.io/en/latest/ https://nornir.readthedocs.io/en/latest/ 4. https://pyneng.readthedocs.io/en/latest/index.html https://pyneng.readthedocs.io/en/latest/index.html
- xnyanta 3y ago
  Had the same reaction as soon as I found out pyATS is a cisco-specific thing. I run very simple networks for events on shoestring hardware/budgets and built a simple wrapper around my own object model using python, jinja and napalm to deploy cisco switches via SSH. Has terraform-like semantics (plan/apply) and lets me be productive and eliminate config drift. Napalm does all of the heavy lifting, it is fantastic. I will probably be integrating it with netbox soon.
- batch12 3y ago
  Looks like he works for Cisco at the moment. Maybe that has something to do with it.
theideaofcoffee 3y ago
I glanced through the guide and it's Windows and Cisco (specifically IOS) heavy: mentions of the old Cisco architecture via Core/Access/Distribution, where larger DC networks have converged onto spine/spline setups, CDP/Cisco Discovery protocol whereas the open-source LLDP is more generic, even the nomenclature of 802.1q VLAN tags: access versus trunk. But I guess if you are starting to automate a legacy office network, it might be useful. More recent non-IOS network OSes that lend themselves to automation, especially in the datacenter, the likes of Cumulus or SONiC are pure linux with some asic-vendor-specific bits and bobs, so I'm unsure of the applicability of this guide to larger, more modern networks. Tools like ansible could be a good fit here, but since they are 'just' linux, might as well use a dedicated config management tool like chef or puppet. Otherwise I think it's well written for someone in a smaller shop wanting to get their feet wet with ansible and other tools but still stuck on IOS.
- jimmar 3y ago
  > old Cisco architecture via Core/Access/Distribution, where larger DC networks have converged onto spine/spline setups Please correct me if I'm wrong, but I see the "old" core/access/distribution layers still relevant. The datacenter spine/spline setup applies to networking between server racks in the data center. > 802.1q VLAN tags: access versus trunk Again, are you saying that these are outdated? I'm not a practicing network engineer, but I know several network engineers and they've told me that understanding 802.1q VLAN tags to segment network traffic has been helpful.
  - darkr 3y ago
    > 802.1q VLAN tags: access versus trunk I think the parent was saying that these are Cisco specific terms; more generic terms would be "untagged" + "tagged".
    - ajsnigrutin 3y ago
      Trunk and access ports are like kleenex and bandaids. Yes, technically cisco terminology, but used everywhere.
      - iso1631 3y ago
        Absolutely, here's a config from one of my aristas(with bits snipped) interface Ethernet1 switchport trunk native vlan 899 switchport trunk allowed vlan 801 switchport mode trunk interface Ethernet13 switchport access vlan 311 And on a Juniper set interfaces xe-0/2/1 unit 0 family ethernet-switching interface-mode trunk set interfaces xe-0/2/1 unit 0 family ethernet-switching vlan members Mgmt_B set interfaces xe-0/2/1 unit 0 family ethernet-switching vlan members Audio_2 .... set interfaces ge-0/0/19 unit 0 family ethernet-switching interface-mode access set interfaces ge-0/0/19 unit 0 family ethernet-switching vlan members Audio_2 When Cisco, Arista, Juniper all use access vs trunk it's hardly a vendor specific term
  - kazen44 3y ago
    > Please correct me if I'm wrong, but I see the "old" core/access/distribution layers still relevant. The datacenter spine/spline setup applies to networking between server racks in the data center. this is correct. The place where spine-leaf really shines is when used in combination with evpn-vxlan. You can then encapsulate every tenant network inside a VXLAN domain and route those between your leafs switches through your spine layer. This is basically a clos fabric which is non-blocking, and is very easy to expand horizontally. It also gives you nice features like ARP suppression[0]. These features are important in a DC fabric because ARP flooding is traffic which is not revenue generating, and should be minimized as much as possible. For normal Enterprise/Office network, running an evpn-vxlan fabric is usually far to complex for the benefits involved. [0] https://satishdotpatel.github.io/how-does-arp-suppression-work-mp-bgp-evpn/ https://satishdotpatel.github.io/how-does-arp-suppression-wo...
nu11ptr 3y ago
I do network automation for a profession. I build tools (technically compilers) that take a proprietary object model designed for our private cloud and translate that into Ansible (v1) or Terraform (v2) code. At our company, I actually call using these tools in isolation doing it "manually". This is because the largest benefit of automation, I believe, is the abstraction gained from the new object model and being to to generate and store the inputs for Ansible/Terraform in a database. If you have to track and specify all the inputs into Ansible/Terraform and write the playbooks/HCL manually it is my experience you don't actually save all that much work. However, when you have an object model specifically designed for your use case, you can deliver a new client network in literally minutes (essentially nothing more than the cloud model, exactly what AWS/Azure, etc does for their networking). The downside is most enterprises don't have people like me to write the code to do this, and writing it for a single deployment would likely not see the gains that we see as a managed service provider.
- xnyanta 3y ago
  This model is probably more common than you think, I don't see how anyone would be doing this any other way in a scalable fashion.
- tguvot 3y ago
  i worked on a product that did something similar for telecoms. had a closed loop automation and graphical designer for object model. it was 10 years ago. looking today at all the manual work with playbooks/etc, it's astonishing. feels like things didn't move forward at all in past decade
  - dopylitty 3y ago
    Even in the big public clouds the user facing networking really hasn't progressed beyond a layer of lipstick on top of the kludges that were created for connecting physical servers 40 years ago. For instance in AWS you still have to care about BGP and ASNs if you want to follow the most seamless approach to create a multi-region mesh of VPCs. Why should I have to care about that? AWS already knows where all the packets came from and where they're going and should just put them in the right place. I don't care how they get there and I certainly shouldn't have to care about BGP attributes[1]. 1. https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-peerings.html https://docs.aws.amazon.com/network-manager/latest/cloudwan/...
metadat 3y ago
Direct link to the PDF: https://github.com/automateyournetwork/automate_your_network/raw/main/Automate%20Your%20Network%20-%20John%20Capobianco%20-%20Paperback.pdf https://github.com/automateyournetwork/automate_your_network...
dvno42 3y ago
Hey this is cool! Thanks for sharing your hard work. I have been living this for the past few years building an automation product[0] and services company to lower the barrier of entry and have tested many of these methodologies. We’ve also written many different runbooks/playbooks for complicated workflows. I’d like to share a couple experiences/opinions: Netconf and vendor apis are lovely when available and working well. Many devices don’t support this and falling back to SSH (sometimes even telnet) is a must for automation. Imo, you could add value to your book by touching on Ktbyer's Netmiko/Paramiko[1] as well as their nuances (timeouts, dealing with interactive prompts, etc). AAA is a big component of automation too. Having something in place to handle authn/authz (radius/tacacs) enables consistency for access across vendors. This also enables least privileged accounts and rotation/limited lifetime of creds when used with something like Hashicorp Vault[2]. I think you briefly mentioned secrets management though Ansible vault. Another technology that may be worth mentioning is Textfsm[3] in conjunction with Netmiko. When we automate workflows for clients, there’s often times where the data we need to parse isn’t easily parsable. Using and expanding on textfsm makes this doable. Lastly, much automation may only be one firmware change away from breaking. Even with the big vendors, bugs are common that are (ime) low priority to the OEM. Keep this in mind when writing runbooks/playbooks, try to rely on features and output that are unlikely to change across versions. [0]https://realmhelm.com https://realmhelm.com [1]https://github.com/ktbyers/netmiko https://github.com/ktbyers/netmiko [2]https://github.com/hashicorp/vault https://github.com/hashicorp/vault [3]https://github.com/google/textfsm https://github.com/google/textfsm
- Cyph0n 3y ago
  +1 to textfsm: it is an extremely powerful approach to reliably parse CLI-based outputs. I used to do some IOS-XR device automation when I worked at Cisco - mainly for integration testing - and I (and other teams) used it heavily. This ties in to your point about how you often need to fallback to SSH or Telnet. For example, a lot of platform-specific data isn’t exposed through standard interfaces, but almost everything is available through a CLI. There are also times when you have no choice but to use the CLI - for example, when re-imaging or reloading a device.
betaby 3y ago
ctr+f 'yang' - nothing ctr+f 'netconf' - nothing
SergeAx 3y ago
> I believe in open source software But... PDF is not "open source", it is literally a compiled binary blob :) This is pretty cool book though. If author reading this: can you please publish real source files for the book?

157 points by hjuutilainen 3y ago

The author states they have evolved from Ansible to pyATS[1], but pyATS is a Cisco project. With Cisco's poor code project and open source track record, I'm not sure how this is much of an improvement, and IMHO, it's arguably worse. For possible alternatives, check out NAPALM[2] and Nornir[3]. It's also worth checking out Python for Network Engineers[4]. 1. https://developer.cisco.com/docs/pyats/ https://developer.cisco.com/docs/pyats/ 2. https://napalm.readthedocs.io/en/latest/ https://napalm.readthedocs.io/en/latest/ 3. https://nornir.readthedocs.io/en/latest/ https://nornir.readthedocs.io/en/latest/ 4. https://pyneng.readthedocs.io/en/latest/index.html https://pyneng.readthedocs.io/en/latest/index.html

xnyanta 3y ago
Had the same reaction as soon as I found out pyATS is a cisco-specific thing. I run very simple networks for events on shoestring hardware/budgets and built a simple wrapper around my own object model using python, jinja and napalm to deploy cisco switches via SSH. Has terraform-like semantics (plan/apply) and lets me be productive and eliminate config drift. Napalm does all of the heavy lifting, it is fantastic. I will probably be integrating it with netbox soon.
batch12 3y ago
Looks like he works for Cisco at the moment. Maybe that has something to do with it.

theideaofcoffee 3y ago

I glanced through the guide and it's Windows and Cisco (specifically IOS) heavy: mentions of the old Cisco architecture via Core/Access/Distribution, where larger DC networks have converged onto spine/spline setups, CDP/Cisco Discovery protocol whereas the open-source LLDP is more generic, even the nomenclature of 802.1q VLAN tags: access versus trunk. But I guess if you are starting to automate a legacy office network, it might be useful. More recent non-IOS network OSes that lend themselves to automation, especially in the datacenter, the likes of Cumulus or SONiC are pure linux with some asic-vendor-specific bits and bobs, so I'm unsure of the applicability of this guide to larger, more modern networks. Tools like ansible could be a good fit here, but since they are 'just' linux, might as well use a dedicated config management tool like chef or puppet. Otherwise I think it's well written for someone in a smaller shop wanting to get their feet wet with ansible and other tools but still stuck on IOS.

jimmar 3y ago
> old Cisco architecture via Core/Access/Distribution, where larger DC networks have converged onto spine/spline setups Please correct me if I'm wrong, but I see the "old" core/access/distribution layers still relevant. The datacenter spine/spline setup applies to networking between server racks in the data center. > 802.1q VLAN tags: access versus trunk Again, are you saying that these are outdated? I'm not a practicing network engineer, but I know several network engineers and they've told me that understanding 802.1q VLAN tags to segment network traffic has been helpful.
- darkr 3y ago
  > 802.1q VLAN tags: access versus trunk I think the parent was saying that these are Cisco specific terms; more generic terms would be "untagged" + "tagged".
  - ajsnigrutin 3y ago
    Trunk and access ports are like kleenex and bandaids. Yes, technically cisco terminology, but used everywhere.
    - iso1631 3y ago
      Absolutely, here's a config from one of my aristas(with bits snipped) interface Ethernet1 switchport trunk native vlan 899 switchport trunk allowed vlan 801 switchport mode trunk interface Ethernet13 switchport access vlan 311 And on a Juniper set interfaces xe-0/2/1 unit 0 family ethernet-switching interface-mode trunk set interfaces xe-0/2/1 unit 0 family ethernet-switching vlan members Mgmt_B set interfaces xe-0/2/1 unit 0 family ethernet-switching vlan members Audio_2 .... set interfaces ge-0/0/19 unit 0 family ethernet-switching interface-mode access set interfaces ge-0/0/19 unit 0 family ethernet-switching vlan members Audio_2 When Cisco, Arista, Juniper all use access vs trunk it's hardly a vendor specific term
- kazen44 3y ago
  > Please correct me if I'm wrong, but I see the "old" core/access/distribution layers still relevant. The datacenter spine/spline setup applies to networking between server racks in the data center. this is correct. The place where spine-leaf really shines is when used in combination with evpn-vxlan. You can then encapsulate every tenant network inside a VXLAN domain and route those between your leafs switches through your spine layer. This is basically a clos fabric which is non-blocking, and is very easy to expand horizontally. It also gives you nice features like ARP suppression[0]. These features are important in a DC fabric because ARP flooding is traffic which is not revenue generating, and should be minimized as much as possible. For normal Enterprise/Office network, running an evpn-vxlan fabric is usually far to complex for the benefits involved. [0] https://satishdotpatel.github.io/how-does-arp-suppression-work-mp-bgp-evpn/ https://satishdotpatel.github.io/how-does-arp-suppression-wo...

nu11ptr 3y ago

I do network automation for a profession. I build tools (technically compilers) that take a proprietary object model designed for our private cloud and translate that into Ansible (v1) or Terraform (v2) code. At our company, I actually call using these tools in isolation doing it "manually". This is because the largest benefit of automation, I believe, is the abstraction gained from the new object model and being to to generate and store the inputs for Ansible/Terraform in a database. If you have to track and specify all the inputs into Ansible/Terraform and write the playbooks/HCL manually it is my experience you don't actually save all that much work. However, when you have an object model specifically designed for your use case, you can deliver a new client network in literally minutes (essentially nothing more than the cloud model, exactly what AWS/Azure, etc does for their networking). The downside is most enterprises don't have people like me to write the code to do this, and writing it for a single deployment would likely not see the gains that we see as a managed service provider.

xnyanta 3y ago
This model is probably more common than you think, I don't see how anyone would be doing this any other way in a scalable fashion.
tguvot 3y ago
i worked on a product that did something similar for telecoms. had a closed loop automation and graphical designer for object model. it was 10 years ago. looking today at all the manual work with playbooks/etc, it's astonishing. feels like things didn't move forward at all in past decade
- dopylitty 3y ago
  Even in the big public clouds the user facing networking really hasn't progressed beyond a layer of lipstick on top of the kludges that were created for connecting physical servers 40 years ago. For instance in AWS you still have to care about BGP and ASNs if you want to follow the most seamless approach to create a multi-region mesh of VPCs. Why should I have to care about that? AWS already knows where all the packets came from and where they're going and should just put them in the right place. I don't care how they get there and I certainly shouldn't have to care about BGP attributes[1]. 1. https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-peerings.html https://docs.aws.amazon.com/network-manager/latest/cloudwan/...

metadat 3y ago

Direct link to the PDF: https://github.com/automateyournetwork/automate_your_network/raw/main/Automate%20Your%20Network%20-%20John%20Capobianco%20-%20Paperback.pdf https://github.com/automateyournetwork/automate_your_network...

dvno42 3y ago

Hey this is cool! Thanks for sharing your hard work. I have been living this for the past few years building an automation product[0] and services company to lower the barrier of entry and have tested many of these methodologies. We’ve also written many different runbooks/playbooks for complicated workflows. I’d like to share a couple experiences/opinions: Netconf and vendor apis are lovely when available and working well. Many devices don’t support this and falling back to SSH (sometimes even telnet) is a must for automation. Imo, you could add value to your book by touching on Ktbyer's Netmiko/Paramiko[1] as well as their nuances (timeouts, dealing with interactive prompts, etc). AAA is a big component of automation too. Having something in place to handle authn/authz (radius/tacacs) enables consistency for access across vendors. This also enables least privileged accounts and rotation/limited lifetime of creds when used with something like Hashicorp Vault[2]. I think you briefly mentioned secrets management though Ansible vault. Another technology that may be worth mentioning is Textfsm[3] in conjunction with Netmiko. When we automate workflows for clients, there’s often times where the data we need to parse isn’t easily parsable. Using and expanding on textfsm makes this doable. Lastly, much automation may only be one firmware change away from breaking. Even with the big vendors, bugs are common that are (ime) low priority to the OEM. Keep this in mind when writing runbooks/playbooks, try to rely on features and output that are unlikely to change across versions. [0]https://realmhelm.com https://realmhelm.com [1]https://github.com/ktbyers/netmiko https://github.com/ktbyers/netmiko [2]https://github.com/hashicorp/vault https://github.com/hashicorp/vault [3]https://github.com/google/textfsm https://github.com/google/textfsm

Cyph0n 3y ago
+1 to textfsm: it is an extremely powerful approach to reliably parse CLI-based outputs. I used to do some IOS-XR device automation when I worked at Cisco - mainly for integration testing - and I (and other teams) used it heavily. This ties in to your point about how you often need to fallback to SSH or Telnet. For example, a lot of platform-specific data isn’t exposed through standard interfaces, but almost everything is available through a CLI. There are also times when you have no choice but to use the CLI - for example, when re-imaging or reloading a device.

betaby 3y ago

ctr+f 'yang' - nothing ctr+f 'netconf' - nothing

SergeAx 3y ago

> I believe in open source software But... PDF is not "open source", it is literally a compiled binary blob :) This is pretty cool book though. If author reading this: can you please publish real source files for the book?