Skip to content


Category: Data Center

This exercise has been completed on Nexus 7K titanium emulator. I am running two N7K titanium machines inside vmware workstation and have connected 7 network adapters to emulate the physical interfaces in N7K, i am using titanium emulator version 5.2.1, if you are curious to know about the N7K titanium emulator and how to install inside vmware workstation, i will be writing more stuffs on it, soon.


I have connected my network adapters as follows on both machines, we can check which physical Ethernet interface on N7K machine is connected to network interface card on vmware workstation by simply disconnecting and watching ” show interface brief” on N7K console  which interface is going down.


Right now my Ethernet2/1 on both N7K is connected to Vmnet4 on both vmware machines.We need to confirm and recheck else our virtual machines will not talk to each other, they should be on the same Vmnet. Lets start with our configuration.

Step 1:- Lets change the hostname of the device and assign N7K-1 to one machine and assign N7K-2 to other machine, by our old IOS familiar command “hostname N7K-1″.

N7K# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
N7K(config)# hostname N7K-1  –>Same for other machine  as N7K-2.

Step-2:- Assign ip address of N7K-1 Ethernet2/1 as and unshut the interface, also assign ip address of N7K-2 Ethernet2/1 as and unshut the interface.Verify connectivity by pinging across the interfaces.

N7K-1(config)# ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=254 time=1.747 ms
64 bytes from icmp_seq=1 ttl=254 time=1.262 ms
64 bytes from icmp_seq=2 ttl=254 time=1.423 ms
64 bytes from icmp_seq=3 ttl=254 time=1.396 ms
64 bytes from icmp_seq=4 ttl=254 time=1.58 ms

— ping statistics —
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 1.262/1.481/1.747 ms

Step-3 Check VRF membership of all interfaces inside N7K by ” show vrf interface”, by default there should be one “management” vrf for interface “mgmt0″and all other interfaces assigned to “default” vrf. Create a Loopback0 interface on N7K-1 and N7K-2 as and Also Create a VRF instance OSPF-VRF and assign it to interface Eth2/1 and Loopback0. Verify vrf assignment by “sh vrf interface”


We can also verify with “sh run vrf OSPF-VRF” to check the interface vrf assignment.


Step 4 Configure the same on N7K-2 with the loopback0 address of and vrf as OSPF-VRF and assign it to interface eth2/1 and loopback0.


Step 5 Next we start our OSPF configuration , this Lab is completed under vrf just to show the vrf functionality along with OSPF, we can also run OSPF without vrf instance  in NXOS like we do in IOS. First we enable the “feature ospf” and configure some basic OSPF commands and assign Area 0 to interface Eth2/1 on both N7K and assign Area 1 and Area 2 to both Loopback0. And in the end we will verify our OSPF neighborship and routes received.


Step 6 :- We will verify our configuration with various show commands. And play around with OSPF.


We can also clear ospf process by “clear ip ospf neighbor vrf OSPF-VRF” and by command”restart ospf 1″ . And watch hellos send and received and other ospf traffic related parameters by “sh ip ospf traffic vrf OSPF-VRF”.


We can also configure OSPF authentication in NXOS same as IOS.


This lab has been completed on Nexus 7010 with following hardware and software installed, it can be seen here in my previous post.

In this task we will configure ACLs using the atomic programming feature of Cisco NX-OS
Software. In addition, we will investigate the method used to modify, validate and re sequence ACLs.

Step 1 On your N7K switch create two object-groups, one named ALLOWSUBNETS and the other BADPORTS.

N7k Object1

N7k Object2

Step 2 Exit global configuration mode and reenter using the configure session command.
Name our session ACL-CHECKER
N7K11-pod3# configure session ACL-CHECKER

Step 3 Create an IP access list named BIG-ACL.
N7K11-pod3(config-s)# ip access-list BIG-ACL

Step 4 Assign the object-groups named ALLOWSUBNETS and BADPORTS created in Step 1 above to the IP access list BIG-ACL.
N7K11-pod3(config-s-acl)# permit ip addrgroup ALLOWNETS any
N7K11-pod3(config-s-acl)# deny tcp any port-group BADPORTS any

Step 5 Add the following deny statements to the access list named BIG-ACL.
N7K11-pod3(config-s-acl)# deny tcp any
N7K11-pod3(config-s-acl)# deny tcp any
N7K11-pod3(config-s-acl)# deny tcp any
N7K11-pod3(config-s-acl)# deny tcp any
N7K11-pod3(config-s-acl)# exit

Step 6 Assign the IP access list BIG-ACL to the port channel interface within our pod
VDC in the ingress direction.
N7K11-pod3(config-s)# interface port-channel 1

N7K11-pod3(config-s-if)# ip access-group BIG-ACL in
N7K11-pod3(config-s-if)# exit

Step 7 Step 7 Verify the configuration session ACL-CHECK.
N7K11-pod3(config-s)# verify
Verification Successful

Step 8 If the operation in Step 7 was successful, then commit the session to the running
N7K11-pod3(config-s)# commit
Commit Successful

N7k Object3

We can check our access-list just configured by ” show access-list BIG-ACL”. Also we can insert multiple sequence in between our access-list sequence, here we will insert sequence 11 to 19 between 10 and 20.

N7k ACL1

We can resequence the ACL also by using “resequence ip access-list BIG-ACL [start-seq] [end-seq]” and we can verify our new restructured ACL.

N7k ACL2

We can use several Show commands like “show ip access-list”, ” sh access-list”to check our configuration like “, one cool command is “Show running-config aclmgr”

N7k ACL3


This exercise has been done on Nexus 7010 chassis with following pieces installed:-

  • Dual supervisor modules, dual power supplies, dual system fans, dual fabric fans, and three fabric modules per chassis.
  • One 48-port 1 Gigabit Ethernet I/O module per chassis.
  • One 32-port 10 Gigabit Ethernet I/O module per chassis with SFP+ SR optical transceivers installed.
  • Cisco NX-OS LAN Enterprise License.
  • Cisco NX-OS LAN Advanced Services License.

First we verify the hardware discussed above.

N7K Module

The above output shows One 48-port 1 Gigabit Ethernet I/O module and One 32-port 10 Gigabit Ethernet I/O module and three fabric modules per chassis installed  in the Nexus 7010. The rest of the hardware can be easily seen by our very old IOS familiar command “Show environment” and NXOS version can be seen by “Show Version” command.

Without AAA, IOS relies on privilege levels.  Privilege levels (0-15) defines locally what level of access a user has when logged into an IOS device, i.e. what commands are permitted.

  • Privilege Level 1 — Normal level on Telnet; includes all user-level commands at the router> prompt.
  • Privilege Level 15 — Includes all enable-level commands at the router# prompt.

NX-OS uses a different concept for the same purpose, known as User Roles. User Roles contain rules that define the operations allowed for a particular user assigned to a role. There are default User Roles:

  • Network-Admin—Complete read-and-write access to the entire NX-OS device (only available in the default VDC).
  • Network-Operator—Complete read access to the entire NX-OS device (Default User Role).
  • VDC-Admin—Read-and-write access limited to a VDC (VDCs are not yet available on Nexus 5000).
  • VDC-Operator—Read access limited to a VDC (Default User Role).

VDC(s) allow the partitioning of a single physical Nexus 7000 device into multiple logical devices. This logical separation provides the following benefits:

  • Administrative and management separation
  • Change and failure domain isolation from other VDCs
  • Address, VLAN, VRF, and vPC isolation

We will discuss VDC in detail in upcoming posts.

When a NX-OS device is setup for the first time, during the first login, a Network-Admin account must be specified and subsequently be used to login. Arguably a bit more secure that IOS. Any additional users created locally after that will by default receive the User Role “Network-Operator“, unless specified implicitly.

Note:-User Roles are local to a switch and only relevant in the absence of AAA being configured.

When logging into a N5K or a N7K system VDC, the default User-Roles assigned is “network-operator”. When logging into a VDC, the default User-Roles is “vdc-operator”.

Apart from above system default roles we can create custom roles as per our requirement.Lets create a new role AFROZ and assign read and write privileges to this role.

In the below figure we have created a new role AFROZ, assigned read and read-write privileges with feature-group L3-ROUTE , feature-group L3-ROUTE has all routing protocol feature.

N7k role

Additionally we can configure our role(AFROZ) to permit or deny a feature. Here we will deny all Vlans except the range 1-100 and deny all VRF instances except for vrf INTERSWITCH and Limit access to all interfaces except the first two 1 Gigabit Ethernet and port channel interface  and verify our configuration.

N7k role deny

Now we will assure that strong passwords are supported and that any roles are distributed
between adjacent Cisco Nexus devices. We will check our password strength by attaching the role AFROZ to a new user. Create a new user named “afroz” and assign the password “afroz123” obviously without quotes. As we have already enabled password strength now NXOS tells us that the current password it weak , so now we will assign a password Rbac@AFROZ, and now the NXOS system excepts it.

Then we will check the status of the configured “roles” and apply the user role configuration changes in the temporary database to the running configuration and distribute the user to role configuration by “role distribute” to adjacent nexus devices.

N7k rbac









Data Center Architecture High Level Overview:-


1. Data Center Networking

2.Application Networking

3.Data Center Security

4.Unified Computing System

5.Unified Fabric

6.Storage Networking

7.Operations & Management

8.Services Component

9.Branch Operations

Overview Of Each Component and Devices Used:-

1.Data Center Networking:- Data Center networking accesses services and connectivity at the data link and network layer.

DC Networking Provides:-

  • Layer 2 services
  • Layer 3 services (as needed)
  • Server access and aggregartion
  • Core Switching
  • 10G and/or 1G connectivity.
  • VLANs
  • Virtual Devices Contexts or VDC

Products from Cisco in DC networking includes:-

  • Nexus switch family(7000,5000,3000,2000,1000)
  • Catalyst switch family(6500,4500,4900)


2.Application Networking :- Application networking assures efficient use of network resources.It Provides :-

  • Server load balancing
  • WAAS(Wide Area Application support)
  • Application Control

Products from Cisco and Other Vendors:-

  • Application Control Engine (ACE) from cisco and  LTM or GTM from F5
  • WAAS from cisco and Riverbed Products like Riverbed Steelhead
  • Wide Area Application Engine or WAE from cisco


3.Data Center Security:-Security protects critical business assets and information from internal and external threats. Security provides in depth protection of network devices, applications and operating system via:

  • Firewall security and deep packet inspection
  • Access Control
  • Host and Network intrusion detection
  • Attack mitigation systems
  • Anti-virus systems

Products from Cisco and Other Vendors:-

  • Catalyst firewall service modules
  • ASA firewall appliances
  • Juniper , Checkpoint and other vendors security product range


4.Unified Computing System:-UCS brings together compute, network and system management into a scalable, cohesive system. UCS is the only product range from cisco in the current market of its class.UCS provides:-

  • Single converged system
  • Programmable infrastructure
  • Unified Fabric

Products from Cisco:-

  • UCS B-Series chassis-based servers
  • UCS C-Series Rack mount servers


5.Unified Fabric:-Unified fabric is based on open standards, and integrates silos to allow access across the data center. Unified fabric provides:-

  • 10G server access
  • Fibre Channel over Ethernet(FCoE) functionality
  • I/O consolidation
  • LAN/SAN server connectivity

Products from Cisco:-

  • Nexus 5000 and Nexus 2000 with appropriate NXOS image and license


6.Storage Networking(SAN):-Storage networking constitute both the actual storage components and the link between storage and servers.SAN provides:-

  • Data Replication
  • Data Deduplication
  • SAN switching
  • Storage connectivity
  • Data vault
  • Disk and Tape subsystems

Products from Cisco:-

  • MDS 9000 SAN switch family


7.Operations & Management:-It integrates data center operations, management and services. It provides:-

  • Equipment and application health monitoring
  • Backup and restore operations
  • Security functions – network and physical security
  • Environmental monitoring
  • Policy management
  • Provisioning of services

Products from Cisco:-

  • Data Center Network Manager (DCNM)
  • UCS Manager (UCSM)


8.Services Component:-Services include network add-ins, software and appliances. It provides

  • ANS- app control, server load balancing, WAAS
  • Security
  • Firewall
  • Intrusion Detection

Products from Cisco:-

  • ASA
  • ACE
  • WAAS


9.Branch Operations:- Branches can pose challenges to Data Center operations. Branches are usually considered part of Data Center. However, when goals include changes to business models, including mobility and virtual options, branch needs must be considered.

Products from Cisco:-

  • Branch Router (ISR)
  • LAN Switch (Catalyst)
  • UCS C-Series server
  • ASA
  • UC Express




Data Center Migration is one of the challenging job in networking world, and the reason behind this we have less documentation available online for this. Recently i did DC migration for a client in Vancouver, Canada. Below are the main highlights, i will be writing more in detail soon.

1. Network:-The first objective is to understand the network in and out from every aspect and user termination to current DC. We need to document  different type of users that are connecting to current DC, like VPN remote users, Site 2 Site VPN tunnel traffic, AVPN layer 3 MPLS cloud users, local remote site users connecting directly via OPTEWAN(layer 2 point to point MPLS) ,etc. Document everything related to current network and the network where we are planning to migrate, like allocation of Public ips, different routing protocols,technology and devices used, WAN and LAN vendors, etc.

2. Firewall:-Firewalls and the rules associated are the major concern and always create issues while migration, so we need to carefully document and consult client and vendors to understand the seriousness of each and every rule associated with the firewall.

3.Load Balancing:-Load balancing of web servers is again major concern and for this we need to understand the current DC setup, devices used, servers associated, etc. In most of the DC’s F5 LTM is used for load balancing, we need to carefully document F5 related configs and integration of F5 to the network from the current DC and plan according to the network setup of the new DC.

4.Storage:-Storage migration is also very important, here we have two options either migrate the storage physically or purchase a high bandwidth link( 1gig or so)from the WAN provider and migrate the entire storage over  the WAN circuit. The second option will take time depending on the WAN speed and amount of storage.

5.Virtual Machine:-Now a days almost every company is using virtual servers instead of many physical servers. The idea is to use few physical server and build as many Virtual servers on top of one physical chassis, this way cutting cost , also administration and other server related tasks will become simpler. So the migration of Virtual Machines is also very important , we need to understand and document VM’s from the application point of view and the way to migrate all VM’s over the WAN.