To Log into a Switch from Current Switch

1. You can perform this either from user mode or privileged mode:
a. SW1>ssh -l user_name ip_address
b. SW1#ssh -l user_name ip_address
2. After you pivot into the system from either of the above commands, type in the user’s password (the user you are using to log into the distant switch).

Configure Default Gateway on Cisco Switch

1. First, escalate into global configuration mode:
a. SW1(config)#Config Terminal
2. In global configuration mode you can now make a global setting indicating your default gateway for the switch:
a. SW1(config)#ip default-gateway ip_address

Configure Gateway of Last Resort

1. First, log into the switch you wish to configure.
2. Next, escalate up to Configure Terminal and then type the following:
    a. #conf t
    b.  #ip default-gateway 172.16.X.X
    c.  #end
    d.  #write
3. Next you can direct your traffic through this route if you wish, as a gateway of last resort:
    a. #ip route 0.0.0.0 0.0.0.0 172.16.X.X
    b. #end
    c. #write
4. Verify by typing in the following:
    a.  #show ip route
    b.  Then verify you see a gateway of last resort listed.


Reference:
https://www.cisco.com/c/en/us/support/docs/ip/routing-information-protocol-rip/16448-default.html

Clearing UCSM error F0933 “named VLAN for vNIC cannot be resolved” errors

1.    First off upon logging onto your UCSM you find this lovely error:

2.    So what is happening here is you done messed up A A Ron! A VLAN was deleted from the global LAN Cloud area and more than likely is still attached to a vNIC template out there. There can be some other causes, but this has been the most common issue I have found with this error. Your UCS has a VLAN it does not know what to do with.
3.    So, from the error screen, go ahead and go to your LAN section in your UCS and go to your vNIC templates section:

4.    Now we can go ahead and select this VLAN we thought was gone, and click on the Delete at the bottom.

5.    Once you delete the VLAN, the errors should go down or start going away. You want to check all your vNIC templates to ensure it has been removed.

Video Issues Upon Logon in Linux

Sometimes you will get login issues such as a blank screen, login loops, and other ways you cannot just logon within Linux distros. The best way to do this is the following:

1. First, when you get to the boot menu and see your Linux distro (s), hit the “e” key to enter edit mode. This will bring up the kernel startup options.
2. You want to find where it stays “quiet splash” screen and then change “quiet splash” to “nomodeset”. This will basically start the system with no graphics drivers which may be causing you video issues upon login.
3. Hold CTRL+X to startup your Linux system.
4. You Should now be able to login to the GUI with no default drivers which may be causing issues.

Reference:

https://wiki.ubuntu.com/Kernel/KernelBootParameters

Edit Alarms in vSphere

1.    First log into your vSphere.
2.    Next, click on the “Home” icon in the top left of your vSphere screen.

3.    Next, click on “Global Inventory Lists” on the dropdown.

4.    Then click on vCenter Servers.

5.    Click on the vCenter server you wish to edit alarms on.

6.    Next, click on the “Monitor” tab.

7.    In the monitor tab, you have three options:
a.    All Issues: This will list all the issues in your vCenter.
b.    Triggered Alarms: This will show any currently triggered alarms in your current vCenter.
c.    Alarm Definitions: This is the tab you want to click on. This tab is all of the definitions of the alarms in vCenter. Definitions are just the configurations/descriptions of your alarms.
8.    Click on the alarm you wish to edit. You will see a menu in the right of your alarm definition screen displaying your selected alarm.
9.    Next, you want to click on the “Edit” button in the top right of the menu showing your alarm definition attributes.

10.    Finally this is the popup you can configure/re-configure your alarms:
a.    General: just allows you to select what will trigger the alarm, virtual machines, clusters, hosts, etc. Items in your vCenter you wish to know about.
b.    Triggers: This will allow you to set the specific alert you wish to trigger. You can have VM Memory usage, CPU usage, power, etc.
c.    Actions: This is where you will put in what actions you wish the system to automatically perform.
11.    You will fill out the appropriate sections to monitor what you wish to keep track of and then click Next to go through the menus and Finish to finalize your edit.

Reference:
https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.monitoring.doc/GUID-79AC1262-D701-4BC8-8F8D-F046AE0578CF.html

Upload Firmware into UCSM (GUI)

1.    First off, log into your UCSM system you wish to upload to.
2.    Next, click on the following tabs:
    a.    Equipment
    b.    Firmware Management
    c.    Installed Firmware
3.    Next, under the “Installed Firmware” section, click on “Download Firmware”. This option means the Fabric Interconnects (FIs) themselves will download the software into their flash. Not you downloading the software yourself. It is confusing at first, but remember this is you pushing the firmware to the FIs and they download it.
4.    After you click this, then you can select the file you wish to upload and then click “OK”.
5.    Your software will start uploading into the FI so it can download. If you need to check the progress of the download, click on the “Download Tasks” tab to the right of the “Installed Firmware” tab.
6.    Finally, you can verify your firmware was uploaded/downloaded by selecting the “Packages” tab and you should see your new firmware upload in here.

Reference:
https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-manager/GUI-User-Guides/Firmware-Mgmt/4-0/b_UCSM_GUI_Firmware_Management_Guide_4-0/b_UCSM_GUI_Firmware_Management_Guide_4-0_chapter_0100.html#task_0EE4036D9F434685A3CD94EBC0501B66

Terminating UCSM Sessions Manually

1.    Sometimes your UCSM web sessions will get hung up and prevent you from logging in. The following will help you remove the selected sessions you wish to terminate and then allow access again.
2.    First SSH into your UCSM FIs.
3.    Next type in the following:
    a.    #scope security
    b.    #show user-session “local or remote”
    c.    #delete user-session “local or remote” UserName SessionID
        i.    This will be the select username you wish to terminate along with their Session ID. Be CAREFUL, some users might have SSH or the Web interface, or both up. Make sure you select the right Session ID.
    d.    #commit-buffer
4.    Now you should be able to close out all of your web browsers you have the UCSM open in, go back and try to re-log on.

5.    Finally, another way to terminate is you can terminate all Cisco Integrated Management Controller (CIMC) sessions. The above method is preferred since you can specify which one, but continue the following if you are still having issues:
    a.    #scope security
    b.    #show cimc-sessions “local or remote”
    c.    #show cimc-sessions “local or remote” UserName
    d.    #scope security
    e.    #terminate cimc-sessions “local or remote”-user UserName
    f.    This will close cimc sessions. Are you sure? (yes/no): yes
6.    Again, close out of your browsers with the USCM open in and then re-open and try to authenticate again.

Reference:
https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-manager/CLI-User-Guides/Admin-Management/3-1/b_Cisco_UCS_Manager_CLI_Administration_Mgmt_Guide_3_1/b_Cisco_UCS_Manager_CLI_Administration_Mgmt_Guide_3_1_chapter_0111.html

Re-issue Certificate for UCSM

1.    You will sometimes have a certificate expire on your UCSM. This will show up as a “Major” error in your UCSM errors/logs. This can be cleared by re-issuing a new certificate within the fabric interconnects.
2.    First, log into the fabric interconnects through SSH.
3.    Once you have logged into the FI, use the following commands to re-issue new certificate:
    a.    #scope security
    b.    #scope keyring default
    c.    #set regenerate yes
    d.    #commit-buffer
        i.    WARNING!: This WILL disconnect your web sessions. Be wary if you have others currently working in the UCSM, for it will disconnect them all.
4.    It can take up to 3-5 minutes to re-generate a new certificate for the UCSM. Give it a few minutes then use the following commands to verify the new key has been created:
    a.    #scope security
    b.    #show keyring detail
    c.    The following should be showing:
        i.    RSA key modules: ModXXXX
        ii.    Trustpoint: XXX
        iii.    Cert Status: Valid (here it should show valid or self-signed, not expired)
5.    You should now be able to log back into your UCSM and see the error cleared.

Changing admin Password in UCSM 3.2 (GUI)

1.    First, log into the UCSM you wish to manage.
2.    Navigate to the following:
    a.    >Admin
    b.    >All
    c.    >User Management
    d.    >Locally Authenticated Users Tab
    e.    >Click on the admin account below
    f.    >Finally, click on the Info tab below
3.    Here you can type in the new password and then confirm in the “Confirm Password” box.
4.    Click “OK” and your new password is updated.

Reference:

https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/gui/config/guide/2-2/b_UCSM_GUI_Configuration_Guide_2_2/configuring_role_based_access_control.html

Restore Cisco IOS to Cisco Device

1.    Login to the system device.
2.    Once you are in the Cisco device, you can perform the following to load the new Cisco IOS:
    a.    #copy tftp flash
    b.    Address or name of remote host []? The host name or IP you wish to pull from.
    c.    Source filename []? The name of the file you are uploading.
    d.    Destination filename []? Name you want it to show up as in the flash.
    e.    Erase flash: before copying []? You might or might not have this option. This is mostly for old systems and you do not have enough space to hold both the previous and the new IOS.
    f.    If you select erase, it will start erasing the system and upload up the new IOS.
    g.    Use the #show flash command to ensure your new IOS is there.
    h.    #reload
3.    Verify your new IOS version is now loaded through the #show version command upon reload.

Loading Cisco Device from a Backup Config File

1.    Log into your Cisco device.
2.    From user mode, type in the following:
    a.    #copy tftp statup-config
    b.    Address or name of remote host []? The host IP address or the host name of the system you wish to backup from.
    c.    Source filename []? Type in the name of your backup file.
    d.    You should start to see an “Accessing tftp://x.x.x.x” on your screen.
3.    Wait for the system to restore from the backup file.
4.    Now you can reload the device and load to the restored backup file.

Backup Cisco Device Config

1.    First off, log into your Cisco device.
2.    Launch your TFTP program you wish to have pull the file (I utilize SolarWinds free TFTP too, but you can use any out there.). Just make sure your program is running and ready to pull the config.
3.    You can execute the following commands to backup your config:
    a.    #copy running-config tftp:
    b.    Address or name of remote host []? YourIPAddressYouCopyingTo
    c.    Destination filename [default]? NameforYourFile
4.    Then you just need to wait for the file copy/transfer to finish. Then check your file destination to verify it finished copying.

Mess With ROMMON Cisco Device

•    While in ROMMON you have three important commands:
    o    >confreg
            You can use this to actually change the boot register config of the device.
    o    >set
            The set command will allow you to actually assign IP addresses to your interfaces.
    o    >tftpdnld
            This will allow you to download your IOS with this command.
    o    >xmodem
            This is horrible, you can use this to upload your IOS through the console port. Good luck with this, but maybe you might have to do this.

Quick Cheat Sheet for How a Cisco Device Boots

1.    Checks the configuration register
    a.    2100: ROMMON
    b.    2101: RXBOOT
    c.    2102: Boot Normally
    d.    2142: Ignore NVRAM
2.    Check for “BOOT SYSTEM” commands in the startup config.
3.    Looks for the first IOS image in flash.
4.    Broadcast for a TFTP server

Adding VLANs to Cisco UCSM & HyperFlex

I know there are a few more ways to do these procedures, but this is just one example in this overall SOP.

1.    First, from the UCS manager, go to the LAN section on the left hand side, then expand LAN> Policies> root>Sub-Organizations>YourDataCenter>vNIC Templates>vNIC Template vm-network-a, and vNIC Template vm-network-b

2.    Once you navigate here, you can click on the “Modify VLANs” Action under the “General” tab.

3.    On the menu on the pop-up, click on “Create VLAN”

4.    On the Create VLANs screen, fill in your VLAN Name/Prefix with a name you wish to call this VLAN, example: MyTestVLAN-100. Then put in your VLAN number, example: 100

 5.    Now make sure to go to the second “vNIC Template vm-network-b” or whatever other vm-template policies you have to ensure it added there too (it should do this automatically with linked templates, but always good to check).

6.    Oh we are not done yet folks! Now, we have the VLAN added to the vm-network templates, now we can add the VLANs to the actual servers so they can start processing the VLAN tagging. This part is easy, just check the box on the new VLAN/VLANs you wish to add to the vm-network templates.

7.    Now, in the top tabs, next to the “General” tab, you can click on VLANs and verify your VLAN has appeared and is now showing in the VLANs for the UCS vNIC Template. You can now add your Port-Group/VLAN information into the VMware vSphere and your Cisco equipment. Ensure the VLAN tag is connected all the way through the Layer 2 systems for it to fully pass through.


Reference:
https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-manager/GUI-User-Guides/Network-Mgmt/4-0/b_UCSM_Network_Mgmt_Guide_4_0/b_UCSM_Network_Mgmt_Guide_4_0_chapter_0110.html

Re-acknowledging a Blade Server in UCSM

1.    Sometimes you will need to re-acknowledge a server in order to clear alarms in the UCS/HyperFlex system. Start off by ensuring the server is in maintenance mode if it is linked in vCenter (this will help offload important VMs if in vMotion, and prevent issues in vCenter).
2.    Next, log into your UCS manager.
3.    Once you are logged into the UCS manager, navigate to the Equipment tab in the top left of the screen.
4.    From the Equipment tab, expand the following:
    a.    Equipment>Servers>Server you wish to work on.
5.    From here, click on the “Server Maintenance” in the “Actions” section. Also, I found out if you use the "reset" option it is displayed here too, a few ways you can get to it.
6.    Select “Re-Acknowledge” from the selection and then click “OK”.
7.    The process can take anywhere from 10-40 minutes depending on the system you are utilizing.
8.    Verify errors have cleared.

Reference:

https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-manager/GUI-User-Guides/Server-Mgmt/3-1/b_Cisco_UCS_Manager_Server_Mgmt_Guide_3_1/b_Cisco_UCS_Manager_Server_Mgmt_Guide_3_1_chapter_01001.html#task_DAD508E1ACC2406B81D00C1099E442EF

HyperFlex Tips & Commands

Storage Controller VM Commands

•    Sometimes the storage VMs will not start up correctly after a shutdown. The storage VMs might need to be rebooted/services started if you are having issues.
•    #service_status.sh
    o    This command will give you a status of the services running on the storage controller VM.
•    #stmgr restart
    o    This will restart the storage manager service on the controller VM. Sometimes this will stop and you will get errors in the Cisco HyperFlex Connect webpage.
•    #stcli cluster start
    o    This will start up the cluster between the nodes/servers. Sometimes the cluster is offline or not started and executing this command will bring the system back up online.
•    #stcli cluster info
    o    This command you can obtain an overall view of the cluster state. You can see replication factor, whether the cluster is online or offline, and also if the cluster is healthy.
•    #stcli cluster storage-summary
    o    This will give you the status of the storage cluster itself.
•    #stcli cluster storage-summary --detail
    o    This will give you further detail into the cluster health/status of the storage.

ESXi (Node) Commands

•    []vim-cmd vmsvc/getallvms
    o    This will show you the VMs currently running on the node/server.


References:

Cisco HyperFlex CLI Guide 3.5
https://www.cisco.com/c/en/us/td/docs/hyperconverged_systems/HyperFlex_HX_DataPlatformSoftware/CLIGuide/3_5/b_HyperFlexSystems_CLIReferenceGuide_3_5.html

Cisco HyperFlex Data Platform Administration Guide 3.5
https://www.cisco.com/c/en/us/td/docs/hyperconverged_systems/HyperFlex_HX_DataPlatformSoftware/AdminGuide/3_5/b_HyperFlexSystems_AdministrationGuide_3_5.html
 

Remove Licenses for VMware through vSphere 6.5

1.    First log on to your vSphere web interface.
2.    Next, click on the home in the top left and then click on Administration

3.    Under the Navigator pane on the left, under Administration, click on Licenses under the Licensing section.

4.    Click on the Licenses tab to view your licenses.

5.    Select a license then click on the X Remove Licenses option. Here it is unavailable at the moment because I have assigned assets to them. But this will allow you to remove the licenses from your old vCenter/vSphere.


Reference: https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vcenterhost.doc/GUID-0A6414DC-557F-4C83-A43B-0B3FBDBFA77B.html

Changing VLAN Root Bridge ID on a Layer 3 Switch

1.    First, you will need to identify the VLAN you wish to change. With Cisco switches (not sure about others yet), you can change the root bridge ID for each VLAN individually. Warning! This may/will cause a re-convergence on the network. My lab environment noticed very little issues, but again just a home lab and a medium/large network this can take down your network as everything tries to readjust. Again, all of my SOPs/tips have been done in a home lab, always follow proper procedures before performing changes on your live/home network.
2.    Make sure you design/choose your switch you wish to be root carefully. Usually, you want your big/fast switches to be the root so you don’t have your traffic handicapped at a small/slow switch. It is all depending on how you wish to design it.
3.    Once you identify your VLAN, perform the following to change the priority:
    a.    #configure terminal
        i.    #spanning-tree vlan XXX priority 4096
            1.    Numbers start from 0 and increment in the number 4096 intervals. So example: 0, 4096, 8192, 12288 etc.
        ii.    #exit
    b.    #write
4.    Sometimes this is quick, sometimes this can take a little time for the switches to readjust. Just a few simple commands, but can make a huge difference on the speed of your network.

Reference: https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2960/software/release/12-2_53_se/configuration/guide/2960scg/swstp.html