Category Archives: Cisco UCS

Cisco UCS related posts

Isolated PVLANS will not work with Cisco UCS and VMware vDS

This is something  I came up against several years ago but never got around to posting it, the workaround is to deploy a Nexus1000v, the PVLANs will be defined within the Nexus and will never traverse the upstream network.

Here’s the response from Cisco

“In a nutshell in End Host mode on the Fabric Interconnects have no unknown unicast flooding functionality and does not learn Mac addresses on the uplinks.

Because the Vmware DVS cannot terminate the PVLANs they will need to extend into the external LAN switching infrastructure.
Therefore, all community/isolated VLANs have to be defined on UCS and on the external LAN switch(es) as well.

This is fine if no communication is required between the isolated PVLAN and any external host on the Primary VLAN.

Where the design requires an external promiscuous port then you need to set the UCS Fabric Interconnects in switch mode. That is traffic that enters the promiscuous port is classified in the primary VLAN. Therefore from a UCS perspective there are no server-side MAC-table entries in the primary VLAN because servers are in an isolated PVLAN. So no communication is possible.

As such, switch mode is a must for bi-directional communication. Here the fabric interconnects will do Mac-Learning on the uplink ports as well as the server ports.”

 

Cisco UCS and SolidFire ISCSI Boot

Note: At the time of writing Cisco has not certified booting UCS blades from Solidfire ISCSI boot disks.

In order to successfully boot a Cisco UCS B Series blade from a SolidFire ISCSI array you will need to create a custom ESXi ISO that includes a firewall rule to open ports 3261-3264.

Explanation :

It seems that Solidfire arrays will present volumes on ports 3260-3264, so you may get one server to see it’s boot disk without issue but the next one will not work.  The ESXi firewall by default is opened on port 3260 only, there is no rule for 3261,3262,3263 and 3264 so traffic coming back from the Solidfire array is blocked and as a result you will not see the bootdisk.

Workaround

1. Create a custom VIB that opens firewall ports, you will find a good guide here, alternatively PM me for 5.1 firewall bundle.

2. Create a custom ISO using powercli

Add-EsxSoftwareDepot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

Add-EsxSoftwareDepot  d:\temp\firewall3261-offline-bundle.zip

Get-EsxImageProfile | Sort-Object “ModifiedTime” -Descending | format-table -property Name,CreationTime 

New-EsxImageProfile -CloneProfile ESXi-5.1.0-20130402001-standard -Name SFBoot-ESXi-5.1.0-20130402001-standard -AcceptanceLevel CommunitySupported

Add-EsxSoftwarePackage -ImageProfile SFBoot-ESXi-5.1.0-20130402001-standard -SoftwarePackage firewallrule

Export-EsxImageProfile -ImageProfile SFBoot-ESXi-5.1.0-20130402001-standard -ExportToISO -FilePath d:\temp\SFBoot-ESXi-5.1.0-20130402001-standard.iso

UCS Service Profile Template creation for SolidFire ISCSI Boot

Note: At the time of writing Cisco has not certified booting UCS blades from Solidfire ISCSI boot disks.

UCS Pre-Requisites:

1. VNIC Templates created

2. UUID Pool Created

3. Boot Policy Created  (can be created during service profile template creation)

4. Maintenance Policy Created

5.  Authentication Policy  Created (used to authenticate with the ISCSI target, can be created during service profile template creation )

6. Initiator IP address policy created (IP addresses assigned to the ISCSI vNICs)

7. Create dedicated ISCSI VLAN, use dedicated uplinks for the ISCSI VLAN

Create Service Profile Template

ISCSI 1

VNIC Creation Select ‘Expert’

ISCSI2

Click add to create the iscsi vnics from vnic templates

ISCSI3

Create ISCSI VNICS, click ‘add’ in the lower pane

ISCSI4

Name the vnic

choose the overlay vnic previously created

do not set iscsi adapter policy

Select ISCSI VLAN

Do not select MAC Address Assignment

ISCSI5

Should look like this

ISCSI6

No VHBA’s

ISCSI7

Leave default click next

ISCSI8

Leave default

ISCSI9

Create boot policy if not already created

ISCSI13

Select manintenance policy if required, recommend using user ack!

ISCSI14

Leave defaults

ISCSI15

Create Policies as required

ISCSI16

Service Profile template is created…

Create service profile from template

ISCSI17

Once created select the new Service Profile  in the left pane and browse to ‘boot order’ in the tab selections

ISCSI18

Select the ISCI vNIC, you will NOT be able to modify the ‘set iscsi boot parameters’ as the service profile is bound to a template.

Work around is to unbind the service profile from it’s template, Cisco say they will fix this, no date as yet!

Go to general tab and click ‘unbind’, now you will be able to modify iscsi boot parameters

ISCSI18

Choose authentication profile for accessing the iscsi target

Select IQN pool

Select Initator IP address pool, create if not there.

ISCSI20

Create iscsi target

Enter the IQN ID provided to you by the storage team

Do not select authentication profile

Enter IP address of ISCSI target.

ISCSI21

You are now ready to boot from the SolidFire Array, my next post will go through booting the ESXi host!

Passwordless SSH between the UCS and a remote Linux Server

1. On the remote Linux/Unix server create the user

useradd -m ucsuser -c “UCS user” -d /home/ucsuser

2. Change the directory permissions

chown ucsuser /home/ucsuser

3. Generate SSH key for the new user

ssh-keygen (accept defaults and leave passphrase empty)

4. Copy the public Key to a text file.

cd .ssh/

cat id_rsa.pub

5. SSH to the UCS Manager, at the CLI type the following commands

scope security

create local-user ucsuser

set password

6. Copy the Public Key obtained in step 4 and paste it into the CLI with inverted commas.

set sshkeyssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAuo9VQ2CmWBI9/S1f30klCWjnV3lgdXMzO0WUl5iPw85lkdQqap+NFuNmHcb4K

iaQB8X/PDdmtlxQQcawclj+k8f4VcOelBxlsGk5luq5ls1ob1VOIEwcKEL/h5lrdbNlI8y3SS9I/gGiBZ9ARlop9LDpD

m8HPh2LOgyH7Ei1MI8=

commit-buffer

Now you should be able to SSH from the remote linux server to the UCS without entering a password.

Schedule Cisco UCS Backups

A quick and easy ‘expect’ script that runs from cron on a nightly basis, it’s not the most graceful script you’ll ever see, but it gets the job done.

I understand that scheduled backups are available in Cisco UCS release 2.1

1. Create a ‘backup’ user on the UCS and the remote Unix/Linux Server, in our case we used the VMA as the backup location.

2. Configure passwordless ssh between the UCS and the remote server, more details can be found here.
 
3. Copy the script below to the remote Linux/Unix server and put it the home directory of the the ‘backup’ user created in step 1.

cat /home/bckup/ucsbackup_full.sh

#!/usr/bin/expect -f

# Expect script to run full backup of the UCS

# Set Variables

set UCS 10.255.2.3 #UCS VIP Address
set DESTPASSWD ucsb@ckup   
set DESTIP 10.255.1.2 # IP Address of remote Unix/Linux Server
set DESTUSER backupuser
set DESTDIR /var/ftp/pub/ucsbackups/

 # Connect to UCS

spawn ssh $UCS

# Delete previous night backup as UCS allows only one object to exist

 send   “scope system\n”

send   “delete backup $DESTIP\n”

send   “commit-buffer\n”

send   “exit\n”

# Run New Backup and SCP to the shared area on the VMA

 send   “scope system\n”

send   “create backup scp://$DESTUSER@$DESTIP$DESTDIR full-state enabled \n”

expect “Password:”

send   “$DESTPASSWD\n”

send   “commit-buffer\n”

send   “exit\n”

expect  eof

4. Edit the crontab of the ‘backup’ user and schedule it to run on Sunday morning at 3am.

crontab -l
0 3 * * 7 /home/bckup/ucsbackup_full.sh > /home/bckup/full.txt