Add and Configure LUNs in Solaris

Posted by Elumalai M

Ensure the following OS packages/patches are installed

This one should be on the first Solaris install DVD

SUNWqlc

These two dowload from SUN if needed

120965-01

119131-13

# Note: If HBA's are in server when Solaris is installed these should all have been installed

VIEW HBA’s:

Once the OS is updated and a reboot -- -r has been done check if SUN/Solaris is seeing the HBA's.

Run the luxadm command:

luxadm -e port

Example:

luxadm -e port

/devices/pci@1d,700000/SUNW,qlc@2,1/fp@0,0:devctl CONNECTED

/devices/pci@1d,700000/SUNW,qlc@2/fp@0,0:devctl CONNECTED

Above is showing two HBA's 2 and 2,1

OK, good to go.

Before assigning LUN’s

Collect Current HBA information

fcinfo hba-port -l |grep HBA

HBA Port WWN: 210000e08b1c829a

HBA Port WWN: 210000e08b1c2395

Collect LUNs Solaris already knows about

fcinfo remote-port -sl -p 210000e08b0c5518 > 210000e08b0c5518.out

fcinfo remote-port -sl -p 210100e08b2c5518 > 210100e08b2c5518.out

cfgadm -al -o show_SCSI_LUN > currentLUNs.out

(Note added LUNS when available will be seen here as "unconfigured"

Scan deeply LUNs attached to each HBA

This shows HBA’s

luxadm -e port

/devices/pci@1c,600000/pci@1/SUNW,qlc@4/fp@0,0:devctl CONNECTED

/devices/pci@1c,600000/pci@1/SUNW,qlc@5/fp@0,0:devctl CONNECTED

Run following command on each Controller

luxadm -e dump_map /devices/pci@1c,600000/pci@1/SUNW,qlc@4/fp@0,0:devctl

luxadm -e dump_map /devices/pci@1c,600000/pci@1/SUNW,qlc@5/fp@0,0:devctl

Assigning LUN’s

Now is the time to assign LUNs

Assign LUNs.

Configure New LUN’s on Solaris

LUNS should “Just Show Up” on Solaris 10.

Cgfadm –al

Example output

c1::2200000c50401277 disk connected unconfigured unknown

New LUNs show as unconfigured until cfgadm is used.

When LUNs appear configure them

cfgadm -c c1::2200000c50401277
Also command can be done globally for each controller:
cfgadm -c configure c1
cfgadm -c configure c2

It does not effect previously configured LUNs.

If they do not, a few things to try if LUNs don’t show up

Check for legacy txt in sd.conf. Solaris 10 does not need this, and it just slows up booting.

Text in this file “my theory is” can mess finding new LUNs in Solaris 10.

Update sd.conf

vi /kernel/drv/sd.conf

Add new LUN IDs created on Hitachi

After Solaris 9, this should not be needed. 10 does not need this.

Instruct Solaris to re-read sd.conf

update_drv -f sd

9 and 10 can do this. 8 will failed.

Scan scsi bus so Solaris can see the new luns

devfsadm

Find new LUNs

cfgadm -al -o show_SCSI_LUN

(Note added LUNS when available will be seen here as "unconfigured"

If LUNs do not appear, server reboot will be needed.

reboot -r

Create Label for LUN

This is all only valid if you Are Not running Veritas

Now run a format command

# Note: the disk only needs to be labeled on once. The following servers only need mount it

Run format command

format

A disk looking like the following should show up.

c6t600A0B800021E8B90000536B456B26B3d0

/scsi_vhci/ssd@g600a0b800021e8b90000536b456b26b3

Select the disk number. It will need a volume name if it does not already have one.

Since this is the Export directory for Solaris "export" is a good volume name. Solaris does

not like disks without a volume name (label).

Some commands in format to look at are:

format> current

Current Disk = c6t600A0B800021E8B90000536B456B26B3d0

/scsi_vhci/ssd@g600a0b800021e8b90000536b456b26b3

(SUN and Solaris see this as a known disktype)

format> type

select type 19

format> volname

Enter 8-character volume name (remember quotes)[""]:"export"

Ready to label disk, continue? y

format> save

Saving new disk and partition definitions

Enter file name["./format.dat"]:

format> quit

This only needs to be done from one machine

Get LUN Info for Mounting

run the luxadm command to get LUN info

luxadm probe

Example:

root@j2-apps01 # luxadm probe

No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):

Node WWN:200400a0b821eab1 Device Type:Disk device

Logical Path:/dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2

Each server will have a different Path, to the same LUN

Create New Filesystem on LUN/Volume

Now create a new filesystem (format)

Run newfs command on LUN found in luxadm probe command:

newfs /dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2

Example:

root@j2-apps01 # newfs /dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2

newfs: construct a new filesystem /dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2: (y/n)? y

/dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2: 1073676288 sectors in 32766 cylinders of 512 tracks, 64 sectors

524256.0MB in 10922 cyl groups (3 c/g, 48.00MB/g, 5824 i/g)

super-block backups (for fsck -F ufs -o b=#) at:

32, 98400, 196768, 295136, 393504, 491872, 590240, 688608, 786976, 885344,

Initializing cylinder groups:

...............................................................................

............................................................

super-block backups for last 10 cylinder groups at:

1072703520, 1072801888, 1072900256, 1072998624, 1073096992, 1073195360,

1073293728, 1073392096, 1073490464, 1073588832,

Now the disk is mountable and solaris can understand it

This only needs to be done from one machine

A LUN is a physical disk to Solaris at this point

Edit /etc/vfstab to mount the new LUN

Now edit /etc/vfstab to mount the new LUN

The new LUN/disk in this example (from luxadm) is c6t600A0B800021E8B90000536B456B26B3d0s2

The device path is:

/dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2

Add the folling line to /etc/vfstab:

/dev/dsk/c6t600A0B800021E8B90000536B456B26B3d0s2 /dev/rdsk/c6t600A0B800021E8B90000536B456B26B3d0s2 /export/home ufs 1 yes logging

Edit this to match LUN ID of your system

This will mount at boot

Run mount command to test if LUN mounts:

mount /export/home

There is no output if it works

Run the mount command again to see new LUN and mount point

mount

Example:

root# mount |grep export

/export/home on /dev/dsk/c6t600A0B800021E8B90000536B456B26B3d0s2 read/write/setuid/devices/intr/largefiles/logging/xattr/onerror=panic/dev=1d80022 on Tue Nov 28 15:33:06 2006

Veritas Volume Setup

If you use Veritas follow these steps

# instruct veritas to scan for new luns

vxdctl enable

# check for new luns on veritas level

vxdisk -o alldgs list

c7t2d11s2 auto:none - - online invalid

c7t1d12s2 auto:none - - online invalid

# initialize new disks

/etc/vx/bin/vxdisksetup -i c7t2d11

# initialize new disk group with disk c3t3d30

vxdg init oraclelogs c7t2d11=c7t2d11

# if group already exists

vxdg -g oraclelogs adddisk c7t1d12=c7t1d12

# check the status of the new disks

vxdisk -o alldgs list

# Check freespace of the diskgroup

# vxdg free

# Size can be gotten from vxdg free

# make a volume of max size (41853696)

vxassist -g oraclelogs make oralogvol01 41853696

# If not all space is used, or task is to grow volume

## After making 19g check free space of new Volume

# vxassist -g oraclelogs maxgrow oralogvol01 Volume oralogvol01 can be extended by 2007040 to: 41852928 (20436Mb) # growto space available vxassist -g oraclelogs growto oralogvol01 41852928

# check the new volume

vxprint -htr

# create a filesystem on the new volume

mkfs -F vxfs /dev/vx/rdsk/oraclelogs/oralogvol01

mkfs -F vxfs /dev/vx/rdsk/oraclelogs/oralogvol02

# make a mount point

mkdir /oralog01

# mount the new filesystem at the new mount point

mount -F vxfs /dev/vx/dsk/oraclelogs/oralogvol01 /oralog01

# verify the new mounted filesystem

cd /oralog01

# verify the size on solaris level