Wednesday, December 30, 2009

Fast way to unencapsulate rootdisk on Veritas

1. boot single user mode alternate boot device like cdrom, spare disk or net
ok> boot cdrom -s

2. fsck the root filesystem
# fsck -y /dev/rdsk/c0t0d0s0 (assuming root disk is c0t0d0)

3. mount the root filesystem on /a
# mount /dev/dsk/c0t0d0s0 /a

4. cp /a/etc/system /a/etc/system.vxvm

5. vi /a/etc/system and remove the following 2 entries

rootdev:/pseudo/vxio@0:0
set vxio:vol_rootdev_is_volume=1

6. cp /a/etc/vfstab /a/etc/vfstab.vxvm

7. cp /a/etc/vfstab.prevm /a/etc/vfstab

8. rm /a/etc/vx/reconfig.d/state.d/root-done

9. touch /a/etc/vx/reconfig.d/state.d/install-db

10. reboot

Monday, December 28, 2009

df: cannot canonicalize .: Permission denied

I often use Solaris Live Upgrade to run quarter patching process in one of my customers. It's a very safe way to patch your system and you also have an untouched disk available (just in case you need to back out).

 

During this year, we've experienced a "not-well-known" issue with Live Upgrade. If you go to a directory (it usually happens with /var) and try the command below, you get a permission denied message (it seems to happen only with older versions of LU):

 

$ sudo pkginfo -l SUNWluu
   PKGINST:  SUNWluu
      NAME:  Live Upgrade 2.0 10/01 (usr)
  CATEGORY:  application
      ARCH:  sparc
   VERSION:  11.8,REV=2001.10.30.17.21

$ cd /var
$ df -k .
df: cannot canonicalize .: Permission denied

 

This happens because Live Upgrade doesn't do a proper copy of the mount-point directory's permission (it usually does a chmod 750 on the mount-point dir, instead of a 755). It also denies a normal user to list/create files on the file system (can impact some apps to work properly).

 

File system directory un-mounted would look like this:

 

$ ls -ld /var
drwx------  41 root     sys         1024 Mar 16  2009 /var
$

 

We have opened a case with Sun to help us troubleshooting (unfortunately they didn't help us finding a permanent solution for this issue).

 

So, based on the history above I decided to develop by myself a permanent fix (feel free to use/suggest any new idea you might have):

 

1 - This issue only happens with Solaris 8 and/or 9 (haven't seen any similar issue on S10).

 

2 - If you want to permanently avoid this (and other issues) when you are starting to create the NewBE, I suggest you to install the Solaris 10 version of Live Upgrade on your Solaris 8/9 (you can find the packages in the following directory tree)

 

Solaris_10/Product/SUNWlur

Solaris_10/Product/SUNWluu

 

Copy the whole directory tree (use tar) to your server and remove the old packages:

 

pkgrm SUNWluu SUNWlur

 

Now, install the new packages:

 

pkgadd –d . SUNWluu SUNWlur

 

 

Common errors that happens using older versions of Live Upgrade:

 

cpio: Error with fstatat() of "opt/ecc/exec/MSR520/diskqueue/SST/GMACBR120.00101806c8be_ENW.que", errno 2, No such file or directory

 

cpio: Cannot open "./usr/emc/API/symapi/ipc/storwatchd", skipped, errno 122, Operation not supported on transport endpoint

cpio: pathname is too long

 

 

3 – If you want to be 100% safe from this issue (also after upgrading your Live Upgrade), you can use the following simple script that my co-worker developed together with myself:

 

#!/sbin/sh

#

# Script to fix the issue regarding the mountpoint permissions.

#

 

 

case "$1" in

'start')

        echo "Fixing mountpoint permissions..."

        for i in `grep -v "^#" /etc/vfstab| awk '{print $3}'| egrep -v '^-$|^/tmp$|^/$|^/proc$|^/dev/fd$' `

        do

        echo $i

        chmod u+rwx,g+rx,o+rx  $i

        done

        ;;

 

*)

        echo "Usage: $0 { start  }"

        exit 1

        ;;

 

esac

exit 0

 

 

Copy this output and save it to a file on /etc/init.d (we named it as mountpoint_fix_permissions).

 

Then, do the following to make this script always run when you reboot the server:

 

# chmod 755 /etc/init.d/mountpoint_fix_permissions

# ln /etc/init.d/mountpoint_fix_permissions /etc/rcS.d/S65mountpoint_fix_permissions

 

# ls -li S65mountpoint_fix_permissions /etc/init.d/mountpoint_fix_permissions

    122723 -rwxr--r--   2 root     other        383 Dec 19 19:44 /etc/init.d/mountpoint_fix_permissions

    122723 -rwxr--r--   2 root     other        383 Dec 19 19:44 S65mountpoint_fix_permissions

 

 

This script will grep /etc/vfstab looking for file system paths and will chmod 755 all of them before they get mounted (to avoid the issue).

 

This is what will happen when the script runs:

 

VxVM starting special volumes ( swapvol rootvol var )...

Fixing mountpoint permissions...

/var

/apps

/u001

/u003

/u010

/u011

/u012

/u013

/u014

/u099

/u100

/u110

/u510

 

Now you should be all set to avoid complaints from your customer :-)

 

PS: This issue can sometimes impact on application’s behavior (so make sure that you have it fixed).

 

Bug Reference:

 

http://forums.sun.com/thread.jspa?threadID=5065412

 

This is bug #4697677