Lights out

We lost power while we were out on Thursday and did not get power again until about noon on Saturday. The spare box I built the router in was not setup in bios to power on after a power outage and the xen box (owl) did not come back up.

The biggest problem on owl was a combination of selinux and a xen bug, so I upgraded the kernel and moved the virtual machines to /var/lib/xen/images where selinux thinks they should exist. I did create a soft link to /xen so the configs would work. On an up note, the VMs now start on boot correctly which had been a problem.

My brother was caring for our dog while we traveled last week. Oreo was on a chain with a clasp and someone stole her while my parents were out and my brother was sleeping. Now I have two crying girls – my wife and daughter – and my 2 year old son doesn’t understand that she is gone and not coming back.

If you care to read the extended version of my upgrade and relinking it is in the “More…”

Detailed Description
    SELinux denied xen access to /. If this is a XEN image it has to have a file
    context label of xen_image_t. The system is setup to label image files in
    /var/lib/xen/images correctly.  We recommend that you copy your image file
    to this directory. If you really want to have your xen image files in this
    directory, you can relabel the / to be a xen_image_t file/directory using
    chcon.  If you do this you should also execute semanage fcontext -a -t
    xen_image_t $TATGET_PATH to add this new path to the system defaults.If you
    did not intend to use / as a xen image it could indicate either a bug or an
    intrusion attempt.

Allowing Access

    You can alter the file context by executing chcon -t xen_image_t /

    The following command will allow this access:

    chcon -t xen_image_t /

Shutdown all xen VMs

[root@owl /]# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
/dev/mapper/VolGroup00-LogVol03 on /home type ext3 (rw)
/dev/mapper/VolGroup00-LogVol02 on /tmp type ext3 (rw)
/dev/mapper/VolGroup00-LogVol01 on /var type ext3 (rw)
/dev/mapper/VolGroup00-LogVolStore01 on /xen type ext3 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
[root@owl /]# vi /etc/fstab
#/dev/VolGroup00/LogVolStore01 /xen                    ext3    defaults        1 2
/dev/VolGroup00/LogVolStore01 /var/lib/xen/images       ext3    defaults        1 2

[root@owl /]# umount /xen
[root@owl /]# rmdir /xen
[root@owl /]# mount -a
[root@owl /]# ln -s /var/lib/xen/images /xen

I am getting a number of selinux denials and the networking is not working. On the console I am getting the error “xen_net: memory squeeze in netback driver”. Searching indicated this was a bug that was fixed. I need to update the kernel and xen tools.

I made the mistake earlier of running the chcon in the selinux dialog and it changed all the selinux contexts on the root / and upon rebooting the system no longer functioned. I added “selinux=0″ to the kernel line and found that the kernel options were actually being passed on the module line immediately following the kernel line. This is out of the ordinary, but at least the system booted successfully. After getting the system up I used system-config-security to disable selinux.

[root@owl boot]# ls -l
total 14798
-rw-r--r-- 1 root root   61057 Jul 10  2007 config-2.6.18-8.1.8.el5xen
-rw-r--r-- 1 root root   61053 Mar 15  2007 config-2.6.18-8.el5xen
drwxr-xr-x 2 root root    1024 Aug  6  2007 grub
-rw------- 1 root root 2330839 Aug  6  2007 initrd-2.6.18-8.1.8.el5xen.img
-rw------- 1 root root 2348337 Aug  6  2007 initrd-2.6.18-8.1.8.el5xenU.img
-rw------- 1 root root 2330731 Aug  7  2007 initrd-2.6.18-8.el5xen.img
lrwxrwxrwx 1 root root      37 Aug  6  2007 initrd-2.6-xenU.img -> /boot/initrd-2.6.18-8.1.8.el5xenU.img
drwx------ 2 root root   12288 Aug  7  2007 lost+found
-rw-r--r-- 1 root root   80032 Apr  1  2007 message
-rw-r--r-- 1 root root   84906 Jul 10  2007 symvers-2.6.18-8.1.8.el5xen.gz
-rw-r--r-- 1 root root   84906 Mar 15  2007 symvers-2.6.18-8.el5xen.gz
-rw-r--r-- 1 root root  868084 Jul 10  2007 System.map-2.6.18-8.1.8.el5xen
-rw-r--r-- 1 root root  868062 Mar 15  2007 System.map-2.6.18-8.el5xen
-rw-r--r-- 1 root root 2076151 Jul 10  2007 vmlinuz-2.6.18-8.1.8.el5xen
-rw-r--r-- 1 root root 2074835 Mar 15  2007 vmlinuz-2.6.18-8.el5xen
lrwxrwxrwx 1 root root      33 Aug  6  2007 vmlinuz-2.6-xenU -> /boot/vmlinuz-2.6.18-8.1.8.el5xen
-rw-r--r-- 1 root root  274228 Jul 10  2007 xen.gz-2.6.18-8.1.8.el5
-rw-r--r-- 1 root root  274722 Mar 15  2007 xen.gz-2.6.18-8.el5
-rwxr-xr-x 1 root root  608568 Jul 10  2007 xen-syms-2.6.18-8.1.8.el5
-rwxr-xr-x 1 root root  608564 Mar 15  2007 xen-syms-2.6.18-8.el5
[root@owl boot]# yum update kernel*
=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 kernel-xen              i686       2.6.18-53.1.21.el5  updates            14 M
Updating:
 kernel-headers          i386       2.6.18-53.1.21.el5  updates           789 k
Removing:
 kernel-xen              i686       2.6.18-8.el5     installed          35 M

Transaction Summary
=============================================================================
Install      1 Package(s)
Update       1 Package(s)
Remove       1 Package(s)

Removed: kernel-xen.i686 0:2.6.18-8.el5
Installed: kernel-xen.i686 0:2.6.18-53.1.21.el5
Updated: kernel-headers.i386 0:2.6.18-53.1.21.el5
Complete!
initrd-2.6-xenU.img -> /boot/initrd-2.6.18-8.1.8.el5xenU.img
vmlinuz-2.6-xenU -> /boot/vmlinuz-2.6.18-8.1.8.el5xen
[root@owl ~]# cd /boot
[root@owl boot]# mkinitrd --with=xennet --with=xenblk /boot/initrd-2.6.18-53.1.21.el5xenU.img `uname -r`
[root@owl boot]# ln -sf initrd-2.6.18-53.1.21.el5xenU.img initrd-2.6-xenU.img
[root@owl boot]# ln -sf vmlinuz-2.6.18-53.1.21.el5xen vmlinuz-2.6-xenU
[root@owl boot]# shutdown -r now
[root@owl boot]# yum update
=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 sos                     noarch     1.7-9.2.el5      updates           108 k
     replacing  sysreport.noarch 1.4.3-10.el5

Updating:
...

Installing for dependencies:
 device-mapper-multipath  i386       0.4.7-12.el5_1.4  updates           2.0 M
 dnsmasq                 i386       2.39-2.el5       base              150 k
 keyutils-libs           i386       1.2-1.el5        base               18 k
 yum-metadata-parser     i386       1.0-8.fc6        base               22 k

Transaction Summary
=============================================================================
Install      5 Package(s)
Update     169 Package(s)
Remove       0 Package(s)

Total download size: 234 M

[root@owl boot]# shutdown -r now

All of the VMs started and email is working again.

Leave a Reply

You must be logged in to post a comment.