EDIT: I eventually ran in to problems with my SR on fakeraid. So please don’t try this if you can’t afford to loose data.
In this article i want to describe my experiences with Citrix XenServer 5.0 Express and particular in combination with cheap PC hardware.
Let’s start of by describing the hardware.
Motherboard: Asus P5Q ( Intel® P45 chipset with ICH10R)
Processor: Intel Core2Quad Q9300 2.5Ghz 6MB cache FSB 1333 (Intel VT, XD)
Memory: Kingston Valueram 4GB DDR2-800 (kit of 2)
Storage: Samsung Spinpoint 750GB x2
Graphics: ATI Radeon X1300 PCI-E
Network: 3Com 3C905 PCI (onboard NIC’s not recognized)
I want to use the onboard RAID 1 to create a redundant storage repository for placing all the VM’s in.
When i started the setup by booting from the CD i only had the option to install on either of the disks.
Setup did not see or care about the RAID1 mirror i had already created trough BIOS tools.
After some research on the internet it seemed that most people would point out to use software raid instead of “Fakeraid” (this is how they call onboard firmware raid). But after researching more i found out that Citrix XenServer does not support software raid either, out of the box.
More research led me to the conclusion that fakeraid is easy to get working under linux, especially if the software (dmraid) is already present.
At least easy if the raidset is not your bootdisk.
I added an 80GB IDE drive i had lying around, to the system. I booted the CD and ran setup. I choose NOT to create a “Storage Repository” and installed on the 80GB IDE drive. After booting up the system i went to the console and typed the following command:
dmraid -ay
This gave me an output stating the raidset was active and was to be found at:
/dev/mapper/isw_ccjhgbjbcc_XEN-MIRROR
Note: XEN-MIRROR was the name i gave the raidset in the BIOS tools.
I then looked up the command for manually creating the the “Storage Repository” as LVM thus wiping the disks clean.
xe sr-create host-uuid=YOUR-OWN-UUID name-label=FAKERAID1 type=lvm content-type=user shared=false device-config-device=/dev/mapper/isw_ccjhgbjbcc_XEN-MIRROR
To get your UUID you can issue the following command:
xe host-list
After this i started my XenCenter on the windows management PC, and i saw the new SR (Storage Repository) when the right click menu, i choose this SR as default. Next i started making my virtual machines on it.
One particular challenging problem is, for instance, using a locally (to the XenServer) attached SCSI tape drive.
I read about PCI passtrough options called “pciback” and “pcifront” but this is only documented for the original old XenSource and highly unsupported by Citrix. I almost gave up, but stumbled upon a different approach. Since the tape drive is a SCSI device we can expose it as an ISCSI target with the help of some ISCSI software. I read a great articly about it here:
http://www.wlug.org.nz/XenNotes (article from Daniel Lawson). I hope the author doesn’t mind me copy/pasting his article, but i would not want the knowledge to be lost.
Making a tape drive available to a guest via iSCSI
This is specifically for Citrix XenServer, although the principles will of course work in other Xen implementations
I recently had a scenario where I was replacing two Windows servers with XenServer guests. This was fine, but we needed a way to backup to the existing SCSI DDS4 DAT drive. After failing to make PCI passthrough work, I settled on the much nicer method of providing the tape drive via an iSCSI target on the XenServer Host (Dom0). Here is how I achieved this.
Note 1: This is totally unsupported by Citrix
Note 2: I’ve used the XenServer terminology “host” instead of Dom0, as this applies to the Citrix commercial implementation of Xen. It will probably work fine on OSS Xen, but you can just install the normal kernel dev packages and ignore the DDK stuff.
Note 3: This is for XenServer 4.1.0, but the principles are the same for previous versions. Just ensure you understand each step rather than following blindly.
* Download the Xen DDK from citrix.com
* The DDK is an ISO containing a VM with a development environment. Import it to your Xen host, and start it.
* Download iscsitarget 0.4.14 (you MUST use this version, as the patch for rawio support will not cleanly apply to 0.4.15) from http://optusnet.dl.sourceforge.net/sourceforge/iscsitarget/iscsitarget-0.4.14.tar.gz into your DDK VM.
* Download the patch for rawio support from http://sourceforge.net/mailarchive/attachment.php?list_name=iscsitarget-devel&message_id=1170171101.2822.23.camel@localhost.localdomain&counter=1 into your DDK VM (assuming you’re saving it in /tmp)
* Now do the following:yum install kernel-devel bison flex
tar -zxvf iscsitarget-0.4.14.tar.gz
cd iscsitarget-0.4.14
patch -p0 < /tmp/raw.p
make* scp the entire iscsitarget-0.4.14 directory to your destination Xen host, and on that host (after enabling the base repo in /etc/yum.repos.d/CentOS-Base.repo) do:
yum install make gcc
cd iscsitarget-0.4.14
make install
mkdir /lib/modules/`uname -r`/kernel/iscsi
cp kernel/iscsi_trgt.ko /lib/modules/`uname -r`/kernel/iscsi
depmod -aqThe last three steps are required because make install will not copy the kernel module correctly outside the target environment.
* Now edit your /etc/ietd.conf and configure the tape as per the following example snippet (cat /proc/scsi/scsi for the correct HCIL values for your SCSI tape drive, this is an example only):
Target iqn.2007-04.com.example:tape0
Lun 0 H=1,C=0,I=6,L=0,Type=rawio
Type 1* Save and do /etc/init.d/iscsi-target start
* Modify /etc/sysconfig/iptables to allow port 3260 tcp from the IP addresses running the initiator.
* Attach to the target using the initiator of your choice.
More on this later, but i have it working, just not stable yet.
6 replies on “Citrix XenServer 5: SR on fakeraid (onboard Intel Raid) and tape drive trough ISCSI to windows VM”
It works, thanks, but, as long as I boot the xenserver it says that the SR is unplugged, and so far I did’t find any solution to this.
Kind regards
On my system it never said it was unplugged.
On server reboot the SR would be available and VM’s would start.
However i still am confused about what happend when things went really bad and i lost all data.
dmraid would appear NOT to be active but Xenserver was using the /dev/mapper/mirrorname and the SR was plugged and running. Was it only using 1 of the disks and not mirroring or was dmraid somehow auto started when the mirror is accessed. I have no idea.
Fact is, i highly encourage everyone NOT to use this fake raid solution.
I’m now running VmWare ESXi and i do NOT mirror the disks. Instead i use 1 disk for my VM’s and the other disk is used to store full VM backups for 2 days. I have a post up about it. It actually works perfect !
Oh and Citrix have a KB article describing how to do this:
http://66.165.176.77/article/CTX116108
And this came from Kiekeboe blog:
What to do if the device is lost after a reboot?
My work-around is to run dmraid -ay after each boot, but before XenServer mounts its Storage Repositories.
The simplest way to do this is to edit “/etc/init.d/xenservices”, and add “dmraid -ay” just after the “start() {” line.
Both without quotes of course.
The better way is to create your own init.d script, but I’m not going to explain that.
Could not find any Xen server 5.5 DDK in the Xen support site, any hits for this. I did this before, and did not work, Then upgraded XenServer 5 to 5.5 and the system crashed completely, leaving my server unusable. Logged in with an external Linux Boot (live CD) but could not find any documentation on xen about the internal file structure, nor the os to boot, the partition was cleared by the upgrader (CD burned with Xen 5.5 iso) and after wobbling a lot, I decided to format the server and install a fully new XenServer 5.5, loosing all VM’s inside the 2 HDD storage, could not find out how to save them. Citrix did not respond to the support incident nor have I a SLA or contract, so damn!. I’ll Do it all again, but now I could not find XenServer 5.5 DDK
I’m not using Xenserver anymore. I have turned over to the VmWare side.
But I think I can help a little.
After you have booted your new xenserver try this:
1) dmraid -ay
This is to activate the fakeraid and if succesful you will get a response with the path to the raid /dev/mapper/some-name-xxyyy
Write the path down.
2) pvscan
Try this to see if your SR is automatically recognized. If ok you will see an SR with the name VG_XenStorage-UUID. Note the UUID part.
3) xe sr-introduce uuid=UUID type=lvm name-label=”Local storage” content-type=user
4) xe host-list
Find the UUID of the Xenserver itself. Note this.
5) xe pbd-create sr-uuid=UUID-of-storage-see-point2 device-config:device=/dev/mapper/some-name-xxxyyy host-uuid=UUID-of-xenserver-itself-see-point4
Note the PBD UUID you will receive
6) xe pbd-plug uuid=PBD-UUID-see-point5
If all went well, your SR is back visible in XenCenter.
Hope it works for you.
To get the dmraid -ay automatically on every boot see the comments or the link i gave in a comment earlier.