Tag Archives: linux

Tutorials: Installing Windows Vista / XP on Xen

6 Jul

Tutorials: Installing Windows Vista / XP  on Xen

I ran across this tut this morning while doing research for my at home setup. Pretty straight forward, so I’m sharing for yours and my reference.

In this chapter we will work through steps involved in installing and running Windows XP and Windows Vista as virtualized Xen HVM guests.

Pre-Requisites for Installing a Windows Xen Guest

This chapter assumes that the Xen host (domain0) system is running a Xen enabled kernel and that the standard Xen tools (specifically xm) are also installed. In addition, full virtualization requires that the host system hardware include either Intel-VT or AMD-V virtualization support. In order to verify this run the following commands on the host system:

For Intel CPUs:

grep vmx /proc/cpuinfo

For AMD CPUSs:

grep svm /proc/cpuinfo

If neither of the above command produce output then it is likely these features are unavailable and that your hardware is unable to run Xen HVM guests. Whilst you can still run modified operating systems (such as Linux) as paravirtualized guests you will not be able to virtualize Windows using Xen. It is also possible that the host CPU supports one of these features, but that the functionality is disabled by default in the system BIOS. To verify this, access the host system’s BIOS configuration menu during the boot process and look for an option related to virtualization support. If such an option exists and is disabled, enable it, boot the system and repeat the above commands.

Preparing to Install Windows

In order to install Windows as a Xen HVM guest the first requirement is disk space. The space assigned to a Windows guest may be either a physical disk drive attached to the host system or a disk image. To create a disk image simply use the dd command. For example, the following command line created a 6Gb disk image file called xenwin.img:

dd if=/dev/zero of=xenwin.img bs=1024k seek=6144 count=0

Preparing the Windows XP / Windows Vista for Xen HVM Installation

Installation will require a Windows XP or Vista installation DVD. Installation may be performed directly from the DVD drive, or the DVD image may be extracted from the DVD to an ISO image residing on the filesystem of the host. To extract the ISO image from the DVD insert it into a DVD drive, unmount it if it is auto-mounted and execute the following command (substituting /dev/hdb for the matching device on your chosen Linux distribution):

dd if=/dev/hdb of=Windows.iso

Once the copy is completed the DVD can be removed from the drive and stored safely, and the Windows.iso file used to perform the installation.

[edit] Configuring the Xen Windows Guest Configuration File

Once the disk or disk image is available the next step is to create a suitable Xen HVM configuration file. The following lines are standard and should be used unchanged for all HVM configurations:

import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'

kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

Next the amount of memory to be assigned to the Windows XP or Windows Vista guest needs to be declared, in this case 512Mb:

memory = 512

Next, the amount of shadow memory may be defined. This should be equal to 2KB per MB of domain memory, plus a few MB per vcpu. In general 8Mb is sufficient:

shadow_memory = 8

The domain name and networking choices now need to be made. If you wish to bridge to the networking device on the host system use the following vif setting:

name = "xenhvm"

vif = [ 'type=ioemu, bridge=xenbr0' ]

Now, the most important part of the configuration is to specify the disk and CD/DVD settings. If a physical disk and CD/DVD drive are to be used this setting will read something like:

disk = [ 'phy:/dev/sdb,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]

The above line maps physical disk drive /dev/sdb on the host to /dev/hda on the guest and /dev/hdc on the host as the cdrom device on the guest (note that the CD/DVD device name on your host may be different depending on your Linux distribution).

Alternatively, if a disk image file is to be used as the disk drive the line might instead read:

disk = [ 'file:/home/xen/xenwin.img,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]

Alternatively, if you have opted to use an ISO image of the Windows XP or Windows Vista installation DVD this can be referenced in place of the CD/DVD device:

disk = [ 'file:/home/xen/xenwin.img,hda,w', 'file:/home/xen/Windows.iso,hdc:cdrom,r' ]

Next the boot order of the guest needs to be defined. Clearly the CDROM device needs to be first in the boot order, at least until Windows is installed on the disk drive:

# boot on floppy (a), hard disk (c) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
boot="dc"

Some configuration settings now need to be made to enable a graphical console for our Windows guest operating system. Xen provides a choice of VNC or SDL for supporting a graphical console when the guest is running. For example the following settings select VNC:

vnc = 1
sdl = 0

whilst the following selects SDL:

vnc = 0
sdl = 1

Both SDL and VNC work very well in terms of displaying Windows in a graphical console, although VNC has some distinct advantages over SDL. Firstly, VNC provides greater flexibility than SDL in terms of remote access to the domainU graphical console. With VNC it is possible to connect to the graphical console from other systems, either on the local network or even over the internet. Secondly, when you close a VNC viewer window the guest domain continues to run allowing you to simply reconnect to carry on where you left off. Closing an SDL window, however, immediately terminates the guest domainU system resulting in possible data loss.

By default Xen does not automatically start the VNC console when the domainU guest starts up. In order to have the graphical console spawned automatically, change the vncconsole = value to 1:

vncconsole=1

After selecting the graphical display options a number of other settings are required and/or recommended:

acpi = 1
apic = 1
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
stdvga=0
serial='pty'

One final setting is needed to avoid problems with the mouse pointer:

/read: Installing and running windows vista / xp as a Xen HVM DomU Guest

n this chapter we will work through steps involved in installing and running Windows XP and Windows Vista as virtualized Xen HVM guests.

Contents

[hide]

// //
// //

//

[edit] Pre-Requisites for Installing a Windows Xen Guest

This chapter assumes that the Xen host (domain0) system is running a Xen enabled kernel and that the standard Xen tools (specifically xm) are also installed. In addition, full virtualization requires that the host system hardware include either Intel-VT or AMD-V virtualization support. In order to verify this run the following commands on the host system:

For Intel CPUs:

grep vmx /proc/cpuinfo

For AMD CPUSs:

grep svm /proc/cpuinfo

If neither of the above command produce output then it is likely these features are unavailable and that your hardware is unable to run Xen HVM guests. Whilst you can still run modified operating systems (such as Linux) as paravirtualized guests you will not be able to virtualize Windows using Xen. It is also possible that the host CPU supports one of these features, but that the functionality is disabled by default in the system BIOS. To verify this, access the host system’s BIOS configuration menu during the boot process and look for an option related to virtualization support. If such an option exists and is disabled, enable it, boot the system and repeat the above commands.

[edit] Preparing to Install Windows

In order to install Windows as a Xen HVM guest the first requirement is disk space. The space assigned to a Windows guest may be either a physical disk drive attached to the host system or a disk image. To create a disk image simply use the dd command. For example, the following command line created a 6Gb disk image file called xenwin.img:

dd if=/dev/zero of=xenwin.img bs=1024k seek=6144 count=0

[edit] Preparing the Windows XP / Windows Vista for Xen HVM Installation

Installation will require a Windows XP or Vista installation DVD. Installation may be performed directly from the DVD drive, or the DVD image may be extracted from the DVD to an ISO image residing on the filesystem of the host. To extract the ISO image from the DVD insert it into a DVD drive, unmount it if it is auto-mounted and execute the following command (substituting /dev/hdb for the matching device on your chosen Linux distribution):

dd if=/dev/hdb of=Windows.iso

Once the copy is completed the DVD can be removed from the drive and stored safely, and the Windows.iso file used to perform the installation.

[edit] Configuring the Xen Windows Guest Configuration File

Once the disk or disk image is available the next step is to create a suitable Xen HVM configuration file. The following lines are standard and should be used unchanged for all HVM configurations:

import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'

kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

Next the amount of memory to be assigned to the Windows XP or Windows Vista guest needs to be declared, in this case 512Mb:

memory = 512

Next, the amount of shadow memory may be defined. This should be equal to 2KB per MB of domain memory, plus a few MB per vcpu. In general 8Mb is sufficient:

shadow_memory = 8

The domain name and networking choices now need to be made. If you wish to bridge to the networking device on the host system use the following vif setting:

name = "xenhvm"

vif = [ 'type=ioemu, bridge=xenbr0' ]

Now, the most important part of the configuration is to specify the disk and CD/DVD settings. If a physical disk and CD/DVD drive are to be used this setting will read something like:

disk = [ 'phy:/dev/sdb,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]

The above line maps physical disk drive /dev/sdb on the host to /dev/hda on the guest and /dev/hdc on the host as the cdrom device on the guest (note that the CD/DVD device name on your host may be different depending on your Linux distribution).

Alternatively, if a disk image file is to be used as the disk drive the line might instead read:

disk = [ 'file:/home/xen/xenwin.img,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]

Alternatively, if you have opted to use an ISO image of the Windows XP or Windows Vista installation DVD this can be referenced in place of the CD/DVD device:

disk = [ 'file:/home/xen/xenwin.img,hda,w', 'file:/home/xen/Windows.iso,hdc:cdrom,r' ]

Next the boot order of the guest needs to be defined. Clearly the CDROM device needs to be first in the boot order, at least until Windows is installed on the disk drive:

# boot on floppy (a), hard disk (c) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
boot="dc"

Some configuration settings now need to be made to enable a graphical console for our Windows guest operating system. Xen provides a choice of VNC or SDL for supporting a graphical console when the guest is running. For example the following settings select VNC:

vnc = 1
sdl = 0

whilst the following selects SDL:

vnc = 0
sdl = 1

Both SDL and VNC work very well in terms of displaying Windows in a graphical console, although VNC has some distinct advantages over SDL. Firstly, VNC provides greater flexibility than SDL in terms of remote access to the domainU graphical console. With VNC it is possible to connect to the graphical console from other systems, either on the local network or even over the internet. Secondly, when you close a VNC viewer window the guest domain continues to run allowing you to simply reconnect to carry on where you left off. Closing an SDL window, however, immediately terminates the guest domainU system resulting in possible data loss.

By default Xen does not automatically start the VNC console when the domainU guest starts up. In order to have the graphical console spawned automatically, change the vncconsole = value to 1:

vncconsole=1

After selecting the graphical display options a number of other settings are required and/or recommended:

acpi = 1
apic = 1
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
stdvga=0
serial='pty'

One final setting is needed to avoid problems with the mouse pointer:

usbdevice='tablet'

Bringing all these settings together gives us a minimal Xen HVM configuration file for a Windows guest which looks similar to the following:

import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'

kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

memory = 512
shadow_memory = 8
name = "xenhvm"
vif = [ 'type=ioemu, bridge=xenbr0' ]
disk = [ 'file:/home/nas/xen/xenguest.img,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
# boot on floppy (a), hard disk (c) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
boot="dc"

sdl=0
vnc=1
vncconsole=1
vncpasswd=''
stdvga=0
serial='pty'
usbdevice='tablet'

A number of other configuration options are available to modify the guest environment but the above settings are the minimal settings recommended to run a Windows guest with Xen. Once the configuration file is complete save it as xenwin.cfg.

[edit] Starting the Xen Windows HVM Guest

With the configuration complete the next step is to start up the Xen guest environment:

xm create xenwin.cfg
Using config file "./xenwin.cfg".
Started domain xenhvm

After a few seconds the VNC or SDL console (depending on the configuration settings) will appear and the Windows installation process will begin. The following figure shows the installation running in a VNC window :

Windows installation viewed using VNC

Going back to ubuntu

15 Sep

So before I even start this, let me say I haven’t been much of a Ubuntu fan. Theres more than enough posts (hopefully I’ll get them back up soon) where I rant about Ubuntu and my frustration with the OS. I’ve never really been a fan of ubuntu, part of it has to do with rackspace, part of it ubuntu in general, but mostly to be completely honest is preference. That Said

Why I’m switch back to ubuntu

It all started with our servers. First I moved a few servers over to ubuntu because in the end it was easier. On top of community support and more up to date packages, apt kicks yum’s ass. Next more of our servers moved to ubuntu for various reasons, leaving only 2 servers using centos. If 10 of 12 servers are running ubuntu: It’s time to go all in.

Server side I grew up on redhat, but ubuntu has finally won me over. While I’m used to doing things the redhat way I’m finding less and less support for redhat as the community and vendors openly embrace Ubuntu.

Goodbye openSuse

Before I start the Ubuntu love fest let me still say openSuse Rocks! But sometimes I don’t want to compile or spend hours / days finding out how to run something obcure thats fully supported or heavily documented with Ubuntu. Yeah finding solutions once was the part of the fun, but I’m getting old and lazy, sometimes I just want a quick fix (a few hours max).

D-day

New years I’m planning to spend the day (after recovering from the hang over) moving all my boxes over to ubuntu, but hopefully I’ll get a few more openSuse related posts in before that. So good bye openSuse – Hello Ubuntu. For better or worst, the community and vendors have spoken.

Sid

video: time to party like it's 1234567890

15 Feb

From: http://revision3.com/content/1234567890/

A time like this can only happen on one planet Earth. Why? Well, because only on planet Earth would a species base its numerical system by a factor of 10 that is tied to the rotation of its planet on its axis (and around its sun). Although every individual number has a uniqueness quantification, only a species like us humans would find the existential irony of 1,234,567,890 seconds.

Check It Out:
Revision3

fix: why is my volume so low? linux

9 Feb

recently the max volume on my laptop for some reason shifted to a lower setting. the fix?

sudo alsamixer

how to: boot a vmware machine via command line

19 Dec

I’m using vmware 6, so I don’t know if this works in 5 or server editions.

vmware installs a script called vmrun that allows you to perform almost anything you need to do on a vmware machine pragmatically.

sidney@my-host:~> /usr/bin/vmrun
vmrun version 6.0.4 build-93057

Usage: vmrun COMMAND [PARAMETERS]

Authentication flags
-gu
-gp

POWER COMMANDS           PARAMETERS           DESCRIPTION
start                    Path to vmx file     Start a VM or Team
                         or vmtm file
                         [gui|nogui]
stop                     Path to vmx file     Stop a VM or Team
                         or vmtm file
                         [hard|soft]
reset                    Path to vmx file     Reset a VM or Team
                         or vmtm file
                         [hard|soft]
suspend                  Path to vmx file     Suspend a VM or Team
                         or vmtm file
                         [hard|soft]

SNAPSHOT COMMANDS        PARAMETERS           DESCRIPTION
listSnapshots            Path to vmx file     List all snapshots in a VM
snapshot                 Path to vmx file     Create a snapshot of a VM
                         Snapshot name
deleteSnapshot           Path to vmx file     Remove a snapshot from a VM
                         Snapshot name
revertToSnapshot         Path to vmx file     Set VM state to a snapshot
                         Snapshot name

GUEST OS COMMANDS        PARAMETERS           DESCRIPTION
runProgramInGuest        Path to vmx file     Run a program in Guest OS
                         Program
                         [Program arguments]
fileExistsInGuest        Path to vmx file     Check if a file exists in Guest OS
                         Path to file in guest
setSharedFolderState     Path to vmx file     Modify a Host-Guest shared folder
                         Share name
                         New host path
addSharedFolder          Path to vmx file     Add a Host-Guest shared folder
                         Share name
                         Host path
removeSharedFolder       Path to vmx file     Remove a Host-Guest shared folder
                         Share name
listProcessesInGuest     Path to vmx file     List running processes in Guest OS
killProcessInGuest       Path to vmx file     Kill a process in Guest OS
                         process id
runScriptInGuest         Path to vmx file     Run a script in Guest OS
                         Interpreter path
                         script_text
deleteFileInGuest        Path to vmx file     Delete a file in Guest OS
                         File
createDirectoryInGuest   Path to vmx file     Create a directory in Guest OS
                         Directory path
deleteDirectoryInGuest   Path to vmx file     Delete a directory in Guest OS
                         Directory path
listDirectoryInGuest     Path to vmx file     List a directory in Guest OS
                         Directory path
copyFileFromHostToGuest  Path to vmx file     Copy a file from host OS to guest OS
                         Path on host
                         Path in guest
copyFileFromGuestToHost  Path to vmx file     Copy a file from guest OS to host OS
                         Path in guest
                         Path on host
renameFileInGuest        Path to vmx file     Rename a file in Guest OS
                         Original name
                         New name

GENERAL COMMANDS         PARAMETERS           DESCRIPTION
list                                          List all running VMs
upgradevm                Path to vmx file     Upgrade VM file format, virtual hw
installtools             Path to vmx file     Install Tools in Guest OS

Examples:
  vmrun list
  vmrun upgradevm w2k.vmx
  vmrun reset w2k.vmx soft
 

Yippie!

how to: use random numbers in shell scripts

18 Dec

Ever wish you could use a random number in your shell scripts? Luckily you can:

my_random_number=$RANDOM

linux returns a random number everytime the $RANDOM variable is accessed.

facebook shares their memcached patches and linux tweaks

12 Dec

Facebook so rocks, they have posted not only their memcached patches, but also explain what they needed to change in linux to properly utilize them. facebook +1, myspace – well they still suck

From: http://www.facebook.com/note.php?note_id=39391378919

If you’ve read anything about scaling large websites, you’ve probably heard about memcached. memcached is a high-performance, distributed memory object caching system. Here at Facebook, we’re likely the world’s largest user of memcached. We use memcached to alleviate database load. memcached is already fast, but we need it to be faster and more efficient than most installations. We use more than 800 servers supplying over 28 terabytes of memory to our users. Over the past year as Facebook’s popularity has skyrocketed, we’ve run into a number of scaling issues. This ever increasing demand has required us to make modifications to both our operating system and memcached to achieve the performance that provides the best possible experience for our users.

Check It Out:
Engineering @ Facebook’s Notes

how to: write shell scripts

10 Dec

This is imho the best guide for shell scripting. I’m always referencing it when writing a shell script ( was it -n or -z?)

From: http://linuxcommand.org/writing_shell_scripts.php#contents

With the thousands of commands available for the command line user, how can you remember them all? The answer is, you don’t. The real power of the computer is its ability to do the work for you. To get it to do that, we use the power of the shell to automate things. We write scripts.

Scripts are collections of commands that are stored in a file. The shell can read this file and act on the commands as if they were typed at the keyboard. In addition to the things you have learned so far, the shell also provides a variety of useful programming features to make your scripts truly powerful.

What are scripts good for? A wide range of tasks can be automated. Here are some of the things I automate with scripts:

Check It Out:
LinuxCommand.org: Writing shell scripts.

project: benchsid – automate benchmarks with siege, ab, and httperf using shell scripts

7 Dec

While benchmarking performance for deathy death match I realized something: Benchmarking sucks!

That said, and me being the lazy person I am, I asked myself how can I make this easier? My answer: benchsid, two shell scripts that make benchmarking easier.

disclaimer

This is a 1,2,3 trick pony – use it as such

These scripts are offered as is and I am not responsible for anything you do, create, break, etc.

Both scripts are released under the do whatever the hell you want, but don’t blame me license. This of course isn’t a real license, but you get the point.

Make benchmarks easier, if only a bit

I’ve tried as much as possible to keep your commands to a minimum. If running as root you can run a complete benchmark, save both the results and you benchmark servers top + free output, and reboot with 3 commands. See lazy right? :)

monitor.sh – run this script on the machine your benchmarking
bench.sh – run this script on the machine your benchmarking from

you do not need to run these on separate machines, but I recommend it

How do I use…

monitor:

 ./monitor.sh
 ./monitor.sh help

bench:

 ./bench.sh
 ./bench.sh help

If your still having issues post a comment

monitor.sh

Monitor should be ran on the machine your benchmarking. It automates the process of recording free (free -m) and top, and if running as root will reboot the server when you tell it to stop monitoring a benchmark

Sounds good so far right?

monitor.sh process overview

start process

clear out any benchmarks that will be overwritten
save output from free to a file (free -m)
save output from top to a file
start a top process to monitor to a file while we benchmark
confirm that we are up and running

stop process

stop top
save output of free to file
save output of top to file
reboot server

monitor.sh: umm why?

monitor.sh saves the information about the server before, during and after your benchmark has been run. By running top during the benchmark process (10 second intervals), we can get a real time overview of both server load and the loads created by applications

bench.sh

bench.sh is the benchmarking portion. bench.sh can benchmark your server using one of three programs: ab, siege, or httpref. I considered adding an option to do all three, but honestly that more poor mans dos than benchmark.

bench.sh first will perform a warm-up cycle of 2000 requests to your server using your desired (or default) concurrency. After that it will perform 3 benchmark cycles of your desired number of requests and concurrency.

All cycle results (including warm up) are saved to the filesystem.

download

download benchsid: http://gregsidberry.com/httpdocs/wp-content/uploads/2008/12/benchsid.tar.gz

known issues

siege output – siege outputs a couple of variables to screen that aren’t included, or properly recorded, in the created file. I’ve tried output redirection, script, etc, but haven’t been able to find a fix that keeps everything in one script

input validation – there is no input validation, so please don’t put these files in a web public location.

funyuns – I’m out of funyuns, please send more

enjoy
Sid

news: china forces internet cafes to use government controlled operating system

3 Dec

Requirements that Internet cafes in a southern Chinese city install Chinese-developed operating systems are raising new concerns over cyber snooping by authorities, a U.S. government-funded radio station reported Wednesday.

The new rules that went into effect Nov. 5 are aimed at cracking down on the use of pirated software, said Hu Shenghua, a spokesman for the Culture Bureau in the city of Nanchang.

Internet cafe operators are required to remove unlicensed software and replace it with legitimate copies of either Microsoft Windows or China’s homegrown Red Flag Linux operating system while paying a fee, he said.

However, Radio Free Asia said cafes were being required to install Red Flag Linux even if they were using authorized copies of Windows. It quoted Xiao Qiang, director of the California-based China Internet Project, as saying the new rules would help authorities regulate Internet cafes that now operate on the margins of the law, and allow them to undertake heightened surveillance.

China Internet cafes switching to Chinese OS – Salon.com