Tag Archives: windows

Tutorials: Installing Windows Vista / XP on Xen

6 Jul

Tutorials: Installing Windows Vista / XP  on Xen

I ran across this tut this morning while doing research for my at home setup. Pretty straight forward, so I’m sharing for yours and my reference.

In this chapter we will work through steps involved in installing and running Windows XP and Windows Vista as virtualized Xen HVM guests.

Pre-Requisites for Installing a Windows Xen Guest

This chapter assumes that the Xen host (domain0) system is running a Xen enabled kernel and that the standard Xen tools (specifically xm) are also installed. In addition, full virtualization requires that the host system hardware include either Intel-VT or AMD-V virtualization support. In order to verify this run the following commands on the host system:

For Intel CPUs:

grep vmx /proc/cpuinfo

For AMD CPUSs:

grep svm /proc/cpuinfo

If neither of the above command produce output then it is likely these features are unavailable and that your hardware is unable to run Xen HVM guests. Whilst you can still run modified operating systems (such as Linux) as paravirtualized guests you will not be able to virtualize Windows using Xen. It is also possible that the host CPU supports one of these features, but that the functionality is disabled by default in the system BIOS. To verify this, access the host system’s BIOS configuration menu during the boot process and look for an option related to virtualization support. If such an option exists and is disabled, enable it, boot the system and repeat the above commands.

Preparing to Install Windows

In order to install Windows as a Xen HVM guest the first requirement is disk space. The space assigned to a Windows guest may be either a physical disk drive attached to the host system or a disk image. To create a disk image simply use the dd command. For example, the following command line created a 6Gb disk image file called xenwin.img:

dd if=/dev/zero of=xenwin.img bs=1024k seek=6144 count=0

Preparing the Windows XP / Windows Vista for Xen HVM Installation

Installation will require a Windows XP or Vista installation DVD. Installation may be performed directly from the DVD drive, or the DVD image may be extracted from the DVD to an ISO image residing on the filesystem of the host. To extract the ISO image from the DVD insert it into a DVD drive, unmount it if it is auto-mounted and execute the following command (substituting /dev/hdb for the matching device on your chosen Linux distribution):

dd if=/dev/hdb of=Windows.iso

Once the copy is completed the DVD can be removed from the drive and stored safely, and the Windows.iso file used to perform the installation.

[edit] Configuring the Xen Windows Guest Configuration File

Once the disk or disk image is available the next step is to create a suitable Xen HVM configuration file. The following lines are standard and should be used unchanged for all HVM configurations:

import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'

kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

Next the amount of memory to be assigned to the Windows XP or Windows Vista guest needs to be declared, in this case 512Mb:

memory = 512

Next, the amount of shadow memory may be defined. This should be equal to 2KB per MB of domain memory, plus a few MB per vcpu. In general 8Mb is sufficient:

shadow_memory = 8

The domain name and networking choices now need to be made. If you wish to bridge to the networking device on the host system use the following vif setting:

name = "xenhvm"

vif = [ 'type=ioemu, bridge=xenbr0' ]

Now, the most important part of the configuration is to specify the disk and CD/DVD settings. If a physical disk and CD/DVD drive are to be used this setting will read something like:

disk = [ 'phy:/dev/sdb,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]

The above line maps physical disk drive /dev/sdb on the host to /dev/hda on the guest and /dev/hdc on the host as the cdrom device on the guest (note that the CD/DVD device name on your host may be different depending on your Linux distribution).

Alternatively, if a disk image file is to be used as the disk drive the line might instead read:

disk = [ 'file:/home/xen/xenwin.img,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]

Alternatively, if you have opted to use an ISO image of the Windows XP or Windows Vista installation DVD this can be referenced in place of the CD/DVD device:

disk = [ 'file:/home/xen/xenwin.img,hda,w', 'file:/home/xen/Windows.iso,hdc:cdrom,r' ]

Next the boot order of the guest needs to be defined. Clearly the CDROM device needs to be first in the boot order, at least until Windows is installed on the disk drive:

# boot on floppy (a), hard disk (c) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
boot="dc"

Some configuration settings now need to be made to enable a graphical console for our Windows guest operating system. Xen provides a choice of VNC or SDL for supporting a graphical console when the guest is running. For example the following settings select VNC:

vnc = 1
sdl = 0

whilst the following selects SDL:

vnc = 0
sdl = 1

Both SDL and VNC work very well in terms of displaying Windows in a graphical console, although VNC has some distinct advantages over SDL. Firstly, VNC provides greater flexibility than SDL in terms of remote access to the domainU graphical console. With VNC it is possible to connect to the graphical console from other systems, either on the local network or even over the internet. Secondly, when you close a VNC viewer window the guest domain continues to run allowing you to simply reconnect to carry on where you left off. Closing an SDL window, however, immediately terminates the guest domainU system resulting in possible data loss.

By default Xen does not automatically start the VNC console when the domainU guest starts up. In order to have the graphical console spawned automatically, change the vncconsole = value to 1:

vncconsole=1

After selecting the graphical display options a number of other settings are required and/or recommended:

acpi = 1
apic = 1
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
stdvga=0
serial='pty'

One final setting is needed to avoid problems with the mouse pointer:

/read: Installing and running windows vista / xp as a Xen HVM DomU Guest

n this chapter we will work through steps involved in installing and running Windows XP and Windows Vista as virtualized Xen HVM guests.

Contents

[hide]

// //
// //

//

[edit] Pre-Requisites for Installing a Windows Xen Guest

This chapter assumes that the Xen host (domain0) system is running a Xen enabled kernel and that the standard Xen tools (specifically xm) are also installed. In addition, full virtualization requires that the host system hardware include either Intel-VT or AMD-V virtualization support. In order to verify this run the following commands on the host system:

For Intel CPUs:

grep vmx /proc/cpuinfo

For AMD CPUSs:

grep svm /proc/cpuinfo

If neither of the above command produce output then it is likely these features are unavailable and that your hardware is unable to run Xen HVM guests. Whilst you can still run modified operating systems (such as Linux) as paravirtualized guests you will not be able to virtualize Windows using Xen. It is also possible that the host CPU supports one of these features, but that the functionality is disabled by default in the system BIOS. To verify this, access the host system’s BIOS configuration menu during the boot process and look for an option related to virtualization support. If such an option exists and is disabled, enable it, boot the system and repeat the above commands.

[edit] Preparing to Install Windows

In order to install Windows as a Xen HVM guest the first requirement is disk space. The space assigned to a Windows guest may be either a physical disk drive attached to the host system or a disk image. To create a disk image simply use the dd command. For example, the following command line created a 6Gb disk image file called xenwin.img:

dd if=/dev/zero of=xenwin.img bs=1024k seek=6144 count=0

[edit] Preparing the Windows XP / Windows Vista for Xen HVM Installation

Installation will require a Windows XP or Vista installation DVD. Installation may be performed directly from the DVD drive, or the DVD image may be extracted from the DVD to an ISO image residing on the filesystem of the host. To extract the ISO image from the DVD insert it into a DVD drive, unmount it if it is auto-mounted and execute the following command (substituting /dev/hdb for the matching device on your chosen Linux distribution):

dd if=/dev/hdb of=Windows.iso

Once the copy is completed the DVD can be removed from the drive and stored safely, and the Windows.iso file used to perform the installation.

[edit] Configuring the Xen Windows Guest Configuration File

Once the disk or disk image is available the next step is to create a suitable Xen HVM configuration file. The following lines are standard and should be used unchanged for all HVM configurations:

import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'

kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

Next the amount of memory to be assigned to the Windows XP or Windows Vista guest needs to be declared, in this case 512Mb:

memory = 512

Next, the amount of shadow memory may be defined. This should be equal to 2KB per MB of domain memory, plus a few MB per vcpu. In general 8Mb is sufficient:

shadow_memory = 8

The domain name and networking choices now need to be made. If you wish to bridge to the networking device on the host system use the following vif setting:

name = "xenhvm"

vif = [ 'type=ioemu, bridge=xenbr0' ]

Now, the most important part of the configuration is to specify the disk and CD/DVD settings. If a physical disk and CD/DVD drive are to be used this setting will read something like:

disk = [ 'phy:/dev/sdb,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]

The above line maps physical disk drive /dev/sdb on the host to /dev/hda on the guest and /dev/hdc on the host as the cdrom device on the guest (note that the CD/DVD device name on your host may be different depending on your Linux distribution).

Alternatively, if a disk image file is to be used as the disk drive the line might instead read:

disk = [ 'file:/home/xen/xenwin.img,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]

Alternatively, if you have opted to use an ISO image of the Windows XP or Windows Vista installation DVD this can be referenced in place of the CD/DVD device:

disk = [ 'file:/home/xen/xenwin.img,hda,w', 'file:/home/xen/Windows.iso,hdc:cdrom,r' ]

Next the boot order of the guest needs to be defined. Clearly the CDROM device needs to be first in the boot order, at least until Windows is installed on the disk drive:

# boot on floppy (a), hard disk (c) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
boot="dc"

Some configuration settings now need to be made to enable a graphical console for our Windows guest operating system. Xen provides a choice of VNC or SDL for supporting a graphical console when the guest is running. For example the following settings select VNC:

vnc = 1
sdl = 0

whilst the following selects SDL:

vnc = 0
sdl = 1

Both SDL and VNC work very well in terms of displaying Windows in a graphical console, although VNC has some distinct advantages over SDL. Firstly, VNC provides greater flexibility than SDL in terms of remote access to the domainU graphical console. With VNC it is possible to connect to the graphical console from other systems, either on the local network or even over the internet. Secondly, when you close a VNC viewer window the guest domain continues to run allowing you to simply reconnect to carry on where you left off. Closing an SDL window, however, immediately terminates the guest domainU system resulting in possible data loss.

By default Xen does not automatically start the VNC console when the domainU guest starts up. In order to have the graphical console spawned automatically, change the vncconsole = value to 1:

vncconsole=1

After selecting the graphical display options a number of other settings are required and/or recommended:

acpi = 1
apic = 1
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
stdvga=0
serial='pty'

One final setting is needed to avoid problems with the mouse pointer:

usbdevice='tablet'

Bringing all these settings together gives us a minimal Xen HVM configuration file for a Windows guest which looks similar to the following:

import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'

kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

memory = 512
shadow_memory = 8
name = "xenhvm"
vif = [ 'type=ioemu, bridge=xenbr0' ]
disk = [ 'file:/home/nas/xen/xenguest.img,hda,w', 'phy:/dev/hdb,hdc:cdrom,r' ]
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
# boot on floppy (a), hard disk (c) or CD-ROM (d)
# default: hard disk, cd-rom, floppy
boot="dc"

sdl=0
vnc=1
vncconsole=1
vncpasswd=''
stdvga=0
serial='pty'
usbdevice='tablet'

A number of other configuration options are available to modify the guest environment but the above settings are the minimal settings recommended to run a Windows guest with Xen. Once the configuration file is complete save it as xenwin.cfg.

[edit] Starting the Xen Windows HVM Guest

With the configuration complete the next step is to start up the Xen guest environment:

xm create xenwin.cfg
Using config file "./xenwin.cfg".
Started domain xenhvm

After a few seconds the VNC or SDL console (depending on the configuration settings) will appear and the Windows installation process will begin. The following figure shows the installation running in a VNC window :

Windows installation viewed using VNC

Dear windows… Why the **** did I try you out again?

18 Sep

So I got lazy a few months ago and installed windows because I didn’t wanta take the time to do a mythtv setup and didn’t know if I wanted to switch back to Ubuntu or upgrade to the lastest version of openSuse. Well that and silverlight on linux still won’t work with netflix. Ok pretty much the only reason I’m still on vista is netflix. Yeah there’s boxee, but as already mentioned I’m being lazy till I get back to cali.

Anyways there’s a point to this post

Great moments of WTF

So there is a Vista bug that causes the entire OS to crash when using a USB drive. It took about 3-10 crashes a day and and 10 minutes on google to confirm it wasn’t just in my head. Anytime my seagate terabyte drive starts getting decent read / writes going, the entire OS crashes… Umm wtf. Thinking about it, I think that explains why Windows 7 kept crashing as well…

Are you friggin kidding me

So far I’m able to replicate the issue with a seagate, maxtor, and western digital drive. All I have to do is get some torrents going, while converting a video file to dvd, and I suddenly remember why I finally went 100% linux on all my boxes. So the question of course is, how the hell can an OS be this frggin buggy? On linux I’ve downloaded more torrents faster, while doing alot more other operations on a slower external with no issues, and I’ve never even heard of a USB Drive crashing a mac. Is it just me or does windows suddenly seem more backwards than usual?

exaile! on windows

18 Jul

I’m really liking exaile! and have been recommending it to friends. of course their first question is , can it run on mac / windows?

well as far as windows yes : heres a few links to help you run exaile! on windows :

screen shots : http://potatosaladx.blogspot.com/2007/01/exaile-029b-on-windows.html
Patch : http://www.exaile.org/trac/ticket/200

news: china forces internet cafes to use government controlled operating system

3 Dec

Requirements that Internet cafes in a southern Chinese city install Chinese-developed operating systems are raising new concerns over cyber snooping by authorities, a U.S. government-funded radio station reported Wednesday.

The new rules that went into effect Nov. 5 are aimed at cracking down on the use of pirated software, said Hu Shenghua, a spokesman for the Culture Bureau in the city of Nanchang.

Internet cafe operators are required to remove unlicensed software and replace it with legitimate copies of either Microsoft Windows or China’s homegrown Red Flag Linux operating system while paying a fee, he said.

However, Radio Free Asia said cafes were being required to install Red Flag Linux even if they were using authorized copies of Windows. It quoted Xiao Qiang, director of the California-based China Internet Project, as saying the new rules would help authorities regulate Internet cafes that now operate on the margins of the law, and allow them to undertake heightened surveillance.

China Internet cafes switching to Chinese OS – Salon.com

framework: progress – alpha 0.02

21 Sep

So after years of partial frameworks, and various concepts, I’ve finally started my framework.

First off there is NO windows support. I’m sure that’s not the best way to go, but I can’t think of one reason to host a php app on windows. As noted windows isn’t supported, but I’m working on a linux installer that will handle installation of third party apps, that the framework allows you to use.

I was hoping to use the framework in its current form on a project, but due to deadline concerns its only used partially. I’m hoping to setup a test server for security testing, so if you like expoliting stay tuned.

So here’s an overview of whats done:

  • automatic cleaning of all GET / POST / COOKIE data.
  • extremely extendable
  • OOP (of course)
  • database sessions
  • geo locations
  • user system
  • html purifier integration
  • session hijacking prevention
  • xss / sql injection protection (still being worked on / 60-70%)
  • caching system
  • made to scale
  • media conversion

There is still much to be done before I release a beta, but thought I’d get try and get some feedback. This framework is a combination of a few of my previous concepts / framework ideas:

  1. bacon – security features are all taken from the bacon framework concept, the encryption features are not fully integrated yet
  2. color – old framework idea from my joop media days
  3. evo – framework concept for virtual worlds

The main goal of this framework is to have a simple framework that doesn’t force you to things this way or that way. Something that gives you the basic and a base to work from.

right now the only draw backs I see with the system are the number of db queries used (caching system also handles db caching), and right now it’s speed isn’t we’re i want it.

If you have any ideas of features to add in, let me know.

Anyways I’ll hopefully have a beta ready in a few months.

microsoft: our os sucks so badly we need to trick people into thinking its cool

6 Sep

I’m watching tv, only to see a commercial where windows is market testing a new os. “Holy crap” , I thought to myself, “they realized zista is shite and are already releasing a new os, thats gonna piss off some people”, but a smart move since as mentioned, vista is shite.

Then again, I’d never heard of the code name they used. Well guess waht it was vista.

So I’ve decided due to my all nighter, lack of sleep, and for everyones amusement, to tell you how my demo would have went.

I’m running a core2duo 2ghz with 4gigs of ram. I dual boot into vista 64 from time to time because, well actually i have not sane reason: I just do.


Loading vista: hmm looks kinda sweet
login screen: nice look
login: is this slower than xp?
system startup: crap this is slow

-I should note I only have firefox, antivirus, vmware, a firwall, pidgin, and the adobe web / creative suite installed. This is an install thats at best 2 months old, and rarely used.

start firefox: system hangs, kill firefox
start firefox: system hangs, wait 10 minutes

start vmware centos image: system degrades to p4 (pre ht) running xp, and under heavy load
check load : cpu 100%, ram 60%,

say crap why did I load into vista

reboot

opensuse 11 loads

I start up an xp vmachine with 313mb of ram, centos5 vmachine with 256 of ram, firefox, evolution, utorrent, gimp, pidgin, exalie, and a bunch of folders (forgot to mention opening my downloads folder almost crashes vista) including my downloads folder. Oh yeah I should mention I normally have 10-30 tabs open in firefox3.

open flash cs3, dreamweaver cs3, and virtual folders in xp

check my stats: cpu1 26%, cpu2 31%, ram 41%, swap 0%, upload 84k.

wonder why an xp machine thats barely using my cpu, and that has almost no ram out performs vista…..

and now to code, and eventually find sleep

media: Everybody needs sox

5 Sep

Most of us love / hate / use ffmpeg. Well meet your other new best friend: Sox

From the site:

Welcome to the home of SoX, the Swiss Army knife of sound processing
programs. SoX is a cross-platform (Windows, Linux, Solaris, etc.)
command line utility that can convert various formats of computer audio
files in to other formats. It can also apply various effects to these
sound files, and, as an added bonus, SoX can play and record audio
files on many major platforms.

Sox allows you to do almost anything you could want to do with audio.

Need to build an audio file from user selected tracks? no problem.

Mix together 15 different files? no problem.

You get the point. If your using ffmpeg, then I recommend you also install sox. It’s fairly simple to use, actually its easier to use than ffmpeg IMHO.

http://sox.sourceforge.net/

linux: openSuse compared after some time

17 Aug

Its been over a month, so i figured its time to do an after action report.

Simply put I still LOVE openSuse. Of course theres a few things that need work, but overall its a wonderful distro that I strongly recommend.

Ok so for a bit of back story I’ve been using windows since i think 3.1. When i was a kid my dad taught me to use the green screens, and i’ve been using linux off and on since about 2000. Also I’ve rarely used kde.

Umm so wheres the review

Sorry if your expecting stats, screen shots, etc. Its is pretty much nothing more than opinion. Take it as you like, I’m just hoping to share my experience – hopefully helping someone else find their ideal os sooner.

4 machines – similar experiences

I’ve installed opensuse on the 4 machines: a new toshiba core2duo laptop, a p4 (pre ht) dell laptop, an amd x2 hp desktop, and a p4 ht dell desktop.

my experience has been pretty consistent acroos all machines with the following exceptions:

  1. There is no audio on the dell laptop. I’m sure this could be fixed, but the dell is more of a toy than something i actually use.
  2. Wifi refuses to work during initial setup on the x2. I’ve used both a linksys pci card and belkin usb device. Both are detected, but no signals are detected. As an fyi I’m able to stabily connect to my network from 50-80 (rough estimate )  feet from my place, so there are no signal issues here.
  3. Media support can be a bit annoying to get up and running.
  4. firefox / internet seems to run slower

now on to detials

Wifi

The wifi support in openSuse is by far awesome, for the most part it puts windows wireless support to same, and of course any and all of my past experiences with wireless on linux.

On the p4 dell I plugged in a wireless card that on ubuntu doesn’t even get reconigzed. Guess what? It worked, no drivers to install, no new device settings, it was up running and displaying my network quickly.

I have yet to have any wireless device not install and display in my connections list. This is a big plus for me as I’ve had bad experiences with wifi on ubuntu.

networking

I have yet to actually setup my network (sorry I’ve been lazy), but as far as getting a basic network setup it’s been no problem.

Theres really nothing to say here other than it works, I haven’t setup myth servers, file servers, etc. I do however have lamp servers running with no issues (disclaimer: I only use centos for development / production lamp servers).

Graphics

You knwo how they say a cool comp won’t help you pick up chicks? Their right, but it sure as hell looks cool when you bring someone over.

Compiz + emerald work damn near flawless. I’m not a gamer so i tend to get your basic video card. I’ve had no issues with eye candy. And to all you mac users: yes my eye candy kicks your eye candy in the arse.

I have the majority of my rendering set to medium levels and only experience lags when recording desktop.

You do not need a top of the line graphics to enjoy good eye candy. Running awn, compiz, and emerald barely taxes any of my systems except the dell p4 laptop which only has 512mb of ram.

Also of note: DUAL SCREEN SUPPORT! yes i know thats in caps, but seriously why can’t everyone handle dual screens like this. I briefly tried ubuntu 8.01 after a few months of 7.10 and honestly opensuse jsut feels more natural to me. Yes i prefer openSuse’s dual screen support to windows.

Eye candy

As already mentioned eye candy is everywhere in openSuse, imho they have the best layouts / designs for the bootscreen, login, everything, etc.

Speed

Fast – that simple, I experience occasional program hangs, and in total 3-4 system hangs, but then again i was messing with stuff.

So compare something already

Here is a list of os and what they do better than opensuse. Understand I’m comparing this way because it’s easier to just point out what another os does better, than to point out everything it does worst.

mint linux : media support, newbie friendly

Seriously I don’t think there’s another distro thats handling media support better than mint linux right now. That said the same support can be installed on openSuse fairly easily.

ubuntu : newbie friendly, media support

mint linux is based on ubuntu, so of course ubuntu is gonna beat openSuse in the same areas. media support is better on mint linux, but again open suse can be setup witht he same media support.

Where i think opensuse loses the majority of its newbie friendly points is yast. Yes yast is powerful, but dependency support is lacking. While ubuntu allows a newer linux user to quickly get a feel, we all know where and when its lacking so I’ll leave that alone. If openSuse improves their package management, denpency support, and loses the one-click installs for something less annoying, imho it will be on par with ubuntu. I should however point out that the stability of packages has not been a problem at all on openSuse as it was on ubuntu.

fedora : nada

I haven’t touched fedora in years, but based on feedback, theres no need to compare. If your looking at fedora look at either ubuntu or openSuse. also does anyone remember when fedora was the centos of linux?

slackware: old school

openSuse is slackware based, but much easier to use. Slackware however has more of the old school linux feel.

centos: anything lamp

centos imho is the only os to host on (well other than rhel). So simply put its better as a lamp envirnoment. Most software is tested on rhel / centos so you’ll have less install issues on centos. Lastly centos has been the most stable hosting os i’ve used. I left ubuntu after 3 days when i tried using it as a hosting envirnoment

rhel: see centos

mandrake linux:

to me openSuse just feels more linuxy – i know thats not a word but hopefully it makes since

closing thoughts

sorry I’m not really big on testing everything, just testing till you find what works best. There are plenty of new and old distros that may work for you. look around, google and ask around. openSuse might not be the best distro for you, but i highly recommend it to everyone. If your a developer that deals with lamp i highly recommend you try openSuse. I’m basing this on the fact that i’m a lamp developer.

That said my ex roommate was a windows network admin, he loved ubuntu, and prob will love opensuse. Speaking of which I should shoot him an email.

elsid Out

how to: Fix utorrent hidden issue on ubuntu

20 Jun

I have utorrent running via wine, when suddenly it stops displaying…

remove the settings.dat and settings.dat.old from {wine}/c_drive/windows/profiles/{user}/Application Data/utorrent

install the latest version. uncheck hide to task bar, close to taskbar

cheers
greg

php security in a nutshell

9 May

I have a friend I’m teaching foundation security to. This post is for him, but also as a protest to some of the materials I’ve found when looking for reference material for him.

Security at it’s simplest form is common sense. ask yourself, how can I make sure I get exactly what I want? How do I make sure I only give what I want. One article mentions xss attacks, and only says prevent them. Why? Thats the question alot of people have when starting why? So why not teach them how to do it first?

How to avoid sql injection / xss, and other misc attacks.

As mentioned this is part rant, part helpful. I’ll explain the following tips and why / how you do it.

  • always use require_once, or include_once. why? it keeps someone from getting your files stuck in loops.
  • clean everything that calls, enters, looks at your db.
  • typecast whenever you expect a certain type of variable.
  • control access and check permissions
  • use your own sessions
  • track everything in some form
  • setup php correctly
  • hide whats not to be seen / accessed

first off let me say I’m by no means a security god. Actually I’m not even an advanced user. Sad as it is maybe to say: I’ve never used pear. that said, the majority of attacks / exploits can be easily avoided. Why? because the majority of attacks on the web don’t come from hackers they come from script kiddies. We can be lax with our own stuff ( like this blog ), but any application you build for a client should at least have the basics.

Enough ranting now to the meat and tators…. I’ll keep everything short and sweet. fyi – this is pretty much a brain dump, so prob not in “good form”.

why do we use the _once functions?

if you have a file that loads another file, say index.php?get=/calender.php

what happens if someone changes get to /index.php? yeap your suck in a loop, unless you use require_once / include_once

simple huh?

State changes

Your first question is prob, what the hell is a state change? a state change is simply any change, anytime you change something, whether in the db, a file, an upload: it should always use post. why? Post can be hacked yes, but it’s harder to hack post.

imagine we’re using an online game
ex: update=1&user=87897&add_money=8.

so any user who can add will know: hey i can change add_money to 100 and gain 100 points. On top of that any user can now see all your get vars. Why does that matter? The less they know about your vars, the harder it is for a kiddie / developer to exploit it?

why else? It makes it easier to validate changes. Why? Honestly I don’t even remember why right now, but hopefully you won’t hold that against me

all users are evil

I know kinda overzealous, but you need to have this mindset, why? users will accidentally mess up your system every chance they get. And script kiddies love telling you how l33t they are if they do something as simple as figure out how to make a game page display a different page.

as for making a game page display a different page, honestly: who cares ( yes that was me venting). But in order to prevent accidents, or worst kiddie hacks, control everything! I’m not saying make your app so restrictive that users hate it, but – actually an example would fit best.

today the team made a new flash widget that requires user data.
ex1: pass user data as flash vars, then use loadvars to pass update to server

or more secure
ex2: use loadvars to recieve user data from server, and then pass update to server with loadvars.

in this example we could have honestly used either way as the update script validates all data and any real changes are driven off the database, not the state change, but you get the point. Its one less thing worry about if a user decides to try and change the vars passed, and also one less file to update if we change something.

Users will enter strings when numbers should be entered, upload swf’s when you only want images – you get the point. And the point is validate, and whenever possible take the control from the frontend and move it to the backend.

Be as lazy as possible

I say I am a smart lazy person. building 30 different files takes more time than building one file, and using includes, or a template structure. Pretty lazy huh? but also easier to update and more secure. The more files you have that each have their own independent / copy + pasted code, the more opportunities you have for a slip up. Make one file, and let it handle the logic. You’ll have more freetime, and get more sleep. Or maybe you’ll just spend that time working on more projects. See being lazy is a good thing, but only if done correctly.

we can take this a step further and say why even ftp into the server, it’s so time consuming. why not just build a backend that not only manages your files, but controls access to them – Thats more of a teaser than anything, but try it out, you’ll like the results.

typecast whenever possible

imagine we’re using an online game
ex: update=1&user=87897&add_money=8

ok so what if someone changes add_money to a delete statement, or attempts some form of sql injection. whats the simplest way to defeat it? $money=(integer) $_GET[' add_money'];

Yeap one simple change is all it took to defeat the sql injection. why? Typecasting is basically a way of forcing something to be something. huh? if i want a value to always be an integer, i use (integer). If I want a double i use (double), string (string).

Yeah it’s that simple. the only issue i’ve run into is that you can’t use typecasting in defining function / method params. huh?

ex: function foo((integer)$f=0)

that will cause an error, but you can do

ex: function foo($f=0){

$f=(integer)$f;
}

Make sense? of course I can’t force something to (mocha frap with extra mocha) $coffee, but thats life. good now on to more, or learn more about typecasting

Validate, validate, validate

using typecasting is great for numbers, but theres other ways to validate your data. the best and most powerful being regular expressions

ex: preg_replace(‘/[^a-z0-9]/i’,”,$value);

The above regex replaces any non alpha numeric characters in value. spend sometime getting comfortable with regex as its an extremely powerful and useful feature. Not just for validating data, but regex has many other uses as well.

Be a neat freak, or cleaning your sql

By now you understand sql injection, if not

ok so now we all understand it. basically its a cool way of saying, someones trying to make my query do bad things, but saying it like that would make me should like a user, so we say sql injection and confuse the heck out of clients :p

we just saw how to prevent one form of sql attack. now lets see how we can handle preventing them at the query level.

Whenever data is sent to your db it should always be cleaned. Me I like to make sure both the table, columns are cleaned using a function that makes sure tables / column names follow a standard, and a cleaning function for actual data. Why? When developing an app from scratch you normally have freedom over how tables, columns are named. I prefer to keep all tables and columns lower case, and only allow _ as a special char (non alpha numeric character). what does it look like?

ex: //convert name to proper db format
function dbProperObjectName($objectName){
//if you want to use caps is table / column names then please uncomment this
$objectName=strtolower($objectName);
return @preg_replace(“/([\\x00-\\x2d\\x3a-\\x40\\x5b-\\x60\\x7b-\\xff{$this->mSystemDatabase['restricted_chars']}\\x2f])/e”, ‘_’, $objectName);
}

You can ignore the {$this->mSystemDatabase['restricted_chars']} thats some carry over from the db class. If you don’t understand what heck that says I’ll explain. first I’m changing $objectName to all lowercase, if it’s not already. then we’re using a regular expression (regex) to clean our string of anything thats is not a letter or number and replacing it with _. why does this matter? because if for any reason our table name contains a sql injection, when ran it will only return nothing. why? because if $objectName was SELECT * FROM HOME, it will now be select___from_home. which will return nothing because select___from_home isn’t a valid table. See and you thought cleaning wasn’t fun.

Ok you do windows, but what else?

As much fun as cleaning a table name maybe, we really need to make sure our data is safe. why? ummm because I say so. There are many reasons, ranging from controlling content, preventing xss, sql injection. But I like to think you’ll do it because users are evil :)

ex: //strip bad things from a string you plan to use in a query
function dbFriendlyValue($value=false,$fixNewlines=true,$allowedTags=[pass your list of allowed tags here]){
//if no value then just return 0, use this because empty returns false if $value =0
if($value===false) return 0;

//convert to string for checking, this is fine for text / numeric values
$value=(string)$value;

//strip slashes if magic quotes enabled
if( get_magic_quotes_gpc() ) $value = stripslashes( $value );

//clear white space
$value=trim($value);

//fix \r\n
$value=str_replace(“\r\n”, “\n”, $value);

//clear tags (except allowed) or just use html entities
$value=(!empty($allowedTags)) ? strip_tags($value,$allowedTags) : htmlentities($value, ENT_QUOTES);

//change newlines to <br>
if($fixNewlines) $value=nl2br($value);

//clear any bad sql we might find untested regex
$value=@preg_replace(‘/(insert(\s?)into|\).(\s?)values.(\s?)\(|DELETE.(\s?)FROM|CREATE.(\s?)[datbsetl]{5,8}|alter.(\s?)[datbsetlcoumn]{5,8}|drop.(\s?)[datbsetlcoumn]{5,8}|update.\s?(.*?).\s?set|alter.(\s?)[datbsetlcoumn]{5,8})/i’,”,$value);

//add slashes
$value=(@mysql_real_escape_string($value)) ? @mysql_real_escape_string($value) : addslashes($value);

return $value;
}

woah what the hell was that? it was me doing the windows and the oven. lets break it down

when calling the function we pass the value, whether to fix newlines ( default : true) , and the string containing allowed tags if any.
next we make sure we have a value to clean, if not return 0, just in case the function is being used to create a sql statement. we check for magic quotes because if this value came from a submitted variable and magic quotes is on, it will add slashes. if its on the strip those slashes so we can continue.

i’ll skip trim and str_replace, now we’re at strip tags. php is pretty good at striping tags, but you want another option use htmlentities( $value, ENT_QUOTES)

and now on to our regex. this is untested ( sorry still building the class ), but points you in the right direction. the regex searches the value for any sql statements and strips them. lastly we add slashes to our value to make its sql / db safe.

woah – we’ve covered alot. almost done

setup php right

TURN OF REGISTER GLOBALS! yes thats all caps for a reason. Also disable magic quotes and change the headers sent my apache to hide version / software information. Can’t turn of register globals? try this function:

function clearRegisteredGlobals(){
global $_GET,$_REQUEST,$_POST,$_SESSION,$_COOKIE,$_FILES;

//check if register globals is on – register globals check taken fron drupal installed patch : http://drupal.org/files/issues/register_globals_check-D6_3.patch
//get php ini setting
$register_globals = trim(ini_get(‘register_globals’));
//check ini value
if(!empty($register_globals) && strtolower($register_globals) != ‘off’){
//ok now lets clear the variables set with register globals

//make array of superglobals
$registered=$_REQUEST;
$registered=(!empty($_POST)) ? array_merge($registered,$_POST) : $registered;
$registered=(!empty($_GET)) ? array_merge($registered,$_GET) : $registered;
$registered=(!empty($_SESSION)) ? array_merge($registered,$_SESSION) : $registered;
$registered=(!empty($_COOKIE)) ? array_merge($registered,$_COOKIE) : $registered;

foreach($registered as $var=>$void){
@unset($GLOBALS[$var]);
}
}
}

Hide everything

hide everything – that simple. if a folder, file, etc doesn’t / should be seen hide it. How? well if you use the .inc file extension like me, configure apache to handle .inc files with php. another option, or added protection: use htaccess to prevent access to .inc files, this will not effect your scripts, just web browsing.

In addition to hiding your inc files, don’t allow access to directories that aren’t needed to view your site. so you images directory should allow access, but your lib, class, or inc folder shouldn’t.

Important files (db config) should be in a directory outside of your hosting directory, but if you name it .inc, or .php and follow these directions you should be ok.

Lastly – turn off directory browsing.

Control access

ever part of your site should have an access level. So areas like your home page, public areas would a level 0. areas a users settings page would be a 1 (making sure only the user can access it of course), and your admin area – thats another story. Your admin area is the heart / backbone / investors dream of your site. That said protect it! all users in your admin should have an access level, and different parts of admin should have different access requirements.

ex: moderator – login, see’s flagged post area, does not see links to other areas, can not access other areas. manager: login, sees users, can add or remove users, but can not access critical site areas, and can not add a user >= his level. Admin can do almost everything, and lastly: rot – your root account can be named anything, but only allow one account full control over the system.

so quick review: users should only be able to see and access areas within their permissions scope, users should never be add / give users permissions >= their permissions.

Lastly, track everything your admin users do. You can go as far as adding an approval system for changes, tying your backend to svn to undo / redo changes, it’s pretty much up to you and the project / budget.

using sessions

sessions are like raymond, everybody loves them. But if your depending fully on php sessions you should make some changes. There should only be 1-2 cookie and session variables sent ( you can also send session id with get ), everything else should be handled internally in your application. Which means session / user validation, tracking, and variables.

misc

  • using isset only tells you if a var is set, not if it contains a value, use empty instead.
  • instead of adding more columns to your db for certain options, you can build a flag system. this allows you to add new options, without always adding a new column.
  • encrypt sensitive data (ssn’s, sin’s, cc data, phone taps, next weeks lotto numbers)
  • aes is your safest bet if your using encryption
  • if your using encryption, you need to spec out an information access process, permissions system (more than just roles)
  • kiss, the simpler it is to the end user, the less likely they are to break, figure out how to exploit it.
  • Variable names shouldn’t match table column names
  • separate code from design. not so much security a being smart and lazy – saves alot of work in the future

Read the php manual, you’ll find lots of good advice / functions in the classes.

Are we done?

Yes, hopefully someone gets something out of this, and I kept my promise of short and sweet. a quick google is all you’ll need to learn more about a subject. so right click -> search google

Gotta question, feedback, or recommendation? leave a comment

Cheers Sid / Greg