Sid Sidberry Reads

I’m using vmware 6, so I don’t know if this works in 5 or server editions.

vmware installs a script called vmrun that allows you to perform almost anything you need to do on a vmware machine pragmatically.

sidney@my-host:~> /usr/bin/vmrun
vmrun version 6.0.4 build-93057

Usage: vmrun COMMAND [PARAMETERS]

Authentication flags
-gu
-gp

POWER COMMANDS           PARAMETERS           DESCRIPTION
start                    Path to vmx file     Start a VM or Team
                         or vmtm file
                         [gui|nogui]
stop                     Path to vmx file     Stop a VM or Team
                         or vmtm file
                         [hard|soft]
reset                    Path to vmx file     Reset a VM or Team
                         or vmtm file
                         [hard|soft]
suspend                  Path to vmx file     Suspend a VM or Team
                         or vmtm file
                         [hard|soft]

SNAPSHOT COMMANDS        PARAMETERS           DESCRIPTION
listSnapshots            Path to vmx file     List all snapshots in a VM
snapshot                 Path to vmx file     Create a snapshot of a VM
                         Snapshot name
deleteSnapshot           Path to vmx file     Remove a snapshot from a VM
                         Snapshot name
revertToSnapshot         Path to vmx file     Set VM state to a snapshot
                         Snapshot name

GUEST OS COMMANDS        PARAMETERS           DESCRIPTION
runProgramInGuest        Path to vmx file     Run a program in Guest OS
                         Program
                         [Program arguments]
fileExistsInGuest        Path to vmx file     Check if a file exists in Guest OS
                         Path to file in guest
setSharedFolderState     Path to vmx file     Modify a Host-Guest shared folder
                         Share name
                         New host path
addSharedFolder          Path to vmx file     Add a Host-Guest shared folder
                         Share name
                         Host path
removeSharedFolder       Path to vmx file     Remove a Host-Guest shared folder
                         Share name
listProcessesInGuest     Path to vmx file     List running processes in Guest OS
killProcessInGuest       Path to vmx file     Kill a process in Guest OS
                         process id
runScriptInGuest         Path to vmx file     Run a script in Guest OS
                         Interpreter path
                         script_text
deleteFileInGuest        Path to vmx file     Delete a file in Guest OS
                         File
createDirectoryInGuest   Path to vmx file     Create a directory in Guest OS
                         Directory path
deleteDirectoryInGuest   Path to vmx file     Delete a directory in Guest OS
                         Directory path
listDirectoryInGuest     Path to vmx file     List a directory in Guest OS
                         Directory path
copyFileFromHostToGuest  Path to vmx file     Copy a file from host OS to guest OS
                         Path on host
                         Path in guest
copyFileFromGuestToHost  Path to vmx file     Copy a file from guest OS to host OS
                         Path in guest
                         Path on host
renameFileInGuest        Path to vmx file     Rename a file in Guest OS
                         Original name
                         New name

GENERAL COMMANDS         PARAMETERS           DESCRIPTION
list                                          List all running VMs
upgradevm                Path to vmx file     Upgrade VM file format, virtual hw
installtools             Path to vmx file     Install Tools in Guest OS

Examples:
  vmrun list
  vmrun upgradevm w2k.vmx
  vmrun reset w2k.vmx soft
 

Yippie!

Ok I’m kinda confused are they trying to replace java, or activex, both…?

From: http://google-code-updates.blogspot.com/2008/12/native-client-technology-for-running.html

Modern PCs can execute billions of instructions per second, but today’s web applications can access only a small fraction of this computational power. If web developers could use all of this power, just imagine the rich, dynamic experiences they could create. At Google we’re always trying to make the web a better platform. That’s why we’re working on Native Client, a technology that aims to give web developers access to the full power of the client’s CPU while maintaining the browser neutrality, OS portability and safety that people expect from web applications. Today, we’re sharing our technology with the research and security communities in the hopes that they will help us make this technology more useful and more secure.

Check It Out:
Google Code Blog: Native Client: A Technology for Running Native Code on the Web

While benchmarking performance for deathy death match I realized something: Benchmarking sucks!

That said, and me being the lazy person I am, I asked myself how can I make this easier? My answer: benchsid, two shell scripts that make benchmarking easier.

disclaimer

This is a 1,2,3 trick pony – use it as such

These scripts are offered as is and I am not responsible for anything you do, create, break, etc.

Both scripts are released under the do whatever the hell you want, but don’t blame me license. This of course isn’t a real license, but you get the point.

Make benchmarks easier, if only a bit

I’ve tried as much as possible to keep your commands to a minimum. If running as root you can run a complete benchmark, save both the results and you benchmark servers top + free output, and reboot with 3 commands. See lazy right?

monitor.sh – run this script on the machine your benchmarking
bench.sh – run this script on the machine your benchmarking from

you do not need to run these on separate machines, but I recommend it

How do I use…

monitor:

 ./monitor.sh

 ./monitor.sh help

bench:

 ./bench.sh

 ./bench.sh help

If your still having issues post a comment

monitor.sh

Monitor should be ran on the machine your benchmarking. It automates the process of recording free (free -m) and top, and if running as root will reboot the server when you tell it to stop monitoring a benchmark

Sounds good so far right?

monitor.sh process overview

start process

clear out any benchmarks that will be overwritten
save output from free to a file (free -m)
save output from top to a file
start a top process to monitor to a file while we benchmark
confirm that we are up and running

stop process

stop top
save output of free to file
save output of top to file
reboot server

monitor.sh: umm why?

monitor.sh saves the information about the server before, during and after your benchmark has been run. By running top during the benchmark process (10 second intervals), we can get a real time overview of both server load and the loads created by applications

bench.sh

bench.sh is the benchmarking portion. bench.sh can benchmark your server using one of three programs: ab, siege, or httpref. I considered adding an option to do all three, but honestly that more poor mans dos than benchmark.

bench.sh first will perform a warm-up cycle of 2000 requests to your server using your desired (or default) concurrency. After that it will perform 3 benchmark cycles of your desired number of requests and concurrency.

All cycle results (including warm up) are saved to the filesystem.

download

download benchsid: http://gregsidberry.com/httpdocs/wp-content/uploads/2008/12/benchsid.tar.gz

known issues

siege output – siege outputs a couple of variables to screen that aren’t included, or properly recorded, in the created file. I’ve tried output redirection, script, etc, but haven’t been able to find a fix that keeps everything in one script

input validation – there is no input validation, so please don’t put these files in a web public location.

funyuns – I’m out of funyuns, please send more

enjoy
Sid

Now this is pretty wicked. They’ve used memcache as an L2 (secondary) cache for mysql. Wonder if there’d be any benefit if your running via vps?

Memcached as a L2 Cache for Innodb – The Waffle Grid Project
By matt

A few months ago I was at dinner with Yves Trudeau discussing what all consultants discuss in the late hours after a long day of hard work… how to improve performance and scalability. I brought up an idea to him to utilize memcached as an L2 cache for innodb. At first he was skeptical, but as we talked he was more and more intrigued by the idea. The idea was simple, add a set to memcached when something hit the LRU… then issue a get from memcached when you do not find the data locally stored in the buffer pool but before you read from disk. Starting from that point, you can work out any of the issues that would be sure to follow. So Yves continued emailing me asking me questions… then he sent me a note that he had made huge progress with the idea. Huge progress means that he wrote version 0.1 and had it working. That’s when the Idea really turned into a project.

We called it the Waffle Grid Project. Why? A waffle sort of looks like a grid diagram doesn’t it? And I like waffles, they happen to be very tasty. Having a working patch… We burned the midnight oil the last few weeks testing and fixing the code, building a proof of concept, and benchmarking it. So does it work? Yes, it does. Pretty well in fact. Take a look at some of the benchmarks below for a better idea.

Basically what this patch enables you to do is have a central node ( standard run of the mill database ) with several servers acting as remote L2 cache. An example: 1 Main Mysql server with 128GB of memory, 4 remote servers with 64GB of memory each… giving you a L2 cache of about 256GB. With a fast private network the L2 cache should return data faster then can be retrieved off of disk.

Here is what this would look like:

Memcached as a L2 Cache for Innodb – The Waffle Grid Project

I’m now officially up and running with wordpress 2.6.5, only I’m nolonger able to edit posts in wordpress. After a bit of snooping the issue appears to be a new flag that is added to the db whenever I open a post to edit it: _edit_lock

To clear _edit_lock run:

delete from wp_postmeta where meta_key='_edit_lock'

. You’ll need to replace wp_ with whatever your table prefix is. This isn’t even a real solution as the lock will be put back in place the moment a post auto saves. Anyone else running into this?

Lucky for me I normally use scribefire

Came across this today while looking into fail to ban. This is a pretty good overview of applications you can use to secure your server. The directions are for ubuntu, but source packages and alts are just a google away of course.

Quick ‘n Dirty Security and Visibility

1. Introduction
2. Firewall
2.1 FireHOL
2.2 ufw
3. Intrusion Prevention
3.1 Fail2Ban
3.2 sshdfilter
4. Intrusion Detection
4.1 chkrootkit
4.2 rkhunter
5. Log Monitoring
5.1 LogWatch and Dropbear
6. Security Update Notification

Introduction

This article describes some of the tools and utilities I use to better secure and monitor my servers. Simply following the article below does not get you a secure box, you also need to ensure any services you have running are correctly configured and you should disable, or better yet remove, any services or software you do not require.

Quickn Dirty Security and Visibility

So I’m trying to give a friend a few of the basics they’d need to know for managing a web project. Not a bible, just a quick read to hit the ground running. Here is what I came up with.

Your team:

designers – creative artistically, creates look and feel, communicate, produces things you can see
developers – creative technically, creates applications foundation, communicate, leave alone for extended periods, produces things that are used, but not always seen.
cto – head geek, usually smarter than you.
ceo – head head, smile and nod – might be cool
cfo – head money – nuff said
creative director – awesomely artistic, likes coffee
techincal director – almost head geek
vp anything – has friends, probably knows what their doing
assitant – love them, bring them coffee, and be really really nice to them
project manager – you, makes sure designers / developers are making progress, middle man between client, developers / designers, and everyone else (maybe)

Basecamp:

where you document everything. Covers your ass. Anything changes, legals, documents, or client communication happens here.

Cover your ass use BC

Deadline:

in house milestone / deadline=client review milestone / deadline- (1 – 2 weeks)
client milestone / deadline = milestone / deadline + (1 – 2 weeks)
client review milestone / deadline = milestone / deadline – (2-3 weeks)

Terms to know:

fubar – we’re fucked
need more time – your fucked
great – needs review
fun – expect delays
breach of contract – see fubar * 2
contract – covers your ass
invoice – pays your ass
server – where everything runs
ui / client side – where everything is seen
php, ruby, python – what developer use to make things run
css / stylesheet – what designers / developers use to layout what you see
ajax – those cool effects
flash – more cool effects
flv – youtube video
harvest – where everyone tracks time
bc – where you track clients / team
skype – online meeting
delay – no sleep
billable hours – what you can get paid for
over hours – what you might get paid for
favor – 5/1 to get your account established
hail mary -  a favor that empties your favor account
wireframe – lines that show how something will look
mock up – test design
voip – the phone
im / aim – chat
ping – contact

And there you have it, my quick read, more web filler.

sid

When I first started this experiment I planned on using a custom php framework. Recently I realized that its kinda pointless to attempt to do something like this and at the same time lock myself into something that may not be the best solution….

enter google, research, testing, and little sleep. Result: codeigniter was the best choice. Huh? Here’s why.

snakes not on the plane

I should say from jump: If I knew and had the time to learn python / django it would have been the winner, sadly I do not.

ruby, rails, hype

I’m a fan of ror, and of course when I decided to take what I’ve created so far and migrate it into something usable I of course thought of ruby on rails. Sadly I saw more than a few hurdles.

First off ror doesn’t play we’ll with others, meaning you shouldn’t run rails in a shared environment. The whole concept behind this experiment is a small start-up with a shoe string budget and 2-3 websites / apps. Yes its possible to host multiple rails apps on one server / vps, but it’s not recommended. There is of course passenger, but that leads into the next point. php +1

Ror is a resource hog, there I said it. When compared to php, rails is more resource intensive than php. Of course the answer is to optimize and scale out, but remember I’m trying to keep the monthly hosting budget under 100-120 bucks (yes seriously shoe string). So php was the choice here.

So far php seems like the best choice for what I’m trying to do, but I needed more than just a few issues I could work around. You can’t work around speed and optimizing. ruby is faster than php via command line, but ror is not amazingly faster when used via web. When you add an opcode cache into the picture ror gets is butt kicked hands down. Of course this is comparing a language to a framework, enter codeigniter.

Codeigniter still out performs ror with an opcode cache. That is just one optimization and php shows drastic improvement.

I wasn’t ready to abandon ror yet, simply put: why use an imitator if you can have the real deal. Codeigniter is a great framework, but it makes more since to actually use rails right?

In the end the answer is no. Rails has a higher level of maintainability out the box, but does less code, easier maintenance, and of course the ror cool factor out weight slower speed, higher cost to deploy, and fewer production optimizations? No.

MVC while not as strict in codeigniter, is there. That combined with OOP will make using a php framework easier to maintain, not ror easy, but easy enough for production imho.

framework, shamework

So we now know why php won over ruby, but why pick codeigniter? I originally started out writing a custom framework, which is always fun to do from time to time to push yourself, but in the end you see the downside and benefits of doing so. Since this entire idea revolves around a small start-up we also need to take development time into consideration.

A custom framework takes alot of trial and error, coding, and recoding, and still more coding. Using a php framework I can reduce the time it takes to get up and running, while still building what I want and need. Yes there’s some overhead compared to a custom framework, but in the end a start-up isn’t yahoo. You should be back at the planning table long before you reach yahoo numbers. Thats not to say you shouldn’t plan for massive growth (hence scalable), just that traffic / users / data on yahoo’s scale is far beyond my scope of experience and even I know its another elephant to eat.

When comparing php frameworks, i was originally looking at cakephp, but after some research found it to slow and with to much overhead. There are other frameworks, but only codeigniter and cakephp met my needs. 2-1=1 (or less than one mr. V), so codeigniter was the winner.

One thing I should note is how poor most of the php frameworks I looked at perform out the box. Yes rails (and merb) are better out the box. Luckily a few simple optimizations speeds things up.

all your base are belong to theory

So far everything has been pretty much all theory, research, and testing. No worries, the meat and potatoes are coming shortly.

the next few posts will cover setting everything up(centos5): memcached / memcachedb, ngninx, apache, varnish, mysql, s3 + rsync, varnish -> s3 relations, etc.

If you feel like following along head over to http://linode.com and setup a few machines, or setup a team in vmware.

Worth Reading

Here are few articles that I came across while researching, some are more on topic than others, but all have some value.

PHP vs Java vs Ruby:
http://www.cmswire.com/cms/industry-news/php-vs-java-vs-ruby-000887.php

Ruby vs PHP performance (cli)
http://izumi.plan99.net/blog/index.php/2008/01/17/ruby-vs-php-performance/

Ruby on Rails Fans
http://shiflett.org/blog/2006/feb/ruby-on-rails-fans

The performance test of 6 leading frameworks
http://www.alrond.com/en/2007/jan/25/performance-test-of-6-leading-frameworks/

I’ve been seriously getting my kicks with scalability fora number of months, so why not start on ongoing series where I talk about what I’ve learned / found?

So welcome to to the first: Building Something Scalable – An ongoing experiment. This post covers caching. I’ll cover delivery in the next post.

Keep in mind language wise I’m using php, but the general advice should be sound, regardless of language. If you disagree with something or have a better way, feel free to comment.

So now that I’ve ranted off 4 topics, maybe I should expand on them a bit.

Use Caching

Caching is a good thing, but use caching is a pretty vague statement, so let’s expand.

Caching isn’t just a one ring to rule them all type of solution. Its actually a fixture of a number of different solutions, that work together to boost your site / applications overall performance.

Database caching

I consider database caching a 2 part solution. You have the mysql query cache, but I also like to have a server side query cache as well. Why? I tend to use oop and having a server side query cache allows me to cut some overhead both appilcation wise, and by preventing me from having to connect / query mysql.

The big issue with server side query caches is stale queries. The mysql query cache prevents stale caches automaticly, but with a server side query cache we’ll need to set a TTL (time to live). I tend to go with something really low like 5-10 seconds.

5-10 seconds may seem pointless, but it allows higher traffic applications to handle a number of requests with fewer queries to mysql. This takes some of the load from mysql, so  your database is performing under less load than it would have without the server side query cache.

There is plenty of information on the mysql query cache online, so fire up google and start researching. For your server side cache here a few things to keep in mind

1. keep your cache in a secure location. If your using a file cache this means outside of your web directory
2. hashing is a quick and painless way to uniquely id your queries. md5(’select * from table1′) will allows return md5(’select * from table1′) if done correctly.
3. prevent cache filename collisons.
4. do a light weight encoding on cache files. base64_encode / base64_decode are quick and easy to use. They’re not secure, but its a good idea to add some basic obfuscation
5. keep your TTL low. Your query cache should try to stay as fresh as possible.

Opcode Cache

Php is compiled / ran at runtime (when you request a page / script). Opcode caches store the compiled code so that your code doesn’t have to be compiled for every request. Opcode caches can increase your codes performence by up to 90%, but then again, any increase helps the overall perforence of your site / application.

There are a number of opcode caches avaible for php. I prefer xcache, but there are a number of other opcode caches available for php.

Static content cache

Static content unlike dynamic content is, well static. Your probably wondering: Why cache something thats already static? Simple, performence. Static content though is cached / served differently than dynamic content. I know this touching more on delivery, but its still worth mentioning.

Static content is often served through a CDN (content delivery network) or a web cache. A CDN and web cache act similarly, except that a cdn has a number of servers setup in various locations.

Content Delivery Network
A CDN acts just as it’s name says: It delivers your content via its network of servers. The CDN selects a server closest to the location of the user, and serves your content from that location. Whats the benifit? Faster delivery of your content. Is it worth it? That’s a question only you can answer. Do some research, compare the solutions, check you budget – and you’ll have your answer.

Web Cache
A web cache or reverse proxy simply put delivers your content faster. I’m not to well versed in the science of it all, but here’s a basic break down of what I do know:

Web caching software like vanish (in the past squid was the standard) handle servering static content better than apache, and with a smaller footprint. The web cache creates a cache of your content when requested and then delivers your content from its memory / disk cache.

The most obvious benefit from all this? Reduced server load. Apache is a resource hog (there I said it), but that will be covered in a future post in this series. By moving static content delivery to software created just for this task your freeing resources and of course getting content to users quicker.

Output Cache
So far we’ve looked at a number of ways to increase the speed of dynamic and static content, but there’s still one major item left out: Output Caching.

As your scripts / application generates pages, you can cache them to be served for future requests. Output caches in general can be as basic or complex as you need them. A few things to keep in mind.

1. stale content, your cache should have aTTL(time to live) that prevents it from serving stale content
2. filename collisons – your naming scheme should prevent filename collisons
3. store your cache outside of your web folder
4. logged in users / vs non logged in users – come up with a solution that deals with this.

Variable / Object Caches

Your code has objects and variables, often some of these objects are database intensive. An object / Variable cache is a way to store your objects and variables. The thing to keep in mind with these types of caches is speed.

It makes no sense to cache something like $var=1+1;. You can run that command quicker than you would access it from the cache. A good example of somethign to cache would a class object that runs a number of queries on the database, but accesses content that doesn’t change as often. By caching this object you can prevent a few database queries (or cache calls). Or a class object that generates a number of child class objects.

I could go on and on about this subject but lets get to the point. If your application is running on only one server use a file cache. If your application uses more than once server look into memcached / memcachedb.

Thats it.
Hopefully that was short and sweet, the next post will cover delivery.

Greg – Out

Finding that your network connection / internet seems to be moving much slower than it should?

First check your connection, then make sure ipv6 is not enabled.

Hope this helps someone