Tag Archives: svn

simple things that f up projects

26 Nov

So while reading the comments from my last post I started wondering, whats some of the things that tend to “f” up projects? after further thought here is what i came up with. I’ll try and expand it when i expand the quick start: web project management post

communication – design to developer, developer to designer, client to pm, pm to developer / designers
building one trick ponies – why build one trick pony sites, apps? their hard to expand, reuse, etc
lack of solid ui design from start
general lack of knowledge on web optimizing
file access / version control – git / svn, problem solved
using basecamp as a run on forum

short, sweet, sleep

php security in a nutshell

9 May

I have a friend I’m teaching foundation security to. This post is for him, but also as a protest to some of the materials I’ve found when looking for reference material for him.

Security at it’s simplest form is common sense. ask yourself, how can I make sure I get exactly what I want? How do I make sure I only give what I want. One article mentions xss attacks, and only says prevent them. Why? Thats the question alot of people have when starting why? So why not teach them how to do it first?

How to avoid sql injection / xss, and other misc attacks.

As mentioned this is part rant, part helpful. I’ll explain the following tips and why / how you do it.

  • always use require_once, or include_once. why? it keeps someone from getting your files stuck in loops.
  • clean everything that calls, enters, looks at your db.
  • typecast whenever you expect a certain type of variable.
  • control access and check permissions
  • use your own sessions
  • track everything in some form
  • setup php correctly
  • hide whats not to be seen / accessed

first off let me say I’m by no means a security god. Actually I’m not even an advanced user. Sad as it is maybe to say: I’ve never used pear. that said, the majority of attacks / exploits can be easily avoided. Why? because the majority of attacks on the web don’t come from hackers they come from script kiddies. We can be lax with our own stuff ( like this blog ), but any application you build for a client should at least have the basics.

Enough ranting now to the meat and tators…. I’ll keep everything short and sweet. fyi – this is pretty much a brain dump, so prob not in “good form”.

why do we use the _once functions?

if you have a file that loads another file, say index.php?get=/calender.php

what happens if someone changes get to /index.php? yeap your suck in a loop, unless you use require_once / include_once

simple huh?

State changes

Your first question is prob, what the hell is a state change? a state change is simply any change, anytime you change something, whether in the db, a file, an upload: it should always use post. why? Post can be hacked yes, but it’s harder to hack post.

imagine we’re using an online game
ex: update=1&user=87897&add_money=8.

so any user who can add will know: hey i can change add_money to 100 and gain 100 points. On top of that any user can now see all your get vars. Why does that matter? The less they know about your vars, the harder it is for a kiddie / developer to exploit it?

why else? It makes it easier to validate changes. Why? Honestly I don’t even remember why right now, but hopefully you won’t hold that against me

all users are evil

I know kinda overzealous, but you need to have this mindset, why? users will accidentally mess up your system every chance they get. And script kiddies love telling you how l33t they are if they do something as simple as figure out how to make a game page display a different page.

as for making a game page display a different page, honestly: who cares ( yes that was me venting). But in order to prevent accidents, or worst kiddie hacks, control everything! I’m not saying make your app so restrictive that users hate it, but – actually an example would fit best.

today the team made a new flash widget that requires user data.
ex1: pass user data as flash vars, then use loadvars to pass update to server

or more secure
ex2: use loadvars to recieve user data from server, and then pass update to server with loadvars.

in this example we could have honestly used either way as the update script validates all data and any real changes are driven off the database, not the state change, but you get the point. Its one less thing worry about if a user decides to try and change the vars passed, and also one less file to update if we change something.

Users will enter strings when numbers should be entered, upload swf’s when you only want images – you get the point. And the point is validate, and whenever possible take the control from the frontend and move it to the backend.

Be as lazy as possible

I say I am a smart lazy person. building 30 different files takes more time than building one file, and using includes, or a template structure. Pretty lazy huh? but also easier to update and more secure. The more files you have that each have their own independent / copy + pasted code, the more opportunities you have for a slip up. Make one file, and let it handle the logic. You’ll have more freetime, and get more sleep. Or maybe you’ll just spend that time working on more projects. See being lazy is a good thing, but only if done correctly.

we can take this a step further and say why even ftp into the server, it’s so time consuming. why not just build a backend that not only manages your files, but controls access to them – Thats more of a teaser than anything, but try it out, you’ll like the results.

typecast whenever possible

imagine we’re using an online game
ex: update=1&user=87897&add_money=8

ok so what if someone changes add_money to a delete statement, or attempts some form of sql injection. whats the simplest way to defeat it? $money=(integer) $_GET[' add_money'];

Yeap one simple change is all it took to defeat the sql injection. why? Typecasting is basically a way of forcing something to be something. huh? if i want a value to always be an integer, i use (integer). If I want a double i use (double), string (string).

Yeah it’s that simple. the only issue i’ve run into is that you can’t use typecasting in defining function / method params. huh?

ex: function foo((integer)$f=0)

that will cause an error, but you can do

ex: function foo($f=0){

$f=(integer)$f;
}

Make sense? of course I can’t force something to (mocha frap with extra mocha) $coffee, but thats life. good now on to more, or learn more about typecasting

Validate, validate, validate

using typecasting is great for numbers, but theres other ways to validate your data. the best and most powerful being regular expressions

ex: preg_replace(‘/[^a-z0-9]/i’,”,$value);

The above regex replaces any non alpha numeric characters in value. spend sometime getting comfortable with regex as its an extremely powerful and useful feature. Not just for validating data, but regex has many other uses as well.

Be a neat freak, or cleaning your sql

By now you understand sql injection, if not

ok so now we all understand it. basically its a cool way of saying, someones trying to make my query do bad things, but saying it like that would make me should like a user, so we say sql injection and confuse the heck out of clients :p

we just saw how to prevent one form of sql attack. now lets see how we can handle preventing them at the query level.

Whenever data is sent to your db it should always be cleaned. Me I like to make sure both the table, columns are cleaned using a function that makes sure tables / column names follow a standard, and a cleaning function for actual data. Why? When developing an app from scratch you normally have freedom over how tables, columns are named. I prefer to keep all tables and columns lower case, and only allow _ as a special char (non alpha numeric character). what does it look like?

ex: //convert name to proper db format
function dbProperObjectName($objectName){
//if you want to use caps is table / column names then please uncomment this
$objectName=strtolower($objectName);
return @preg_replace(“/([\\x00-\\x2d\\x3a-\\x40\\x5b-\\x60\\x7b-\\xff{$this->mSystemDatabase['restricted_chars']}\\x2f])/e”, ‘_’, $objectName);
}

You can ignore the {$this->mSystemDatabase['restricted_chars']} thats some carry over from the db class. If you don’t understand what heck that says I’ll explain. first I’m changing $objectName to all lowercase, if it’s not already. then we’re using a regular expression (regex) to clean our string of anything thats is not a letter or number and replacing it with _. why does this matter? because if for any reason our table name contains a sql injection, when ran it will only return nothing. why? because if $objectName was SELECT * FROM HOME, it will now be select___from_home. which will return nothing because select___from_home isn’t a valid table. See and you thought cleaning wasn’t fun.

Ok you do windows, but what else?

As much fun as cleaning a table name maybe, we really need to make sure our data is safe. why? ummm because I say so. There are many reasons, ranging from controlling content, preventing xss, sql injection. But I like to think you’ll do it because users are evil :)

ex: //strip bad things from a string you plan to use in a query
function dbFriendlyValue($value=false,$fixNewlines=true,$allowedTags=[pass your list of allowed tags here]){
//if no value then just return 0, use this because empty returns false if $value =0
if($value===false) return 0;

//convert to string for checking, this is fine for text / numeric values
$value=(string)$value;

//strip slashes if magic quotes enabled
if( get_magic_quotes_gpc() ) $value = stripslashes( $value );

//clear white space
$value=trim($value);

//fix \r\n
$value=str_replace(“\r\n”, “\n”, $value);

//clear tags (except allowed) or just use html entities
$value=(!empty($allowedTags)) ? strip_tags($value,$allowedTags) : htmlentities($value, ENT_QUOTES);

//change newlines to <br>
if($fixNewlines) $value=nl2br($value);

//clear any bad sql we might find untested regex
$value=@preg_replace(‘/(insert(\s?)into|\).(\s?)values.(\s?)\(|DELETE.(\s?)FROM|CREATE.(\s?)[datbsetl]{5,8}|alter.(\s?)[datbsetlcoumn]{5,8}|drop.(\s?)[datbsetlcoumn]{5,8}|update.\s?(.*?).\s?set|alter.(\s?)[datbsetlcoumn]{5,8})/i’,”,$value);

//add slashes
$value=(@mysql_real_escape_string($value)) ? @mysql_real_escape_string($value) : addslashes($value);

return $value;
}

woah what the hell was that? it was me doing the windows and the oven. lets break it down

when calling the function we pass the value, whether to fix newlines ( default : true) , and the string containing allowed tags if any.
next we make sure we have a value to clean, if not return 0, just in case the function is being used to create a sql statement. we check for magic quotes because if this value came from a submitted variable and magic quotes is on, it will add slashes. if its on the strip those slashes so we can continue.

i’ll skip trim and str_replace, now we’re at strip tags. php is pretty good at striping tags, but you want another option use htmlentities( $value, ENT_QUOTES)

and now on to our regex. this is untested ( sorry still building the class ), but points you in the right direction. the regex searches the value for any sql statements and strips them. lastly we add slashes to our value to make its sql / db safe.

woah – we’ve covered alot. almost done

setup php right

TURN OF REGISTER GLOBALS! yes thats all caps for a reason. Also disable magic quotes and change the headers sent my apache to hide version / software information. Can’t turn of register globals? try this function:

function clearRegisteredGlobals(){
global $_GET,$_REQUEST,$_POST,$_SESSION,$_COOKIE,$_FILES;

//check if register globals is on – register globals check taken fron drupal installed patch : http://drupal.org/files/issues/register_globals_check-D6_3.patch
//get php ini setting
$register_globals = trim(ini_get(‘register_globals’));
//check ini value
if(!empty($register_globals) && strtolower($register_globals) != ‘off’){
//ok now lets clear the variables set with register globals

//make array of superglobals
$registered=$_REQUEST;
$registered=(!empty($_POST)) ? array_merge($registered,$_POST) : $registered;
$registered=(!empty($_GET)) ? array_merge($registered,$_GET) : $registered;
$registered=(!empty($_SESSION)) ? array_merge($registered,$_SESSION) : $registered;
$registered=(!empty($_COOKIE)) ? array_merge($registered,$_COOKIE) : $registered;

foreach($registered as $var=>$void){
@unset($GLOBALS[$var]);
}
}
}

Hide everything

hide everything – that simple. if a folder, file, etc doesn’t / should be seen hide it. How? well if you use the .inc file extension like me, configure apache to handle .inc files with php. another option, or added protection: use htaccess to prevent access to .inc files, this will not effect your scripts, just web browsing.

In addition to hiding your inc files, don’t allow access to directories that aren’t needed to view your site. so you images directory should allow access, but your lib, class, or inc folder shouldn’t.

Important files (db config) should be in a directory outside of your hosting directory, but if you name it .inc, or .php and follow these directions you should be ok.

Lastly – turn off directory browsing.

Control access

ever part of your site should have an access level. So areas like your home page, public areas would a level 0. areas a users settings page would be a 1 (making sure only the user can access it of course), and your admin area – thats another story. Your admin area is the heart / backbone / investors dream of your site. That said protect it! all users in your admin should have an access level, and different parts of admin should have different access requirements.

ex: moderator – login, see’s flagged post area, does not see links to other areas, can not access other areas. manager: login, sees users, can add or remove users, but can not access critical site areas, and can not add a user >= his level. Admin can do almost everything, and lastly: rot – your root account can be named anything, but only allow one account full control over the system.

so quick review: users should only be able to see and access areas within their permissions scope, users should never be add / give users permissions >= their permissions.

Lastly, track everything your admin users do. You can go as far as adding an approval system for changes, tying your backend to svn to undo / redo changes, it’s pretty much up to you and the project / budget.

using sessions

sessions are like raymond, everybody loves them. But if your depending fully on php sessions you should make some changes. There should only be 1-2 cookie and session variables sent ( you can also send session id with get ), everything else should be handled internally in your application. Which means session / user validation, tracking, and variables.

misc

  • using isset only tells you if a var is set, not if it contains a value, use empty instead.
  • instead of adding more columns to your db for certain options, you can build a flag system. this allows you to add new options, without always adding a new column.
  • encrypt sensitive data (ssn’s, sin’s, cc data, phone taps, next weeks lotto numbers)
  • aes is your safest bet if your using encryption
  • if your using encryption, you need to spec out an information access process, permissions system (more than just roles)
  • kiss, the simpler it is to the end user, the less likely they are to break, figure out how to exploit it.
  • Variable names shouldn’t match table column names
  • separate code from design. not so much security a being smart and lazy – saves alot of work in the future

Read the php manual, you’ll find lots of good advice / functions in the classes.

Are we done?

Yes, hopefully someone gets something out of this, and I kept my promise of short and sweet. a quick google is all you’ll need to learn more about a subject. so right click -> search google

Gotta question, feedback, or recommendation? leave a comment

Cheers Sid / Greg

Install yum and subversion on media temple dv 3.5

29 Mar

Just upgraded one of my clients servers to dv 3.5. dv 3.5 uses centos 5 so when you install yum you’ll need to use that version

 

rpm -Uvh http://mirror.centos.org/centos/5/os/i386/CentOS/yum-3.0.5-1.el5.centos.5.noarch.rpm

Subversion is now included in the developer tools package. Login to your account center and install developer tools from there.

via hello, hellm 

unfuddle : basecamp with svn

10 Mar

checking out unfuddle today, cheaper plans with svn integration / hosting. I’ll test drive it after i get some time

Unfuddle is a secure, hosted project management solution for software development teams.

We know that you don’t have time to get a server up and running and then configure and secure Subversion repositories, ticketing systems and valuable project management tools.

Unfuddle: Free Subversion Hosting, Bug and Issue Tracking

MySpace Access Class – Closed

15 Oct

THIS PROJECT HAS BEEN SHUTDOWN, CEASED, SHOT, STABBED, ETC

Why?

I’m shutting down this project due to spam concerns, lack of time, and because it can be done a lot better.

Next Steps

http://myspaceapi.info It’s not live, and i have no idea when i’ll start or finish it, but i’m aiming for an end of the year launch. I’ve realized a lot of flaws in both the execution and direction of this project, and will release a web based service / api that i hope better meets the actual needs of users vs a class of functions i toss together as time permits i hope to release a full fledged api for myspace. a big thanx to Design Is Fuel. They’ve pointed me into a much better direction and i think everyone will be happy with the results.

the class as it stands is open source – basically that means everything you can download from this page is yours and the web’s.

To everyone that contributed : Don’t worry i’m axing the entire class and working from the ground up. but please send me your info if it’s not listed so i can add it. once the api gets up i’ll release a developers resource.

—–here’s the old page

This version is released as is.

After trying to play catch up with myspace i’ve chosen to rewrite the entire class. this should hopefully allow for a better application. this version was released on june 18, 2007, and is the last version to be released before the rewrite.

You’ll notice this version includes what is the start of documentation. from this date forward i’ll include the entire documentation site as part of the zip.

Project Page

This will be the permanent page for this project from now on.
http://elsid.net/myspace-access-class/

What is the MySpace Access Class?
The access class is a php4/php5 class to allow anyone to access myspace remotely. It is not a proxy service.

Requirements?
php4/php5, curl

Demo / Documentation : http://myspacedemo.elsid.net

File : http://myspacedemo.elsid.net/accessClass.zip

Features

  • blog posting
  • comment posting
  • friendslist navigation
  • myspace navigation
  • music player extraction
  • functions to pull data from user page
  • error support (includes php errors)
  • friend adding
  • settings for debugging mode, error displaying and recovery
  • logs out of myspace at close
  • functions for collecting hashes, hidden fields, etc.
  • url tokens are automatically extracted / included
  • captha functions (needs testing)
  • bulletin posting

Who makes this?

Primary Developer :
Myself – Greg Sidberry – poetics5 [ at ] yahoo.com

Contributers :
Brad Turcotte – brad [at] bradsucks.net
initial development, feedback, getMyspaceFriends()

FB ( still no idea what his name is) – thegreatfb [ at ] gmail.com
extractText(), extractMyspacePlayer()

Contributing

I love it when other people make functions, keeps me from having to. If your trying to write a function and it doesn’t work – contact me.

Please send any code as a php file. I try to fix any broken functions that are sent to me, and include or update all functions sent to me.

Common Questions

is this an extension? no this is a php4 class. It will work with wordpress, drupal, pretty much anything running php.

whats new? friend requests, bulletin posting

do you plan on ever fixing your site? one day

Do you offer support? yeap, actually about 4 of the functions in the class come from people trying to make new functions and contacting me for help.

i’m adding a function, do i need to let you know? yeap, send it over so i can add it.

why are you so damn sexy? genetics

will you always offer support? yeap, as long as there is a page there is support

umm i’d like to hire you. :) yippie money, you can contact me directly at poetics5 [ at ] yahoo.com, or contact the guys that feed me http://designisfuel.com

why are you writing a class you don’t even use? i like to help my friends, and by friends i mean people i don’t know.

i’d like to give you money! yippie paypal : greg [ at ] sidneyblack.net

umm i can’t make this work? see the support question, i’d say rtfm – but there isn’t one.

Licensing, usage, and restrictions

If you use this in a commercial project – please make a donation.
If you add a function – Send it to me.
Do Not Use This Class To Spam!!!!

Other than that I love linkage love, comments, and suggestions…

This Class is provided as is, and is in a constant state of development. all versions are released as alpha.

Future Tasks

I’d like to put together proper documentation for this project, svn access, and trac.

Development

If you’d like to contribute, or would like to help with the development process please contact me.

at the moment this project has an ActiveCollab area setup for potential developers.

textmate official bundles

29 Aug

heres a link to some of the bundles in dev for textmate.

http://macromates.com/svn/Bundles/trunk/Review/Bundles/

fyi – dreamweaver svn access

14 Aug

If your using an apache / svn setup you can access repos using webdav in dreamweaver

update — if you use webdav you won’t be able to commit / add new files. but you can commit them from a regular client, then use dreamweaver to update them. also svn doesn’t seem to be tracking revision numbers from dreamweaver / webdav

sweet trac theme

2 Aug

finally a sweet theme for trac. no body likes trac’s defualt theme, and i’m not a fan of most of the ones i’ve found – till tonight.

Screen Shot :
http://svn.dnite.org/trac_skittlish_theme/0.10/skittlishtheme/htdocs/screenshot.png
you can install with :
easy_install http://svn.dnite.org/trac_skittlish_theme/0.10/

This theme requires the trac theme engine of course:
http://trac-hacks.org/wiki/ThemeEnginePlugin

tying a custom sytem into plesk

29 Jul

ok well after stayng up for most of this weekend i’m still gonna end up having to use what i already had yesterday. but maybe my time wasted can help someone out there.

1. plesk’s cmd line subdomain creator does not tie into dns. so after slaving to have everything work perfectly with plesk – this is just another hill to conquer.

2. with the default mysql / plesk setup you have to use grant to add a user

3. there are there tables in the psa database that contain the info needed to display a subdomain in a users subdomain panel.

heres the code that finally worked for me

function processOnServer($clientName){
global $baseDir;
//make our password using the user name and adding some md5 salt to the tail
//each password must be unique, as it's how we'll find the ids later
$pass=md5($clientName);
$passLen=13-strlen($clientName);
$passEnd=substr($pass,0,$passLen);
$password=$clientName.$passEnd;
//setup the client dir location
$clientDir=$baseDir.$clientName;

$dbConn = mysql_connect('db_server', 'user', 'pass');
if (!$dbConn) {
    die('Could not connect: ' . mysql_error());
}

$dbName=$clientName."_trac";
//create db
$sql = "CREATE DATABASE $dbName";
mysql_query($sql, $dbConn);

//select mysql db
mysql_select_db("mysql",$dbConn);
//add client user
$sql="GRANT ALL ON $dbName.* TO '$clientName' IDENTIFIED BY '$password'";
mysql_query($sql, $dbConn);
//select plesk db
/*plesk configuration - i'm leaving this till another day
mysql_select_db("psa",$dbConn);
//add account
$sql="INSERT INTO `accounts` (`id`, `type`, `password`) VALUES (NULL, 'plain', '$password')";
mysql_query($sql, $dbConn);
///find the new users id
$sql="SELECT id,password FROM accounts WHERE password = '$password' ORDER BY id DESC";
$findAccountId=mysql_query($sql, $dbConn);
list($newId) = mysql_fetch_row($findAccountId);

//add user to plesk for subdomain
$sql="INSERT INTO `sys_users` (`id`, `login`, `account_id`, `home`, `shell`, `quota`) VALUES (NULL, '$clientName', $newId, '$clientDir', '/bin/false', 0)";
mysql_query($sql, $dbConn);
//find the new new id
$sql="SELECT id,account_id FROM sys_users WHERE account_id=$newId ORDER BY id DESC";
$findAccountId2=mysql_query($sql, $dbConn);
list($newerId) = mysql_fetch_row($findAccountId2);

//add subdomain to plesk
$sql="INSERT INTO `subdomains` (`id`, `dom_id`, `name`, `displayName`, `sys_user_type`, `sys_user_id`, `ssi`, `php`, `cgi`, `perl`, `python`, `fastcgi`, `miva`, `coldfusion`, `asp`, `asp_dot_net`, `ssl`, `same_ssl`) VALUES (NULL, 1, '$clientName', '$clientName', 'native', $newerId, 'false', 'true', 'false', 'false', 'false', 'false', 'false', 'false', 'false', 'false', 'true', 'false')";
mysql_query($sql, $dbConn) or die('Could not connect: ' . mysql_error());
*/
//make config file
$fileName = md5($clientName);
makeApacheFiles($clientName,$fileName.".conf");
makeTracFiles($clientName,$fileName.".ini",$password);
makeAuthFiles($clientName,$fileName.".authz");

//finish the svn and trac setup
passthru("shell_script $clientName $password $fileName");

}

the codes choppy (i’ve been up for over 30 hours – so i’m sure it looks worse to use :) )

the system i’ve built uses a custom frontend app to create, manage subdomains, and do automated installs of svn repos / trac, etc.

if you look over that code you’ll see there’s three psa tables that need fixing to allow your client to also view your subdomains in plesk.

also the ftp fix mentioned on there site won’t work on mediatemple. when i find one i’ll post it.

in addtional to all this. the psa database is only a place holder for the frontend, and in no way reflects the actually server (unless you do everything from plesk)

if you add a db or a dns record you’ll have to also update the psa db.

don’t know if i’m still making sense, off to bed – sid

does alexa matter?

27 Jul

well tonight (as a break for an svn / trac /amazon s3 system – notice thread in posts) i checked my alexa rank. i’m like 688,*** weird i only get about 3-4 000 hits a month (i’m guesstamating the blackout periods) so why am i only 120*** away from an old client that i know has an entire team devoted to only myspace advertising, and with a min of 5000+ pageviews per day?……

a quick google shows plenty of people who aren’t to happy with their stats either….

not that it matters as this is basically a blog to share info (and to assist with my bad memory).

anyways yippie i rank higher than 75% of my clients :) which really doesn’t matter, come one whose really gonna keep visiting a design firms site