Sid Sidberry Reads

The last time I used red5 was over a year ago. I ended up going back to fms due to time constrains, and some issues with streaming.

I’m setting up an in-house media server so I decided to revisit red5. All the issues I had about red5 have been cleared up and they are actually pushing past fms.

Anyways more later, I’ve got more testing researching to do

Sid

I’m doing a project to create a flash chat that will be expanded into a flash world next phase, and will eventually need video and audio for additional phases. Its no secret I’m a fms fan boy, by I’ve been trying to find cheaper alternatives because honestly, while fms by me fine to develop with, the pricetag isn’t worth it.

I looked at electro server a few months ago when it was still version 3, but now it looks really promising. check it out. I’ll try and write a basic tut once i get some free time. (famous words) till then check it out:

ElectroServer 4 – Socket-servers

Powered by ScribeFire.

I’ve been posting alot on linux htpc, mythtv, and netowrking here and there, thought i should prob explain what i’m looking to do.also i need to start documenting stuff for myself.

current systems

right now i have 4 boxes.

1. p4 2.4 laptop
2. p4 2.8 laptop
3. pentium m 1.8 laptop
4. amd x2 4600 desktop

at the current moment there all being used as follows

1. p4 2.4 – my son plays with it
2.  p4 2.8 – ubuntu play box
3. pm 1.8 – goes back and forth between being a ubuntu box and my xp mobile dev machine
4. x2 4600 – vista – recording / watching tv, torrents, web, and sometimes dev

storage / drives

I currently have about 30-40 gigs per laptop, a 74 raptor and 250gb on the x2, and about 600-750 gb on exturnals.

In addition to all this i also have 3 exturnal burners (2 dvd, 1 cd) and a bunch more exturnals in storage.

the old network

up until about 6 months ago i had a windows domain running that piggybacked off one of my old domains for exturnal dns, and maintained it’s own internal dns, dhcp, web (apache and iis), file, remote access (via web based interface ), web based proxy server (to get around nmci’s proxy at work),  fms server, and intranet. I’ve gotten rid of about 3 machines since then.

I liked the setup, but in honesty i just regreted going back to windows when i knew it was time to finally go fully to linux.

current network

my current network is in one word crap, it consists of three routers:

1. old linksys network router
2. belkin 5 port wireless router
3. motarola 6 port voip router

each machine has it’s own web server / development envirnoment, and needless to say files are easily managed

the new network

the new network is currently spec’d as follows. the spec is a work in progress, so there are still some changes

I’ve decided to move all video recording and file servering / storage to one place – the fs box. as of right now the spec for this machine includes a total of 6 750gb’s in raid 10, and two in raid 0. dual gigabit, 4 gb of ddr2, and a core2 duo.

this box will serve as the mythtv backend (everything else, except one box, will be enabled as a myth frontend). all machines will also use the fs box for file storage / backups. each system will still have their own fs, but the files that need to be shared (projects, music, video, pictures, apps, torrents, backups, user network dirves, vmware images, etc) will all live on the server. I’m still debating wheter to do boot from network.

in addition to fs box, I’ll also be building a new box for myself that uses the htpc form factor. right now the specs are zalman 160 (don’t know which one yet) or ahanix MCE701. core2quad, 4gb ddr2, 2 74 raptors in raid 0, and a 8600gts.

my current x2 will be changed into the living room machine (tv, general browsing, music, movies ) for the wife. the p4 2.8 will be used for general wireless stuff (wife / friends), and the pm will be my mobile box. I don’t know what the p4 2.4 will be used for yet – prob just to mess around with.

Right now i’m still specing everythng out and doule checking support, implementation, planning ,etc.  but plan to start purchasing parts either as my christmas present or right before my move (no newegg shipping where i’m going).

seems like a big undertaking, and it is but you need to have fun every now and then, and once completed i prob won’t get another chance to do something big again for awhile.

to be continued…

Seacrest out

Powered by ScribeFire.

I decided to document my last event. don’t know what this falls under, but hopefully it helps others.

so lets get started

the event

I was contacted by the client to do a live webcast for a horse show. I would setup everything needed prior to the event, and also come to the event location for a day and a half to insure everything ran smoothly, was setup, etc.

the setup

We used fms installed on a media temple extreme server. The extreme server was to be used initially because we didn’t know how many users would be watching. if and when traffic to the feed started to elevate we’d contact mt to upgrade to a nitro server.

for broadcasting we used flash media encoder

day one

I arrived at the event to learn we still needed to pick up some of the equipment to access the video feed. We got to circuit city (no fry’s nearby) and decided on the WinTV-HVR-950 model 1145 from hauppauge.

after arriving at the event, this is the day before the actual event, we found out the a/v people wouldn’t arrive till the morning of the event so the day was pretty much spent making sure everything was running, installed, etc.

day two – seehorse politics

day two started off badly to say the least. the company doing the video “seehorse” apparently had some issues because they had originally wanted to do the web feed. this lead to us receiving the feed used for internal broadcasting which completely sucked and had no audio, we dealt with this the best we could.

luckily they also had brought in outside video people to assist with the filming of the event. Namely Thomas “Rick” Rommarito. I can’t say enough about Rick Rommarito. Rick saw our feed and told us not to worry. Rick ran us a direct feed to the video, audio, and was a great help. keep in mind that seehorse video’s staff had pretty much told me and the company that had hired me to blow off (with out saying it of course). we we’re pretty much ignored by the seehorse video team. so thank you again to Thomas “Rick” Rommarito for everything he did. He is the only reason we got a good video feed or audio. If you found this article while searching on seehorse, please don’t use them, contact rick directly at : rickbommarito[at][sbcglobal][dot][net]. Very knowledgeable a/v guy.

day two – the server

so half way through the day we finally had a good video and audio feed, after a few compatibility issues and tweaking we we’re up and running. here are screen shots from both fms admin, and top to show you how we’re on resources. the cpu usage for the fms process is wrong, so look at the actual cpu usages. for any one interested we maintained about 220-230 concurrent users throughout the day. once that number started to increase is when the server started having issues (end of day). I’d say keep users at about 200 concurrent if your going to be on an extreme.

early in the day we realized we’d need to upgrade to the nitro asap. I contacted mt, and was told we could be upgraded by the following morning. near the end of the day when i contacted mt again for an eta we found out all nitros were sold out for at least a week.

*update* – turns out there was a communication issue. one of the techs had a nitro on standby, but the night shift wasn’t in the loop. found this out the following week

edgecast to the rescue

with our upgrade plan now shot we needed a new host. after limelight told us there’s a 7 day wait, and Akamai seems to have outsourced their entire operation. speaking with someone at Akamai was just pointless, not to mention I didn’t hear back from a sales rep for two days, and that was also outsourced.

*update – I received a call from their guys two weeks later, asking what type of service i want!*

Edgecast was a different story, they knew we we’re 3 steps from being screwed and had us up and running within hours, I’ve already spoken on them in a previous post, but i wanted to make sure i mentioned them again. it normally takes a few days to get everything setup, but james seriously helped us out.

day three

day three had us with a brand new video feed, a new feed to computer setup, and edgecast streaming.

first off the edgecast streaming is great, any future events will use them, screw maintaining our own fms servers.

by the end of the day everything was finally running smoothly, and i got a flight heading back home

issues
our biggest issue was video, which luckily for us Thomas Rick Rommarito handled that.

next up was our server, we’d previously spoken to mt to make sure the nitro was ready on standby. *update – see previous update * I can’t be mad at mt though, as they have proven over and over again to be rock solid and caring. they even contacted me twice after about the nitro issue. edgecast was amazing. their support, network, and customer service was amazing. both myself and the client we’re happy with them, and now the client will be rolling out an entire service using the edgecast system, which leaves me happy to build with their system again.

after those two major issues the reoccurring issue was stream quality. horse shows are very motion intensive, and although we had a good feed, we didn’t have good lighting at all. i recommend any hauppauge product, because I use them at home, and every other encoder tested lacked the features it had. we tryed a few different one’s at the clients request, but in the end always went back to the hauppauge.

to overcome some of the lighting issues i used hauppauge’s built in features to adjust the feed which helped produce a better feed. the client had purchased a number of other encoders because he wanted to split the video / audio with another device. I’ll never again use a pyro product. once we started using the pyro adapter it created nothing but issues.

so once we had the hauppauge device back as the video source i was able to tweak settings and get a decent quality stream that had little lag, good motion, and decent to to good quality. I found the sweet spot to be at a 380 bit rate. the client wanted clear motion, but also needed the horses and people to be of good quality also staying around 380 offered the best motion and quality without being to bandwidth heavy. we broadcasted at a custom 16:9 resolution.

another issue was that the client had an outsource team that developed, and maintained the broadcast site. any changes had to go directly through them. they’re a good group, but often the need to contact them for changes, as well as time differences we’re frustrating. I tried to use the player the client already had in place, but in the end just hacked out a quick and simple player when i finally got completely frustrated.

in the end

so what did i learn? motion intensive events are a different experience than non motion intensive ones. don’t use pyro products. use edgecast, the quality of our stream when using their network was worth it.

I’ve normally ran our own fms server for broadcasts events. I won’t ever again, the cost and benefits of edgecasts service just makes setting up and maintaining a fms server pointless. It’s less of a headache, and you don’t need to monitor anything. In the end the cost of using edgecast is less when you add the man hours, maintenance, headaches, chain smoking, etc.

media temple dv extremes are good for around 200-210 concurrent users, after than your gonna start feeling it.

also something I realized : widescreen formats (16:9) are better then 4:3 for live streams. I found that when using a wide screen format the viewing experience was alot better, especially at smaller sizes than traditional 4:3 format. i was able to reduce the viewer and stream size by going to widescreen, and the viewing experience was actually better.

Server Resources

that’s all folks

yes i know there’s more than goes into live streams, but this is just me giving a brief overview from an experience. hopefully this helps someone, and wasn’t just good intent, bad execution

Sid

If you’d like to see the event streams recorded check out : http://www.iequine.com/

**this is a rant, sorry but needed to vent**

so I’m bout to doze off when i get a message on skype. A clients dv has been locked up since 4pm. its now 11:20. I have no idea what happened, hell I’ve only been on it 3 times. but come on its an extreme thats only running rails and this is the same server mt lost the data on.

why would it now be maxed out cpu and memory wise when it handled the rails site for 2 weeks, and was able to take a licking from fms and keep going….

no friggin clue, but i may start recommending someone else if this becomes a trend

this is the first bad week in over 6 years of hosting with them. I still have faith, but can’t really defend them when the clients pissed and right.

I guess I’m just frustrated. some of the issues have nothing to do with mt. but all the client sees is the mt issues, regardless of it there were video issues on location, or brief communacation issues with the team, or hell me not seeing that they would get the traffic they got and getting a cdn to begin with. All they see is that we had an issue it got fixed, and then they feel mt dropped a major ball. to top it off the not migrating the database thing completely broke the camels back. Now their borderline back to windows streaming.

this was the clients first time using flash instead of windows streaming. I’m frustrated and sad to see them go back to windows, even though this broadcast has gotten the highest reviews from users than any other they’ve done. many of which have out right said : the quality and ease of this years broadcast was better than the vegas event, etc. It’s like you show someone a better way because you want to help them. But i also understand that to them windows is easy and familiar. not to mention their pissed off because our feed is now branded with a competitor, and theres nothing we can do.

In the end i guess the blame rests on me. They had on idea what type of traffic they would get, instead of starting small and having the nitro as a back up, I should have just put them on the cdn to jump, might have cost more from start, but then we’d never have had these issues with mt.

its been a long day, hopefully i’ll have some fun code posts tomorrow

the documentation was a little confusing.

for broadcasting use the url provided for publishing with out the trailing /

so if your publishing point is : rtmp://fmsbcastserverurl/20xxxx/ , use rtmp://fmsserverurl/20xxxx

to view your stream use the player url without the stream name (that got me for a bit). their documentation says to include you stream name, don’t.

so if your player url is rtmp://fmsviewingserverurl/20xxxx/examplestream

change it to rtmp://fmsviewingserverurl/20xxxx and use that in your netconnection

everything else works as normal set your stream ids, etc. Also the jw flv player is fully compatible with this: http://www.jeroenwijering.com/?item=JW_FLV_Player

just add the following to your swfobject call for the player :

so.addVariable(“file”, “rtmp://fmsviewingserverurl/20xxxx”);
so.addVariable(“id”, “yourstreamid”);

enjoy, sid

*update, forgot to mention if your using the flvplayer component the just follow their guide*

came across this issue on a fresh fms install (installed by someone else)

./shmrd: error while loading shared libraries: libnspr4.so: cannot open shared object file: No such file or directory

yum install *nspr

will grab the seamonkey library, no idea why a browser files are needed, but whatever it works now

the search : running FMS on CentOS, fms not working on centos

answer :
http://elsid.net/2007/06/23/media-temple-install-flash-media-sever-2-on-media-temple-dv-or-similar-centos-setups/

first make sure yum is installed. go here for a full media temple yum install how to:
http://blog.hellm.com/post/5

ok you have yum, now run the following

yum install compat-libstdc++-296.i386
yum install compat-libstdc++-33.i386

don’t for get to add a group and user for fms and give the user a password

groupadd fms
useradd -g fms fms
usermod -p yourPassword fms

lastly run the install, but we need to tell it to only warn about platform compatibility

./installFMS -platformWarnOnly

you should now be running the install

after testing both servers for a simple application that feeds 5 live feeds with a user chat, and a admin message oneway chat. i’ve decided to go with fms. this is only due to out the box performance and in no way am i not recommending it.

Why

I decided on fms for one reason – out the box streaming. red5 can stream just as well as fms, but i don’t know java.

straight out the box fms handled the streams extremely well, with very little lag, and no dropped frames when load tested.

to red5’s defense it handled everything well when doing a live demo for the client. also for a beta application it’s only a few steps behind the fms server. I would still recommend red5 to anyone, and plan on learning java with the focus on working with red5.

I believe with in one or two more releases i’ll find myself moving back to red5, but for now it’s fms. FYI – i’m using version .6

final thoughts

I don’t think the cost of fms just for a step or two up in quality of a live stream is reason enough to justify choosing it. But then again it’s not my budget, and i want to give the client the best possible results.

i should also note, red5 has alot of features i have yet to test: many of which fms doesn’t.

I’ll keep the site update as i continue testing