I’ve been slacking on the posts recently. There has been a lot going on. I’ve switched jobs and I’m now working for E.W. Scripps’ Online Division. E.W. Scripps is newspaper company based in Knoxville and also formally part of HGTV, FOOD Networ, etc. I was brought on to help split out the web servers during the company split.
We’ve made a lot of progress on getting things running more effeciently. We’ve recently upgraded from Postgres 8.1.3 to 8.3.3 which made a huge improvement. Even though Postgres claims that writes don’t block reads and reads don’t block writes, we were seeing a lot of processes getting hung waiting on a commit to finish. I think that it was having problems starting a new transaction while a transaction was commiting. Postgres 8.3 introduced asyncronous commits, so that we no longer have to wait for everything to be flushed to disk before we received the commited status. Now we immediately get a commited status and transactions are written to disk as they can be.
This has been a HUGE performance gain for out database. Before we had seem the database hit loads over 300 while getting Drudged. Now we never see the database get above a load of 10, they ususally over around 3 to 4.
Next we will be working on clustering Postgres. I have evaluated CyberCluster, PGCluster, Slony, PGPoolII and Sequoia. The only one that seems to be reliable and meet our needs has been Sequoia, so we will see how that goes.
Oh, and I’ve also got a new MacBook Pro. More on that latter.
Ever been frustrated by a processes that you could not kill on a Linux box? I know I have, especially working with Samba and NFS. Good news, there is a new sleep state that will help eliminate the problem and more code makes use of it. In fact most of the NFS code has already been converted. Check out the link here and read more about it: http://lwn.net/Articles/288056/
I’ve tried the firefox 3 beta a couple times on my linux box. Every time I try it the fonts and toolbars are obnoxiously big. I couldn’t find any help on this. It seemed like a dpi issue, so I went to about:config and searched for dpi. There it was layout.css.dpi. Mine was set to -1, I changed it to 150 which seemed about right.
Type about:config in you browser window. Then type dpi in the filter text box and double click the layout.css.dpi to change the value. I changed mine to 150.
Two things I ran across today that I need to look further into.
http://latencytop.org/ - This shows what processes are waiting on and looks very interesting.
The other is Group Scheduling in Linux 2.6.24 and above. This allows you to put processes in a group so that one set of processes can share the same resources. For example, you can assign two different users 50% of the processor and one use could be running 50 compiles but should not affect the performance of the second user.
As I said in my previous post, I learned a couple things about bash today. I had a nice for loop that outputted the info I wanted, but not in the right order. I thought to myself, man if this were php I would just put it in an array and sort it before looping across it. I bet this will be a pain in bash.
BUT, I was pleasantly surprised to find out how simple this really was. If you want to perform an action the output of a loop in bash then you just pipe done into the fuction you want performed. See the example below.
for VAR in $VARS; do
{someaction}
done | sort -n
Cool, right?
Learned a couple things about bash today. I was wanting to print something out nicely, so I was using tabs to separate fields. As you know, nothing ever lines up if you do not have values very similar in size. So I was about to search Google for the problem when I had a moment of sheer brilliance. 
I thought I’ll just concatenate a bunch of spaces on the end of the variable and then use cut to grab how many characters I want. I was very proud of myself, but then bash quickly humbled me by condensing all my spaces I had added to the variable.
So, back to the all knowing Google. Turns out you have to quote your variable when you use it if you don’t want bash to truncate the spaces. So I quoted my variable when using it and all is good.
BUFFER="$VAR "
NICE_VAR=`echo "$VAR" | cut -c -15`
echo "$VAR $VAR2"
When trying to install an older version of the JRE on a newer Linux distro I was getting the following error:
Unpacking...
tail: cannot open `+480' for reading: No such file or directory
Checksumming...
1
The download file appears to be corrupted. Please refer
to the Troubleshooting section of the Installation
Instructions on the download page for more information.
Please do not attempt to install this archive file.
This is because tail has changed in the newer Linux distros. the +{number of lines} does not work unless you proceed it with a -n. So, edit the .bin file you downloaded make the following changes:
Replace: tail +480 $0 > $outname
With: tail -n +480 $0 > $outname
Replace:
if expr $sum1 != 22444 || expr $sum2 != 13542 ; then
echo “The download file appears to be corrupted. Please refer”
echo “to the Troubleshooting section of the Installation”
echo “Instructions on the download page for more information.”
echo “Please do not attempt to install this archive file.”
exit 1
fi
With:
# if expr $sum1 != 22444 || expr $sum2 != 13542 ; then
# echo “The download file appears to be corrupted. Please refer”
# echo “to the Troubleshooting section of the Installation”
# echo “Instructions on the download page for more information.”
# echo “Please do not attempt to install this archive file.”
# exit 1
# fi
I usually use phpMyAdmin to dump my table to SQL, but sometimes it is easier to do it from the command line. Use the following syntax to dump a table to SQL.
mysqldump -p -e -c --add-drop-table -r table.sql dbname table
Warning: The –add-drop-table will cause it to drop the existing table when you import.
You can import this with the command
mysql -p dbname < table.sql
Looks like this filesystem has some promise, hope it matures quickly.
http://oss.oracle.com/projects/btrfs/
I was thinking to myself it would be really great if Dell made available a yum repository for open manage. So I did a search and sure enough the do! This makes my job a lot easier
http://linux.dell.com/repo/hardware/
Also check out http://linux.dell.com/wiki/index.php/Repository/software
and http://linux.dell.com/wiki/index.php/Repository/firmware
Trylinux.org