ESX Server, NIC Teaming, and VLAN Trunking

Following article is from blog.scottlowe.org

...
There are actually two different pieces described in this article. The first is NIC teaming, in which we logically bind together multiple physical NICs for increased throughput and increased fault tolerance. The second is VLAN trunking, in which we configure the physical switch to pass VLAN traffic directly to ESX Server, which will then distribute the traffic according to the port groups and VLAN IDs configured on the server. I wrote about ESX and VLAN trunking a long time ago and ran into some issues then; here I’ll describe how to work around the issues I ran into at that time.
...

Full article:
http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking

Other useful technical paper about VLAN on ESX is at
http://www.vmware.com/resources/techresources/412

SSH Tunel - howto (Czech Language)

http://www.cryptofest.cz/2003/slajdy/sshtunel/ssh.html

How to Set up VNC on Debian GNU/Linux

Installation and usage

You've just installed Debian, but your wife wants her monitor back. That's OK, you were planning on running it headless, anyway. But, wouldn't it be nice to check out some of those groovy GUI apps? Don't fret, VNC will let you interact with a desktop environment from just about any platform available.

Install vncserver (as root):
apt-get install vncserver

Choose your desired window size and color depth, then, as an ordinary user, open a terminal and type:
vncserver -geometry 1024x768 -depth 24
or
vncserver -geometry 1024x768 -depth 24 -pixelformat rgb565

This will prompt you to create a password:

You will require a password to access your desktops.

Password:
Verify


The server will start and tell you where to access it:

New 'X' desktop is foobar:1

Starting applications specified in /etc/X11/Xsession
Log file is /home/jorey/.vnc/foobar:1.log


Open the VNC viewer on your remote machine, enter the hostname:screen and password (use a hostname or IP that your client machine understands), and your Linux desktop will open in a window! Network speed and processor power will affect performance, but it's amazing how many apps will run fine under VNC. You might not be able to play Frozen Bubble, but you can use productivity applications without any trouble.

To kill the server enter a command similar to this, using the appropriate settings:
vncserver -kill :1

The reason for killing the session is that you may need to edit the default configuration file that vncserver creates for you, for example to get the vncserver to run the K desktop environment instead of twm, you may want to edit the $HOME/.vnc/xstartup file to replace the line:


twm &


with this line is you use KDE:


startkde &


and with this line if you use GNOME:


gnome-session &


before launching the vncserver again using:


vncserver :1 -geometry 1024x768 -depth 16 -pixelformat rgb565


VNC over OpenSSH

The following is just a summary of the full explanation. To use VNC over OpenSSH, first you need to run an ssh session on the machine where you will be running the vncviewer, to request that ssh listen on a particular port on your local machine, and forward communication on that port down the secure connection to a port on the machine running the vncserver.

For example:

ssh -L x:localhost:y vncserver_machine

means "Start an SSH connection to the vncserver_machine, and also listen on port x on my machine, and forward any connections there to port y on the vncserver_machine."

Now, the VNC protocol normally uses port 59xx, where xx is the display number of the server. So a VNC server on a Windows machine, which normally uses display number 0, will listen on port 5900. The first VNC server on linux will probably use display number 1, and subsequent servers would use 2, 3, etc. and so the vncservers on linux will be listening on ports 5901, 5902 and so forth. By forwarding these ports to a remote machine running vncserver, you can make the remote VNC server appear to be a server running on your local machine. So, imagine you had a VNC server running as display :1 on vncserver_machine, and you wanted a secure connection to it from your local machine. You could start the ssh session using:

ssh -CL 5902:localhost:5901 vncserver_machine

After that, starting up the vncviewer as follows on your local machine: vncviewer -encodings "copyrect hextile" localhost:2 would actually connect to display :1 on the vncserver_machine.

Note that the above OpenSSH command-line is deliberately meant to accept incoming connections only from the local machine. This means that to use the ssh connection that we have just set up, we must connect to it from the same machine, using the special name localhost, rather than using the local machine's own unique name.

This article is compiled from following resources:

http://www.joreybump.com/code/howto/debian/vnc.html
http://www.penlug.org/twiki/bin/view/Main/TightVNC

X Window Servers

"X Server" for Windows XP and Vista
http://mediakey.dk/~cc/x11-for-windows-xp-and-vista/

X Ming "X Server"
http://www.straightrunning.com/XmingNotes/

"X Server" for Mac OS X

http://www.apple.com/downloads/macosx/apple/
macosx_updates/x11formacosx.html

@@VMWARE@@ UTS_RELEASE

Paolo Conti wrote how to hack VMware tools to work on linux kernels 2.6.18

[CITATION FROM http://www.atlink.it/~conti/2007/12/19/vmware-uts_release/]

Well, VMWare tools sometimes fails to install into a Linux guest with recent kernel.

The error is something like this: The directory of kernel headers (version @@VMWARE@@ UTS_RELEASE) does not match your running kernel (version 2.6.18.2-34-default). Even if the module were to compile successfully, it would not load into the running kernel.

With this bug you cannot sync the time with your hosting server, automate shutdown tasks, etc… This problem exist because the kernel source code structure is changed in recent kernels (I guess > 2.6.18). The VMWare tools installation script is looking for the string “#define UTS_RELEASE $kernel_number” into /usr/src/kernels/$(uname -r)*/include/linux/version.h but the UTS_RELEASE variable is now into the file utsrelease.h.

To fix this, you can patch the VMWare tools installation script or just add the content of utsrelease.h to version.h. I suppose the second solution is the fastest one

To do this:

cd /usr/src/kernels/$(uname -r)*/include/linux
cat utsrelease.h >> version.h

Happy virtualization folks!

Eth0 disappears after cloning

When I used VMWare Cloning of Debian Gold image everything was OK except networking. Eth0 disappeared and Eth1 came up. It's due to MAC address persistent association in /etc/udev/rules.d/z25_persistent-net.rules. Solution is to avoid persistent association. If you open this file you'll see that old MAC address is associated with eth0 so you can change this MAC address. However, the easiest solution is just delete file /etc/udev/rules.d/z25_persistent-net.rules. File will be created after next reboot but with MAC address of current network card.

System Management & Monitoring server - ZENOSS

Looks interesting http://www.zenoss.com I would like to practically test it. Hope to have time soon.

Apple advertisment

Very nice Apple advertisment against Microsoft ;-)

Very interesting blog about DB & STORAGE performance

Christian has a lot of practical experiences with DB & Storage performance tunning. Look at http://christianbilien.wordpress.com/ and read some articles there. I have similar practical experiences.

AJAX suite

I found very interesting AJAX suite of dynamic web components. See at Zapatec web site.

VMWare appliances with pre-installed Operating Systems

Pre-installed OS images for VMWare should speed up deployment of particular OS. You can find out some images at http://www.thoughtpolice.co.uk/vmware/

Internet FailOver script for FreeBSD written in Perl

I wrote Perl script to automatically detect internet uplink failure and switch over to backup internet link. When primary link is up again script will switch it back. Script must be run in crontab as often as you wish.

#!/usr/bin/perl
use Net::Frame::Device;
use Net::Ping;

$uplink1_interface="sis0";
$uplink2_interface="sis1";
$lan_interface="sis2";
$primary_gateway="10.0.3.1";
$secondary_gateway="10.10.1.1";

# print current date and time
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = gmtime(time);
$year=$year+1900;
print "GMT Time: $year-$mon-$mday $hour:$min:$sec\n";

my $device_uplink1 = Net::Frame::Device->new(dev => $uplink1_interface);
my $device_uplink2 = Net::Frame::Device->new(dev => $uplink2_interface);
my $device_lan = Net::Frame::Device->new(dev => $lan_interface);

print "Current default gateway:", $device_lan->gatewayIp, "\n";
print "Primary default gateway:", $primary_gateway, "\n";
print "Secondary gateway:", $secondary_gateway, "\n";

print "Checking primary internet uplink ...\n";
if (check_uplink($primary_gateway)) {
if ($device_lan->gatewayIp ne $primary_gateway) {
# set default route to $primary_gateway
`route delete default`;
`route add default $primary_gateway`;

# Change uplink interface to uplink1_interface
change_nat_interface($uplink1_interface);

# restart IPFW and NATD daemon
`/etc/rc.d/ipfw restart`;

print "Default gateway has been changed to $primary_gateway via interface $uplink1_interface\n";
} else {
print "Current default gateway is set to primary gateway and is ok\n";
}
} elsif (check_uplink($secondary_gateway)) {
if ($device_lan->gatewayIp ne $secondary_gateway) {
# set default route to $secondary_gateway
`route delete default`;
`route add default $secondary_gateway`;

# Change uplink interface to uplink2_interface
change_nat_interface($uplink2_interface);

# restart IPFW and NATD daemon
`/etc/rc.d/ipfw restart`;

print "Default gateway has been changed to $secondary_gateway via interface $uplink2_interface\n";
} else {
print "Current default gateway is set to secondary gateway and is ok\n";
}
} else {
print "Any uplink is up and working\n";
}

print "----\n";

######################################################
# Ping IP address and decide if it's reachable or not
# 1.parameter: IP
######################################################
sub check_uplink {
my ($ip) = @_;

my $p = Net::Ping->new("icmp");
my $ok = $p->ping($ip);
$p->close();

return $ok; # 0 - false; 1 - true
}

######################################################
# Change NATD interface file
# 1.parameter: network interface
######################################################
sub change_nat_interface {
my ($nat_if) = @_;

$filename="/tmp/natd_iface";
if (open(F, "> $filename")) {
print "change_nat_iface: writing $nat_if to $filename\n";
print F $nat_if;
close F;
} else {
print "change_nat_iface: Cannot write to $filename\n";
return 0;
}

return 1; # 0 - false; 1 - true
}

VMWare Certified Professional

I have passed VMWARE VCP Exam on 12-Dec-2007. CANDIDATE ID: VCP026715

FreeBSD 6.2 and CDMA modem Eurotel Any Data

Howto setup CDMA in FreeBSD 6.2

Excelent documentation for Soekris

Soekris is extremely good hardware for embedded computing and applications.
Documentation and howto scenarios are available at Ultradesic website.
FreeBSD configuration of serial console is documented at Handbook

How to build VMWare image for VMWare Player free of charge

Very good howto is available at this link.

Here is copy of this article ...

If you haven't done so yet, download the Free VMware Player.

Next, you need the qemu-img.exe program that comes with QEMU. If you are using Windows (like I do) you can download QemuInstall-0.7.2.exe. After downloading this program, install it. Start a command prompt and go to the installation directory of QEMU, for example:

cd C:\Program Files\Qemu

and create a VMware disk file as follows:

C:\Program Files\Qemu>qemu-img.exe create -f vmdk WindowsXPPro.vmdk 2G
Formating 'WindowsXPPro.vmdk', fmt=vmdk, size=2097152 kB

A file "WindowsXPPro.vmdk" with a maximum disk size of 2G (the actual file is much smaller about 320 KB) has been created. You might want to move this file to a different folder.

Note: I have made several empty VMware virtual disk files of various sizes available as a single 21 KB download.

Now, create an empty text file, and rename it to "WindowsXPPro.vmx" (use the same name as in the previous step, but use vmx as the extension instead). Open the file in an editor and enter the following values:

config.version = "8"
virtualHW.version = "3"
ide0:0.present = "TRUE"
ide0:0.filename = "WindowsXPPro.vmdk"
memsize = "64"
MemAllowAutoScaleDown = "FALSE"
ide1:0.present = "TRUE"
ide1:0.fileName = "auto detect"
ide1:0.deviceType = "cdrom-raw"
ide1:0.autodetect = "TRUE"
floppy0.present = "FALSE"
ethernet0.present = "TRUE"
usb.present = "TRUE"
sound.present = "TRUE"
sound.virtualDev = "es1371"
displayName = "Windows XP Pro"
guestOS = "winXPPro"
nvram = "WindowsXPPro.nvram"
MemTrimRate = "-1"

ide0:0.redo = ""
ethernet0.addressType = "generated"
uuid.location = "56 4d 5c cc 3d 4a 43 29-55 89 5c 28 1e 7e 06 58"
uuid.bios = "56 4d 5c cc 3d 4a 43 29-55 89 5c 28 1e 7e 06 58"
ethernet0.generatedAddress = "00:0c:29:7e:06:58"
ethernet0.generatedAddressOffset = "0"

tools.syncTime = "TRUE"
ide1:0.startConnected = "TRUE"

uuid.action = "create"

checkpoint.vmState = ""

Insert your Windows XP Professional CD ROM, and double click on the file you just created. The VMware Player should boot the CD, and you can install Windows XP Pro.
Windows XP Pro up and running in a virtual machine
Windows XP Pro up and running in a virtual machine

Other versions of Windows
I first tried to install Windows 2000 Professional. When I got the installation process running I stopped the virtual machine, because I wanted to try XP. The value of guestOS was set to "windows2000pro" in the vmx configuration file.

Speed of disk subsystem can be business critical

INTRODUCTION
As a DELL System/Solution Consultant I designed hardware infrastructure for one Czech commercial ISP which wants to provide IPTV and VoD. ISP choosed software IPTV/VoD/DRM solution based on Linux OS. Together with software provider we choosed several servers PE 2970 which are AMD (x86_64) based servers. Streamer server needs cost-effective however fast enough disk subsystem. Unfortunatly, 2.5" hdd are available only as 10k rpm so streamer has eight 2.5" SAS 73GB 10k rpm in RAID5 where RAID is provided by internal DELL Power Edge RAID controller PERC 5/i (re-branded LSI MegaRAID SAS). Software provider has load tests which are able to recognized if system is good enough for requested solution. Software provider has reference hardware configuration with reference results. Reference hardware is server SuperMicro with eight 3.5" SCSI 36GB 10k rpm connected to RAID controller Areca ARC-1260. Disk subsystem is tested by real load test and also by synthetical disk benchmark utility "iozone". We can try to tune disk subsystem and check progress by synthetical benchmark tool.

RESULTS & TUNING
Here are results from reference hardware (SuperMicro):
1 THREAD
iozone -s 32g -r 1m -i 0 -i 1 -t 1 -b /tmp/test.xls
Initial write 299951.25 kB/s
Rewrite 254075.39 kB/s
Read 386271.91 kB/s
Re-read 388288.41 kB/s

2 THREADS
iozone -s 16g -r 1m -i 0 -i 1 -t 2 -b /tmp/test.xls
Initial write 327977.77 kB/s
Rewrite 321502.41 kB/s
Read 312530.73 kB/s
Re-read 315091.88 kB/s

10 THREADS
iozone -s 4g -r 1m -i 0 -i 1 -t 10 -b /tmp/test.xls
Initial write 293753.92 kB/s
Rewrite 281009.19 kB/s
Read 262086.25 kB/s
Re-read 260864.43 kB/s

Software provider tested DELL PE 2970 with these results:
1 THREAD
iozone -s 32g -r 1m -i 0 -i 1 -t 1 -b /tmp/test.xls
Initial write 339329.00 kB/s
Rewrite 325063.56 kB/s
Read 337726.91 kB/s
Re-read 320971.62 kB/s

2 THREADS
iozone -s 16g -r 1m -i 0 -i 1 -t 2 -b /tmp/test.xls
Initial write 356046.38 kB/s
Rewrite 359441.64 kB/s
Read 193787.55 kB/s
Re-read 194154.85 kB/s

10 THREADS
iozone -s 4g -r 1m -i 0 -i 1 -t 10 -b /tmp/test.xls
Initial write 296494.20 kB/s
Rewrite 281730.82 kB/s
Read 147723.06 kB/s
Re-read 148728.67 kB/s

We can see significant difference between reference hardware and DELL hardware. DELL is better in write throughtput but worse in read throughtput. Default parameters of DELL servers are preconfigured for database servers where are usually different requirements then for streaming applications. Streaming applications don't need write but read performace. DELL received request from software provider to optimize infrastructure for better read performance.

Tunning of disk subsystem is not easy task and it's depended on lot of aspects. In this particular environment we have OS Linux Debian 4.0 (kernel 2.6.x), filesystem XFS, Raid 5, PERC 5/i. Debian is not certified and supported operating system so customer cannot use standard DELL tech-support but sometimes DELL can help to their customers in some particular complex enterprise solutions.

For increase read performace - especially sequence reads - is very important to set up read-ahead cache. PERC 5/i can be in three modes - adaptive, read ahead and non-read ahead. PERC is by default in adaptive mode which means that PERC use internal algorithm to automaticaly recognize when to use read-ahead. In this particular solution we can explicitly set up "read ahead" mode in RAID management. Another very important point is to set up read-ahead in operating system Linux block device layer.

Linux kernel 2.6
Set the value to 8192 blocks using the blockdev command, for example
blockdev --setra 8192 /dev/sda
this example is setting up 4MB Cache (8192 blocks of 512-byte sector)
which is aligned with default XFS parameters see. xfs_info for current XFS parameters

DELL IOZONE tests reults on DELL PE 2970 with tuned block device layer:
2 THREADS
iozone -s 16g -r 1m -i 0 -i 1 -t 2 -b /tmp/test.xls
Initial write 290692.31 kB/s
Rewrite 359531.20 kB/s
Read 503044.62 kB/s
Re-read 496045.61 kB/s

10 THREADS
iozone -s 4g -r 1m -i 0 -i 1 -t 10 -b /tmp/test.xls
Initial write 297497.92 kB/s
Rewrite 279933.16 kB/s
Read 473969.33 kB/s
Re-read 481384.16 kB/s


CONCLUSION
It's possible to successfully tuned up disk subsystem by set up read-ahead parameters. Read throughtput of DELL PE 2970,PERC 5/i,8xhdd 10k rpm was increased approximately 3 times so we achieved great results in synthetical benchmark (iozone) and overcome reference hardware results. What we can see is that disk performance lowers with more concurrent threads. Local RAID controllers are designed as disk storage for one server where I/O stress is not so high. If someone is looking for storage without I/O stress issues he should focus to SAN disk arrays which are designed for environments with lot of servers, proceses and threads.

Monitoring server

Server for monitoring IT services is crucial for effective management of IT infrastructure and to guarantee SLA. Two years ago we developed something like that and in the end it was realy usefull piece of software. ZABBIX is very interesting open-source project for network and service monitoring. For further information check out http://www.zabbix.com

Fedora Core 5 on USB hard disk

I have DELL notebook Latitude D610 with preinstaled company standard image. We have lot of restrictions and security constraints in this image. It's absolutly correct from security and company's point of view. But because I'm working as solution/system consultant I have to test a lot of systems, softwares and solutions to have a practical experiences with latest technologies. So I had two possibilities. To buy another hardware for testing or use a company notebook with another hard disk and another OS. Second posibility is chiper and more convinient for me. You can pretty easy demount internal HDD from DELL Latitude 610 but it's not so professional. Much better way is to use external USB hard disk and during the boot press F12 key and choose a boot device. Because we are suggesting virtualization tou our customers I decide to install linux and VMWare. I choosed linux distro Fedora Core 5 (FC5) as hosted OS. There are some hacks haw to install FC5 to external USB disk. First of all you have to choose different type of installation method from DVD/CD medium. You have to use option "linux expert" which is not sugested in instalation wizard. Only "linux expert" and "linux rescue" are loading USB modules so it's working with external USB disk. After common CD/DVD installation you have to reboot. But after reboot system is not booting propperly from external hard disk. You can see "kernel panic" on the screen. The reason is that generic instalation doesn't contain USB modul. There is work-arround. You have to boot from CD/DVD to rescue mode (linux rescue). Installed system is mounted to /mnt/sysimage/. You have to mount boot image by command
mount /dev/sda1 /mnt/sysimage/boot
set a chroot directory
chroot /mnt/sysimage
and create new initrd image with modules for USB storage
mkinitrd ––preload=ehci–hcd ––preload=usb–storage ––preload=scsi_mod ––preload=sd_mod /boot/initrd-2.6.15-1.2054_FC5.img 2.6.15-1.2054_FC5

This work-arround change booting image to new boot image with USB disk support and after reboot Linux OS should boot from external USB hard disk without any problem.

VMWare instalation to be continued ...

Business proceses and Workflow Software - Part II.

Continued from part I. ... We found (with my collegue Juraj) much more enterprise software suits for BPM. It looks that BPM is trendy and lot of vendors try to catch market share. You can find these vendors in Gartner Magic Quadrant: Tibco Software, Lombardi, Savvion, Pegasystems, Fuego, IBM, and others. After our "hot" discussion about the best tool to use for workflow application we decide to use OpenSource JBoss engine jBPM and develop other components in-house in SUN Java Studio Creator which looks like pretty interesting RAD (Rapid Application Development) tool for web based applications. The reason of this decision is that we have programmers background and it looks like best way for us. It's not a common way of course :) I will write later about our experiences.