Tuesday, July 28, 2009

Order matters when adding doubles

I recently discovered (or rather, was told) that summing a set of doubles from a database query may produce different values, even if the database is in a read only state and no programmatic or system errors occurred. Struck me as a bit odd until I realized that order in which the results returned from a query are not guaranteed unless an order by is specified. As a result the sequence in which the doubles are added together may change, thus the sum of those doubles may be different!

An easy way to explain this is that if the doubles are returned in an order where large doubles appear first, the small doubles do not have time to cumulate and add up to anything meaningful. Instead these doubles may be truncated or even ignored because they individually can not impact the current rolling sum. I wanted to explore this a bit further, so i wrote a java program to generate some doubles and add them up in random, ascending and descending order.

Lesson of the day? If you're creating an application that requires all the significant figures available to you in a double either very careful in your arithmetic or use a datatype with precision to spare!

java AddingDoubles 10000 10
Sum in random order: 5.022355325590349E12
Sum in decending order: 5.022355325590318E12
Sum in increasing order: 5.022355325590351E12

Code for AddingDoubles.java below.


import java.util.Random;
import java.util.Date;

/** Adding doubles demonstrates how precision may be lost
 depending on the order in which doubles are added. This effect
 is especially true when some doubles are small relative to others.
 */
 
public class AddingDoubles {
    
    public double m_data[];
    
    public AddingDoubles(int nSmall, int nBig, int nMultiplier) {
        m_data = new double[nSmall + nBig];
        Random r = new Random();
        
        // create small doubles
        for(int i = 0; i < nSmall; i++) {
            m_data[i] = r.nextDouble();
        }
        
        // create 'large' doubles
        for(int i = 0; i < nBig; i++) {
            m_data[i + nSmall] = r.nextDouble() * nMultiplier;
        }
        
        // randomize the array
        shuffle(m_data);
    }
    
    private static void shuffle(double[] a) {
        int n = a.length;
        
        for (int i = 0; i < n; i++) {
            int r = i + (int) (Math.random() * (n-i)); 
            double temp = a[i];
            a[i] = a[r];
            a[r] = temp;
        }
    }
    
    public void sort( ) {
        java.util.Arrays.sort( m_data );
    }

    public double sum( boolean reverse ) {
        double total = 0;
        
        int i = reverse ? m_data.length - 1: 0;
        int inc = reverse ? -1: 1;
        
        while( i >= 0 && i < m_data.length ) {
            total += m_data[i];
            i += inc;
        }
        
        return total;
    }
    
    public static void main(String[] args) {
        int s = Integer.parseInt(args[0]);
        int b = Integer.parseInt(args[0]);
        AddingDoubles ad = new AddingDoubles(s,b,1000000000);
        
        System.out.println( "Sum in random order: " + ad.sum( true ) );
        ad.sort();
        System.out.println( "Sum in decending order: " +ad.sum( true ) );
        System.out.println( "Sum in increasing order: " + ad.sum( false ) );
    }
    
}

Sunday, July 5, 2009

ESXi 4.0 on the Thinkpad T61 (Intel ICH8M)

I sometimes need an ESX instance that I can potentially destroy at will. I also didn't want to have to have to purchase a supported system as the cost is many times that of normal commodity hardware. An ESXi whitebox (i.e. systems that are not on VMware's Hardware Compatibility List (HCL)) are perfect for this. ESXi found my T61 laptop's Intel ICH8M SATA controller and the 82566MM NIC. It allowed me to create a VMFS (but forced me to completely wipe the local drive, there were no options to only create the VMFS in free space). I could even create VMs and boot them. So, on the surface it all looked fine. But I quickly discovered performance was worse than expected (and I wasn't expecting much on a laptop HD), but what really doomed the setup was the that sometimes the storage would just stopped responding for minutes at a time. I'd open up storage browser and the system would just sit.

So, the summary is that yes ESXi will boot on a Thinkpad T61 (or X61), but you're going to need some NFS or iSCSI storage to create stable setup that will run VMs. I ended up switching to Hyper-V for now, but ultimately I'll move to KVM.

Tuesday, June 23, 2009

OCZ Vertex SSD performance on the Lenovo X61

I recently replaced my 160GB 7200 RPM hard drive in my Lenovo X61 with a 60 GB OCZ Vertex SSD. The OCZ Vertex uses the new Indilinx controller instead of the Jmicron controller. The Indilinux uses an FPGA and 64 MB cache vs JMicron's much smaller cache (I've heard on the order of KB?). The performance difference is certainly noticable. Everything starts instantly, searching on outlook is very quick, boot time is about 25 seconds.

But at the end of the day, I really don't need this much speed. My 7200 drive was probably fast enough and the reduction in heat and increases in battery life were measurable, but minor. Another limitation is that the SATA I controller on the X61 is saturated by this SSD drive. My testing indicated a max read speed of about 100 MB/s vs. 150+ MB/s on a SATA II desktop controller. So, all in all not a tremendous boost in performance.

But, the drive's ability to withstand over 15Gs of shock (while operating) is a great reason for everyone to have one of these! No more 'clicking' noises coming from your hard drive, no worries about losing all your data if the machine takes a fall while operating and I have to admit that virtualization technologies (like sun's Virtual Box) do show significant speed ups.

OCZ Vertex 60GB

Sun Cloud Partner Lightning Talk

VMLogix just announced its first Cloud offering: LabManager Cloud Edition. It's now open to a limited beta where we will be collecting feedback before an open beta in a few months.

In preparation for this launch, our partners at Sun Microsystems granted me a 5 minute speaking spot during the Cloud Lightening talks during Community One West.

I had a lot to cover in a few minutes, but the main points were:

  1. Cloud solutions will be adopted by developers first because they are by nature the people that like to tinker with new technology (and use it to tackle old problems)
  2. The development and test lab is a natural, low risk area to start experimenting with the cloud. Especially compared to production or even UAT/Staging systems.
  3. Cloud computing solves real problems in dev/test - such as faster provisioning of complex environments, better test matrix coverage and better issue reproduction.

I close it off explaining that the pay as you go model is especially attractive in the dev/test lab because demand is often spiky.

Sun posted the entire talk over at Sun Channel. You'll find plenty of other speakers and coverage there too.



Sun Cloud partner Jim Singh, Director of Technology at VMLogix, delivers a five-minute lightning talk on using the Sun Cloud for Virtual Lab Automation solutions.

Monday, June 22, 2009

Moving a windows server from on premise (VMware ESX or RHEL 5) to EC2

One use case that I expect to be fairly common as Cloud Computing gains adoptions is the need to migrate certain existing systems (phyiscal or virtual) to a cloud provider. It seems that today one must setup the Windows or Linux machine again in the cloud, which might be fine for one or two systems, but what if you had 100?

My experiment to automatically migrate systems into the cloud did not go as well as I had hoped. EC2 (and all other infrastructure as a server (iaas) cloud providers) do not accept uploads of Windows AMIs, OVF, VHD or VMDK. EC2 does accept Linux AMIs, but that's not what I was attempting. That left me with limited options:

1) Install virtual box under in a linux AMI and use OVF to send my VMs to EC2. This would probably work, but hardly seemed stable. I tried it on Windows, but starting virtual box hard locked the VM.

2) Use ntbackup to perform a shadow volume copy of the on premise system and restore the backup into the EC2 instance. Amazingly, this does works - assuming you have a full day to spend on it. The issue is that the windows system must be setup perfectly for EC2 usage before you take the backup.

The real gotcha here is that EC2 doesn't allow console access to the VM, you only have access via RDP. Since you are never really in front of the terminal, any driver failures and you're out of luck. I found that moving from a RHEL 5.3 VM to EC2 was the most reliable means of migration, because the paravirtualization drivers are the same (EC2 runs on open source xen and RHEL).

So, how do you prepare your windows instance before backup?

0) Migrate everything to a RHEL 5.3 system running Xen

1) Enable autlogon as an administrator account. This is critical to allow new hardware to be detected and process the environment change from RHEL 5 to Amazon EC2.

Here is my batch script to do this:
REM enable autologin
net user administrator password
reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /f /v AutoAdminLogon /t REG_SZ /d 1
reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /f /v DefaultUsername /t REG_SZ /d "administrator"
reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /f /v DefaultPassword /t REG_SZ /d "password"
REM Disable Shutdown Event Tracker reg add "HKLM\Software\Policies\Microsoft\Windows NT\Reliability" /f /v ShutdownReasonUI /t REG_DWORD /d 0
2) Disable driver signing checking and the new server wizard.

.reg file

REGEDIT4  [HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\DriverSearching] "DontSearchWindowsUpdate"=dword:00000001 "DontPromptForWindowsUpdate"=dword:00000001 "DontSearchCD"=dword:00000001 "DontSearchFloppies"=dword:00000001  [HKEY_CURRENT_USER\Software\Policies\Microsoft\Windows NT\Driver Signing] "BehaviorOnFailedVerify"=dword:00000000  [HKEY_USERS\.DEFAULT\Software\Policies\Microsoft\Windows NT\Driver Signing] "BehaviorOnFailedVerify"=dword:00000000  [HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\PlugPlay\Parameters] "SupressUI"=dword:00000001  [HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\CurrentVersion\MYS] "DisableShowAtLogon"="dword:00000000  [HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\srvWiz] @=dword:00000000 "CYSMustRun"=dword:00000001

3) Use this autoit script to accept and process any found new hardware wizards and other system messages. You can get autoit from here: http://www.autoitscript.com/autoit3/

$allowed = 240 * 1000
$time = TimerInit()
While $allowed > TimerDiff($time)
Select
Case WinExists('Service Control Manager')
WinActivate('Service Control Manager')
Case WinExists('Confirm File Replace')
WinActivate('Confirm File Replace')
Case WinExists('Found New Hardware Wizard')
WinActivate('Found New Hardware Wizard')
Case Else
Sleep(5000)
ContinueLoop
EndSelect

Sleep(250)

Select
Case WinExists('Service Control Manager')
Send('{ENTER}')
Case WinActive('Confirm File Replace')
Send('!a')
Case WinActive('Found New Hardware Wizard', 'Cannot Install this Hardware')
Send('{TAB 2}{ENTER}')
Case WinActive('Found New Hardware Wizard', 'The wizard has finished')
Send('{ENTER}')
Case WinActive('Found New Hardware Wizard', 'Completing the Found')
Send('{ENTER}')
Case WinActive('Found New Hardware Wizard', 'Welcome to')
Send('!n')
EndSelect
WEnd

4)Set the autoit script to start on boot by adding it to the all users startup program group.

5) You may want to capture a movie of everything going on in the system, just in case things fail. So use http://camstudio.org/blog/camstudio-command-line-v01-released to do a screen movie via the command line. You probably want to save this to an EBS volume on E.

6) You'll want to install the EC2Config tools into your image. [this section left intentially vague, because I can't tell you where or how to get these.]

7) Make sure your system is set to DHCP. EC2 doesn't like static IPs. Make sure that the MAC address is not overriden and the system will come up with the original MAC. I was thinking of just removing the NIC all together, but then you have to save the backup in the next step to a seperate drive and boot the drive in a new windows instance.

8) Reboot! make sure the system logs in automatically and runs autoit. You may want to remove your network card from device drivers and then reboot just to make sure it can be added correctly.

9) backup the entire system (c drive and system state). Compress it. Send to EC2 and restore.

10) Reboot the instance when ready. You should see a message saying EC2Config is restrating this.

Simple, right?

Tuesday, June 16, 2009

ESX Update 4 and ICH7 motherboards

ESX 3.5 Update 3 and 4 no longer allow VMFS on ICH7 controllers - but improves access to CD/DVD ROM drives attached directly to the host and allows guest operating systems to access them. That's probably the right trade off for most people, but my ESX whitebox could no longer find its VMFS storage after the update to ESX Update 3. I found http://communities.vmware.com/message/1222880 VMware communities discussion and resolved the issue. In short the steps were:

1) download the update ESX350-200803213-UG from http://support.vmware.com/selfsupport/download/
2) extract VMware-esx-drivers-scsi-ata_piix-350.1.05-82663.i386.rpm from that tar file
3) rpm2cpio VMware-esx-drivers-scsi-ata_piix-350.1.05-82663.i386.rpm > VMware-esx-drivers-scsi-ata_piix-350.1.05-82663.i386.cpio
4) mkdir ata_piix; cd ata_piix
4) cat ../VMware-esx-drivers-scsi-ata_piix-350.1.05-82663.i386.cpio | cpio -id
5) cp * / [This step will copy the ata_piix drivers to their correct locations on the filesystem]

The initial boot of the host is done from the initrd, which is a packaging of essential drivers and other components that are loaded right after the kernel. The initrd mounts the root filesystem after it's done loading. As a result, the initrd needs these the older version of the ata_piix drivers as well).



Repackage the initrd in /boot/initrd-2.4.21-57.ELvmnix.img and run esxcfg-pciid:


1) cp /boot/initrd-2.4.21-57.ELvmnix.img /root/initrd-2.4.21-57.ELvmnix.img.gz
2) gzip -d initrd-2.4.21-57.ELvmnix.img.gz
3) mkdir initrd
4) mount -o loop ./initrd-2.4.21-57.ELvmnix.img initrd
5) cp /usr/lib/vmware/vmkmod/ata_piix.o to /root/initrd/lib/vmware/ata_piix.o
6) umount initrd
7) gzip initrd-2.4.21-57.ELvmnix.img
8) mv /boot/initrd-2.4.21-57.ELvmnix.img /boot/initrd-2.4.21-57.ELvmnix.img.orig
9) mv /root/initrd-2.4.21-57.ELvmnix.img.gz /boot/initrd-2.4.21-57.ELvmnix.img
10) esxcfg-pciid


Reboot esx and cross your fingers. I later found this to be slow/unstable on my ICH7 based motherboard and upgrading to the ASUS P5Q-EM ICH10R based board, which works out of the box with update 4 (once you add an Intel e1000 NIC)

Asus p5q-em and Intel e1000 NIC, both on Amazon.com

Monday, June 8, 2009

Dell Mini 9 (Atom N270 CPU) and virtualization

The Dell Mini 9's N270 Atom processor does not have VT support, which Xen requires for full virtualization. So, there's no way to use open source Xen to run windows guests on the mini 9. VMware Server or Virtual Box might work - haven't tested either one. ESX (full install) doesn't see the network card on this machine. I haven't tested ESXi though ...