- Asus P5BP-E/4L motherboard
This motherboard supports an Intel S775 processor, has VGA and audio onboard and most importantly, the LAN controllers on this motherboard are ESX certified (Broadcom 57xx chipset). - Intel Q6600 Quad Core processor (2.4 GHz) and 8 GB ECC RAM (4x 2GB)
Just to be sure I have enough CPU power and memory resource pools :) - Dell Perc 5i Integrated SAS Controller
My colleagues advised me that storage was the biggest bottleneck in their ESX whiteboxes (based around the very nice Asus P5M2/SAS board). I decided to go for a dedicated hardware controller. I picked up the Dell Perc 5i controller, which is more or less a rebranded LSI Logic 8408 SAS controller on EBay with 256MB of RAM and a battery backup unit for about 175 EUR.
The main advantage of SAS controllers is that they also support the (cheaper) SATA consumer drives. A quick test confirmed this; I had absolutely no problems at all with this controller & even flashed the latest LSI Logic firmware to it :).
Maybe of interest for some: the later Dell firmwares and also the later LSI logic firmwares for this controller provide support for Write Back without a BBU present. - SATA to SAS cables
The Dell Perc 5i has SFF-8484 SAS connectors on board, so I purchased two Adaptec SFF-8484 to 4xSATA cables from a nearby store to attach all the drives. - 8 Seagate SATA harddisks (4x 1TB and 4x 200GB)
Space... loads of space.
The installation of ESX 3.5 was a piece of a cake & and I can confirm that the above hardware works like a charm. For those interested, I also noticed that ESX 3.5 supports the ICH7 SATA controllers (found on many consumer motherboards as well). I think -- but this has to be confirmed by someone else -- that you need to configure your ICH7 disks in a RAID before the ESX kernel will accept them as a storage pool.
20 comments:
Hi there,
Just wanted to report that you do not need to create a raid in order to have the sata hdd recognized by ESX. I have tested this with an Asus barebone (V3-P5945GC@90 Euro)with a single samsung 750GB drive. ESX installs successfully the only issue I have is that the onboard nic is not recognized. Hope this helps.
Grtz,
Herc.
One reason for an actual SAS/SATA controller is RAID support. If you don't have RAID I assume ESX 3.5/3i will see the the storage as JBOD. A drive failure could be bad.
How many VMs are you running? How's the performance?
Cheers,
James
Do you know if 3.5 works on intel ESB2 controller?
James,
At the moment I am running 7 virtual machines, which is quite a low load:
- Two Windows 2008 servers
- Two Windows 2003 servers
- Two Windows XP machines
- One m0n0wall firewall/router
The total CPU taken by these 7 virtual machines at idle is 440MHz, the total memory taken 2.65 GB (512Mb for the W2K8 machines, roughly 300Mb for the XP machines, the rest for the W2K3 servers).
Regarding the question whether the Intel ESB2 controller is compatible, the only answer I can give is to look at the official hardware compatibility guides (http://www.vmware.com/support/pubs/vi_pages/vi_pubs_35.html) which document the kernel modules that ESX supports. If it is not there, it is not supported. Feel free to provide a piece of hardware for testing :D.
Bye,
Tim
Regarding the ICH7 controller ESX will only recognize drives connected in a non-RAID mode. That's the case with most of the embedded (low end) RAID controllers found on lower end motherboards, as they require a software component to function which ESX does not provide.
I've had my eye on that PERC5i SAS controller for some time now. But I keep hearing rumors about it being slow. Care to comment how's the beast performing, from your own experience?
Regarding the speed of the Dell Perc 5i controller: I've read people mentioning write speeds of up to 250 MB/s on that controller, so the throughput capacity surely is there. It depends a lot on the disks you use on the controller and whether you put it in a 1x/4x/8x PCIe slot.
I regularly see peaks of 60MB/s write on a four-SATA-disk 7200 RPM RAID5 array (a network copy to a virtual fileserver), without any other virtual machine noticing anything of that load. That 60 MB/s is roughly the network throughput you can attain over a gigabit network.
By stressing a virtual machine itself, I can achieve write speeds of up to 150 MB/sec, again without any other virtual machine noticing performance loss. I suspect that even higher performance can be gained by using 10k/15k RPM disks in a real SAS setup.
Having said that, do realize that the Perc 5i is an "old" controller and that there are more performant controllers on the market. However, for a second hand $175 deal (which is already expensive if you look at some eBay bargains you can get today), you don't hear me complain at all :D.
Tim,
Where did you buy your P5BP-E/4L? The only sites online that have it are in Australia or the UK?
Thanks,
Tim,
Does ESX 3.5 also work with the local SATA controller on this mainboard? I first want to use local storage and after a few weeks a dedicated RAID controller.
Thanks!
Yes, this board uses an Intel ICHx SATA controller which is supported by ESX 3.5.
Hi tim
this looks ideal for my requirements, have you tested ESXi with this setup ?
cheers
Hello,
I also want to know if you have tested this setup with esxi?
Would appreciate an answer, as I am in the process of building an esx/esxi compatible system my self.
Hi Thomas & anonymous poster,
I have not explicitly tested the board with ESXi, but since the hardware compatibility list is the same for ESX 3.5 and ESX 3i, I do not expect any issues at all.
Keep up the good work.
Tim, nice system ... may I ask what disk you built in?
Cheers,
Hannes
Hi Hannes,
I put in 8 Seagate Barracuda disks. 4 x 7200.10 200GB series for VM storage and 4x 1TB 7200.11 series storage for data storage.
Bye!
Tim
Tim,
Nice machine. I have been putting a whitebox together as well. This is my first experience with raid and a server quality machine. I am a web developer not really an expert in Systems or networks.
Anyway, I also bought a perc5i off ebay. I have a question, I can definitely configure this via setup ctr+r, but what if I need to configure this remotely running esx 3.5? Do you know of a way?
Also, I configured mine raid 10 64k stripe. Do you have any performance tips for me? I do have the battery backup. I will be running only a few VMs - Centos Server, Window 2008, and mono firewall.
-Thanks
Adam
How do you tell if a disk has failed?
Hey DM,
I blogged about how you can use the LSI tools under VMware ESX to detect disk failures. For more information, see:
http://timjacobs.blogspot.com/2008/05/installing-lsi-logic-raid-monitoring.html
Bye!
Tim
Post a Comment