The Mac Mini VMware ESXi 5 Server – Part 3 (Follow Up)

The Mac Mini VMware ESXi 5 Server – Part 3 (Follow Up)

January 18th, 2012 // 9:31 pm @

The Mac Mini VMware ESXi 5 Server – Part 3 (Follow Up)

This posting will get regular updates in regards to comments and questions that occur in the previous 2 sections.  If you haven’t seen the other two sections, then please follow the links below

[Update for quick links 1/30/2012]

To make linking to the other pages easier….

Overview

From the numerous comments and questions from the Installation of the blog entry, we have decided to section up each question that we can reasonably answer.  Since we’re actually using these boxes on a daily use, we’re not as interested in making them dual boot or how we can make them into portable ESXi hosts when we want to.  If we get more equipment in the future, we’ll try those ideas out, but for the most part we just want it to work.

As for general stability, over a 15-20 day period, everything has been stable and running 24/7 and there have been no unexpected crashes, errors, or alarms appearing via the ESXi console.

Uptime (1/18/2012)

The question of uptime, came about from Timo.  From the two Mac Mini Servers that we have in use, here’s some screen shots.  As you can clearly see, one of the servers has had an uptime of 15 days already while the second one which was purchased a few days later after the theory was proved, has an uptime of 9 days.  (This is in reference to January 18, 2012)

Upgraded Memory (1/18/2012)

As for upgrading the memory, to fully make use of the advantages of this setup, 16 gb of most desired.  Here’s a screen shot of one of the Mac Mini ESXi hosts with 16 gb of memory and the VM utilization.

Network Card (1/18/2012)

The network card responded as Gigabit and Full Duplex.  We have not seen any slow file copies or VM migrations / copies from one Mac Mini to the other.

 Any thing else you would like to see?

Post some comments and questions and we’ll try our best to answer them as quickly as possible.


Category : Blog

51 Comments → “The Mac Mini VMware ESXi 5 Server – Part 3 (Follow Up)”


  1. Timo Sugliani

    5 years ago

    Many thanks,

    Hmm, I’m getting very interested for a homelab with such size factor/power :-)

    Reply

  2. Matt

    5 years ago

    Again, this is a great Proof of concept and many thanks for documenting it so well. As for something I’d like to see, do you have any iSCSI targets laying around looking to become storage for some VMs? I’m wondering how the single 1Gb/s ethernet port would/could handle the demands of iSCSI and all other network activity of Host and Guests. Second, is it possible to get the wireless adapter functioning under ESXi 5? If the wireless adapter does function you could devote the 1Gb/s ethernet to the iSCSI and route the other network traffic through the wireless. I can’t imagine the vSphere Client over wireless N speeds would be too terrible…??

    Reply

    • Paraguin

      5 years ago

      If you have any suggestions / programs for measuring I/O with a VM and iSCSI target, I’m more than willing to try to find someone with some spare SAN storage either SOHO or Enterprise who will lend me some time. As for the WiFi chipset, that’s something else we’re looking into for the long run though we’re somewhat doubtful since you’d also need a WiFi manager through the CLI. If someone has made an ESXi server with WiFi on and able to connect to an access point, then show us and we’ll work on the driver issue.

      Reply

  3. Timo Sugliani

    5 years ago

    Just for my personal information : Did you also try with the normal mac mini (core i5) as I don’t need much horsepower but only memory (I’ll be having many VMs but with very little load on them, so don’t think I’ll need the extra power or Hard Drive.

    Because i5 + 16GB could come up with a rather interesting entry price compared to the mac mini server ‘edition’.

    Thanks in advance,

    Reply

    • Paraguin

      5 years ago

      I’m pretty confident that it’ll work. I have a Mac Mini i5 with 16 gb of ram running VMware Fusion to act as the VCenter and backup servers. Best of luck.

      Reply

  4. eloscurosecreto

    5 years ago

    This is very interesting. Thank you for the eloquently providing this information to the world. From reading ‘Part 2′ (technical section), it seems that the HDMI and USB interfaces are fully functional. Have you tested the ThunderBolt interface? I’m interested in finding out if ESXi detects this and if it can be used for attaching storage. I’d like to take full advantage of the 1010Gbit/s.

    Reply

    • Paraguin

      5 years ago

      No. I believe the only thunderbolt functional storage is the one sold by Apple so you’d have to try that yourself unless someone else already purchased it and figured out if VMware added the drivers for it.

      Reply

  5. LakritsTroll

    5 years ago

    Really appreciate the work in getting this up and running, thanks for a great step-by-step instruction!
    Now, if there was a way of making it start automatically after a power outage, I would be totally home free!!!

    Reply

  6. eloscurosecreto

    5 years ago

    Thank you for the prompt response.

    What storage options (other than the internal disks) are available (externally) with ESXi 5 on the Mac Mini, if any?

    I would like to be able to attach some sort of external device via thunderbolt (which I will test), FW or USB.

    My lack of understanding ESXi may come into play within the aforementioned questions, my I apologies in advance.

    Reply

    • Paraguin

      5 years ago

      iSCSI would be the best option since it’s cheap and easily available. The I/O on the network card may cause some VMs to load slower as it loads off the network storage, but it would be very viable option. You can choose to build your own iSCSI storage server or buy something that is pre-configured to be iSCSI compatible.

      Reply

  7. eloscurosecreto

    5 years ago

    FireWire (in my environment) would be best for me. Have you had a chance to play around with the vmdirectpath options yet?

    Reply

    • Paraguin

      5 years ago

      If you look at the screen shot above in this post, you will see that DirectPath is not supported with this version of ESXi.

      Reply

  8. peter

    5 years ago

    Thanks for this great blog item. I followed it as the main guide for my Mac Mini 2011 esxi 5 installation. And it’s doing fine, I have it on my desk as a test rig.
    The tip about installing rollout 2 is correct, but you still have to reinstall the BroadComm driver again – you need the latest patch to run OSX guest and it deinstalls the driver again.
    One thing though. After 1 week I’m trying to login using vSphere client and I get an error. Taking over the mini’s screen and restarting the management network fixed the problem.

    Reply

  9. Pres

    5 years ago

    Question,

    Can a datastore be mounted through thunderbolt? I am thinking if there was some way to plug in external thunderbolt drives, better yet ssd raided setup and get 10gig connection to 6gbps drives, the performance on a vm would be pretty good through a $1000 server.

    If you can’t use thunderbolt, is there a way for a second nic booting off esxi5, to use iscsi?

    Reply

  10. nick

    5 years ago

    has anyone tried adding perhaps a usb nic card for an additional network interface?

    Reply

  11. Andrew

    5 years ago

    How have you found the noise from the Mac Mini server machines?
    I am still trying to weigh up using a mac mini, or a Shuttle SZ68R5 with an i7-2600s – or wait for an Opteron 3250/ 3280 based solution.

    I am looking for a quiet solution.

    Reply

    • Paraguin

      5 years ago

      I have two of them running ontop of a laptop cooler and leave it in a storage area with decent air flow. Overall, they are quiet and unless I really ramp up some VMs, the overall noise is minimal. I have about 8+ VMs running at most times.

      Reply

  12. Myatix

    5 years ago

    Great blog…
    Have you tried installing ESXi on an SD card instead of the Mac Mini harddrive? I was thinking about running ESXi from an SD card in the Mac Mini and then using a NAS with iSCSI for my VM Datastore. I would be really interested in hearing if anyone has tried this? The idea is I would like the Mac Mini to boot from it’s own hard disk into Lion when the SD card isn’t mounted???

    Is this possible? How do I go about making a bootable ESXi SD card?

    Thanks for any tips…

    Reply

  13. blahblahguy

    5 years ago

    I was unable to install directly to the SD card, the esxi installation did not recognize it/show it as an option… But…

    I have a micro SD card and have SD and USB adapters for it, so I installed esxi to the USB (micro SD card in usb adapter), then after installation I moved the micro SD card to the SD adapter. I can select the SD slot from the mac efi boot selection screen, ESXi boots fine, but when I make changes to the network, or install VMs, the network changes and vm inventory is lost at reboot.

    I’m not sure if ESXi is mounting the SD card in read-only mode, or if ESXi has an absolute path to the location where it would store network/vm configuration information and is trying to access something on the now non-existant USB drive. In order to permanently change the IP, I had to move the micro SD card back to the USB adapter, boot up, make the change, shut down, then when I move the micro SD back to the SD adapter and boot up, the network change is in place.

    I’m not a vmware professional so I don’t know exactly how ESXi handles it’s configuration storage and haven’t had much time to try to get it working, but wanted to respond and let you know the experience with SD installation.

    I’m not sure where to go from here to get it fully configurable on the SD slot. If anyone has any command tips of where to go from here I can try them and report back, maybe we can get it to work. I don’t have much time to do research on it unfortunately as vmware isn’t my main gig.

    Reply

  14. david

    5 years ago

    love the idea, not liking the cost.

    i have found that an HP DC5800 makes a great option. low power, < 35W at idle, you can get quad core, but only up to 16GB RAM. add PCI-E lowprofile multi-port NICs.

    you can find them on ebay for about $300 with the quad core CPU. runs ESX 4.1 for certain, and probably 5.0

    Reply

  15. chau nguyen

    5 years ago

    I’ve successfully followed your guide to install esxi 5 on mac mini.
    The problem i have now is I can’t install solaris 10 on it.
    There always an error 36 when decompressing at the beginning of installation.
    I even make sure it have more than enough RAM.

    Reply

  16. Emmanuel

    5 years ago

    I just tested ESX 5.0 Update 1 on my Mac Mini Server, unfortunatly ESXi does not recognize my Thunderbolt attached Storage (Promize).
    Do you know if a driver is released for Thunderbolt Mac Mini ?

    Reply

    • vSean

      5 years ago

      Since the interesting half of the thunderbolt data stream is really a serial PCIe bridge, you’d need to find a driver vib for the storage controller promise is using. A bit of googling tells me its probably a PM8011, and the closest matching PCIe card would be the Promise Supertrack line. Those are HCL, but are all async drivers (not included). The one you want is probably “stex version 4.07.0000.75″ but with the MyVMware go live the driver downloads are still a jumbled mess of broken links. They are there, if you did around the site long enough it will turn up. I post in the hopes you can get it to work. Good luck.

      Reply

  17. vSean

    5 years ago

    Just an FYI, there is a newer tg3 driver available. Search for tg3-3.123b.v50.1-682322.zip release notes say 4/3/12 even though the vmware site has last years date on the download.

    Release notes include some potential PSOD related fixes under vmware.

    I was hoping it would enable the SD card reader bits of the broadcom chip, but no luck. Seems stable on the ethernet side though.

    Reply

  18. Mleo

    5 years ago

    So is it recommended to upgrade the tg3-3.123b.v50.1-682322.zip?

    Also is it recommended for the newer version of ESXi?

    Reply

  19. vSean

    5 years ago

    The thunderbolt nic works in the mini under ESXi. Plug it in, reboot, done.

    Reply

    • Paraguin

      5 years ago

      Thanks for the update!

      Reply

      • vSean

        5 years ago

        It also uses the TG3 driver:

        ~ # esxcfg-nics -l
        Name PCI Driver Link Speed Duplex MAC Address MTU Description
        vmnic0 0000:02:00.00 tg3 Up 1000Mbps Full 40:6c:8f:23:6a:17 1500 Broadcom Corporation NetXtreme BCM57765 Gigabit Ethernet
        vmnic32 0000:0c:00.00 tg3 Up 1000Mbps Full 40:6c:8f:37:50:b8 1500 Unknown Unknown

        ~ # ethtool -i vmnic32
        driver: tg3
        version: 3.123b.v50.1
        firmware-version: 57762-a1.10
        bus-info: 0000:0c:00.0

        ~ # vmkchdev -l|grep vmnic32
        000:012:00.0 14e4:1682 106b:00f6 vmkernel vmnic32
        ~ #

        Reply

    • Melvin

      5 years ago

      Hi Sean, where you find the Thurderbolt NIC and how much does it cost?

      Reply

  20. Joel Carnat

    5 years ago

    First of all, thank you for the post sets. Make me nearly regret the Whitebox I just build :) ))

    I would like to know if you can access “Hardware sensors” ; do you get CPU/MB temperature and FAN RPM ?

    Thanks.

    Reply

    • Paraguin

      5 years ago

      The hardware sensors do show healthy / non healthy, but they do not show the actual speed of the fans or CPU of the motherboard. I was also somewhat disappointed in that.

      Reply

  21. Melvin

    5 years ago

    Hi Mate,

    It great to find your article which help me advance my ESX planing where at lowest CAPEX & OPEX GREENER compare to those Server type with high Watt usage.

    1. I wonder do you think this supportability will have challenge in new model coming Ivy Bridge?

    2. Also have you tried loaded with addition USB-Ethernet?

    Reply

    • Paraguin

      5 years ago

      There’s no way of knowing what Apple will do with the next version. All we can do is sit and wait.

      As for the addition USB-ethernet, I have not tried it myself as one NIC card is enough to satisfy my needs. The two boxes I have currently have an uptime of over 5 months.

      Reply

      • Melvin

        5 years ago

        So is mainly playing with the VLAN :) ,

        I tot to have a separate Management IP or Port to simplify the need of Managed Switches here.

        Thanks for the Info, will try out the Thunderbolt & USB :)

        Reply
  22. [...] http://paraguin.com/2012/01/18/the-mac-mini-vmware-esxi-5-server-part-3-follow-up/ Rating 3.00 out of 5 [Comments (0)]  [link]  [TB] [...]

    Reply

  23. svenc

    5 years ago

    2 questions:

    1) Thunderboldt – EtherNet-NIC

    I am using ESXi 5.0.0 upd1 build 768111 on a Mac Mini 2011 (i5 CoreDuo, 16GB RAM, 128GB Samsung 830 SSD).

    The Thunderbolt – EtherNet-adaptor isn’t recognized (rebooted many times).

    any idea?

    2) 10.7.4 (Client) stops during boot for about 5 minutes (Apple logo is shown, nothing else). After that “break” it boots and is working without any problems. But a boot time of > 6min. seems to be a bit to long.

    The kernel.log displays:

    Aug 2 03:16:13 miniESXi kernel[0]: Waiting for DSMOS…
    Aug 2 03:16:13 miniESXi kernel[0]: macx_swapon SUCCESS
    Aug 2 03:16:13 miniESXi kernel[0]: memctl: Loaded @ 0xffffff7f80acee34: Info 0xffffff7f80ad2020 Name com.vmware.kext.vmmemctl Version 1.0.0 built May 29 2012 07:04:05
    Aug 2 03:17:49 miniESXi kernel[0]: SMC::smcGetKeyFromIndexAction ERROR: failed on index 0
    Aug 2 03:17:49 miniESXi kernel[0]: SMC::smcGetKeyFromIndexAction ERROR 0 kSMCTimeoutError(0xb7) fKeyHashTable=0x0xffffff800a8ab000
    Aug 2 03:17:49 miniESXi kernel[0]: SMC::smcInitHelper ERROR smcGetKeyFromIndex(0) kSMCTimeoutError(0xb7)
    Aug 2 03:21:10 miniESXi kernel[0]: SMC::smcGetKeyInfoAction ERROR DPLM kSMCTimeoutError(0xb7) fKeyHashTable=0×0
    Aug 2 03:22:51 miniESXi kernel[0]: SMC::smcWriteKeyAction ERROR CLKH kSMCKeyNotFound(0×84) fKeyHashTable=0×0
    Aug 2 03:22:51 miniESXi kernel[0]: DSMOS has arrived

    the tip with replacing the AppleLSIFusion.kext (which is version 3.0.5 on 10.7.4) to an older version (3.0.0) from 10.7.0/10.7.1 does not work. With that patch the machine doesn’t boot at all.
    Does the error come from this DSMOS-message or is it the SMC-error?
    DSMOS must be the “Don’t Steal Mac OS X.kext”, correct?

    I tried the unlock-all-patch (1.10), no change.
    I installed the latest patches to the Hypervisor, no change.

    With 10.7.2 the machine is booting in about 30-40 seconds.

    Many thanks in advance.

    svenc

    Reply

  24. svenc

    5 years ago

    First problem solved:
    This link describes that another driver is needed.

    http://www.virtuallyghetto.com/2012/06/thunderbolt-ethernet-adapter-in-apple.html

    But why isn’t that driver included in the patches?

    ok, second problem still occurs…

    Many thanks in advance

    svenc

    Reply

  25. Pascal

    5 years ago

    Hello,

    It seems it is anylonger possible to install or run ESXi 5.0 on a rMBP (MacBook Pro 15′ Retina).
    No issue to boot on CD or USB key to install or run ESXi 5.0 but I receive the following error message on screen:

    NOT_REACHED bora/vmkernel/hardware/intel/vtd.c:4463
    cr0=0x8001003b cr2=0×0 cr3=0x8ae2f000 cr4=0×2078
    *PCUPU0

    Any solution yet?

    Reply

    • Pascal

      5 years ago

      ESXi5 ISO used: 5.0.0 build 623860

      Reply

  26. Titobal

    5 years ago

    Very interesting article. I’m going to test this this week.

    I will try this with iscsi via freenas for the storage, it would be REALLY great to be able
    to support the thunderbolt drives like the Pegasus from Promise too…

    I’ll continue to watch the articles.

    Thanks for your very cool work.

    Reply

  27. Melvin

    5 years ago

    Hi Mate,

    Need help here i lack of CD Drive, may i know anyone has step to make the ISO to load into USB and successful boot on Mac Mini.

    Reply

  28. Adam

    5 years ago

    Sean, have the thunderbolt nic ( from Apple ) but my esxi5 doesn’t recognise it
    What build of esxi5 are you using? are you testing on an i7 or i5 mac mini ?

    Reply

    • vSean

      4 years ago

      Using the 5.1 release version currently, but the thunderbolt nic does require the newer tg3 driver. 3.123b.v50.1

      The version from the original guide does not support the thunderbolt nic.

      Reply

  29. oturn

    5 years ago

    Hopefully some of you folks running ESXi on the Mac Mini are still watching this thread…

    I’ve had ESXi 5.0, and now 5.1 running on my Mac Mini for about 6 months now. The only problem I’ve seen is very serious….the occasional purple screen of death, or PSOD. The system will run flawlessly for weeks or months, but if it’s pushed hard and the CPU’s ramp up, it will get a PSOD.

    Here’s are parts of the PSOD error:
    PCPU 0 locked up. Failed to ack TLB invalidate (total of 1 locked up, PCPU(s): 0).
    *PCPU5:2805/hostd-worker

    VMware’s KB articles don’t help me out too much. Any suggestions? Could this be a heat issue?

    Reply

    • Paraguin

      4 years ago

      Hm, the most we’ve seen is we burned out a NIC port on a switch and thought the system crashed. We do have a laptop cooler underneath the Mac Mini servers and sometimes a small fan blowing at them to circulate air better.

      Reply

  30. Nicolas

    4 years ago

    Hello,

    Has anyone tested the same setup with the new Mac Mini announced this week ?
    I’m wondering if there’s any change… and if it would work with 32GB ram…

    Thanks

    Nicolas

    Reply

    • Robbo

      4 years ago

      Have just started to test this on the Mac-Mini server released a few days ago (I think apple are labeling it Version 6.2). PSOD with the hardware during installation.

      Had tried with both 5.0, and with 5.1 (Build 799733)

      I haven’t done a Ram upgrade, just using the standard 4GB that comes with the hardware.

      Found this site, which described the PSOD perfectly.

      http://communities.vmware.com/message/2136780

      Which recommended that we put a setting in on initial setup

      ” iovDisableIR=TRUE”

      This has allowed us to install the Hypervisor. Have yet to install the Broadcomm driver, but so far the setup looks just as straight forward, and looks to be working.

      I haven’t tested with 5.0.

      Reply

  31. Tony Spencer

    4 years ago

    I’ve tried installing via SuperDrive on the new 2012 Mac Mini server using the standard 4 GB (two 2GB) of 1600MHz DDR3 memory and I can’t even get started. I received a purple screen upon install that complained “No usable memory below 4GB”

    http://www.evernote.com/shard/s27/sh/da19132e-5b52-4614-9675-8b456fdbb8d6/4e4849dd28aa38556031a937d0d525da

    Has anyone else hit this wall? I’m going to try third party RAM from Corsair later.

    Reply

  32. Melvin

    4 years ago

    Hi Mate,

    Just wonder as of now my Mac mini with ESXi 5.1 running with custom driver update as per above, if there is new patch from VMWare will it break my Driver?

    Reply

  33. laputa

    4 years ago

    not working for new MacMini6,1, any new iso please?

    Reply

Leave a Reply → Robbo

Latest Posts

Your Ad Here