Installation using the cdrom was succesful but after restarting the server won’t boot from the USB stick.
Make sure you set the correct BIOS options to allow to boot from USB.
If still doesn’t boot than it probably has to do with GPT/MBR formatting of the USB stick done by VMWARE.
You need to boot from the ESXi install CD again and right after you press enter to choose “ESXi5.1 installer ISO …” you see in the lower righthand corner the text “Shift + o” press this key combination (shift and the letter o). Now you see the text “runweasel” remove any chars after this, type a space and then “formatwithmbr”.
Now install as normal but now VMware should format your USB stick as MBR instead of GPT and you should be able to boot from it after the install finishes.