Aufgrund einer Anfrage meines Kunden und seinen umfangreichen Fragen zu VSAN 6.1, zum Design, zur Architektur und Konfiguration mit Microsoft Clustering, habe ich folgende Informationen zusammengestellt.
Ich hoffe, dass diese Zusammenstellung auch anderen Kunden hilft, ihre VSAN Umgebung zu planen und zu „designen“.
VSAN 6.0/6.1 Whats New in VSAN 6.1?
Hardware Compatibility Guide for VSAN
VSAN 6.1 Feature Walkthroughs
VSAN Design & Architecture
This article will answer all of your questions and provides recommendations to server, storage and network configurations and best-practices,
However, I like to point out some additional topics you have to consider when setting up your VSAN cluster.
- Besides the configuration regarding vCPUs per VM and RAM used and assigned per VM you have to consider how much components do you have per VM and thus per host. A component is the leaf of a RAID tree, which is a storage object. VMDKs are storage objects, as well as snapshots, the VM home namespace and swap. A VM can have multiple storage objects associated with it, and thus it can have many components. The number of components per VM depends on the configuration of a VM, which RAID configuration is used, the number of replicas, the number of snapshots, and the number of NumberOfFailuresToTolerate settings. The witness is also a component. The maximum number of components per host is 9.000 in VSAN 6.1. Suppose, you have 100 VMs per host, the number of components per VM is limited to 90.
- There are some other limitations you have to consider, which you can review in this document, page 11ff: http://www.vmware.com/files/pdf/products/vsan/VSAN_Design_and_Sizing_Guide.pdf
- It is very important that the hosts in a cluster have thee same configurations.
- do not enable power management https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1018206
- At least you need 32 GB memory for a full configured VSAN system, this is according to Cormac Hogans post 5 disk groups with 7 disks in each group. The memory is not only used for VSAN but to perform workloads. http://cormachogan.com/2015/11/10/vsan-design-sizing-memory-overhead-considerations/
- Understand memory consumption by VSAN 6.0/6.1 ( Note: this will change in future releases ): https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2113954
- Technical Whitepaper about the caching algorithm of VSAN: http://www.vmware.com/files/pdf/products/vsan/vmware-virtual-san-caching-whitepaper.pdf
It is strongly recommended to create and use your own storage policies, although the policy might use the same settings as the default ( none ) storage policy. As in Cormacs article explained, even if you use the default ( none ) storage policy you can choose either thick or thin provisioning, but the default policy uses LZT ( Lazy-Zeroed Thick ). But if you choose thin, it will use thin.
Virtual SAN ( VSAN ) Stretched Cluster
- VSAN stretched cluster requires VSAN 6.1 ( vSphere 6.0 Update 1 ).
- Topology of Stretched Cluster and why you have to consider L3 network with VSAN 6.1 for witness. VMware recommends L2 network between data sites and L3 network for witness.
- Best Practices and Performance
- Alternatively to stretched cluster configurations there is also the option to use ROBO configuration: http://www.mrvsan.com/virtual-san-robo/ This feature is provided with VSAN 6.1 ( vSphere 6.0U1 ).
VSAN and Microsoft Clustering Services ( MSCS )
Please review this KB Article carefully: https://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1037959&sliceId=2&docTypeID=DT_KB_1_1&dialogID=10460707&stateId=1%200%2010476229
- MSCS (Windows Server Failover Clustering (WSFC)) is supported with VMware Virtual SAN (VSAN) version 6.1 and later.
- With VSAN 6.1 it is not necessary to use RDMs only, when using iSCSI because then SCSI bus-sharing is not needed.
- This article by Simon Todd describes why SCSI bus-sharing isn’t necessary with VSAN ( and thus, why RDMs-only with MSCS in a CAB configuration aren’t an option anymore ). With VSAN it is possible to use VMDKs as iSCSI targets and the in-guest iSCSI Software as iSCSI initiator. http://www.mrvsan.com/microsoft-windows-failover-clustering-on-virtual-san/ „The Cluster nodes themselves would be standard Windows 2003 / 2008 / 2012 virtual machines with the Software iSCSI Initiator enabled and used to access the LUNs being presented by the iSCSI Target, again since the in guest iSCSI will manage the LUNs directly there is no need to share VMDKs or RDMs using SCSI BUS Sharing on the virtual machine SCSI adapter, the Cluster nodes themselves can also reside on Virtual SAN, so you would typically end up with each of the cluster virtual machines accessing the shared storage via their in guest iSCSI initator.“
- If it is not an option to use iSCSi you have to use RDMs, because VMFS is supported only for CIB ( Cluster in a Box ). See KB article above.
- Sharing RDMs between two virtual machines without a clustering solution is not supported. For more information, see Microsoft Clustering on VMware vSphere: Guidelines for Supported Configurations (1037959).
- Further information how to setup a Microsoft Cluster with ESXi 6.0/ESXi 6.0U1: http://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-60-setup-mscs.pdf
- KB article regarding MSCS support on ESXi in general: https://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1004617
Backup and DR with VSAN
- VMware recommends to VMware vSphere Data Protection Advanced with VSAN https://www.vmware.com/files/pdf/products/vsan/VMware-vSphere-Data-Protection-Advanced.pdf
- Virtual SAN interoperates with VMware vSphere Data Protection™ Advanced, a backup and recovery solution designed for vSphere environments and powered by EMC® Avamar®. vSphere Data Protection Advanced provides agentless backup and recovery of virtual machines running on VMware vSphere VMFS, NFS, and Virtual SAN datastores. Backups are deduplicated using a variable-length segment algorithm, resulting in a significant reduction in backup data storage capacity consumption. Backup data can also be moved offsite using reliable, secure, network-efficient replication.
- vSphere Data Protection Advanced is deployed as a Linux® based virtual appliance. A vSphere Data Protection Advanced virtual appliance consists of multiple virtual disks that contain the guest operating system (OS), vSphere Data Protection Advanced application, and backup data. vSphere Data Protection Advanced routinely performs checks to verify the integrity of the virtual appliance and the backup data it contains.