In this article we show an experimental deployment of a test S/4HANA system running SAP HANA 2.0 SP3 on vSphere 6.7 with persistent memory (PMEM) – this is the first version of HANA that supports PMEM and is PMEM aware. Note: SAP HANA is not yet supported by SAP on vSphere on persistent memory. We introduce a demo of live migration/vMotion of an SAP HANA workload between two ESXi hosts configured with PMEM devices. vSphere 6.7 presents PMEM as an NVDIMM device to the virtual machine – this is referred to as vPMEM. Please read the beginning of this blog for more background on PMEM and the different methods to deploy applications with PMEM on vSphere.
SAP support for PMEM is based on Intel® Optane™ DC persistent memory in app-direct mode. SAP HANA PMEM aware code places a large part of HANA memory, the “column store” , in PMEM – this is byte-addressable and more efficient than block based access. This provides the following benefits: a more cost-effective solution whereby SAP HANA memory is stored in cheaper PMEM devices instead of DRAM; with DRAM SAP HANA pre-loads column store data into memory after startup – with PMEM this initial load of data is not necessary.
vSphere supports vMotion of vPMEM. vPMEM vMotion is conducted as compute vMotion, where vPMEM is transferred as memory along with vRAM.
Overview of Installation
The following diagram depicts the vMotion demo scenario.
SAP HANA has different memory regions:
- Column store corresponds to all the column based tables and takes up a large part of SAP HANA memory. This is stored in PMEM and is byte-addressable.
- Remaining memory regions include space for working data and the row store corresponding to row-based tables. This data is stored in DRAM.
Each ESXi host was configured with PMEM devices. vSphere presents PMEM as a NVDIMM device to the Guest-OS in the VM. The following vSphere client screenshot shows the NVDIMM device shown as virtual hardware in the VM “edit settings” function.
The Guest OS Linux version requirements for PMEM are defined in SAP KB article 2618154 – SAP HANA Persistent Memory – Release Information. In the Guest-OS the Linux “ndctl” utility was used to manage the NVDIMM and create a device that supports DAX. A filesystem was created from the NVDIMM device and mounted using the DAX option. A few database parameters were set to activate PMEM in SAP HANA – they are summarized in SAP KB article 2700084 – FAQ: SAP HANA Persistent Memory.
Demo
The demo of vMotion of the S/4HANA system deployed on vPMEM is available here. The demo steps include:
- Overview of the configuration: the SAP HANA database VM; the S/4HANA application server VM.
- Start an SAP batch workload – this was an SAP Client Copy.
- Start vMotion.
- Verify workload is running during vMotion and after successful completion of vMotion.
- Check PMEM usage of source and target ESXi hosts after vMotion in the vSphere client. PMEM usage falls on the source host and increases on the target host.
Summary
- SAP HANA 2.0 SP3 is the first version that supports persistent memory (PMEM) and is PMEM aware.
- vSphere 6.7 presents PMEM as an NVDIMM device to the virtual machine – this is referred to as vPMEM.
- We described a proof-of-concept of an S/4HANA system running SAP HANA 2.0 SP3 deployed on vPMEM. A demo is available showing vMotion of the SAP HANA virtual machine between two ESXi hosts configured with PMEM. vMotion completed successfully with no impact to the SAP workload.
- vPMEM vMotion support provides the benefits of managing compute resources in an ESXi cluster: minimize planned downtime of SAP HANA on vPMEM due to ESXi host maintenance; optimally distribute compute resources required by SAP HANA on vPMEM with the VMware Distributed Resource Scheduler.
