In part 1 we introduced the concept of SAP HANA Application Workload guidance and using example business requirements to come up with a workload and vSphere cluster design for the SAP environment. In part 2 we looked at storage, network and security design for the proposed customer environment. In part 3 we looked at monitoring & management, backup/recovery and disaster recovery for SAP S4/HANA. In this final part we look at validating the design we built over the past three parts and conclude the four part blog series.
Validation of an SAP design is often difficult because of the absence of publicly available validation and performance tools. This design utilizes best practices derived from vendor testing conducted in SAP labs. The SAP HANA database tier is critical to the infrastructure and must be validated. So as part of this SAP S/4HANA VVD solution, some SAP standard validation tools were used to exercise the designed infrastructure.
As part of the application workload guidance process, the design was created to depict an example customer environment based on common business requirements. Even though the design is based on best practices developed by VMware, it is a good practice to conduct performance validation for the solution.
Validation was performed for the SAP HANA database tier for the SAP S/4HANA modules that were designed earlier. The database tier was deployed in its own dedicated cluster on large memory hosts with 44 cores and 1TB RAM each. The VMs were deployed on SLES 12 for SAP with memory and CPU sizes from the infrastructure design. All VMware best practices were adhered to in the deployment of the various SAP S/4HANA deployments.
Validation Tests:
Contents
Standardized OLTP and OLAP tests for SAP S/4 HANA were used for the validation. The SAP HANA systems in the design had 8, 16, or 32 vCPUs each. The validation tests were run across these three sizes of VMs and were compared for performance and scalability. The storage used for the SAP HANA deployments was initially SAP HANA TDI storage hosted on the Pure Storage FlashArray//M50. The VMs then were migrated via vSphere vMotion to the all-flash VMware vSAN configuration. The tests were then repeated.
OLTP Tests:
The OLTP tests included inserts, updates, deletes, joins, and join–select operations. These tests were run for the various VM sizes.
Results for Tests with Storage on Pure Storage FlashArray//M50
| 8 vCPU | 16 vCPU | 32 vCPU | |
| Insert | 13.7115 | 8.318 | 4.912 |
| Delete | 13.131 | 7.802 | 4.4335 |
| Update | 3.2555 | 1.845 | 1.091 |
| Select | 222.2555 | 110.454 | 63.018 |
| Join–Select | 6.1375 | 3.152 | 1.983 |
Table 11. OLTP Time Comparison for Various Operations on TDI Storage
The time in seconds shows the duration required to run these tests. Lower times are better because they imply that the test ran more quickly.
Figure 16. OLTP operations Time Comparison for Different HANA VM Sizes on TDI Storage
Results from OLAP Type Tests on Pure Storage FlashArray//M50
Two different types of OLAP tests were performed on the SAP HANA database servers: VDM and TPC-DS. The results of these tests are shown in Table 24.
VDM
The performance for this test is measured in queries per second (QPS). There are two different types of queries, labeled QPS1 and QPS2. The QPS for the two types of queries are compared in Table 24 for the three sizes of SAP HANA database VMs.
| vCPU | QPS1 | QPS2 |
| 8 | 3.925833333 | 45.16535 |
| 16 | 7.694538462 | 80.1825 |
| 32 | 11.4115 | 135.5245 |
Table 12. QPS Time Comparison Across Different HANA VM Sizes on TDI Storage
Figure 17. Query Time Comparison for Different Query Types on TDI Storage
TPC-DS is a standard benchmark for measuring the performance of decision support solutions. The benchmark models several generally applicable aspects of a decision support system, including queries and data maintenance. It includes a set of 99 queries that test various aspects of OLAP-based systems. The entire suite of queries was run against the SAP HANA databases, and the top 25 queries were compared for the three sizes of database servers. The results are shown in Figure . The 8-vCPU database system is used as the baseline and represents 100 percent from a query time perspective. The 16- and 32-vCPU systems’ query times are compared with the baseline as shown in Figure 32.
Figure 18. TPC-DS Query Time Comparison for Different HANA VM Sizes on TDI Storage
Results for Tests with Storage on All-Flash VMware vSAN
OLTP tests were run multiple times, and the average runtimes were calculated and tabulated as shown in Table 25.
| 8 vCPU | 16 vCPU | 32 vCPU | |
| Insert | 16.094 | 7.607 | 5.145 |
| Delete | 13.6 | 7.304 | 4.44 |
| Update | 3.255 | 1.844 | 1.035 |
| Select | 239.958 | 110.496 | 67.87 |
Table 13. OLTP Time Comparison for Different Operations on VSAN
Figure 19. OLTP operations Time Comparison on VSAN
Results from OLAP-Type Tests on All-Flash VMware vSAN
Two different types of OLAP tests were performed on the SAP HANA database servers: VDM and TPC-DS. The results of these tests are shown in Table 26.
VDM
The performance for this test is measured in queries per second (QPS). There are two different types of queries, labeled QPS1 and QPS2. The QPS for the two types of queries are compared in Table 26 for the three sizes of SAP HANA database VMs.
| vCPU | QPS1 | QPS2 |
| 8 | 4.08 | 45.33 |
| 16 | 7.08 | 79.45 |
| 32 | 11.75 | 136.35 |
Table 14. QPS Time Comparison Across Different HANA VM Sizes on VSAN
Analysis
The results of the tests clearly show that the vSphere platform provides linear scalability and consistency in performance across different types of SAP S/4HANA database workloads. Linear CPU, memory, and I/O performance and scalability are reflected in these results.
TPC-DS on All-Flash VMware vSAN
TPC-DS is a standard benchmark for measuring the performance of decision support solutions. The benchmark models several generally applicable aspects of a decision support system, including queries and data maintenance. It includes a set of 99 queries that test various aspects of OLAP-based systems. The entire suite of queries was run against the SAP HANA databases, and the top 25 queries were compared for the three sizes of database servers. The results are shown in . The 8-vCPU database system was used as the baseline and represents 100 percent from a query time perspective. The 16- and 32-vCPU systems query times are compared with the baseline as shown in Figure 40.
Analysis
The results of the tests clearly show that the vSphere platform provides linear scalability and consistency in performance across different types of SAP S/4HANA database workloads. Linear CPU, memory, and I/O performance and scalability are reflected in these results.
Conclusion:
Virtualizing SAP HANA scale-out systems enables an organization to benefit from all supported VMware virtualization solutions and options, such as live migration via vSphere vMotion, to increase SLAs and reduce TCO. The recent joint SAP and partner testing and the subsequent release of the VMware virtualized BW-EML benchmark of an SAP HANA scale-out system have shown reliable operation with a very small impact on overall system performance. SAP HANA scale-out support in controlled availability provides additional benefits for the customer by offering additional consultation from SAP, VMware, and the hardware partner. This ensures that virtualized SAP HANA configurations work well and run optimally in a customer’s virtualized environment.
A comprehensive implementation of the Software-Defined Data Center (SDDC) using the prescriptive advice found in this Application Workload Guidance Design document will lead to the successful “end-to-end” and linearly scalable virtual SAP HANA system. The components used within this set of tests constitute a reference architecture. Each component, although ideal for this type of implementation, is interchangeable with a variety of products widely available through the industry. The overhead imposed by the SDDC is extremely minimal. Even when using VMware vSAN, the tests in this report reveal that any overhead introduced by VMware vSAN processing is negligible.
This Application Workload Guidance Design document based on a VMware Validated Design, in addition to providing an end-to-end infrastructure for running business-critical applications, includes a VVD infrastructure that is designed, deployed, and validated for the entire SAP S/4HANA landscape. The SAP Quick-Sizer approach was used to convert business requirements to allocated compute resources. The sizing and designing of the SAP S4/HANA infrastructure followed SAP and VMware best practices. All VVD components were leveraged to build out the end-to-end solution for SAP S/4HANA. The following are some of the major components of the VVD that were used in this infrastructure:
- vSphere for virtualized compute and infrastructure
- High-performance Dell R630 and R730 PowerEdge Servers
- Pure Storage–based SAP TDI and Western Digital all-flash VMware vSAN storage
- Brocade Generation 6 SAN fabric
- Separate vSphere clusters for management, SAP applications, and SAP HANA databases
- vSphere HA and vSphere FT for simplified high availability
- VMware NSX for networking with microsegmentation for security on Brocade VDX switches
- vRealize Automation with Adapter for SAP Landscape Management
- vRealize Operations with Blue Medora SAP dashboards for operations and capacity planning
The solution was then successfully validated to meet and exceed the requirements. The results of the validation tests clearly show that vSphere offers a robust platform for running SAP S/4HANA production workloads.
This set of tests was conducted in a truly comprehensive manner, both broad in scope and striking in depth. In addition to what has been highlighted in this voluminous work, there are components of VMware technology that were used as “common practice” during the testing procedure that are well accepted in “everyday” virtualized environments and therefore don’t necessarily warrant detailed or complex analysis and explanation in the text. For example, vSphere vMotion was used on the virtual SAP HANA VMs to transition them from servers using the Pure Storage array to those configured to use VMware vSAN. This is a highly impressive capability, but it is one that in 2017 is so commonly employed in virtualized environments that it is unnecessary to highlight.
The time has come to definitively recognize that the well-known VMware motto “No Application Left Behind,” which was adopted in the vSphere 6.x time frame and signifies that every implemented application and database is a candidate for virtualized infrastructure, is an observable fact. The benefits and value of the VMware SDDC will enhance any SAP HANA implementation.
More details from this blog series can be found in this comprehensive Whitepaper.
