Several customers have asked what vFabric SQLFire can do for their applications and how they can modify the architecture of a custom Java application accordingly. These customers typically run custom Java applications against RDBMS that have reached the limits of scalability and response time with their current architecture. They want to make the change only if they are not too invasive.
To answer these questions fairly, we simulated a customer scenario with no specific assumptions about specialized tuning. First, we took Spring Travel with RDBMs as-is and ran a load test against it. Then we converted the Spring Travel schema to run against vFabric SQLFire, also without tuning, and plotted the results side by side.
We also wanted to demonstrate how quickly we could make this change without assumptions about any code intrusion/change, so we simulated how a developer might download the Spring Travel application, run the DDlUtils conversion utility to generate the SQLFire schema and the data load file, then quickly test to see the performance improvements. We felt this would answer the customers’ questions without bias.
This post summarizes the results. The details of the conversion of the RDBMS schema and data can be found in the vFabric SQLFire POC Jumpstart Service Kit, or vFabric SQLFire Accelerator Service Kit. (Contact your local VMware account team for details on these service offerings.). The conversion process took one day, and the total process—downloading the Spring Travel application, installing vFabric SQLFire, running the schema and data conversion utility, and running the load test—took 3 days. We iterated the results for another week for verification.
NOTE: You can download Spring Travel from: http://www.springsource.org/download. Navigate to the download link under Spring Web Flow 2.3.0. For vFabric SQLFire, see: http://www.vmware.com/products/application-platform/vfabric-sqlfire/overview.html
Figure 1. Spring Travel on Traditional Disk-Based RDBMS versus vFabric SQLFire
Performance Results
Contents
The results are plotted in Table 1. The legend for the columns is as follows:
- Threads: The number of concurrent Spring Travel application threads executed during the two load tests.
- SQLF R/T (ms): The response time of Spring Travel application in milliseconds.
- SQLF CPU %: The percentage of CPU utilization at peak for the SQLFire VMs.
- RDBMS R/T (ms): The Spring Travel application response time in milliseconds when running against the traditional disk-based RDBMS.
- RDBMS CPU %: The percentage of CPU utilization at peak on the RDBMS VM.
The results tested for a range of 18 to 7200 concurrent threads. ”Failed” indicates that Spring Travel running against the traditional disk-based RDBMS failed to respond. Since it was essentially frozen, we collected no data from it.
Table 1. Spring Travel Results with RDBMS versus SQLFire
Spring Travel Response Time versus Concurrent Threads Test Results
Figure 2 shows response time along the vertical axis and concurrent number of threads along the horizontal axis. The red line represents Spring Travel running against a traditional disk-based RDBMS, and the blue line represents Spring Travel running against SQLFire. The data shows that as the number of concurrent threads increases along the horizontal axis, the Spring Travel response time increases in a linear fashion when running against the disk-based RDBMS but remains constant, as indicated by a fairly low and flat blue line, when running against SQLFire.
Figure 2 – Spring Travel Response Time versus Concurrent Threads
Scalability Test Results
This test demonstrates the extent of scalability of both configurations. When Spring Travel ran against the RDBMS, after reaching 1850 concurrent threads and getting close to a response time of 172 milliseconds, the system stopped responding, indicating that it had reached the scalability limit. This is indicated by the red line on Figure 3. On the other hand, Spring Travel running against SQLFire continued to function to the limit of 7200 concurrent threads and a response time of 984 milliseconds. This is indicated by the blue line.
NOTE: At approximately 3600 concurrent threads, SQLFire started to overflow to disk, and the response time increased. In a normal situation, you can use appropriate sizing of available RAM to contain this kind of overflow.
Figure 3. Spring Travel Response Time versus Concurrent Threads – Scalability Test
CPU versus Concurrent Threads Test Results
Figure 4 shows the CPU %, measured for the duration of the test, of the RDBMS VM in red and the SQLFire VMs in blue. Using the RDBMS, Spring Travel peaked at approximately 80% CPU and 1850 concurrent threads. At this point, it completely failed to respond. The SQLFire-based Spring Travel configuration, on the other hand, continued to 98% CPU utilization at 7200 concurrent threads and was still responsive – this is at approximately 984 milliseconds f Spring Travel application response time. The red and blue lines crossed over at approximately 1000 concurrent threads, indicating that Spring Travel with SQLFire handled much higher loads at a steadier CPU utilization increase.
Figure 4 – Spring Travel Application CPU versus Concurrent Threads
Summary of Findings
This simulation shows that:
- Using the DDlUtils utility to convert the schema and data of the RDBMS associated with Spring Travel application was relatively straightforward.
- The installation of vFabric SQLFire was also straightforward.
- Spring Travel pointing to vFabric SQLFire scaled approximately 4x when compared to Spring Travel pointing to an RDBMS.
- The response times of SQLFire were 5x to 30x faster with vFabric SQLFire. Further, the response times on SQLFire were more stable and constant with increased load.
- The configuration of Spring Travel with an RDBMS has a response time that increases linearly with increased load.
- The break point for Spring Travel against an RDBM was at 80% CPU utilization for about 1850 concurrent threads, after which Spring Travel stopped responding. The SQLFire version of Spring Travel continued to pace ahead at 98% CPU utilization and achieved 7200 concurrent threads.
- NOTE: The assumption here in the test across the two configurations was that the total compute resource was the same, meaning a true apples-to-apples comparison. The RDBMS VM was eight vCPU and 4GB of RAM, while the SQLFire VMs were of 2GB and two vCPU each. In configuration cases VM memory reservation was set.
Thank you for reading! Looking forward to seeing you at VMworld 2012, I will blog about my VMworld sessions shortly.
Emad Benjamin
