Compare Apache Tomcat performance on x86_64 and arm64 CPU architectures
The majority of the software developers usually do not think about the CPU architecture their software will run on. I do not have official statistics but in my experience most of the software for desktop and backend applications runs on x86_64 architecture (Intel and AMD processors) and most of the mobile and IoT devices run on ARM architecture. The developers write their software for the respective CPU architecture using some high level programming language and do not think what kind of Assembly instructions are being executed at runtime. And this is the purpose of the high level programming languages — to let the compiler deal with the low level hardware instructions and simplify our task to focus only on the high level business related problems.
Life is simple and beautiful but there are times when a big player in the laptop and desktop hardware and software manufacturing comes and says that our software will have to run on a different architecture — first from PowerPC to Intel and now from Intel to ARM64 (sources: Bloomberg & AppleInsider). Due to the lower consumption of electricity even several of the bigger cloud providers started providing ARM64 virtual machines (e.g. Amazon AWS, HuaweiCloud, Linaro). And here comes the uncertainty:
- Will my software run on the new CPU architecture ?!
- What kind of changes I will have to do to make it work ?!
- Will it perform as good as before ?!
To be able to answer these questions you will have to roll up your sleeves and test!
You can deploy your software on any of the cloud providers. Some of them give free trial period! Or if you are on a low budget you can experiment on RaspberryPi.
Depending on what programming language you use to write your software you might need to do some changes or not at all! If you use an interpreted language (e.g. Python, Perl, Ruby, JVM, …) then the chances the interpreter already supports ARM64 are pretty high and you are good to go without any changes! But if your software needs to be compiled then you will need to adapt your toolchain and make sure that there are ARM64 binaries for all your dependencies! Depending on your software development stack your mileage may vary!
Once our software runs fine on the new architecture we will be able to check whether it performs as good as before. Recently some users have asked in Apache Tomcat mailing lists whether ARM64 architecture is supported. Since Apache Tomcat is written mostly in Java it “Just Works”. If you need to use libtcnative and/or mod_jk then you will need to build them yourself on ARM64. Apache Tomcat team uses TravisCI to test both Java and C code on ARM64 and there are no known issues at the moment!
To compare the performance of two versions of some software usually you will run the tests on the same hardware but in this case since we use different CPU architectures this is impossible. For my tests I have used two VMs with similar specifications:
- The x86_64 processor is:
- The ARM64 processor is:
Both VMs have same amount of RAM, disk and network connectivity.
The test application is based on Spring Boot (2.2.7), runs an embedded Apache Tomcat 9.0.x + OpenSSL 1.1.1h-dev and Apache Apr 1.7.x nightly builds and has a single REST controller that exposes a PUT endpoint for creating an entity, a GET endpoint to read it, a POST endpoint to update it and a DELETE endpoint to remove it. It uses Memcached as a database.
For load testing I have used Apache JMeter 5.2.1 and wrk from its master branch. JMeter is used for a real case scenario with 1000 simultaneous users, ramp-up period and think time between the HTTP requests. And wrk is used to test the maximal throughput.
JMeter is executed with these arguments:
The httpclient4.** properties are needed to reuse the HTTPS connections, otherwise Keep-Alive was not effective. Update: Apache JMeter 5.3.0 has been released with these properties updated so there is no need to set them explicitly anymore!
The results from both JMeter and wrk are parsed with Logstash, stored in Elasticsearch and visualized by Kibana.
JMeter’s response times:
As you can see the results for HTTPS were not very good before May 8th. The HTTPS connections were not reused and TLS handshake has been done for each request, despite request header “Connection: keep-alive”. Since there was no such issue with wrk I’ve asked at JMeter mailing lists and they gave me the httpclient4 arguments above. (Thank you, Philippe Mouawad!). With or without the HttpClient tweak we see that the response times are very similar for x86_64 and arm64. That’s awesome!
For the throughput test with wrk I have run it with these parameters:
i.e. 8 threads will hit the server for 30 seconds using 96 HTTP(S) connections.
To collect the summary in a CSV file I used this custom Lua script:
the results show that Tomcat on x86_64 is twice faster than on arm64:
I will try to find out what is the reason for this difference and share it with you in a follow up post. If you have any ideas I would be happy to test them!
Happy hacking and stay safe!