Solving Java Cold Starts – Quarkus and GraalVM

When it comes to software engineering it’s generally best to use the correct tool for the job, but of course we all have personal preferences. Any collection of Developers (company, team, individual et al.) should be striving to use the best available technologies, however, they often have experience and predilections that may contradict or subvert that mindset. You may be aware that X is better than Y at solving the problem you are facing, but if you can get a solution together using Y in a fraction of the time because you know it inside out, technology Y will definitely be tempting.

With that in mind I’d like to talk about Java being used in a ‘function as a service’ cloud solution; in this example AWS Lambda. I’ve put some effort into researching some of the major available languages available for Lambda, the drawbacks of Java (primarily cold starts), and how to overcome them.

What Are Cold Starts?

I was recently part of a team who implemented a serverless solution using AWS Lambda, and with little discussion it was decided that we would use Python. However, after the project had started one of my peers, who I’m going to call ‘Ella’, started asking, “Why did we have to use Python and not Java?”. My response at the time was an uninformed, “Doesn’t Java have some issues when running on Lambda”, to which I was swiftly told that “Java is a perfectly functional AWS Lambda language”. It was the next week before Ella remembered that the Java Lambda she had previously built, timed out every Monday morning and needed to be pinged ahead of time to work properly. This is exactly the Java cold start problem.

What’s going on here is Amazon managing its resources with regards to the Lambda. Your Lambdas are not magic, they in fact run on Firecracker containers. AWS has a number of these ready to go and if you get a lot of requests then it will increase this number. However, if you are getting no requests for an extended period, then they will scale down to zero. Then when your Lambda gets called it has to create a new container for you and start up Java before it can respond. For interpreted languages like Python, this is a small task, but for compiled languages like Java, it takes a bit of time.

A Bit of Code to Prove The Point

I’ve put together some code to illustrate this problem. The high level concept is easy; write a simple hello world Lambda, let a weekend pass, then run it and figure out how long it takes. Easy, right? Well yes, that would be fine if I was a sane man, but I wanted to go further than that. Firstly, I wanted to compare the speed of Java to a few of the more common Lambda languages (Golang, Node and Python). This was tricky because I had never even looked at Golang code before, but I did it anyway. Then I thought it would be nice to have the exact number of milliseconds it takes. So I decided to wrap my Lambdas in AWS API Gateway restful service, which can capture the time the call was made. Then the code in my Lambdas can just do some basic maths. Finally, I wanted to script it all so you can do the same experiment too.

Here are some code snippets, if you want to skip my GitHub repo completely.

I wrote some scripts to setup, invoke and tear down these functions using the AWSCLI. The invoke script pares down the responses to just what I care about and prints them to the console. The results were:

As you can see Java has no issues warm, but when cold it is more than four times slower than its closest competitor.

The Default Solution

As Ella said, the solution is to ping your Lambda before its going to be called. For something like a website’s backend code, that’s all the time. On AWS that means setting up a CloudWatch Events Rule. I’ve had a CTO tell me this is the only solution. However, it doesn’t sit right with me, setting up a whole new subsystem to ensure my code runs correctly. It feels like a patch job, a hack. Also, this solution doesn’t deal with the problem of scaling up when you have too many requests. Those new containers are still going to start slowly. For me personally, the solution is to use Python, or indeed any other language that I used in my examples above.

The Quarkus and GraalVM Solution

What? You want more? You want a satisfying solution? OK, fine.

At one of my beloved Java Meetups here in Manchester Edson Yanaga came to talk to us about Quarkus, which is a plugin for Java that makes your code load all its resources lazily. This way you can redeploy your code in seconds with a minimal memory footprint. Furthermore, Quarkus optimizes your code for use with GraalVM, which is an alternative JRE that allows you to build native apps. The advantage being these apps can start up in milliseconds. So you see where this is going right? At the end of the talk I asked Edson about deploying to something like AWS Lambda, and he put me onto the ‘integration layer’ that they have created.

And of course that lead to me writing more code. I had to tweak my solution to make it work with this layer but I made a Quarkus and a Quarkus plus GraalVM version of the above Lambdas.

So the only thing that we need to worry about there is the @Named(“test”). That name is referred to in the properties file, to tell Quarkus where our Lambda code is. I put together the ‘event.getPath().substring(1)’ section because I didn’t want to write two versions of the code; one for Quarkus on its own and one for with GraalVM.

Running my ‘wait a few days’ experiment nets me the results.

We can see that Quarkus on its own gave us nothing good in terms of time, because it is another dependency that needed to be started up with my code. On the other hand, Quarkus and GraalVM together startup from cold in a third of the time of Java on its own! It’s still not as fast as any of the other languages I looked at, but it is a lot better.

When to Use Java Over Another Language

We got our Java Lambda’s cold start time down to a reasonable level. One that will not make people think that your website is broken if they are the first one on it after a few days of silence. The downside being that we had to take extra time to compile to a native image, thus sacrificing Java’s ‘write once run anywhere’ feature. However, you may be in a position where no-one is willing or able to write in another language, or where adding another language to your solution would be messy. Likewise, you might not care about cold starts, so long as your code runs eventually. Your solution may even involve hitting your Lambda at regular intervals anyway. There may be a specific Java library that is advantageous to employ for your solution. The advantages of using a strongly typed object oriented language still stand on a Lambda. All of these are reasonable rationale for choosing Java.

Personally I’d default away from Java, but if you have good reasons, I’d absolutely recommend using Quarkus and GraalVM. So there you go, you can write AWS Lambdas in Java now.