Categories
Uncategorized

Comparing Static Site Generator Build Times

There are so many static site generators (SSGs). It’s overwhelming trying to decide where to start. While an abundance of helpful articles may help wade through the (popular) options, they don’t magically make the decision easy.

I’ve been on a quest to help make that decision easier. A colleague of mine built a static site generator evaluation cheatsheet. It provides a really nice snapshot across numerous popular SSG choices. What’s missing is how they actually perform in action.

One feature every static site generator has in common is that it takes input data, runs it through a templating engine, and outputs HTML files. We typically refer to this process as The Build.

There’s too much nuance, context, and variability needed to compare how various SSGs perform during the build process to display on a spreadsheet — and thus begins our test to benchmark build times against popular static site generators.

This isn’t just to determine which SSG is fastest. Hugo already has that reputation. gjstatic I mean, they say it on their website — The world’s fastest framework for building websites — so it must be true!

This is an in-depth comparison of build times across multiple popular SSGs and, more importantly, to analyze why those build times look the way they do. Blindly choosing the fastest or discrediting the slowest would be a mistake. Let’s find out why.

The tests
The testing process is designed to start simple — with just a few popular SSGs and a simple data format. A foundation on which to expand to more SSGs and more nuanced data. For today, the test includes six popular SSG choices:

Eleventy
Gatsby
Hugo
Jekyll
Next
Nuxt
Each test used the following approach and conditions:

The data source for each build are Markdown files with a randomly-generated title (as frontmatter) and body (containing three paragraphs of content).
The content contains no images.
Tests are run in series on a single machine, making the actual values less relevant than the relative comparison among the lot.
The output is plain text on an HTML page, run through the default starter, following each SSG’s respective guide on getting started.
Each test is a cold run. Caches are cleared and Markdown files are regenerated for every test.
These tests are considered benchmark tests. They are using basic Markdown files and outputting unstyled HTML into the built output.

In other words, the output is technically a website that could be deployed to production, though it’s not really a real-world scenario. Instead, this provides a baseline comparison among these frameworks. The choices you make as a developer using one of these frameworks will adjust the build times in various ways (usually by slowing it down).

For example, one way in which this doesn’t represent the real-world is that we’re testing cold builds. In the real-world, if you have 10,000 Markdown files as your data source and are using Gatsby, you’re going to make use of Gatsby’s cache, which will greatly reduce the build times (by as much as half).

The same can be said for incremental builds, which are related to warm versus cold runs in that they only build the file that changed. We’re not testing the incremental approach in these tests (at this time).

The two tiers of static site generators
Before we do that, let’s first consider that there are really two tiers of static site generators. Let’s call them basic and advanced.

Basic generators (which are not basic under the hood) are essentially a command-line interface (CLI) that takes in data and outputs HTML, and can often be extended to process assets (which we’re not doing here).
Advanced generators offer something in addition to outputting a static site, such as server-side rendering, serverless functions, and framework integration. They tend to be configured to be more dynamic right out of the box.
I intentionally chose three of each type of generator in this test. Falling into the basic bucket would be Eleventy, Hugo, and Jekyll. The other three are based on a front-end framework and ship with various amounts of tooling. Gatsby and Next are built on React, while Nuxt is built atop Vue.

Basic generators Advanced generators
Eleventy Gatsby
Hugo Next
Jekyll Nuxt
My hypothesis
Let’s apply the scientific method to this approach because science is fun (and useful)!

My hypothesis is that if an SSG is advanced, then it will perform slower than a basic SSG. I believe the results will reflect that because advanced SSGs have more overhead than basic SSGs. Thus, it’s likely that we’re going to see both groups of generators — basic and advanced — bundled together, in the results with basic generators moving significantly quicker.

Let me expand on that hypothesis a bit.

Linear(ish) and fast
Hugo and Eleventy will fly with smaller datasets. They are (relatively) simple processes in Go and Node.js, respectively, and their build output will reflect that. While both SSG will slow down as the number of files grows, I expect them to remain at the top of the class, though Eleventy may be a little less linear at scale, simply because Go tends to be more performant than Node.

Slow, then fast, but still slow
The advanced, or framework-bound SSGs, will start out and appear slow. I suspect a single-file test to contain a significant difference — milliseconds for the basic ones, compared to several seconds for Gatsby, Next, and Nuxt.

The framework-based SSGs are each built using webpack, bringing a significant amount of overhead along with it, regardless of the amount of content they are processing. That’s the baggage we sign up for in using those tools (more on this later).

But, as we add thousands of files, I suspect we’ll see the gap between the buckets close, though the advanced SSG group will stay farther behind by some significant amount.

In the advanced SSG group, I expect Gatsby to be the fastest, only because it doesn’t have a server-side component to worry about — but that’s just a gut feeling. Next and Nuxt may have optimized this to the point where, if we’re not using that feature, it won’t affect build times. And I suspect Nuxt will beat out Next, only because there is a little less overhead with Vue, compared to React.

Jekyll: The odd child
Ruby is infamously slow. It’s gotten more performant over time, but I don’t expect it to scale with Node, and certainly not with Go. And yet, at the same time, it doesn’t have the baggage of a framework.

At first, I think we’ll see Jekyll as pretty speedy, perhaps even indistinguishable from Eleventy. But as we get to the thousands of files, the performance will take a hit. My gut feeling is that there may exist a point at which Jekyll becomes the slowest of all six. We’ll push up to the 100,000 mark to see for sure.

Leave a Reply

Your email address will not be published. Required fields are marked *