Multiline
Comment

I’m a big fan of using Nunjucks macros to build pre-rendered components in Eleventy projects. Plopping an element into place is a nice, simple affair:

{%- from "system/component.njk" import component with context -%}

<aside class="theme-gray--s | sidebar">
	{{ component( "widget-today", {
		date: now,
		scope: "sidebar"
	} ) }}
</aside>

And unlike includes, values are passed directly to the macro and don’t depend on variables that potentially pollute the global scope.

There is a frustrating limitation, however:

Important note: […] please be aware that you cannot do anything asynchronous inside macros.

In other words, shortcodes that perform asynchronous operations — like optimizing remote images using the Eleventy Image plugin — will simply render an empty value when called from inside a macro:

{% macro render( cardImage, description, title, url ) %}
<div class="card">
	{% image cardImage.alt, cardImage.src %}
	<h2 class="card__title">{{ title }}</h2>
	<p class="card__description">{{ description }}</p>
</div>
{% endmacro %}

<div class="card">
	<h2 class="card__title">Lorem Ipsum</h2>
	<p class="card__description">Dolor sit amet, consectetur adipiscing elit, sed do eiusmod.</p>
</div>

This bums me out every single time I bump into it, and I grumble to myself that there must be a better way to make things work than cobbling together an ugly, weird workaround.

Friends, I’m thrilled to announce: I’ve cobbled together a beautiful, weird workaround for rendering content asynchronously that plays nicely with good ol' synchro-only Nunjucks macros.

Rethinking the role of shortcodes

It’s common for a single shortcode to perform a bunch of different actions behind the scenes. For example, when an {% image %} shortcode that's backed by the Eleventy Image plugin is called like this:

{% image "Nearly completed puzzle with a single missing piece", "https://unsplash.com/photos/B-x4VaIriRc/download?w=1920" %}

the underlying function is typically responsible for a handful of operations:

• Fetching the remote image and stashing a copy in a local cache
• Crunching numbers about image metadata (ex., height and width)
• Converting the image to a few different formats
• Generating new variations of those images at different sizes
• Building out a <picture> element that ties the new images together, and returning it to the template

From a page-rendering standpoint we’re primarily interested in the final step, rendering HTML, and this is precisely where macros get into trouble.

The shortcode can’t build HTML to return to the template without first fetching the remote image, and fetching a remote image is an inherently asynchronous process. Poor old Nunjucks just wasn’t designed to be patient in this scenario, so it moves on without rendering any HTML at all.

This feels like an intractable problem, until we stop and ask: who says the HTML has to come from the shortcode?

Defer asynchronous work with structured placeholders

What if we simplified the role of shortcodes considerably and, instead of expecting them to deliver browser-ready HTML, just asked them to package all the pertinent information into a placeholder with a known format? Like this:

<!-- NAMESPACE { "key1": "value", "key2": "value2" } -->

Now this is something Nunjucks macros can handle! It would be left to some later process to parse the placeholder and perform the asynchronous image processing that used to be handled by the shortcode, but at least it's no longer rendering an empty value.

Here’s how a simple placeholder-injecting {% image %} shortcode might be implemented:

// src/_eleventy/shortcodes/image.js

const path = require( "path" );

/**
 * @param {string} alt
 * @param {string} src
 * @return {string}
 */
module.exports = function( alt, src )
{
	if( !alt )
	{
		throw new Error( `Missing required alt attribute for image '${src}'` );
	}

	const properties = JSON.stringify({
		alt,
		src,
	});

	return `<!-- IMAGE ${properties} -->`;
};

We can now use our familiar shortcode to define alt and src properties for a remote image and have that information stored in a placeholder rendered by the template:

<!-- IMAGE { "alt": "Nearly completed puzzle with a single missing piece", "src": "https://unsplash.com/photos/B-x4VaIriRc/download?w=1920" } -->

Kinda neat! Wrapping the data in an HTML comment, a trick I copped from Eleventy Edge, ensures that even if placeholder replacement goes completely off the rails our data isn't rendered in a browser for the world to see.

But how do we get from a placeholder to displaying an actual image? To answer that, we have to figure out which process should be responsible for picking up where the shortcode left off.

Admittedly, I didn't have a clear idea about how to tackle this at first. Maybe a post-build script of some sort that iterates over every HTML file in the build output folder, reads the contents, performs a find-and-replace, then writes the updated contents back to disk? It's definitely do-able, but that sounds like a lot of file-juggling code, and isn't that why we're using Eleventy in the first place?

It wasn't until my teammate Nick recently reminded me about Eleventy transforms that the pieces finally came together.

Perform asynchronous work in transforms

Transforms are functions that modify the contents of a template that has already been rendered by its engine. Excitingly, they are permitted to operate asynchronously, making them an ideal place in Eleventy's lifecycle to convert placeholder data into rendered content.

This is what a transform function that accompanies our placeholder shortcode might look like:

// src/_eleventy/transforms/async-optimize-images.js

const Image = require( "@11ty/eleventy-img" );

module.exports = function( content )
{
	if( !this.outputPath || !this.outputPath.endsWith( ".html" ) )
	{
		// We're not interested in transforming any non-HTML output
		return content;
	}

	// Find all relevant placeholders on the page
	const placeholderPattern = new RegExp( "<!-- IMAGE {[^}]+} -->", "g" );
	const placeholders = content.match( placeholderPattern );

	if( placeholders )
	{
		return new Promise( ( resolve, reject ) =>
		{
			const promises = placeholders.map( async (placeholder) =>
			{
				// Extract structured data properties
				const propertiesPattern = /{[^}]+}/;
				const propertiesString = placeholder.match( propertiesPattern );

				if( propertiesString )
				{
					const { alt, src } = JSON.parse( propertiesString );

					// Perform async work
					const metadata = await Image( src, {/* options */} );

					// Replace placeholders with <picture> element
					const html = Image.generateHTML( metadata, imageAttributes );
					content = content.replace( placeholder, html );
				}
			} );

			// Wait for async work to finish or error out
			return Promise.all( promises )
				.then( () => resolve( content ) )
				.catch( ( error ) => reject( error ) );
		}
	}
}

It works like this:

1. The function parses the contents of each HTML page looking for one or more strings with the same shape as our <!-- IMAGE {...} --> placeholder.
2. If it encounters any, it grabs the JSON blob and hands the alt and src values off to Eleventy Image's asynchronous optimization function.
3. When images are done being measured, squooshed, and converted, it replaces the original placeholder string with a <picture> element generated by the plugin

And that's it! Each time the site is built, the {% image %} shortcode inside our macro generates a placeholder:

<div class="card">
	<!-- IMAGE { "alt": "Nearly completed puzzle with a single missing piece", "src": "https://unsplash.com/photos/B-x4VaIriRc/download?w=1920" } -->
	<h2 class="card__title">Lorem Ipsum</h2>
	<p class="card__description">Dolor sit amet, consectetur adipiscing elit, sed do eiusmod.</p>
</div>

that is eventually parsed and processed by our transform function, and replaced with a big, beautiful <picture> element:

<div class="card">
	<picture>
		<source type="image/avif" srcset="/images/W2AWOHQQ8u-800.avif 800w, /images/W2AWOHQQ8u-1000.avif 1000w, /images/W2AWOHQQ8u-1200.avif 1200w, /images/W2AWOHQQ8u-1400.avif 1400w" sizes="(min-width: 1237px) 748px, (min-width: 768px) 59vw, 100vw">
		<source type="image/jpeg" srcset="/images/W2AWOHQQ8u-800.jpeg 800w, /images/W2AWOHQQ8u-1000.jpeg 1000w, /images/W2AWOHQQ8u-1200.jpeg 1200w, /images/W2AWOHQQ8u-1400.jpeg 1400w" sizes="(min-width: 1237px) 748px, (min-width: 768px) 59vw, 100vw">
		<source type="image/webp" srcset="/images/W2AWOHQQ8u-800.webp 800w, /images/W2AWOHQQ8u-1000.webp 1000w, /images/W2AWOHQQ8u-1200.webp 1200w, /images/W2AWOHQQ8u-1400.webp 1400w" sizes="(min-width: 1237px) 748px, (min-width: 768px) 59vw, 100vw">
		<img alt="Nearly completed puzzle with a single missing piece" loading="lazy" decoding="async" src="/images/W2AWOHQQ8u-800.jpeg" width="1400" height="933">
	</picture>
	<h2 class="card__title">Lorem Ipsum</h2>
	<p class="card__description">Dolor sit amet, consectetur adipiscing elit, sed do eiusmod.</p>
</div>