raw Software
RAW Software Administration Web Servers

Combine CSS and JavaScript with lighttpd

Robert Eisele

Combining stylesheets and scripts used to be one of the most reliable ways to make a page load faster. Under HTTP/1.1, every file added request headers, competed for a small pool of connections, and often waited for another response to finish. One CSS bundle and one JavaScript bundle could remove much of that delay.

The transport has changed, but the engineering question has not disappeared. HTTP/2 multiplexes concurrent exchanges over one connection, and HTTP/3 gives requests independent QUIC streams. That sharply reduces the original connection bottleneck. It does not make every extra file free, nor does it make one enormous bundle ideal. The useful target is now a small number of coherent, cacheable artifacts rather than the fewest possible requests.

What the original lighttpd technique did

The 2010 implementation encoded an ordered file list in the request URL. A Lua script attached through mod_magnet parsed that list, concatenated the files with cat, ran CSS Tidy or YUI Compressor, produced a gzip copy, and cached the result under a hash of the request URI. It also required a private Lighttpd patch to expose an internal hash function to Lua.

That design had an attractive fast path: once generated, the bundle lived as a static file. Its miss path is not suitable for a current deployment. Request-derived paths reached shell commands, the cache key described the URL rather than the source contents, and several external processes ran inside Lighttpd's event loop. The official mod_magnet documentation is explicit: a blocking operation in a magnet script pauses the server for every other request. The old lighty.env, lighty.header, and lighty.content interfaces are also deprecated in favor of the request object introduced with Lighttpd 1.4.60.

Decide what belongs together

A bundle is a cache boundary. Put files together when they are usually requested together and change at roughly the same rate. Splitting a stable vendor dependency from frequently changing application code can preserve a useful browser cache entry. Route-specific code should remain separate when most visitors never need it. CSS order must stay deterministic because later rules can override earlier ones; JavaScript modules should be bundled from their import graph rather than concatenated in filename order.

Measure before choosing the split. Compare a cold load and a repeat load over the protocols actually served in production. The browser waterfall, transferred bytes, compression ratio, cache hits, and rendering milestones reveal whether another bundle removes latency or merely causes more unchanged code to be invalidated.

Build once and publish immutable names

The normal solution is a build-time pipeline. A JavaScript bundler can resolve imports, remove unreachable code, minify output, and extract CSS. The final bytes receive a content hash, producing names such as app.a1841d7c97cdb2f3.css and app.6d2cb950532a263f.js. HTML is rendered from the generated manifest:

<link rel="stylesheet" href="/assets/app.a1841d7c97cdb2f3.css">
<script type="module" src="/assets/app.6d2cb950532a263f.js"></script>

A changed byte creates a new URL, while an unchanged artifact keeps its old name. This makes long-lived caching safe and removes request-time invalidation logic. The following small Node.js build illustrates the naming rule for already ordered legacy files. It deliberately does not replace a module-aware compiler or a CSS minifier; those transformations should run before the digest is calculated.

import { createHash } from 'node:crypto';
import { mkdir, readFile, writeFile } from 'node:fs/promises';

const bundles = [
  {
    key: 'css',
    files: ['src/css/reset.css', 'src/css/layout.css', 'src/css/home.css'],
  },
  {
    key: 'js',
    files: ['src/js/runtime.js', 'src/js/home.js'],
  },
];

await mkdir('public/assets', { recursive: true });
const manifest = {};

for (const bundle of bundles) {
  const chunks = await Promise.all(
    bundle.files.map((file) => readFile(file, 'utf8')),
  );
  const source = chunks.join('\n');
  const digest = createHash('sha256')
    .update(source)
    .digest('hex')
    .slice(0, 16);
  const filename = `app.${digest}.${bundle.key}`;

  await writeFile(`public/assets/${filename}`, source);
  manifest[bundle.key] = `/assets/${filename}`;
}

await writeFile(
  'public/assets/manifest.json',
  JSON.stringify(manifest, null, 2) + '\n',
);

Deploy the artifacts before switching HTML to the new manifest, and retain the previous generation long enough for pages and caches that still reference it. A partial deployment otherwise turns perfect cache keys into perfect 404s. Source maps, license notices, CSS URLs, and module chunks belong to the same release transaction.

Let lighttpd serve the result

Lighttpd only needs to serve static files, negotiate content encoding, and assign a long freshness lifetime. Current mod_deflate replaces the older mod_compress module and supports Brotli and gzip. mod_expire sets Cache-Control: max-age for the matching URL space:

server.modules += ( "mod_deflate", "mod_expire" )

deflate.mimetypes = (
  "text/css",
  "text/javascript",
  "application/javascript"
)
deflate.allowed-encodings = ( "br", "gzip" )
deflate.cache-dir = "/var/cache/lighttpd/deflate"

$HTTP["url"] =~ "^/assets/app\.[0-9a-f]{16}\.(css|js)$" {
  expire.url = ( "" => "access plus 1 years" )
}

Create the deflate cache directory with ownership and permissions suitable for the Lighttpd account. Lighttpd keys compressed static responses by URL and ETag, but it does not purge stale compressed entries automatically, so a scheduled cleanup policy is part of the deployment. If an upstream or application already emits Cache-Control, avoid adding a conflicting second field.

The year-long lifetime is correct only because the URL contains a content hash. For mutable names such as /assets/app.js, use short freshness and validators instead. In HTTP caching, max-age is a duration in seconds, not an absolute Unix timestamp. ETag and Last-Modified validators let a stale representation be revalidated; they do not make a mutable URL immutable.

If bundles must be generated at runtime

Runtime generation can still be justified for tenant themes, plugin sets, or other combinations unknown during deployment. Keep the static-file fast path, but move generation to a bounded FastCGI service or another backend. The same hit/miss boundary works well for an on-demand derivative cache: Lighttpd serves an existing result directly and rewrites only a miss.

The public URL should identify a registered bundle, not contain arbitrary paths. For example, /generated/dashboard.42.css can map server-side to one ordered allowlist of files. Reject unknown bundle names, mixed extensions, traversal segments, symlinks outside the source root, excessive source counts, and oversized output. Never interpolate a URL-derived value into a shell command.

The cache key must cover the ordered source bytes, transformation options, compiler version, and bundle type. On a miss, acquire a per-key lock, write a temporary file on the target filesystem, flush and validate it, and publish it with an atomic rename. Concurrent requests should share one generation rather than start a process stampede. Apply time, memory, output-size, and worker-count limits, then let subsequent requests return to the static handler.

The miss backend should not stream the completed artifact. Enable Lighttpd's current X-Sendfile support on that backend and constrain it to the generated-artifact root:

fastcgi.server = ( "/bundle.fcgi" =>
  ( "bundle" =>
  ( "socket" => "/run/bundle-generator.sock",
    "check-local" => "disable",
    "x-sendfile" => "enable",
    "x-sendfile-docroot" => ( "/srv/www/generated/" )
  )
  )
)

After an atomic rename, the FastCGI process emits an absolute, URL-safe path in X-Sendfile. Lighttpd reads the file, derives Content-Length, and handles byte ranges without copying the body through the application process. x-sendfile-docroot is not optional hardening: an enabled backend is trusted to name files that its own operating-system account might not be able to read.

A small native FastCGI front end can perform this final handoff without starting a language runtime. Here generate_registered_bundle() represents the bounded implementation that resolves an allowlisted bundle, computes the complete cache key, locks, generates, validates, and atomically publishes it. Its returned path and media type come from server-owned data.

#include <fcgi_stdio.h>
#include <stddef.h>
#include <stdlib.h>
#include <string.h>

#define GENERATED_ROOT "/srv/www/generated/"

int generate_registered_bundle(
  const char *request_uri,
  char *path,
  size_t path_size,
  const char **media_type
);

static int is_generated_path(const char *path)
{
  const size_t root_length = sizeof(GENERATED_ROOT) - 1;

  if (strncmp(path, GENERATED_ROOT, root_length) != 0) {
    return 0;
  }

  const char *relative = path + root_length;
  return *relative != '\0'
    && strstr(relative, "..") == NULL
    && strspn(relative, "abcdefghijklmnopqrstuvwxyz0123456789-./")
      == strlen(relative);
}

static int is_media_type(const char *media_type)
{
  return media_type != NULL
    && (strcmp(media_type, "text/css") == 0
      || strcmp(media_type, "application/javascript") == 0);
}

int main(void)
{
  while (FCGI_Accept() >= 0) {
    const char *request_uri = getenv("REQUEST_URI");
    const char *media_type = NULL;
    char path[4096] = {0};

    if (request_uri == NULL
      || generate_registered_bundle(
        request_uri, path, sizeof(path), &media_type
      ) != 1) {
      printf("Status: 404 Not Found\r\n\r\n");
    } else if (!is_generated_path(path) || !is_media_type(media_type)) {
      printf("Status: 500 Internal Server Error\r\n\r\n");
    } else {
      printf("Status: 200 OK\r\n");
      printf("Content-Type: %s\r\n", media_type);
      printf("X-Sendfile: %s\r\n\r\n", path);
    }
  }

  return 0;
}

mod_magnet remains useful for cheap request manipulation, but not for compilation, shell execution, or blocking disk pipelines. If Lua participates at all, it should validate a small identifier and rewrite to an existing static path or an external backend. No user-controlled list should become a filesystem walk.

Common correctness failures

Bundling is no longer a universal request-count trick. It is a deliberate choice about dependency graphs, invalidation, and delivery. Build-time content hashes make that choice predictable; Lighttpd can then do what it does best: serve stable files with validators, compression, and cheap cache hits.

References

Related Stuff

raw / Administration
28 Apr
·
Robert Eisele

Generate and Cache Thumbnails on Demand with lighttpd

A production-oriented design for lazy image derivatives on a mostly static lighttpd host. Existing thumbnails stay on the static-file path, while cache misses enter a bounded FastCGI worker. Fixed presets, content-addressed source names, format and size validation, shell-free ImageMagick execution, resource policies, per-target locks, same-filesystem temporary files, and atomic renames constrain the expensive path. Immutable caching, preset versioning, cleanup, concurrency testing, and operational metrics complete the lifecycle.
raw / Administration
24 Jul
·
Robert Eisele

Add HTTP Response Headers in lighttpd

A modern and historical guide to adding response headers to lighttpd error pages. Fixed headers use mod_setenv directly. The complete C source recovered from the retired 2008 FastCGI archive is preserved inline, followed by a hardened replacement that validates status, header, and sendfile values without dynamic allocation. Current FastCGI configuration uses x-sendfile with a mandatory x-sendfile-docroot boundary, and all former page and download URLs redirect to this canonical article.
raw / Web Development
10 Jun
·
Robert Eisele

Faster PHP behind FastCGI

A historically preserved and technically corrected account of a 2011 FastCGI optimization experiment. A lighttpd 1.4.28 patch exported the connection descriptor and cached server time, a custom PHP 5.3.6 fork wrote its output buffer to a tmpfs file, and X-Sendfile handed the completed response back to lighttpd. The repaired example supplies the missing fopen mode, writes both buffering branches, initializes every path, escapes gzip arguments, handles descriptor reuse, and states the cleanup, same-host, latency, security, and measurement limits of the design.
All Software