How to test internal links and routes automatically

You rename an article, change the slug and publish. The new page works. The five pages that pointed to the old address now lead to a 404 error, and nobody notices until a reader complains.

This kind of failure is silent because the build, the stage that generates the site, usually finishes without errors. This tutorial shows how to create an automated check of internal links and routes for a static site, with examples in Astro, and ends with the case of Token & Tech itself.

What exactly to test

Before choosing a tool, define what counts as an error. For a static site, five checks cover most problems:

1. Internal links. Every href and src pointing to the site itself leads to a file that exists.
2. Expected routes. Every page that should exist has been generated. The most practical reference list is the sitemap, the XML file that lists the site’s URLs for search engines. Under the official protocol, each URL goes in a <loc> tag and must be complete, starting with the protocol (https://).
3. Trailing slash. The address format is consistent (/article/ or /article, not both).
4. Canonical. The rel="canonical" tag indicates which URL represents the page when there are duplicate versions. Google treats it as a “strong signal” and recommends using absolute addresses (Google Search Central).
5. Hreflang, on multilingual sites. Each language version must list itself and all the others. According to Google, if two pages do not point to each other, the tags are ignored.

The first two are universal. The last three depend on your site’s rules, and that is where your own script usually comes in.

Step 1: generate the site and test the output

Test the result of the build, not the source files. A link written in Markdown can turn into a different address after passing through components, layouts and plugins. What the reader receives is the final HTML.

In Astro, astro build writes the site to dist/. With the build.format: "directory" option, which is the default, each page becomes a folder with an index.html: /about becomes /about/index.html (configuration reference). This makes checking simple: to know whether /about/ exists, just check whether the file dist/about/index.html exists.

The same reference recommends combining this format with trailingSlash: "always", so that the development server and the build behave the same way. There is a caveat: for prerendered pages, the trailing slash is handled by the hosting, which may not follow this setting. That is why it is worth testing the trailing slash explicitly.

Step 2: start with a ready-made tool

For broken links, there are mature open-source tools. Two options, with commands taken from each one’s documentation:

lychee. A link checker written in Rust that reads HTML and Markdown. The project README documents the useful options here: --offline checks only local files and blocks network access; --root-dir sets the folder used to resolve absolute links, such as /en/, and must be an absolute path. One possible use:

lychee --offline --root-dir "$PWD/dist" --no-progress dist

lychee exits with code 2 when at least one link fails, which is enough to stop an automation. There is also an official GitHub action (lycheeverse/lychee-action).

linkinator. A Node.js tool with a command line and an API. According to its README, when given a local folder it starts a temporary server and crawls the site as if it were published. The --recurse option follows links on the same domain:

npx linkinator ./dist --recurse

It accepts --skip with regular expressions to ignore addresses and has retry options for 429 and 5xx responses, useful when you also check external links.

An editorial recommendation: separate internal links from external ones. Internal links must always pass, and the test can be strict. External links depend on third-party servers, which go down, block bots or rate-limit requests. Checking them with the same strictness produces failures that are not yours. Run them in a separate task, with caching and retries, and without blocking publication.

Step 3: test routes with the site served

A file existing in dist/ does not prove that the server delivers it with a 200 status. To test server behavior, serve the build and make real requests.

The astro preview command starts a local server with the contents of dist/, by default on port 4321. The documentation warns that it is not designed for production (CLI reference), which is not a problem in a test.

With Playwright, a testing library that controls browsers, you can go through the sitemap and check the status of each route. One caveat: according to the page.goto documentation, the method does not throw when the server responds with a valid HTTP status, including 404 and 500. You need to check the status explicitly:

import { test, expect } from "@playwright/test";
import { readFileSync } from "node:fs";

const paths = [...readFileSync("dist/sitemap.xml", "utf8").matchAll(/<loc>([^<]+)<\/loc>/g)]
  .map((m) => new URL(m[1]).pathname);

for (const path of paths) {
  test(`route ${path}`, async ({ page }) => {
    const res = await page.goto(`http://localhost:4321${path}`);
    expect(res?.status()).toBe(200);
  });
}

This test is slower than checking files, because it opens a browser. For hundreds of routes, run it only on a sample or only in the main pipeline.

Step 4: write the rules only your site has

Generic tools do not know that your canonical must match the sitemap URL, that each page must have exactly one H1 or that hreflang must be reciprocal. These rules fit in a short script that reads the files in dist/ and fails if it finds a problem.

The basic structure has three parts: load the list of URLs from the sitemap, open the HTML of each one and accumulate the errors. At the end, if there are errors, the script prints the list and exits with code 1. That detail is what makes the test work: a script that only prints warnings gets ignored within a few weeks.

Documented case: the Token & Tech checker

The site you are reading is built with Astro and has its own checker, which runs on the build output: scripts/check-dist.mjs, called by npm run check. It uses no external dependencies, only native Node.js modules.

For each URL in the sitemap, the script checks that the file exists, that there is <html lang>, a title, a description, a single H1 tag and a social sharing image, and that the canonical matches the page’s own URL. Then it cross-checks hreflang: each page must reference itself, have an x-default and receive a link back from each version it references.

The link check goes through all generated HTML. The core excerpt, summarized:

for (const m of h.matchAll(/(?:href|src)="(\/[^"#?]*)[^"]*"/g)) {
  const p = m[1];
  if (!p.endsWith("/")) { errors.push(`${rel}: internal link without trailing slash ${p}`); continue; }
  if (p !== "/" && !existsSync(fileFor(p))) errors.push(`${rel}: broken link ${p}`);
}

Files with extensions, such as images, are checked separately. An internal link without a trailing slash also counts as an error, which keeps the address format consistent.

There is a process detail. The site publishes article by article, and one text may cite another that is not yet live. For this, an earlier step (scripts/postbuild.mjs) turns links in article bodies that point to pages missing from dist/ into plain text. When the cited article is published, the next build restores the link, without touching the original text. The checker runs afterwards and, if any broken link is still left in the HTML, exits with code 1 and lists the problem.

The other half of prevention happens before the build: the collection schema, which validates the content of each article, described in Astro Content Collections.

Key takeaways

• Test dist/, not the source code. Use the sitemap as the list of expected routes.
• Start with lychee or linkinator for broken links. Write a script only for your site’s own rules.
• Separate internal links (strict, block publication) from external ones (with caching and retries, non-blocking).
• Make sure the test actually fails, with a non-zero exit code, and that it runs on every publication.