Ugrás a tartalomra
IngyenesAudit a Drupal-oldaladrólÍrásos audit a Drupal-oldaladról, három munkanapon belül
Összes cikk

Backend és fejlesztés2026. nov. 5.6 perc olvasás

Bot handling in Drupal: botbuster, and why real protection lives in the WAF

Valentin ZsigmondAlapító és vezető fejlesztő

Some pages cost more than others. A search results page, a faceted catalog or a listing with many filter combinations has to be built fresh for almost every request, and crawlers love them, because every combination is a new link. When a scraper finds those paths, the site spends its capacity on pages nobody reads.

We maintain BotBuster (machine name botbuster) for exactly this case. It is a small module with no dependencies outside core, and it can take real load off a site. It also has a clear scope, because it runs inside Drupal. This article explains what the module does and how to set it up, and then where we put the protection that does the heavy lifting: at the edge, in front of the site.

What botbuster does

BotBuster puts a browser check in front of the paths you choose. A request to a protected path without a valid token does not reach your page. It gets a small challenge page instead, which confirms the visitor is using a browser, stores a token and reloads the original address. From then on the visitor goes straight through.

That keeps simple scrapers and scripted tools away from your most expensive pages. A real visitor sees a short pause once, and nothing after that until the token expires.

The module writes its challenge page and its runtime settings as files, so it needs a private file system. Set the path in settings.php before you install it; the module's requirements check refuses to install without one.

// settings.php: keep private files outside the web root.
$settings['file_private_path'] = '../private';

The private file system keeps those files out of the web root, where anyone could read them. When the configuration is saved, and on every cache clear, the module renders the challenge page ahead of time, so answering with it needs no theme, no render pipeline and no configuration lookup.

Choosing the paths to protect

Configuration lives at /admin/config/system/botbuster and is stored in the botbuster.settings config object, so it exports and deploys like any other configuration. Here is the protection part, as you can keep it in your config sync directory:

ddos_protection:
  enabled: true
  protected_paths: |-
    /search
    /search/*
    /ro/search
    /ro/search/*
  token_lifetime: 86400

Protection is enabled by default, but the path list starts empty, so a fresh install protects nothing until you add paths. Each line is a pattern where * matches anything. The token lifetime is picked from a list in the form, from one hour to seven days.

List every address a protected page answers on. On a multilingual site, /search does not cover /ro/search, so add the language versions, as in the example.

Where it sits in the request

BotBuster is an HTTP middleware, registered in botbuster.services.yml:

services:
  botbuster.ddos_protection_middleware:
    class: Drupal\botbuster\Middleware\DdosProtectionMiddleware
    arguments: ['@file_system']
    tags:
      - { name: http_middleware, priority: 300, responder: true }

Drupal stacks its middlewares by priority, highest on the outside. At 300, BotBuster wraps the internal page cache, which sits at 200. So it decides before Drupal looks for a cached page, before routing and before a session starts. The core of its handle() method is short (comments and an installer guard trimmed):

$config = $this->getConfig();

if (empty($config['enabled'])) {
  return $this->httpKernel->handle($request, $type, $catch);
}

$protected_paths = $config['patterns'] ?? [];

if ($this->isProtectedPath($request, $protected_paths)) {
  $cookie_name = $config['cookie_name'] ?? 'botbuster_token';
  $token = $request->cookies->get($cookie_name);
  $token_lifetime = $config['token_lifetime'] ?? 86400;

  if (!$this->isValidToken($token, $token_lifetime)) {
    $new_token = $this->generateToken($token_lifetime);
    return $this->createChallengeResponse($request, $new_token, $cookie_name);
  }
}

return $this->httpKernel->handle($request, $type, $catch);

Unprotected paths pass straight through. A protected request either carries a valid token or gets the challenge, and nothing in between. The token is issued and signed by the server, so a client cannot make one up.

What visitors and integrations see

The challenge response is a 503 that is never cached, so no proxy keeps it and hands it to the next visitor. After a GET, the page reloads the original address, query string included. A challenged POST is never replayed: the page asks the visitor to go back and submit the form again. AJAX and API callers get the 503 and must pass through a normal page first, so keep API routes off the list.

The token is stored in a cookie marked Secure, so the site must be served over HTTPS; on plain HTTP the browser does not keep it and the visitor sees the challenge again. If a CDN or reverse proxy sits in front of the site, it has to forward that cookie to Drupal, or visitors loop on the challenge. The settings form warns when it detects proxy headers, and the cookie name is configurable for proxies that only pass certain names.

What a module inside Drupal is for

Look again at where the middleware runs. By the time it decides anything, the web server has accepted the connection, a PHP worker has been assigned and Drupal's kernel has booted. The challenge is much cheaper than a search page, but it is still an answer from PHP.

That shapes how to use it:

  • A filter, not a flood defense. It takes simple scripted traffic off a few expensive paths. Large volumes of traffic have to be stopped before they reach the application at all.
  • Expensive paths only. A crawler that does not run JavaScript gets a 503 on every protected path. Protect costly, low-value paths like search and facets, never the content you want indexed.
  • Part of a whole. On its own, it is a useful layer for one narrow problem. Treat it as one layer among several, not as the site's security.

None of this makes the module less useful. It makes it a targeted tool: it keeps scripted traffic off the handful of paths that are expensive to build. The broad sorting of traffic belongs somewhere cheaper.

Real protection lives at the edge

The cheapest request to handle is one that never reaches PHP. That is why every site we host sits behind an edge layer in front of the application, with a web application firewall your visitors never notice and bot handling. Its rules are tuned against each site's real traffic, so legitimate visitors are not the ones being stopped.

The edge sees every request before the application does, across the whole platform, and sorts traffic there. Legitimate crawlers keep their access, unidentified automation is kept to a reasonable pace, as described in keeping automated traffic in its lane, and floods and known bad actors are stopped without spending a PHP worker. Real visitors never come near the limits. What reaches Drupal has already been sorted.

Using both

On our platform, the edge does the main work and most sites need nothing more. BotBuster still has a place as a second, narrow layer: a search or facet path that remains expensive even after the edge has filtered traffic, or a site that runs somewhere without an edge of its own. In both cases, configure it the same way. Pick a few costly paths, include language versions, make sure any proxy forwards the cookie, and leave content pages open to crawlers.

The module is on drupal.org, and issues and patches are welcome there. If bot load is costing your site capacity and you want a second pair of eyes on where to stop it, get in touch.

A szerző

Valentin Zsigmond

Alapító és vezető fejlesztő

Full-stack Drupal-mérnök, aki közel két évtizedes pályája során nagy fejlesztőcsapatokat vezetett nagyvállalati környezetben. A Tilizy Digitalt azért alapította, hogy a senior szintű szakértelmet közvetlenül juttassa el azokhoz a cégekhez, amelyeknek szükségük van rá. Számos Drupal-fejlesztő mentora és az SDX megalkotója, amely egy Drupal-bővítmény modern, React és Vue alapú felületek építéséhez.

Tetszett a cikk?

Ha ez közel áll hozzád, képzeld el, mire jutnánk együtt a Drupal oldaladon.