Putting a Drupal site behind Cloudflare can make a substantial difference to performance and security.

But it does not mean unwanted automated traffic will suddenly disappear.

We recently investigated a Drupal site that was already behind Cloudflare but was still showing heavy server load, large numbers of PHP requests and rapidly growing cache tables.

At first glance, that seems contradictory.

If Cloudflare is sitting in front of the site, why are bots still reaching Drupal?

Because Cloudflare is a layer between visitors and your server — not an automatic instruction to block every crawler that might be expensive for your application.

Cloudflare can proxy a request and still send it to Drupal

When a hostname is proxied through Cloudflare, requests reach Cloudflare first.

Cloudflare can:

• serve cached content
• apply security rules
• identify some automated traffic
• rate-limit requests
• block traffic matching configured rules

But if a request is allowed and Cloudflare does not serve the requested page from its own cache, the request is passed to your origin server.

Drupal still has to deal with it.

So a request path may look like:

Bot → Cloudflare → web server → PHP → Drupal → database

Cloudflare being present does not necessarily remove PHP, Drupal or the database from that chain.

Dynamic Drupal pages are where the problem becomes expensive

Static files are usually cheap.

A request for an image, stylesheet or JavaScript file can often be served without Drupal doing very much at all.

Dynamic pages are different.

A request might cause Drupal to:

• bootstrap the application
• load configuration
• execute a View
• query the database
• check permissions
• render entities
• build cache metadata
• render HTML
• store the result in cache

One request is normally unremarkable.

Thousands of different dynamic requests are not.

This is particularly important with Drupal Views, search pages and faceted navigation.

Our problem was not simply "lots of bots"

One of the sites we investigated contained a resource directory with faceted filters.

Those filters generated URLs representing different combinations of the same underlying content.

For example:

/national-resources

could become variations containing different themes and filter combinations.

To a visitor, these were simply filtered views of one directory.

To Drupal, they could be distinct URLs requiring distinct page-cache entries.

Automated crawlers began exploring those combinations.

The result was thousands of different URLs reaching Drupal.

At one point, approximately 4,670 out of roughly 5,000 cache_page entries belonged to variants of the same resource directory.

Cloudflare was working.

It just wasn't being asked to prevent those requests from reaching the application.

A bot does not have to look malicious

This is an important distinction.

When people hear "bot traffic", they often imagine:

• vulnerability scanners
• brute-force attacks
• obviously fake user agents
• thousands of requests from one IP address

Modern crawler traffic does not necessarily look like that.

During our investigation, requests were spread across many IP addresses.

Some identified themselves as legitimate search crawlers.

Others presented ordinary browser user agents.

The behaviour was the problem.

A crawler repeatedly discovering new faceted URLs can cause substantial application load without doing anything that looks like a conventional attack.

That makes simple IP blocking much less effective.

Cloudflare cannot know which Drupal URLs are expensive

Cloudflare sees HTTP requests.

Drupal understands what those requests mean inside the application.

That difference matters.

Consider these two URLs:

/about-us

and:

/national-resources?theme=a&type=b&audience=c&region=d

Cloudflare may simply see two valid HTTP requests.

Drupal knows that the second might involve an exposed View, several filters, database queries and a new cache entry.

Without additional rules, Cloudflare has no reason to assume that one should be allowed and the other should not.

This is why effective bot control often requires some understanding of the application itself.

Check the origin access logs

If you suspect bots are still reaching Drupal, the origin web server logs are one of the most useful places to look.

Don't rely solely on what you think Cloudflare ought to be blocking.

Look at what the server is actually receiving.

Useful questions include:

• Which URLs are being requested most frequently?
• Are requests concentrated around one View or search page?
• Are query strings involved?
• How many different URL combinations are being generated?
• Which user agents are making the requests?
• Are requests coming from one IP or many?
• Are the same routes being requested continuously?
• Are PHP processes increasing at the same time?

The access logs tell you what actually reached the origin.

That is what matters to Drupal performance.

robots.txt and Cloudflare rules solve different problems

These tools are often confused.

A robots.txt rule tells compliant crawlers what they should not crawl.

It can be very useful for preventing legitimate search engines from exploring worthless faceted combinations.

But robots.txt is advisory.

A badly behaved crawler can simply ignore it.

Cloudflare rules operate differently.

They can actively prevent matching requests from reaching the origin.

So the two approaches can complement each other:

robots.txt

Use it to tell legitimate search engines not to crawl areas or parameter combinations that provide no useful search value.

Cloudflare security rules

Use them where requests genuinely need to be blocked, challenged or rate-limited before they reach Drupal.

Neither should be used blindly.

Don't block all query strings

After discovering that query-string URLs are creating thousands of Drupal cache entries, it can be tempting to block anything containing ?.

That is usually a bad idea.

Drupal and other web applications legitimately use query parameters for things such as:

• exposed Views filters
• search
• pagination
• tracking parameters
• administrative interfaces
• AJAX requests
• external integrations

The better solution is normally to identify the problematic route and the parameter patterns being abused.

A narrowly targeted rule is much safer than a site-wide ban.

Cache hits and origin requests are not the same thing

Another misconception is that "using Cloudflare" means Cloudflare should cache every page.

That is rarely appropriate for a Drupal application.

Many pages are dynamic, personalised or affected by cookies, query strings and application behaviour.

Even when Drupal itself has a cached copy of a page, the origin server may still need to receive the request and serve that cached response.

That is cheaper than rebuilding the page, but it is not free.

And when crawlers continually request new URL combinations, there may be no existing cached page to serve.

Drupal has to build each new variation first.

Cloudflare can actually hide the scale of the issue

A site may appear reasonably responsive in a browser while its origin server is working much harder than expected.

Cloudflare may be serving static assets efficiently and absorbing some traffic at the edge.

Meanwhile, expensive dynamic requests can still be reaching Drupal.

The first visible symptom might therefore be something indirect:

• high CPU
• growing database cache tables
• increasing PHP workers
• slow administrative pages
• intermittent 503 or 508 errors
• unusually high database activity

This is why server monitoring still matters after Cloudflare has been introduced.

Make sure the origin cannot simply be bypassed

There is another issue worth checking.

Cloudflare can only protect traffic that actually passes through it.

If the origin server remains publicly reachable through another hostname or its IP address, automated systems may be able to connect directly.

That does not mean every Drupal installation needs an elaborate network configuration.

But if you are relying on Cloudflare as a security layer, it is worth knowing whether requests can bypass it.

The same principle applies to services that Cloudflare's normal web proxy is not intended to handle.

SSH, database connections and web traffic are different services and should not be treated as though enabling the Cloudflare proxy automatically protects everything running on the server.

A practical Drupal bot investigation

When we suspect Cloudflare-protected Drupal infrastructure is still receiving too much automated traffic, the useful sequence is:

1. Confirm that the affected hostname is proxied

Make sure the requests you are investigating should actually be travelling through Cloudflare.

2. Check server resource usage

Determine whether PHP, the database, Drush or another process is consuming the resources.

3. Inspect the origin access logs

Find the routes responsible for the request volume.

4. Look for URL multiplication

Faceted search, exposed filters and search parameters are common sources of enormous crawl spaces.

5. Decide whether the traffic should exist

Some crawler access is useful.

Some is worthless.

Some is actively harmful.

6. Control compliant crawlers

Use appropriate crawl directives where there is no search benefit in indexing or traversing the URLs.

7. Stop abusive traffic before Drupal

Where necessary, use targeted Cloudflare controls to block, challenge or limit requests before they reach PHP.

8. Fix the Drupal-side cause as well

If one View can produce hundreds of thousands of crawlable combinations, bot blocking alone may only hide a design problem.

Cloudflare is not a substitute for understanding your application

The lesson from this incident was not that Cloudflare had failed.

It was doing what it had been configured to do.

The problem was that Drupal exposed a very large collection of faceted URLs, crawlers discovered them, and those requests were allowed through to the application.

Drupal then did exactly what it was designed to do: it rendered and cached them.

So when a Drupal site behind Cloudflare is still being hammered by bots, don't start with:

Why isn't Cloudflare stopping them?

Start with:

What are the bots requesting, why is Drupal making those requests expensive, and which of them should ever be reaching the origin at all?

That usually leads to a much better fix.