A large Drupal cache database does not necessarily mean Drupal is failing to clear its cache.

We recently investigated a site where the cache tables appeared to be behaving normally: cron was running and the number of records in cache_page was being kept at around 5,000.

But the table itself had grown to hundreds of megabytes.

At one point, roughly 5,000 cached pages were consuming more than 800 MB.

The culprit was not broken cache clearing.

It was faceted search.

Faceted search can create thousands of cacheable URLs

Faceted search is extremely useful. It lets users narrow a directory, catalogue or resource library using combinations of filters.

But every combination of those filters can create a different URL.

For example:

/national-resources

might become:

/national-resources?theme=employment

and then:

/national-resources?theme=employment&type=guidance

and:

/national-resources?theme=employment&type=guidance&region=wales

To the visitor, these are simply different views of the same resource directory.

To Drupal's page cache, they can be completely different pages.

With several facets and many possible values, the number of possible URL combinations can become enormous.

That was the underlying problem on this site.

Cache row count and cache size are different things

It is easy to look at a Drupal cache table and concentrate on the number of rows.

If cron runs and old cache entries are removed, the row count may remain reasonably stable. That does not mean the amount of data being cached will remain stable as well.

A cache table containing 5,000 small entries is very different from one containing 5,000 fully rendered filtered pages.

In our case, Drupal was successfully trimming the number of records.

The problem was that faceted searches were continually generating substantial new cache entries to replace them.

Crawlers make the problem much worse

Faceted search creates the potential crawl space.

Bots then discover it.

Search-engine crawlers and other automated systems can follow links between different combinations of facets and keep requesting new URLs.

Looking at the affected cache table showed how far this had gone.

Of approximately 5,000 cache_page records, around 4,670 were variants of the same resource listing page.

Roughly 94% of the cache entries contained query strings.

Web server logs showed hundreds of requests to different versions of the resource directory over a relatively short period.

So the bots were certainly contributing to the load.

But they weren't the fundamental cause.

Without the faceted URL structure, there would not have been thousands of different pages for them to crawl and Drupal to cache.

Why faceted search creates a crawler trap

Suppose a directory has five filters, each with ten possible values.

Users will normally choose only a handful of sensible combinations.

A crawler does not necessarily behave like a user.

If every facet creates links to further filtered states, it can continue exploring combinations of:

• themes
• categories
• regions
• resource types
• audiences
• topics

The number of possible URLs grows very quickly.

Drupal then sees requests for what appear to be different pages:

Facet combination → unique URL → Drupal renders View → page is cached

Repeat that thousands of times and cache_page can become very large even though Drupal's cache system itself is working correctly.

Why cron doesn't solve it

Cron can remove old cache entries.

It cannot stop new faceted URLs being requested.

So the cycle becomes:

Crawler finds facet URL → Drupal renders it → cache entry created → old cache entry eventually removed → another facet URL is discovered → another cache entry created

The cache may therefore remain at roughly the same number of rows while continuing to contain hundreds of megabytes of data.

Clearing the cache manually has the same limitation.

It removes the current entries, but it does nothing about the faceted search URLs causing them to be created.

How to recognise this problem

If cache_page or another Drupal cache table becomes unexpectedly large, look at what is actually being cached before changing Drupal's cache configuration.

In particular, check:

• how many cache records exist
• the average size of those records
• which routes account for most of them
• whether those URLs contain query parameters
• whether they belong to a View with exposed or faceted filters
• whether automated traffic is requesting large numbers of different combinations

If thousands of cache records point back to one filtered directory or View, the cache is probably the symptom rather than the problem.

Fix the facets, not just the cache

The long-term solution is to control how faceted URLs are exposed and crawled.

Depending on the site, that might involve:

• preventing crawlers from following unwanted facet combinations
• using appropriate robots.txt rules
• adding noindex directives where appropriate
• reviewing how Views exposes filter links
• limiting unnecessary combinations
• improving canonicalisation
• blocking abusive automated traffic at Cloudflare
• reconsidering whether every filtered state should be independently cacheable or discoverable

Crawler controls are useful, but they should be viewed as part of the solution rather than the explanation for the problem.

The fundamental issue is that faceted search can turn one Drupal page into thousands of technically distinct URLs.

Drupal's cache may actually be doing exactly what it should

When a Drupal cache table reaches several hundred megabytes, the natural reaction is to assume something is wrong with caching.

Sometimes there is.

But in this case, Drupal was doing exactly what it had been asked to do.

It received requests for thousands of different faceted-search URLs, rendered them and cached the results.

The real question was therefore not:

Why isn't Drupal clearing its cache?

It was:

Why have we created thousands of cacheable versions of the same page?

Once we identified the faceted search as the culprit, the behaviour of the cache made perfect sense.