A Note About 'X-Magento-Tags'

Recommended Extension

Open Dependency
Magento 2 Free Sitemap Based Cache Warmer Extension

`X-Magento-Tags` Generation

Again, it's a plugin in Magento_PageCache module.

\Magento\PageCache\Model\Layout\LayoutPlugin::afterGetOutput

/**
 * Retrieve all identities from blocks for further cache invalidation.
 *
 * @param Layout $subject
 * @param mixed $result
 * @return mixed
 */
public function afterGetOutput(Layout $subject, $result)
{
    if ($subject->isCacheable() && $this->config->isEnabled()) {
        $tags = [];
        $isVarnish = $this->config->getType() === Config::VARNISH;

        foreach ($subject->getAllBlocks() as $block) {
            if ($block instanceof IdentityInterface) {
                $isEsiBlock = $block->getTtl() > 0;
                if ($isVarnish && $isEsiBlock) {
                    continue;
                }
                $tags[] = $block->getIdentities();
            }
        }
        $tags = array_unique(array_merge([], ...$tags));
        $tags = $this->pageCacheTagsPreprocessor->process($tags);
        $this->response->setHeader('X-Magento-Tags', implode(',', $tags));
    }

    return $result;
}

See this line.

$tags[] = $block->getIdentities();

Tags and Identities

Tags come from the "identities" of all blocks on the page.

Note those blocks must implement \Magento\Framework\DataObject\IdentityInterface, otherwise they don't have "identities". \Magento\Cms\Block\Page::getIdentities is a good example.

How Cache Purging in Varnish and CDN Works

Check this VCL.

if (req.method == "PURGE") {
    if (client.ip !~ purge) {
        return (synth(405, "Method not allowed"));
    }
    # To use the X-Pool header for purging varnish during automated deployments, make sure the X-Pool header
    # has been added to the response in your backend server config. This is used, for example, by the
    # capistrano-magento2 gem for purging old content from varnish during it's deploy routine.
    if (!req.http.X-Magento-Tags-Pattern && !req.http.X-Pool) {
        return (synth(400, "X-Magento-Tags-Pattern or X-Pool header required"));
    }
    if (req.http.X-Magento-Tags-Pattern) {
      ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
    }
    if (req.http.X-Pool) {
      ban("obj.http.X-Pool ~ " + req.http.X-Pool);
    }
    return (synth(200, "Purged"));
}

Note this part.

if (req.http.X-Magento-Tags-Pattern) {
  ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
}

The cache is purged by X-Magento-Tags-Pattern(regular expression).

Purge via CLI

Actually, you can purge the cache manually if you know the tags.

# Suppose we need to purge tags: cat_c_3 and cat_p_5
curl -X PURGE -H 'X-Magento-Tags-Pattern: cat_c_3,.*,cat_p_5(,|$)' https://your-store.com/product1.html

Why Magento uses `Tag` as identifier instead of something like URL?

We give an example to answer this question. Suppose a product with ID = 5 has its product name updated, all pages contain this product, that is, the product's page itself, category pages, if the product is on homepage then the homepage, if a blog post has the product then the blog post page, all of their cache is needed to be purged.

That's why Magento uses Tag like cat_p_5 for cache purging, otherwise if the identifier is a URL, only one page can be purged.

`ESI` Block

(Note ESI feature only works with Varnish and some CDNs.)

ESI stands for "Edge Side Includes". To make a block an ESI block, you need to add ttl attribute to <block /> in layout XML(The ttl must be greater than 0).

Tag Exclusion

Check the \Magento\PageCache\Model\Layout\LayoutPlugin::afterGetOutput method again.

if ($isVarnish && $isEsiBlock) {
    continue;
}

When Varnish (or CDN) is employed, identities of ESI blocks are skipped. To answer why they are skipped, you need to know how ESI block works.

`ESI` Mechanism

When Magento renders a page and meets an ESI block, it will render a placeholder rather the real block content. Then Varnish or CDN fetches the real block content and replace the placeholder with it before generating the final response.

So the tags of ESI blocks should be separated from the main page.

When do we need `ESI` block and what can we benefit from it?

The answer is to prevent purging everything if a common block is changed.

The most typical example is the menu bar, it is almost on every page. If you updated the menu, for example, changed a category's name, only the menu ESI block will be purged.

Another benefit is that the ESI block can have different lifetime than the main page. Usually the ESI block's lifetime is much shorter. For example, the main page's cache lifetime is 1 day and the ESI block's lifetime is only 1 hour.

Code

The code below shows how X-Magento-Tags are added to ESI block and the generation of block content response.

\Magento\PageCache\Controller\Block\Esi

/**
 * Returns block content as part of ESI request from Varnish
 *
 * @return void
 */
public function execute()
{
    $response = $this->getResponse();
    $blocks = $this->_getBlocks();
    $html = '';
    $ttl = 0;

    if (!empty($blocks)) {
        $blockInstance = array_shift($blocks);
        $html = $blockInstance->toHtml();
        $ttl = $blockInstance->getTtl();
        if ($blockInstance instanceof \Magento\Framework\DataObject\IdentityInterface) {
            $response->setHeader('X-Magento-Tags', implode(',', $blockInstance->getIdentities()));
        }
    }
    $this->translateInline->processResponseBody($html);
    $response->appendBody($html);
    $response->setPublicHeaders($ttl);
}

Closing Thoughts

We hope this blog explained the "How It Works" clearly regarding the X-Magento-Tags.

When designing a block, you should consider:

Does the block need identities?
Does the block need ESI mechanism?

Still have questions? Contact us.

If you found this blog post helpful, please share it!