Frontend Revalidation Batching

Problem

The frontend revalidation endpoint has a rate limit of 100 calls per minute. When multiple entities are updated (e.g., batch imports, crawl processing), each entity change triggers a separate revalidation call, causing 429 errors.

Evidence: Sentry issue BEANS-BACKEND-5Q shows 918 occurrences of HTTP 429 errors from the frontend.

Solution

Batch revalidation tags and send one request per "operation window" instead of one request per entity change.

Architecture

Current Flow

Entity Update → CacheInvalidationSubscriber → FrontendRevalidationService::revalidate() → HTTP POST
Entity Update → CacheInvalidationSubscriber → FrontendRevalidationService::revalidate() → HTTP POST
Entity Update → CacheInvalidationSubscriber → FrontendRevalidationService::revalidate() → HTTP POST
(3 HTTP requests)

New Flow

Entity Update → CacheInvalidationSubscriber → FrontendRevalidationService::queueTags()
Entity Update → CacheInvalidationSubscriber → FrontendRevalidationService::queueTags()
Entity Update → CacheInvalidationSubscriber → FrontendRevalidationService::queueTags()
End of Request/Message → FrontendRevalidationService::flush() → HTTP POST (1 request with merged tags)

Implementation Steps

Step 1: Modify FrontendRevalidationService

Add tag collection instead of immediate dispatch:

final class FrontendRevalidationService
{
    /** @var string[] */
    private array $pendingTags = [];

    // Rename current revalidate() to queueTags() - collects without sending
    public function queueTags(array $backendTags): void
    {
        $frontendTags = $this->mapToFrontendTags($backendTags);
        $this->pendingTags = array_unique(array_merge($this->pendingTags, $frontendTags));
    }

    // New method to send collected tags
    public function flush(): void
    {
        if ($this->pendingTags === []) {
            return;
        }

        $tagsToSend = $this->pendingTags;
        $this->pendingTags = []; // Clear before sending to avoid duplicates on retry

        $this->sendRevalidationRequest($tagsToSend);
    }

    // Extract current HTTP logic to private method
    private function sendRevalidationRequest(array $frontendTags): void
    {
        // Current HTTP POST logic with Sentry error handling
    }
}

Step 2: Update CacheInvalidationSubscriber

Change from revalidate() to queueTags():

// In CacheInvalidationSubscriber
$this->frontendRevalidation->queueTags($tagsToInvalidate);
// Instead of: $this->frontendRevalidation->revalidate($tagsToInvalidate);

Step 3: Create FrontendRevalidationFlushListener

Flush pending tags at end of HTTP requests:

#[AsEventListener(event: KernelEvents::TERMINATE, priority: -100)]
final readonly class FrontendRevalidationFlushListener
{
    public function __construct(
        private FrontendRevalidationService $revalidationService,
    ) {}

    public function onKernelTerminate(): void
    {
        $this->revalidationService->flush();
    }
}

Step 4: Handle Messenger Workers

For async message processing, flush after each message. Options:

Option A: Messenger middleware (preferred)

final class FrontendRevalidationMiddleware implements MiddlewareInterface
{
    public function __construct(
        private FrontendRevalidationService $revalidationService,
    ) {}

    public function handle(Envelope $envelope, StackInterface $stack): Envelope
    {
        try {
            return $stack->next()->handle($envelope, $stack);
        } finally {
            $this->revalidationService->flush();
        }
    }
}

Register in config/packages/messenger.php:

'buses' => [
    'messenger.bus.default' => [
        'middleware' => [
            FrontendRevalidationMiddleware::class,
        ],
    ],
],

Option B: Event listener on WorkerMessageHandledEvent

#[AsEventListener(event: WorkerMessageHandledEvent::class)]
public function onMessageHandled(): void
{
    $this->revalidationService->flush();
}

Step 5: Optional - Time-based batching for workers

For high-throughput scenarios, add time-based windowing:

private ?float $windowStart = null;
private const BATCH_WINDOW_SECONDS = 1.0;

public function queueTags(array $backendTags): void
{
    if ($this->windowStart === null) {
        $this->windowStart = microtime(true);
    }

    $frontendTags = $this->mapToFrontendTags($backendTags);
    $this->pendingTags = array_unique(array_merge($this->pendingTags, $frontendTags));

    // Auto-flush if window exceeded
    if ((microtime(true) - $this->windowStart) >= self::BATCH_WINDOW_SECONDS) {
        $this->flush();
    }
}

public function flush(): void
{
    $this->windowStart = null;
    // ... rest of flush logic
}

Files to Modify

1. src/Service/Api/FrontendRevalidationService.php - Add batching logic
2. src/EventSubscriber/CacheInvalidationSubscriber.php - Change to queueTags()
3. src/EventListener/FrontendRevalidationFlushListener.php - New file for HTTP request flush
4. config/packages/messenger.php - Add middleware (if using Option A)
5. src/Messenger/Middleware/FrontendRevalidationMiddleware.php - New file for Messenger flush

Testing

1. Unit test: FrontendRevalidationService collects and deduplicates tags
2. Unit test: flush() sends one request with all collected tags
3. Integration test: Multiple entity updates result in single revalidation call
4. Manual test: Verify no more 429 errors in Sentry after deployment

Rollback

If issues arise, revert to immediate dispatch by: 1. Changing queueTags() back to revalidate() with immediate HTTP call 2. Removing the flush listener and middleware