Feature Implementation Plan: Refactor List Endpoints for Caching with DTOs¶

📋 Todo Checklist¶

[ ] Create DTOs for entities returned in paginated lists (e.g., VarietyDTO, ProcessingMethodDTO).
[ ] Create a generic EntityToDtoMapper service to convert Doctrine entities to their DTO counterparts.
[ ] Refactor the repository methods for list endpoints (e.g., VarietyRepository::findByRequest) to return DTOs instead of entities.
[ ] Update the caching logic in these methods to store the paginated DTO results.
[ ] Update the corresponding controller methods to handle the DTOs.
[ ] Write unit tests for the new mapper and integration tests for the updated endpoints.
[ ] Final Review and Testing

🔍 Analysis & Investigation¶

Codebase Structure¶

New DTOs: This plan will introduce a new src/DTO/ directory (if it doesn't already exist for this purpose) to house the API-facing data transfer objects.
New Mapper Service: A new, reusable service will be created at src/Service/Api/Mapper/EntityToDtoMapper.php.
Repositories: The findByRequest (or similar) methods in repositories like VarietyRepository and ProcessingMethodRepository will be the primary focus of the refactoring.
Controllers: The corresponding controller methods will be slightly adjusted to handle the results from the refactored repository methods.

Current Architecture & Problem¶

Problem: The current implementation attempts to cache Doctrine Paginator objects that contain Doctrine entities. These entities are often "proxy objects" with a live database connection, which cannot be safely serialized and cached. This is the root cause of the "pagination caching issue."
Solution: This plan solves the problem by introducing a layer of Data Transfer Objects (DTOs). The new process will be:
1. Fetch the paginated list of Doctrine entities from the database.
2. Convert these entities into simple, serializable DTOs.
3. Cache the paginated result containing the DTOs. This decouples the API representation from the database entity, making the cache safe, reliable, and efficient.

Dependencies & Integration Points¶

Symfony/PropertyAccess: This component is likely already installed and is the perfect tool for the EntityToDtoMapper to automatically and efficiently map data from entities to DTOs without writing tedious manual setters and getters.
Symfony Cache: The existing api.cache pool will be used to store the DTO results.

Considerations & Challenges¶

Boilerplate: The main trade-off is the introduction of some boilerplate code for creating the DTO classes. However, this is a one-time cost that pays significant dividends in architectural cleanliness and reliability.
Nested DTOs: For entities that have relationships (e.g., a Variety has a Species), the DTOs will also need to have nested DTOs (e.g., VarietyDTO will have a ?SpeciesDTO $species property). The mapper service must be able to handle these nested structures.

📝 Implementation Plan¶

Prerequisites¶

No new external dependencies are required.

Step-by-Step Implementation¶

Create the DTOs
- Files to create: src/DTO/Api/VarietyDTO.php, src/DTO/Api/SpeciesDTO.php, src/DTO/Api/ProcessingMethodDTO.php, etc.
- Changes needed: For each entity returned in a list, create a corresponding DTO. These should be simple, readonly classes with public properties that exactly match the fields exposed in the API's JSON response.
```
// Example for VarietyDTO.php
namespace App\DTO\Api;

final readonly class VarietyDTO
{
    public function __construct(
        public string $id,
        public string $name,
        public ?SpeciesDTO $species,
        // ... other public properties
    ) {}
}
```

Create the EntityToDtoMapper Service

Files to create: src/Service/Api/Mapper/EntityToDtoMapper.php

Changes needed: Create a service that handles the conversion. Use the PropertyAccessor for efficient mapping.

// Example method in EntityToDtoMapper.php
public function mapVarietiesToDtos(iterable $varieties): array
{
    $dtos = [];
    foreach ($varieties as $variety) {
        $speciesDto = $variety->getSpecies() ? new SpeciesDTO(...) : null;
        $dtos[] = new VarietyDTO(
            id: $variety->getId(),
            name: $variety->getName(),
            species: $speciesDto,
            // ... map other properties
        );
    }
    return $dtos;
}

Refactor Repository Methods to Use DTOs

Files to modify: VarietyRepository.php, ProcessingMethodRepository.php, etc.
Changes needed: Update the findByRequest methods to perform the mapping and cache the DTOs.

Example Refactoring:

// In VarietyRepository.php
public function findByRequest(Request $request, ...): array // Note: Return type is now array
{
    $cacheKey = $this->cacheKeyGenerator->createKey($request);

    return $this->apiCache->get($cacheKey, function (ItemInterface $item) use ($request, ...) {
        $item->tag(['varieties_list']);

        // 1. Get the paginator of ENTITIES (don't change this part)
        $qb = $this->createQueryBuilder('v');
        $this->filterService->apply($qb, $request);
        $qb->setMaxResults($limit)->setFirstResult($offset);
        $paginator = new Paginator($qb->getQuery());

        // 2. Map the entities to DTOs
        $varietyDtos = $this->entityToDtoMapper->mapVarietiesToDtos($paginator->getIterator());

        // 3. Return a simple, cacheable array
        return [
            'items' => $varietyDtos,
            'totalItems' => count($paginator),
        ];
    });
}

Update Controller Methods

Files to modify: VarietyController.php, ProcessingMethodController.php, etc.

Changes needed: The controllers now receive a simple array instead of a Paginator object. Adjust them to build the final JSON response from this array.

// In VarietyController.php
public function getVarieties(Request $request): JsonResponse
{
    // ... get page and limit ...
    $result = $this->varietyRepository->findByRequest($request, $limit, $offset);

    return $this->json([
        'items' => $result['items'],
        'pagination' => [
            'page' => $page,
            'limit' => $limit,
            'totalItems' => $result['totalItems'],
            'totalPages' => (int) ceil($result['totalItems'] / $limit),
        ],
    ]);
}

Testing Strategy¶

Unit Tests:
- Write comprehensive unit tests for the EntityToDtoMapper service. Provide it with mock entities and assert that the resulting DTOs have the correct structure and data.
Integration Tests:
- Update the integration tests for the list endpoints (e.g., /api/varieties).
- Call the endpoint twice. On the second call, verify that the database was not hit (proving the cache works).
- Assert that the JSON response structure is correct and contains the data from the DTOs.

🎯 Success Criteria¶

The Doctrine proxy serialization errors are completely resolved.
All paginated list endpoints are now successfully and reliably cached.
The API architecture is cleaner, with a clear separation between database entities and API data structures (DTOs).
The overall performance and resilience of the API under load are significantly improved.