Building a High-Performance Data API with Async Django

Why async Django matters now

For years, Django's synchronous architecture was a compromise worth making — the ORM, the admin, the ecosystem. But as Python backends increasingly serve AI agents, real-time data pipelines, and high-concurrency workloads, blocking I/O became a genuine bottleneck.

Django 4.1 introduced async views. Django 6.0 has made async a first-class citizen. We built an entire production data API on this foundation — and the results were worth the migration overhead.

Architecture overview

The API serves structured business data to both human-facing dashboards and AI agents. The requirements were demanding: low latency at scale, full OpenAPI documentation, and a Model Context Protocol (MCP) server so AI agents could discover and query endpoints autonomously.

The stack:

• Django 6.0 with async views and ORM queries
• Django REST Framework with custom serialisers and viewsets
• PostgreSQL with carefully indexed query patterns
• Redis for response caching on expensive aggregation queries
• AWS Lambda for background processing with SQS triggers
• An MCP server exposing API tools and data schemas to AI agents

Async views: what actually changed

Moving to async views isn't just adding async def. The gains come from non-blocking database calls, concurrent external API requests within a single request lifecycle, and the ability to serve more requests per worker without thread contention.

The pattern we settled on for database access uses Django's async ORM consistently:

async def get_company_data(request, company_id):
    company = await Company.objects.aget(pk=company_id)
    financials = await sync_to_async(
        list
    )(Financial.objects.filter(company=company).select_related('period'))
    return Response(CompanySerializer(company).data)

For bulk operations we use sync_to_async with explicit queryset evaluation — async Django's ORM support is excellent for single-object lookups but still benefits from this pattern for complex querysets.

The MCP server layer

Model Context Protocol is a standard for exposing tools and data sources to AI agents. We built an MCP server that sits alongside the REST API and exposes the same data through a tool-discovery interface — meaning AI agents can query the data platform without hardcoded API knowledge.

The MCP server defines tools with JSON schemas describing parameters and return types, and maps them to the same service layer used by the REST API. This eliminated duplication and ensured the agent-facing interface stayed in sync with the human-facing one.

Migrating 1,000 tests to PyTest

The legacy test suite used Django's built-in TestCase. It worked, but it was slow — 8 minutes for 1,000 tests on every CI run. We migrated to PyTest with pytest-django and made several targeted improvements:

• Replaced setUp/tearDown with pytest.fixture and scope="module" where appropriate
• Introduced a persistent test database that rolls back at the transaction level rather than recreating between tests
• Added CircleCI dependency caching for the Python virtual environment
• Parallelised tests with pytest-xdist across available CI workers

Result: 2 minutes 40 seconds for the same test suite. A 67% reduction in CI time, compounded across dozens of deploys per week.

Documentation as a first-class concern

We generated full OpenAPI 3.0 documentation from DRF serialisers using drf-spectacular. Every endpoint has example request/response pairs, authentication requirements, and error schemas. The documentation is served as an interactive Swagger UI and consumed directly by the MCP server's tool definitions.

Lessons learned

Async Django is production-ready in 2025, but it rewards careful architecture. The biggest pitfalls are implicit synchronous calls in middleware and third-party packages that block the event loop. Profile first, convert incrementally, and test under realistic concurrency before assuming the gains are uniform across your views.

The MCP server pattern is worth the investment. As AI agents become first-class API consumers, designing for machine-readability from the start — clear schemas, consistent naming, discoverable endpoints — pays dividends that extend beyond the AI use case.