Architecture Documentation with arc42

This guide demonstrates how to document software architecture using the arc42 template combined with Sphinx-Needs directives. This approach enables systematic architecture documentation with full traceability between architecture decisions, components, and requirements.

What is arc42?

arc42 is a proven, practical, and pragmatic template for software architecture documentation and communication. It provides a standardized structure with 12 sections covering all aspects of software architecture.

Why Combine arc42 with Sphinx-Needs?

By integrating arc42 with Sphinx-Needs, you gain:

  • Traceability: Link architecture elements to requirements, design decisions, and test cases

  • Filtering: Query and display specific architecture elements based on criteria

  • Consistency: Ensure all architecture components are properly documented

  • Navigation: Automatic cross-references between related architecture elements

  • Reporting: Generate architecture overviews, dependency matrices, and more

arc42 Structure Overview

The arc42 template consists of 12 sections:

  1. Introduction and Goals

  2. Constraints

  3. Context and Scope

  4. Solution Strategy

  5. Building Block View

  6. Runtime View

  7. Deployment View

  8. Crosscutting Concepts

  9. Architecture Decisions

  10. Quality Requirements

  11. Risks and Technical Debt

  12. Glossary

Custom Sphinx-Needs Types for arc42

Define custom need types in your conf.py to represent arc42 elements:

needs_types = [
    # ... existing types
    {
        "directive": "arch-goal",
        "title": "Architecture Goal",
        "prefix": "AG_",
        "color": "#BFD8D2",
        "style": "node"
    },
    {
        "directive": "constraint",
        "title": "Constraint",
        "prefix": "CON_",
        "color": "#FEDCD2",
        "style": "node"
    },
    {
        "directive": "building-block",
        "title": "Building Block",
        "prefix": "BB_",
        "color": "#DF744A",
        "style": "node"
    },
    {
        "directive": "arch-decision",
        "title": "Architecture Decision",
        "prefix": "ADR_",
        "color": "#DCB239",
        "style": "node"
    },
    {
        "directive": "quality-goal",
        "title": "Quality Goal",
        "prefix": "QG_",
        "color": "#BFD8D2",
        "style": "node"
    },
]

Section 1: Introduction and Goals

Quality Goals

Quality Goal: High Performance QG_001
status: open
tags: performance, quality

The system must handle 10,000 concurrent users with response times under 200ms for 95% of requests.

Quality Goal: Maintainability QG_002
status: open
tags: maintainability, quality
links incoming: ARCH_CTX_001, ADR_001, ARCH_QA_001

Code must follow clean architecture principles to ensure long-term maintainability and testability.

Quality Goal: Security QG_003
status: open
tags: security, quality

All data transmission must be encrypted, and authentication must use industry-standard protocols (OAuth 2.0, JWT).

Stakeholders

Specification: Product Owner STAKE_001
tags: stakeholder

Expectations: Feature completeness, time-to-market, ROI

Influence: High - Sets priorities and acceptance criteria

Specification: Development Team STAKE_002
tags: stakeholder

Expectations: Clear requirements, technical feasibility, maintainable codebase

Influence: High - Implements and maintains the system

Section 2: Constraints

Technical Constraints

Constraint: Programming Language: Python CON_001
status: accepted
tags: technical, language
links incoming: BB_004

The system must be implemented in Python 3.12+ to leverage existing team expertise and tooling.

Constraint: Cloud Platform: AWS CON_002
status: accepted
tags: technical, infrastructure
links incoming: ARCH_DV_001

Deployment must use AWS services due to existing enterprise agreement and compliance requirements.

Organizational Constraints

Constraint: Team Size CON_003
status: accepted
tags: organizational
links incoming: ADR_001

Development team is limited to 5 developers, requiring architecture that minimizes coordination overhead.

Constraint: Budget Limitation CON_004
status: accepted
tags: organizational, budget

Infrastructure costs must not exceed $5,000/month in production.

Section 3: Context and Scope

Business Context

Architecture: System Context ARCH_CTX_001
links outgoing: QG_001, QG_002, QG_003
links incoming: BB_001

External interfaces and system boundaries:

@startuml
!define RECTANGLE class

actor User
actor Admin

rectangle "EAC System" {
    component [API Gateway] as API
    component [Documentation Engine] as DocEngine
    component [Build System] as Build
}

database "Document Store" as DB
cloud "External Systems" {
    component [Version Control] as VCS
    component [CI/CD] as CICD
}

User --> API : Browse/Search Docs
Admin --> API : Manage Content
API --> DocEngine : Process Requests
DocEngine --> Build : Generate Docs
Build --> DB : Store Artifacts
DocEngine <--> VCS : Fetch Sources
Build <--> CICD : Trigger Builds
@enduml

Technical Context

Architecture: Technical Interfaces ARCH_CTX_002

Integration points with external systems:

  • REST API: JSON over HTTPS, OAuth 2.0 authentication

  • Git Integration: SSH/HTTPS for repository access

  • Database: PostgreSQL 14+ with connection pooling

  • Message Queue: Redis for background job processing

Section 4: Solution Strategy

Architecture Decision: Use Microservices Architecture ADR_001
status: accepted
links outgoing: QG_001, QG_002, CON_003
links incoming: ADR_002, BB_002, BB_003

Context: Need to scale different components independently while maintaining team autonomy.

Decision: Adopt microservices architecture with domain-driven design.

Consequences:

  • ✅ Independent scaling and deployment

  • ✅ Technology diversity where appropriate

  • ❌ Increased operational complexity

  • ❌ Need for service mesh and API gateway

Architecture Decision: Event-Driven Communication ADR_002
status: accepted
links outgoing: ADR_001, QG_001

Context: Microservices need to communicate without tight coupling.

Decision: Use event-driven architecture with Redis Streams for inter-service communication.

Consequences:

  • ✅ Loose coupling between services

  • ✅ Better fault tolerance

  • ❌ Eventual consistency challenges

  • ❌ More complex debugging

Section 5: Building Block View

Level 1: System Overview

Building Block: EAC Documentation System BB_001
links outgoing: ARCH_CTX_001
links incoming: BB_002, BB_003, BB_004

Top-level system components:

  • API Gateway: Entry point for all external requests

  • Documentation Service: Core documentation processing

  • Build Service: Sphinx documentation builds

  • Storage Service: Document artifact management

  • Search Service: Full-text search capabilities

Level 2: Component Details

Building Block: API Gateway BB_002
links outgoing: BB_001, ADR_001
links incoming: ARCH_BB_001, ARCH_DV_001

Responsibilities:

  • Request routing and load balancing

  • Authentication and authorization

  • Rate limiting and throttling

  • Request/response transformation

Technologies:

  • Kong API Gateway

  • OAuth 2.0 / JWT

  • Redis for rate limiting

Building Block: Documentation Service BB_003
links outgoing: BB_001, ADR_001

Responsibilities:

  • Parse and validate documentation sources

  • Manage documentation projects

  • Coordinate build triggers

  • Handle version management

Technologies:

  • Python/FastAPI

  • PostgreSQL for metadata

  • GitPython for repository access

Building Block: Build Service BB_004
links outgoing: BB_001, BB_003, CON_001

Responsibilities:

  • Execute Sphinx builds in isolated environments

  • Generate multiple output formats (HTML, PDF)

  • Handle build artifacts

  • Report build status

Technologies:

  • Python/Celery for task queue

  • Docker for build isolation

  • Sphinx + sphinx-needs

Component Relationships

Architecture: Component Interaction ARCH_BB_001
links outgoing: BB_002, BB_003, BB_004
        graph TD
   User[User/Client] -->|HTTPS| AG[API Gateway]
   AG -->|REST| DS[Documentation Service]
   AG -->|REST| SS[Search Service]
   DS -->|Events| BS[Build Service]
   DS -->|Read/Write| DB[(PostgreSQL)]
   BS -->|Store| S3[S3 Storage]
   BS -->|Index| SS
   BS -->|Clone| Git[Git Repository]

   style AG fill:#BFD8D2
   style DS fill:#DF744A
   style BS fill:#DF744A
   style SS fill:#DF744A
    

Section 6: Runtime View

Architecture: Documentation Build Flow ARCH_RV_001
links outgoing: BB_003, BB_004

Sequence of events when a documentation build is triggered:

@startuml
actor User
participant "API Gateway" as AG
participant "Doc Service" as DS
participant "Build Service" as BS
participant "Git Repo" as Git
participant "Storage" as S3

User -> AG: POST /projects/{id}/build
AG -> DS: Trigger Build
DS -> DS: Validate Request
DS -> BS: Queue Build Job
DS --> User: 202 Accepted (Job ID)

BS -> Git: Clone Repository
Git --> BS: Source Files
BS -> BS: Run Sphinx Build
BS -> S3: Upload Artifacts
BS -> DS: Update Build Status
DS --> User: Notification (Success)
@enduml

Architecture: Search Query Flow ARCH_RV_002

How search requests are processed:

        sequenceDiagram
   participant U as User
   participant AG as API Gateway
   participant SS as Search Service
   participant ES as Elasticsearch
   participant Cache as Redis Cache

   U->>AG: GET /search?q=architecture
   AG->>Cache: Check Cache
   alt Cache Hit
      Cache-->>AG: Cached Results
      AG-->>U: Return Results (Fast)
   else Cache Miss
      AG->>SS: Forward Query
      SS->>ES: Execute Search
      ES-->>SS: Results
      SS->>Cache: Store in Cache
      SS-->>AG: Results
      AG-->>U: Return Results
   end
    

Section 7: Deployment View

Architecture: Production Deployment ARCH_DV_001
links outgoing: CON_002, BB_002, BB_003, BB_004

AWS infrastructure layout:

@startuml
!include <C4/C4_Context>

skinparam rectangle {
   BackgroundColor<<cdn>> LightBlue
   BackgroundColor<<lb>> LightGreen
   BackgroundColor<<app>> LightYellow
   BackgroundColor<<db>> Pink
   BackgroundColor<<storage>> Orange
}

package "AWS Cloud" {
   rectangle "CloudFront\nCDN" as cdn <<cdn>>

   package "VPC" {
      package "Public Subnet" {
         rectangle "Application LB\nLoad Balancer" as alb <<lb>>
      }

      package "Private Subnet" {
         rectangle "API Gateway\nt3.medium" as app1 <<app>>
         rectangle "Doc Service\nt3.medium" as app2 <<app>>
         rectangle "Build Workers\nt3.large" as worker <<app>>
         database "PostgreSQL\ndb.t3.medium" as db <<db>>
      }
   }

   storage "S3 Bucket\nArtifacts" as s3 <<storage>>
}

cdn --> alb
alb --> app1
alb --> app2
app2 --> db
app2 --> worker
worker --> s3
@enduml

Section 8: Crosscutting Concepts

Logging and Monitoring

Specification: Structured Logging CROSS_001
tags: logging, monitoring

All services use structured JSON logging with:

  • Correlation IDs for request tracing

  • Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL

  • Centralized logging via AWS CloudWatch

Security Concepts

Specification: Authentication and Authorization CROSS_002
tags: security
links outgoing: QG_003
  • OAuth 2.0 for user authentication

  • JWT tokens for session management

  • Role-based access control (RBAC)

  • API key authentication for service-to-service

Error Handling

Specification: Error Handling Strategy CROSS_003
tags: error-handling

Consistent error responses across all services:

{
   "error": {
      "code": "BUILD_FAILED",
      "message": "Documentation build failed",
      "details": "Sphinx error: ...",
      "timestamp": "2025-11-12T10:30:00Z",
      "trace_id": "abc123"
   }
}

Section 9: Architecture Decisions

Decision Records

All major architecture decisions are documented using the ADR format shown above. Key decisions include:

ID

Title

Status

Links

ADR_001

Use Microservices Architecture

accepted

ADR_002

Event-Driven Communication

accepted

Section 10: Quality Requirements

Quality Scenarios

Specification: Performance Scenario QS_001
links outgoing: QG_001

Scenario: 1000 users simultaneously request documentation pages

Expected Response:

  • 95% of requests complete within 200ms

  • No failed requests

  • CPU utilization stays below 70%

Specification: Availability Scenario QS_002

Scenario: Single service instance fails

Expected Response:

  • Automatic failover within 30 seconds

  • No data loss

  • Service degradation < 5%

Quality Tree

Architecture: Quality Attribute Tree ARCH_QA_001
links outgoing: QG_001, QG_002, QG_003
        graph TD
   Q[Quality Goals] --> P[Performance]
   Q --> M[Maintainability]
   Q --> S[Security]

   P --> P1[Response Time < 200ms]
   P --> P2[Throughput > 10k req/s]

   M --> M1[Test Coverage > 80%]
   M --> M2[Clear Architecture]

   S --> S1[Encryption at Rest]
   S --> S2[Secure Authentication]
    

Section 11: Risks and Technical Debt

Specification: Risk: Scaling Challenges RISK_001
status: open
tags: risk, scaling

Risk: Build service may not scale to handle peak loads

Probability: Medium

Impact: High - Could cause build delays

Mitigation:

  • Implement auto-scaling for build workers

  • Add build queue prioritization

  • Monitor build times and queue lengths

Specification: Technical Debt: Legacy API Endpoints DEBT_001
status: open
tags: technical-debt

Description: Some API endpoints use deprecated patterns

Impact: Inconsistent API design, harder to maintain

Planned Resolution: Q2 2026 - API v2 migration

Section 12: Glossary

Specification: Documentation Artifact GLOSS_001
tags: glossary

A compiled output from the documentation build process, typically HTML, PDF, or other formats.

Specification: Build Job GLOSS_002
tags: glossary

An asynchronous task that processes documentation sources and generates artifacts.

Specification: Sphinx-Needs Directive GLOSS_003
tags: glossary

A reStructuredText directive that creates traceable documentation elements (requirements, specs, etc.).

Traceability Views

Architecture to Requirements

Show which architecture components fulfill which quality goals:

ID

Title

Links

ADR_001

Use Microservices Architecture

ADR_002

Event-Driven Communication

ARCH_BB_001

Component Interaction

BB_001

EAC Documentation System

BB_002

API Gateway

BB_003

Documentation Service

BB_004

Build Service

Complete Architecture Overview

Visualize all architecture elements and their relationships:

No needs passed the filters

Summary

This approach combines the proven arc42 template with Sphinx-Needs’ traceability features to create:

Structured Documentation: Follow industry-standard architecture documentation template

Full Traceability: Link architecture decisions to requirements, components, and quality goals

Living Documentation: Keep architecture docs synchronized with implementation

Automated Reporting: Generate views, matrices, and reports from architecture data

Version Controlled: All architecture documentation in Git alongside code

Best Practices

  1. Start Simple: Don’t fill all 12 sections at once. Focus on what’s important for your project.

  2. Keep It Updated: Review and update architecture docs with each sprint/iteration.

  3. Use IDs Consistently: Follow ID prefix conventions

  4. Link Liberally: Connect related architecture elements using the :links: option.

  5. Visualize: Use diagrams (PlantUML, Mermaid, Draw.io) to illustrate concepts.

  6. Review Together: Make architecture reviews part of your regular process.

Additional Resources