# HTTP Sender Plugin (HSP)
## Project Implementation Plan

**Project**: HTTP Sender Plugin (HSP) for Diagnostic Data Collection
**Version**: 1.0
**Status**: Ready for Implementation
**Last Updated**: 2025-11-19

---

## Executive Summary

### Project Overview

The HTTP Sender Plugin (HSP) is a Java-based diagnostic data collection system that polls HTTP endpoints and transmits collected data via gRPC to the Collector Sender Core. The system implements hexagonal architecture pattern with comprehensive requirement traceability using **Test-Driven Development (TDD)** methodology.

**Key Metrics**:
- **Total Requirements**: 62 unique requirements (100% traced)
- **Architecture Score**: 7.0/10 (after decisions)
- **Development Approach**: **Test-Driven Development (TDD)**
- **Test Coverage Target**: 95% line, 90% branch
- **Estimated Duration**: 10-12 weeks
- **Team Size**: 4-6 developers
- **CI/CD Platform**: Gitea with Gitea Actions / Drone CI

### Project Status

| Phase | Status | Completion |
|-------|--------|------------|
| Requirements Analysis | ✅ Complete | 100% |
| Architecture Design | ✅ Complete | 100% |
| Architecture Review | ✅ Complete | 100% |
| Decision Record | ✅ Complete | 100% |
| **Implementation** | 🎯 **READY TO START** | 0% |
| Testing | ⏳ Pending | 0% |
| Deployment | ⏳ Pending | 0% |

---

## 📚 Critical Document Index

### Must-Read Documents (In Order)

| Priority | Document | Purpose | Audience |
|----------|----------|---------|----------|
| **1** | [DataCollector SRS.md](../requirements/DataCollector%20SRS.md) | Source requirements (62 reqs) | All team members |
| **2** | [ARCHITECTURE_DECISIONS.md](ARCHITECTURE_DECISIONS.md) | Decisions on review findings | All team members |
| **3** | [architecture/system-architecture.md](architecture/system-architecture.md) | Complete system design | Developers, Architects |
| **4** | [architecture/java-package-structure.md](architecture/java-package-structure.md) | Implementation blueprint | Developers |
| **5** | [testing/test-strategy.md](testing/test-strategy.md) | Testing approach | Developers, QA |

### Reference Documents

| Document | Purpose | When to Use |
|----------|---------|-------------|
| [ARCHITECTURE_REVIEW_REPORT.md](ARCHITECTURE_REVIEW_REPORT.md) | Independent architectural review | Understanding design rationale |
| [REQUIREMENT_REFINEMENT_VERIFICATION.md](REQUIREMENT_REFINEMENT_VERIFICATION.md) | Requirement traceability verification | Validating requirement coverage |
| [traceability/requirements-traceability-matrix.md](traceability/requirements-traceability-matrix.md) | Req → Code → Test mapping | Finding what implements what |
| [testing/test-requirement-mapping.md](testing/test-requirement-mapping.md) | Test coverage matrix | Writing tests |
| [architecture/component-mapping.md](architecture/component-mapping.md) | Component details | Implementing components |
| [diagrams/architecture-diagrams.md](diagrams/architecture-diagrams.md) | Visual architecture | Understanding system flow |

### Interface Specifications

| Document | Interface | Purpose |
|----------|-----------|---------|
| [IF_1_HSP_-_End_Point_Device.md](../requirements/IF_1_HSP_-_End_Point_Device.md) | HTTP polling | Endpoint data collection |
| [IF_2_HSP_-_Collector_Sender_Core.md](../requirements/IF_2_HSP_-_Collector_Sender_Core.md) | gRPC transmission | Data transmission protocol |
| [IF_3_HTTP_Health_check.md](../requirements/IF_3_HTTP_Health_check.md) | Health monitoring | Status endpoint specification |
| [HSP_Configuration_File_Specification.md](../requirements/HSP_Configuration_File_Specification.md) | Configuration | Config file schema |

---

## 🎯 Implementation Phases

### Phase 0: Project Setup (Week 1) ✅ COMPLETE
**Status**: ✅ Complete (Design & Planning)

**Deliverables**:
- ✅ Requirements analysis (62 requirements)
- ✅ Architecture design (Hexagonal pattern)
- ✅ Architecture review and decisions
- ✅ Complete documentation

---

### Phase 1: Foundation & Quick Wins (Weeks 1-2)

**Objective**: Implement approved enhancements and establish project foundation

**Duration**: 2 weeks
**Team**: 3-4 developers
**Effort**: 15-18 person-days

#### Week 1: Enhancements Implementation

**1.1 Rate Limiting Implementation** (1 day)
- **Owner**: Backend Developer
- **Files**:
  - `src/main/java/com/siemens/coreshield/hsp/adapter/outbound/http/RateLimitedHttpPollingAdapter.java`
  - `src/test/java/com/siemens/coreshield/hsp/adapter/outbound/http/RateLimitedHttpPollingAdapterTest.java`
- **Requirements**: Req-FR-16 (enhanced)
- **Dependencies**: None
- **Deliverables**:
  - Rate limiter decorator for HttpPollingPort
  - Configurable requests-per-second limit
  - Unit tests with 95% coverage
  - Configuration schema update

**1.2 Backpressure Controller** (2 days)
- **Owner**: Backend Developer
- **Files**:
  - `src/main/java/com/siemens/coreshield/hsp/application/BackpressureController.java`
  - `src/main/java/com/siemens/coreshield/hsp/application/BackpressureAwareCollectionService.java`
  - Tests
- **Requirements**: Req-FR-26, Req-FR-27 (enhanced)
- **Dependencies**: BufferManager interface
- **Deliverables**:
  - Buffer usage monitoring (100ms intervals)
  - Backpressure signal (80% threshold)
  - HTTP polling skip logic
  - Unit and integration tests

**1.3 Test Coverage Enhancement** (3-5 days)
- **Owner**: QA Engineer + Developers
- **Scope**: Raise coverage from 85%/80% to 95%/90%
- **Requirements**: Req-Norm-2 (EN 50716 compliance)
- **Tasks**:
  - Analyze coverage gaps with JaCoCo
  - Write missing unit tests
  - Add MC/DC tests for critical paths
  - Configure PIT mutation testing
  - Update CI/CD with new thresholds
- **Deliverables**:
  - 95% line coverage, 90% branch coverage
  - MC/DC coverage for safety-critical components
  - Updated Maven POM configuration
  - Coverage reports in CI/CD

#### Week 2: Project Foundation

**1.4 Maven Project Setup** (1 day)
- **Owner**: Build Engineer
- **Files**:
  - `pom.xml`
  - `.gitignore`
  - `README.md`
- **Tasks**:
  - Create multi-module Maven structure
  - Configure JaCoCo with 95%/90% thresholds
  - Configure JUnit 5, Mockito, WireMock
  - Configure gRPC and Protocol Buffers
  - Set up fat JAR packaging
- **Deliverables**:
  - Buildable Maven project
  - All dependencies configured
  - CI/CD integration ready

**1.5 Port Interfaces** (2 days)
- **Owner**: Architect + Senior Developer
- **Files** (8 interfaces):
  - Primary Ports: `IConfigurationPort`, `IHealthCheckPort`, `ILifecyclePort`
  - Secondary Ports: `IHttpPollingPort`, `IGrpcStreamPort`, `ILoggingPort`, `IBufferPort`
- **Requirements**: All port-related requirements
- **Deliverables**:
  - 8 complete port interfaces with Javadoc
  - Interface method signatures
  - Exception definitions
  - 100% requirement traceability annotations

**1.6 Domain Models** (2 days)
- **Owner**: Domain Expert + Developer
- **Files**:
  - `DiagnosticData.java` (value object)
  - `Configuration.java` (value object)
  - `HealthCheckResponse.java` (value object)
  - `BufferStatistics.java` (value object)
- **Requirements**: Req-FR-22, FR-23, FR-24, NFR-7, NFR-8
- **Deliverables**:
  - Immutable value objects
  - JSON serialization support
  - Base64 encoding (DiagnosticData)
  - 100% test coverage

**Phase 1 Success Criteria**:
- ✅ All enhancements implemented and tested
- ✅ Maven project builds successfully
- ✅ All port interfaces defined
- ✅ All domain models implemented
- ✅ Test coverage at 95%/90%
- ✅ Zero compilation errors

---

### Phase 2: Core Services (Weeks 3-4)

**Objective**: Implement business logic and orchestration

**Duration**: 2 weeks
**Team**: 4 developers
**Effort**: 20 person-days

#### Week 3: Configuration & Buffer

**2.1 ConfigurationManager** (2 days)
- **Owner**: Senior Developer
- **Files**:
  - `ConfigurationManager.java`
  - `ConfigurationValidator.java`
  - Tests
- **Requirements**: Req-FR-9 to FR-13
- **Deliverables**:
  - Load configuration from file
  - Validate all parameters
  - Terminate with exit code 1 on failure
  - Log validation errors
  - 95% test coverage

**2.2 BufferManager** (2 days)
- **Owner**: Concurrency Expert
- **Files**:
  - `BufferManager.java`
  - `BufferStatistics.java`
  - Tests (including stress tests)
- **Requirements**: Req-FR-26, FR-27, Req-Arch-7, Arch-8
- **Deliverables**:
  - Thread-safe circular buffer (ArrayBlockingQueue)
  - FIFO overflow handling (discard oldest)
  - Atomic statistics tracking
  - Concurrent stress tests (1000 producers/consumers)
  - Performance benchmarks

**2.3 CollectionStatistics** (1 day)
- **Owner**: Developer
- **Files**:
  - `CollectionStatistics.java`
  - Tests
- **Requirements**: Req-NFR-8
- **Deliverables**:
  - Atomic counters (totalPolls, totalErrors)
  - Time-windowed queue (30s metrics)
  - Thread-safe implementation
  - Unit tests

#### Week 4: Data Services

**2.4 DataCollectionService** (3 days)
- **Owner**: Senior Developer
- **Files**:
  - `DataCollectionService.java`
  - Tests
- **Requirements**: Req-FR-14 to FR-24
- **Dependencies**: IHttpPollingPort, IBufferPort, ILoggingPort
- **Deliverables**:
  - HTTP endpoint polling orchestration
  - Virtual thread pool for concurrent polling
  - Data validation (size limits)
  - JSON serialization with Base64
  - Statistics tracking
  - Unit tests with mocks
  - Integration tests with WireMock

**2.5 DataTransmissionService** (3 days)
- **Owner**: Senior Developer
- **Files**:
  - `DataTransmissionService.java`
  - Tests
- **Requirements**: Req-FR-25, FR-28 to FR-33
- **Dependencies**: IGrpcStreamPort, IBufferPort
- **Deliverables**:
  - Single consumer thread
  - Batch accumulation (4MB or 1s limits)
  - gRPC stream management
  - Reconnection logic (5s retry)
  - receiver_id = 99
  - Unit tests with mock gRPC
  - Integration tests with gRPC test server

**Phase 2 Success Criteria**:
- ✅ All core services implemented
- ✅ Thread safety verified
- ✅ Unit tests at 95% coverage
- ✅ Integration tests passing
- ✅ Performance benchmarks meet requirements

---

### Phase 3: Adapters (Weeks 5-7)

**Objective**: Implement infrastructure adapters

**Duration**: 3 weeks
**Team**: 4 developers
**Effort**: 24 person-days

#### Week 5: Secondary Adapters (Outbound)

**3.1 HttpPollingAdapter** (3 days)
- **Owner**: HTTP Expert
- **Files**:
  - `HttpPollingAdapter.java`
  - `RetryHandler.java`
  - `BackoffStrategy.java`
  - Tests
- **Requirements**: Req-FR-14 to FR-21
- **Deliverables**:
  - Java 11+ HttpClient implementation
  - 30s timeout configuration
  - Retry 3x with 5s intervals
  - Linear backoff (5s → 300s)
  - Per-endpoint semaphore (no concurrent connections)
  - Size validation (1MB limit)
  - Unit tests with mocks
  - Integration tests with WireMock

**3.2 ExponentialBackoffAdapter** (1 day)
- **Owner**: Developer
- **Files**:
  - `ExponentialBackoffAdapter.java` (decorator)
  - `ExponentialBackoffStrategy.java`
  - Tests
- **Requirements**: Req-FR-18 (enhanced)
- **Deliverables**:
  - Exponential backoff with jitter
  - Configurable strategy selection
  - Unit tests
  - Performance comparison tests

**3.3 FileLoggingAdapter** (1 day)
- **Owner**: Developer
- **Files**:
  - `FileLoggingAdapter.java`
  - Tests
- **Requirements**: Req-Arch-3, Arch-4
- **Deliverables**:
  - Java Logger with FileHandler
  - Log to temp directory (hsp.log)
  - Rotation (100MB, 5 files)
  - Thread-safe logging
  - Integration tests

#### Week 6: gRPC & Primary Adapters

**3.4 GrpcStreamAdapter** (3 days)
- **Owner**: gRPC Expert
- **Files**:
  - `GrpcStreamAdapter.java`
  - `TransferService.proto` (Protocol Buffers)
  - Tests
- **Requirements**: Req-FR-28 to FR-33, Req-NFR-4
- **Deliverables**:
  - gRPC client with bidirectional stream
  - Stream lifecycle management
  - Reconnection on failure (5s)
  - Batch serialization
  - Synchronized stream access
  - receiver_id = 99
  - Unit tests with mock server
  - Integration tests with gRPC test server

**3.5 ConfigurationFileAdapter** (1 day)
- **Owner**: Developer
- **Files**:
  - `ConfigurationFileAdapter.java`
  - Tests
- **Requirements**: Req-FR-9, FR-10
- **Deliverables**:
  - JSON file loading (./hsp-config.json)
  - Jackson ObjectMapper configuration
  - Error handling
  - Unit tests

**3.6 HealthCheckController** (2 days)
- **Owner**: Developer
- **Files**:
  - `HealthCheckController.java`
  - HTTP server setup
  - Tests
- **Requirements**: Req-NFR-7, NFR-8
- **Deliverables**:
  - GET /health endpoint (localhost:8080)
  - JSON response with 6 required fields
  - Service status determination
  - Dependency status aggregation
  - HTTP server (embedded Jetty or similar)
  - Integration tests

#### Week 7: Application Entry Point

**3.7 HspApplication (Main)** (3 days)
- **Owner**: Senior Developer
- **Files**:
  - `HspApplication.java`
  - Startup orchestration
  - Tests
- **Requirements**: Req-FR-1 to FR-8, Req-Arch-5
- **Deliverables**:
  - Startup sequence implementation
  - Dependency injection (manual or framework)
  - Component initialization order
  - gRPC retry loop (5s)
  - Wait for gRPC before HTTP polling
  - Health check server startup
  - Integration tests (full startup)

**Phase 3 Success Criteria**:
- ✅ All adapters implemented
- ✅ gRPC communication working
- ✅ HTTP polling working
- ✅ Health check accessible
- ✅ Application starts successfully
- ✅ Integration tests passing

---

### Phase 4: Testing & Validation (Week 8)

**Objective**: Comprehensive testing and validation

**Duration**: 1 week
**Team**: 2 QA + 2 Developers
**Effort**: 16 person-days

#### 4.1 Integration Test Suite (2 days)
- **Owner**: QA Engineer
- **Scope**:
  - HttpCollectionIntegrationTest (WireMock)
  - GrpcTransmissionIntegrationTest (gRPC test server)
  - EndToEndDataFlowTest (IF1 → IF2 complete flow)
  - ConfigurationFileIntegrationTest
  - CircularBufferIntegrationTest
- **Requirements**: All integration test requirements
- **Deliverables**:
  - 20+ integration test scenarios
  - Mock server configurations
  - Test data generators

#### 4.2 Performance Tests (2 days)
- **Owner**: Performance Engineer
- **Tests**:
  - PerformanceConcurrentEndpointsTest (1000 endpoints)
  - PerformanceMemoryUsageTest (< 4096MB)
  - PerformanceVirtualThreadTest
  - PerformanceStartupTimeTest
- **Requirements**: Req-NFR-1, NFR-2, Req-Arch-6
- **Deliverables**:
  - JMH benchmarks
  - Performance baseline measurements
  - Memory profiling reports

#### 4.3 Reliability Tests (1 day)
- **Owner**: QA Engineer
- **Tests**:
  - ReliabilityStartupSequenceTest
  - ReliabilityGrpcRetryTest
  - ReliabilityHttpFailureTest
  - ReliabilityBufferOverflowTest
  - ReliabilityPartialFailureTest
- **Requirements**: Resilience requirements
- **Deliverables**:
  - Failure injection scenarios
  - Recovery validation
  - Error logging verification

#### 4.4 Compliance Tests (1 day)
- **Owner**: Compliance Specialist
- **Tests**:
  - ComplianceErrorDetectionTest
  - ComplianceIso9001Test
  - ComplianceEn50716Test
  - ComplianceAuditLoggingTest
- **Requirements**: Req-Norm-1, Norm-2, Norm-3
- **Deliverables**:
  - Compliance reports
  - Audit trail validation
  - Quality metrics collection

#### 4.5 Coverage Validation (1 day)
- **Owner**: QA Lead
- **Tasks**:
  - Run JaCoCo coverage analysis
  - Verify 95% line, 90% branch coverage
  - Run PIT mutation testing
  - Generate coverage reports
- **Deliverables**:
  - Coverage reports (HTML, XML)
  - Mutation test results
  - Gap analysis (if any)

**Phase 4 Success Criteria**:
- ✅ All test categories executed
- ✅ 95% line coverage, 90% branch coverage achieved
- ✅ 1000 concurrent endpoints supported
- ✅ Memory usage < 4096MB
- ✅ All reliability scenarios pass
- ✅ Compliance requirements met

---

### Phase 5: Integration & Deployment (Weeks 9-10)

**Objective**: End-to-end integration and deployment preparation

**Duration**: 2 weeks
**Team**: 2 Developers + 1 DevOps
**Effort**: 12 person-days

#### Week 9: End-to-End Testing

**5.1 E2E Test Scenarios** (3 days)
- **Owner**: QA Engineer + Developer
- **Scenarios**:
  - E2EStartupAndCollectionTest
  - E2EFailureRecoveryTest
  - E2EPerformanceTest (1000 endpoints, sustained)
  - E2EConfigurationReloadTest (future feature)
- **Deliverables**:
  - Complete system validation
  - Real component interaction (no mocks except external systems)
  - Long-running tests (24+ hours)
  - Chaos engineering scenarios

**5.2 Documentation Finalization** (2 days)
- **Owner**: Tech Writer + Developer
- **Tasks**:
  - User guide (installation, configuration, operation)
  - Operations manual (monitoring, troubleshooting)
  - API documentation (health check endpoint)
  - Javadoc completion
- **Deliverables**:
  - Complete user documentation
  - Operations runbook
  - Javadoc for all public APIs

#### Week 10: Deployment Preparation

**5.3 Packaging & Distribution** (2 days)
- **Owner**: DevOps Engineer
- **Tasks**:
  - Fat JAR with all dependencies
  - Startup scripts (Linux, Windows)
  - Configuration templates
  - Log rotation setup
- **Deliverables**:
  - Deployable artifacts
  - Installation scripts
  - Configuration examples

**5.4 Deployment Guide** (1 day)
- **Owner**: DevOps Engineer
- **Tasks**:
  - System requirements documentation
  - Installation instructions
  - Configuration guide
  - Troubleshooting guide
- **Deliverables**:
  - Complete deployment documentation
  - Quick start guide
  - FAQ

**5.5 Production Validation** (2 days)
- **Owner**: QA + DevOps
- **Tasks**:
  - Deployment in staging environment
  - Smoke tests in staging
  - Performance validation
  - Security scan
- **Deliverables**:
  - Staging deployment successful
  - Production readiness checklist
  - Sign-off documentation

**Phase 5 Success Criteria**:
- ✅ E2E tests passing (24+ hours)
- ✅ Documentation complete
- ✅ Deployable artifacts created
- ✅ Staging deployment successful
- ✅ Production readiness approved

---

## 📊 Project Schedule

### Gantt Chart (High-Level)

```
Week:  1    2    3    4    5    6    7    8    9   10
       |----|----|----|----|----|----|----|----|----|----|

Phase 1: Enhancements & Foundation
       [████████████]

Phase 2: Core Services
                    [████████████]

Phase 3: Adapters
                                [█████████████████████]

Phase 4: Testing
                                                   [████]

Phase 5: Integration & Deployment
                                                       [████████]
```

### Milestones

| Milestone | Week | Deliverable | Criteria |
|-----------|------|-------------|----------|
| **M1: Foundation Complete** | 2 | Enhanced system + Foundation | Maven builds, ports defined, enhancements tested |
| **M2: Core Services Complete** | 4 | Business logic | All services implemented, unit tests pass |
| **M3: Adapters Complete** | 7 | Infrastructure | Application runs end-to-end |
| **M4: Testing Complete** | 8 | Test suite | 95%/90% coverage, all tests pass |
| **M5: Production Ready** | 10 | Deployable system | Staging validated, docs complete |

---

## 👥 Team Structure

### Required Roles

| Role | Count | Responsibilities |
|------|-------|-----------------|
| **Senior Developer** | 2 | Core services, complex components, architecture decisions |
| **Backend Developer** | 2-3 | Adapters, utilities, integration |
| **QA Engineer** | 1-2 | Test strategy, test implementation, validation |
| **DevOps Engineer** | 1 | Build, CI/CD, deployment |
| **Architect** (Part-time) | 0.5 | Architecture guidance, code reviews |
| **Tech Writer** (Part-time) | 0.5 | Documentation |

**Total Team Size**: 4-6 full-time + 2 part-time = **5-7 FTE**

### Team Assignment by Phase

| Phase | Developers | QA | DevOps | Other |
|-------|-----------|-----|--------|-------|
| Phase 1 | 3-4 | 1 | 1 | Architect |
| Phase 2 | 4 | 1 | - | - |
| Phase 3 | 4 | 1 | - | - |
| Phase 4 | 2 | 2 | - | - |
| Phase 5 | 2 | 1 | 1 | Tech Writer |

---

## 🧪 Test-Driven Development (TDD) Approach

### TDD Methodology

**ALL development MUST follow TDD Red-Green-Refactor cycle**:

```
┌─────────────────────────────────────────────────────────┐
│ TDD Cycle (Red-Green-Refactor)                          │
├─────────────────────────────────────────────────────────┤
│                                                          │
│  1. RED: Write a failing test                           │
│     • Write test for next requirement                   │
│     • Test fails (no implementation yet)                │
│     • Commit test to Git                                │
│                                                          │
│  2. GREEN: Make the test pass                           │
│     • Write minimal code to pass test                   │
│     • All tests pass (new + existing)                   │
│     • Commit implementation to Git                      │
│                                                          │
│  3. REFACTOR: Improve the code                          │
│     • Clean up code, remove duplication                 │
│     • All tests still pass                              │
│     • Commit refactored code to Git                     │
│                                                          │
│  4. REPEAT: Next requirement                            │
│     • Move to next test case                            │
│     • Cycle continues                                   │
└─────────────────────────────────────────────────────────┘
```

### TDD Rules (Non-Negotiable)

1. **Write tests FIRST, code SECOND**
   - No production code without a failing test
   - Test defines the interface and behavior
   - Implementation satisfies the test

2. **One Test at a Time**
   - Focus on single requirement/behavior
   - Small, incremental steps
   - Frequent commits (multiple times per day)

3. **All Tests Must Pass**
   - Never commit broken tests
   - CI pipeline enforces this
   - Fix immediately if build breaks

4. **Test Coverage Mandatory**
   - 95% line coverage minimum
   - 90% branch coverage minimum
   - Enforced by CI pipeline

### TDD Workflow Example

**Example: Implementing BufferManager**

```bash
# Step 1: RED - Write failing test
$ git checkout -b feature/buffer-manager
$ # Create BufferManagerTest.java
$ cat > src/test/java/.../BufferManagerTest.java << 'EOF'
@Test
void shouldAddMessageToBuffer_whenSpaceAvailable() {
    // Given
    BufferManager buffer = new BufferManager(300);
    DiagnosticData data = new DiagnosticData("http://test", new byte[]{1,2,3});

    // When
    boolean result = buffer.offer(data);

    // Then
    assertThat(result).isTrue();
    assertThat(buffer.size()).isEqualTo(1);
}
EOF

$ mvn test  # FAILS (BufferManager doesn't exist)
$ git add src/test/java/.../BufferManagerTest.java
$ git commit -m "test: add BufferManager offer() test (RED)"

# Step 2: GREEN - Minimal implementation
$ # Create BufferManager.java
$ cat > src/main/java/.../BufferManager.java << 'EOF'
public class BufferManager {
    private final BlockingQueue<DiagnosticData> buffer;

    public BufferManager(int capacity) {
        this.buffer = new ArrayBlockingQueue<>(capacity);
    }

    public boolean offer(DiagnosticData data) {
        return buffer.offer(data);
    }

    public int size() {
        return buffer.size();
    }
}
EOF

$ mvn test  # PASSES
$ git add src/main/java/.../BufferManager.java
$ git commit -m "feat: implement BufferManager offer() method (GREEN)"

# Step 3: REFACTOR - Improve code (if needed)
$ # Add javadoc, improve naming, etc.
$ git commit -m "refactor: add javadoc to BufferManager"

# Step 4: REPEAT - Next test case
$ # Write test for overflow behavior (Req-FR-27)
$ cat >> src/test/java/.../BufferManagerTest.java << 'EOF'
@Test
void shouldDiscardOldest_whenBufferFull() {
    // Test implementation...
}
EOF
$ # Continue RED-GREEN-REFACTOR cycle...
```

### TDD for Each Component Type

**Port Interfaces** (Test-First Design):
```java
// 1. RED: Write test defining interface contract
@Test
void shouldPollEndpoint_whenUrlProvided() {
    IHttpPollingPort httpPort = new HttpPollingAdapter(config);

    CompletableFuture<byte[]> result = httpPort.pollEndpoint("http://test");

    assertThat(result).isCompletedWithValue(expectedData);
}

// 2. GREEN: Define interface to satisfy test
public interface IHttpPollingPort {
    CompletableFuture<byte[]> pollEndpoint(String url);
}

// 3. GREEN: Minimal adapter implementation
public class HttpPollingAdapter implements IHttpPollingPort {
    @Override
    public CompletableFuture<byte[]> pollEndpoint(String url) {
        // Minimal implementation
    }
}
```

**Domain Services** (Behavior-Driven):
```java
// 1. RED: Test business logic behavior
@Test
void shouldRejectOversizedData_whenFileExceeds1MB() {
    DataCollectionService service = new DataCollectionService(...);
    byte[] largeData = new byte[2_000_000];  // 2 MB

    assertThatThrownBy(() -> service.validateData(largeData, "http://test"))
        .isInstanceOf(OversizedDataException.class);
}

// 2. GREEN: Implement validation
public class DataCollectionService {
    public void validateData(byte[] data, String url) {
        if (data.length > 1_048_576) {  // Req-FR-21: 1MB limit
            throw new OversizedDataException(url, data.length);
        }
    }
}
```

**Adapters** (Integration-Tested):
```java
// 1. RED: Test adapter with real infrastructure (mocked)
@Test
void shouldRetryThreeTimes_whenHttpFails() {
    // Use WireMock to simulate failures
    stubFor(get("/endpoint")
        .willReturn(aResponse().withStatus(500)));

    HttpPollingAdapter adapter = new HttpPollingAdapter(config);

    assertThatThrownBy(() -> adapter.pollEndpoint(url).join())
        .hasCauseInstanceOf(PollingFailedException.class);

    // Verify 3 retry attempts (Req-FR-17)
    verify(3, getRequestedFor(urlEqualTo("/endpoint")));
}

// 2. GREEN: Implement retry logic
```

### TDD Daily Workflow

**Morning** (Start of Day):
1. Pull latest from main/develop
2. Review test coverage report
3. Pick next user story from sprint backlog
4. Create feature branch

**During Development**:
1. Write test (RED)
2. Commit: `git commit -m "test: description (RED)"`
3. Write code (GREEN)
4. Commit: `git commit -m "feat: description (GREEN)"`
5. Refactor (REFACTOR)
6. Commit: `git commit -m "refactor: description"`
7. Repeat 5-10 times per day

**End of Day**:
1. Push feature branch: `git push origin feature/name`
2. Create pull request in Gitea
3. CI pipeline runs (all tests must pass)
4. Request code review

**Code Review** (TDD-Focused):
- ✅ Tests exist for all code paths
- ✅ Tests were committed BEFORE implementation
- ✅ Tests follow AAA pattern (Arrange-Act-Assert)
- ✅ Coverage threshold met (95%/90%)
- ✅ All tests passing in CI

### TDD Metrics & Monitoring

**Track in Gitea/CI**:
- Test-to-code commit ratio (should be ~1:1)
- Test coverage trend (should increase or stay at 95%+)
- Test execution time (should be < 5 minutes for unit tests)
- Flaky test rate (should be < 1%)

**Sprint Retrospective Questions**:
- Did we follow TDD for all code?
- Where did we skip tests first?
- What slowed down TDD workflow?
- How can we improve TDD practices?

---

## 🎯 Sprint Planning Guide

### Sprint Structure (2-week sprints with TDD)

**Sprint 1 (Weeks 1-2)**: Phase 1 - Foundation

**TDD Workflow**:
1. Day 1-2: Team TDD training/workshop
2. Day 3-10: Strict TDD for all stories
3. Daily: Pair programming sessions (TDD pairs)
4. End of sprint: TDD retrospective

- User Stories (with TDD acceptance criteria):
  - US-1.1: Implement rate limiting
    - ✅ Tests written first for RateLimiter
    - ✅ RED-GREEN-REFACTOR documented in commits
    - ✅ 95% coverage achieved
  - US-1.2: Implement backpressure
    - ✅ Tests written first for BackpressureController
    - ✅ Integration tests with BufferManager
    - ✅ 95% coverage achieved
  - US-1.3: Achieve 95%/90% test coverage
    - ✅ JaCoCo configured with thresholds
    - ✅ CI pipeline enforces coverage
    - ✅ Mutation testing enabled (PIT)
  - US-1.4: Set up Maven project
    - ✅ TDD-friendly project structure
    - ✅ Test dependencies configured
    - ✅ Gitea Actions/Drone CI configured
  - US-1.5: Define all port interfaces
    - ✅ Tests define interface contracts
    - ✅ Tests committed before interfaces
  - US-1.6: Implement domain models
    - ✅ Tests for immutability, serialization
    - ✅ TDD for JSON/Base64 encoding

**Sprint 2 (Weeks 3-4)**: Phase 2 - Core Services
- User Stories:
  - US-2.1: Implement ConfigurationManager
  - US-2.2: Implement BufferManager
  - US-2.3: Implement CollectionStatistics
  - US-2.4: Implement DataCollectionService
  - US-2.5: Implement DataTransmissionService

**Sprint 3 (Weeks 5-6)**: Phase 3 Part 1 - Secondary Adapters
- User Stories:
  - US-3.1: Implement HttpPollingAdapter
  - US-3.2: Implement ExponentialBackoffAdapter
  - US-3.3: Implement FileLoggingAdapter
  - US-3.4: Implement GrpcStreamAdapter

**Sprint 4 (Week 7)**: Phase 3 Part 2 - Primary Adapters & Application
- User Stories:
  - US-4.1: Implement ConfigurationFileAdapter
  - US-4.2: Implement HealthCheckController
  - US-4.3: Implement HspApplication main

**Sprint 5 (Week 8)**: Phase 4 - Testing
- User Stories:
  - US-5.1: Complete integration test suite
  - US-5.2: Execute performance tests
  - US-5.3: Execute reliability tests
  - US-5.4: Execute compliance tests
  - US-5.5: Validate coverage targets

**Sprint 6 (Weeks 9-10)**: Phase 5 - Integration & Deployment
- User Stories:
  - US-6.1: Execute E2E test scenarios
  - US-6.2: Finalize documentation
  - US-6.3: Create deployable artifacts
  - US-6.4: Validate in staging environment

### Story Point Estimation

| Component Type | Story Points | Rationale |
|---------------|--------------|-----------|
| Simple Adapter | 3-5 | Straightforward implementation, clear interfaces |
| Complex Adapter | 8-13 | gRPC, HTTP with retry/backoff logic |
| Core Service | 13-21 | Business logic, concurrency, multiple dependencies |
| Domain Model | 2-3 | Immutable value object, serialization |
| Port Interface | 1-2 | Interface definition only |
| Integration Test Suite | 8-13 | Multiple scenarios, mock setup |
| Enhancement (Rate Limiting) | 3-5 | Well-defined, single responsibility |

**Total Story Points**: ~180-220 (for 10-week project)
**Velocity Target**: 30-40 points per 2-week sprint

---

## 🔧 Technical Setup

### Development Environment

**Required Software**:
- **JDK**: OpenJDK 25 (Java 25 features required)
- **Build**: Maven 3.9+
- **IDE**: IntelliJ IDEA / Eclipse with Java 25 support
  - IDE Plugin: TDD/Test Runner (live test feedback)
  - IDE Plugin: Coverage visualization (EclEmma/IntelliJ Coverage)
- **Version Control**: Git 2.40+
- **SCM Platform**: Gitea (self-hosted)
- **CI/CD**: Gitea Actions or Drone CI
- **Testing**: JUnit 5, Mockito, WireMock, gRPC Testing
- **Coverage**: JaCoCo (with 95%/90% enforcement)
- **Mutation Testing**: PIT (for test quality validation)
- **Profiling**: JProfiler / YourKit (for performance testing)

**Dependencies** (from pom.xml):
```xml
<dependencies>
    <!-- Core -->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>2.0.9</version>
    </dependency>

    <!-- gRPC -->
    <dependency>
        <groupId>io.grpc</groupId>
        <artifactId>grpc-netty-shaded</artifactId>
        <version>1.60.0</version>
    </dependency>
    <dependency>
        <groupId>io.grpc</groupId>
        <artifactId>grpc-protobuf</artifactId>
        <version>1.60.0</version>
    </dependency>
    <dependency>
        <groupId>io.grpc</groupId>
        <artifactId>grpc-stub</artifactId>
        <version>1.60.0</version>
    </dependency>

    <!-- Protocol Buffers -->
    <dependency>
        <groupId>com.google.protobuf</groupId>
        <artifactId>protobuf-java</artifactId>
        <version>3.25.0</version>
    </dependency>

    <!-- JSON -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.16.0</version>
    </dependency>

    <!-- Rate Limiting -->
    <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>32.1.3-jre</version>
    </dependency>

    <!-- Testing -->
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter</artifactId>
        <version>5.10.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.mockito</groupId>
        <artifactId>mockito-core</artifactId>
        <version>5.7.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.github.tomakehurst</groupId>
        <artifactId>wiremock</artifactId>
        <version>3.0.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>io.grpc</groupId>
        <artifactId>grpc-testing</artifactId>
        <version>1.60.0</version>
        <scope>test</scope>
    </dependency>
</dependencies>
```

### CI/CD Pipeline (Gitea)

**Platform**: Gitea with Gitea Actions (or Drone CI)

**Option 1: Gitea Actions** (.gitea/workflows/ci.yml):
```yaml
name: HSP CI/CD Pipeline

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  compile:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 25
        uses: actions/setup-java@v3
        with:
          java-version: '25'
          distribution: 'temurin'
      - name: Compile
        run: mvn clean compile

  unit-test:
    needs: compile
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 25
        uses: actions/setup-java@v3
        with:
          java-version: '25'
          distribution: 'temurin'
      - name: Run Unit Tests
        run: mvn test
      - name: Generate Coverage Report
        run: mvn jacoco:report
      - name: Check Coverage Threshold
        run: |
          mvn jacoco:check \
            -Djacoco.line.coverage=0.95 \
            -Djacoco.branch.coverage=0.90
      - name: Upload Coverage Report
        uses: actions/upload-artifact@v3
        with:
          name: coverage-report
          path: target/site/jacoco/

  integration-test:
    needs: unit-test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 25
        uses: actions/setup-java@v3
        with:
          java-version: '25'
          distribution: 'temurin'
      - name: Run Integration Tests
        run: mvn verify -P integration-tests

  performance-test:
    needs: integration-test
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 25
        uses: actions/setup-java@v3
        with:
          java-version: '25'
          distribution: 'temurin'
      - name: Run Performance Tests
        run: mvn verify -P performance-tests

  package:
    needs: integration-test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 25
        uses: actions/setup-java@v3
        with:
          java-version: '25'
          distribution: 'temurin'
      - name: Package Fat JAR
        run: mvn package -P fat-jar
      - name: Upload Artifact
        uses: actions/upload-artifact@v3
        with:
          name: hsp-jar
          path: target/hsp-*.jar

  deploy-staging:
    needs: package
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/develop'
    steps:
      - name: Deploy to Staging
        run: |
          # Deploy to staging environment
          echo "Deploying to staging..."
      - name: Run Smoke Tests
        run: |
          # Run smoke tests
          echo "Running smoke tests..."
```

**Option 2: Drone CI** (.drone.yml):
```yaml
kind: pipeline
type: docker
name: hsp-pipeline

steps:
  - name: compile
    image: maven:3.9-eclipse-temurin-25
    commands:
      - mvn clean compile

  - name: unit-test
    image: maven:3.9-eclipse-temurin-25
    commands:
      - mvn test
      - mvn jacoco:report
      - mvn jacoco:check -Djacoco.line.coverage=0.95 -Djacoco.branch.coverage=0.90

  - name: integration-test
    image: maven:3.9-eclipse-temurin-25
    commands:
      - mvn verify -P integration-tests

  - name: performance-test
    image: maven:3.9-eclipse-temurin-25
    when:
      branch:
        - main
    commands:
      - mvn verify -P performance-tests

  - name: package
    image: maven:3.9-eclipse-temurin-25
    commands:
      - mvn package -P fat-jar

  - name: deploy-staging
    image: alpine:latest
    when:
      branch:
        - develop
    commands:
      - echo "Deploy to staging environment"
      - # Add deployment commands

trigger:
  branch:
    - main
    - develop
  event:
    - push
    - pull_request
```

**Quality Gates**:
- Code coverage: ≥ 95% line, ≥ 90% branch
- Zero critical security vulnerabilities
- All tests passing (unit + integration)
- Code review approved (Gitea PR review)
- Documentation updated
- TDD workflow followed (tests committed before implementation)

---

## ⚠️ Risk Management

### High Priority Risks

| Risk | Probability | Impact | Mitigation | Owner |
|------|------------|--------|------------|-------|
| **gRPC integration complexity** | Medium | High | Early prototype, gRPC expert on team | Tech Lead |
| **Virtual thread debugging issues** | Medium | Medium | Use Java 25 LTS, thorough logging | Senior Dev |
| **Buffer overflow in production** | Low | High | Monitor buffer metrics, make configurable | DevOps |
| **Test coverage not achieved** | Medium | High | Allocate sufficient time, start early | QA Lead |
| **Performance requirements not met** | Low | High | Early benchmarking, profiling tools | Perf Engineer |

### Medium Priority Risks

| Risk | Probability | Impact | Mitigation |
|------|------------|--------|------------|
| Schedule slippage | Medium | Medium | Weekly progress reviews, buffer in schedule |
| Team availability | Low | Medium | Cross-training, documentation |
| Requirement changes | Low | Medium | Frozen requirements, change control process |
| Integration issues | Medium | Low | Continuous integration, early testing |

### Accepted Risks (Per ARCHITECTURE_DECISIONS.md)

| Risk | Severity | Acceptance Rationale |
|------|----------|---------------------|
| No TLS encryption | HIGH | Deployment in isolated network, future release |
| Buffer size 300 | MEDIUM | Meets current requirements, configurable |
| No circuit breaker | MEDIUM | Retry mechanisms sufficient, manual intervention |
| No graceful shutdown | LOW | Continuous operation design, acceptable data loss |

---

## 📈 Success Criteria

### Technical Success Criteria

- ✅ **Functionality**: All 62 requirements implemented and verified
- ✅ **Test Coverage**: 95% line coverage, 90% branch coverage
- ✅ **Performance**: 1000 concurrent endpoints, < 4096MB memory
- ✅ **Reliability**: Zero data loss in normal operation, graceful degradation
- ✅ **Code Quality**: Zero critical bugs, passing code review
- ✅ **Documentation**: Complete user and operations documentation

### Project Success Criteria

- ✅ **Schedule**: Delivered within 10-12 weeks
- ✅ **Budget**: Within allocated person-days (60-80 days)
- ✅ **Quality**: Production-ready deployment
- ✅ **Stakeholder Satisfaction**: Approved by product owner
- ✅ **Compliance**: Meets ISO-9001, EN 50716 requirements

---

## 📋 Deliverables Checklist

### Code Deliverables

- [ ] Source code (Java 25, Maven project)
- [ ] Port interfaces (8 interfaces)
- [ ] Domain models (4 value objects)
- [ ] Core services (5 services)
- [ ] Adapters (7 adapters + 2 enhancements)
- [ ] Application main (HspApplication)
- [ ] Configuration schema (JSON)
- [ ] Protocol Buffers definitions (gRPC)

### Test Deliverables

- [ ] Unit tests (95% coverage, 90 branch)
- [ ] Integration tests (20+ scenarios)
- [ ] Performance tests (4 benchmark suites)
- [ ] Reliability tests (5 failure scenarios)
- [ ] Compliance tests (4 normative requirements)
- [ ] E2E tests (4 system-level scenarios)
- [ ] Coverage reports (JaCoCo, PIT)

### Documentation Deliverables

- [ ] User guide (installation, configuration, operation)
- [ ] Operations manual (monitoring, troubleshooting)
- [ ] API documentation (health check endpoint)
- [ ] Javadoc (all public APIs)
- [ ] Deployment guide (system requirements, installation)
- [ ] Architecture documentation (updated with as-built)
- [ ] Test reports (coverage, performance, compliance)

### Deployment Deliverables

- [ ] Fat JAR executable
- [ ] Startup scripts (Linux, Windows)
- [ ] Configuration templates
- [ ] Installation scripts
- [ ] Staging deployment (validated)
- [ ] Production deployment artifacts
- [ ] Runbook (operations procedures)

---

## 📞 Communication Plan

### Status Reporting

**Daily Standups** (15 minutes):
- What was completed yesterday?
- What will be completed today?
- Any blockers?

**Weekly Status Report**:
- Progress against milestones
- Risks and issues
- Metrics (velocity, coverage, bugs)
- Next week's plan

**Sprint Reviews** (Every 2 weeks):
- Demo completed functionality
- Retrospective
- Sprint planning for next sprint

### Stakeholder Communication

| Stakeholder | Frequency | Format | Content |
|-------------|-----------|--------|---------|
| Product Owner | Weekly | Email + Meeting | Progress, risks, decisions needed |
| Architecture Team | Bi-weekly | Meeting | Technical decisions, reviews |
| QA Team | Daily | Chat + Meeting | Test status, blockers |
| DevOps Team | Weekly | Meeting | Build, deployment, infrastructure |

---

## 🎯 Next Steps for Implementation Team

### Immediate Actions (Week 1, Day 1)

1. **Team Kickoff Meeting** (2 hours)
   - Review project plan
   - Review architecture documents
   - **Emphasize TDD mandatory approach**
   - Assign roles and responsibilities
   - Set up communication channels

2. **TDD Training Workshop** (4 hours, Day 1 afternoon)
   - TDD principles and Red-Green-Refactor
   - Live TDD demonstration (pair programming)
   - Practice session: Implement simple component with TDD
   - Git workflow for TDD (commit patterns)
   - Code review for TDD compliance

3. **Environment Setup** (1 day, Day 2)
   - Install JDK 25, Maven, IDE
   - Configure IDE for TDD (test runners, coverage tools)
   - Clone repository from Gitea
   - Set up Gitea Actions or Drone CI access
   - Verify build environment
   - Test TDD workflow (write test, fail, pass, commit)

4. **Document Review** (2 days, Days 3-4)
   - Read DataCollector SRS.md (all team members)
   - Read ARCHITECTURE_DECISIONS.md (all team members)
   - Read system-architecture.md (developers)
   - Read test-strategy.md (QA team)
   - Review TDD examples from architecture docs

5. **Sprint 1 Planning** (2 hours, Day 5)
   - Break down Phase 1 into user stories
   - Estimate story points
   - Assign TDD pairs (pair programming)
   - Set sprint goals with TDD acceptance criteria
   - Define "Definition of Done" (must include TDD compliance)

### First Week Goals

- [ ] Team fully onboarded with TDD training complete
- [ ] Development environment set up (including TDD tools)
- [ ] Gitea repository configured with CI/CD pipeline
- [ ] TDD workflow validated (sample component with full TDD cycle)
- [ ] Sprint 1 planned and started with TDD pairs assigned
- [ ] Rate limiting: Tests written (RED phase)
- [ ] Backpressure: Tests written (RED phase)
- [ ] Maven project structure created with test framework configured

---

## 📚 Appendix

### A. Requirement Categories

- **Functional (33)**: Req-FR-1 to FR-33
- **Non-Functional (8)**: Req-NFR-1 to NFR-8
- **Architectural (8)**: Req-Arch-1 to Arch-8
- **Testing (4)**: Req-Test-1 to Test-4
- **Normative (6)**: Req-Norm-1 to Norm-6
- **User Stories (3)**: Req-US-1 to US-3

**Total: 62 unique requirements**

### B. Technology Stack Summary

| Category | Technology | Version |
|----------|-----------|---------|
| Language | Java | 25 |
| Build | Maven | 3.9+ |
| Concurrency | Virtual Threads | Java 25 |
| RPC | gRPC Java | 1.60+ |
| Serialization | Protocol Buffers | 3.25+ |
| JSON | Jackson | 2.16+ |
| HTTP Client | Java HttpClient | Java 25 |
| Logging | Java Logging API | Java 25 |
| Testing | JUnit 5 | 5.10+ |
| Mocking | Mockito | 5.7+ |
| HTTP Mock | WireMock | 3.0+ |
| Coverage | JaCoCo | 0.8.11+ |
| Mutation | PIT | 1.15+ |

### C. Key Contacts

| Role | Contact | Responsibility |
|------|---------|----------------|
| Product Owner | TBD | Requirements, decisions, approval |
| Tech Lead | TBD | Architecture, technical decisions |
| QA Lead | TBD | Test strategy, quality assurance |
| DevOps Lead | TBD | Build, deployment, infrastructure |
| Project Manager | TBD | Schedule, budget, communication |

---

## 📄 Document Control

**Document**: PROJECT_IMPLEMENTATION_PLAN.md
**Version**: 1.0
**Status**: Approved for Implementation
**Created**: 2025-11-19
**Last Updated**: 2025-11-19
**Next Review**: After Phase 1 completion

**Approval**:
- [ ] Product Owner
- [ ] Tech Lead
- [ ] Project Manager

**Change History**:
| Version | Date | Changes | Author |
|---------|------|---------|--------|
| 1.0 | 2025-11-19 | Initial plan created | System Architect |

---

**END OF PROJECT IMPLEMENTATION PLAN**

For questions or clarifications, refer to specific documents listed in the Critical Document Index or contact the project manager.