Skip to content

CIS Reporting Engine — Architecture & Developer Guide

Target Platform: Spring Boot 4.1.2 | Java 25 | Jakarta EE 11 | Oracle Database 23ai
Package: com.dw.ngms.cis.reporting
Git Branch: feat/error-framework


1. Executive Summary & Design Principles

The CIS Reporting Engine (com.dw.ngms.cis.reporting) provides analytical data extraction, statutory report compilation, and document generation across all Cadastral modules. It addresses the historical scaling bottlenecks identified in performance audits (unbounded heap retention, repeated calendar lookups, and injection vulnerabilities in dynamic SQL) through a modern, bounded streaming architecture.

Architectural Invariants

  1. Bounded Heap Footprint: CSV reports stream directly from JDBC cursors to disk artifacts via buffered chunks (CsvArtifactWriter). Large exports (100k+ rows) execute within a bounded heap budget.
  2. Safe Typed Query Construction: DynamicQueryBuilder binds named parameters strictly through server-owned column definitions, eliminating SQL concatenation vulnerabilities.
  3. Multi-Tenant Scope Authorization: ReportScopeAuthorizer enforces user jurisdiction (province, office, role boundaries) at the query level before execution.
  4. Durable Asynchronous Job Queue: Reports exceeding synchronous timeout thresholds are dispatched as background jobs (ReportJobService) tracked in Oracle (REPORT_EXPORT_JOB) and stored in S3/disk.
  5. Rate-Limited Download Slots: ReportDownloadSlots bounds concurrent file downloads to protect backend bandwidth.

2. Dedicated Architectural Diagrams

2.1 Component & Pipeline Topology

flowchart TD
    subgraph Client_Layer [Clients & Gateways]
        ANGULAR["Angular Frontend"]
        BFF["cis-bff"]
    end

    subgraph REST_API [Reporting API Layer]
        RESOURCE["ReportJobResource / ReportingResource"]
        LIMITS["ReportJobLimits & ReportDownloadSlots"]
        ADVICE["ReportJobErrorAdvice"]
    end

    subgraph Core_Engine [com.dw.ngms.cis.reporting.core]
        REGISTRY["ReportDefinitionRegistry"]
        AUTH["ReportScopeAuthorizer"]
        BUILDER["DynamicQueryBuilder"]
        SCOPED["ScopedReport Interface"]
    end

    subgraph Job_Orchestration [com.dw.ngms.cis.reporting.jobs]
        SERVICE["ReportJobService"]
        SCHEDULER["ReportJobScheduling"]
        TELEMETRY["ReportJobTelemetry"]
        STORE["ReportJobStore"]
    end

    subgraph Artifact_Renderers [Artifact Generation]
        CSV["CsvArtifactWriter (Streaming Bounded Buffer)"]
        PDF["PdfArtifactRenderer (Thymeleaf / Flying Saucer)"]
    end

    subgraph Storage [Persistence & Storage]
        ORACLE[("Oracle DB: REPORT_EXPORT_JOB")]
        FS[("S3 / File System Storage")]
    end

    %% Flow
    ANGULAR & BFF --> RESOURCE
    RESOURCE --> LIMITS
    RESOURCE --> AUTH
    AUTH --> REGISTRY
    RESOURCE --> SERVICE
    SERVICE --> STORE
    STORE --> ORACLE

    SERVICE --> SCOPED
    SCOPED --> BUILDER
    BUILDER --> ORACLE

    SCOPED --> CSV & PDF
    CSV & PDF --> FS
    SERVICE --> TELEMETRY

2.2 Background Report Job Lifecycle Sequence Diagram

sequenceDiagram
    autonumber
    actor User as Cadastral Officer
    participant Resource as ReportJobResource
    participant Auth as ReportScopeAuthorizer
    participant JobSvc as ReportJobService
    participant Store as ReportJobStore
    participant Builder as DynamicQueryBuilder
    participant Writer as CsvArtifactWriter
    participant S3 as S3/File Storage

    User->>Resource: POST /api/v1/reporting/jobs (type="ARCHIVE_PRODUCTION", filters, format="CSV")
    Resource->>Auth: authorizeUserScope(user, filters)
    Auth-->>Resource: Authorized Scope (Province: GAUTENG)

    Resource->>JobSvc: submitReportJob(reportDef, filters, scope)
    JobSvc->>Store: createJobRecord(status="PENDING")
    Store-->>JobSvc: jobId=5024
    JobSvc-->>Resource: JobCreatedResponse(jobId=5024, status="PENDING")
    Resource-->>User: HTTP 202 Accepted { jobId: 5024 }

    Note over JobSvc: Asynchronous Worker Execution
    JobSvc->>Store: updateStatus(jobId=5024, "PROCESSING")
    JobSvc->>Builder: buildBoundedQuery(reportDef, filters, scope)
    Builder-->>JobSvc: Prepared SQL & Named Parameters

    JobSvc->>Writer: streamResultsToArtifact(sql, params)
    loop Bounded Row Chunks
        Writer->>Writer: Fetch batch (1000 rows) -> Flush CSV buffer
    end
    Writer->>S3: saveArtifact(jobId=5024, file)
    S3-->>Writer: artifactUri

    JobSvc->>Store: updateStatus(jobId=5024, "COMPLETED", artifactUri)

    Note over User: Polling or Notification
    User->>Resource: GET /api/v1/reporting/jobs/5024/download
    Resource->>Store: getJob(5024)
    Store-->>Resource: Job(status="COMPLETED", artifactUri)
    Resource-->>User: HTTP 200 File Stream (Content-Disposition: attachment)

2.3 Class Hierarchy Diagram

classDiagram
    direction TB

    class ScopedReport {
        <<Interface>>
        +getReportId() String
        +generateReportData(filters, scope) Object
        +supportsFormat(format) boolean
    }

    class BackgroundCsvReport {
        <<Interface>>
        +streamCsv(filters, scope, writer) void
        +getEstimatedCount(filters, scope) long
    }

    class DynamicQueryBuilder {
        -NamedParameterJdbcTemplate jdbcTemplate
        +buildQuery(criteria, allowedColumns) PreparedQuery
        +bindScopeRestrictions(sql, userScope) String
    }

    class ReportScopeAuthorizer {
        +enforceProvinceScope(user, requestedProvinces) List~String~
        +validateDateBounds(filters) void
    }

    class ReportJobService {
        -ReportJobStore store
        -ReportDefinitionRegistry registry
        -ThreadPoolTaskExecutor executor
        +submitJob(request) JobHandle
        +pollJobStatus(jobId) JobStatusDto
        +acquireDownloadSlot(jobId) InputStream
    }

    class CsvArtifactWriter {
        -int bufferSize
        +writeChunk(rows) void
        +flush() void
        +close() void
    }

    class PdfArtifactRenderer {
        -TemplateEngine templateEngine
        -ReportPdfUserAgent userAgent
        +renderToPdf(templateName, model, outputStream) void
    }

    ScopedReport <|.. BackgroundCsvReport
    BackgroundCsvReport <|.. ArchiveProductionReport
    BackgroundCsvReport <|.. ReservationProductionReport
    BackgroundCsvReport <|.. LodgementProductionReport
    BackgroundCsvReport <|.. ExaminationProductionReport

    ReportJobService --> ScopedReport : executes
    ReportJobService --> DynamicQueryBuilder : queries
    ReportJobService --> ReportScopeAuthorizer : checks
    ScopedReport --> CsvArtifactWriter : writes CSV
    ScopedReport --> PdfArtifactRenderer : renders PDF

3. Supported Report Catalog

The reporting framework orchestrates reports across all business modules: 1. Archive Reports (com.dw.ngms.cis.reporting.reports.archive): - ArchiveProductionReport: Monthly throughput of archived packets and microfilms. - ScanningProductionReport: Scan volumes and DPI compliance statistics. - PSDTReport: Plan Safe document tracking audit. 2. Reservation Reports (com.dw.ngms.cis.reporting.reports.reservation): - ReservationProductionReport: Parcels reserved, extended, and lapsed per province. - ReservationTransferReport: Transferee changes and surveyor ownership shifts. 3. Lodgement & Examination Reports (com.dw.ngms.cis.reporting.reports.workflow): - WorkflowProductionReport: Cadastral turn-around times per office against statutory SLAs. - WorkflowUserProductionReport: Examiner daily productivity metrics. 4. User Management Reports (com.dw.ngms.cis.reporting.reports.usermanagement): - UserSummaryReport: Active internal and external professional surveyors. - UserHourlyLoginSummary: Security session concurrency and access audit.


4. Developer Guide: Adding a New Scoped Report

To add a new analytical report to the framework:

  1. Implement BackgroundCsvReport in the appropriate subpackage:
    @Component("LODGEMENT_SUMMARY_REPORT")
    @RequiredArgsConstructor
    public class LodgementSummaryReport implements BackgroundCsvReport {
    
        private final DynamicQueryBuilder queryBuilder;
    
        @Override
        public String getReportId() {
            return "LODGEMENT_SUMMARY";
        }
    
        @Override
        public void streamCsv(ReportFilters filters, UserScope scope, CsvArtifactWriter writer) {
            writer.writeHeader("Lodgement ID", "Province", "Status", "Lodged Date", "Surveyor");
            queryBuilder.streamRows("SELECT * FROM VW_LODGEMENT_EXPORT", filters, scope, rs -> {
                writer.writeRow(rs.getString("ID"), rs.getString("PROVINCE"), 
                                rs.getString("STATUS"), rs.getDate("LODGED_DATE"), rs.getString("USER_NAME"));
            });
        }
    }
    
  2. Register the report ID in ReportDefinitionRegistry.
  3. The engine automatically handles scope authorization, job persistence, chunked streaming, and download rate-limiting.