Skip to main content

​
Sprint 17: Telemetry & Analytics (The “God View”)

With the core mechanics, automated content factory, macro-economy, and premium consumable systems locked in, PCMTG is rapidly approaching a 1.0 release candidate. However, operating a highly volatile, user-driven digital economy requires absolute observability. Sprint 17 focuses on deploying our enterprise data warehouse using Google Cloud BigQuery. We will construct asynchronous pipelines to monitor hyperinflation, track quadrant win rates, and audit the esoteric Euclidean math at scale, all without introducing latency into the 60-second debate loop.

​
🛠 Context & Required Skills for AI Agents

When working on this sprint, agents must adhere to the following technical contexts:
  1. BigQuery Storage Write API: We do not use standard INSERT statements to move data into BigQuery. The Cloud Run engine must use the @google-cloud/bigquery-storage library to stream batches of resolved match data asynchronously.
  2. Cost Control (Partitioning & Clustering): BigQuery charges per byte scanned. To prevent cost overruns, all tables MUST be time-partitioned (e.g., DATE(resolved_at)) and clustered by highly queried fields (e.g., winning_quadrant, tx_type).
  3. Non-Blocking Execution: Telemetry ingestion must fire after the prisma.$transaction completes and must never block or delay the HTTP 200 OK response returned to the Next.js client.

​
🗺 Sprint 17 Roadmap

​
[x] Phase 0: BigQuery Provisioning & Schema Definition

  • Objective: Initialize the data warehouse with cost-optimized table structures.
  • Tasks:
    • Create the GCP BigQuery dataset: pcmtg_telemetry.
    • Define and provision the MatchHistory table.
      • Schema: lobby_id, resolved_at, outcome_type, winning_quadrant, target_vector_x, target_vector_y, target_vector_z.
      • Optimization: Partition by DATE(resolved_at), Cluster by winning_quadrant and outcome_type.
    • Define and provision the EconomyLogs table.
      • Schema: tx_id, player_id, tx_type, currency_ticker, amount, timestamp.
      • Optimization: Partition by DATE(timestamp), Cluster by tx_type and player_id.

​
[x] Phase 1: Storage Write API Pipeline Integration

  • Objective: Stream real-time events asynchronously using the Google Cloud BigQuery Storage Write API.
  • Tasks:
    • Integrate the @google-cloud/bigquery-storage client library in the Node.js backend.
    • Implement an asynchronous, non-blocking telemetry queuing utility TelemetryQueue.
    • Dispatch MatchHistory telemetry payload asynchronously after the Prisma transaction resolves on /debate/resolve and /debate/sweep.
    • Dispatch EconomyLogs telemetry payload asynchronously after any economy transaction resolves on /market/* or /inventory/*.

​
[x] Phase 2: Analytics & Materialized Views

  • Objective: Support high-performance, cost-effective querying via BigQuery Materialized Views.
  • Tasks:
    • Create materialized views for tracking hourly hyperinflation metrics by currency_ticker.
    • Create materialized views for tracking quadrant-specific win rates across lobbies.
    • Implement automated audit queries verifying that local Euclidean ELO matchmaker math perfectly mirrors the raw logs.

​
[x] Phase 3: Looker Studio Dashboards (Visual “God View”)

  • Objective: Assemble premium real-time visualization dashboards.
  • Tasks:
    • Connect Looker Studio to the pcmtg_telemetry dataset using partitioned/clustered filters to minimize scan costs.
    • Build real-time timeseries charts for player wealth distribution and active currency velocity.
    • Generate a geographical/matrix coordinate heatmap displaying Euclidean consensus drift in real-time.

​
🏁 Retrospective & Conclusion

Sprint 17 has successfully established an enterprise-grade, cost-controlled observability platform for PCMTG. By deploying our BigQuery data warehouse with time-partitioned and clustered table structures, we ensure optimal query performance and minimal scanning costs. The Storage Write API was integrated via a fully asynchronous, non-blocking pipeline, ensuring that real-time telemetry streaming never introduces latency into the core 60-second debate loop. Finally, the creation of BigQuery materialized views and the design of the Looker Studio dashboard (the “God View”) enable premium, real-time analytics and coordinate-drift visualizations, delivering absolute observability over our game-theoretic economy.