Week 8: The Bimodal Convergence

This week represents the "Grand Finale" of the A.E.G.I.S. development phase. The primary objective was to close the loop on the bimodal sensing architecture by fusing the Wi-Fi CSI Kinetic Engine with the mmWave Biological Engine. After weeks of isolating the physics of wave-scattering and the telemetry of radar presence, I successfully integrated these subsystems into a unified, high-fidelity monitoring framework.

The transition from disparate hardware modules to a singular, synchronised system is arguably the most rigorous phase of the project. It requires meticulous resource management to ensure the Raspberry Pi 5 can ingest two high-speed serial streams while simultaneously calculating complex matrix math and rendering a real-time telemetry dashboard.

The Integration Challenge: Engineering Synchronicity

In a bimodal system, "Synchronicity" refers to the alignment of two different physical perspectives. My goal was to create an environment where the Wi-Fi CSI (detecting kinetic energy) and the mmWave Radar (detecting human respiration) could "handshake" in real-time. This ensures that if a high-velocity impact occurs, the system immediately queries the radar to verify if a biological target is present at floor level.

Redesigning the Master Telemetry Dashboard (GUI)

To support this bimodal stream, I redesigned the GUI for the ILI9341 2.8-inch SPI display. I abandoned the basic text-only readout in favour of a high-contrast telemetry layout. Using the Pillow (PIL) library, I partitioned the screen into logical diagnostic zones:

• Telemetry Panel (Left): Real-time numerical readouts for Wi-Fi Variance (σ), Radar State, and Target Distance (D cm).
• Confidence Engine (Right): A dynamic percentage display representing the heuristic probability of a fall.
• Kinetic Variance Bar (Centre): A colour-coded horizontal bar that visualises signal instability, transitioning from Cyan (Nominal) to Red (Critical).

The Software Stack and Processing Pipeline

The entire backend is architected in Python 3.11. Python was selected as the integration bridge because it allows for the seamless orchestration of low-level C++ binaries (via spidev and lgpio) with high-performance mathematical libraries like NumPy.

Core Data Sources & Engines:

• NumPy Physics Engine: For the real-time calculation of the Standard Deviation (σ) across 64 complex Wi-Fi subcarriers.
• Serial Hex-Parser: A custom-built parser for the HLK-LD2410C radar, performing bitwise shifts to reconstruct Little-Endian distance bytes.
• Threading Module: Utilised to manage three parallel execution threads to prevent CPU "bottlenecking":
  • Thread 1: CSI data ingestion via /dev/ttyUSB0 (115,200 baud).
  • Thread 2: Radar telemetry ingestion via /dev/ttyAMA0 (256,000 baud).
  • Thread 3: The Master Logic Loop and GUI rendering engine.

Implementation: The Weighted Confidence Loop

The AegisMaster.py script is the new "Command and Control" centre of the project. It implements a Heuristic Weighted Engine to decide when to trigger an alert.

Because Wi-Fi signals can be "noisy," we cannot rely on a single spike. Instead, the system assigns "Confidence Points":

• Initial Wave-Shift: If σ>3.0, add 35% confidence.
• Critical Velocity: If σ>7.5, add an additional 35% confidence.
• Bio-Verification: If the Radar reports a STATIONARY or MOVING target, add 30% confidence.

If the total confidence exceeds 65%, the GUI is instructed to override the telemetry and render the "FALL DETECTED" alert.

Troubleshooting and Technical Hurdles

Challenge 1: The Serial "Traffic Jam"

• Problem: The LD2410C radar broadcasts data at 256,000 baud. During testing, I noticed the radar distance on the screen was "lagging" several seconds behind the person’s actual movement. The Pi 5 was receiving data faster than the Python script could parse it, causing a massive buffer backlog.
• Resolution: I implemented a Serial Flush Logic. At the start of every parsing cycle, the code calls self.ser.reset_input_buffer() if the in_waiting count exceeds 2,000 bytes. This ensures the system only processes the absolute latest frame, restoring real-time performance.

Challenge 2: The "Ghost" Occupancy Problem

• Problem: The Wi-Fi variance would occasionally spike when a door was closed or a heavy object was moved, nearly triggering a false alarm.
• Resolution: I updated the Logic Gate to strictly require the Radar’s "State 2" (Stationary) or "State 1" (Moving) before the confidence could reach the 65% trigger. If the radar reports "State 0" (Empty), the alert is suppressed, proving that the disturbance was non-biological.

Week 8 Checklist

• Bimodal Fusion: Successfully combined Wi-Fi CSI and mmWave Radar into a singular logic loop.
• Asynchronous Optimisation: Resolved serial latency issues through buffer management and threading.
• GUI Finalisation: Completed the 320x240 telemetry dashboard with real-time variance bars.
• Empirical Validation: Verified that a "chair drop" (high variance/no radar) does not trigger an alert, while a "human fall" (high variance/radar presence) hits 100% confidence.