Layer 2 & 3 Complete ML Pipeline Trained Layer 4-5 In Progress

Institutional-Grade
Quantitative Trading Pipeline

An end-to-end, multi-asset architecture that evaluates, filters, and executes algorithmic strategies using ML meta-labeling and dynamic market regime detection.

6
Execution Layers
900,000+
Signals Processed
1:3
R/R Pipeline
3
Forex Pairs Active

Data Flow Pipeline

Every trade passes through six deterministic layers before execution

L0 Qualify
L1 Regime
L2 Signals
L3 ML Gate
L4 Risk
L5 Dashboard

The 6-Layer Architecture

A strict, instrument-agnostic pipeline that evaluates every asset independently before aggregating risk at the portfolio level

0

Strategy Qualification Engine

Offline Backtesting Sandbox
Complete

A rigid backtesting sandbox that mathematically validates trading strategies before live deployment.

  • Process: Strategies tested against decades of historical OHLCV data with standardized ATR-based risk profiles
  • Promotion Criteria: Minimum 20 trades, positive Expectancy, Profit Factor > 1.15
  • Output: Strategies marked PROMOTED or REJECTED, inserted into Dim_Strategy_Registry
  • Philosophy: No strategy touches live data until it proves a mathematical edge
1

Market Regime Detection

The Weather Report
Complete

Dynamically classifies the current market state for each asset independently using unsupervised learning.

  • Algorithm: K-Means clustering on ATR (volatility) + ADX (trend strength)
  • Regimes: Trending_HighVol, Trending_LowVol, Ranging_HighVol, Ranging_LowVol
  • Validation: Silhouette Score for cluster quality evaluation
  • Output: Regime tags persisted to Fact_Market_Regime SQL table
2

Live Strategy Bank

Signal Generators
Complete

Promoted strategies run continuously on incoming H1 candle data, generating raw trade signals.

  • Process: PROMOTED strategies from Layer 0 scan live price action
  • Output: Raw BUY/SELL signals with ATR-based stop/target distances
  • Key Rule: Signals are NOT sent to the broker—they go to the AI Gatekeeper
  • Timeframe: H1 (hourly) candles across all monitored pairs
  • Current Status: Successfully ingested over 915,400 raw mathematical signals across 3 assets into the Fact_Signals database
3

ML Meta-Labeler

The AI Gatekeeper
Complete

XGBoost classification model that contextualizes and filters raw signals. It does not predict price—it evaluates trade probability in the current regime using a dynamic 1:3 ATR-based target variable.

  • Features: EMA_50, EMA_200, RSI, ADX, ATR, session flags, H4 trend alignment
  • Temporal: London/NY session flags, London-NY overlap detection
  • Threshold: Trades must score > 0.75 confidence to be APPROVED
  • Training: 70/30 chronological split, 300 boosting rounds, depth 4
  • Dynamic Target: The AI no longer looks blindly 24 hours ahead; it is trained to hunt for asymmetrical Risk:Reward paths (Stop Loss = 1 ATR, Take Profit = 3 ATR)
  • Bake-offs: We conduct "ML Bake-offs" using Optuna to tune XGBoost, LightGBM, Random Forest, and PyTorch LSTMs
4

Dynamic Risk & Portfolio Correlation

The Shield
In Progress

Two-part risk layer: trade-level ATR stops and portfolio-level correlation guard.

  • Layer 4 (Trade Risk): Stop Loss = Close − 1.0 ATR, Take Profit = Close + 2.0 ATR (buys). Fixed pip stops prohibited.
  • Layer 4.5 (Portfolio): 30-day rolling correlation matrix blocks correlated trades
  • Example: Blocks EUR_USD buy if GBP_USD buy already open (USD correlation)
  • Risk Ratio: Strict 1:2 R/R across all setups
5

Telemetry & Visualization

The Dashboard
Planned

Real-time oversight dashboard for human monitoring and system auditing.

  • Tech: Python Streamlit or Power BI dashboard
  • Metrics: Active regimes, AI veto rates, confidence scores, live expectancy
  • Decay Detection: Flags promoted strategies that begin decaying in live markets
  • Oversight: Full trade audit trail with entry/exit reasoning

Qualified Trading Strategies

Each strategy must pass Layer 0's rigorous mathematical qualification before live deployment

Trend EMA ADX Trend

50/200 EMA crossover confirmed by ADX > 25 trend strength filter. Captures momentum in established trends.

Profit Factor > 1.15 1:2 R/R ATR Stops

Range Bollinger Range

Bollinger Band mean reversion combined with RSI < 30 oversold filter. Profits from range-bound markets.

Profit Factor > 1.15 1:2 R/R ATR Stops

Trend Donchian Trend

20-period Donchian channel breakouts. Rides momentum when price breaks through established highs/lows.

Profit Factor > 1.15 1:2 R/R ATR Stops

Range Stochastic Range

Stochastic oscillator oversold bounce strategy. Enters when price reaches statistical extremes in ranging conditions.

Profit Factor > 1.15 1:2 R/R ATR Stops

Research Methodology

How we develop, validate, and deploy trading strategies

Deterministic Strategy Design

Strategies encode strict mathematical rules—not black-box predictions. Every entry and exit condition is explicit, reproducible, and testable. We never optimize for past data; we validate against it.

ML as Contextual Gatekeeper

Machine Learning is used strictly as a meta-labeler. The XGBoost model doesn't predict price—it evaluates whether a given strategy historically succeeds in the current market regime. This prevents overfitting to noise.

Regime-Aware Backtesting

All backtests account for market regimes. A trend-following strategy is evaluated separately in trending vs. ranging conditions, ensuring we understand precisely when and why strategies work.

Walk-Forward Validation

We use chronological 70/30 train-test splits with no data leakage. Models are never validated on data they've seen. Temporal features (session flags, multi-timeframe alignment) add real-world context.

Technology Stack

Built with production-grade tools designed for reliability and performance

🐍

Python 3.10+

Core language for all pipeline logic, data processing, and ML training

📊

Pandas & NumPy

High-performance data manipulation and numerical computation

🧠

XGBoost & Scikit-learn

Gradient boosting for meta-labeling, K-Means for regime clustering

🗄

SQL Server

Containerized relational database with ODBC v17 driver

🐳

Docker

Container orchestration for database and service infrastructure

📈

Pandas-TA

Technical analysis library for indicator computation (RSI, EMA, ADX, ATR)

🌐

Oanda v20 API

REST API integration for live market data and order execution

🐧

Linux (Fedora)

Development and deployment on Fedora/Ubuntu environments

💻

Streamlit / Power BI

Real-time dashboards for telemetry and performance visualization

Broker & Data Integration

Connected to institutional-grade data feeds and execution

Oanda v20 REST API

Broker
Oanda (Practice Account)
API Version
v20 REST API
Data Granularity
H1 (Hourly Candles)
Active Pairs
EUR/USD, GBP/USD, USD/JPY
Historical Coverage
2008 – Present (18+ Years)
Sync Method
Self-Healing Pagination (5,000 candle batches)

Means of Improvement

Continuous refinement across all system layers

🔄

Strategy Decay Detection

Layer 5 monitors live expectancy. If a promoted strategy begins decaying, it is flagged for demotion back to Layer 0 for re-evaluation.

🎯

Confidence Threshold Tuning

The ML meta-labeler's confidence threshold (currently 0.75) is periodically adjusted based on out-of-sample performance metrics.

📐

Feature Engineering

Expanding the feature set with additional multi-timeframe indicators, volatility surfaces, and cross-asset correlation features.

🧪

New Strategy Research

Continuously developing and testing new algorithmic strategies in the Layer 0 sandbox before promotion to live environments.

🌍

Asset Expansion

Instrument-agnostic architecture allows seamless addition of new forex pairs, commodities, and indices to the pipeline.

Latency Optimization

Moving toward real-time event-driven architecture for faster signal processing and reduced execution slippage.

Development Roadmap

Current progress and upcoming milestones

Layer 0: Strategy Qualification Engine

4 strategies qualified with full backtesting pipeline. Evaluator handles multi-asset, multi-strategy testing.

Layer 1: Market Regime Detection

K-Means clustering implemented. Regime ingestion pipeline writing to SQL database.

Data Infrastructure

Self-healing Oanda sync, SQL Server schema, 18+ years of historical H1 data ingested.

Layer 2: Live Signal Generation Engine

Successful batch processing of SQL data for signal generation from promoted strategies.

Layer 3: Real-Time ML Inference

Optuna hyperparameter tuning for XGBoost, LightGBM, Random Forest, and PyTorch LSTMs.

Layer 4 & 4.5: Risk & Correlation Engine

Implement ATR-based dynamic stops and 30-day rolling correlation matrix.

Layer 5: Telemetry Dashboard

Streamlit/Power BI dashboard with live regimes, veto rates, and expectancy tracking.

Milestone 1 (Current)

Dynamic ATR Exit Optimization & AI Gatekeeper Tuning (March 8-10)

Milestone 2

Live Data Ingestion Automation via CRON (March 12-15)

Milestone 3

Vercel Frontend / API Integration (March 18-20)

Milestone 4

Paper Trading Sandbox Deployment (End of March 2026)