The EG-TD3 Machine Learning Architecture: Evolutionary-Guided Twin Delayed Deep Deterministic Policy Gradient

The EG-TD3 Machine Learning Architecture: Evolutionary-Guided Twin Delayed Deep Deterministic Policy Gradient

Djambong Tenkeu H.-D. (NRU HSE, Moscow, Russia)

Abstract

This paper presents the Evolutionary-Guided Twin Delayed Deep Deterministic Policy Gradient (EG-TD3), a hybrid learning architecture that couples TD3 policy improvement with Differential Evolution for adaptive packet-buffer control under non-stationary traffic. Through empirical evaluation on a standard continuous-control benchmark (Pendulum-v1) and a synthetic traffic-burst queue simulator, we show that EG-TD3 improves training stability (coefficient of variation 0.005 versus 0.258 for TD3) and action smoothness (mean action delta 0.052 versus 0.164 for TD3) while maintaining competitive robustness under observation and action noise. The architecture combines periodic elite transplantation and fitness-guided exploration. Beyond learning results, we outline a service-mesh deployment path based on a split control/data plane. Full EG-TD3 training and multi-layer inference are kept in userspace next to the mesh control plane, because in-kernel machine learning inside eBPF remains constrained by verifier and instruction-set limits. On the node data plane, eBPF programs attached at TC or socket hooks, operating alongside a sidecar proxy such as Envoy, collect buffering and traffic telemetry into BPF maps and enforce quantized control decisions (target buffer size, timeout, and urgency thresholds) at control-loop frequency (10 – 100 Hz), without training on the packet path. Shared BPF maps mediate the interaction: eBPF writes telemetry; the EG-TD3 agent reads state and writes control parameters; eBPF applies the updated policy. Offline or sidecar training on logged trajectories does not block the fast path. An open-source implementation and full hyperparameter specifications are provided for reproducibility.

Keywords

network control; adaptive buffering; deep reinforcement learning; EG-TD3; twin delayed deep deterministic policy gradient (TD3); differential evolution (DE).

Edition

Proceedings of the Institute for System Programming, vol. 38, issue 4, part 1, 2026, pp. 135-152

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2026-38(4)-7

For citation

Djambong Tenkeu H.-D. The EG-TD3 Machine Learning Architecture: Evolutionary-Guided Twin Delayed Deep Deterministic Policy Gradient. Proceedings of the Institute for System Programming, vol. 38, issue 4, part 1, 2026, pp. 135-152 DOI: 10.15514/ISPRAS-2026-38(4)-7.

Full text of the paper in pdf Back to the contents of the volume