Software
The engineering platform grew out of a broader interest in building inspectable, reproducible systems for evaluating AI behavior. TraceVox Research houses the research experiments; TraceVox AI focuses on production evaluation and observability.
TraceVox Research
An open research environment for reproducible experiments in trustworthy AI, reinforcement learning, multimodal agents, corruption robustness, adversarial robustness, evaluation, and AI safety. It records complete per-timestep scientific evidence (environment ground truth, clean and delivered observations, corruption state, policy distribution, value, outcome), replays runs exactly, and exports provenance-carrying research bundles as plain machine-readable JSON. Experiments run on the researcher's own compute — locally and fully offline if desired.
Install: pip install "tracevox-ai[rl]" then tracevox local start
tracevox-ai (PyPI)
The public Python package behind TraceVox Research: CLI, compute runner, and the complete local research environment (backend, persistent storage, and UI) — no account or cloud required for local use.
TraceVox AI (Platform)
Engineering work around production AI observability, evaluation, tracing, safety monitoring, and agent/LLM infrastructure: complete traces of what a system did (runs, tool calls, model events, latency, cost), scored evaluations, and findings attributed to their cause. The same reproducibility-first mindset as the research environment, applied to production AI systems.
Evaluation systems
Autonomous Voice-Agent Evaluator
An autonomous Python system that evaluates a production-style AI medical receptionist through real telephone conversations, rather than a scripted benchmark. A speech-to-speech patient simulator (OpenAI Realtime) converses over Twilio Voice with bidirectional Media Streams and a FastAPI backend, with voice activity detection and barge-in handling. Every call produces inspectable evidence: dual-channel recordings, timestamped diarized transcripts, scenario-based evaluation, structured failure attribution (target agent, simulator, shared, infrastructure, inconclusive, none), and human-reviewed findings consolidated into a bug report.
It is a separate system from TraceVox: it uses TraceVox Autonomous Runs as an independent, failure-isolated assurance layer — asynchronous observability that cannot interrupt the live call. The TraceVox Early Warning layer used here is an explainable deterministic system, not a trained predictive model.
Research connection: this evaluator demonstrates engineering around autonomous-agent assurance and observability, while my PhD research investigates learned and calibrated reliability estimation for autonomous multimodal decision-making. The evaluator does not implement the dissertation trust estimator.