02 / Applied AI engineering · 2026
Human-in-the-loop GenAI data preparation
Deterministic data-quality checks first; LLM recommendations second.
This project treats the LLM as a bounded recommendation layer rather than a statistical oracle. Target leakage checks, split strategy and transformations remain explicit and testable, while Pydantic contracts and repair loops constrain model responses.
Evidence register
81/81
tests
Verified locally with an isolated test configuration
4
workflow stages
Profile, recommend, validate and approve
3×
repair budget
Schema-validated LLM retries
01 / Problem
Generic preprocessing automation can silently introduce leakage, fit transforms on test data or apply plausible-looking choices without an auditable reason.
02 / Approach
The workflow separates deterministic statistical rules from LLM suggestions, versions prompts in YAML, validates structured outputs with Pydantic and pauses for human approval.
03 / Outcome
A CLI, Streamlit interface and FastAPI surface share the same pipeline. A local SQLite cache makes repeated development runs faster and avoids duplicate model calls.
System trace
How the evidence is produced.
Built the agent graph, deterministic safeguards, structured-response layer, API, interface, experiment tracking hooks and automated tests.
- 01Dataset contract → deterministic profiling
- 02LLM recommendation → Pydantic validation
- 03Transformation plan → human approval
- 04Train-only fitting → reproducible artifacts
Validation scope
The audited test run passed all 81 tests. Tests cover statistical fallbacks, split strategies, target checks, response contracts and workflow behavior.
Known limitation
Data samples can be sent to an external model when that provider is enabled. Production use requires explicit consent, redaction, path/URL restrictions and a retention policy.
What is inspectable
- Prompts and model configuration live outside application code.
- Human review is a first-class graph state rather than an interface afterthought.
- Locked dependencies and a dense deterministic test harness support reproducibility.
Next proof to add
- 01Inject the LLM client so deterministic tests never require an API key.
- 02Add PII detection, redaction, encrypted cache storage and time-based deletion.
- 03Restrict remote URLs and local paths before exposing the API beyond localhost.
Main stack