Project ARK
The ML platform that turns a fine-tuned model into a self-improving capability — fine-tuning, eval gates, promotion, and serving, all wired into one loop.
The flywheel
Each stage hands off to the next. Promotion only fires when the eval gate passes — and production traces feed the next training run, closing the loop automatically.
ark.datasetsSynthetic generation or public-set registration with parquet + schema hash on MinIO/S3. Lineage logged as MLflow run.
ark.trainAxolotl YAML → `axolotl train` subprocess → LoRA adapter logged as MLflow fine_tune run tagged with capability, base model, dataset_run_id.
ark.evalRegisters versioned golden set (idempotent on schema hash). External runner scores the new adapter; score becomes the promotion gate.
ark.controlplaneFires only when the eval gate passes: sets MLflow @champion alias on the registry version, writes agent_deployments audit row.
ark.servingDownloads the @champion adapter from MLflow registry, POSTs to vLLM /v1/load_lora_adapter. Production traces carry artifact URIs back to Data.
What's inside
Six Python modules. Each owns one slice of the lifecycle and exposes a narrow API — no cross-module globals, no hidden state.
ark.trainAxolotl fine-tuning → MLflow
Parses an Axolotl YAML, runs `axolotl train` in a subprocess, then logs the LoRA adapter as a `fine_tune` MLflow run tagged with capability, base model, and dataset_run_id (ADR-001).
ark.datasetsSynthetic gen + dataset lineage
synth_gen(...) and register_dataset(parquet, schema_hash) write parquet to MinIO/S3 and record lineage tags as a `synth_gen` MLflow run. Schema hash is content-addressed for idempotency (ADR-009).
ark.evalGolden eval sets + promotion gates
register_eval_set(name, version, parquet, schema_hash) logs the golden set as a stable URI an external runner reads. Idempotent on schema hash. The gate score blocks or allows promotion.
ark.controlplane5 Postgres tables: versioning + audit
Schema (5 tables, URIs only — no inline manifests), store (write versions/bindings/gate results), promote (set @champion alias + write agent_deployments audit row) (ADR-006/011).
ark.serving.deploy_loraMLflow → vLLM adapter sync
Downloads the @champion LoRA adapter from the MLflow registry and POSTs to vLLM's /v1/load_lora_adapter. Non-zero exit with the server's error message on a 4xx (ADR-005).
ark.mlflow_schema12 required run tags enforced
Enforces the 12 standard run tags from schemas/mlflow-run-schema.yaml. Rejects a run missing a required tag or carrying an out-of-enum value — before it hits the tracking server.
Demo: intent classifier
A customer-support intent classifier fine-tuned on the public Bitext dataset (26,872 labeled pairs, 27 intents across 10 categories). Base model: Qwen2.5-0.5B or 1.5B. No GPU required.
26,872 labeled pairs, 27 intents, 10 categories from the public Bitext dataset (CDLA-Sharing-1.0).
ark.datasets.register_dataset(
parquet='bitext-train.parquet',
schema_hash='sha256:a1b2...'
)
# → logs synth_gen MLflow run, uploads to MinIOBase model: Qwen2.5-0.5B or 1.5B. No GPU required — runs on Apple M-series or any x86-64 CPU.
ark.train.run(
config='axolotl/qwen2.5-0.5b-intent.yaml',
capability='intent-classifier'
)
# → LoRA adapter logged as fine_tune MLflow runPromotion only fires when the eval score exceeds the registered gate threshold (0.85 in the demo).
# External runner scores against golden set
# score: 0.91 threshold: 0.85
ark.controlplane.promote(
capability='intent-classifier',
run_id='abc123'
)
# → sets @champion alias, writes audit rowProduction traces carry the model_uri, adapter_uri, and prompt_uri back into the dataset for the next round.
ark.serving.deploy_lora(
capability='intent-classifier@champion'
)
# → downloads adapter, POSTs to vLLM
# → agent-waves resolves @champion at request timeGet started
Clone the repo and run locally in three steps. Requires Python 3.11+, uv, and Docker for the local services (MLflow, MinIO, Postgres).
# 1. Clone and install
git clone https://github.com/utsavanand/project-ark
cd project-ark
uv sync --all-extras
# 2. Start local services (MLflow, MinIO, Postgres)
docker compose -f infra/docker-compose.yml up -d
# 3. Run the intent-classifier demo
uv run python examples/intent-agent/run_flywheel.pyNote: pip install is planned for a future release. For now, clone the repo and use uv sync --all-extras — this installs all optional extras (Axolotl, vLLM adapters, eval tooling). Connection strings for the local services are in infra/README.md.