Notes on reproducible pipelines, caching intermediary results, experiment management, and why a lightweight caching DAG changes how you (and your AI coding agent) build data-science workflows.
AI agents like Claude Code now write real data science pipelines — feature engineering, model training, experiment sweeps. Here's the honest account of where they fail at it, and why a lightweight workflow library removes exactly those failures.
They sound like the same problem. They're not — and mixing them up is why so many ML projects still can't reproduce last week's result.
How data scientists waste hours recomputing steps that never changed — and a lightweight way to fix it.
Your current workflow probably chains several functions together like in the example below. While quick, it likely has many problems: