Data Pipeline Documenter
NotebooksClaudedata pipelineETLdocumentationdata engineering
Prompt
Document this data pipeline thoroughly so another engineer (or future me) can maintain it. Pipeline name: [PIPELINE NAME] Owner / team: [OWNER] Run schedule: [e.g. nightly at 2am UTC] Critical downstream consumers: [WHO / WHAT DEPENDS ON THIS OUTPUT] Please produce: 1. **Data Flow Diagram** (ASCII/text format showing: source → transforms → destination) 2. **Input Sources** table (name, type, location, expected schema, update frequency) 3. **Output / Destination** table (name, type, location, schema changes from input) 4. **Transform Inventory** (list each transformation with plain-English description) 5. **Business Logic** (document any non-obvious rules or calculations) 6. **Failure Modes** (what breaks and how to detect it) 7. **Runbook** (step-by-step: how to run, how to re-run failed jobs, how to backfill) 8. **Change Log template** (date | change | author | impact) Pipeline code: ```python [PASTE YOUR PIPELINE CODE] ```