Data Pipeline Architect
$2.99OfficialDesign robust data pipelines: ingestion patterns, transformation stages, schema evolution, and data quality monitoring.
datadata-pipelineetldata-engineeringschema-evolution· by SkillingMain
What you get
- ✓9-step procedure
- ✓6 pitfalls to avoid
- ✓Installs into 6 tools
- Version
- v1 →
- Last updated
- today
- Length
- 3 min read
- Requires
- Best with a strong model (Claude Sonnet 4)
Works in: Claude Code, Codex, Cline, opencode, OpenClaw, Hermes · Handles multi-file projects
Preview
When to use
Use this skill when designing a new data pipeline, refactoring a brittle ETL process, or planning ingestion for a new data source. It applies to batch, micro-batch, and streaming contexts and covers the full lifecycle: source ingestion, transformation, storage, quality enforcement, and downstream delivery. Reach for it whenever you need a defensible architecture rather than ad-hoc glue scripts.
Inputs to gather
- Source systems and their access patterns (APIs, CDC logs, files, message queues)
- Data volume, velocity, and variety estimates (rows/day, peak rate, schema complexity)
- Freshness requirements for downstream consumers (real-time, hourly, daily)
- Transformation
…
🔒 Buy once ($2.99) to unlock the full playbook, download it, and install it in every tool you use.