499 free skills
Data Engineering skills
Skills for data engineering — ETL pipelines, Airflow DAGs, Spark jobs, and data warehouse modeling.
Sourced from real, public repositories — synced daily, never invented.
499 free skills
Skills for data engineering — ETL pipelines, Airflow DAGs, Spark jobs, and data warehouse modeling.
Sourced from real, public repositories — synced daily, never invented.
15 tools across six categories
13 of them never send your data anywhere
Free · No signup · No trial clock
SEE THE DIRECTORY
data-pipeline
claude-office-skills/skills
Data pipeline and ETL automation - extract, transform, load workflows for data integration and analytics
ETL Pipeline
claude-office-skills/skills
Design and automate Extract, Transform, Load data pipelines for data integration and analytics
data-throughput-accelerator
affaan-m/ecc
Use when large data ingestion, backfill, export, ETL, warehouse loading, manifest catch-up, or table synchronization needs to become much faster while preserving data correctness.
ads-tiktok
agricidaniel/claude-ads
Audit TikTok Ads measurement, Pixel and Events API, mobile-first creative, audiences, Smart+, Shop and commerce campaigns, bidding, budgets, pacing, attribution, and policy. Use for TikTok Ads, TikTok Pixel, Events API, Smart+, TikTok Shop Ads, GMV Max, Spark Ads, or TikTok campaign optimization.
data-analytics
markdown-viewer/skills
Create data pipeline and analytics architecture diagrams using PlantUML syntax with database/analytics stencil icons. Best for ETL pipelines, data lakes, real-time streaming, data warehousing, and BI dashboard design.
accelerated-computing-cudf
nvidia/skills
Official NVIDIA-authored guidance for NVIDIA cuDF GPU DataFrames, pandas acceleration, dask-cuDF, ETL, joins, groupby, CSV/Parquet I/O, nullable semantics, and multi-GPU DataFrame workloads.
netlify-deploy
openai/skills
Deploy web projects to Netlify using the Netlify CLI (`npx netlify`). Use when the user asks to deploy, host, publish, or link a site/repo on Netlify, including preview and production deploys.
netlify-blobs
netlify/context-and-tools
Store and retrieve unstructured objects, file uploads, and cache-like state on Netlify using the @netlify/blobs key/value API from Functions, Edge Functions, and Build Plugins. Use when a task involves saving user file or image uploads, persisting form or contact-form submissions, storing generated output from Background Functions (sitemaps/processed media/bulk-email results), building read-only asset stores, adding client-side blob expiration, or wiring file-based blob uploads at deploy time. Not for per-user, transactional, or relational data (counters/balances/sessions) — reach for Netlify DB there instead.
google-cloud-solution-agentic-analytics-spark-knowledge-catalog
google/skills
>-
scoping-changes
riekelt/principal-engineer
Use when deciding how big a fix should be, when scope is drifting mid-task, when a plan is being quietly trimmed to fit, or when "while you're at it" appears in any form. Encodes size-to-trigger, decompose-not-descope, and drive-to-completion. Use whenever the work is about to grow past its cause or shrink below its promise.
analyzing-data
astronomer/agents
Queries the data warehouse with SQL and answers business questions about data. Use when answering anything that needs warehouse data - counts, metrics, trends, aggregations, joins across tables, data lookups, or ad-hoc SQL analysis (for example "who uses X", "how many Y", "show me Z", "find customers", "what is the count").
pytorch-lightning
k-dense-ai/scientific-agent-skills
Deep learning framework (PyTorch Lightning / lightning package). Organize PyTorch code into LightningModules, configure Trainers for multi-GPU/TPU, implement data pipelines, callbacks, logging (W&B, TensorBoard, MLflow), distributed training (DDP, FSDP, DeepSpeed), for scalable neural network training.
creator-ugc-marketing
eronred/aso-skills
When the user wants to plan, brief, source, or measure organic creator / influencer / UGC marketing for their app — including TikTok creators, Instagram Reels, YouTube Shorts, micro-influencers, paid creator briefs, UGC ad creative for Meta/TikTok, affiliate programs, and seeding strategy. Use when the user mentions "creators", "influencers", "TikTok marketing", "UGC", "user-generated content", "creator briefs", "creator outreach", "influencer marketing", "Reels", "Shorts", "creator gifting", "seeding", "affiliate", "Whop", "TikTok Spark Ads", "Instagram collab posts", or "I want my app to go viral on TikTok". For paid social ad campaigns (Meta/TikTok ad accounts), see ua-campaign. For PR / press, see press-and-pr. For viral in-app loops, see referral-program.
parallel-task-spark
am-will/codex-skills
>
super-swarm-spark
am-will/codex-skills
>
tiktok-ads
kostja94/marketing-skills
When the user wants to set up, optimize, or manage TikTok Ads. Also use when the user mentions "TikTok Ads," "TikTok for Business," "TikTok Pixel," "Events API," "TikTok Spark Ads," or "TikTok video ads." For organic TikTok, use tiktok-captions.
migrating-airflow-2-to-3
astronomer/agents
Guide for migrating Apache Airflow 2.x projects to Airflow 3.x. Use when the user mentions Airflow 3 migration, upgrade, compatibility issues, breaking changes, or wants to modernize their Airflow codebase. If you detect Airflow 2.x code that needs migration, prompt the user and ask if they want you to help upgrade. Always load this skill as the first step for any migration-related request.
spark-training-gotchas
wshobson/agents
Preflight and diagnose the ten known failure modes for ML training on NVIDIA DGX Spark. Use when a training run on DGX Spark fails to start, OOMs below the 128GB limit, slows down mid-run, or before any multi-hour training job on GB10.
analytics-data-analysis
mindrally/skills
Best practices for analytics, data analysis, and visualization using Python, pandas, matplotlib, seaborn, and Jupyter notebooks. Use when performing exploratory data analysis, building data pipelines, creating statistical visualizations, writing Jupyter notebooks, cleaning and transforming datasets, or implementing analytics dashboards.
databricks-pipelines
databricks/databricks-agent-skills
Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL. Invoke BEFORE starting implementation.
data-engineer
sickn33/agentic-awesome-skills
Build scalable data pipelines, modern data warehouses, and real-time streaming architectures. Implements Apache Spark, dbt, Airflow, and cloud-native data platforms.
managed-airflow-migrations
google/skills
Provides guidance for migrating Apache Airflow DAGs in Managed Service for Apache Airflow (MSAA; formerly Cloud Composer). Covers migration to Airflow 2.11.1 (MSAA Gen 2 and 3) and Airflow 3 (MSAA Gen 3), including environment inspection, GCS download/upload and scanning patterns for breaking changes. Use when migrating the DAG code to newer Airflow version. Don't use when checking DAG run failures unrelated to code migration.
warehouse-optimization
nexscope-ai/ecommerce-skills
E-commerce warehouse and inventory optimization advisor. Analyzes inventory health, calculates safety stock and reorder points, performs ABC analysis, evaluates fulfillment costs, and provides actionable recommendations for improving efficiency. Supports all major fulfillment models: Self-fulfillment, Amazon FBA/FBM, Walmart WFS, 3PL, Shopify Fulfillment, TikTok Shop, Dropshipping, and Hybrid setups. No API key required. Use when: (1) reducing stockouts or overstock, (2) calculating safety stock levels, (3) optimizing warehouse costs, (4) improving Amazon IPI score, (5) analyzing inventory KPIs.
ui-craft-dense-dashboard
educlopez/ui-craft
Dense dashboard / admin / Bloomberg / Retool / data-heavy internal tools. Locked knobs: CRAFT=7, MOTION=3, DENSITY=9. IBM Plex + mono numbers, semantic palette, 4/8px grid, sparklines, tabular-nums. Trigger on: dashboard, admin panel, data-dense, analytics, Bloomberg-like, Retool-like.
annotating-task-lineage
astronomer/agents
Annotate Airflow tasks with data lineage using inlets and outlets. Use when the user wants to add lineage metadata to tasks, specify input/output datasets, or enable lineage tracking for operators without built-in OpenLineage extraction.
content-amplifier
aaron-he-zhu/aaron-marketing-skills
Use when the user asks to "amplify influencer content with paid media", "set up whitelisting or Spark Ads", "decide which posts to boost", "repurpose influencer content", "turn one video into multiple ads", or "build a UGC asset library"; produces (paid mode) a content-selection scorecard, a paid amplification strategy (whitelisting/boosting/dark posts), audience targeting, and a budget+optimization plan, or (repurpose mode) a rights-tracked content inventory, a 1-video-to-10+-asset repurposing map, per-format transformation specs, and a 30-day distribution plan. Not for gating whether a deliverable is publishable or FTC-compliant — use creator-content-auditor; not for the always-on brand posting calendar — use social-calendar-builder; not for drafting a net-new idea into platform-native packages — use social-creative-builder. 复用达人内容 / 内容放量.
creating-openlineage-extractors
astronomer/agents
Create custom OpenLineage extractors for Airflow operators. Use when the user needs lineage from unsupported or third-party operators, wants column-level lineage, or needs complex extraction logic beyond what inlets/outlets provide.
cosmos-dbt-core
astronomer/agents
Turns a dbt Core project into an Airflow DAG/TaskGroup using Astronomer Cosmos. Use turning a dbt Core project into an Airflow DAG or TaskGroup with Astronomer Cosmos. Before implementing, verify dbt engine, warehouse, Airflow version, execution environment, DAG vs TaskGroup, and manifest availability.
Data Cleaning Pipeline
aj-geddes/useful-ai-prompts
Build robust processes for data cleaning, missing value imputation, outlier handling, and data transformation for data preprocessing, data quality, and data pipeline automation
spark
wishworldbetter/seedex-skills
Use when the user wants to brainstorm an idea or design a feature/spec. Explores intent and requirements through dialogue, then writes a spec document to docs/spark/ and STOPS. Does not auto-chain to implementation planning or any other skill.
managed-airflow-dag-authoring
google/skills
>-
warehouse-init
astronomer/agents
Initialize warehouse schema discovery. Generates .astro/warehouse.md with all table metadata for instant lookups. Run once per project, refresh when schema changes. Use when user says "/astronomer-data:warehouse-init" or asks to set up data discovery.
sap-datasphere
secondsky/sap-skills
SAP Datasphere development skill with 3 specialized agents, 5 slash commands, and validation hooks. Use when building data warehouses on SAP BTP, creating analytic models, configuring data flows and replication flows, setting up connections, managing spaces and users, implementing data access controls, using the datasphere CLI, or inspecting authenticated Datasphere browser UI state with Microsoft Edge CDP. Covers Data Builder, Business Builder, analytic models, 40+ connection types, real-time replication, task chains, content transport, and data marketplace.
neo4j-spark-skill
neo4j-contrib/neo4j-skills
Use when reading from or writing to Neo4j with Apache Spark or Databricks using the
deploying-airflow
astronomer/agents
Deploys Airflow DAGs and projects. Use when deploying Airflow or answering anything about deployment - deploying DAGs/projects, pushing code, setting up CI/CD, deploying to production or deployment strategies for Airflow.
developing-in-lightdash
lightdash/lightdash
Use when working with Lightdash YAML files, dbt models with Lightdash metadata, the lightdash CLI (deploy, upload, download, preview, lint, warehouse-catalog, sql, set-warehouse, apps create/preview/validate), or managing charts, dashboards, spaces and access, AI agents, scheduled content, data apps, organization Data App themes, data-app external connections, users, groups, custom roles, metrics, and dimensions as code
ml-pipeline-automation
secondsky/claude-skills
Automate ML workflows with Airflow, Kubeflow, MLflow. Use for reproducible pipelines, retraining schedules, MLOps, or encountering task failures, dependency errors, experiment tracking issues.
particle-system
iart-ai/webgl-animation-skills
This skill should be used when the user asks to "build a particle system", "make confetti/snow/smoke/sparks", "create a connected-dot/constellation network background", "add a flow-field or curl-noise particle effect", "render thousands of GPU particles with Three.js Points", or "animate emitters with forces". Covers per-particle integration, forces, flow fields, burst/continuous emission, spatial-grid connected dots, and GPU points + shaders.
motherduck-build-data-pipeline
motherduckdb/agent-skills
Design an end-to-end MotherDuck data pipeline. Use for ETL/ELT workflows -- choosing raw, staging, and analytics boundaries, bulk ingestion paths, transformation sequencing, dlt/dbt integration, publication targets, or whether DuckLake is actually required.
airflow-dag-patterns
sickn33/agentic-awesome-skills
Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.
ae-dataops
thinkingaiagenticengine/ae-cli
AE Data Development and Operations: Data warehouse management, flow orchestration, IDE queries, and data integration
omx
akillness/jeo-skills
Use when you need multi-agent orchestration for OpenAI Codex CLI, especially Codex equivalents of Claude `/team`, `/autopilot`, `/ultrawork`, and `/ultraqa`. Triggers on: omx, $plan, $ralph, $team, $ulw, $ultraqa, $autopilot, $deep-interview, $prometheus-strict. v0.18.14 — 30+ agents, 35+ workflow skills, tmux team runtime, sparkshell, explore, ralplan, prometheus-strict, best-practice-research, autoresearch-goal.
use-spark
readdle/spark-cli-skills
>-
kafka-engineer
404kidwiz/claude-supercode-skills
Expert in Apache Kafka, Event Streaming, and Real-time Data Pipelines. Specializes in Kafka Connect, KSQL, and Schema Registry.
airflow-plugins
astronomer/agents
Builds Airflow 3.1+ plugins that embed FastAPI apps, custom UI pages, React components, middleware, macros, and operator links directly into the Airflow UI. Use when building anything custom inside Airflow 3.1+ that involves Python and a browser-facing interface - creating an Airflow plugin, adding a custom UI page or nav entry, building FastAPI-backed endpoints inside Airflow, serving static assets from a plugin, embedding a React app, adding middleware to the API server, creating custom operator extra links, or calling the Airflow REST API from inside a plugin; also when AirflowPlugin, fastapi_apps, external_views, react_apps, or plugin registration come up.
spark-prairie-fire
hughyau/qiushi-skill
|
python-data-pipeline-designer
jorgealves/agent_skills
Design ETL workflows with data validation using tools like Pandas, Dask, or PySpark. Use when building robust data processing systems in Python.
snowflake-expert
personamanagmentlayer/pcl
Expert-level Snowflake data warehouse platform, virtual warehouses, data sharing, streams, tasks, and SQL optimization
etl-sync-job-builder
patricio0312rev/skills
Designs reliable ETL and data synchronization jobs with incremental updates, idempotency guarantees, watermark tracking, error handling, and retry logic. Use for "ETL jobs", "data sync", "incremental sync", or "data pipeline".
databricks-expert
personamanagmentlayer/pcl
Expert-level Databricks platform, Apache Spark, Delta Lake, MLflow, notebooks, and cluster management
spark-recipe-inbox-by-category
readdle/spark-cli-skills
>-
data-quality-audit
nimrodfisher/data-analytics-skills
Comprehensive data quality assessment against business rules, schema constraints, and freshness expectations. Activate when validating data pipeline outputs before production use, auditing a dataset against defined business rules, or producing a quality scorecard for a data asset.
spark-recipe-end-of-day
readdle/spark-cli-skills
>-
spark-recipe-morning-standup
readdle/spark-cli-skills
>-
data-pipeline-engineer
curiositech/some_claude_skills
Expert data engineer for ETL/ELT pipelines, streaming, data warehousing. Activate on: data pipeline, ETL, ELT, data warehouse, Spark, Kafka, Airflow, dbt, data modeling, star schema, streaming
spark-recipe-multi-account-review
readdle/spark-cli-skills
>-
spark-recipe-weekly-digest
readdle/spark-cli-skills
>-
spark-recipe-meeting-prep
readdle/spark-cli-skills
>-
blueprint
astronomer/agents
Define reusable Airflow task group templates with Pydantic validation and compose DAGs from YAML. Use when creating blueprint templates, composing DAGs from YAML, validating configurations, or enabling no-code DAG authoring for non-engineers.
spark-recipe-calendar-audit
readdle/spark-cli-skills
>-