499 free skills
Data Engineering skills
Skills for data engineering — ETL pipelines, Airflow DAGs, Spark jobs, and data warehouse modeling.
Sourced from real, public repositories — synced daily, never invented.
499 free skills
Skills for data engineering — ETL pipelines, Airflow DAGs, Spark jobs, and data warehouse modeling.
Sourced from real, public repositories — synced daily, never invented.
15 tools across six categories
13 of them never send your data anywhere
Free · No signup · No trial clock
SEE THE DIRECTORY
deploying-go-sdk-bundles
astronomer/agents
Builds, packs, and deploys compiled Airflow Go SDK bundles so the ExecutableCoordinator can run them. Use when the user wants to compile a Go task bundle, asks about `go build`, `go tool airflow-go-pack`, the AFBNDL01 self-contained executable bundle, packing or inspecting a bundle, placing it under `executables_root`, cross-compiling a bundle for workers, `go-sdk` module versioning/tags/pseudo-versions, or getting the bundle onto an Airflow worker (Docker, Kubernetes, or Astro). For the task code see authoring-go-sdk-tasks; for the shared coordinator settings see configuring-airflow-language-sdks.
data-pipelines
kylelundstedt/dotfiles
Use this skill for data pipeline work — ingestion, transformation, analytics, notebooks, and exploration using a DuckDB-centric stack.
data-profiler
datadrivenconstruction/ddc_skills_for_ai_agents_in_construction
Profile construction data to understand characteristics, distributions, quality metrics, and patterns. Essential for data quality assessment and ETL planning.
etl-pipeline
datadrivenconstruction/ddc_skills_for_ai_agents_in_construction
Build automated ETL (Extract-Transform-Load) pipelines for construction data. Process PDFs, Excel, BIM exports. Generate reports, dashboards, and integrate with other systems. Orchestrate with Airflow or n8n.
airflow-dag
datadrivenconstruction/ddc_skills_for_ai_agents_in_construction
Create Apache Airflow DAGs for construction data pipelines. Orchestrate ETL, validation, and reporting workflows.
migrating-dagster-to-airflow
astronomer/agents
Guide for migrating Dagster projects to Apache Airflow 3 on Astro. Use when the user mentions migrating, converting, or porting Dagster (or Dagster+) code to Airflow or Astro, wants to plan or assess such a migration, or asks what a Dagster construct maps to in Airflow. Covers assets, partitions, schedules, sensors, declarative automation, resources, IO managers, ops/jobs, dbt, Pipes, Components, and Dagster+ platform config. Always load this skill as the first step for any Dagster-to-Airflow request.
spark
simota/agent-skills
Proposing new features leveraging existing data/logic as Markdown specifications. Use when brainstorming new features, product planning, or feature proposals are needed. Does not write code.
etl-patterns
majesticlabs-dev/majestic-marketplace
Production ETL patterns orchestrator. Routes to core reliability patterns and incremental load strategies.
ecommerce-data-warehouse
finsilabs/awesome-ecommerce-skills
Build a commerce data warehouse with star-schema tables, ETL pipelines, and dbt models for BigQuery, Snowflake, or Redshift analytics
data-warehouse
pluginagentmarketplace/custom-plugin-sql
Data warehouse design mastery with star schema, dimensional modeling, fact/dimension tables, slowly changing dimensions, and enterprise best practices. Complete schema examples included.
streaming-data
ancoleman/ai-design-components
Build event streaming and real-time data pipelines with Kafka, Pulsar, Redpanda, Flink, and Spark. Covers producer/consumer patterns, stream processing, event sourcing, and CDC across TypeScript, Python, Go, and Java. When building real-time systems, microservices communication, or data integration pipelines.
idea-spark
microsoft/researchstudio
>-
deploying-java-sdk-bundles
astronomer/agents
Builds and deploys compiled Airflow Java SDK bundles so workers can run them. Use when the user wants to package a JVM task bundle into a JAR, asks about the `org.apache.airflow.sdk` Gradle plugin, `./gradlew bundle`, the Maven shade/BOM setup, fat vs thin JARs, the logging integration artifacts (JPL, SLF4J, Log4j 2, JUL), preview/snapshot builds, or getting the JAR onto an Airflow worker (Docker, Kubernetes, or Astro). For the task code see authoring-java-sdk-tasks; for the Airflow coordinator settings see configuring-airflow-language-sdks.
release-mac-app
steipete/agent-scripts
macOS app release: Sparkle, notarization, GitHub Release, Homebrew, closeout.
warehouse-design
kishorkukreja/awesome-supply-chain
When the user wants to design a warehouse, optimize warehouse layout, determine facility size, or configure storage systems. Also use when the user mentions "warehouse layout," "facility design," "warehouse sizing," "storage systems," "material flow," "pick path design," "dock configuration," or "space utilization." For warehouse location selection, see facility-location-problem. For slotting existing warehouses, see warehouse-slotting-optimization.
openai-netlify-deploy
trailofbits/skills-curated
Deploy web projects to Netlify using the Netlify CLI (`npx netlify`). Use when the user asks
alibabacloud-analyticdb-spark-application-analysis-helper
aliyun/alibabacloud-aiops-skills
>
syncfusion-winforms-sparkline
syncfusion/winforms-ui-components-skills
Implement Syncfusion Windows Forms Sparkline controls for compact data visualization. Use this when working with sparklines, trend displays in condensed format, or high-density data visualization. This skill covers line, column, and WinLoss chart types, data point markers, high/low value highlighting, and lightweight graphical representations in Windows Forms applications.
dag-factory
astronomer/agents
Authors Apache Airflow DAGs declaratively from dag-factory YAML configs. Use when building DAGs declaratively from YAML via dag-factory; creating/editing dag-factory templates/YAML configs,reating/editing dag-factory YAML configs, defaults, dynamic tasks, datasets, or callbacks; or validating dag-factory configurations; upgrading or re-pinning dag-factory.
pyspark-transformer
jeremylongshore/claude-code-plugins-plus-skills
Transform pyspark transformer operations. Auto-activating skill for
cometly
membranedev/application-skills
|
netlify-ops
bm629/agent-skills
>
etl-incremental-patterns
majesticlabs-dev/majestic-marketplace
Incremental data loading patterns including backfill strategies, CDC, timestamp-based loads, and pipeline orchestration.
sendspark
membranedev/application-skills
|
etl-core-patterns
majesticlabs-dev/majestic-marketplace
Core ETL reliability patterns including idempotency, checkpointing, error handling, chunking, retry logic, and logging.
kafka
alphaonedev/openclaw-graph
Apache Kafka is a distributed event streaming platform for real-time data pipelines and streaming applications.
matlab-read-pcb-layout
matlab/matlab-agentic-toolkit
Import Gerber, ODB++, Allegro .brd, .mcm files for PCB boards and IC packages. Inspect nets, layers, shapes, and stackups. TRIGGER: user asks to import, read, or open a PCB layout file. Gerber files use gerberRead or PCBReader; ODB++, Allegro .brd, .mcm, and native formats use pcbFileRead. Also when inspecting nets, layers, components, or stackups from an imported board. Invoke BEFORE writing import code — the query API (cadnet, cadnetList, componentList) is specialized. SKIP: EM analysis or S-parameter extraction (use matlab-analyze-em), PDN/IR-drop analysis (use matlab-analyze-pcb-pdn), building custom geometry (use matlab-assemble-pcb-layout), material/stackup definition only (use matlab-manage-pcb-material).
migrating-ai-sdk-to-common-ai
astronomer/agents
Migrates Airflow projects from airflow-ai-sdk to apache-airflow-providers-common-ai 0.4.0+. Use when replacing airflow-ai-sdk with the official Airflow AI provider - migrating LLM decorators (@task.llm, @task.agent, @task.llm_branch, @task.embed), switching from model strings/objects to connection-based LLM configuration, updating imports from airflow_ai_sdk to the new provider, or upgrading an existing common-ai 0.1.x setup to 0.4.x (multimodal prompts, toolsets, embedding operators); also when common-ai provider, AIP-99, a pydanticai connection or migrating away from airflow-ai-sdk come up.
big-data
pluginagentmarketplace/custom-plugin-data-engineer
Apache Spark, Hadoop, distributed computing, and large-scale data processing for petabyte-scale workloads
warehouse-automation
kishorkukreja/awesome-supply-chain
When the user wants to implement warehouse automation, evaluate automation technologies, or design automated material handling systems. Also use when the user mentions "warehouse robotics," "automated storage," "AS/RS," "goods-to-person," "conveyor systems," "sortation," "AMR," "AGV," "automated picking," or "warehouse automation ROI." For warehouse layout design, see warehouse-design. For order fulfillment, see order-fulfillment.
spark-authoring-cli
microsoft/skills-for-fabric
>
documenting-warehouse-sources
posthog/posthog
Write or update the user-facing posthog.com documentation for a PostHog Data warehouse import source. Use when adding a new source doc, fixing an inconsistent or stub source doc, or standardizing the docs at contents/docs/cdp/sources. Covers the canonical template, shared snippets, the auto-rendered <SourceParameters /> and <SourceTables /> components, frontmatter, and the docsUrl/slug rule that prevents 404s.
update-dataset
owid/etl
End-to-end dataset update workflow with PR creation, snapshot, meadow, garden, and grapher steps. Use when user wants to update a dataset, refresh data, run ETL update, or mentions updating dataset versions.
upgrading-dbt-core
dbt-labs/dbt-agent-skills
Use when a user wants to upgrade, update, or migrate a dbt-core project to the latest version — e.g. "upgrade my dbt project," "migrate this off dbt-core 1.5," "get this project running on the latest dbt," "bump the dbt-core version." Upgrades a dbt-core v1 project (on 1.3, 1.4, 1.5, 1.6, or 1.7) all the way to 1.12, applying the required breaking, behavior, and deprecated changes from a data-driven issue corpus. Inputs — starting_version (the project's current dbt-core minor, one of 1.3/1.4/1.5/1.6/1.7) and adapter_type (snowflake/redshift/bigquery/databricks/spark); both are normally supplied by the caller (e.g. the dbt VS Code extension), with fallbacks described in the skill.
stream
simota/agent-skills
Designing ETL/ELT pipelines, visualizing data flows, selecting batch/streaming approaches, and architecting Kafka/Airflow/dbt systems. Use when building data pipelines or managing data quality.
dgx-spark
fandhe-ai/agent-reference-skills
>
spark-consumption-cli
microsoft/skills-for-fabric
>
etl-pipelines
alphaonedev/openclaw-graph
Design and implement ETL pipelines for extracting, transforming, and loading data in data engineering workflows.
spark
alphaonedev/openclaw-graph
Distributed processing framework for large-scale data sets using in-memory computing.
data-engineering-data-pipeline
rmyndharis/antigravity-skills
You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.
gatsby-ecommerce-netlify
rebyteai-template/rebyte-skills
Gatsby e-commerce starter with styled components for Netlify.
spark-sql-optimizer
jeremylongshore/claude-code-plugins-plus-skills
|
astro-supabase-netlify
rebyteai-template/rebyte-skills
Astro with Supabase integration for Netlify.
airflow-hitl
necatiarslan/airflow-vscode-extension
Use when the user needs human-in-the-loop workflows in Airflow (approval/reject, form input, or human-driven branching). Covers ApprovalOperator, HITLOperator, HITLBranchOperator, HITLEntryOperator. Requires Airflow 3.1+.
auditing-warehouse-source-health
posthog/ai-plugin
>
authoring-dags
necatiarslan/airflow-vscode-extension
Workflow and best practices for writing Apache Airflow DAGs. Use when the user wants to create a new DAG, write pipeline code, or asks about DAG patterns and conventions. For testing and debugging DAGs, see the testing-dags skill.
streams
pluginagentmarketplace/custom-plugin-nodejs
Master Node.js streams for memory-efficient processing of large datasets, real-time data handling, and building data pipelines
tanstack-chat-netlify
rebyteai-template/rebyte-skills
Modern chat app with TanStack Router and Claude AI for Netlify.
airflow-dag-generator
jeremylongshore/claude-code-plugins-plus-skills
Generate airflow dag generator operations. Auto-activating skill for
auditing-warehouse-view-health
posthog/ai-plugin
>
computer-vision-warehouse
kishorkukreja/awesome-supply-chain
When the user wants to apply computer vision in warehouses, detect defects, track packages, count inventory, or automate visual inspection. Also use when the user mentions "computer vision," "image recognition," "object detection," "barcode reading," "package tracking," "quality inspection," "YOLO," "RCNN," "image classification," or "warehouse automation with vision." For general ML, see ml-supply-chain.
resolve-beagle
existential-birds/beagle
Use as the follow-up to brainstorm-beagle when a spec has an Open Questions section (or quietly carries latent gaps) that need closing before planning or implementation can begin. Triggers on: \"resolve the open questions\", \"close the gaps in this spec\", \"research the open items\", \"finalize my spec\", \"make this spec implementation-ready\", \"answer the TBDs\". Also triggers whenever the user points at a brainstorm-beagle spec and asks for research, proposals, or answers to unresolved items. Orchestrates parallel research subagents when available (falls back to inline sequential research otherwise), proposes answers one at a time for user approval, then rewrites the spec in place so it arrives at planning with no known gaps. Does NOT write code, design implementation, or create plans — it only produces a complete spec.
carto-connect-datawarehouse
cartodb/agent-skills
Choose and configure the data warehouse engine connection for CARTO (BigQuery, Snowflake, Redshift, Postgres, Databricks, Oracle).
carto-explore-datawarehouse
cartodb/agent-skills
Discover what's in the connected warehouse — schemas, tables, columns, and CARTO named sources.
wms-developer
daemon-blockint-tech/agentic-enteprises-skill
|
carto-query-datawarehouse
cartodb/agent-skills
Write spatial SQL against the connected warehouse — dialect-specific guidance, performance defaults, and CARTO's query/job execution model.
warehouse-location-optimization
kishorkukreja/awesome-supply-chain
When the user wants to optimize warehouse locations, design warehouse networks, or determine optimal warehouse placement for distribution. Also use when the user mentions "warehouse siting," "warehouse network design," "storage facility location," "fulfillment center location," "regional warehouse optimization," "warehouse consolidation," or "distribution warehouse placement." For general facility location, see facility-location-problem. For distribution centers, see distribution-center-network.
carto-import-export-data
cartodb/agent-skills
Import geospatial files into the data warehouse via CARTO, export results back out, and prepare tilesets for fast map rendering.
csv-pipeline
aaaaqwq/agi-super-team
Process, transform, analyze, and report on CSV and JSON data files. Use when the user needs to filter rows, join datasets, compute aggregates, convert formats, deduplicate, or generate summary reports from tabular data. Works with any CSV, TSV, or JSON Lines file.
warehouse-slotting-optimization
kishorkukreja/awesome-supply-chain
When the user wants to optimize warehouse slot assignments, improve pick efficiency, or design warehouse layouts. Also use when the user mentions "slotting optimization," "slot assignment," "ABC slotting," "pick path optimization," "storage location assignment," "warehouse layout optimization," or "forward pick locations." For picker routing, see picker-routing-optimization. For warehouse design, see warehouse-design.