250 free skills · page 3 of 5
Data Engineering skills
Skills for data engineering — ETL pipelines, Airflow DAGs, Spark jobs, and data warehouse modeling.
Real files from public repositories — hand-curated, never invented. Sorted by installs.
3 / 5page
authoring-go-sdk-tasksastronomer/agentsWrites Airflow task logic in Go using the Airflow Go SDK. Use when the user wants to implement Airflow tasks in Go, asks about BundleProvider/RegisterDags, the bundlev1 Registry/Dag interfaces, registering Go tasks (AddTask/AddTaskWithName), dependency injection by parameter type (context.Context, sdk.TIRunContext, *slog.Logger, sdk.Client), or reading connections/variables/XComs from Go. This skill covers the Go-specific native API; the shared Python-stub pattern and conceptual model live in authoring-language-sdk-tasks. For building/packing/shipping the bundle see deploying-go-sdk-bundles; for coordinator config see configuring-airflow-language-sdks.Data Engineering81
data-pipelineskylelundstedt/dotfilesUse this skill for data pipeline work — ingestion, transformation, analytics, notebooks, and exploration using a DuckDB-centric stack.Data Engineering81deploying-go-sdk-bundlesastronomer/agentsBuilds, packs, and deploys compiled Airflow Go SDK bundles so the ExecutableCoordinator can run them. Use when the user wants to compile a Go task bundle, asks about go build, go tool airflow-go-pack, the AFBNDL01 self-contained executable bundle, packing or inspecting a bundle, placing it under executables_root, cross-compiling a bundle for workers, go-sdk module versioning/tags/pseudo-versions, or getting the bundle onto an Airflow worker (Docker, Kubernetes, or Astro). For the task code see authoring-go-sdk-tasks; for the shared coordinator settings see configuring-airflow-language-sdks.Data Engineering81
airflow-dagdatadrivenconstruction/ddc_skills_for_ai_agents_in_constructionCreate Apache Airflow DAGs for construction data pipelines. Orchestrate ETL, validation, and reporting workflows.Data Engineering80data-profilerdatadrivenconstruction/ddc_skills_for_ai_agents_in_constructionProfile construction data to understand characteristics, distributions, quality metrics, and patterns. Essential for data quality assessment and ETL planning.Data Engineering80etl-pipelinedatadrivenconstruction/ddc_skills_for_ai_agents_in_constructionBuild automated ETL (Extract-Transform-Load) pipelines for construction data. Process PDFs, Excel, BIM exports. Generate reports, dashboards, and integrate with other systems. Orchestrate with Airflow or n8n.Data Engineering80migrating-dagster-to-airflowastronomer/agentsGuide for migrating Dagster projects to Apache Airflow 3 on Astro. Use when the user mentions migrating, converting, or porting Dagster (or Dagster+) code to Airflow or Astro, wants to plan or assess such a migration, or asks what a Dagster construct maps to in Airflow. Covers assets, partitions, schedules, sensors, declarative automation, resources, IO managers, ops/jobs, dbt, Pipes, Components, and Dagster+ platform config. Always load this skill as the first step for any Dagster-to-Airflow request.Data Engineering77
sparksimota/agent-skillsProposing new features leveraging existing data/logic as Markdown specifications. Use when brainstorming new features, product planning, or feature proposals are needed. Does not write code.Data Engineering75
etl-patternsmajesticlabs-dev/majestic-marketplaceProduction ETL patterns orchestrator. Routes to core reliability patterns and incremental load strategies.Data Engineering74
ecommerce-data-warehousefinsilabs/awesome-ecommerce-skillsBuild a commerce data warehouse with star-schema tables, ETL pipelines, and dbt models for BigQuery, Snowflake, or Redshift analyticsData Engineering73
data-warehousepluginagentmarketplace/custom-plugin-sqlData warehouse design mastery with star schema, dimensional modeling, fact/dimension tables, slowly changing dimensions, and enterprise best practices. Complete schema examples included.Data Engineering71
streaming-dataancoleman/ai-design-componentsBuild event streaming and real-time data pipelines with Kafka, Pulsar, Redpanda, Flink, and Spark. Covers producer/consumer patterns, stream processing, event sourcing, and CDC across TypeScript, Python, Go, and Java. When building real-time systems, microservices communication, or data integration pipelines.Data Engineering68
idea-sparkmicrosoft/researchstudioConvert an under-specified research direction into ONE reviewer-defensible Oral-level research proposal — grounded in 1947 ICLR/ICML/NeurIPS papers (2021-2025) — via a 5-phase workflow: retrieve recent literature, diagnose the bottleneck, select + generate a candidate using corpus-derived ideation…Data Engineering67deploying-java-sdk-bundlesastronomer/agentsBuilds and deploys compiled Airflow Java SDK bundles so workers can run them. Use when the user wants to package a JVM task bundle into a JAR, asks about the org.apache.airflow.sdk Gradle plugin, ./gradlew bundle, the Maven shade/BOM setup, fat vs thin JARs, the logging integration artifacts (JPL, SLF4J, Log4j 2, JUL), preview/snapshot builds, or getting the JAR onto an Airflow worker (Docker, Kubernetes, or Astro). For the task code see authoring-java-sdk-tasks; for the Airflow coordinator settings see configuring-airflow-language-sdks.Data Engineering64
release-mac-appsteipete/agent-scriptsmacOS app release: Sparkle, notarization, GitHub Release, Homebrew, closeout.Data Engineering64
alibabacloud-analyticdb-spark-application-analysis-helperaliyun/alibabacloud-aiops-skillsAnalyze and troubleshoot AnalyticsDB Spark applications via aliyun CLI and local analysis scripts. You are a Spark-savvy SRE—not just an API caller, but someone who knows how to diagnose failures, detect performance anomalies, and compare execution plans across similar applications.Data Engineering63
warehouse-designkishorkukreja/awesome-supply-chainWhen the user wants to design a warehouse, optimize warehouse layout, determine facility size, or configure storage systems. Also use when the user mentions "warehouse layout," "facility design," "warehouse sizing," "storage systems," "material flow," "pick path design," "dock configuration," or "space utilization." For warehouse location selection, see facility-location-problem. For slotting existing warehouses, see warehouse-slotting-optimization.Data Engineering63
syncfusion-winforms-sparklinesyncfusion/winforms-ui-components-skillsImplement Syncfusion Windows Forms Sparkline controls for compact data visualization. Use this when working with sparklines, trend displays in condensed format, or high-density data visualization. This skill covers line, column, and WinLoss chart types, data point markers, high/low value highlighting, and lightweight graphical representations in Windows Forms applications.Data Engineering63
openai-netlify-deploytrailofbits/skills-curatedDeploy web projects to Netlify using the Netlify CLI (npx netlify). Use when the user asksData Engineering63dag-factoryastronomer/agentsAuthors Apache Airflow DAGs declaratively from dag-factory YAML configs. Use when building DAGs declaratively from YAML via dag-factory; creating/editing dag-factory templates/YAML configs,reating/editing dag-factory YAML configs, defaults, dynamic tasks, datasets, or callbacks; or validating dag-factory configurations; upgrading or re-pinning dag-factory.Data Engineering61
pyspark-transformerjeremylongshore/claude-code-plugins-plus-skillsTransform pyspark transformer operations. Auto-activating skill forData Engineering61
cometlymembranedev/application-skillsCometly is an AI-powered marketing attribution platform. It helps marketers and agencies track and analyze the performance of their marketing campaigns across different channels. This allows them to optimize their ad spend and improve ROI.Data Engineering60
netlify-opsbm629/agent-skillsThis skill lets an agent perform Netlify web-hosting operations CLI-first (the netlify CLI) with a REST fallback on Netlify's official API — no SDK.Data Engineering58etl-incremental-patternsmajesticlabs-dev/majestic-marketplaceIncremental data loading patterns including backfill strategies, CDC, timestamp-based loads, and pipeline orchestration.Data Engineering56sendsparkmembranedev/application-skillsSendspark is a video messaging tool for sales, marketing, and support teams. It allows users to record and share short videos for personalized communication. This helps to build relationships and improve customer engagement.Data Engineering56etl-core-patternsmajesticlabs-dev/majestic-marketplaceCore ETL reliability patterns including idempotency, checkpointing, error handling, chunking, retry logic, and logging.Data Engineering55
kafkaalphaonedev/openclaw-graphApache Kafka is a distributed event streaming platform for real-time data pipelines and streaming applications.Data Engineering54
matlab-read-pcb-layoutmatlab/matlab-agentic-toolkitImport Gerber, ODB++, Allegro .brd, .mcm files for PCB boards and IC packages. Inspect nets, layers, shapes, and stackups. TRIGGER: user asks to import, read, or open a PCB layout file. Gerber files use gerberRead or PCBReader; ODB++, Allegro .brd, .mcm, and native formats use pcbFileRead. Also when inspecting nets, layers, components, or stackups from an imported board. Invoke BEFORE writing import code — the query API (cadnet, cadnetList, componentList) is specialized. SKIP: EM analysis or S-parameter extraction (use matlab-analyze-em), PDN/IR-drop analysis (use matlab-analyze-pcb-pdn), building custom geometry (use matlab-assemble-pcb-layout), material/stackup definition only (use matlab-manage-pcb-material).Data Engineering54migrating-ai-sdk-to-common-aiastronomer/agentsMigrates Airflow projects from airflow-ai-sdk to apache-airflow-providers-common-ai 0.4.0+. Use when replacing airflow-ai-sdk with the official Airflow AI provider - migrating LLM decorators (@task.llm, @task.agent, @task.llm_branch, @task.embed), switching from model strings/objects to connection-based LLM configuration, updating imports from airflow_ai_sdk to the new provider, or upgrading an existing common-ai 0.1.x setup to 0.4.x (multimodal prompts, toolsets, embedding operators); also when common-ai provider, AIP-99, a pydanticai connection or migrating away from airflow-ai-sdk come up.Data Engineering53spark-authoring-climicrosoft/skills-for-fabricThis skill covers two complementary areas: (1) managing Fabric Spark artifacts via REST APIs (workspaces, lakehouses, notebooks, jobs, pipelines) and (2) writing code inside Fabric Notebook cells (PySpark, Scala, SparkR, SQL with correct lakehouse access, notebookutils, and Spark configuration).Data Engineering50warehouse-automationkishorkukreja/awesome-supply-chainWhen the user wants to implement warehouse automation, evaluate automation technologies, or design automated material handling systems. Also use when the user mentions "warehouse robotics," "automated storage," "AS/RS," "goods-to-person," "conveyor systems," "sortation," "AMR," "AGV," "automated picking," or "warehouse automation ROI." For warehouse layout design, see warehouse-design. For order fulfillment, see order-fulfillment.Data Engineering50big-datapluginagentmarketplace/custom-plugin-data-engineerApache Spark, Hadoop, distributed computing, and large-scale data processing for petabyte-scale workloadsData Engineering50
documenting-warehouse-sourcesposthog/posthogWrite or update the user-facing posthog.com documentation for a PostHog Data warehouse import source. Use when adding a new source doc, fixing an inconsistent or stub source doc, or standardizing the docs at contents/docs/cdp/sources. Covers the canonical template, shared snippets, the auto-rendered <SourceParameters /> and <SourceTables /> components, frontmatter, and the docsUrl/slug rule that prevents 404s.Data Engineering50
update-datasetowid/etlEnd-to-end dataset update workflow with PR creation, snapshot, meadow, garden, and grapher steps. Use when user wants to update a dataset, refresh data, run ETL update, or mentions updating dataset versions.Data Engineering49
upgrading-dbt-coredbt-labs/dbt-agent-skillsUse when a user wants to upgrade, update, or migrate a dbt-core project to the latest version — e.g. "upgrade my dbt project," "migrate this off dbt-core 1.5," "get this project running on the latest dbt," "bump the dbt-core version." Upgrades a dbt-core v1 project (on 1.3, 1.4, 1.5, 1.6, or 1.7) all the way to 1.12, applying the required breaking, behavior, and deprecated changes from a data-driven issue corpus. Inputs — starting_version (the project's current dbt-core minor, one of 1.3/1.4/1.5/1.6/1.7) and adapter_type (snowflake/redshift/bigquery/databricks/spark); both are normally supplied by the caller (e.g. the dbt VS Code extension), with fallbacks described in the skill.Data Engineering49
dgx-sparkfandhe-ai/agent-reference-skillsタスクからカテゴリを引き、カテゴリの README.md で目的のページを特定する:Data Engineering47etl-pipelinesalphaonedev/openclaw-graphDesign and implement ETL pipelines for extracting, transforming, and loading data in data engineering workflows.Data Engineering47sparkalphaonedev/openclaw-graphDistributed processing framework for large-scale data sets using in-memory computing.Data Engineering47spark-consumption-climicrosoft/skills-for-fabricApply environment detection from COMMON-CORE.md Environment Detection Pattern to set: - $FABRIC_API_BASE and $FABRIC_RESOURCE_SCOPE - $FABRIC_API_URL and $LIVY_API_PATH for Livy operationsData Engineering47streamsimota/agent-skillsDesigning ETL/ELT pipelines, visualizing data flows, selecting batch/streaming approaches, and architecting Kafka/Airflow/dbt systems. Use when building data pipelines or managing data quality.Data Engineering47
data-engineering-data-pipelinermyndharis/antigravity-skillsYou are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.Data Engineering46spark-sql-optimizerjeremylongshore/claude-code-plugins-plus-skillsThis skill provides automated assistance for spark sql optimizer tasks within the Data Pipelines domain.Data Engineering45
gatsby-ecommerce-netlifyrebyteai-template/rebyte-skillsGatsby e-commerce starter with styled components for Netlify.Data Engineering45astro-supabase-netlifyrebyteai-template/rebyte-skillsAstro with Supabase integration for Netlify.Data Engineering44
airflow-hitlnecatiarslan/airflow-vscode-extensionUse when the user needs human-in-the-loop workflows in Airflow (approval/reject, form input, or human-driven branching). Covers ApprovalOperator, HITLOperator, HITLBranchOperator, HITLEntryOperator. Requires Airflow 3.1+.Data Engineering42auditing-warehouse-source-healthposthog/ai-pluginThis skill produces a project-wide audit of the source and sync side of the data warehouse pipeline — source connections, sync schemas, and webhook push channels.Data Engineering41airflow-dag-generatorjeremylongshore/claude-code-plugins-plus-skillsGenerate airflow dag generator operations. Auto-activating skill forData Engineering41streamspluginagentmarketplace/custom-plugin-nodejsMaster Node.js streams for memory-efficient processing of large datasets, real-time data handling, and building data pipelinesData Engineering41tanstack-chat-netlifyrebyteai-template/rebyte-skillsModern chat app with TanStack Router and Claude AI for Netlify.Data Engineering41authoring-dagsnecatiarslan/airflow-vscode-extensionWorkflow and best practices for writing Apache Airflow DAGs. Use when the user wants to create a new DAG, write pipeline code, or asks about DAG patterns and conventions. For testing and debugging DAGs, see the testing-dags skill.Data Engineering41auditing-warehouse-view-healthposthog/ai-pluginThis skill produces a project-wide audit of materialized views (materialized saved queries) in the data warehouse — which ones are failing, and which are materialized but unused. Use it when the user wants a summary of view health, not a deep-dive on one failure.Data Engineering40computer-vision-warehousekishorkukreja/awesome-supply-chainWhen the user wants to apply computer vision in warehouses, detect defects, track packages, count inventory, or automate visual inspection. Also use when the user mentions "computer vision," "image recognition," "object detection," "barcode reading," "package tracking," "quality inspection," "YOLO," "RCNN," "image classification," or "warehouse automation with vision." For general ML, see ml-supply-chain.Data Engineering40
resolve-beagleexistential-birds/beagleUse as the follow-up to brainstorm-beagle when a spec has an Open Questions section (or quietly carries latent gaps) that need closing before planning or implementation can begin. Triggers on: \"resolve the open questions\", \"close the gaps in this spec\", \"research the open items\", \"finalize my spec\", \"make this spec implementation-ready\", \"answer the TBDs\". Also triggers whenever the user points at a brainstorm-beagle spec and asks for research, proposals, or answers to unresolved items. Orchestrates parallel research subagents when available (falls back to inline sequential research otherwise), proposes answers one at a time for user approval, then rewrites the spec in place so it arrives at planning with no known gaps. Does NOT write code, design implementation, or create plans — it only produces a complete spec.Data Engineering40
wms-developerdaemon-blockint-tech/agentic-enteprises-skillDefine WMS footprint, tenants, automation touchpoints, and handoffs.Data Engineering38
carto-connect-datawarehousecartodb/agent-skillsChoose and configure the data warehouse engine connection for CARTO (BigQuery, Snowflake, Redshift, Postgres, Databricks, Oracle).Data Engineering38carto-explore-datawarehousecartodb/agent-skillsDiscover what's in the connected warehouse — schemas, tables, columns, and CARTO named sources.Data Engineering38sparkpostmembranedev/application-skillsSparkPost is an email delivery service that helps businesses send transactional and marketing emails reliably. It provides APIs and tools for sending, tracking, and analyzing email performance. Developers and marketers use SparkPost to manage their email infrastructure and improve deliverability.Data Engineering37
csv-pipelineaaaaqwq/agi-super-teamProcess, transform, analyze, and report on CSV and JSON data files. Use when the user needs to filter rows, join datasets, compute aggregates, convert formats, deduplicate, or generate summary reports from tabular data. Works with any CSV, TSV, or JSON Lines file.Data Engineering37carto-query-datawarehousecartodb/agent-skillsWrite spatial SQL against the connected warehouse — dialect-specific guidance, performance defaults, and CARTO's query/job execution model.Data Engineering37airflow-operator-creatorjeremylongshore/claude-code-plugins-plus-skillsCreate airflow operator creator operations. Auto-activating skill forData Engineering37Other tasks
same rules, different job
