spark-python-data-source
Build custom Python data sources for Apache Spark using the PySpark DataSource API — batch and streaming readers/writers for external systems. Use this skill whenever someone wants to connect Spark to an external system (database, API, message queue, custom protocol), build a Spark connector or plugin in Python, implement a DataSourceReader or DataSourceWriter, pull data from or push data to a system via Spark, or work with the PySpark DataSource API in any way. Even if they just say "read from X in Spark" or "write DataFrame to Y" and there's no native connector, this skill applies.
Works with
This skill's source license couldn't be confirmed as safe to mirror here, so it isn't inlined. View the full skill directly on its source repository.
View on GitHubMore Data Engineering skills
data-pipeline
claude-office-skills/skills
Data pipeline and ETL automation - extract, transform, load workflows for data integration and analytics
ETL Pipeline
claude-office-skills/skills
Design and automate Extract, Transform, Load data pipelines for data integration and analytics
data-throughput-accelerator
affaan-m/ecc
Use when large data ingestion, backfill, export, ETL, warehouse loading, manifest catch-up, or table synchronization needs to become much faster while preserving data correctness.
