spark-and-distributed-processing
From the author
Guides agents through batch and distributed data processing design. Use when implementing or reviewing Spark-based pipelines, or managed distributed runtimes such as Glue and EMR.
Install
npx skills add vaquarkhan/data-engineering-agent-skills --skill spark-and-distributed-processing -a claude-code
Installs to .claude/skills/spark-and-distributed-processing/ · Each command fetches the skill straight from GitHub with the open skills CLI.
Byte-identical to the scanned commit, with the repository’s licence and a SOURCE.txt.
Want your agent to suggest skills while you type? See how to use skills from your IDE.
Safety checks on SKILL.md
- Ships scripts we have not reviewedLow
Why 72
aiqbench computes the trust score from public signals. It is not a security audit, so read a skill before you install it.
Similar skills
Closest to this one by what they do and when they are used.
- spark-engineerJeffallan/claude-skillsUse when writing Spark jobs, debugging performance issues, or configuring cluster settings for Apache Spark applications, distributed data processing pipelines, or big data workloads. Invoke to write DataFrame transformations, optimize Spark SQL queries, implement RDD pipelines, tune shuffle operations, configure executor memory, process .parquet files, handle data partitioning, or build structured streaming analytics.Data★ 12.1K
- streamsimota/agent-skillsDesigning ETL/ELT pipelines, visualizing data flows, selecting batch/streaming approaches, and architecting Kafka/Airflow/dbt systems. Use when building data pipelines or managing data quality.Data★ 95
- streaming-and-messaging-systemsvaquarkhan/data-engineering-agent-skillsGuides agents through event streaming and real-time data pipeline design. Use when working with Kafka, Kinesis, Flink, stream processing, windowing, stateful consumers, or near-real-time publish flows.Data★ 59
- senior-data-engineerborghei/Claude-SkillsData engineering for batch and streaming pipelines with Airflow, dbt, Spark, and Kafka. Use when designing data architectures, building pipelines, adding data-quality checks, optimizing ETL/ELT, or troubleshooting pipeline failures.Data★ 978
- pipeline-planning-and-task-breakdownvaquarkhan/data-engineering-agent-skillsBreaks approved data specifications into safe, verifiable implementation tasks. Use when a data project spans multiple steps, systems, or files and needs dependency-aware sequencing.Planning★ 59
- apache-beam-unified-batch-and-streamvaquarkhan/data-engineering-agent-skillsGuides agents through Apache Beam pipelines that unify batch and streaming logic. Use when designing Beam transforms, windowing, runners, replay behavior, or portability across execution backends.Other★ 59
Ratings & reviews
—
no reviews
Tried this skill? Rate it to help others. Sign in
Discussion
Join the discussion. Sign in