data-engineering-master
De la autor
数据工程 — 数据平台从业者的认知操作系统, 覆盖把数据从源系统搬运成可靠 / 可查询 / 可信赖形态供分析 / ML / 数据产品消费的全生命周期 (生成 → 摄取 → 存储 → 转换 → 服务 + 安全/数据管理/DataOps/数据架构/编排/软件工程 六条暗流, Reis & Housley 框架): 摄取与集成 (批 + CDC 变更数据捕获 Debezium + EL 工具 Fivetran/Airbyte/Meltano/dlt + Kafka Connect + schema drift) / 存储与文件表格式 (对象存储数据湖 + 列存 Parquet/ORC/Arrow/Avro + 开放表格式 Apache Iceberg/Delta Lake/Apache Hudi + lakehouse + 分区/compaction) / 转换与建模 (ELT dbt/SQLMesh + Spark + 维度建模 Kimball + Inmon + Data Vault + 大宽表 OBT + 渐变维 SCD + 增量模型 + 语义/指标层) / 编排与工作流 (Apache Airflow/Dagster/Prefect/Mage/Kestra/Apache DolphinScheduler + DAG + 幂等 + 回填 backfill + 数据资产调度) / 批流与实时 (Apache Kafka/Apache Flink/Spark Structured Streaming/Kinesis/Pulsar/Redpanda + Lambda vs Kappa + watermark/窗口/exactly-once + 流式 SQL Materialize/RisingWave + 实时 OLAP ClickHouse/Apache Druid/Apache Pinot/StarRocks/Apache Doris) / 数仓与查询引擎 (Snowflake/BigQuery/Redshift/Databricks SQL/Trino/Presto/DuckDB/Polars + 存算分离 + MPP) / 数据质量测试与可观测性 (dbt tests/Great Expectations/Soda + 数据契约 + Monte Carlo data downtime + 新鲜度/量/schem
Instalare
npx skills add swaylq/master-skill --skill data-engineering-master -a claude-code
Se instalează în .claude/skills/data-engineering-master/ · Fiecare comandă descarcă skill-ul direct de pe GitHub cu utilitarul open source skills.
Identic octet cu octet cu commit-ul scanat, cu licența depozitului și un fișier SOURCE.txt.
Vrei ca agentul să-ți propună skill-uri în timp ce scrii? Vezi cum folosești skill-urile din IDE.
Verificări de siguranță pe SKILL.md
- Include scripturi pe care nu le-am verificatScăzut
De ce 76
aiqbench calculează scorul de încredere din semnale publice. Nu este un audit de securitate, așa că citește un skill înainte să-l instalezi.
Skill-uri similare
Cele mai apropiate de acesta prin ce fac și când sunt folosite.
- data-engineeringcbrock84/headcountBuilds and operates data pipelines — ingestion, transformation, orchestration, quality testing, and reliability of data delivery. Use this to design or debug a pipeline, decide batch versus streaming, add data quality checks, handle late or duplicate data, or work out why a dashboard's numbers changed without anyone changing the dashboard.Date★ 2.5K
- fluxtonone-ai/tononeData engineer — databases, migrations, pipelines, schema design, and query optimization. Use when asked to "design a schema", "write a migration", "build a data pipeline", or "optimize this query".Date★ 77
- engineering-data-pipelinestelagod/code-abyssData engineering knowledge reference covering Airflow, Dagster, Kafka Streams, Flink, dbt, and data quality patterns. Use when building data pipelines, ETL workflows, stream processing, or data quality checks.Date★ 244
- data-engineersaeed-vayghan/gemini-agent-skillsExpert data engineer specializing in building scalable data pipelines, ETL/ELT processes, and data infrastructure. Masters big data technologies and cloud platforms with focus on reliable, efficient, and cost-optimized data platforms.DevOps★ 51
- tool.data_engineering.134100a5a52f2e93AI45Lab/OpenARTData engineering practice patterns for data pipelines, ETL/ELT, orchestration, data quality, and infrastructure. Covers Airflow, Dagster, Prefect, Spark, dbt, data lakes, lakehouses, Iceberg, Delta Lake, warehouses (Snowflake, BigQuery, Databricks), streaming (Kafka, Kinesis, Flink), CDC, data contracts, data governance, data mesh, and batch processing. Use when reviewing or building data pipelines, ingestion systems, stream processing, data infrastructure, or data platform architecture. Do not use for SQL modeling or dbt projects (use analytics-engineering), business dashboards (use analytics), or ML model training (use data-science).Date★ 263
- tool.data_engineering.55c8f35d7b3cac72AI45Lab/OpenARTData engineering practice patterns for data pipelines, ETL/ELT, orchestration, data quality, and infrastructure. Covers Airflow, Dagster, Prefect, Spark, dbt, data lakes, lakehouses, Iceberg, Delta Lake, warehouses (Snowflake, BigQuery, Databricks), streaming (Kafka, Kinesis, Flink), CDC, data contracts, data governance, data mesh, and batch processing. Use when reviewing or building data pipelines, ingestion systems, stream processing, data infrastructure, or data platform architecture. Do not use for SQL modeling or dbt projects (use analytics-engineering), business dashboards (use analytics), or ML model training (use data-science).Date★ 263
Note și recenzii
—
nicio recenzie
Ai încercat acest skill? Notează-l ca să-i ajuți pe alții. Autentifică-te
Discuție
Intră în discuție. Autentifică-te