รายละเอียดงานJob Responsibilities :
- Design, develop, and maintain ETL/ELT pipelines using Apache Airflow and other relevant tools.
- Build and optimize data ingestion workflows from various sources on Cloud and relational databases.
- Implement data transformations and processing pipelines to ensure high-quality data for analytics and machine learning.
- Work with streaming data processing frameworks like Kafka.
- Optimize data warehouse performance, partitioning, and indexing strategies.
- Implement and enforce data governance, security, and compliance best practices.
- Monitor and troubleshoot data pipelines, ensuring high availability and reliability.
- Collaborate with software engineers and DevOps teams to integrate data workflows into CI/CD pipelines.
- Document data workflows, pipelines, and system architecture for maintainability and scalability.