Cloud Data Engineering Specialist
Unlock your potential as a Cloud Data Engineering Specialist by working with cutting-edge technology to design, develop, and maintain scalable ETL/ELT pipelines using cloud-based data services.
* Develop high-performance data processing solutions with Apache Spark (PySpark) on cloud-based big data platforms.
* Collaborate with data scientists, analysts, and business stakeholders to deliver clean, reliable datasets that meet business needs.
* Optimize data workflows/pipelines for performance and cost efficiency in a cloud environment.
Key Responsibilities:
* Expertise in columnar and table formats (Parquet, Delta, Hudi, Iceberg).
* Utilization of data quality tools (Great Expectations, Soda) to ensure data integrity.
* Knowledge of serverless compute services like AWS Lambda or Google Cloud Functions.
* Adherence to best practices for API security (Cognito, WAF, IAM Policies) to safeguard sensitive data.