Data Scientist Role Overview
We are currently seeking a highly skilled and experienced Data Engineer to join our team. As an expert in AWS technologies, the ideal candidate will have hands-on experience with API Gateway, Lambda/Fargate, S3, Glue, and CloudWatch.
Key Responsibilities:
* Proven expertise in PySpark for data processing and analysis.
* Strong knowledge of database integration and querying via Athena.
* Experience with data pipelines and ETL processes using Glue.
* Familiarity with infrastructure as code using Terraform or CloudFormation.
* Proficient in version control and CI/CD best practices.
Desirable Skills:
* Knowledge of columnar storage formats such as Parquet, Delta, Hudi, and Iceberg.
* Familiarity with data quality tools like Great Expectations and Soda.
* Understanding of serverless computing using Step Functions, EventBridge, and Kinesis.
* Best practices for securing APIs using Cognito, WAF, and IAM policies.
About Our Organization
Our company is a leading IT services provider with a culture of continuous learning and professional development.