Big Data Architect
We are seeking a highly skilled Data Engineer to join our team. As a key member of the data architecture group, you will be responsible for designing and implementing large-scale data systems using Databricks.
Key Responsibilities:
* Design and develop scalable data pipelines using PySpark.
* Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
* Develop and maintain high-quality code, following best practices and coding standards.
* Work closely with data scientists to ensure seamless integration of data products into our analytics pipeline.
* Stay up-to-date with industry trends and emerging technologies, providing recommendations for improvements and innovation.
Qualifications:
* Strong experience in big data engineering, with expertise in Databricks, Spark, and Scala.
* Hands-on experience with data warehousing, ETL, and data governance.
* Excellent problem-solving skills, with the ability to analyze complex data issues and provide creative solutions.
* Strong communication and collaboration skills, with the ability to work effectively with both technical and non-technical stakeholders.
* Experience with Agile development methodologies and continuous integration/continuous deployment (CI/CD) pipelines.