As a seasoned professional, we are seeking an accomplished Data Engineer to contribute to our Threat Research team.
Key Responsibilities:
* Data Pipeline Development: Design and implement robust data pipelines for ingesting threat intelligence data from various sources into our ecosystem.
* Data Validation: Implement rigorous data validation processes to ensure accuracy, completeness, and consistency of data.
* Collaboration: Collaborate closely with threat analysts to understand data requirements and design effective solutions.
* Automation: Develop automation scripts and workflows for efficient data export processes to external systems or partners.
* Optimization: Continuously optimize and enhance existing data pipelines for improved performance, scalability, and reliability.
* Monitoring and Troubleshooting: Proactively monitor data pipelines and troubleshoot issues as they arise, ensuring uninterrupted data availability and integrity.
* Documentation: Provide detailed technical documentation of specifications, data flows, and procedures for data pipeline maintenance and support.
* Professional Growth: Stay updated on emerging technologies and best practices in data engineering, applying them to our ecosystem.
* Mentorship: Offer technical guidance and support to team members on data engineering best practices and methodologies.