We are seeking an experienced tech lead to join our web scraping development team. This is a leadership role that requires strong technical expertise and the ability to drive architectural decisions.
Responsibilities
* Develop and implement high-parallel crawling technology, manage user sessions, bypass anti-bot systems, and solve CAPTCHAs.
* Lead a team of developers, establish development and release processes, ensure code quality, and make technical decisions.
* Research and develop methods for browser and network-level fingerprinting, as well as protection bypasses for services like Cloudflare and Akamai.
* Set up CI/CD pipelines, configure scaling, monitor system performance using tools like Prometheus and Grafana, and establish SLA processes.
Requirements
* A minimum of 5 years of experience in building high-load backend/data platforms.
* Extensive experience with web scraping and a strong understanding of HTTP, TLS, and proxies.
* Proficiency in asynchronous programming languages like Python and Go, as well as Redis, message queues, and DevOps tools like Kubernetes and Helm.
* Ability to analyze and bypass bot protection systems and make informed technical decisions.
* Fluent English communication skills.
* Strong decision-making skills, mentoring abilities, and accountability for results.
Benefits
* The opportunity to contribute to the development of a key product from scratch.
* High levels of autonomy and influence over architecture and team decisions.
* Flexibility to work remotely and participate in conferences, courses, and other professional development opportunities.
* A competitive compensation package and benefits.
About the Role
* This is a full-time position that offers opportunities for growth and development.
* We are looking for a motivated individual who can take on challenges and drive innovation.