**Python Developer**
Home Office / Full time / LATAM / Only resumes in English
- "ScrumLaunch is a Los Angeles based product innovation studio. We work with leading brands and high growth startups to design and build great products._
**Responsibilities**:
- Develop and maintain web scraping scripts and tools using Python, focusing on efficiency, reliability, and scalability.
- Gather requirements from stakeholders and design effective scraping strategies to extract data from websites, considering high volume scenarios.
- Implement and optimize scraping algorithms to handle complex data structures, dynamic content, and authentication mechanisms.
- Utilize AWS services such as EC2, S3, Lambda, and Glue for data processing, storage, and computation.
- Work with message brokers like Redis, Kafka, or RabbitMQ to handle data ingestion and distribution efficiently.
- Perform data cleansing, transformation, and aggregation to ensure data quality and integrity.
- Collaborate with cross-functional teams to integrate scraped data into data pipelines and analytics systems.
- Utilize Redis as a message broker for efficient and reliable communication between components of the data processing pipeline.
- Monitor and optimize scraping processes, data workflows, and message queues for performance, scalability, and cost-effectiveness.
- Stay up-to-date with the latest web scraping techniques, AWS services, data processing frameworks, and message broker technologies.
- Troubleshoot and resolve issues related to web scraping, data processing, AWS infrastructure, and Redis message broker.
- Provide technical guidance and support to junior developers on web scraping, data processing, and message broker implementations.
**Qualifications**:
- Proven work experience as a Python Developer with expertise in web scraping.
- Strong proficiency in Python programming language and related libraries such as BeautifulSoup, Scrapy, or Selenium.
- Knowledge of AWS services such as EC2, S3, Lambda, Glue, and familiarity with AWS SDKs.
- Experience with message brokers like Redis, Kafka, or RabbitMQ for efficient data ingestion and distribution.
- Solid understanding of data processing techniques, ETL concepts, and data storage technologies.
- Familiarity with database concepts and proficiency in SQL for data manipulation and retrieval.
- Understanding of web protocols (HTTP, HTTPS), web security best practices, and authentication mechanisms.
- Ability to analyze and interpret complex website structures and patterns to devise efficient scraping strategies.
- Strong problem-solving skills and ability to handle high volumes of data efficiently and effectively.
- Excellent communication skills to effectively collaborate with
**We offer**:
- Opportunity to grow as an expert;
- Flexible working conditions;
- 12 days of paid vacation;
- 10 sick days and off days;
- Friendly and open-minded team;
- English classes
**Job Types**: Full-time, Part-time, Temporary, Contract
Ability to commute/relocate:
- Rio de Janeiro, RJ: Reliably commute or willing to relocate with an employer-provided relocation package (preferred)