Senior Python Developer

Job Description:

Mandatory Skills: Python, Playwright, HTML, CSS, SQL, Web crawling, DOM, JSON, JavaScript, WebDriver

Nice-to-Have Skills: Selenium, Scrapy, BeautifulSoup, RegEx, Session/Cookie Management

Roles & Responsibility:

• Design and develop scalable web scraping solutions for structured and unstructured websites.
• Build robust crawlers using Scrapy, Playwright, Selenium, and BeautifulSoup.
• Develop browser automation scripts for dynamic JavaScript-based websites.
• Integrate scraped data with databases, APIs, and downstream applications.
• Handle pagination, authentication, cookies, sessions, anti-bot, proxy and rate limiting.
• Optimize scraping performance using asynchronous programming and parallel processing.
• Monitor scraper health, troubleshoot failures, and implement retry mechanisms.
• Ensure data quality through validation, deduplication, and transformation.
• Collaborate with dev team and customer to deliver data.