EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.
We are seeking a Senior Python Data Engineer to join our team focused on developing and managing efficient data pipelines.
You will work extensively with AWS Glue and PySpark to handle both structured and unstructured data at scale. This role involves optimizing pipelines for performance and cost while ensuring robustness through testing and CI/CD integration. Your expertise will contribute to delivering scalable data solutions in collaboration with IT engineers. If you are experienced in Python programming, data engineering, and AWS technologies, we encourage you to apply and help us enhance our data processing capabilities.
Responsibilities
- Develop and maintain data pipelines using PySpark on AWS Glue
- Implement ETL and ELT processes to handle large-scale structured and unstructured data
- Optimize data pipelines for performance, scalability, and cost-effectiveness on AWS Glue
- Write unit and integration tests for PySpark code using Pytest or unittest frameworks
- Build and manage CI/CD pipelines for AWS Glue workflows
- Troubleshoot and optimize resource utilization and performance on the AWS platform
- Collaborate with IT engineers to deliver reliable and scalable data solutions
- Document processes and solutions to ensure maintainability and knowledge sharing
- Monitor data workflows and resolve issues proactively
- Ensure compliance with data governance and security standards
Requirements
- 4+ years of experience in data engineering or distributed systems development
- Strong programming skills in Python and PySpark
- Hands-on experience with AWS Glue, including SQL usage within the ecosystem
- Knowledge of big data frameworks such as Spark
- Experience with AWS services, including S3, Lambda, Glue Catalog
- Proficiency with SQL databases such as PostgreSQL or SQL Server
- Strong analytical and problem-solving abilities
- Capability to manage multiple priorities in a deadline-driven environment
- Effective communication and organizational skills
- Strong written and verbal English communication skills (B2+)
Nice to have
- Experience with Kubernetes for managing containerized workloads
- Background in regulated industries such as insurance
We offer
- We believe that the greatest strength of the company is its people. EPAM is fully committed to help its employees to reach their full potential and achieve their professional goals through continues learning. With this in mind, we would like to introduce to you few of the many opportunities and services which we believe will help you expand your current knowledge:
- Full access to cutting-edge tools and technologies
- Competitive compensation depending on experience and skills
- All-around Social package: professional & soft skills training, medical & family care programs, sports
- Relocation opportunities
- Free English classes
- Unlimited access to LinkedIn learning solutions
- Continuous experience exchange with experts and professionals worldwide
- Friendly team and comfortable working environment
- Engineering, corporate, and social events within and outside the Company
- Flexible working schedule
- Opportunities for self-realization
