EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.
We are seeking a Senior Data Engineer to develop and manage scalable data pipelines using AWS Glue and PySpark.
You will be responsible for optimizing ETL/ELT processes and ensuring efficient data integration on AWS. This role involves collaboration with IT teams to deliver robust data solutions and maintain CI/CD pipelines. If you have strong Python and big data experience and enjoy working in dynamic environments, we encourage you to apply.
Responsibilities
- Develop and manage data pipelines using PySpark on AWS Glue
- Implement ETL and ELT processes for structured and unstructured data at scale
- Optimize data pipelines for performance, scalability, and cost efficiency within AWS Glue
- Write unit and integration tests for PySpark code using Pytest or unittest frameworks
- Build and maintain continuous integration and continuous delivery pipelines for AWS Glue workflows
- Troubleshoot and optimize resource utilization and performance on AWS platform
- Collaborate closely with IT engineers to deliver reliable data solutions
- Document data engineering processes and workflows
- Manage multiple tasks in a deadline-driven environment
Requirements
- 3+ years of experience in data engineering or distributed systems development
- Strong programming skills in Python and PySpark
- Hands-on experience with AWS Glue, including its ecosystem such as SQL, S3, Lambda, and Glue Catalog
- Knowledge of big data framework,s including Apache Spark
- Experience with SQL databases such as PostgreSQL or SQL Server
- Analytical and problem-solving skills
- Ability to handle multiple tasks in high-intensity, deadline-driven settings
- Proficiency ithe n English language (B2+)
Nice to have
- Familiarity with Kubernetes for managing containerized workloads
- Experience in regulated industries such as insurance
We offer
- We believe that the greatest strength of the company is its people. EPAM is fully committed to help its employees to reach their full potential and achieve their professional goals through continues learning. With this in mind, we would like to introduce to you few of the many opportunities and services which we believe will help you expand your current knowledge:
- Full access to cutting-edge tools and technologies
- Competitive compensation depending on experience and skills
- All-around Social package: professional & soft skills training, medical & family care programs, sports
- Relocation opportunities
- Free English classes
- Unlimited access to LinkedIn learning solutions
- Continuous experience exchange with experts and professionals worldwide
- Friendly team and comfortable working environment
- Engineering, corporate, and social events within and outside the Company
- Flexible working schedule
- Opportunities for self-realization
