{"id":2120,"date":"2025-08-19T00:03:15","date_gmt":"2025-08-19T00:03:15","guid":{"rendered":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/lead-consultant-data-engineer-awspython-spark-kafka-for-etl\/"},"modified":"2025-08-19T00:03:15","modified_gmt":"2025-08-19T00:03:15","slug":"lead-consultant-data-engineer-awspython-spark-kafka-for-etl","status":"publish","type":"post","link":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/lead-consultant-data-engineer-awspython-spark-kafka-for-etl\/","title":{"rendered":"Lead Consultant-Data Engineer, AWS+Python, Spark, Kafka for ETL"},"content":{"rendered":"

Ready to shape the future of work?<\/span>At Genpact, we don\u2019t just adapt to change\u2014we drive it. AI and digital innovation are redefining industries, and we\u2019re leading the charge. Genpact\u2019s , our industry-first accelerator, is an example of how we\u2019re scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. From large-scale models to , our breakthrough solutions tackle companies\u2019 most complex challenges.<\/span>If you thrive in a fast-moving, tech-driven environment, love solving real-world problems, and want to be part of a team that\u2019s shaping the future, this is your moment.<\/span>Genpact (NYSE: G) is an advanced technology services and solutions company that delivers lasting value for leading enterprises globally. Through our deep business knowledge, operational excellence, and cutting-edge solutions \u2013 we help companies across industries get ahead and stay ahead. Powered by curiosity, courage, and innovation, our teams implement data, technology, and AI to create tomorrow, today. Get to know us at and on , , , and .<\/span>Inviting applications for the role of Lead Consultant-Data Engineer, AWS+Python, Spark, Kafka for ETL!<\/span>Responsibilities<\/b><\/span>Develop, deploy, and manage ETL pipelines using AWS services, Python, Spark, and Kafka.<\/span>Integrate structured and unstructured data from various data sources into data lakes and data warehouses.<\/span>Design and deploy scalable, highly available, and fault-tolerant AWS data processes using AWS data services (Glue, Lambda, Step, Redshift)<\/span>Monitor and optimize the performance of cloud resources to ensure efficient utilization and cost-effectiveness.<\/span>Implement and maintain security measures to protect data and systems within the AWS environment, including IAM policies, security groups, and encryption mechanisms.<\/span>Migrate the application data from legacy databases to Cloud based solutions (Redshift, DynamoDB, etc) for high availability with low cost<\/span>Develop application programs using Big Data technologies like Apache Hadoop, Apache Spark, etc with appropriate cloud-based services like Amazon AWS, etc.<\/span>Build data pipelines by building ETL processes (Extract-Transform-Load)<\/span>Implement backup, disaster recovery, and business continuity strategies for cloud-based applications and data.<\/span>Responsible for analysing business and functional requirements which involves a review of existing system configurations and operating methodologies as well as understanding evolving business needs<\/span>Analyse requirements\/User stories at the business meetings and strategize the impact of requirements on different platforms\/applications, convert the business requirements into technical requirements<\/span>Participating in design reviews to provide input on functional requirements, product designs, schedules and\/or potential problems<\/span>Understand current application infrastructure and suggest Cloud based solutions which reduces operational cost, requires minimal maintenance but provides high availability with improved security<\/span>Perform unit testing on the modified software to ensure that the new functionality is working as expected while existing functionalities continue to work in the same way<\/span>Coordinate with release management, other supporting teams to deploy changes in production environment<\/span>Qualifications we seek in you!<\/span>Minimum Qualifications<\/b><\/span>Experience in designing, implementing data pipelines, build data applications, data migration on AWS<\/span>Strong experience of implementing data lake using AWS services like Glue, Lambda, Step, Redshift<\/span>Experience of Databricks will be added advantage<\/span>Strong experience in Python and SQL<\/span>Proven expertise in AWS services such as S3, Lambda, Glue, EMR, and Redshift.<\/span>Advanced programming skills in Python for data processing and automation.<\/span>Hands-on experience with Apache Spark for large-scale data processing.<\/span>Experience with Apache Kafka for real-time data streaming and event processing.<\/span>Proficiency in SQL for data querying and transformation.<\/span>Strong understanding of security principles and best practices for cloud-based environments.<\/span>Experience with monitoring tools and implementing proactive measures to ensure system availability and performance.<\/span>Excellent problem-solving skills and ability to troubleshoot complex issues in a distributed, cloud-based environment.<\/span>Strong communication and collaboration skills to work effectively with cross-functional teams.<\/span>Preferred Qualifications\/ Skills<\/span>Master\u2019s Degree-Computer Science, Electronics, Electrical.<\/span>AWS Data Engineering & Cloud certifications, Databricks certifications<\/span>Experience with multiple data integration technologies and cloud platforms<\/span>Knowledge of Change & Incident Management process<\/span>Why join Genpact?<\/span>Be a transformation leader \u2013 Work at the cutting edge of AI, automation, and digital innovation<\/span>Make an impact \u2013 Drive change for global enterprises and solve business challenges that matter<\/span>Accelerate your career \u2013 Get hands-on experience, mentorship, and continuous learning opportunities<\/span>Work with the best \u2013 Join 140,000+ bold thinkers and problem-solvers who push boundaries every day<\/span>Thrive in a values-driven culture \u2013 Our courage, curiosity, and incisiveness – built on a foundation of integrity and inclusion – allow your ideas to fuel progress<\/span>Come join the tech shapers and growth makers at Genpact and take your career in the only direction that matters: Up.<\/span>Let\u2019s build tomorrow together.<\/span>Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military\/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation.
\n Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a ‘starter kit,’ paying to apply, or purchasing equipment or training.<\/p>\n

Apply Now<\/a> <\/b><\/center><\/p>\n","protected":false},"excerpt":{"rendered":"

Ready to shape the future of work? At Genpact, we don\u2019t just adapt to change\u2014we drive it. AI and digital innovation are redefining industries, and we\u2019re leading the charge. Genpact\u2019s , our industry-first accelerator, is an example of how we\u2019re scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[52],"tags":[],"class_list":["post-2120","post","type-post","status-publish","format-standard","hentry","category-python-jobs"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"jetpack-related-posts":[],"jetpack_shortlink":"https:\/\/wp.me\/pgs0IK-yc","_links":{"self":[{"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/posts\/2120","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/comments?post=2120"}],"version-history":[{"count":0,"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/posts\/2120\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/media?parent=2120"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/categories?post=2120"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobsinindia\/wp-json\/wp\/v2\/tags?post=2120"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}