{"id":52710,"date":"2025-03-18T02:59:54","date_gmt":"2025-03-18T02:59:54","guid":{"rendered":"https:\/\/www.whitehattoolbox.com\/jobs\/data-engineering-lead-aws-python-pyspark-specialist\/"},"modified":"2025-03-18T02:59:54","modified_gmt":"2025-03-18T02:59:54","slug":"data-engineering-lead-aws-python-pyspark-specialist","status":"publish","type":"post","link":"https:\/\/www.whitehattoolbox.com\/jobs\/data-engineering-lead-aws-python-pyspark-specialist\/","title":{"rendered":"Data Engineering Lead &#8211; AWS Python &#038; PySpark Specialist"},"content":{"rendered":"<p><b>Job Description:<\/b><span class=\"br\"><\/span>We are seeking a Data Engineering Lead with 5-9 years of experience to join our dynamic team. In this role, you will collaborate closely with project leads and clients to design and develop scalable data processing systems. You will be responsible for gathering requirements, creating comprehensive end-to-end solutions, and overseeing projects from inception to production. A key aspect of this role involves mentoring team members and converting complex business needs into innovative technical solutions using the latest technologies.<span class=\"br\"><\/span><b>Key Responsibilities:<\/b><\/p>\n<ul>\n<li>Design and develop scalable data processing systems.<\/li>\n<li>Gather and analyze requirements to create effective end-to-end solutions.<\/li>\n<li>Manage projects from conception through to production.<\/li>\n<li>Mentor and guide team members to enhance their technical skills.<\/li>\n<li>Translate complex business requirements into robust technical solutions.<\/li>\n<li>Utilize cutting-edge technologies to solve complex problems.<\/li>\n<\/ul>\n<p><b>Technical Skills:<\/b><\/p>\n<ul>\n<li>Expertise in Python and PySpark: Proficient in Spark architecture and performance tuning.<\/li>\n<li>SQL Proficiency: Strong skills in writing optimized SQL queries with performance tuning.<\/li>\n<li>AWS Data Engineering Services: In-depth experience with Glue, Lambda, API Gateway, and S3.<\/li>\n<li>Business to Technical Translation: Ability to convert complex business requirements into technical solutions.<\/li>\n<li>Unix Knowledge: Good to have working knowledge of Unix.<\/li>\n<li>CloudFormation and CI\/CD: Hands-on experience with CloudFormation and CI\/CD pipelines is desirable.<\/li>\n<\/ul>\n<p><b>Soft Skills:<\/b><\/p>\n<ul>\n<li>Problem-Solving: Strong analytical and problem-solving capabilities.<\/li>\n<li>Communication: Excellent communication skills to convey technical concepts clearly.<\/li>\n<li>Adaptability: Comfortable working in a fast-paced environment and managing multiple priorities.<\/li>\n<li>Collaboration: Effective collaboration with cross-functional teams.<\/li>\n<li>Quality Focus: Proven ability to deliver high-quality results within project timelines.<\/li>\n<\/ul>\n<p><b>Skills:<\/b><span class=\"br\"><\/span>Python,Pyspark,Aws<span class=\"br\"><\/span><b>About Company:<\/b><span class=\"br\"><\/span>UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world\u2019s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients\u2019 organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact\u2014touching billions of lives in the process.<\/p>\n<p><center><b><a rel=\"nofollow noopener\" href=\"https:\/\/www.careerjet.co.in\/jobad\/infce8889219f40a290c0575a70e042882?affid=0852fd5db5a52cbcd75bf945aa534932\" class=\"button purchase\" target=\"_blank\">Apply Now<\/a> <\/b><\/center><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Job Description: We are seeking a Data Engineering Lead with 5-9 years of experience to join our dynamic team. In this role, you will collaborate closely with project leads and clients to design and develop scalable data processing systems. You will be responsible for gathering requirements, creating comprehensive end-to-end solutions, and overseeing projects from inception<\/p>\n","protected":false},"author":1,"featured_media":52711,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[20,71,15,148],"tags":[],"class_list":["post-52710","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-it-jobs","category-it-jobs-jobs-in-uae","category-jobs-in-india","category-python-jobs"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.whitehattoolbox.com\/jobs\/wp-content\/uploads\/2025\/03\/52710-data-engineering-lead-aws-python-pyspark-specialist.png?fit=1306%2C840&ssl=1","jetpack-related-posts":[{"id":167133,"url":"https:\/\/www.whitehattoolbox.com\/jobs\/were-hiring-ai-python-developer\/","url_meta":{"origin":52710,"position":0},"title":"We\u2019re Hiring AI\/Python Developer","author":"Admin","date":"March 27, 2026","format":false,"excerpt":"We are looking for a Python Developer to join our engineering team and help us develop and maintain various software products. Python Developer responsibilities include writing and testing code, debugging programs and integrating applications with third-party web services. To be successful in this role, you should have experience using server-side\u2026","rel":"","context":"In &quot;IT Jobs&quot;","block_context":{"text":"IT Jobs","link":"https:\/\/www.whitehattoolbox.com\/jobs\/category\/jobs-in-india\/it-jobs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":167230,"url":"https:\/\/www.whitehattoolbox.com\/jobs\/were-hiring-ai-python-developer-2\/","url_meta":{"origin":52710,"position":1},"title":"We\u2019re Hiring AI\/Python Developer","author":"Admin","date":"March 27, 2026","format":false,"excerpt":"We are looking for a Python Developer to join our engineering team and help us develop and maintain various software products. Python Developer responsibilities include writing and testing code, debugging programs and integrating applications with third-party web services. To be successful in this role, you should have experience using server-side\u2026","rel":"","context":"In &quot;IT Jobs&quot;","block_context":{"text":"IT Jobs","link":"https:\/\/www.whitehattoolbox.com\/jobs\/category\/jobs-in-india\/it-jobs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":145178,"url":"https:\/\/www.whitehattoolbox.com\/jobs\/were-hiring-data-scientist-python-aws-ml\/","url_meta":{"origin":52710,"position":2},"title":"We\u2019re Hiring Data Scientist ( Python, AWS, ML)","author":"Admin","date":"February 22, 2026","format":false,"excerpt":"What's the role? As a Machine Learning Engineer, you will design, implement, and operationalize scalable ML pipelines and algorithms that support HERE\u2019s Trips and sensor-based data products. You will contribute to end-to-end software development, from problem definition to solution deployment, and collaborate closely with cross-functional teams to bring state-of-the-art ML\u2026","rel":"","context":"In &quot;IT Jobs&quot;","block_context":{"text":"IT Jobs","link":"https:\/\/www.whitehattoolbox.com\/jobs\/category\/jobs-in-india\/it-jobs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":117891,"url":"https:\/\/www.whitehattoolbox.com\/jobs\/were-hiring-software-engineer-iii-uipath2022-python-ai-llm\/","url_meta":{"origin":52710,"position":3},"title":"We\u2019re Hiring Software Engineer III &#8211; UIPath2022 +, Python, AI &#038; LLM","author":"Admin","date":"November 25, 2025","format":false,"excerpt":"Job Category: Software Engineering Job Description: We have an exciting and rewarding opportunity for you to take your software engineering career to the next level. As a Software Engineer III at JPMorgan Chase within the Commercial and Investment Bank's Global Banking Tech Team, you serve as a seasoned member of\u2026","rel":"","context":"In &quot;IT Jobs&quot;","block_context":{"text":"IT Jobs","link":"https:\/\/www.whitehattoolbox.com\/jobs\/category\/jobs-in-india\/it-jobs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":129371,"url":"https:\/\/www.whitehattoolbox.com\/jobs\/were-hiring-le-python-model-automation\/","url_meta":{"origin":52710,"position":4},"title":"We\u2019re Hiring LE &#8211; Python &#8211; Model Automation","author":"Admin","date":"January 14, 2026","format":false,"excerpt":"Overview: Join the Prodapt team in building a unified, enterprise-scale solution for model, data, and service management across the AI\/ML model development. Responsibilities: Design and develop scalable data pipelines and integration workflows. Experience in Integrating ML Models. Implement and optimize big data solutions for batch and real\u2011time processing, ensuring performance,\u2026","rel":"","context":"In &quot;IT Jobs&quot;","block_context":{"text":"IT Jobs","link":"https:\/\/www.whitehattoolbox.com\/jobs\/category\/jobs-in-india\/it-jobs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":130035,"url":"https:\/\/www.whitehattoolbox.com\/jobs\/were-hiring-le-python-model-automation-2\/","url_meta":{"origin":52710,"position":5},"title":"We\u2019re Hiring LE &#8211; Python &#8211; Model Automation","author":"Admin","date":"January 14, 2026","format":false,"excerpt":"Overview: Join the Prodapt team in building a unified, enterprise-scale solution for model, data, and service management across the AI\/ML model development. Responsibilities: Design and develop scalable data pipelines and integration workflows. Experience in Integrating ML Models. Implement and optimize big data solutions for batch and real\u2011time processing, ensuring performance,\u2026","rel":"","context":"In &quot;IT Jobs&quot;","block_context":{"text":"IT Jobs","link":"https:\/\/www.whitehattoolbox.com\/jobs\/category\/jobs-in-india\/it-jobs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pg9e9d-dIa","_links":{"self":[{"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/posts\/52710","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/comments?post=52710"}],"version-history":[{"count":0,"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/posts\/52710\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/media\/52711"}],"wp:attachment":[{"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/media?parent=52710"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/categories?post=52710"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.whitehattoolbox.com\/jobs\/wp-json\/wp\/v2\/tags?post=52710"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}