Senior Data Engineer

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Senior Data Engineer Role: Senior Data Engineer Role Overview We are seeking a Senior Data Engineer to drive AI-led transformation across our analytics and data platforms. This role is designed for a hands-on engineer who can ideate, design, and implement AI-driven solutions, modernize Python and data pipelines, and maintain best-in-class analytics environments. The role combines agentic AI concepts, cloud-based AI services, strong Python/PySpark engineering, and deep analytics expertise across Databricks and Snowflake, while applying solid ETL and DBA fundamentals in AWS.

Priority & Core Responsibilities 1️⃣ AI – Agentic Systems, GenAI & Intelligent Workflow Automation Lead AI ideation by identifying opportunities to eliminate manual effort, reduce operational friction, and improve decision-making. Design and implement agentic AI solutions, including: Multi-agent orchestration for task delegation and workflow execution AI agents for monitoring, diagnostics, reconciliation, and data reasoning Build end-to-end GenAI / LLM-powered workflows, including prompt engineering, chaining, tool use, and evaluation. Design and maintain data pipelines specifically for AI workloads, supporting: Training, fine-tuning, and inference Vector storage, retrieval-augmented generation (RAG), and embeddings Integrate AI solutions with existing data platforms while ensuring governance, observability, and cost control. Apply responsible AI principles, access controls, and auditability for AI-driven systems. 2️⃣ Python Engineering, APIs & PySpark Development Develop production-grade Python applications supporting data, AI, and automation use cases. Design and expose REST APIs using Flask or FastAPI for model inference, AI services, and data access. Improve and refactor existing Python codebases for maintainability, performance, and scalability. Migrate legacy logic and scripts into PySpark-based implementations on Databricks. Follow best practices in: Modular architecture and design patterns Logging, monitoring, and exception handling Unit and integration testing 3️⃣ Analytics Platforms – Databricks & Snowflake Expertise Act as an expert practitioner for Databricks and Snowflake, supporting both analytics and AI-driven workloads. Perform advanced problem solving related to: Performance tuning Cost optimization Query optimization and data layout Define and enforce standards and best practices for analytics and AI workloads on these platforms. Support administration and daily management activities, including: Workspace and resource governance User access and role management Platform usage optimization Enable analytics teams through reusable patterns, templates, and documentation. 4️⃣ ETL & Cloud Data Engineering (AWS) Architect and implement scalable ETL pipelines using modern cloud-native patterns. Apply expert knowledge of ETL frameworks and design principles, including incremental processing and fault tolerance. Build and maintain pipelines using AWS Glue, interacting with S3, IAM, and related AWS services. Ensure reliable orchestration, monitoring, and error handling across data pipelines. Optimize ETL workloads for performance, scalability, and cost efficiency. 5️⃣ DBA Concepts & Data Storage Expertise Apply strong DBA fundamentals to analytics and operational systems. Work with relational and NoSQL databases, including: Amazon RDS (MySQL, PostgreSQL) MongoDB Atlas Apply best practices in: