Senior Data Engineer - Vice President - Python Development

Technology Join a small, high-impact engineering team in Citi Markets Technology building the data foundation for greenfield Generative AI products across asset classes. We are looking for an experienced Data Engineer who combines deep Python expertise with strong engineering judgement and a practical understanding of how to build reliable, high-performance data platforms. The role goes beyond constructing ETL pipelines. You will help define how billions of records from diverse Markets data sources are collected, validated, transformed, governed, and made available to production AI applications with consistently low retrieval latency. If you want to work on ambitious data engineering problems, shape a platform from the ground up, and help define how data is engineered for production AI in Markets, this is the role for you.

We are a fast-moving team specialising in Generative AI within Markets Technology. We build greenfield products that span multiple asset classes and solve real business problems using modern AI and data engineering approaches. Our applications depend on a robust and well-designed data foundation. That means creating pipelines and serving layers that can process billions of records while preserving data quality, provenance, security, and operational reliability. The team is still small enough for every engineer to have genuine influence. Data engineering is a core part of the product architecture, not a downstream support function.

As a Vice President in the team, you will be a hands-on senior engineer responsible for designing and building the data foundation for greenfield Generative AI products across Markets, including our conversational AI platform. You will develop production-grade data pipelines and services, primarily using Python, to ingest, validate, transform, enrich, and serve data from a wide range of internal and external sources. You will help create a curated, high-performance data-serving layer that enables low-latency retrieval by our AI and application services. You will contribute directly to code while also shaping architecture, engineering standards, data models, quality controls, and operational practices. We are looking for someone who can make pragmatic technology choices, challenge assumptions, and take long-term ownership of the platform. What You’ll Do Design, build, and evolve the data architecture supporting Generative AI products across Markets Develop scalable Python pipelines that ingest, validate, transform, enrich, and integrate billions of records from diverse sources Build curated, queryable datasets and high-performance serving layers for low-latency application access Design and optimise the application’s data storage strategies, initially centred on PostgreSQL and Parquet Select suitable processing approaches for each workload, from efficient in-process and columnar processing to distributed frameworks where required Optimise ingestion, transformation, storage, indexing, and query performance Establish robust controls for data quality, reconciliation, lineage, schema evolution, idempotency, and recovery Build production observability into data pipelines, including metrics, logging, alerting, and operational diagnostics Design solutions that respect data classification, entitlements, access controls, and security requirements Contribute directly to code, architecture reviews, technical standards, and the wider engineering direction of the team Build automated tests and CI/CD pipelines that enable reliable and repeatable delivery Collaborate closely with AI engineers, software engineers, architects, product partners, and Markets stakeholders Use AI-assisted engineering tools, including Devin and GitHub Copilot, to improve development quality and productivity What We’re Looking For Extensive hands-on experience in data engineering, software engineering, or a closely related discipline Deep practical expertise in Python and the ability to build