Lead, Data Engineering

Grade Level (for internal use): 11 We are seeking a  hands-on Senior Data Engineer / Data Platform Lead  to support our offshore team in India. This role focuses on executing data pipeline onboarding, migration, and support activities within our enterprise data platform. You will work closely with the Databricks team and internal stakeholders to ensure smooth transition, ongoing refinement, and reliable operation of data pipelines and integrations in a  Databricks-on-AWS lakehouse environment . Databricks on AWS positions data engineering around governed ingestion, transformation, and delivery of high-quality data for analytics and downstream use cases.  This is a delivery-focused role requiring strong technical expertise in data engineering, with responsibilities for supporting data onboarding, pipeline maintenance, platform support, and cloud-native operational excellence. You will lead a small team of engineers, providing technical guidance and ensuring high-quality execution across scalable, secure, and governed data workflows leveraging  Amazon S3 ,  AWS Glue ,  AWS IAM , and  AWS Lake Formation  alongside Databricks capabilities such as  Delta Lake ,  Unity Catalog , and open table format interoperability where appropriate.

Data Pipeline Support & Migration Assist in the migration of existing data pipelines to the Databricks platform, following established patterns and runbooks. Support the ongoing operation, troubleshooting, and optimization of pipelines post-migration. Implement and follow engineering best practices for data ingestion, transformation, and monitoring. Ensure pipelines are reliable, performant, and maintainable. Support data ingestion and storage patterns using  Amazon S3  as the core cloud object store for lakehouse data layers, with Databricks processing and transformation built on top of that storage foundation.  Contribute to migration and optimization activities involving  AWS Glue  for metadata cataloging, schema discovery, or ETL interoperability where required across the enterprise AWS data ecosystem. Support analytical integration patterns with downstream platforms such as  Amazon Athena  when business consumers require governed access beyond Databricks-native consumption paths.  Data Onboarding & Asset-Agnostic Support Support offshore teams in onboarding new data assets to the enterprise platform using standardized, asset-agnostic processes. Help teams prepare, validate, and publish data assets, ensuring consistency and compliance with platform standards. Document onboarding procedures and assist in resolving onboarding issues. Help standardize onboarding into  Bronze, Silver, and Gold  data layers and support practical lakehouse patterns for ingestion, curation, and consumption in Databricks on AWS.  Support metadata-driven onboarding workflows using  AWS Glue Data Catalog  and Databricks governance capabilities to improve discoverability, schema consistency, and reusable ingestion patterns. Assist with secure data onboarding by aligning access permissions and environment controls through  AWS IAM  and  AWS Lake Formation , alongside  Unity Catalog  permissions and governance policies for managed access to datasets.  Platform Integration Support Support integration of data pipelines with the enterprise data mastering platform. Assist in data quality checks, metadata management, and reconciliation activities. Help troubleshoot and resolve issues related to data ingestion and mastering workflows. Support interoperability across AWS and Databricks services for ingestion, transformation, and governed publishing of mastered or standardized datasets. Contribute to table design and data publication patterns using  Delta Lake  and, where applicable,  Apache Iceberg  to support interoperable lakehouse consumption models. Assist with auditability and lineage-oriented controls through metadata, access management, and governed