Principal Engineer - Marketing Technologies

About this role: Wells Fargo is seeking a Principal Engineer to lead observability and reliability engineering for Marketing Technology platforms. This role focuses on enabling end-to-end visibility across a complex data ecosystem spanning SaaS platforms, hybrid cloud, and on-prem systems. This position is responsible for designing and advancing full-stack observability for ETL pipelines, integrations, and data movement across marketing platforms. The goal is to improve reliability, reduce time to detect and resolve issues, and ensure consistent data flow supporting critical customer engagement channels. You will join a focused team of engineers and application support professionals driving adoption of observability, automation, and Site Reliability Engineering (SRE) practices across Marketing Technology. The team partners closely with data engineering, platform teams, and vendors to deliver stable, scalable, and highly visible systems. The role supports all aspects of the platform lifecycle—from instrumentation and monitoring design to incident response and continuous improvement—ensuring strong operational health, proactive alerting, and resilient data pipelines across distributed systems. The team operates across a global footprint, enabling a follow-the-sun support model and consistent platform reliability. In this role, you will: Define observability strategy Establish standards, patterns, and designs for monitoring distributed data systems Drive adoption of tools such as Grafana, Splunk, Prometheus, and OpenTelemetry Enable end-to-end visibility Instrument ETL pipelines, integrations, and APIs with metrics, logs, and traces Build dashboards for data flow health, latency, throughput, and error tracking Monitor data movement across SaaS, hybrid, and on-prem environments Strengthen data reliability Implement proactive alerting for pipeline failures, latency spikes, and data integrity risks Identify and isolate issues across multi-step data hops and integrations Partner with data engineering teams to embed observability into ETL frameworks (Airflow, Informatica) Lead incident response and problem resolution Drive triage for high-severity data and platform incidents Distinguish root causes across data, platform, and vendor layers Perform root cause analysis and implement long-term fixes Develop runbooks and playbooks for repeatable issue resolution Manage change and vendor coordination Ensure observability readiness for vendor patches, hotfixes, and configuration changes Validate monitoring coverage post-change to maintain visibility and alert accuracy Participate in change advisory processes and track vendor release cycles Document change impacts and lessons learned for audit and compliance Drive automation and continuous improvement Automate observability setup for new pipelines and integrations Implement anomaly detection and predictive alerting Continuously refine dashboards, thresholds, and alert logic based on trends Influence architecture and engineering practices Partner with Enterprise Architecture and engineering teams to align solutions with standards Advocate for scalable, observable system design across Marketing Technology Contribute to technology strategy and roadmap decisions Support operational excellence Share responsibility for production support of critical data flows and applications Improve key metrics such as availability, time to detect, and time to recover Promote SRE practices including SLIs, SLOs, and error budgets Required Qualifications: 7+ years of Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education 5+ years designing and implementing observability solutions (e.g., Splunk, Grafana, Prometheus, OpenTelemetry) 3+ years of experience with cloud and container platforms (Kubernetes, OpenShift) Desired Qualifications: Experience with databases (Oracle, SQL Server, PostgreSQL,