Senior Software Engineer – Observability Platform

Job Description

Senior Software Development Engineer (US Federal) | Workday

The Tone:
This is a full-time, hybrid role at Workday, with a primary pay location in Reston, VA. Workday is a Fortune 500 company and a leading AI platform for managing people, money, and agents, shaping the future of work. This role is crucial for empowering real-time insights across Workday’s platforms, infrastructure, and applications, specifically supporting U.S. Federal Government contracts by developing and operating a large-scale distributed data platform in air-gapped AWS cloud regions.

The TL;DR
• Role: Full Time
• Type: Full-time
• Location: Hybrid – Reston, VA
• Pay: $148200–$264000 yearly
• Team: Data Platform and Observability team
• Mission: Develop and operate a large-scale distributed data platform that fuels critically important Workday applications for U.S. Federal Government contracts.
• Tech Stack: Java, Python, Go, Kubernetes, OpenStack, Bare Metal, Prometheus Ecosystem, Elastic Search, Grafana, InfluxData stack, PromQL, InfluxQL, Helm, Ansible, Chef, Terraform, Linux

What You’ll Actually Do
• Service Design: Design and enhance critical Observability services, including Monitoring, Logging, Alerting, and Tracing.
• Data Development: Develop data capture and collection services utilizing leading technologies across diverse infrastructure environments.
• Pipeline Construction: Construct metrics ingestion pipelines, alert definitions, and automate dashboard lifecycle management.
• Cloud Management: Manage and optimize Workday within air-gapped U.S. AWS cloud regions, collaborating with commercial software engineering teams.
• Security Partnership: Partner with the Workday Federal Cyber Defense team to ensure security operations, cybersecurity posture, and platform security health.

The Must-Haves
• Background: BS/MS in Computer Science or a related technical field. Strong software engineering background with a consistent track record of delivering high-quality products at scale. Applicants must be United States citizens.
• Experience: 10+ years building high-performance data ingestion software, addressing concurrency, latency, and efficiency challenges.
• Skills: Exceptional coding skills (Java, Python); solid understanding of high-performance data capture and collection systems, API design, and reliable data delivery; good understanding of Time-Series Databases and Log collection and management; good knowledge of Kubernetes; fluency in Linux; experience with Infrastructure Automation (Ansible, Chef, Terraform).
• Bonus: Demonstrable experience with the Prometheus Ecosystem, Elastic Search, Grafana, and InfluxData stack; experience with PromQL and InfluxQL; hands-on experience with Kubernetes and Helm; preferred DoD 8570/8140 compliant with at least IAT Level Il certification; active TS/SCI w/CI Poly is preferred, or the ability to obtain and maintain one.