System Development Engineer

Our Fortune 500 client (One of the world's companies) in Seattle WA  is looking for hardworking, motivated talent to join their innovative team. 

Are you a System Development Engineer with a passion for collaborating with multiple teams and an interest in working onsite? Don't wait... apply today!
 
Position Title: System Development Engineer
Location: Onsite: Seattle WA 98101 
Duration: 6+ Months with possibilities of extension  

Job Description:
As a member of the Region Flexibility Migration (RFM) team, you will be responsible for driving Amazon's DRAM memory optimization initiative focused on reducing the memory footprint of EMR on EC2 clusters through infrastructure-level tuning, cluster configuration optimization, and instance family right-sizing.

You will analyze cluster metrics, evaluate Spark runtime configurations, and recommend optimal cluster topologies that minimize DRAM consumption while preserving job performance and reliability.
This is a high-impact, independently driven role where you will assess EMR cluster utilization patterns, identify over-provisioned resources, and implement configuration-level optimizations, including instance type migrations (e.g., R-family to C-family), Spark executor tuning, YARN resource allocation adjustments, and dynamic scaling configurations. You will collaborate with service teams and RFM engineering leadership to validate changes and establish reusable frameworks for fleet-wide EMR optimization.

Key Responsibilities
•    Analyze EMR cluster metrics, Spark application telemetry, and YARN resource utilization to identify over-provisioned memory allocations, underutilized executors, and suboptimal cluster configurations that contribute to excessive DRAM consumption.
•    Recommend and implement cluster-level optimizations, including instance family right-sizing (e.g., migrating from memory-optimized R-type to compute-optimized C-type instances), node count adjustments, EBS volume configurations, and spot/on-demand fleet composition changes.
•    Tune Spark runtime configurations at the cluster level, including executor memory/core ratios, YARN container sizing, dynamic resource allocation settings, memory overhead parameters, and shuffle service configurations, to achieve optimal memory utilization without impacting job SLAs.
•    Perform custom operations and iterative experiments using Amazon internal tooling to validate optimization impact: own end-to-end deployment, test execution, metric validation, and derive actionable insights from results.
•    Collaborate with service teams to review cluster architectures, discuss findings, propose optimization plans, and align resolution strategies while communicating effectively across engineering leadership and technical stakeholders.
•    Monitor service health metrics and troubleshoot operational issues during and after optimization activities, ensuring zero degradation to job completion times, data processing throughput, and downstream SLAs.
•    Develop comprehensive operational runbooks, SOPs, documentation, and technical specifications that capture cluster optimization patterns and can be consumed by both human engineers and AI agents to orchestrate optimization workflows at scale.
•    Extract scalable learnings from optimization engagements and develop programmatic frameworks that enable the initiative to scale across hundreds of EMR clusters, including training and enabling other vendor engineers to execute optimization playbooks.
Required Qualifications:
•    Bachelor's degree in computer science, Engineering, or equivalent technical field.
•    5-7+ years of hands-on experience operating, configuring, and tuning EMR clusters in large-scale production environments, with demonstrable results in cost or resource optimization.
•    Deep expertise in the Spark/YARN ecosystem operations, including cluster re-sizing and instance type selection for memory optimization, YARN resource manager configuration, Spark memory model (executor memory, memory overhead, off heap), and dynamic allocation tuning.
•    Strong knowledge of AWS EC2 instance families and their performance characteristics, with the ability to evaluate trade-offs between memory-optimized, compute-optimized, and general-purpose instances for data processing workloads.
•    Proficiency in interpreting CloudWatch metrics, EMR cluster logs, Spark History Server data, and YARN resource manager metrics to perform data-driven capacity analysis and make right-sizing recommendations.
•    Proficiency in using generative AI tools and assistants as part of daily engineering workflows to accelerate problem-solving, analysis, and technical documentation.

Preferred Qualifications:
•    Direct experience with EMR fleet management at scale (50+ clusters), including automated scaling policies, instance fleet configurations, and cost optimization strategies.
•    Experience with Amazon internal tools including Amazon Profiler, CloudWatch, X-Ray, and load/stress testing frameworks for validating cluster optimization impact under production-like conditions.
•    Understanding Spark workload characteristics (batch, streaming, interactive) and ability to recommend cluster configurations tailored to each workload type for optimal memory utilization.
•    Experience developing AI agents or automated workflows that can orchestrate tasks, extract information from services, and coordinate optimization activities across multiple systems.
•    Strong attention to detail and effective communication abilities: able to present findings, propose strategies, and influence service team stakeholders and engineering leadership.
•    Familiarity with CI/CD pipelines, deployment automation, and Amazon deployment technologies and best practices for infrastructure changes.

A Day in the Life:
Your morning might begin by reviewing CloudWatch dashboards for a fleet of EMR clusters: examining memory utilization heatmaps, YARN container allocation efficiency, and identifying clusters running at 30% memory utilization on R5.4xlarge instances. By mid-day, you are running a controlled experiment: migrating a staging cluster from R-family to C-family instances with adjusted Spark configurations, validating that job completion times remain within SLA.
After lunch, you are on a call with a service team, presenting your capacity analysis showing they can achieve 40% DRAM savings by moving to a different instance family with tuned executor configurations. Before wrapping up, you are documenting the migration pattern into a reusable playbook: one that another engineer (or an AI agent) could pick up and apply to a similar cluster topology tomorrow.
You operate with high autonomy, own your end-to-end investigations, and thrive on making infrastructure leaner without breaking production workloads.

About the Team:
The Region Flexibility Migration (RFM) team is part of Amazon's Intelligent Cloud Hosting (ICON) organization. The RFM team was formed in 2024 with the purpose of proactively discovering and mitigating issues that service teams encounter when migrating services across AWS regions. Over the past 2 years, the team has developed deep expertise in AWS service architectures, infrastructure optimization, and large-scale migration programs.
In 2026, the RFM team expanded its mission to include a critical DRAM memory optimization initiative targeting significant memory reduction across Amazon's service fleet. We are looking for an exceptional engineer with deep EMR operations and Spark ecosystem expertise to drive cluster-level optimization across Amazon's data infrastructure

Are you interested ? Please click apply button!

If you are not available or if this is not the right role at the moment, please share the job description with your friends and let us know if any of them show interest.