Analytics Engineer 5 - Identity & Signal
Remote · Not stated
Hiring locations: Remote, United States
Open posting at NetflixCheck the posting for country and time zone requirements before applying.
Employer’s description
In 2022 we launched a new lower-priced, ad-supported tier, and we are building an in-house, world-class ad-tech ecosystem to give our members more choice and to offer advertisers a premium, better-than-linear-TV experience. We are looking for the founding members of this new business area for Netflix. The Ads Forecasting team builds the predictive foundation of the Netflix ads business — the models that tell us, before a campaign ever runs, how much inventory is available and how a campaign will deliver. We forecast supply and demand across audiences, ad products, and formats, and we predict campaign outcomes such as maximum availability, delivery confidence, reach, and frequency, accounting for the ad-serving optimizations that shape delivery. Our forecasts power media planning, underwriting, budget planning, yield, and the public API. This is a brand-new, foundational role. You will build the machine learning models that augment our simulation-based engine for predicting campaign delivery — turning a slow, rules-based simulation into fast, accurate, learnable models of how campaigns deliver against real inventory. You'll own the modeling and prototyping end-to-end and partner closely with our ML engineering team to take models to production. In this role, you will: • Build, prototype, and iterate on supervised machine learning models that predict campaign delivery outcomes — delivery risk, reach, frequency, and contention — to replace the current simulation engine. • Model demand-side campaign outcomes while incorporating supply-side signals, so the models reason about how well available inventory matches what advertisers are trying to achieve (targeting, frequency caps, contention, pacing). • Design rigorous offline and online evaluation frameworks to measure model accuracy, robustness to seasonality and distribution shift, and lift over the simulation baseline. • Own feature engineering and contribute to the team's feature store — turning ad-serving logs, campaign attributes, and supply signals into reusable, well-documented features. • Prioritize explainability and interpretability: your models' outputs must be defensible to sales and media-planning stakeholders making real booking and underwriting decisions. • Partner with ML engineers to deploy models at scale and to monitor production model health and drift, feeding monitoring insights back into the next modeling iteration. • Collaborate with cross-functional partners across product, engineering, and sales to define objectives, constraints, and trade-offs, and to drive adoption of ML-driven forecasts. • Communicate technical decisions, trade-offs, and results clearly to both technical and non-technical audiences at all levels of the company. We are looking for: • Advanced degree (PhD or Master's) in Statistics, Mathematics, Computer Science, or a related quantitative field. • 5+ years of relevant experience building machine learning models on large-scale data. • Deep expertise in supervised learning (e.g. gradient-boosted trees, regression, and related methods) with a strong bias toward interpretable, explainable models. • Strong feature engineering skills and familiarity with feature stores and standard ML lifecycle practice (versioning, evaluation, monitoring, retraining). • Proven ability to prototype algorithms and validate them rigorously against production data. • Strong programming skills in Python and strong SQL. • Working knowledge of ad-serving and campaign concepts — how campaigns are delivered and what creates delivery risk: targeting, frequency caps, contention, bidding, pacing, budget planning, and the core campaign objects/attributes; and the metrics that matter (reach, frequency, impressions, clicks, outcomes). You should understand both the supply side (ad-serving rules and inventory behavior) and the demand side (campaign attributes and advertiser goals). Ads experience is strongly preferred. • Ability to work independently, drive your own projects, and make compelling cases for prioritization. • Ability to communicate technical and statistical concepts clearly to audiences at many levels. • Embodies the Netflix values while bringing a new perspective to continue improving our culture. Nice to have: • Experience at a DSP, SSP, or publisher-side ad platform where predicting campaign outcomes at scale is a core science problem. • Familiarity with our ML stack (Metaflow) or comparable large-scale ML tooling. • Experience partnering with ML engineers to ship and monitor production ML systems. • Experience creating data products, dashboards, or explainability tooling for non-technical stakeholders. • Experience applying GenAI to boost developer/research productivity.