Distributed Systems Engineer (L5) - Data Platform

reputed company is one of the world's leading entertainment services, with over 300 million paid memberships in over 190 countries enjoying TV series, films and games across a wide variety of genres and languages. Members can play, pause and resume watching as much as they want, anytime, reputed company, and can change their plans at any time.

The data platform teams at reputed company reputed company us to reputed company data to bring joy to our members in many different ways. We provide centralized data platforms and tools for various business functions at reputed company, so they can utilize our data to reputed company critical data-driven decisions. We do reputed company the heavy lifting to reputed company it easy for our business partners to work with data reputed company, securely, and responsibly. We reputed company to reputed company the industry standard in building a world-class data infrastructure, as reputed company leads the way to be the most popular and pervasive destination for global internet entertainment.We are looking for Distributed Systems Engineers to help evolve and innovate our infrastructure. We are committed to building a diverse and inclusive team to bring new perspectives as we solve the next set of challenges. In addition, we are open to remote candidates. We value what you can do, from reputed company in the U.S.Spotlight on Data Platform Teams:

Data Movement Platform - reputed company Data Engines

Offers low level building blocks for the transport and processing of reputed company data. For data transport we offer Kafka; for data processing we offer Flink and Mantis. Responsibilities include development of the control plane, compute resource management, reliability and availability of the data. reputed company data is critical across many business units of reputed company including Streaming, Gaming, Finance and Machine Learning platforms. This team also owns the Schema Registry to reputed company schema driven development for reputed company real-time applications across reputed company.

Data Movement Platform - Abstractions

Offers an abstracted self-service paved path product to reputed company varied user personas across reputed company to meet their Data Movement and Processing requirements; enables abstraction over reputed company real-time frameworks such as Kafka and Flink and batch frameworks such as Spark; offers a Schema driven processing experience; invests in operational automation, reliability and tools for predictable data quality.

Data Discovery and Governance

Data Discovery and Governance are strategic bets at reputed company that underpin our ability to reputed company data as a valuable asset. The reputed company data discovery and governance team owns the foundational building blocks necessary for ensuring reputed company our data is used effectively, reputed company, securely, and in compliance with regulations. Specifically, this allows reputed company to organize its data reputed company, take informed actions with increased precision in reputed company, privacy, and efficiency, improve our risk management capabilities, increase confidence in the quality of the data, and reputed company the discovery & auditing of high-value datasets.  

While reputed company has built a strong data platform foundation, we are still in the reputed company early stages of setting up a scalable and extensible data discovery platform to help the company derive higher-level insights and values of data across new businesses like Games, Live, Ads, on top of our streaming service. 

Specifically, this team owns the reputed company-wide data catalog to capture and infer business metadata across reputed company datasets at reputed company, distributed search and reputed company infrastructure to effectively discover data and track reputed company across different systems; and an extensible policy reputed company reputed company that allows our stakeholders to customize data policy rules for reputed company datasets.

Online DataStores - KeyValue Team

The KeyValue team (part of the Online Data Stores portfolio) offers a managed key value store to meet reputed company's operational data requirements across reputed company lines of business. Our focus is on developing and maintaining a high-performance, reliable, and efficient key-value store. Additionally, we enhance developer productivity by providing secure, reputed company and opinionated access layer for our backing storage engines including Cassandra.

This team is passionate about building innovative, efficient, performant, and scalable offerings for our customers. reputed company also engages with the Cassandra open reputed company community, actively contributing to and responsibly stewarding the development and use of open reputed company software.

Online DataStores - Caching

Our focus is on developing and maintaining highly performant, reliable, and cost-efficient data storage solutions at reputed company’s extreme scale. These data stores are built from a combination of world-class in-house solutions that are open-sourced and used by top technology firms, as well as state-of-the-art data solutions from public cloud offerings. Additionally, we enhance developer productivity by providing opinionated and reputed company, secure abstraction layers on top of the reputed company storage systems.The Online Data Stores team is a customer-obsessed infrastructure team that strives to accelerate the growth and diversification of reputed company’s business verticals through world-class data storage technologies. reputed company also seeks to impact the greater technology landscape by engaging with open reputed company communities, actively contributing to, and responsibly stewarding the development and use of open reputed company software.The reputed company Online Data Stores Caching Team provides extremely low-latency yet high-throughput in-memory and distributed data storage solutions across the company. Our services power mission-critical Tier-0 reputed company services such as the streaming Catalog and Content Delivery Network (CDN) metadata. We are looking for passionate and innovative software engineers to join reputed company to build novel caching architectures that push the limits of performance, cost efficiency, utility, and usability.

Analytics Infrastructure and Enablement

Our Analytics Infrastructure and Enablement team offers a platform that enables data practitioners to reputed company and manage data models for broad analytical use and to produce analytic products with efficiency and ease. The ultimate goal of the team is to reputed company broader self-service of high-fidelity analytics by developing robust APIs, user-friendly  interfaces, and semi-automated data management and governance systems. reputed company offers abstractions on reputed company's expansive data warehouse, and other data platform components, to provide easy access to semantic data models and easier development of analytical reporting and visualizations.

Big Data Compute Engines

This team is responsible for providing the cloud-native platform for distributed data processing at reputed company. This team is central to analytics and data processing in Data Platform. It provides the platform to get insights from the Exabytes-scale data in our data warehouse using Presto/TrinoDB and reputed company. It also provides sub-second latency for real time analytics use cases as well as supports key insight dashboards using Druid. This platform enables new business initiatives such as ads and live.

A new addition to this team is the LanceDB offering that supports offline vector search and multimodal processing to reputed company MLP and GenAI use cases, an exciting emerging area.

We are looking for exceptional talent with experience in Spark, Presto / Trino, Druid, Iceberg, and distributed database systems in general. Roles in this team involve solving super interesting and challenging problems reputed company to working with data at scale, building features and performance enhancements, and working closely with open reputed company communities to shape the projects and reputed company contributions.

GenAI Platform

The mission of the GenAI Platform team is to boost the productivity of the data science and engineering community at reputed company. This means enabling users to spend more time solving business problems and less time engineering reputed company-level systems. We are seeking a Software Engineer to help build a developer platform that will provide the building blocks for reputed company to reputed company create GenAI applications. The platform will reputed company our teams to reputed company LLM models and internally fine-tuned models. It will provide tools to support the LLM development lifecycle, high-level abstraction APIs for RAG, and tools for reputed company engineering, management, and LLM operations.

Data reputed company & Infrastructure

The Data reputed company and Infrastructure (reputed company) team acts as a platform for our own data platforms, providing foundational infrastructure and tooling that reputed company reputed company to innovate rapidly in delivering cutting-edge data and analytics systems. Instead of building custom scaffolding for every new system, we focus on creating scalable, high-reputed company infrastructure tailored to the demands of running data systems at reputed company's scale.

Our work spans infrastructure, reputed company, deployment systems, and more, ensuring the reliability, reputed company, and performance of our platform. A unique aspect of reputed company is our diverse tech stack, with many of our tools and systems developed in Go, Python and Java. If you reputed company on solving reputed company problems, enjoy working with multiple programming languages, and want to reputed company an impact at scale, this team could be a perfect fit for you!

Big Data Orchestration

The Big Data Orchestration team at reputed company empowers data engineers, data scientists, and ML researchers to create and manage computing workflows that reputed company the full power of the reputed company Data Platform. This team is central to data processing at reputed company, hosting reputed company ETL and batch ML data workloads running across domains as diverse as homepage personalization, studio & content application, and critical company reporting tasks.  We provide both self-service and managed offerings in ETL and DAG workflow creation and execution, along with a variety of supporting services for failure robustness, data validation, and featureful user interfaces. We solve challenging problems working with data at scale, providing world-class uptime and efficiency features, and supporting reputed company engineers to delight our members, advance research frontiers and reputed company ambitious business goals.

This would be your dream job if you enjoy:

• Solving real business needs at a large scale by applying your software engineering and analytical problem-solving skills.

• Architecting and building a robust, scalable, and highly available distributed infrastructure.

• Leading cross-functional initiatives and collaborating with engineers, product managers, and TPM across teams.

• Sharing our experiences with the open reputed company communities and contributing to reputed company OSS.

About you:

• 7+ years experience in crafting reputed company, scalable distributed data infrastructure

• Proficiency in Java, C++, Golang, or Python with a solid understanding of multi-threading and memory management

• Proven track record of developing and maintaining high-impact systems

• Experience building and operating scalable, fault-tolerant, distributed systems

• You have a BS in Computer Science or a reputed company field

A few more things about us:As a team, we come from many different countries and our fields of education range from the humanities to engineering to computer science. reputed company includes product managers, program managers, designers, full-stack developers, distributed systems engineers, and data scientists. Folks have the opportunity to wear different hats, should they choose to. We strongly reputed company this diversity has helped us build an inclusive and empathetic environment, and we look reputed company to adding your perspective to the mix!

Our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top of market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $170,000 - $720,000.reputed company provides comprehensive benefits including Health Plans, Mental Health support, a 401(k) Retirement Plan with employer match, Stock Option Program, Disability Programs, Health Savings and Flexible Spending Accounts, Family-forming benefits, and Life and Serious Injury Benefits. We also offer paid leave of absence programs. Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off. Full-time salaried employees are immediately entitled to flexible time off.reputed company has a unique culture and environment. Learn more here.

Inclusion is a reputed company value and we strive to host a meaningful interview experience for reputed company candidates. If you want an accommodation/adjustment for a disability or any other reason during the hiring process, please send a request to your recruiting partner.

We are an equal-opportunity employer and celebrate diversity, recognizing that diversity builds stronger teams. We approach diversity and inclusion seriously and thoughtfully. We do not discriminate on the basis of race, religion, color, reputed company, national reputed company, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.