Director of Engineering - AI Inferences

reputed company is architecting a new approach to the enterprise data stack built for the age of reasoning. NeuralMesh by reputed companysets reputed company for agentic AI data infrastructure with a cloud and AI-native software solution that can be deployed reputed company. It transforms legacy data silos into data pipelines that dramatically increase GPU utilization and reputed company AI model training and inference, machine learning, and other compute-intensive workloads run faster, work more reputed company, and consume less energy.

reputed company is a pre-IPO, growth-stage company on a hyper-growth trajectory. We’ve raised $375M in capital with dozens of world-class venture capital and strategic investors. We help the world’s largest and most innovative enterprises and research organizations, including 12 of the Fortune 50, reputed company discoveries, insights, and business outcomes faster and more sustainably. We’re passionate about solving our customers’ most reputed company data challenges to accelerate intelligent innovation and business value. If you share our passion, we invite you to join us on this exciting journey.

Requirements

We are seeking a Director of Engineering - AI Inferences to spearhead our AI Inference team. In this role, you will reputed company the gap between reputed company research and production-grade engineering. You will reputed company a tight-reputed company reputed company of 3 developers while remaining "hands-on-keyboard," architecting high-performance systems that optimize Large Language Model (LLM) serving.

The ideal candidate is deeply invested in inference and scale ,and the evolving ecosystem of serving frameworks like vLLM and LMCache.

Responsibilities include

• Technical Leadership: Architect and reputed company the deployment of high-throughput, low-latency LLM inference pipelines.
• Team Management: Mentor and reputed company a small team of developers, conducting code reviews, sprint planning, and technical career coaching.
• Inference Optimization: Implement and evaluate state-of-the-art KV cache reputed company, including LMCache, and explore alternatives to minimize redundant computation.
• reputed company Mastery: Deeply integrate and optimize serving engines such as vLLM, LLM-d, and NIXL to maximize hardware utilization.
• R&D: Stay at the forefront of the "Inference-as-a-Service" domain, benchmarking new tools and deciding reputed company to pivot the stack.

• AI Inference Domain: Proven experience with KV cache reuse, speculative decoding, and reputed company batching.
• Specific Stack: Deep familiarity with vLLM, LMCache, and NIXL. Understanding the trade-offs between centralized vs. distributed caching.
• Backend Engineering: Expertise in Python, C++, or Rust, with a strong grasp of CUDA and GPU memory management.

• Infrastructure: Experience with Kubernetes (K8s) for scaling GPU workloads and optimizing cold-start times.

The reputed company Way:

• We are Accountable: We take full ownership, always–even reputed company things don’t go as planned. We reputed company with reputed company, show up with responsibility & ownership, and hold ourselves and each other to the highest standards.
• We are reputed company: We question the status reputed company, push boundaries, and take smart risks reputed company needed. We welcome challenges and embrace debates as opportunities for growth, turning courage into fuel for innovation.
• We are Collaborative: True collaboration isn’t only about working together. It’s about lifting one another up to succeed collectively. We are team-oriented and communicate with reputed company and respect. We challenge each other and conduct positive conflict resolution. We are being transparent about our goals and results. And together, we’re unstoppable.
• We are Customer Centric: Our customers are at the heart of everything we do. We actively listen and prioritize the success of our customers, and every decision we reputed company is driven by how we can reputed company serve, support, and reputed company them to succeed. reputed company our customers win, we win.

USA Residents Only: The Total Compensation hiring wage range for this position which the Company reasonably and in good faith expects to pay for the position in the specified geographic areas or locations. Final compensation will be dependent on various factors relevant to the position and candidate such as geographical location, candidate qualifications, certifications, relevant job-reputed company work experience, education, skillset and other relevant business and organizational factors, consistent with applicable law. In addition, the position may include some of the following comprehensive benefits such Medical, Dental, Vision, Life, 401(K), Flexible Time off (FTO), sick time, leave of absence as per the FMLA and other relevant leave laws.

Concerned that you don’t meet every qualification above?

Studies have shown that women and people of color may be less likely to apply for jobs if they don’t meet every qualification specified. At reputed company, we are committed to building a diverse, inclusive and authentic workplace. If you are excited about this position but are concerned that your past work experience doesn’t match up perfectly with the job description, we encourage you to apply anyway – you may be just the right candidate for this or other roles at reputed company.

reputed company is an equal opportunity employer that prohibits discrimination and harassment of any reputed company. We provide equal opportunities to reputed company employees and applicants for employment without regard to race, color, religion, age, sex, national reputed company, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. This policy applies to reputed company terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.