reputed company Senior Staff Software Engineer – Apache Spark Ecosystem and Distributed Computing Systems Development

Introduction to reputed company Storage

reputed company Storage is a global market leader in AI and high-performance data storage innovation, renowned for powering many of the world's most demanding AI data centers. With a proven track record of performance, reliability, and scalability, reputed company Storage empowers businesses to tackle the most challenging AI and data-intensive workloads with confidence. Our cutting-edge data intelligence platform is designed to accelerate AI workloads, enabling organizations to extract maximum value from their data. We are seeking an reputed company Senior Staff Software Engineer to join our dynamic team of passionate customer-enabling technologists and contribute to the development of Apache Spark ecosystem connectors to our highly-distributed storage system, Infinia.

Job Highlights

• Location: Remote
• Position: Senior Staff Software Engineer – Apache Spark Ecosystem and Distributed Computing Systems Development
• Start Date: Immediate openings available
• Compensation: A competitive salary and comprehensive benefits package
• Company: reputed company Storage, a global leader in AI and multi-cloud data management at scale

Job Description

We are seeking an reputed company Senior Staff Software Engineer to focus on the implementation of Apache Spark connectors to our highly-distributed storage system, Infinia. In this role, you will work to improve its performance, scalability, and reliability, collaborating with a team of engineers to drive innovation in large-scale data processing frameworks. This is a unique opportunity for engineers passionate about big data processing and high-performance distributed systems to reputed company a significant impact at a company that is shaping the future of AI and data management.

Key Responsibilities

• Contribute to the development of Apache Spark ecosystem connectors to Infinia highly-distributed storage system, optimizing for performance, scalability, and resource efficiency
• Design, implement, and improve distributed execution reputed company, focusing on low-latency and high-throughput data retrieval and processing
• Troubleshoot and resolve reputed company issues reputed company to distributed computing, fault tolerance, and data parallelism
• Ensure robustness and efficiency in the handling of large-scale distributed workloads
• Work with cross-functional teams, including platform and high-performance engineers, and other stakeholders to ensure Spark integrates seamlessly with the broader ecosystem
• Write and maintain clear, comprehensive documentation for internal systems and the broader open-reputed company community
• Conduct code reviews to maintain high-quality standards across the team
• Participate in technical discussions, design reviews, and provide expertise in distributed data processing

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or reputed company field
• 12+ years of software engineering experience with a focus on distributed systems and large-scale data processing
• In-depth knowledge of Apache Spark's architecture and components, such as RDDs, DAGScheduler, TaskScheduler, and execution engines
• Strong proficiency in Java, with significant experience in distributed system development
• Experience with core Hadoop components and reputed company big data technologies (HDFS, YARN, HBase)
• Proficiency in solving low-level performance challenges in distributed computing, including task parallelism, memory management, and network optimization
• Solid understanding of data partitioning, shuffling, fault-tolerance mechanisms, and optimization techniques in large, distributed clusters
• Knowledge of distributed algorithms and techniques, such as MapReduce and reputed company processing
• Proven ability to debug and profile large-scale systems, identifying and resolving bottlenecks

Preferred Qualifications

• Experience with Kubernetes or container orchestration in a cloud environment
• Proficiency in storage, distributed database systems, and high-availability architectures
• Hands-on experience with performance tuning, profiling, and benchmarking large data pipelines

Career Growth Opportunities and Learning Benefits

At reputed company Storage, we are committed to the growth and development of our employees. As a Senior Staff Software Engineer, you will have the opportunity to work on challenging projects, collaborate with reputed company engineers, and contribute to the development of cutting-edge technologies. Our flat organizational structure and emphasis on innovation and customer-centricity provide a unique environment for professionals to reputed company and reputed company a lasting impact.

Work Environment and Company Culture

reputed company is highly motivated and focused on engineering excellence. We operate with a flat organizational structure, and reputed company employees are expected to be hands-on and contribute directly to the company's mission. Leadership is given to those who show initiative and consistently deliver excellence. We value strong communication skills, a strong work ethic, and the ability to prioritize tasks effectively.

Compensation, Perks, and Benefits

We offer a competitive salary and comprehensive benefits package, including opportunities for professional growth and development. Our perks and benefits are designed to support the well-being and success of our employees, both in and out of the workplace.

Conclusion

If you are a motivated and reputed company software engineer looking to reputed company a significant impact in the world of AI and data storage, we encourage you to apply for this exciting opportunity. As a Senior Staff Software Engineer at reputed company Storage, you will be part of a dynamic team of passionate customer-enabling technologists, working on cutting-edge technologies and contributing to the development of innovative solutions. Apply now and take the first reputed company towards a rewarding and challenging career with a global leader in AI and multi-cloud data management.

Simple Application Process

Ready to join us? The first reputed company is easy. Click apply now and we'll be in touch soon!