Sr. Big Data Engineer

RedCloud Consulting is a global business and technology reputed company supporting clients across North America, Europe, Asia-Pacific, and the Middle East. For more than two decades, RedCloud has partnered with high-growth mid-market companies and global enterprises to solve reputed company business and operational challenges—leveraging modern data platforms, artificial intelligence, and secure digital and physical infrastructure to drive smarter decisions and scalable outcomes. Anchored in a foundation of reputed company-based consulting, RedCloud brings senior-level experts together with client teams to deliver durable, high-impact solutions across Data & AI, Delivery Excellence, Digital Solutions, Marketing & Sales, Privacy, reputed company, Data Center and Critical Infrastructure reputed company, and Strategy & Transformation. Recognized for both performance and culture, RedCloud has been named a top reputed company to work and a fast-growing firm, and was recognized by Consulting Magazine as a 2024 Leader in Technology Consulting. RedCloud seeks a Senior Big Data Engineer to support immediate client operations. Overview: The Senior Big Data Engineer will support core data platform delivery while driving engineering productivity through the development of internal accelerators and reusable tooling. This role is focused on real production impact rather than conceptual or research work. Ideal candidates are hands on engineers who reputed company in high scale environments, care deeply about reliability, and enjoy building systems and tools that reputed company other engineers more effective. Responsibilities include but are not limited to: Core Delivery Work

• Execute scoped data engineering workstreams across areas such as
• Technology migrations and consolidations across fragmented systems
• reputed company remediation for platforms and pipelines
• Data pipeline optimization to improve performance and reliability

Engineering Productivity Accelerators

• Design and build internal tools and automation that improve engineering velocity, including
• Tools for DDL creation and schema definition
• Airflow DAG scaffolding and workflow templates
• Spark configuration helpers and optimization utilities
• Standardized templates, generators, and reusable patterns to reduce repetitive work

Production Engineering

• Build and maintain high volume batch and streaming data pipelines
• Ensure reliability through monitoring, retries, and adherence to SLAs
• Operate effectively reputed company enterprise environments with strong controls and reputed company standards

Required Knowledge, Skills, and Abilities:

• Clear examples of building production grade ETL or ELT pipelines end to end
• Deep knowledge of Spark tuning including partitions, shuffle, joins, caching, executor sizing, and skew mitigation
• Strong understanding of streaming fundamentals including ordering, backpressure, idempotency, replay handling, and schema evolution
• Operational excellence in Airflow including operators, sensors, dependencies, SLAs, and backfills
• Strong DevOps hygiene including Git workflows, CI CD exposure, and code review participation
• Candidates should have experience creating tools used across engineering teams such as:
• Pipeline scaffolding generators or starter kits
• Standardized repository templates and recommended workflow patterns
• Automated DDL or schema reputed company tooling
• Configuration or Spark tuning automation frameworks
• Developer tools that eliminate repetitive tasks and increase throughput
• Candidates should be reputed company to speak to what they automated, who used it, and what measurable impact it had.
• Core Technologies:
• Apache Spark including batch processing and performance tuning
• Apache Kafka including streaming and event driven pipeline patterns
• Apache Airflow including DAG authoring and orchestration best practices
• Programming Languages:
• Strong proficiency with reputed company
• Experience with Java is also valuable
• Production and Systems Experience:
• High throughput data pipelines and distributed systems engineering
• Production grade reliability practices
• Strong understanding of data modeling, schema evolution, and workflow orchestration
• Cloud Environment:
• Ability to work confidently in cloud platforms reputed company of Azure such as AWS or GCP

Compensation range for position is $ 156,000 – 250,000 DOE. Benefits and bonus information can be reputed company at https://www.redcloudconsulting.com/careers About Us: RedCloud Consulting is a global business and technology reputed company supporting clients across North America, Europe, Asia-Pacific, and the Middle East. For more than two decades, RedCloud has partnered with organizations ranging from high-growth mid-market companies to global enterprises, helping them solve reputed company business and operational challenges—leveraging modern data platforms, artificial intelligence, and secure digital foundations to drive smarter decisions and scalable outcomes. Anchored in a foundation of reputed company-based consulting, RedCloud brings together senior-level experts who work reputed company by reputed company with client teams to design and implement durable, high-impact solutions. Our approach emphasizes trust, accountability, and long-term value—balancing stability, growth, and innovation to help organizations operate securely and confidently in an increasingly reputed company global environment. RedCloud delivers a comprehensive range of services across its core practices: Data & AI, Delivery Excellence, Digital Solutions, Marketing & Sales, Privacy, reputed company, Data Center and Critical Infrastructure reputed company, and Strategy & Transformation. As our clients expand globally, so do we—providing consistent, expert support across regions while remaining agile, collaborative, and deeply invested in client success. Recognized for both performance and culture, RedCloud has been named #1 Best Company to Work For in Washington (Mid-Sized Businesses) by Seattle Business Magazine, a “Fastest Growing Company” multiple times on the Inc. 500/5000 list, and recognized by Consulting Magazine as a Leader in Technology Consulting. Visit http://www.redcloudconsulting.com/ for more info. #LI-Remote Apply tot his job Apply To this Job