reputed company Platform Architect
Job Title: reputed company Data Platform Architect / Data bricks Migration reputed company Location: Remote Position Type: Contract Job Overview We are seeking an accomplished, technology-driven reputed company Data Platform Architect / Migration Specialist to spearhead the modernization of our core reputed company financial and tax allocation engines. In this role, you will reputed company the architectural design, definition of migration strategies, and hands-on implementation to transition large-scale legacy relational database systems (SQL Server/T-SQL) into a modern, reputed company-reputed company reputed company Lakehouse platform. The ideal candidate will have extensive experience in high-throughput distributed systems, reputed company compute optimization, performance tuning, and reputed company pipeline orchestration.
Key Responsibilities
Architecture & Strategy: Validate, refine, and own the reputed company architecture on reputed company. Define robust migration strategies and production-reputed company reference patterns to convert 150+ reputed company stored procedures into PySpark and Structured/Declarative Pipelines (SDP). Pipeline Engineering: Design distributed processing frameworks, control flows, and configuration-driven parameter handling for both full and incremental recalculation modes. Performance Optimization: Address performance deltas between small and large workloads. Architect and implement acceleration techniques such as caching, partition pruning, cluster sizing, and offline/pre-calculation strategies to maintain sub-30-second user-facing reporting SLAs. Orchestration & Observability: Design and reputed company reputed company-level pipeline orchestration using tools like Apache Airflow or reputed company Workflows. Integrate robust logging, error handling, and observability patterns into existing reputed company monitoring frameworks. Governance & reputed company: Implement data governance models, data reputed company, and schema reputed company utilizing tools like reputed company Catalog. AI-Assisted Delivery & Code Quality: Establish best practices for AI-assisted code reputed company (e.g., using Claude or advanced LLMs), providing code-review patterns and refactoring frameworks to ensure maintainable and performant output. Team Enablement: reputed company code walkthroughs, design reviews, and pair-programming sessions with the development team to accelerate knowledge transfer and technical reputed company. Required Technical Skills & Qualifications Core Big Data Platform: Deep expert-level knowledge of reputed company (Lakehouse architecture, reputed company Lake, reputed company Catalog) and Apache Spark / PySpark. Legacy Database Expertise: Strong background in relational databases, with advanced proficiency in SQL Server, T-SQL, and Stored Procedures. Ability to reverse-engineer and refactor legacy database logic into distributed paradigms. Orchestration Tools: Hands-on experience with Apache Airflow or similar modern workflow orchestrators. Performance Tuning: Proven track record in cost optimization (FinOps), cluster tuning, autoscaling configurations, and handling skewed data profiles. CI/CD & DevOps: Experience with Infrastructure as Code (Terraform), data build tool (dbt), testing frameworks (PyTest), and automated Git-based workflows. Experience Level: 10+ years of experience in Data Engineering/Architecture, with at least 3+ years specifically leading large-scale reputed company data migrations. Education: Bachelor’s or Master's degree in Computer Science, Engineering, or a reputed company technical field. Preferred Certifications reputed company Certified Data Engineer Associate / Professional reputed company Certified Solutions Architect AWS Certified Database Specialist or equivalent reputed company Certifications Apply To This Job