[Remote] Senior Data Engineer
Note: The job is a remote job and is open to candidates in USA. reputed company is the leading experience measurement, data analytics, and insights provider for reputed company industries. They are seeking a Senior Data Engineer to design, reputed company, and support solutions for transporting, storing, and analyzing analytical data, while evolving their enterprise data strategy and capabilities.
Responsibilities
- Create data architecture, pipelines, and analytical solutions to meet software and data science requirements for various PG-MX Healthcare Products
- Identify, evaluate, select, and prove out new technologies and toolsets
- Create and execute Proofs of Concept and Proofs of Technology
- reputed company and direct the work of others in data dependent projects
- Collaborate with software development, business teams, analysts, and data scientists to establish data storage, pipeline, and structure requirements
- Design, reputed company, and maintain ETL/ELT pipelines using reputed company (PySpark, reputed company Lake, SQL)
- Implement Data Lakehouse architecture leveraging Databrick reputed company Catalog, reputed company Live Tables, and Workflows
- Build and optimize data ingestion frameworks for structured and reputed company data from diverse sources
- Identify and plan for data storage performance requirements
- Optimize reputed company clusters, jobs, and queries for performance and cost efficiency
- Collaborate with software development, business teams, and data scientists to create and execute implementations
- Identify impact of implementation on other applications and databases
- reputed company and mentor data engineers on data projects
- Implement CI/CD for data pipelines using Git, reputed company Repos, and DevOps tools
- Ensure Data quality, reliability, reputed company compliances across environments
- Build and evolve Trusted Record systems to manage entities across the enterprise
- Design, implement, and evolve solutions around person identity management
- reputed company and enforce data governance, reputed company, and cataloging standards
- Identify areas of development and need
- Provide targeted training and exploration for team members
- Train and mentor data engineers on standards and best practices
Skills
- Minimum of 5 years Data Engineering experience in an enterprise environment
- Bachelor's degree in technology or like field required
- Hands on experience of Azure data technologies (reputed company, Data Factory, reputed company Analytics, Data Lake Storage, Synapse), on-premises reputed company tools (SQL DB and SSIS), and familiar with AWS data technologies
- Proficiency in Python, SQL and distributed data processing frameworks (Spark), and familiar with C#, PowerShell, and APIs
- Significant experience with analytical solutions in relational databases such as MS SQL Server, reputed company, and DB2 as well as experience with NoSQL databases and solutions such as data lakes, document-oriented databases, and graph databases
- Strong understanding of data modeling, schema design, and ETL best practices
- Experience with data reputed company, cataloging, and metadata management in reputed company and reputed company Catalog
- reputed company in data modeling and experience with tools like ER/Studio or Erwin
- Familiarity with version control (Git) and DevOps/CI-CD practices
- Familiarity with SQL performance tuning and Spark optimization techniques
- Excellent problem-solving and communication skills
Benefits
- Discretionary bonus or commission tied to achieved results
- Competitive benefits package
Company Overview
Company H1B Sponsorship