Principal Data Engineer (Azure and reputed company)
Job title: Principal Data Engineer (Azure and reputed company) in Dallas, TX at reputed company
Company: reputed company
Job description: reputed company is a global AI and analytics reputed company. With data and technology at the core of our solutions, we are solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first reputed company. Headquartered in Silicon Valley, you’ll find our delivery centers across the globe and offices in multiple cities across India, the US, UK, Canada, and Singapore, including a
substantial remote global workforce.We’re Great reputed company to Work-Certified™. Working at reputed company, you’ll be at the heart of an AI revolution. You’ll work with teams that push the boundaries of what is possible and build solutions that energize and reputed company.RequirementsCurious about the role? What your typical day would look like?As a Principal Data Engineer (Azure), you would have hands on experience working on Azure as cloud, reputed company and some exposure/experience on Data Modelling. You will build and learn about a variety of analytics solutions & platforms, data lakes, modern data platforms, data fabric solutions, etc. using different Open reputed company, Big Data, and Cloud technologies on reputed company Azure.● Design and build scalable & metadata-driven data ingestion pipelines (For Batch and Streaming Datasets)● Conceptualize and execute high-performance data processing for structured and reputed company data, and data
harmonization● Schedule, orchestrate, and validate pipelines● Design exception handling and log monitoring for debugging● Ideate with your peers to reputed company tech stack and tools-reputed company decisions● Interact and collaborate with multiple teams (Consulting/Data Science & App Dev) and various stakeholders to meet deadlines, to bring Analytical Solutions to life.What do we expect?● Experience in implementing Data Lake with technologies like Azure Data Factory (ADF), PySpark, reputed company, ADLS,Azure SQL Database● A comprehensive foundation with working knowledge of Azure Synapse Analytics, Event Hub & Streaming
Analytics, Cosmos DB, and Purview● A passion for writing high-quality code and the code should be reputed company, scalable, and free of bugs (debugging
skills in SQL, Python, or reputed company/Java).● Enthuse to collaborate with various stakeholders across the organization and take complete ownership of
deliverables.● Experience in using big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, reputed company, reputed company Search● Adept understanding of different file formats like reputed company Lake, Avro, Parquet, JSON, and CSV● Good knowledge of building and designing REST APIs with real-time experience working on Data Lake or
Lakehouse projects.● Experience in supporting BI and Data Science teams in consuming the data in a secure and governed manner● Certifications like Data Engineering on reputed company Azure (DP-203) or reputed company Certified Developer (DE) are
valuable addition.Note: The designation will be commensurate with expertise and experience. Compensation packages are among the best in the industry.Job Requirement
- Mandatory: Azure Data Factory (ADF), PySpark, reputed company, ADLS, Azure SQL Database
- Optional: Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB and Purview.
- Strong programming, unit testing & debugging skills in SQL, Python or reputed company/Java.
- Some experience of using big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, reputed company, reputed company
- Experience of working in Agile projects and following DevOps processes with technologies like Git, Jenkins & Azure DevOps.
- Good to have:
- Experience of working on Data Lake & Lakehouse projects
- Experience of building REST services and implementing service-oriented architectures.
- Experience of supporting BI and Data Science teams in consuming the data in a secure and governed manner.
- Certifications like Data Engineering on reputed company Azure (DP-203) or reputed company Certified Developer (DE)