Data Engineering - Lead / Architect
Experience:
Minimum 10
years' experience in designing and data modeling for Data Lake / Data Warehouse / Data Hub projects.
Exp in Designing Big Data Platforms on Cloud (Google Cloud Platform / Azure) and Big Data technologies:
Hadoop, Spark (PySpark) andKafka Exp in writing Spark code in Scala or Java or Python for data ingestion, integration, transformation and data quality Experience in Data Modeling and Data Lake modeling on Cloud Platforms (Google Cloud Platform, Azure) Designing and Building Data Pipelines in Spark and deploying on Spark Engines (EMR, Databricks, Dataproc) Experience in handling and processing different data sources (file, db, streams, event sources, APIs) and file formatslike Parquet, Avro, Orc, Json, CSV Experience in test driven development and automating the test cases.
Experience in Stream processing.
Spark / Flink streaming is optional and a plus.
Experience in Designing and Data Modelling for Datawarehousing/DataLakehouse platforms on Teradata, Redshift,Snowflake and Databricks is a plus.
Proficiency in:
Responsible for logical and physical Data Model Design and Implementation on Hadoop/HDFS, Real-Time Systems,Data Warehouses, and Data Lakes Knowledge of system development lifecycle methodologies, such as waterfall and AGILE.
An understanding of data architecture and modeling practices and concepts including entity-relationship diagrams,normalization, abstraction, denormalization, dimensional modeling practices.
Experience generating physical data models and the associated DDL from logical data models.
Experience enforcing data modeling standards and procedures.
Knowledge of OLTP/OLAP technologies, data strategy disciplines, relational databases, data warehouse developmentand Big Data solutions.
Ability to work collaboratively in teams and develop meaningful relationships to achieve common goals.
Data Engineering - Lead / Architect Recommended Skills Api Agile Methodology Amazon Redshift Apache Flink Apache Hadoop Apache Kafka Estimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
