Our direct client from the UK is seeking a Senior Data Engineer to join their team in developing cutting-edge mobile solutions for the e-commerce industry.
Your responsibilities:
● Build and run Azure based Data Platform
● Handle multiple Terabytes of different domains data with a focus on IoT data using Stream and Batch processing
● Craft and maintain world-class, high-traffic analytical applications and services with a team of experienced data engineers and scientists
● Develop data models and process raw data at scale to make it available for analysis
● Build scalable data pipelines using state-of-the-art technologies in the Microsoft Azure Cloud
● Implement, run, and deploy algorithms in cooperation with data scientists using the Hadoop ecosystem (e.g. Spark, Hive, HDFS) and the Microsoft Azure Stack)
What we expect: ● Certification University degree in Computer Science or related fields of study (Mathematics, Physics)
● At least 6 years of experience in architecting and building Data Lake and Enterprise Analytics Solutions
● Ability to optimize 'Big Data' data pipelines, design Data Lake architecture and data sets
● Experience with Design and Architecture of relational SQL and NoSQL databases
● Advanced hands-on SQL, Python, pySpark, and HDFS knowledge and experience working with relational databases for data querying and retrieval
● Experience with Design and Architecture of Azure big data frameworks: Azure Databricks, Azure Data Factory, Azure ML, SQL Data Warehouse, HDInsights, Stream Analytics, Azure Data Explorer - or their open source counterparts: Kafka, Spark, Blob Storages.
● Knowledge of DevOps Processes (including CI/CD) and infrastructure as a code fundamentals
● Knowledge of Data Quality issues and methods of dealing with them
● Nice to have: experience in designing, building, and integrating REST APIs
● Fluent in English
What we can offer you: ● Stable employment conditions based on an employment contract
● Paid employee referral program
● Private medical care
● Sport card
Image by fullvector on Freepik