Title: Sr. Data Engineer

Duration: 6+ months

Location: Remote

Job Descriptions:

This is a senior developer role. Developer(s) will contribute to the development and deployment of the client's state-of-the-art big data platform.

Strong Spark, Scala/Python, Kafka, and Azure Cloud

Skills & Experience Required:

·        Scala/Python, Spark Structured Streaming, Spark Streaming, Kafka, micro-batching

·        Azure Event Hubs, Databricks, Stream Analytics, Data Explorer, Monitors

·        Reading, writing, and optimizing streaming data at scale – e.g. handling millions of connected devices sending billions of messages per day

·        Reading from and writing to RDBMs and NoSQL DB

Strong understanding of the following:

·        Event-Driven and Domain-Driven Architecture & Implementations

·        Backend development of micro-service within the Cloud platform (in Azure)

Essential Working Knowledge of:

·        Designing experience in High Available and Reliable Solutions

·        Observe, test, debug, and correct faults in a code base

·        Develop and produce documentation, including technical specifications

·        Hands-on experience in the unit test, integration tests, CI/CD process, and code coverage

Secondary Skills

·        Kubernetes

·        Full data management skills – data profiling, data analysis, data analytics, data transformation, data optimization, data governance, data anomaly handling

·        Working knowledge in Yugabyte, Hadoop, HDFS, and relational databases

·        Hands-on experience in Spark Streaming / Spark Structured Streaming