Lead Data Engineer
Job Title: Lead Data Engineer
Duration: 12+ months
Location: Hybrid (Seattle, WA)
Rate: Experience dependent
As a Software Engineer Tech Lead, you will be responsible for leading a team of software engineers in an initiative to migrate an on-premises large-scale data warehouse and associated data pipelines to Azure. You will have core competencies in the big data analytics platforms including Spark/ Databricks as well as Data Warehouse/Data Lakehouse architectures. You will also have very strong project management skills, specifically using Agile Methodologies such as Scrum/Kanban/Lean.
Tops 3 Skills Needed
Experience in big data platforms
Agile / scrum
Experience uplifting/migrating big data to the cloud
Years of Experience:
- 10+ years
Degree or certifications required:
- Bachelor’s degree in Computer Science or related field
Required Background/ skills:
- Experience leading a team of software engineers in large-scale data warehouse and data pipelines migration.
- Experience designing and implementing data pipelines and workflow management
- Experience with data warehouse/lake house architectures and data modeling
- Strong problem-solving skills
- Strong communication and leadership skills
- Strong ability to drive engineering trade-off discussions with the goal of driving consensus
- Experience in Agile methodologies such as Scrum, specifically in the role of Scrum Master
- Experience with Azure Cloud Analytics Platforms including PaaS and IaaS services: Azure Data Lake Storage, Azure Data Factory, Kafka, Azure Synapse
- Experience with Analytics Platforms and related technologies: Spark/Databricks, Apache Streaming, Delta Tables, and Airflow
- Prod modernization – NDA
- Lead a team of software engineers in the migration of a large-scale data warehouse and associated data pipelines.
- Ability to have trade-off discussions, with engineering team, around data pipeline design with the goal of driving consensus on approach.
- Attend cross-functional meetings with project Technical Product Manager (TPM).
- Work with the TPM to translate requirements into backlog items
- Lead the engineering team in backlog grooming and refinement meetings.
- Work with team to prioritize product and engineering backlog items in planning meetings
- Lead daily stand-up meetings and drive action items and ownership around blocking items.
- Communicate schedule/project timeline status to engineering manager and TPM, highlighting key trade-off discussions.
- Mentor junior engineers and help them to grow their skills in story creation, decomposition, and estimation.
- Migration into the cloud
- Scrum master certification
- This is not strong hands on engineering, this would be closer to driving project management. Not looking for someone who is wanting to do the engineering need to lead the projects