Lead Data Engineer (Finance)
We are seeking a highly skilled and motivated Lead Data Engineer. This individual possesses a deep passion for data and thrives on architecting robust, scalable data pipelines that fuel our organization's data-driven initiatives.
The ideal candidate is a hands-on lead, experienced in designing and implementing complex data architectures. They have a strong grasp of data warehousing, ETL processes, and data modeling, and are always at the forefront of emerging technologies.
Excellent communication and collaboration skills are essential, as this role involves close interaction with various stakeholders across the organization. Experience with big data environments is a must, experience in the financial industry is a plus.
What you'll do
- Design, architect, and implement scalable and secure data pipelines on GCP, utilizing services like Dataflow, Pub/Sub, and Cloud Storage.
- Develop and maintain data models, ensuring data quality, consistency, and accessibility for various internal stakeholders.
- Automate data processes and workflows using scripting languages like Python, leveraging technologies like Spark and Airflow.
- Monitor and troubleshoot data pipelines, identifying and resolving performance issues
- proactively.
- Stay up-to-date with the latest trends and advancements in GCP and related technologies, actively proposing and evaluating new solutions.
- Implement data governance best practices, including data security, access control, and lineage tracking.
- Lead security initiatives, design and implement security architecture.
- Lead data quality initiatives, design and implement monitoring dashboards.
- Mentor and guide junior data engineers, sharing knowledge and best practices to foster a high-performing team.
- Role requires a solid educational foundation and the ability to develop a strategic vision and roadmap for D&A's transition to the cloud while balancing delivery of near-term results that are aligned with execution.
What experience you need
- BS degree in a STEM major or equivalent discipline; Master's Degree strongly preferred
- 8+ years of experience as a data engineer or related role, with experience demonstrating leadership capabilities
- Cloud certification strongly preferred
- Expert level skills using programming languages such as Python or SQL and advanced level experience with scripting languages.
- Demonstrated proficiency in all Google Cloud Services
- Experience building and maintaining complex data pipelines, troubleshooting complex issues, transforming and entering data into a data pipeline in order for the content to be digested and usable for future projects; Proficiency in Airflow strongly desired
- Experience designing and implementing advanced to complex data models and experience enabling advanced optimization to improve performance
- Experience leading a team with Git expertise strongly preferred
What could set you apart
- Expert level skills using Java.Self-starter that identifies/responds to priority shifts with minimal supervision.
- Strong communication and presentation skills
- Strong leadership qualities
- Demonstrated problem solving skills and the ability to resolve conflicts
- Experience creating and maintaining product and software roadmaps
