Our professional services client is looking for an experienced Data Engineer to join an expanding team and help build smart data applications for a massive Canadian retail client.
Role & Responsibilities
- Write well-structured, beautiful, clean code.
- Design smart ways of storing and displaying complex data.
- Check the quality of your contributions by supporting automated testing
- Help shape strategic analytics roadmaps and architectures.
- Be aware of new technologies and be keen to review and explore them when appropriate.
- Support and train your analyst colleagues to use new modelling and analytical tools.
- Give your opinions, backed by data and investigation, to the design forum.
- Share ideas by encouraging debate
- Extend the web applications to cover more predictive and prescriptive functionality.
- Undertake refactoring models to operate on distributed computing platforms.
- Add new functionality and components to link model outputs.
- Creating connectors to additional external data sources.
Experience & Skills Required
- Educated to degree level in a relevant discipline
- Have a deep understanding of platform architecture
- Demonstrable experience writing code.
- Possess excellent knowledge of Unix scripting and one of R, Python, or Java, and Scala is a must.
- Understand cloud computing and its security, automation and manageability
Great communication skills with the ability to converse at all levels in the business.
Technologies you will be expected to use:
- Application architectures; version control.
- Amazon Web Service APIs.
- Apache Spark, mldb.ai, Zeppelin and Apache Hadoop, Hive and HBase.
- AWS EMR, EC2, S3.
- Docker, Docker Compose and Jenkins.
- Pentaho Data Integration and PostgreSQL.
- Bitbucket, Docker Hub and JIRA.