Description:
About the job At Minutes to Seconds, we match people having great skills with tailor-fitted jobs to achieve well-deserved success. We know how to match people to the right job roles to create that perfect fit. This changes the dynamics of business success and catalyzes the growth of individuals. Our aim is to provide both our candidates and clients with great opportunities and the ideal fit every time. We have partnered with the best people and the best businesses in Australia to achieve success on all fronts. We’re passionate about doing an incredible job for our clients and job seekers. Our success is determined by the success of individuals at workplace
We would love the opportunity to work with YOU!!
Our client is looking for a Data Engineer to join a dynamic team in Sydney on a permanent basis.
Requirements Primary Functions:
- Ensure reliable performance of the data warehouse through monitoring, performance tuning and intelligent scheduling of workloads.
- Manage the existing data platform infrastructure and perform necessary upgrades
- Manage and configure data engineering workflows
- Ensure integrity and accuracy of data models by developing data quality checks, reconciliation processes and thorough testing practices.
- Contribute to the ongoing support of releases in production.
- Monitor and ensure adequate logging and alerting are set up for data solutions.
- Maintain enterprise data warehouse/data lake and data engineering solutions on the cloud at an enterprise scale.
- Supports the development of agreed standards and tools
- Work with data consumers to design flexible data marts that are reliable, easy to use and efficient.
- Minimum of 3 years’ experience in a similar role with a background in data and support
- Time management and organisational skills to manage competing deadlines and conflicting priorities.
- Understanding of architecture patterns and technical Implementation details relating to security, compliance and governance for BI applications and Infra deployment on cloud
- Strong skills in Jira and Confluence.
- Problem solving
- Proficient with SQL with the ability to write very complex SQL queries and to query multiple data sources.
- Good understanding and experience with ETL, data structures and data modelling (e.g. data warehousing and data marts) and how to best optimise them
- Design and document data models, including the underlying principles, their relationship to the information architecture, and managing integrity and changes to those models.
- Experience in building/managing the cloud infrastructure required for a data platform in AWS.
- Proven expertise working with very large and complex datasets.
- Good understanding of Git and CI/CD pipeline technologies. Ideally buildkite, terraform, AWS CDK and Docker containers.
- Experience with schema design and dimensional data modelling.
Desirable/Bonus
- Experience in AWS Technologies like Redshift, EMR, EC2, CloudFormation, Security groups, VPCs, DMS, S3, EKS, Athena, Redshift, Glue, RDS, EC2, Snowflake, Kubernetes, Docker, Airflow and Lambda, but not essential
- Experience using Apache Spark for data processing
- Experience with data streaming applications (e.g., kinesis, Rabbitmq).
- Experience using Terraform for Infrastructure as code
- Experience maintaining Tableau Server or other Linux based applications.