Connecting...

Location
Sydney
Salary
Up to $1145 per day (Inclusive of super)
Job Type
Contract
Ref
BH-183956
Contact
Megan Edwards
Contact email
Email Megan
Posted
3 months ago
  • 6 month initial contract
  • Up to $1145 per day inclusive of super
  • Hybrid working - 2 days in CBD office
This role is working for a NSW government Department based in the CBD. They are moving their data analytics platform from on-premise SQL servers to Microsoft Fabric  / Databricks. The Azure data engineer will be responsible for developing best data engineering practices, building and maintaining the data pipelines, data models and standards for the organisation.  You will ensure new features and subject areas are modelled to integrate with existing structures and provide a consistent view.

Key accountabilities
  • Strategic understanding of how to align data systems with business objectives: The ability to understand the overall objectives and strategy of the business, identifying how data can support decision-making, and ensuring that the data infrastructure aligns with the organization's goals.
  • Architecting and delivering scalable data engineering best practices for Azure fabric: Hands on ETL data pipeline building leveraging Microsoft Fabric / ADF, Databricks and Delta Lake. Utilising control tables to ensure flexibility and simplified data ingestion reducing the number of data pipelines for raw data to be ingested from on-premise SQL systems (SQL2014, SQL2019) to Microsoft Fabric. Familiarity with deployment of Medallion architecture (Bronze, Silver, Gold) leveraging Python and SQL.
  • Azure devops champion: Be the data analytics Azure devops champion, introducing best practices (CI/CD, Git repos, version control, testing automation), tools, and mindset shifts to data analytics team enabling a DevOps culture and improve the overall efficiency of their workflows, particularly in delivering data analytics solutions
  • Data quality: Working with business stakeholders to identify business data quality rules. Leveraging DLT/DBT/great expectations libraries to manage data quality rules on data ingestion. These rules flag null values, out of range data or invalid data records.
  • Migrating on-premise code base (stored procedures / views) to data lake architecture: Simplifying and migration of existing legacy on-premise SQL code (stored procedures and views) to the cloud, specifically to a data lake architecture in Microsoft Fabric, organised into Bronze, Silver, and Gold layers. The migration must ensure that the legacy code is refactored for optimal performance in the cloud, while preserving the business logic. Close collaboration with business teams to gather requirements, challenge existing assumptions, and ensure that the cloud solution is effective for facilitating insights in Power BI
  • Data governance - Setup and maintain of active directory groups regarding provisioning and handling of PII data classifications in Bronze, Silver, Gold layers. This includes identifying of PII data ensuring that sensitive information is properly secured and governed.
The occupant of this role must demonstrate:
  • Tertiary qualifications or equivalent experience in a quantitative discipline, such as computer science, statistics or economics
  • Demonstrated extensive experience in one or more of the following: data analytics, Microsoft Azure data engineering architecture, project delivery;
  • Proven high-level ability to think critically and analyse, interpret and evaluate information;
  • Exceptional written and verbal communication skills, including the ability to communicate well with our diverse stakeholders, and to prepare complex correspondence and reports;
  • Experience delivering high-level strategic advice, proposals and business cases with the ability to assess business needs and identify business opportunities;
  • Demonstrated ability to work independently and as part of a team;
  • Previous experience in staff supervision and leading teams.
 If the above is a match to your skillset and experience please apply NOW and you will be contacted for a further discussion if your profile is deemed a suitable fit.

You will be contacted for a further discussion if your profile is a suitable match to the role requirements.
Only WORD FORMAT resumes will be accepted.