This is an exciting opportunity for a skilled Data Engineer to join a leading financial institution in Johannesburg. The role focuses on designing, building, and optimising robust data pipelines and architectures to support the organisation's growing data needs and next-generation product initiatives. If you have a passion for data wrangling, enjoy building data systems from the ground up, and thrive in a dynamic environment, this position could be a perfect fit.

About the Role

The successful candidate will be instrumental in expanding and refining the organisation's data and data pipeline architecture. This involves ensuring optimal data flow and collection for various cross-functional teams. You will be a key player in supporting software developers, database architects, data analysts, and data scientists on critical data initiatives, ensuring that the data delivery architecture remains consistent and efficient across all ongoing projects. This is a self-directed role requiring comfort in supporting the data requirements of multiple teams, systems, and products simultaneously. A significant part of the responsibility will involve optimising or even re-designing the existing data architecture to lay the groundwork for future products and data-driven strategies.

Furthermore, you will be responsible for the ongoing maintenance, improvement, cleaning, and manipulation of data residing in the business’s operational and analytics databases. This includes understanding and aiding in the implementation of database requirements, analysing performance metrics, and proactively troubleshooting any issues that arise. Expertise in SQL development is paramount, as you will provide essential support to the Data and Analytics teams in database design, data flow management, and analytical activities. The role also plays a crucial part in the development and deployment of innovative big data platforms designed for advanced analytics and complex data processing.

The core responsibilities of this Data Engineer position are multifaceted and crucial to the organisation's data strategy. Key duties include:

• **Data Pipeline Architecture:** Creating and maintaining optimal data pipeline architectures, and assembling large, complex datasets that meet both functional and non-functional business requirements.
• **Process Improvement:** Identifying, designing, and implementing internal process improvements, which includes automating manual processes, optimising data delivery mechanisms, and re-designing infrastructure for enhanced scalability.
• **ETL Development:** Building the essential infrastructure for the efficient extraction, transformation, and loading (ETL) of data from a diverse range of sources, leveraging both SQL and big data technologies.
• **Analytics Tool Implementation:** Implementing analytics tools that effectively utilise the data pipeline to deliver actionable insights into key business areas like customer acquisition and operational efficiency.
• **Stakeholder Collaboration:** Working closely with stakeholders across Executive, Product, Data, and Design teams to address data-related technical challenges and support their infrastructure needs.
• **Data Security and Governance:** Ensuring data is appropriately segregated and secured across national boundaries, often involving multiple data centres.
• **Tooling for Data Teams:** Implementing data tools for analytics and data scientist team members, empowering them to build and optimise products and drive innovation.
• **Platform Innovation:** Leading innovation through the exploration of new big data technologies, benchmarking their capabilities, making recommendations, and implementing them for platforms.
• **Database Management:** Developing and implementing scripts for routine database maintenance, monitoring, performance tuning, and other critical operational tasks.
• **ETL Package Design:** Designing and developing scalable ETL packages, potentially using tools like Abinitio, to populate databases from business source systems and create necessary aggregates.
• **Big Data Platform Support:** Overseeing large-scale data Hadoop platforms and supporting the rapid growth of data within the business.
• **Data Migrations:** Managing and executing data migrations across different databases and servers, such as SQL Server to Oracle migrations.
• **Data Store Definition:** Defining and implementing data stores based on system and broader business requirements.
• **Model Validation:** Performing thorough testing and validation to ensure the accuracy of data transformations and verification used in machine learning models.
• **Data Quality Assurance:** Ensuring adherence to proper data governance and quality standards across the Data and Analytics department and the organisation as a whole.
• **Ad-hoc Analysis:** Conducting ad-hoc analyses of data stored in business databases, including writing SQL scripts, stored procedures, functions, and views.
• **Troubleshooting and Solutions:** Troubleshooting data issues within and across the business, and presenting effective solutions.
• **Database Optimisation:** Proactively analysing and evaluating business databases to identify and recommend areas for improvement and optimisation.
• **Data Modelling:** Analysing complex data elements and systems, including data flow, dependencies, and relationships, to contribute to conceptual, physical, and logical data models.
• **Data Warehousing Support:** Collaborating with the data warehousing department to support the implementation of the data warehouse for new big data platforms.
• **Industry Best Practices:** Staying abreast of industry trends and best practices, advising management on new data engineering strategies to enhance departmental performance, improve data governance, and promote informed decision-making.

What They're Looking For

To excel in this role, candidates should possess advanced working knowledge of SQL and experience with relational databases, including query authoring. Familiarity with a variety of database systems is essential. You should have a proven track record in building and optimising big data pipelines, architectures, and datasets. Experience in performing root cause analysis on both internal and external data and processes to answer specific business questions and identify improvement opportunities is highly valued.

Strong analytical skills, particularly when working with unstructured datasets, are a must. The ability to build processes that support data transformation, metadata management, dependency tracking, and workload management is also critical. Candidates should demonstrate a history of successfully manipulating and processing large and complex datasets. A proactive approach to problem-solving and a collaborative spirit are key attributes for success in this team-oriented environment.

---

Data Engineering roles in South Africa, particularly within the financial sector, are in high demand. Salaries for experienced Data Engineers can range significantly based on expertise, years of experience, and the specific technologies mastered, but often fall within the R600,000 to R1,200,000 per annum range, sometimes higher for senior or specialised positions. Major banks, telecommunications companies, and large retail groups are common employers of data professionals. Career progression typically involves moving into senior engineering roles, team lead positions, or specialised areas like data architecture or machine learning engineering.

Qualifications:

• The Data Engineer must have a bachelor’s degree in computer science, Applied Mathematics, Engineering, or any other technology related field. An equivalent of the same in working experience is also accepted for the position.
• 5 years of experience in managing financial services data, especially as a data architect
• Advanced data analysis skills using Excel, Abinitio,SQL and reporting tools to import, analyse and report on data
• Relative certifications

Important Closing Date Note

Take note that applications will not be accepted on the below date and onwards, kindly submit applications ahead of the closing date indicated below.