Short Description:
A data engineer is a data professional who uses his expertise in data engineering and programming to build systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. Their ultimate goal is to make data accessible so that organizations can use it to evaluate and optimize their performance.
Qualifications:
- Hands-on experience with SQL database design;
- In-depth knowledge of industry standard techniques for data processing, data management, data delivery, data mining, data analysis;
- Fluent in developing, maintaining, testing, and optimizing data pipelines;
- IT knowledge - SQL (on Oracle / other databases);
- Understanding, at a detailed level, of all architectural dependencies of technologies in IT environments;
- Strong understanding of Datawarehouse and relational database concept;
- Strong understanding of Big Data concepts;
- General knowledge of the bank’s products and services / all banking products and services;
- Knowledge of Azure data factory/ Azure Databricks is a big plus.
Job Responsibilities:
- Develop algorithms to transform data into useful, actionable information;
- Collect data from different sources, process it, and store it in a ready-to-use format in the company’s data warehouse, data lake or cloud platforms;
- Build and optimize ‘big data’ data pipelines, architectures, and data sets;
- Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement;
- Able to work with big data technologies such as Hadoop and Spark to efficiently process and analyze massive datasets;
- Able to work with cloud tools hosted in cloud providers, such as Azure or Google Cloud;
- Collaborate with management to understand company objectives.
