Big Data Engineer

Job description

Description About the job Join our Team - Your career journey starts here - not just a job, but a future. Our Potential Opportunity the employer Automation Intelligence is seeking a Big Data Engineer to join our dynamic team. This role is pivotal in supporting Industrial Digitalization Solutions across industries such as Oil & Gas, Petrochemicals, Utilities, and Manufacturing. Design and operate end-to-end big data and analytics solutions building scalable ETL/ELT pipelines, integrating industrial and enterprise data, optimizing Hadoop/Cloudera platforms, and delivering real-time, predictive dashboards that drive operational efficiency, safety, compliance, and data-driven decision-making. the employer’s commitment to providing cutting-edge Digital Solutions ensures you will contribute to transformative projects that enhance operational efficiency and productivity. Our Digital Solutions are a testament to our dedication to driving innovation and optimizing industrial processes, reflecting our focus on excellence, reliability, and overcoming industry challenges. What You’ll Lead and Deliver Design, develop, and maintain robust ETL/ELT pipelines for both pre-production and production environments. Ingest data from diverse sources, including industrial sensors (pumps, boilers, storage tanks), relational databases (SQL, Oracle), and enterprise applications. Implement data integration and transformation logic using Scala, PySpark, Sqoop, and Apache NiFi within the Cloudera/Hadoop ecosystem. Manage and schedule data workflows using Oozie or similar orchestration tools. Optimize data storage and query performance using Hive, Impala, and HDFS for efficient analytics. Develop interactive and real-time dashboards in Power BI and Tableau to monitor critical business and operational KPIs. Translate complex technical datasets into user-friendly visualizations for stakeholders, enabling data-driven decision-making. Design dashboards for specific use cases such as: i) Real-time monitoring of sensor data (pumps, boilers) to improve plant uptime and save costs. ii) Track safety permits (ESWP), monitor workplace hazards (SHER, risk scores), and perform root-cause analysis on incidents. Support the development of predictive models, such as Hazard Prediction dashboards, by preparing and engineering the necessary data features. Perform data analysis and ad-hoc reporting to answer complex business questions related to plant manufacturing, energy, and safety. Ensure data accuracy, consistency, and reliability across all reporting and analytics platforms. Assist in the maintenance, configuration, and performance tuning of the Big Data platform (Cloudera) and BI servers (Tableau/Power BI). Collaborate on the integration of industrial systems (e.g., OSI PI System) with the big data lake for advanced analytics. Requirements To Be Successful in This Role, You Must Have: Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, Telecommunication, or a related field. 5+ years of experience in data engineering, big data development, or business intelligence. Strong hands-on experience with the Apache Hadoop ecosystem (Cloudera distribution preferred) including HDFS, Hive, Impala, and Oozie. Expert-level proficiency in PySpark and Scala for building data pipelines. Experience with real-time data ingestion tools like Apache NiFi and data migration tools like Sqoop. Strong SQL skills and experience with relational databases (e.g., Microsoft SQL, Oracle, IBM DB2, Amazon RDS). Proficiency in Power BI and Tableau, including dashboard design, server administration, and performance optimization. Working knowledge of Python for data analysis and scripting. Cloud (Optional but beneficial): Experience with AWS data services (Amazon RDS, Glue, Redshift) is a plus. Industrial Systems (Optional but beneficial): Familiarity with OSI PI System for sensor data integration. Local Saudi Nationals would be preferred for this role. We Offer Competitive Benefits: Global Expo

Requirements

• Design data models
• Create ETL processes
• SQL proficiency
• Python or Scala programming
• Experience in cloud platforms or big data systems (e.g. AWS, Azure, GCP, Spark, Hadoop)

Skills

Benefits

• Health & Wellness: Comprehensive medical insurance for you and your dependents.
• Work-Life Balance: Paid time off, including annual leave and holidays.