Big Data Engineer

Job description

You will be responsible for designing and developing scalable big data solutions that enable advanced analytics across the organization. You will work with large data sets and leverage cutting‑edge technologies to ensure efficient and reliable processing. Design, build, and maintain scalable data pipelines using Apache Spark, Hadoop, and Kafka. Collaborate with data scientists, analysts, and IT teams to identify requirements and deliver solutions that meet business needs. Ensure high data quality and integrity by implementing robust validation and testing processes. Optimize storage solutions for performance and cost efficiency in cloud and on‑premises environments. Monitor and troubleshoot data processing workflows to ensure timely delivery. Document data architectures, processes, and workflows for knowledge sharing and compliance. Stay updated on industry trends and best practices in big data technologies and methodologies

Requirements

• Experience with cloud platforms
• Preferred: experience with machine learning frameworks and algorithms
• Knowledge of data governance and security best practices
• Excel
• AWS cloud
• Azure cloud
• Google Cloud
• Bachelor's degree in Computer Science, IT or related field
• 3+ years experience as a Big Data Engineer or similar role
• Proficiency with Apache Spark, Hadoop, Apache Kafka
• Strong coding skills in Java, Scala or Python
• Familiarity with data processing frameworks and ETL tools
• Analytical mindset and excellent problem‑solving abilities
• Strong communication skills for technical and non‑technical stakeholders
• Self‑motivated, proactive and able to manage multiple tasks and deadlines
• Familiarity with containerization and orchestration tools like Docker and Kubernetes

Skills