Big Data Engineer

Job Summary

We are looking for a highly skilled Big Data Engineer to design, develop, and optimize large-scale data processing systems that support analytics, reporting, and AI initiatives. The ideal candidate should have strong expertise in distributed computing, cloud data platforms, ETL pipelines, and big data technologies.

Key Responsibilities

• Design and maintain scalable big data solutions.
• Develop ETL and ELT pipelines for enterprise data systems.
• Optimize data ingestion, transformation, and storage processes.
• Work with Hadoop, Spark, Kafka, and distributed systems.
• Build cloud-native data architectures.
• Collaborate with analytics and AI teams.
• Ensure data quality, governance, and security.
• Implement automation and monitoring solutions.
• Troubleshoot performance and scalability issues.
• Support data warehousing and reporting initiatives.
• Develop APIs and integration services.
• Maintain technical documentation and standards.
• Support real-time and batch processing workflows.
• Participate in Agile development and sprint planning.
• Research and implement emerging data technologies.

Required Skills

• Expertise in Python, Scala, or Java.
• Strong knowledge of Spark, Hadoop, and Kafka.
• Experience with cloud data platforms.
• Knowledge of SQL and NoSQL databases.
• Familiarity with data warehousing concepts.
• Strong analytical and troubleshooting skills.
• Understanding of data governance and security practices.

Qualifications

• Bachelor’s degree in Computer Science or related field.
• 4+ years of big data engineering experience.
• Experience working with large-scale enterprise datasets.

Preferred Qualifications

• Experience with Databricks or Snowflake.
• Knowledge of machine learning pipelines.
• Cloud certifications are a plus.