Job Details

100% Remote Position
Position Type: Contract

Hadoop/Dataproc Platform Analyst

We are seeking a highly skilled Hadoop/Dataproc Expert to support a strategic initiative focused on the reverse engineering, analysis, and retirement/migration of our existing Hadoop/Google Cloud Dataproc platform. This role is critical to enabling the project team to fully understand the current data ecosystem and prepare for a seamless transition.

Key Responsibilities:

Platform Analysis & Assessment: Conduct a comprehensive review of the Hadoop/Dataproc environment, including cluster configurations, resource usage, and job execution patterns. Identify and document all active components, services, and dependencies.

Data & Metadata Extraction: Pull and catalog all data sources, datasets, and associated metadata across the platform. Map data lineage and relationships to support reverse engineering efforts.

Feed & Log Analysis: Analyze incoming and outgoing data feeds, including batch and streaming pipelines. Review system logs, audit trails, and job histories to identify integration points and data flow patterns.

Collaboration & Documentation: Work closely with data engineers, architects, and project leads to provide insights and technical guidance. Document findings in a structured format to support migration planning and platform decommissioning.

Required Skills & Experience:

• Proven expertise in Hadoop ecosystem (HDFS, Hive, Pig, Spark, etc.) and Google Cloud Dataproc.
• Strong understanding of platform management, including cluster operations, job orchestration, and performance tuning.
• Familiarity with cloud migration strategies and legacy platform retirement best practices.
• Excellent analytical, communication, and documentation skills.