Responsibilities:
- Design, develop, and maintain Flink applications using Java for stream processing.
- Deploy and manage Flink applications on Amazon Managed Service for Apache Flink.
- Collaborate with data engineers and data scientists to integrate Flink applications with other data processing systems.
- Optimize and troubleshoot Flink jobs to ensure high performance and reliability.
- Monitor and maintain the health of Flink clusters and data pipelines.
- Implement best practices for data management and monitoring.
Qualifications
- 5+ years of software development experience with a focus on big data processing.
- 3+ years of hands-on experience with Apache Flink.
- Proficiency in Java for Flink application development.
- Experience with AWS services, specifically Amazon Managed Service for Apache Flink.
- Excellent problem-solving, teamwork, and communication skills.
Additional Breakdown
Technical Skills
Proficiency in Apache Flink:
- Strong understanding of Flink’s architecture and core concepts.
- Hands-on experience with Flink’s DataStream API for stream processing.
- Familiarity with Flink's Table API and SQL.
Programming Languages:
- Proficiency in Java for developing Flink applications.
Big Data Ecosystem:
- Experience with data ingestion and integration using Apache Kafka.
- Understanding of data processing and transformation in Flink.
AWS and Amazon Managed Service for Apache Flink:
- Experience deploying and managing Flink applications on Amazon Managed Service for Apache Flink.
Monitoring and Logging:
- Proficiency in setting up logging and alerting for Flink applications using AWS CloudWatch.
Soft Skills
Problem-Solving:
- Strong analytical and problem-solving skills.
- Ability to troubleshoot and resolve issues in Flink applications.
Team Collaboration:
- Excellent teamwork and collaboration skills.
- Experience working in an Agile/Scrum development environment.
Communication:
- Strong written and verbal communication skills.
- Ability to explain complex technical concepts to non-technical stakeholders.
Adaptability:
- Ability to quickly learn new technologies and adapt to changing requirements.
- Proactive in seeking out new challenges and opportunities for improvement.
Attention to Detail:
- High attention to detail and commitment to producing high-quality, reliable code.
Experience:
- 5+ years of experience in software development with a focus on big data processing.
- 3+ years of hands-on experience with Apache Flink.
Optional but Preferred Skills
Performance Tuning:
- Experience with performance tuning and optimization of Flink jobs.
