Director, Big Data Architecture – Big Data Solutions

Job Description

Sr Java Big Data Architect | Photon

The Tone:
This is a full-time role at Photon, located in Mississauga, Canada. In this position, you will be instrumental in architecting and designing large-scale, distributed big data solutions. Your work will enable the company to effectively handle high-volume data processing and advanced analytics. This role is crucial for scaling Photon’s data capabilities and driving significant impact through robust system design.

The TL;DR
• Role: Senior Architect
• Location: In-person, Mississauga, Ontario, Canada
• Mission: Architect and design large-scale, distributed big data solutions using Java and big data technologies to handle high-volume data processing and analytics.
• Tech Stack: Java 1.8+, Apache Spark, Hadoop, Cloudera Distribution, Hive, Impala, Yarn, Kafka, Junit, Python, Unix

What You’ll Actually Do
• Architecture: Architect and design large-scale, distributed big data solutions leveraging Java and various big data technologies to process high volumes of data.
• Development: Develop and implement high-volume data processing and analytics solutions within the Big Data ecosystem, focusing on Spark and MapReduce.
• Core Java/Spark Implementation: Apply advanced concepts of Java 1.8+ for data structures, memory management, and design patterns, alongside core Java and Spark architectural principles.
• Quality Assurance: Ensure the robust quality and reliability of developed solutions through comprehensive Junit testing and other relevant testing frameworks.
• Team Leadership: Lead and manage global technology teams, fostering adherence to industry practices and standards in project delivery.

The Must-Haves
• Background: This role requires a Senior Architect with a minimum of 15 years of experience in applications development, specializing in designing and architecting robust, large-scale distributed big data solutions. Candidates must demonstrate deep expertise in Java and the broader big data ecosystem.
• Experience: A proven track record of over 15 years in developing applications, specifically with hands-on experience in architecting high-volume big data processing and analytics solutions. This includes substantial, advanced hands-on experience with Java (core 1.8 or above), Apache Spark (preferably Java Spark), and strong familiarity with the Hadoop ecosystem (Cloudera distribution), including Hive, Impala, Yarn, and Kafka. Experience in managing global technology teams is also required.
• Skills: Expertise in advanced Java (1.8+), Apache Spark, and big data technologies. Strong data analysis and programming skills, particularly with large datasets and understanding of no-SQL joins. Proficient in Java and Spark architecture and design principles, and experienced with Junit/testing frameworks.
• Bonus: Practical experience with Unix and Python for shell scripting.