Handshake AI – AI Content Evaluation

Job Description

Handshake AI Fellow — Dusker | Handshake

The Tone:
This is a fully remote, asynchronous role at Handshake, offering significant flexibility to work from anywhere on your own schedule, adapting to your personal commitments. Handshake is pioneering an AI program, Handshake AI, which is designed to enhance the artificial intelligence’s grasp of complex real-world tasks and various professional workflows. This particular position is absolutely vital for the program’s success, as it involves both the meticulous creation of specialized training content and the rigorous evaluation of AI-generated outputs. Your work will directly contribute to refining AI capabilities, ensuring the technology can perform with greater accuracy, higher quality, and a deeper understanding of practical applications. This opportunity allows individuals from diverse professional and academic backgrounds to shape the future of AI without requiring prior experience in the field of artificial intelligence itself.

The TL;DR
• Role: Early Career
• Type: Contract
• Location: Remote
• Mission: Help AI better understand real-world tasks and workflows.

What You’ll Actually Do
• Content Development: Apply your professional and academic expertise to meticulously create diverse training content, serving as foundational material for AI systems to learn from. This involves translating complex real-world scenarios and workflows into structured data and examples.
• Accuracy Evaluation: Rigorously evaluate AI-generated responses against established benchmarks and real-world expectations to determine their factual correctness and precision. This critical step ensures that the AI’s outputs are reliable and truthful.
• Quality Assessment: Conduct comprehensive assessments of the overall quality of AI-generated responses, focusing on aspects like coherence, relevance, completeness, and adherence to specific output formats. This goes beyond mere accuracy to improve the user experience and utility of the AI.
• Expertise Application: Utilize your unique professional and academic background to provide nuanced insights, guiding the AI to better understand and navigate the complexities inherent in various real-world tasks and intricate workflows. Your input helps bridge the gap between AI capabilities and human expectations.
• Project Management: Adhere strictly to project-specific guidelines received upon onboarding, managing your workload asynchronously and independently to meet project objectives and deliver high-quality contributions consistently. This ensures alignment with the Handshake AI program’s standards.

The Must-Haves
• Background: This role is ideal for Early-Career professionals or those looking to transition or expand their skill sets, as prior experience in Artificial Intelligence is not a prerequisite. Candidates are expected to bring a robust foundation of professional and academic expertise, cultivated through their career and educational journeys, which will be directly applied to the innovative work of AI training and evaluation.
• Experience: No specific years of experience in AI are required. However, candidates should possess a background demonstrating the successful application of their professional or academic knowledge in problem-solving, analysis, or content creation within their respective fields.
• Skills: Essential skills include a keen ability to synthesize complex information from professional and academic domains, exceptional attention to detail for evaluating accuracy, and a strong commitment to assessing the quality and relevance of AI-generated content. The capacity for independent, asynchronous work and adherence to detailed project guidelines is also crucial.