Rubrics Expert
Required Skills
Rubric Design & Development
Scoring Methodology & Calibration
Assessment Validation & Quality Control
Performance Analysis & Feedback Structuring
About micro1
micro1 is a data engine that helps AI labs train foundational models and enterprises build AI agents. We provide frontier evaluations and reinforcement learning environments used to improve LLM capabilities, as well as contextual evaluations used to monitor and improve AI agents in enterprise settings. Our data engine includes an AI recruiter agent that sources and vets domain experts, a data platform that enables rapid production of high-quality training data, and a pipeline performance system that ensures both quality and velocity.
Our goal is to have 1 billion people doing meaningful work by contributing their expertise to the development of frontier AI models. We’ve raised $40M+ in funding, and our AI recruiter has powered more than 1 million AI-led interviews as our global network of experts expands to form the human intelligence layer for AGI.
Job Description
Job Title: Rubrics Expert
Job Type: Contract (Full-time or Part-time)
Location: Remote
Job Summary:
Join our customer’s team as a Rubrics Expert and play a pivotal role in shaping the quality and impact of assessments across diverse learning environments. You’ll leverage your expertise in rubric design and validation to ensure fair, consistent, and actionable evaluations, directly influencing educational and professional development outcomes.
Key Responsibilities:
- Design and develop robust, scalable scoring rubrics tailored to varying assessment needs and content areas.
- Establish and refine scoring methodologies, ensuring clarity, reliability, and alignment with learning objectives.
- Lead calibration sessions with assessors and subject matter experts to maintain scoring consistency and inter-rater reliability.
- Validate assessments and rubrics through data-driven analysis, pilot testing, and continuous quality control measures.
- Analyze performance data using advanced assessment analytics to pinpoint trends and inform rubric revisions.
- Develop clear, actionable feedback structures based on rubric results to support learner growth and achievement.
- Collaborate cross-functionally with instructional designers, educators, and stakeholders to drive assessment excellence.
Required Skills and Qualifications:
- Proven experience in rubric design.
- In-depth knowledge of scoring methodologies, calibration techniques, and assessment validation processes.
- Strong analytical skills to interpret performance data and inform rubric improvements.
- Demonstrated expertise in quality control and continuous improvement within assessment systems.
- Exceptional written and verbal communication skills, with a passion for clarity and detail.
- Ability to structure actionable, meaningful feedback based on rubric-driven evaluations.
- Self-motivated, process-oriented, and adaptable to remote collaboration.
Preferred Qualifications:
- Experience with digital assessment tools, large-scale assessment projects, or remote teams.
- Background in professional development or training for educators and assessors.