Initial Research Findings
After a round of user interviews with both math and chemistry instructors, we found the following:
91% of participants rated the importance to author questions as a 4 out of 5
Most common question types requested: multiple choice, free response, numeric entry, matching
Instructors want to write multi-part questions
Existing tools in other online homework systems are too complex
Instructors want folders to organize their questions
“The way I teach a concept may be different than how the homework system presents it, so I author questions to fit my teaching style and share question templates with other instructors teaching the course.”
Competitor product — Pearson MyMathLab
Project Scope
Allow instructors to:
Create their own questions
Organize and manage authored questions
Create assignments using authored questions
Manually grade free-response questions
Consider student experience
Potential Business Impact
Appeal to roughly 25–30% of the market that currently authors questions
We previously did not win roughly $3–5 million dollars in potential revenue due to lacking this feature
Design Goals
User Flow
Usability Testing
During user testing, we asked instructors to try to create some sample math questions that we provided them. We observed that instructors took a bit of time to digest the various fields and dropdowns on the page, and it was not clear enough what the “Type” dropdown corresponded to. To improve the readability of the form, we split the form into “Question” and “Answer” sections. We also learned that questions tend to be longer and decided to increase the size of the Question input to help the content stay in view. We added an additional multiple choice input, because most multiple choice questions have 4 options. Lastly, we provided a help tooltip on the top of the page linking instructors to additional onboarding resources.
While other question types are automatically graded, free-response questions are open ended and require manual grading. In the original design, instructors would grade the attempts by clicking into the Score input field, manually typing in a score or using the + and - buttons to adjust the score, and clicking “save”. After observing the user flow and noting that instructors usually give zero or full score to the attempts, I introduced floating “grading buttons” for instructors to quickly mark each student attempt as correct or incorrect. Instructors can also edit their grade for a previously graded attempt.
High-Fidelity Prototype
Question Organization
Instructors are able to edit, move, preview, delete, and organize their questions into folders on the My Questions page. They can create up to 3 folder levels and navigate back up the tree structure using breadcrumb links at the top of the page. We also added the ability to drag and drop questions into folders for ease of use.
Manual Grading
Floating grading buttons allow users to quickly mark each student attempt as correct or incorrect.
Floating Button States
At ALEKS, we are strong advocates of designing for accessibility. Whenever creating new buttons and components, I ensure all UI states are accounted for, are distinctive, and that color combinations are passing WCAG 2.0 AA.
Matching — Instructor Experience
To write a matching question, instructors select the Matching question type and enter prompt and answer pairs. They can also add extra incorrect answer choices, or distractors, a need that we discovered through user testing.
Matching — Student Experience
To keep as much content above the fold as possible, I proposed to minimize any inserted images. I also worked with our developers to make sure the drag and drop feature is accessible.
Next Steps
Instructors’ feedback have been positive in usability testing sessions and instructors have made suggestions for additional features that we are exploring for a v2 release:
Instructors want more detailed settings for certain question types
Instructors want more onboarding for question creation (tooltips, wizard, video)
Instructors want sample questions (already working on it!)