6 CMU-HCII students
1 client, Edmentum, an leading company in K-12 classroom management in the US
2 mentors, professors from CMU-HCII, who are experts in learning science and learning experience design
By solving the current usability problems in Edmentum’s current data platform, Sensei, to encourage teachers to use the data on it more frequently to help their students make learning progress.
As the product designer of my team, I applied my product design expertise in improving the usability of Sensei, Edmentum's current data platform. As a result, we developed Sensei Pro with optimized product flow, new on-demand features, and more intuitive UI. According to the user testing result, we made it to 100% positive user satisfactory compare to the older version of Sensei. :)
“Everything was easy [in Sensei Pro]. I’ve never used this ever and I was able to navigate through this more easily than something [Sensei] I’ve been working with for 3 years.”
— A user who is a teacher and have been using Sensei for years
A note at the beginning:
During the project collaboration time period, our team iteratively designed and tested Sensei Pro. We sat together and everyone was able to provide feedback on every iteration of design solutions. When it came to high fidelity UI design, I particularly crafted the students interface and the assignments interface.
After the termination of collaboration, I looked back at the entire product design process and expanded my work to ensure that the design solutions fully addressed the research questions and concretely solved the needs from customers. Here, I will present my thinking process followed by my work.
To view the original team project report, click here.
Introducing Sensei Pro
See what we crafted after 8 months’ hard work from research to shipping
Interesting Fact: in Japanese, sensei means teacher.
An interactive data platform that helps teachers manage their class.
Behind the stage
Why we made Sensei pro this way & how we made it there
One of the major products of our client is Study Island, a K-12 classroom management tool that has been used by teachers over 24,000 schools. Study Island’s data platform, Sensei, contains data of students from comprehensive kinds of learning activities to help teachers tailor their teaching. Sensei is considered the most valuable feature of this product and is highly valued by our client for the potential customer loyalty it could gain and the potential revenue it could bring. However, because of Sensei’s usability issues, a quite amount of its teacher users have never used it to access the data, nor use the data to adjust their instruction. Our client is eager to find a way to encourage teachers to use the data in Sensei in ways that truly help their students’ learning. They believe that this problem could be greatly relieved or even solved by a through research and design towards existing user experience problems, and that’s why they got us.
It took us 8 months to identify targeted user groups, define problems, design solutions, and test them with real users. We did a great job by delivering Sensei Pro, an upgraded Sensei to our client, with a testing result of 100% positive user satisfactory (which is unbelievable!).
Through research, we identified 3 major problems that happen to teacher users with different experience levels of using Sensei. These problems result in infrequent access to Sensei, and blocks teachers from effectively using data in Sensei to inform their teaching.
1. Hard to access data
“I feel like Sensei in Study Island has a lot of data that I can’t access because I don’t know how.”
Teachers find that the interfaces in Sensei are unintuitive. This results its data inaccessible to them.
2. Hard to interpret data
“I wish it [Sensei] could give me an easier time identifying which areas students need more support on by reading the data to drive my instruction.”
Teachers find that the data on sensei is hard to interpret. This blocks them from effectively utilizing the data to guide their teaching.
3. Hard to take actions on data
“I wish it [Sensei] could possibly give me recommendations when I see the data and tell me what the next logical step is.”
During user research, we found that a few experienced teachers who have been using Sensei for years had their way of accessing and interpreting data, although all described doing so as “exhausting and unproductive”. Unfortunately, none of these experienced users had a way of taking actions with the information interpreted from the data because it easier “unclear” or “vague”. A quite amount of users want to have a “smart recommendation dot” in Sensei that they can “do something with one click of that button”.
First of all, what we aim to achieve through our solutions to the above problem?
— An upgraded data platform from Sensei that brings teachers a seamless user experience in accessing, interpreting, and taking actions on the data, and lead to a result that teachers frequently access the data platform, interact with the data, and use the insights disclosed from the data to inform their teaching and help their students’ learning.
And Sensei Pro did it! We powered Sensei Pro with on-demand new features and appropriately applied design methodologies to solve the problems that users care the most. Existing users now get to experience the version of Sensei that they have never experienced but always dream to!
As a bonus, we give Sensei Pro a fresh new look that is more elegant, more delightful to use :)
1. hard to access data
Flat Card View
The current data platform on the left requires teachers to recall quite amount of detailed informations to access a certain data report. This shuts teachers down from actively accessing the data.
To solve this accessibility issue, I applied the design principle Recognition rather than Recall to design a flat card view on the right that saves teachers tons of time when accessing a particular data report.
Recognition rather than Recall
Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another.
— Jakob Nielsen
Another accessibility issue with the current data platform is that it does not provide teachers with a place to save their frequently visited data reports and quickly access them again.
We solved this problem by adding a shortcut feature on top of each report, which enables teachers to star the report, save it to the permanent toolbar, and quickly access it again with one click.
2. hard to interpret data
The current data platform fails to provide teachers with an easy way to interpret data. This caused teachers to suffer from figuring out who needs help on which topic or assignment, which questions are the class’s most frequent mistakes, and how have the class or an individual student mastered a topic.
by Mastery Level
To help teachers quickly identify struggling students, we group students based on their mastery level within each topic. Teachers no longer have to read the raw data to figure out who are struggling at which topic by themselves. Instead, they can easily identify struggling students with a simple click.
by Score Range
For each assignment, we group students based on their score range to help teachers easily identify the most struggling students.
For Frequent Mistakes
Our question card design enables teachers to quickly visualize class performance on a question, and which incorrect answer choice was the most selected one by the class.
For All Questions
We apply matrix design on this data mapping to help teachers visualize the class’s performance on all questions.
Each row (x axis) informs teachers about which questions this individual student did not answer correctly.
Each column (y axis) informs teachers on how many students did this question wrong.
For Individual Mastery Level
One way to measure how much a student has mastered a collection of knowledge components is to see which score the student got on the corresponding topic and its subtopics. Often the times, teachers want to know these scores of each individual student in the class so that they can tailor their instruction to fit that student’s need. Unfortunately, the way these data are visualized in Sensei is unintuitive, and teachers are suffering from extracting useful information out of this exhausting data map.
In Sensei Pro, I designed a more intuitive, straightforward, and handy experience for teachers to read the data of their students and track whether each individual has mastered a topic and its subtopics. In this interface, topics are primary data points and are represented as colored blocks in the chart. Subtopics are secondary data points and are represented in the box on the right of the chart. Colors represents the score range. Teachers are able to get a sense of how each individual student has mastered a topic with just a glance of this chart. If teachers need to dig into subtopics, they can always click on each block to get the information.
In an earlier iteration, I placed informations of individual student’s mastery level towards subtopics in a box under its topic. It is supposed to be shown once the corresponding color box is clicked.
The user testing results show that teachers are easily get lost when interacting with this design because of the constantly moving box.
In the final solution, I present the informations of subtopics in a box on the right of the chart. The position of this box is fixed, which resulted positive testing results. Teachers described it as “easy to get used to” and “never need to worry about where I am”.
For Assignment Progress Indication
On the assignment interface, I applied micro-interaction to indicate the completion progress of an assignment. For each “In Progress” assignment, the blue bar on the top gives the teacher a sense on how the assigned student groups are making progress towards the completion of the assignment. If an assignment remains as “almost done” for a while, the teacher would quickly notice it from the length of the blue bar and would be able to send a reminder to individual students who are delayed in a timely manner.
Besides smart grouping and data visualization, we apply color coding methodology to help teachers quickly understand the class’s mastery level and an individual student’s mastery level on each topic and each assignment in a simple and straightforward way.
3. hard to take actions on data
Our recommendations are generated based on students’ mastery level from the data to help teachers make individualized instructions. We applied the Proximity design principle to place the recommendation right next to its source data, which makes the reasons behind each recommendation explicit to teachers.
To assign individualized practices to specific student groups, all teachers need to do is to click on the “assign” button.
Placing the related data nearby to help teachers understand the link between recommendations and data.
In early iterations of user testing, teachers often failed to visually link a recommendation with its supportive data. They were confused about where a recommendation comes from regardless the fact that its supportive data is right there.
To solve this problem, we optimized our design solution to place supportive data nearby the relevant recommendation. As a result, teachers are able to view both at the same time and easily figure out the connection in between them.
Through reducing teachers’ cognitive load, this optimization saves teachers’ time on digesting the information they need when making decisions on whether or not to take a recommendation.
Balance Actionable Recommendations with Teacher Control
While our research on literature review and user interview shows that teachers like to have an actionable recommendation feature like this, it also informed us that teachers want to be in control when making decisions about teaching.
To balance useful actionable recommendations with enough teacher control, we provide teachers with the freedom of editing questions before they take the recommendation. As a result, they have the flexibility to provide more individualized help to targeted students using their own judgement.
Besides customized question editing, we also leave teachers the freedom of accepting each recommendation or not.
If teachers change their mind after rejecting a recommendation, they always have the choice to come back later to accept it.
How do these solutions go with users? Let’s take a look.
In 5 iterations, we conducted a total of 26 usability testing sessions with 3 teachers and 7 educational professionals. The 3 teachers are the current data platform users. The 7 education professionals are non-users. We choose to test with them because our client wants to see if Sensei 2 lowers the entry barrier for new users compare to Sensei.
We used two metrics to measure the success of Sensei 2. They are: Time of task completion, and Number of clicks.
During each testing session, the user is required to perform the following tasks on Sensei and Sensei Pro. For each task, we observe the user’s interaction with both platforms, and record the user’s time of completion and number of clicks, respectively. After the user finished performing all tasks, we ask the user to rate Sensei and Sensei Pro respectively in responding to the usability of the platform, with 1 being “hard to use” and 10 being “easy to use”.
Here are the tasks users are required to perform during usability testing:
Access a previously created assignment
identify students who are scored in a certain percentage range on the assignment
Access a student’s individual data report
Access a specific topic
Identify struggling students within a topic
Assign more practice to students who are struggling
The results are positive. Compare to the experience of using the current data platform Sensei, they averagely performed 47% faster on a task, with 75% less clicks when using Sensei Pro. Averagely, they rated Sensei Pro 30% higher than Sensei.
What do teachers think?
Delightfully, all 3 teachers pointed out that Sensei Pro is much better than the current data platform because it is more intuitive to use, easily adaptable, and lowers their cognitive load by providing quick and direct access to the exact data piece they need at the right time.
“My not-so-computer-savvy friends could do much better with [Sensei Pro] because it’s so much more clear.” — Teacher A
“Everything was easy. I’ve never used this ever and I was able to navigate through this more easily than something I’ve been working with for 3 years.” — Teacher B
By integrating data into one place, Sensei Pro makes it more efficient for teachers to interact with data to inform their teaching, and saves their time.
“It’s nice because all the data is right there. I don’t have to go in and find the class report and look on my own, this has it all right there” — Teacher A
“All the stuff I want to see is in one place.” — Teacher C
reflection & future improvement
Although we did a great job on delivering a solution that made both the customers and the client happy, there are still corners and edges that our solution did not fully address. Here, I would love to provide a few suggestions to the client’s newly founded UX team to carry on:
Test Sensei Pro with a larger scale of users and various user types. Due to time limitation, our team only got 3 users for usability testing. More user users participating in the testing will ensure a safer release of Sensei Pro as it would be more acceptable to the entire user population.
Let teachers notice Sensei Pro and easily access it from Study Island. As the data platform of a K-12 classroom management product Study Island, Sensei has been overlooked by teachers. Because of its position and the visual style of its entrance, teachers either have never noticed it, or thought of it as an advertisement channel rather then a data platform containing useful resource. The greatness of Sensei Pro can only be experienced after the user enters it. Thus, it is extremely important for our client to find the best way to expose Sensei Pro by making its entrance conspicuous and its visuals data-styling.
Sensei Pro is designed within the scope of single class at a time. More work needs to be done on the product flow targeting users who teach multiple classes within the same semester and need to switch among classes.