Project Overview

Background

8 months

6 CMU-HCII students

1 client, Edmentum, an leading company in K-12 classroom management in the US

2 mentors, professors from CMU-HCII, who are experts in learning science and learning experience design

project goal

By solving the current usability problems in Edmentum’s current data platform, Sensei, to encourage teachers to use the data on it more frequently to help their students make learning progress.

My contribution

As the product designer of my team, I applied my product design expertise in improving the usability of Sensei, Edmentum's current data platform. As a result, we developed Sensei Pro with optimized product flow, new on-demand features, and more intuitive UI. According to the user testing result, we made it to 100% positive user satisfactory compare to the older version of Sensei. :)

“Everything was easy [in Sensei Pro]. I’ve never used this ever and I was able to navigate through this more easily than something [Sensei] I’ve been working with for 3 years.”

— A user who is a teacher and have been using Sensei for years

A note at the beginning: 
During the project collaboration time period, our team iteratively designed and tested Sensei Pro. We sat together and everyone was able to provide feedback on every iteration of design solutions. When it came to high fidelity UI design, I particularly crafted the students interface and the assignments interface.
After the termination of collaboration, I looked back at the entire product design process and expanded my work to ensure that the design solutions fully addressed the research questions and concretely solved the needs from customers. Here, I will present my thinking process followed by my work.

To view the original team project report, click here.


Introducing Sensei Pro

See what we crafted after 8 months’ hard work from research to shipping

Interesting Fact: in Japanese, sensei means teacher.
product structure.png

Sensei pro

An interactive data platform that helps teachers manage their class.

 

Behind the stage

Why we made Sensei pro this way & how we made it there

One of the major products of our client is Study Island, a K-12 classroom management tool that has been used by teachers over 24,000 schools. Study Island’s data platform, Sensei, contains data of students from comprehensive kinds of learning activities to help teachers tailor their teaching. Sensei is considered the most valuable feature of this product and is highly valued by our client for the potential customer loyalty it could gain and the potential revenue it could bring. However, because of Sensei’s usability issues, a quite amount of its teacher users have never used it to access the data, nor use the data to adjust their instruction. Our client is eager to find a way to encourage teachers to use the data in Sensei in ways that truly help their students’ learning. They believe that this problem could be greatly relieved or even solved by a through research and design towards existing user experience problems, and that’s why they got us.

It took us 8 months to identify targeted user groups, define problems, design solutions, and test them with real users. We did a great job by delivering Sensei Pro, an upgraded Sensei to our client, with a testing result of 100% positive user satisfactory (which is unbelievable!).

 

The problems

Through research, we identified 3 major problems that happen to teacher users with different experience levels of using Sensei. These problems result in infrequent access to Sensei, and blocks teachers from effectively using data in Sensei to inform their teaching.

Screen Shot 2018-09-24 at 3.31.34 PM.png

1. Hard to access data

“I feel like Sensei in Study Island has a lot of data that I can’t access because I don’t know how.”

Teachers find that the interfaces in Sensei are unintuitive. This results its data inaccessible to them.

Screen Shot 2018-09-24 at 3.31.42 PM.png

2. Hard to interpret data

“I wish it [Sensei] could give me an easier time identifying which areas students need more support on by reading the data to drive my instruction.”

Teachers find that the data on sensei is hard to interpret. This blocks them from effectively utilizing the data to guide their teaching.

Screen Shot 2018-09-24 at 3.31.50 PM.png

3. Hard to take actions on data

“I wish it [Sensei] could possibly give me recommendations when I see the data and tell me what the next logical step is.”

During user research, we found that a few experienced teachers who have been using Sensei for years had their way of accessing and interpreting data, although all described doing so as “exhausting and unproductive”. Unfortunately, none of these experienced users had a way of taking actions with the information interpreted from the data because it easier “unclear” or “vague”. A quite amount of users want to have a “smart recommendation dot” in Sensei that they can “do something with one click of that button”.

 

The solutions

First of all, what we aim to achieve through our solutions to the above problem?

— An upgraded data platform from Sensei that brings teachers a seamless user experience in accessing, interpreting, and taking actions on the data, and lead to a result that teachers frequently access the data platform, interact with the data, and use the insights disclosed from the data to inform their teaching and help their students’ learning.

And Sensei Pro did it! We powered Sensei Pro with on-demand new features and appropriately applied design methodologies to solve the problems that users care the most. Existing users now get to experience the version of Sensei that they have never experienced but always dream to!

As a bonus, we give Sensei Pro a fresh new look that is more elegant, more delightful to use :)

Screen Shot 2018-09-25 at 10.50.09 AM.png

1. hard to access data

Flat Card View

The current data platform on the left requires teachers to recall quite amount of detailed informations to access a certain data report. This shuts teachers down from actively accessing the data.

To solve this accessibility issue, I applied the design principle Recognition rather than Recall to design a flat card view on the right that saves teachers tons of time when accessing a particular data report.

Recognition rather than Recall

Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another.

— Jakob Nielsen

Flat card view

Shortcut

Another accessibility issue with the current data platform is that it does not provide teachers with a place to save their frequently visited data reports and quickly access them again.

We solved this problem by adding a shortcut feature on top of each report, which enables teachers to star the report, save it to the permanent toolbar, and quickly access it again with one click.

Shortcut feature

 
Screen Shot 2018-09-25 at 10.54.22 AM.png

2. hard to interpret data

The current data platform fails to provide teachers with an easy way to interpret data. This caused teachers to suffer from figuring out who needs help on which topic or assignment, which questions are the class’s most frequent mistakes, and how have the class or an individual student mastered a topic.

Smart Grouping

by Mastery Level

To help teachers quickly identify struggling students, we group students based on their mastery level within each topic. Teachers no longer have to read the raw data to figure out who are struggling at which topic by themselves. Instead, they can easily identify struggling students with a simple click.

Smart grouping by mastery level

Smart Grouping

by Score Range

For each assignment, we group students based on their score range to help teachers easily identify the most struggling students.

Smart grouping by score range

Data Visualization

For Frequent Mistakes

Our question card design enables teachers to quickly visualize class performance on a question, and which incorrect answer choice was the most selected one by the class.

Data visualization for frequent mistakes

Data Visualization

For All Questions

We apply matrix design on this data mapping to help teachers visualize the class’s performance on all questions.

Each row (x axis) informs teachers about which questions this individual student did not answer correctly.

Each column (y axis) informs teachers on how many students did this question wrong.

Data visualization for all questions

data visualization

For Individual Mastery Level

One way to measure how much a student has mastered a collection of knowledge components is to see which score the student got on the corresponding topic and its subtopics. Often the times, teachers want to know these scores of each individual student in the class so that they can tailor their instruction to fit that student’s need. Unfortunately, the way these data are visualized in Sensei is unintuitive, and teachers are suffering from extracting useful information out of this exhausting data map.

The unintuitive way of viewing individual mastery level in Sensei

In Sensei Pro, I designed a more intuitive, straightforward, and handy experience for teachers to read the data of their students and track whether each individual has mastered a topic and its subtopics. In this interface, topics are primary data points and are represented as colored blocks in the chart. Subtopics are secondary data points and are represented in the box on the right of the chart. Colors represents the score range. Teachers are able to get a sense of how each individual student has mastered a topic with just a glance of this chart. If teachers need to dig into subtopics, they can always click on each block to get the information.

The data visualization of individual mastery level in Sensei Pro is intuitive, straightforward, and easy to view

In an earlier iteration, I placed informations of individual student’s mastery level towards subtopics in a box under its topic. It is supposed to be shown once the corresponding color box is clicked.

The user testing results show that teachers are easily get lost when interacting with this design because of the constantly moving box.

In the final solution, I present the informations of subtopics in a box on the right of the chart. The position of this box is fixed, which resulted positive testing results. Teachers described it as “easy to get used to” and “never need to worry about where I am”.

micro-interaction

For Assignment Progress Indication

On the assignment interface, I applied micro-interaction to indicate the completion progress of an assignment. For each “In Progress” assignment, the blue bar on the top gives the teacher a sense on how the assigned student groups are making progress towards the completion of the assignment. If an assignment remains as “almost done” for a while, the teacher would quickly notice it from the length of the blue bar and would be able to send a reminder to individual students who are delayed in a timely manner.

Micro-interaction for assignment progress indication

Color Coding

Besides smart grouping and data visualization, we apply color coding methodology to help teachers quickly understand the class’s mastery level and an individual student’s mastery level on each topic and each assignment in a simple and straightforward way.

Color coding

Screen Shot 2018-09-25 at 10.57.19 AM.png

3. hard to take actions on data

Actionable recommendation

Our recommendations are generated based on students’ mastery level from the data to help teachers make individualized instructions. We applied the Proximity design principle to place the recommendation right next to its source data, which makes the reasons behind each recommendation explicit to teachers.

To assign individualized practices to specific student groups, all teachers need to do is to click on the “assign” button.

Proximity

Placing the related data nearby to help teachers understand the link between recommendations and data.

Actionable recommendation

In early iterations of user testing, teachers often failed to visually link a recommendation with its supportive data. They were confused about where a recommendation comes from regardless the fact that its supportive data is right there.  

To solve this problem, we optimized our design solution to place supportive data nearby the relevant recommendation. As a result, teachers are able to view both at the same time and easily figure out the connection in between them.

Through reducing teachers’ cognitive load, this optimization saves teachers’ time on digesting the information they need when making decisions on whether or not to take a recommendation.

Actionable recommendation: iterations on placing related data nearly

Before: Next Step Recommendation and its supportive data Skills by Class are visually separated.

After: Next Step Recommendation and its supportive data are placed next to each other for a stronger visual link.

Balance Actionable Recommendations with Teacher Control

While our research on literature review and user interview shows that teachers like to have an actionable recommendation feature like this, it also informed us that teachers want to be in control when making decisions about teaching.

To balance useful actionable recommendations with enough teacher control, we provide teachers with the freedom of editing questions before they take the recommendation. As a result, they have the flexibility to provide more individualized help to targeted students using their own judgement.

Actionable recommendation: balance smart recommendation with teacher control (1)

Besides customized question editing, we also leave teachers the freedom of accepting each recommendation or not.

If teachers change their mind after rejecting a recommendation, they always have the choice to come back later to accept it.

Actionable recommendation: balance smart recommendation with teacher control (2)

 

success measurement

How do these solutions go with users? Let’s take a look.

In 5 iterations, we conducted a total of 26 usability testing sessions with 3 teachers and 7 educational professionals. The 3 teachers are the current data platform users. The 7 education professionals are non-users. We choose to test with them because our client wants to see if Sensei 2 lowers the entry barrier for new users compare to Sensei.

We used two metrics to measure the success of Sensei 2. They are: Time of task completion, and Number of clicks.

During each testing session, the user is required to perform the following tasks on Sensei and Sensei Pro. For each task, we observe the user’s interaction with both platforms, and record the user’s time of completion and number of clicks, respectively. After the user finished performing all tasks, we ask the user to rate Sensei and Sensei Pro respectively in responding to the usability of the platform, with 1 being “hard to use” and 10 being “easy to use”.

Here are the tasks users are required to perform during usability testing:

  1. Access a previously created assignment

  2. identify students who are scored in a certain percentage range on the assignment

  3. Access a student’s individual data report

  4. Access a specific topic

  5. Identify struggling students within a topic

  6. Assign more practice to students who are struggling

 

The results

The results are positive. Compare to the experience of using the current data platform Sensei, they averagely performed 47% faster on a task, with 75% less clicks when using Sensei Pro. Averagely, they rated Sensei Pro 30% higher than Sensei.

Median Time of Task Completion from Teachers and Education Professionals

Note: When performing task 3 & 6, educational professionals tried to access a feature on a static high fidelity prototype in Figma that aren’t implemented yet.

Number of clicks from teachers and education professionals when performing task 1: accessing a previously created assignment, and task 3: accessing a student’s individual data report

When asked to give a rating regarding to the usability of both products within 1 to 10, both teachers and educational professionals rated Sensei Pro higher than Sensei

 

What do teachers think?

Delightfully, all 3 teachers pointed out that Sensei Pro is much better than the current data platform because it is more intuitive to use, easily adaptable, and lowers their cognitive load by providing quick and direct access to the exact data piece they need at the right time.

“My not-so-computer-savvy friends could do much better with [Sensei Pro] because it’s so much more clear.” — Teacher A

“Everything was easy. I’ve never used this ever and I was able to navigate through this more easily than something I’ve been working with for 3 years.” — Teacher B

By integrating data into one place, Sensei Pro makes it more efficient for teachers to interact with data to inform their teaching, and saves their time.

“It’s nice because all the data is right there. I don’t have to go in and find the class report and look  on my own, this has it all right there” — Teacher A

“All the stuff I want to see is in one place.” — Teacher C

 

reflection & future improvement

Although we did a great job on delivering a solution that made both the customers and the client happy, there are still corners and edges that our solution did not fully address. Here, I would love to provide a few suggestions to the client’s newly founded UX team to carry on:

  1. Test Sensei Pro with a larger scale of users and various user types. Due to time limitation, our team only got 3 users for usability testing. More user users participating in the testing will ensure a safer release of Sensei Pro as it would be more acceptable to the entire user population.

  2. Let teachers notice Sensei Pro and easily access it from Study Island. As the data platform of a K-12 classroom management product Study Island, Sensei has been overlooked by teachers. Because of its position and the visual style of its entrance, teachers either have never noticed it, or thought of it as an advertisement channel rather then a data platform containing useful resource. The greatness of Sensei Pro can only be experienced after the user enters it. Thus, it is extremely important for our client to find the best way to expose Sensei Pro by making its entrance conspicuous and its visuals data-styling.

  3. Sensei Pro is designed within the scope of single class at a time. More work needs to be done on the product flow targeting users who teach multiple classes within the same semester and need to switch among classes.

 

Design iteration Gallery