Once upon a time…a Pocket Prep Story

Pocket Prep has a particular challenge in engaging our users: We’re a study tool for over 100 niche professional certification exams, and people typically don’t love to study.

These exams are difficult, require an incredible investment of time, and hundreds, if not thousands, of dollars to surmount. Our apps are a supplementary study tool to reinforce what users have learned in their textbooks, classes, and workshops.

Pocket Prep, like many start ups, outsourced its design and development for years as it built a user base. However, over the past 3 years, we’ve been growing a small but mighty product team with four developers and two designers (including myself) to craft our study products. In the summer of 2020, we set out to redesign our outdated web platform, originally designed and developed by an agency, to elevate our UI and give our users a more efficient and enjoyable studying experience.

A graphic of a dated dashboard view on a desktop in the background and the old Pocket Prep mobile app interface in the foreground.
The Pocket Prep web platform (left) and app (right) interfaces in 2020.

I hope this review of our process – what we did, what went well, lessons learned – will inform other small product teams embarking on their own legacy projects. Legacy redesigns can be overwhelming, but they are an  opportunity to elevate the user experience and success of a company. Let’s get into it.

Case Study Outline:

  1. Where we started
  2. Define & align
  3. Research
  4. Wireframes & user testing
  5. Visual designs
  6. Handoff and development
  7. Retrospective
  8. Closing & thanks
Dated web dashboard interface in background and updated web interface in foreground.
Original web platform interface vs. updated stats dashboard interface preview.

Where we started

About 75% of our 200,000+ monthly active users studied with our mobile iOS and Android apps in 2020. The remaining portion used our legacy web platform, a remnant of outsourced agency design and development from 2017. It was outdated, lacked our recent feature additions, and was minimally maintained.

Legacy redesigns are a common challenge for any product-driven company; it can be a daunting effort to refine a motley collection of features cobbled together over years of disjointed decisions. It’s important to maintain expected user behavior vs. improving workflows. It’s also a chance to incorporate the product into the larger visual design system it inhabits.

Our design team often acts as a facilitator for collaboration and company alignment on Pocket Prep’s product initiatives. Redesigning our web app was our UX team’s first chance to demonstrate the value of a full design process: thoughtful, thorough, and user-driven.

Screenshots of old interface: stats, quiz set up, and taking a quiz.
Our legacy web platform in 2020.

The first step of our design process brought the team together to define and align on goals.

Define and align

A common (and inadequate) reason to redesign a legacy product is to make it look better.

A redesign is a massive investment in resources. Our product team needed goals driving our work in order to be successful beyond aesthetics.

Key stakeholders – CEO, CTO, Engineering, Design – sat down in our Define and Align meeting to evaluate why we should redesign our web platform in the first place. Facilitated through a sticky note session in Whimsical, everyone had the chance to share their hopes and concerns with the web product as it existed and important elements to consider as we rebuilt.

Pocket Prep Design Process: Define and Align, Research, Wireframes, Visual Design, and Dev Handoff with the opportunity to talk to users and gather feedback between each step.
Pocket Prep Design Process

An all-hands kickoff meeting, while important for the success of the project, also lent itself to the kind of culture our UX Lead and I wanted to cultivate as key values in our young product team. Aligning on our goals and expectations early gave our team a shared idea of success to work towards and set the tone for a collaborative project.

Through our discussion, we distilled two overarching goals:

Elevate our web platform to match our mobile app’s features.

Not only were our web users getting a sub par experience, but we were making our work harder. On the product side, we had to design and code for two systems or choose to further the gap between our platforms. Our marketing team faced a challenge in communicating feature announcements that were only relevant for a portion of our users. Every item had an asterisk for *only mobile apps. This created a disjointed experience for our users and made our marketing team’s job more difficult.

Enhance web features to take advantage of the screen real estate a desktop format offers.

Initial user behavior data showed that our web users preferred a desktop format for long study sessions. We wanted to advance our desktop experience to account for this preference.

Research

In a crunched design process, competitor and user research is often the first phase to get scrapped in favor of “speed” and “efficiency”. This is a huge mistake. It typically results in poor design and incredibly expensive rework.

“It’s better to spend three weeks researching than to spend one year building the wrong thing.”
Jeffrey Zeldman

Research allows us to get an understanding of the competitive landscape, compare our goals with trends in the industry, and—most importantly—gather data on what our users care about most.

As an in-house team, it’s easy to feel like we know our users. We wanted to check our own assumptions and understand the details of who our users were, what their study habits entailed, and how they used Pocket Prep. To do that, we sent a survey to our web platform users and gathered user behavior data in-product.

Survey

Our goals for the survey were to understand:

  • Why our users chose Pocket Prep
  • What they found useful about the web platform
  • What they would improve about our product
  • Basic statistics around their study habits
Screenshot of user survey and invite email. Over 650 studiers responded.
We reached out to web platform users to gather feedback on our existing web and mobile platforms. Over 660 people responded.

Over 650 users submitted responses. Most of our survey questions were open ended, which made parsing through the data manually a pain, but it meant that common themes were born of authentic free response feedback.

Common themes were:

  • Requests for a Full Practice Exam to imitate the real exam experience
  • Strong explanations that accompanied our quiz questions were a strong selling point for users
  • Appreciation for performance statistics by exam subject
  • Appetite for customizability around quiz controls
  • A craving for community and encouragement to get them through the anxiety of studying for such intensive exams
  • Love for the on-the-go nature of our app to fit in regular small study sessions
Graph for users' time spent studying: 3% studied over days; 5% studied over 1-2 weeks; 13% studied 2-4 weeks; 26% 1-2 months; 21% 2-3 months; 19% 3-6 months; 13% more than 6 months.
This data was helpful for us to know distribution of our users' study timelines and went on to influence our design for "extremes" in the interface.
Text: What is frustrating about using Pocket Prep? Repeating questions, wish there were more questions for a mock exam, want app features in desktop version
Our survey measured what users find frustrating about using Pocket Prep. They responded with a common bug, wishing we had more questions (especially for a mock exam), and incorporating mobile app features into our web platform.
Graph of reasons users use the web platform
It was notable that desktop users preferred the web platform because it's closer to the exam experience; that informed our interface choices while designing our quiz mode.

While a survey is a great way to collect large scale self-reported user data, it’s also important to pay attention to user behavior where possible.

User Behavior

We pulled device usage data from our web platform and found, top level, that 64% of our users were on desktop, 2% on tablet, and 34% on mobile. We also found that our web users’ average session length was twice as long as mobile app users, which aligned with survey feedback that desktop was preferred for longer, more focused study sessions.

Graph: 2020 Web Usage by Device. 64% Desktop, 34% Mobile, 2% Tablet
More than 1/3 of our web platform usage sessions were on mobile browsers even though we have a mobile app. This was important to consider as we designed a responsive product.

We reconvened as a team to discuss these findings and consolidated several additional feature concepts based on community feedback to move forward with.

Wireframes and user testing

From this foundation of user data, I utilized Whimsical to build out the basic information architecture and user flows. From there, I wireframed core functionality of existing features in the mobile app and potential features we identified through our initial user research and internal team discussions.

The rough nature of wireframes enables stakeholders to focus on the functionality rather than the sexy visuals and respond with a level of fidelity appropriate to the design. We could iterate through good and bad ideas quickly without feeling a need to achieve pixel perfection.

Wireframes for new features and navigation layouts.
Wireframes allow for quick information architecture and layout iterations.

Wireframing key layouts and workflows allowed us to consider:

  • How might this work?
  • How might the layout roughly look?
  • What legacy concepts might be enhanced?
  • Is a new concept technically feasible?
  • What are some promising ideas we can evaluate in user testing?

After a round of clarifying edits with stakeholders, we arrived at a good point to verify functionality and potential feature value through focused user testing.

I pulled 10 strong responses from the customer survey and reached out for 1:1 interviews to test wireframe prototypes (with an Amazon gift card as compensation).

I conducted eight one-hour Zoom user tests with Pocket Prep users over the course of two weeks. We tested onboarding flows, discussed the value of specific feature ideas, and tracked how users interacted with several of our wireframed prototype flows.

Eight illustrated faces of the users we tested
We tested a range of users with different exams and levels in their careers.

A side note on the value of user testing:

If you or your team questions the value of user testing, ours cost $800 and about three weeks of my time to prepare, conduct, and process feedback into a presentation. It breathed life into the work we were doing and personally gave me energy in this long process. It was a reminder that everyone from college students to advanced professionals use our app to prepare for what is an important and weighty step in their careers. It was humbling and inspiring to talk 1:1 about their journeys and how they used Pocket Prep to move forward in their goals.

From our testing, we found that users valued the following features most:

  • Full Practice Exam a mock exam unique to their chosen certification exam that would imitate the experience of the real exam — includes precise number of questions, a pre-established time limit, etc.
  • Review Tab a place to search, filter, and re-study questions they already answered
  • Notes feature a way to highlight specific content and save it into a separate Notes section in the platform
“The question, the explanation, everything. No, [the Review tab] is really a slam dunk.”
Laurye, ASWB LCSW Studier

Finally, we tested three navigation layout options to gather data on how best to organize our core tabs.

Three wireframe screens pictured with different options for layout and Question of the Day placement.
Most of our users preferred a top nav layout and opted to move the Question of the Day feature from its own tab to the home page.

Our team had discussed moving the popular Question of the Day feature from its own separate tab to join the other five quiz modes we offer. Of the options we displayed, most users found the top navigation with the Question of the Day front and center (alongside other study modes) on the “home” study tab to be most clear. This user feedback validated a significant shift in our product’s navigation and information architecture.

“I prefer it [on the study tab]....Because it's a reminder that, if nothing else, here's the material that you've been studying so hard to do. And I would absolutely answer it because it was in front of me.”
Sarah, Registered Dietician Studier

Though considering user feedback is paramount to building a successful product, we needed to balance user preference with the team’s appetite and workload. After presenting user testing findings to the larger team, we determined we would pursue the Full Practice Exam and Review tab features but table the Notes feature; it would be a larger development lift than we were able to commit to for a V1 launch.

Armed with user test data that validated our layout, flow, and feature decisions, we moved on to visuals.

Visual Design

Our design team works in Sketch for our UI design. In developing the web platform’s visual designs, I needed to hold the existing mobile app’s UI in mind for continuity of components and brand feel while also optimizing for the screen real estate a desktop format afforded.

I also had to consider an upcoming product initiative to bring our mobile design to the level of our fresh web design; if I pushed the web app design too far, achieving parity with our mobile apps would be more difficult.

Our new pocketprep.com website had just launched. I took the web platform’s visual design stage as an opportunity to inject some brand delight and friendliness into the interface through colors and patterns from the website. I had a fresh visual language to play with.

Survey feedback illustrated how many of our users log into our app and feel immense anxiety for their exam. I wanted to create an interface that felt safe, friendly, and encouraging. With this in mind, I introduced a home page banner that shifted in color throughout the day and greeted users with encouragement and humor.

Home page banner colors and greetings shift by time of day.
Home page banner colors and greetings shift throughout the day for a touch of warmth and encouragement.

While opportunities for playfulness were fun to explore, there were also practical visual functionalities to consider. Though our “web” product was ideally for desktop use, it also required design for full responsiveness across mobile, tablet, and desktop breakpoints.

Study tab shown across desktop, tablet, and mobile screens.
A fully responsive design across all breakpoints and dark mode to ensure a pixel perfect experience no matter the preferred browser format.

We incorporated dark mode as a moment of delight and a savvy way to acknowledge the studying preferences of our night owl users.

All of these elements were incorporated into vignettes to ensure our designs were as consistent as possible across web and mobile, from light/dark mode colors to individual UI components.

Image navigation component states across hover focus, dropdown, etc.
Design creates UI documentation called "vignettes" to capture detailed component states and design notes for development handoff.

Hand-off and development

Upon hand-off of visual designs to our development team, the devs and I collaborated frequently to ensure designs were implemented correctly and improved when Colin and Robby saw problems.

Previously, Pocket Prep’s design and development were entirely built by agencies before our co-founders chose to build an internal team. The benefit of in-house UX and dev teams is deep and consistent collaboration to ensure implementation adheres to the design. A strong design doesn’t matter if dev and design don’t get the time to ensure implementation reflects it.

To maintain that accurate adherence, our UX and dev teams held eight one-hour independent VQA (Visual Quality Assurance) meetings to review implemented features and point out areas where designs didn’t match the dev environment. These meetings were a significant investment of time from both the design and dev sides but crucial to ensure the final product our users experienced matched the designs.

From a functionality standpoint, the benefit of two in-house devs who maintain our CMS and understand the structure of our data was crucial in solving edge cases that inevitably arose. It resulted in a higher standard of code for good maintenance and better performance.

Through design and dev challenges, we were able to work through issues thoroughly and collaboratively as a team.

Graphics across dark mode and light mode.
Designing across dark and light mode acknowledged our night owl studiers.

A note on Accessibility

When I first joined Pocket Prep I advocated for stronger Web Content Accessibility Guidelines (WCAG) adherence as a company. An accessible product would be a smart business decision and ethically the right thing to pursue. It’s a testament to our leadership and larger team that they immediately supported an accessibility initiative fully with their time and funding.

Pocket Prep partnered with Ablr, a digital accessibility company, to ensure our designs for web and mobile reached accessibility standards. We knew we had a lot of room to grow, and our product team as a whole felt that reaching WCAG compliance was an important element of Pocket Prep’s mission to make “Education accessible for all”.

I could write a separate post about accessibility work alone (as could our developers). For now, this is an ongoing initiative that our product team is actively working on across our web and mobile platforms. You can find our preliminary Accessibility Statement for our products here.

Image to show checking color contrast with Stark color tool.
Pocket Prep design and development teams have prioritized WCAG accessibility compliance for our study products.

A retrospective

This project ultimately took 8 months to design and 6 months to build.

We followed a rigorous design process driven by community feedback, user testing, and design systems thinking to not only build a strong product but to lay a foundation our design and development teams could build upon in future products.

Our launch was incredibly smooth – only a smattering of small bugs. We took the proper time needed to build a strong product.

Some things we did well:

  • CEO, CTO, and dev teams were involved in goal setting, brainstorming, user flows, and information architecture discussions
  • We held regular check-in’s for constructive feedback on designs so all parties were informed and had buy-in by the time I delivered final designs
  • We took advantage of our new brand (fresh from the website redesign) and incorporated it subtly into our product for a cohesive visual language
  • We gathered feedback from our users early and often
  • We held a rigorous VQA process through implementation so the product was built with high fidelity to the designs

What we could have done better:

  • I designed the whole product and delivered it in its entirety to development. They caught visual inconsistencies that required visual rework across the product. If I had delivered one section at a time (ie, onboarding, then the study tab), then I could have made adjustments by learning from each section’s handoff. 
  • We brought our marketing team into our design process mid-way, when we should have involved them as a key stakeholder from the Define & Align stage.

Closing & thanks

We are grateful for the generous feedback from our Pocket Prep users – over 650 of them provided thoughtful suggestions and insights into how they use our product day-to-day in order to study for their exams.

Thanks to all of our team members, each of whom contributed to a successful launch. Special shoutout to our product team:

Pocket Prep is excited to present our new web platform — and now mobile apps — to our users after over a year of work. We’re excited to hear back from our users and keep improving.