IDM for Microsoft Active Directory

CLEVER

Earned Clever $1M ARR by building empathy with Active Directory admins

Dec 2022 - Apr 2023

TIMELINE

1 product manager
1 product marketing manager
4 engineers

TEAM

User researcher
Designer

ROLE

Survey design
Usability testing

FOCUS

At-a-glance summary

Question

The identity management (IDM) team wanted to know: what are the core customer needs Clever must support when we add Microsoft to our IDM product as an identity provider?

Approach

I chose to conduct mixed methods research in phases to gain alignment from the IDM team as we iteratively learned and made decisions. With this approach, I was able to build confidence amongst the team as we charted Microsoft territory that was unfamiliar to us all:

  • Survey to assess the market landscape by gathering quantitative information at scale.

  • 1:1 interviews to understand how district administrators use Microsoft Active Directory (AD) by gaining deeper qualitative insights.

  • Moderated usability testing to evaluate our initial solution by getting honest behavioral feedback from users paired with an opportunity to ask follow-up questions. 

  • Secondary research on Microsoft AD to empathize with admins and fill our team’s gaps in understanding by building an AD instance and finding every piece of information we were asking customers to provide Clever.

Impact

  • Research informed product improvements, which helped the team reach its business objective: grow the IDM business by supporting customers who use a Microsoft identity provider.

  • The team met this business objective by earning $1.03M annual recurring revenue (103% to key result) and achieving a 70% conversion rate for IDM customers who used AD as an identity provider (116% to key result).

Context

  • In 2021, Clever releases its first identity management product, Clever IDM, which automates Google account provisioning for school districts. 

  • The product is successful, but customers need an account provisioning solution for Microsoft as well as Google.

  • I join the team at the end of 2022 as a researcher and designer.

Adding Microsoft to the Identity Management (IDM) product

CONTEXT

CONTEXT

Opportunity

  • Although the team had already identified the need to support Microsoft, they did not yet know which Microsoft identity provider to focus on first: Active Directory or Azure.

  • My task as a researcher was to deeply understand customer context and needs for the Microsoft identity provider that Clever IDM would support in its next release.

Assessing the market landscape

Research goals

PLANNING

  • The team knew there was an opportunity for us to expand our IDM product to support customers who use Microsoft as an identity provider, but they needed a deeper understanding of customer needs in this segment. 

  • The purpose of setting goals and criteria was to ensure we had the information we needed to make decisions about how to approach the market.

Research goal

Determine the market share of Microsoft Active Directory vs. Microsoft Azure as a school district’s primary identity provider

Describe our competitive positioning for identity provisioning in school districts.

Determine the biggest gaps school districts have with identity provisioning today.

Success criteria

We have enough data from districts to make a decision on which Microsoft identity provider to support first

We have enough data from districts to understand the size of the addressable market and who our main competitors are.

We have enough data from school districts to prioritize features that will differentiate our solution

METHODOLOGY

Survey

Since our research goals focused on developing a broad understanding of the market landscape and where our identity provisioning solution would be competitive, I chose to send a survey to gather quantitative information at scale.

Building confidence with current customers

RECRUITMENT

  • We chose to reach out to our current customers because we were only serving a small fraction of them with our identity provisioning solution.

  • To feel confident in our survey results, we wanted to recruit at least 100 district administrator participants. We ultimately exceeded our recruitment goal and had 199 survey participants.

  • To go as broad as possible and understand the market landscape we reached out to all segments of our customer base.

Big

Districts with 60+ schools

Mid

Districts with 20-59 schools

Scaled

Districts with 1-19 schools

  • As their primary identity provider, 52% of participants reported AD and <1% reported Azure. Of those who reported AD as their primary identity provider, 41% reported Azure as an additional identity provider they use.

  • Further, of the customers who reported Google as their primary identity provider, which our product already supported, those who reported using an additional identity provider unanimously reported AD.

For Microsoft districts, Active Directory (AD) is the source of truth identity provider

SURVEY INSIGHT

Data

I found that AD was the the leading primary identity provider across district segments

Decision

The team prioritized supporting AD as the next identity provider for our product to support

  • 74% of mid districts did not have an existing third party automated solution for identity provisioning.

  • The same was true for 65% of scaled districts and 0% of big districts.

Clever has the greatest addressable market with the mid district segment

SURVEY INSIGHT

Data

When asked how they create user accounts, mid districts had the highest rate of choosing “Automated process - internal scripts” or “Manual account creation”

Decision

The team focused on the mid district segment as our primary market

The most compelling value propositions are hands off, secure, and low cost

SURVEY INSIGHT

These value propositions received the highest amount of “very compelling” or “somewhat compelling” ratings.

Data

I used a 4-point likert scale without a neutral option to ensure a response about a specific opinion. Hands off, secure, and low cost revealed the biggest gaps admins had with identity provisioning.

Decision

The team led with these value propositions on our in-product sign up page for IDM

Understanding how admins use AD

Research goals

PLANNING

  • Now that we had a broad view of the market landscape and knew we would focus on AD as the next identity provider to support, we wanted to get a deeper understanding of how customers used AD.

  • The purpose of setting goals and criteria was to ensure we had the information we needed to prioritize specific product and design decisions.

Research goal

Describe the top needs of districts that use AD when provisioning accounts

Illustrate how OU organization is different in districts that use AD than districts that use Google

Determine if districts use multiple identity systems and how the differences are managed

Describe what districts think about the differences between AD and Azure as well as how they rank provisioning systems.

Success criteria

We have enough data to decide whether the OU structuring will be the same or different than our existing product that supports Google.

We have enough data to decide if we should prioritize supporting multiple identity systems and, if so, how the design should support multiple systems.

We have enough data to understand how AD and Azure should interact with one another in the design.

We have enough data to prioritize the features of our design.

METHODOLOGY

1:1 interviews

Because we had identified Active Directory as the next identity provider to support from our survey data, and wanted to dig into how district administrators use Active Directory, I chose to do 1:1 interviews to get deeper qualitative insights.

Follow-up with survey participants

RECRUITMENT

  • In our survey, we included a question that asked participants if they would be open to a follow-up interview.

  • We emailed all 33 participants who indicated that they would be open to an interview and reported using AD, 24 scheduled a call with us, and 10 showed up.

  • Automated syncs between their school information system and Active Directory

  • One product that can do account provisioning for Google, Active Directory, and Azure

  • Prevent duplicate instances of accounts once provisioned

The top needs for districts when provisioning AD accounts are:

INTERVIEW INSIGHT

Data

An admin explains how they manually provision accounts today: “Accounts are built in Active Directory then passed over to Google. The same OU exists in both places.”

Decision

Leverage what we built for Google to automate syncs and prevent duplicate instances of accounts. Prioritize multi-destination to ensure IDM can do provisioning for more than one identity provider per district.

  • Cloud-based systems like Google and Azure are easy to connect to other systems and manage.

  • Admins don’t get these same benefits with AD because it is a legacy, on-premise platform, but it is deeply embedded into their existing processes.

Districts will continue using AD even though they prefer cloud-based systems like Google and Azure

INTERVIEW INSIGHT

Data

An admin shows us their complex, legacy AD instance where they manage accounts.

Decision

The team doubled-down on AD as the next identity provider IDM would support and focused on making the connection to Clever step simple.

Evaluating our initial solution

Research goals

PLANNING

  • Once we had our initial IDM for AD flow built, we wanted to evaluate how well our initial solution was meeting customer needs. 

  • The purpose of setting goals and criteria was to ensure we had the information we needed to make adjustments to the IDM for AD flow based on user feedback.

Research goal

Describe how admins move through the flow

Illustrate how self-serve support tools are used in the flow

Determine admins understanding of what they are doing in the flow

Success criteria

We have enough data to make decisions on how to modify the self-serve support tools included in the flow

We have enough data to make decisions on the language and framing of steps in the flow

We have enough data to make a decision on which steps need usability improvements

Since we had built an initial solution and wanted to evaluate how well it would meet customer needs, I chose moderated usability testing. This method would allow us to get honest behavioral feedback from users paired with an opportunity to ask follow-up questions and go deeper on particular usability issues. 

Moderated usability testing

METHODOLOGY

In-product pathway to capture interest 

RECRUITMENT

  • To recruit for usability testing we include a “Email me when AD is available” button to capture districts that were interested in adding AD to IDM. 

  • We reached out to all 23 districts that expressed interest through this pathway to see if they would be interested in testing our flow, 11 districts responded, and 9 and completed testing.

The two biggest hurdles in the flow are the connect and preview steps

USABILITY INSIGHT

  • For the connect step, all participants either completed with difficulty or did not complete.

  • For the participants who made it to the preview step, all completed with difficulty.

Admins can’t easily connect Clever to AD because they don’t know where to find what we’re asking for

USABILITY INSIGHT

Finding specific required information such as their LDAPS URL, Bind account, and Base DN were challenging for admins to find because they rarely connected their AD instances to other systems.

An admin struggles to complete part of the connect step:
“Bind account? Oh boy, I’m gonna have to look that up…”

  • After admins completed the IDM for AD flow, they could download a csv file that showed a preview of how Clever would manage their AD identities.

  • Showing a user email rather than user principal name conflated Google account provisioning with AD, which confused admins.

  • Admins weren’t clear on what the headings “forward data” and “reverse data” referred to.

The language in the preview step confuses admins, which makes them hesitant to provision accounts

USABILITY INSIGHT

When admins didn’t understand the headings of their preview file, they were hesitant to provision accounts.

  • Many of the participants had these legacy AD systems passed to them and their role is to keep them running without full knowledge of how it works.

  • We’re putting their core job at risk by asking them to do specific things without supporting them with advice and documentation.

For many of the participants, what we’re asking them to do in our IDM for AD flow is disruptive and scary.

USABILITY INSIGHT

An admin has difficulty because this legacy AD system was passed on to them:
“I’ve added the Clever IPs to our firewall, but I’m not sure about the LDAPS URL…”

  • By the end of usability testing I shared with my team that the connect step was causing the most problems for admins: they didn’t know where to find what we were asking for.

  • To provide accurate advice and documentation we had to learn where admins could find the AD information we were asking for.

Our team needed to learn more about the AD information we were asking for

SHARING AND ACTIVATION

I had hit a wall with my research methods up to this point. We needed to learn more about the AD information we were asking admins to provide Clever.

Becoming an AD admin to build empathy

Research goals

PLANNING

  • This phase of our research focused on building our understanding of where to find the specific pieces of AD information we were asking admins to provide Clever.

  • The purpose of setting goals and criteria was to ensure we had the information we needed to make decisions on how we guided admins to provide the AD information we needed.

Research goal

Describe the steps admins must take to find the AD information Clever needs to connect

Success criteria

We have enough data to create walkthrough materials that guide admins to provide Clever the required AD information

Secondary research on Active Directory

METHODOLOGY

  • Since both admins and our team were unclear on where to find specific information in an AD instance, I chose secondary research to empathize with admins and put myself in their shoes as they navigated AD to find the information Clever was requesting.

  • As a result, I shared the usability testing results with my engineering team and asked them to set up our own AD instance. This way, I could find every piece of information we were asking customers to provide Clever in the connect step.

I asked my eng team to set up our own AD instance so that I could become the customer and experience how an admin would find the AD information Clever was requesting.

  • After a few days of digging around the AD instance my team had set up, I found where all of the information we were requesting lived.

  • I recorded step-by-step videos of where each piece of information lived to document what I had learned.

All of the information Clever asks for can be found, but it takes some digging

SECONDARY RESEARCH INSIGHT

In this step-by-step video, I showed where admins could find one of the pieces of information required for the connect step: a bind account username and password.

Impact

Research informed product improvements

IMPACT

Using my findings from the usability testing and secondary research, I collaborated with my product manager to prioritize product improvements and my engineers to build the improvements I designed.

Before

The flow conflated asking for an LDAPS URL with allow listing Clever’s IP addresses on a firewall. This confused admins and didn’t proactively address a common error in the LDAPS step.

After

The flow separated allow listing Clever’s IP addresses on a firewall as step one and providing an LDAPS URL as step two. This reduced cognitive load and proactively addressed the most common error in the LDAPS step.

Before

The flow directed admins to contact support if they were having trouble with any of the information we were asking for in the connect step, but our support team often could not help.

After

The flow directed admins to a help center page, specific to the information Clever was requesting, including the step-by-step videos I had recorded. We made this update for LDAPS URL, bind account, and base DN.

After (help center page for bind account)

The help center page that showed admins how to find their bind account username and password in AD, then provide those credentials to Clever.

Before

The final step of the flow asked admins to run a final test of their AD configuration. The errors associated with this test were vague and provided minimal paths to recovery.

After

The final test of the AD configuration included errors that highlighted the specific step that had an issue, provided a more detailed explanation of the error, and provided a link to the help center article specific to that error.

Before

Admins could download a csv file that showed a preview of how Clever would manage their AD identities. The email, forward data, and reverse data headings confused admins.

After

The csv file had headings that matched what admins reported would make more sense: UPN, existing data, and planned data. We also switched the heading order to match admins’ expectations: existing before planned data.

Iteration to IDM helped team achieve business objective

IMPACT

Business objective: Grow the IDM business by supporting customers who use a Microsoft identity provider.

$1.03M

Annual recurring revenue (ARR)
after adding AD functionality

103% to key result

70%

Conversion rate for AD
(completed onboarding and purchased)

116% to key result

Reflections

IMPACT

Start with desk research first

Going into this project, we assumed that admins would know where specific information lived in their AD instances. That assumption was wrong but, even if it was true, I should have built my own understanding of where that information lived prior to beginning other research to have a clearer sense of the admin’s context surrounding the Clever customer experience. If I was to do this project again, I would have started with desk research first to build my understanding of the admin's context in AD.

Get a sense of participants’ familiarity with AD before usability testing

In our recruitment for usability testing we relied solely on customer interest without using any screener or questionnaire to understand participants’ familiarity with AD. This led to a group of participants that tended to be more experienced with AD. If I were to do this project again, I would have gathered this information on familiarity in a screener or questionnaire prior to usability testing. That way, we could have created a more balanced pool of participants that included both admins that were experienced and inexperienced with AD.