IDP Password Strength Testing
Overview
When: Dec 2019
Length: 2 weeks
Role: Primary researcher and designer
Team: 1 Product Designer (myself) and 2 Product Managers
Task: Research and design a new account creation user flow with a focus on encouraging users to create better passwords
Define
Vacasa had created an Identity Provider (IDP) team that was tasked with creating a backend tool to connect and manage guest and owner accounts.
Part of this process meant that new and existing users would need to make sure their passwords met certain standards. The IDP team approached me to design a new account creation flow that would encourage users to create stronger passwords without introducing too much friction.
Ideate
In brainstorming with the IDP team and the design team we came up with three scenarios to test:
- Standard password input with a list of password criteria and checking off that criteria as the user meets them.
- 'Strength meter' that indicates to the user the strength of their password through a bar filling and changing in color.
- Hybrid solution that includes both the criteria and the meter.
Build
In order to test these I created the designs in Sketch and then pulled the flow into FramerX. Using FramerX's React code component features I created the various password components.
The criteria component would update as the user typed and if their password met the criteria then the line would mark itself as complete.
For the strength meter, I found a npm package that checked passwords and returned a strength value. I then used that value to update a bar's color and length in the prototype.
For the purpose of the user testing I added tracking for certain actions:
- Attempting to sign in without an account
- Exiting the password input without a qualifying password or a password 'soft fail'
- Attempting to advance to the next screen without a qualifying password or a password 'hard fail'
.svg)
Results
My assumption coming into these tests was that the strength meter would lead to longer and more complex passwords than the criteria and that the hybrid would lead to similarly strong passwords with less friction for the user.
After multiple rounds of testing on UserTesting.com, I found that the criteria only led to the strongest passwords and the least amount of user friction. Users entered longer and more complex passwords on the criteria prototype while experiencing less soft and hard fails. The average number of soft plus hard fails for the criteria was 1, compared to the average of 4 for the strength meter and 3 for the hybrid.
Lookback
What I liked:
- Data and test-driven decisions led to the implementation of a better user flow. Rather than designing on my assumptions around the strength meter, I allowed the data to lead the way and influence the final design.
- Secondary test metrics like 'Sign In Attempts' notified me of other pain points outside of the password focus. This led to the implementation of a 'Create Account' button rather than a small 'Sign Up' link.
What I learned:
- Creating concise and detailed scripts for user testing flows that usually require context.
- Implementing complex code components in FramerX. Including adding an npm package for checking password strength.
What I would do differently:
- Split the 'Create Account' button change into a different test. While it was a minor change, it could have affected other test metrics or user feedback about the whole flow.