Commentary from a concept testing participant
Job Ad Optimizer is a feature I designed that is included with The Predictive Index’s (PI) Hire application. Hire uses PI’s Behavioral Assessment data to help recruiters find candidates who align best with a job’s targeted behaviors, so they can hire the right people for roles where they’ll thrive.
Job Ad Optimizer (in-development name: Job Ad Analyzer) goes a step further, using AI language processing to evaluate a hiring team’s job ad text against the job’s targeted behaviors, helping recruiters adjust the language in their job ad to better attract the right candidates.
The end result? Higher quality applicants to the roles they post.
How could we show recruiters value before they had candidates?
Hire’s updated free version of the software was showing low usage past an initial entry point into the software; people were entering and creating jobs but were not going far enough in the process to unlock candidate insights, as the value of the tool was hard to see until they had applicants in the system. Because the hiring process can take time, this meant product value wasn’t immediately apparent without candidates.
Job Ad Optimizer was proposed in the hopes of boosting usage and showing users more immediate value.
With Job Ad Optimizer available for free, our user (an HR recruiter) would be more likely to…
Create a job in the software when the tools to refine a job description are right at her fingertips
Engage the hiring manager by sharing the job description from within the tool
Unlock candidate insights because she’ll have more certainty that candidates are a close fit for open roles
Personas existed, but lacked a UX lens.
At The Predictive Index, we had an existing set of personas built by our marketing team, aligned to each product. For Hire, our star persona and primary user was Tara, a recruiter. While our personas were built on data compiled by the marketing team, the information they held was primarily messaging-oriented. This meant additional research and detail would often be filled in by the UX team as part of our process to ensure we aligned with Tara’s needs as a user.
Enter the User Stories.
Due to timeline constraints, we couldn’t do formal research to fill out the persona, but we could make some assumptions in the form of user stories and validate them when we took the initial ideas to concept testing. (More on that later!)
User Story #1
As a recruiter/hiring manager, I want to write a job description that will attract a good number of candidates who fit my criteria for the role so that I can fill job recs successfully with candidates who will thrive in the role they apply for.
User Story #2
As a recruiter/hiring manager, I want to write a job description that is inclusive and free of bias, so that I can begin incorporating more inclusive measures into my hiring processes and attract a balanced selection of candidates from diverse backgrounds.
User Story #3
As a recruiter/hiring manager, I want to make sure that the job description I post will represent the role expectations as accurately as possible so that potential employees don’t get surprised or impacted should they accept the job.
Designing for an AI language processing tool
I partnered as product design lead with a product manager and team of developers who were constructing the AI model that would drive the Job Ad Optimizer feature. This particular feature was begin developed by a separate scrum team from the core product, though we worked closely with the core team at each step to ensure our feature could be seamlessly folded into the workflow of the Hire product. To that end, I regularly communicated with the primary designer on Hire to exchange feedback as I worked through the designs.
Because Job Ad Optimizer’s main functionality would be driven by an AI language processing model, I also made sure to bring the developers in during the first stage of ideation for feedback, to ensure our designs wouldn’t be promising something that wasn’t technologically feasible. Our entire organization was remote, so all communication was done over a combination of Slack and Zoom.
As design lead for this particular feature, I organized the initial product lab alignment kickoff sessions to pitch our idea to the greater product team, created all design iterations from initial concept to executable high definition designs, and led all user research efforts during the early phases of the project.
Working with constraints of technology, feasibility, and time
As scope was concerned, we worked through the definition and design phases for roughly a quarter, giving us ample time to conduct research in the early stages. While the development team was doing research on the technical side in order to build their AI model, I was able to run parallel and conduct competitor and user research while planning the project.
Technological feasibility was the biggest constraint; we had no product or feature that required an AI language model to function, so the team was limited by what they could research and ramp up to build in the time we had. We preserved our ultimate design vision as a longer-term guide to steer the feature’s roadmap but pared down the initial release capabilities to match what development could currently produce.
Discovery to definition: competitive research and getting alignment
With the feature proposed and given a quarter to work on, the product manager and I worked together to establish the requirements for the feature.
To zero in on capabilities for the feature, I conducted a competitive analysis on competitors in the job ad writing arena such as Datapeople.io, and on AI writing tools such as ProWritingAid and Grammarly.
Armed with this knowledge, we determined basic capabilities, flagged what would be out of scope (and why), logged outstanding questions for further research, then presented it to the product lab stakeholders for further feedback and input.
As part of the presentation, I drafted a user flow to showcase where the feature fit in Hire’s existing workflow.
User flow showing Tara’s journey through the feature, relative to where it fits in Hire’s overall workflow.
Concept testing: validating early ideas with the customer base
Once everyone was aligned and our pitch for the high concept of the feature was agreed upon, I began drafting key screens for the experience. With these screens ready to go, I recruited participants who aligned with our persona via Respondent and conducted a series of concept test interviews.
The feature was well-received overall, with a majority of participants excited by the prospect of what Job Ad Optimizer could help them do. We gathered useful data on the job ad writing process on the user’s side of things, which gave us helpful context around how to prioritize certain functionality. We did get some feedback that led to iterating on the UI and layout as we carried the designs forward and dug deeper into the interaction design.
WHAT WE TESTED
Early draft of the post-analysis screen used in concept testing.
WHAT WE LEARNED
Design and development partnering on iteration and feedback loops
I presented the findings to the stakeholders, fellow designers, and the development team along the way, and proceeded to flesh out the designs in greater detail with feedback from my designer teammates. I presented each iteration to the development team as we went, gathering their feedback and iterating as their technical discoveries fed into the requirements further.
Some features had to be removed or scaled down due to technical constraints discovered along the way, such as inclusive language scoring: we intended to keep it as part of the future vision, but it required a separate model to be built and trained, which we didn’t have the resources to tackle before launch.
Low fidelity wireframes of the design
A sample of the core screen from the final high fidelity designs that were built for launch
Smooth sailing to launch: setting the precedent for a great product design & development process
Once the high-fidelity design was complete, I walked the development team through the final version and fielded the remaining questions one more time, then it was in their hands. I was on call from that point forward if they needed some design adjustments along the way.
The Job Ad Optimizer feature successfully launched its initial version in the Spring of 2023 when it was added to the free version of Hire.
It was the most impactful feature in the product with the highest revenues in the portfolio, boosting adoption by 37% and engagement by 36%, and task success by 26%, compared to <5% other features offered.
This feature was one of the smoothest journeys from discovery to launch I’ve ever been a part of, in no small part because of the incredibly strong collaboration and active communication between myself, the product manager, and the development team. The processes we tried on this project would also go on to become defining milestones in our overall process within the product lab.
