It didn't work. Adding the value questions created more friction and led to a lower conversion rate (10%) compared to the control (24%). Although, for those who completed the value questions, the conversion rate was 46% (note: this result could be attributable to a self-selection bias).
Adding value questions to a job search survey may increase friction, which could lead to less completed job searches. It it weren't for this added friction, there may be an interesting case for evoking value-based thinking.
According to the Government Accountability Office, the American retirement system designed to help workers and retirees faces major challenges in the days and years ahead.
The average American does not save enough for a comfortable retirement; nearly half of all families have no retirement savings at all. Additionally, Social Security, faces long-term funding challenges, and may not provide adequate income for millions of older Americans who depend on the benefits for most or all of their income.
To combat this crisis, we reviewed, and performed a survey around an AARP project focused on helping people over 50 earn extra money through flexible contracting work. AARP’s “Extra Money” webpage educates users around gig economy jobs, like Uber or TaskRabbit, based on their skillset (“I can...”) and what they enjoy doing (“I like...”) as well as the potential pros and cons of certain gig economy jobs.
We conducted a behavioral diagnosis of the “Extra Money” website and noted the following key insights:
There were several questions in the skills assessment tool. There were over twelve questions for users to answer regarding their skills and interests before they were presented with potential job opportunities. We worked with AARP to decrease the number of questions, reducing unnecessary barriers, while still allowing users to attain enough information to take action.
There were several questions in the skills assessment tool. There were over twelve questions for users to answer regarding their skills and interests before they were presented with potential job opportunities. We worked with AARP to decrease the number of questions, reducing unnecessary barriers, while still allowing users to attain enough information to take action.
The website focused on educating users on job opportunities but was less effective in communicating how to take action. Once a user determined a gig opportunity might be right for them, without a deadline or a reason to act in the short term, we hypothesized that users would be in a “discovery” mindset versus an “action” mindset when searching through the potential jobs. We designed an experiment to test our hypothesis.
We focused on trying to increase users’ intrinsic motivation to apply for job opportunities if such opportunity made sense for the individual user. We hypothesized that getting users to list their life values and internalize why getting a contracting job could fit in with those values might lead to higher intrinsic motivation.
To test if framing jobs in terms of values would increase the number of users clicking on a specific job opportunity, we modified AARP’s search tool. In the control condition, users answered four questions, detailing their likes, resources, willingness to engage in certain activities, and skill sets. In our experimental condition, users took an additional survey with two questions on their values and then answered the same four questions as in the control condition. Approximately 15,500 users entered our experiment.
First, we analyzed conversion rates across the two conditions, measuring the number of people who successfully searched for a job. Our control condition generated a 24% conversion rate. In contrast, our experimental condition generated just a 10% conversion rate, driven by a high drop-off rate at the start of our two-question survey. If users completed our two-question survey, however, our conversion rate increased to 46% (note: this result could be attributable to a self-selection bias). In short, when users pass our survey, they become more engaged and are more likely to successfully search for a job. The effect of the additional friction added to the flow by the values questions was more impactful than the additional motivation provided by the values questions, resulting in lower completion rates in our experimental condition.
The most important self-reported values were relationships with family and friends, independence, and learning new skills.
The least important self-reported values were music, government or politics, and belonging to a group.
Respondents cared more about having a flexible schedule than about earning more money.
The most common words used in the open-text question were “can” and “like”, as people described what they are able to do and what they enjoy doing.
The AARP Extra Money team plans to use our results and learnings to improve the search function and language used on their website.