It is unclear if this intervention "worked". Report does not explicitly say if any of the conditions were a control, so it is hard to guage efficacy compared to control. While the results seem to point to the stacked bar being the most effective design (especially when an "overdraft risk" is present), the interpretations of each design was fairly low. While this study is a good way to preliminarily understand design efficacy, it seems that another round of testing with clearer explanations may lead to more reliable results.
The key lessons learned are the following: 1) Users need clearer explanations of the graphs that were used within the survey; 2) the stacked bar visual gave users the strongest impression of overspending, and; 3) rate of spend required either clearer explanation or a higher data visualization literacy on behalf of the users.
With about half of Americans living paycheck-to-paycheck in 2020, appropriately planning for upcoming expenses is more important than ever. While there are a plethora of budgeting tools on the market, many of these budgeting tools are better at helping individuals assess their past spending than they are at providing instant feedback on how individuals are doing compared to their goals, and informing current and future spend decisions. Individuals also tend to forget about upcoming obligations, such as bills, and to anchor on the wrong number when determining how much money they have left, and what kinds of purchases they can afford.
Branch [see page 38 of the 2020 Annual Report] was interested in applying behavioral science to improve upon how a user’s spending, balance, and bills were communicated in budgeting data visualizations. The aim was to make more salient: (1) the amount of money available to spend after upcoming bills were deducted, and (2) the impact of recent spending behavior on money available for the remainder of the pay period. The Common Cents Lab designed a survey to test three different redesigns of the budgeting screen to gauge behavioral intentions, interpretation of new graphics, and relative preferences between redesigns.
The Common Cents Lab tested three budgeting visuals that communicated either good financial standing (i.e., “looks great”) or overspending (i.e., “overdraft risk”). The screens differed in how they communicated a starting amount, as well as how much of the remaining funds were committed to upcoming bills, versus available to spend:
Progress bar: Displayed a big “left to spend” amount at the top (available balance - upcoming bills), as well as the amount the user should have spent by today if they were to spend their paycheck at an even rate across the pay period.
Gas gauge: Displayed a small available balance (full length of bar) and subtracted for the user their upcoming bills to get to their large “left until payday” amount (what the dial points to).
Stacked bar: The full length of the bar represented their full paycheck, and was broken down into what had been spent, what was still in their account but committed to upcoming bills, and what was available to spend (“amount left”).
We had three hypotheses regarding how users would respond to these visuals:
Users would choose to decrease spending more often when shown an “overdraft risk” message.
Users would choose the most expensive decision in a hypothetical scenario less often when shown an “overdraft risk” message.
Users would correctly interpret the features of their assigned budgeting visual.
The experiment was implemented as a Qualtrics survey delivered directly to Branch users through a push notification in the Branch app.
The survey consisted of three parts:
Behavioral Questions: Participants were randomly shown one of the three budgeting visuals displaying either positive feedback (i.e., “looks good”) or negative feedback (i.e., “overdraft risk”), for a total of six conditions. Based on this screen they were asked: (1) how they would respond in a hypothetical scenario where they could purchase food, (2) how and whether they would change their spending if this screen reflected their current financial status, and (3) how this screen affected their perceptions of how they spend their money. There were 1,145 respondents that completed this section and were included in this part of the analysis.
Visual Interpretation: Participants were shown the budgeting visual they had been assigned and were asked to identify the meaning of the different features of the data visualization (e.g., the length of the bar, the fill of the gauge, etc.). There were 1,082 respondents that completed this section and were included in this part of the analysis.
Choosing between Visuals: Participants were shown all three sets of budgeting visuals (progress bar, gas gauge, and stacked bar) and were asked which set of visuals would be the most helpful to see used in the Branch app with their reasoning for their selection. There were 1,043 respondents that completed this section and were included in this part of the analysis.
Very few users responded that they would increase their spending regardless of whether they saw a “looks great” versus or an “overdraft risk” visual. The main trade-off was between choosing to maintain or decrease spending. Users presented with gas gauge or stacked bar visuals showed a significant difference in spending decisions when presented with “looks great” versus “overdraft risk” versions. They were more likely than expected to choose to “spend at the same rate” when presented with a “looks great” version. They also chose to “decrease spending” more than expected when presented with an “overdraft risk” version. This effect is highly significant for the stacked bar (p = 0.0003) and gas gauge (p = 0.0007) budget visuals. Responses for the progress bar users shows a similar trend but did not meet statistical significance (p = 0.1639).
All three visuals helped users curb spending when presented with an “overdraft risk” version versus a “looks great” version. For all three visuals, users said they were significantly more likely to cook dinner than get fast food or Uber Eats when presented with an overdraft risk. This effect is significant for the progress bar (p = 0.007) and gas gauge (p = 0.004), but is most visible when users rely on the stacked bar budget visual (p < 0.0001).
Users also interpreted the stacked bar correctly more often than the other two visuals, with 65% of users correctly interpreting the full length of the bar and 41% correctly interpreting the amount left. One reason for this could be that the bar represented their full paycheck, while the progress bar and gas gauge used the user’s available balance or the length (in days) of a pay period as reference points.
This study demonstrated that, at least when making hypothetical decisions, the amount an individual anchors on does have the ability to affect their spending decisions. The challenge comes in visually representing those anchor amounts in a way that is clear and intuitive, particularly when trying to introduce individuals to new numerical anchors that contrast their existing mental models. The key lessons we learned in testing out concepts aimed at doing so are:
Users needed clearer explanations of what was represented by the “left to spend” amount than was shown in these survey mockups.
Users got the strongest impression that they had been spending too much from the visual that showed them not just how much money they had left, but also the fraction of their paycheck they had already spent down (i.e., the stacked bar visual).
Understanding a rate of spend (the amount the user should have spent by today if they were to spend their paycheck at an even rate across the pay period) required either clearer explanation or a higher level of data visualization literacy than was represented by this user base.