Things to Consider When Setting Up A UX Diary Study

What is a UX Diary Study?  

UX diary studies are a qualitative research method that asks participants to record their perceptions, thoughts, opinions, and actions to a particular prompt. They are typically longitudinal, lasting a few days to a few weeks or even months. In simple terms, UX diary studies aim to gather data about users while using a particular product or performing a specific task to make design improvements that increase customer satisfaction. 

Setting Objectives 

Study objectives will serve as your compass throughout the project. You can refer to them at each step of the research process to ensure you're gathering the right type and amount of information. If you are working within a team on a UX diary study, it is important to consider everyone's input. Typically for a study, 3-5 objectives are standard. 

When writing your objectives, think of how the idea for the study came about. Your goals must be clear, concise, and achievable. Here are some question prompts that might help you create impactful objectives: 

• What is our current customer experience like when using our product(s)?
• Which parts of the process are easy/difficult for customers? 
• Why are customers choosing one product over another? 
• What are the 3-5 things we must know about how customers use this product to serve them better?

You will have to customize these questions to fit your specific project goals, but they should help get a conversation started. Once you have written some draft objectives, it's time to iterate and refine them until they meet the below criteria:

Achievable 

Create achievable objectives that can be satisfied through your chosen methodology, such as a UX diary study. You'll be able to assess if your objectives are feasible in the pre-planning phase. For UX diary studies, you'll want to consider if your target audience can answer or complete the tasks presented and if you are conducting the correct number of interviews to fully understand the questions at hand. 

Specific 

Vague objectives are the bane of every UX study's existence. They muddy the waters and make it challenging to inspire action. For example, if you are getting negative feedback on one of your products, instead of an objective saying, 'uncover negative customer experiences,' you would specify which product is causing problems. A specific objective would read more like this: 'uncover the causes of negative customer feedback during the online booking process.

Measurable 

Since UX studies generally aim to collect qualitative and anecdotal data, you must consider the appropriate metrics for your research. In some cases, you will use frequency counts or attitudinal questions. These are great tools to measure the prevalence of a particular issue or the thoughts and feelings of the participant while completing a task. For example, you might want to know how often a participant is experiencing negative emotions regarding your product. 

Timeline 

The complexity of your study will depend on how long you want to collect data. Generally, UX diary studies span from a few days to several weeks. You and your team will need to decide how long you think it will take to gather enough information to meet the project's objectives. It's also essential to keep the fielding time concise so you don't lose participants' interest. You can mitigate drop-off by adequately setting expectations during recruiting and compensating participants appropriately. 

Selecting a Target Audience 

Setting target audience criteria is critical to the success of your UX diary study. This is where you want to be extremely specific about your objectives and determine how your target audience will help or hurt your chances of meeting them. 

Screening Questions 

Screener questions should be used to determine who does and does not qualify for your diary study. If you are using a database of existing customers, you should have a lot of the information needed to qualify respondents already available. If you want to include potential customers, you should be a lot more detailed and specific with your screener questions. 

When writing screener questions, you don't want to make it obvious which answer options will be qualified. Poor screener questions can lead to unusable results. For example, if you want to include people who purchase organic foods in your study, then you would not want to ask, 'do you purchase organic foods?' instead, you should ask, ‘when was the last time you purchased organic foods.' The latter option is more discreet and will ensure you include only qualified respondents. 

How Many Audiences 

Companies will often have multiple target audiences, segments, or personas. When conducting a UX diary study, you'll likely want feedback from multiple, if not all, of these different customer groups. It's standard to have at least 5 participants per target audience. This ensures consistent themes and hopefully some differences in feedback from each group. 

Creating Your Prompts

Map to Objectives 

As previously mentioned, your objectives serve as the compass for your study, so you'll want to refer to them often to stay on track. One trick is putting your objectives into a table and writing down prompt ideas for each. This way, you can trace each question to a specific objective and desired outcome to keep your study concise and focused. 

Here is a sample table that may be helpful to map objectives to prompts. 

• Objective
• Prompt
• Outcome

Test Internally

After mapping out your prompts and objectives, you will need to test your prompts internally. Set aside a few hours with your team to do trial runs of the study. At this point, you will want to show your test participants precisely what the experience will look like. If you are using an online platform, like EthOS, you will want to have all of your prompts programmed into the tool. 

During the testing phase, you want to ensure that:

• Respondents will be able to answer the questions how you intended 
• The flow of the questions makes sense 
• The questions are easy to read and understand
• You have covered all of your objectives thoroughly
• Each question is displayed correctly and with the appropriate media (if applicable)

When testing your remote diary study prompts amongst your team members, take notes on how the questions are answered, any further questions you had, or anywhere there are obstacles for the respondent. Of course, when you're using a remote platform like EthOS, you can easily create follow-up tasks for respondents. 

Iterate 

Finally, once you have mapped your objectives and done internal testing, it's time to iterate. During this phase of UX diary study preparation, your team will make the final edits to your objectives and prompts to ensure your research question will be answered when it comes time to field. 

Ready to Field 

Finally, the time has come. After a thorough and thoughtful planning period, you will be ready to field your study. All the prep work will be worth it, as it will help ensure you have a smooth fielding experience. 

Consider using a remote diary study tool like EthOS, as it will alleviate many of the logistical challenges usually associated with diary studies since the platform allows you to edit to prompts, set follow-ups, and track the study's progress all from one dashboard. 

UX diary studies are a powerful tool for taking deep dives and collecting detailed information about how your customers use your products, leading to actionable and impactful results.