– Performed by Android Team, October 2021

– Phabricator: T293344 + T294078

Contents

1. Recommendations
2. Problems
3. Participants
4. Analysis of tasks (Robin)
5. Analysis of tasks (Jazmin)
6. Analysis of follow up questions

1. Recommendations

1. Discoverability of article talk pages
   • Accessing talk pages needs to be more obvious, the current way to access a talk page via overflow menu or footer is not enough from a discoverability angle. It’s going to be interesting to hear talk page access preferences from experienced users (question #5)
2. Performance
   • Users are used to instant publication of messages. Why does it take so long to publish new topics and reply on talk pages? What can we do to improve the speed? (this problem is visible in the Japanese usability test). We need to investigate on it.
   • A note on perceived performance: can we take the user to the next screen with and indicate its publication or error later to keep users in the flow.
3. Understanding of both article and user talk pages
   • The terminology “talk pages“ is not self explanatory. People were looking at in the tests, even reading it out loud and weren’t associating it with discussions. Is there an opportunity to use more newcomer friendly language in the Android app?
   • We need to better convey that a discussion page is about conversations, e.g. with a more conversational design (see “More design related observations”)
   • To further improve the understanding, consider elements of onboarding to talk pages, e.g. with tooltips once we settled on the access point of talk pages.
4. Disorientation when tapping items on notifications home
   • Wikimedia Commons etc. notifications
     • Implement native talk pages for Wikimedia Commons.
     • The fact that certain notification types take the user outside app with a different design was a source of confusion for almost all participants.
   • “Left a message on your talk page” notifications
     • Did not take users to the corresponding message but ****to the overview. This was confusing to the participants as they expected a dedicated message. Please investigate on why this is happening. For details, watch the French test video (task #22).
   • Milestone notifications
     • Milestone notifications take users to the corresponding article, article or user talk page. How can this be optimized? There is no indication after tapping a milestone notification, this is unexpected. What can we do to improve this? Please watch the French test video to see it in action (task #17, 15:35min).
5. Understanding the diff view
   • Participants had difficulties understanding the diff view
   • Participants had difficulties to identify the changes that have been made in the diff view
   • Participants were confused that certain diff screens take users outside the app
   • ❓What can we do to improve this?
6. More design related observations
   • Optimize design/display of the snackbar, edit history info and FAB button. When these three elements are displayed all together, it can potentially confuse users.
   • Read/unread consistency (notifications + talk pages)
     • Unread/read indication consistency (blue dot, treatment of type)
     • Allow users to manually mark conversations as read/unread
   • Design article and talk page list view in a more conversational way. It was unclear to participants what this page is all about. Investigate on what elements convey a space for active discussions (e.g. forum)
   • Diff screen view
     • Optimize clarity on diff screen, e.g. use strike through for deleted parts. Colors could not be identified by some participants.
     • Optimize visual treatment of summary of the diff screen as doesn’t stand out enough
   • Keyboard wasn’t always visible when tapping “Reply”, e.g. for the French participant. Make sure it’s available right aw
   • One user requested a “more modern UI“ in the test, I think that summarizes the impression I had in the test. The article and user talk pages list and subject views need refinement re: information density and visual design.

2. Problems (see slide deck)

1. Discoverability of article talk pages
   • Participants had difficulties to find and access article talk pages via overflow menu or footer
   • Most participants tapped the article edit pencil first to access talk pages and expected it there
   • ❓What can we do to improve discoverability and of article talk pages?
2. Performance
   • Participants perceived the performance of posting and replying to messages as slow
   • A participant from Japan compared the performance to other services they use and was disappointed
   • Users are used to instant publication of messages
   • ❓What can we do to improve the current (perceived) performance?
3. Understanding of both article and user talk pages
   • Participants had difficulties associating “talk pages” with article discussions
   • Participants were looking at an article talk page, were even reading it out loud, but still weren’t sure about its purpose
   • Participants did not understand difference between article and user talk page or how user talk pages work
   • ❓What can we do to improve understanding of talk pages? Do we need to make sure users comprehend the difference between article and user talk pages or are these just “spaces” on Wikipedia to discuss?