Sprint 2: Search and interaction model December 2015

[Research Plan] (#research-plan)
[Conversation Guide] (#conversation-guide)
[Learnings] (#learnings)

##Research Plan

###1. Advanced search, reporting and exporting

Hypotheses we’re testing:

• We believe that building an advanced search that allows users to search by a combination of status, requester, cardholder, vendor, time frame or amount.
• Will result in requesters and approvers feeling confident to find a given request from Pegasus or another system
• We will know we’re right when requesters and approvers can quickly find requests requests with a combination of attributes
• We believe that allowing requesters and approvers to save their searches as reports.
• Will result in requesters and approvers feeling they can quickly recall commonly used views of the data
• We will know we’re right when approvers know how to save and repeat a search
• We believe that allowing users to run reports and export in CSV
• Will result in users being able to use C2 data in other applications
• We will know we’re right when users can quickly export a CSV (and, eventually, import it elsewhere)

Other questions we’re trying to answer:

• Does the search design make sense to people?
• Do people have any kind of link in their mind between reports and search results? Does save as a report make sense to people? Is it confusing to people?
• What attributes are important to people? Less, specific attributes, what are their habits? Trying to combine many different attributes? Or just a few?
• How do users currently locate requests?
• Work habits in locating request. In general, work habits and how C2 fits into it.
• What does a subscription look like?
• For 18F, what types of reports would be valuable to them?
• How much are people interested in just finding one request versus groups of requests?

###2. Interaction model

Hypotheses we’re testing:

• We believe that organizing requests either “Excel style” or “Outlook style”
• Will result in users having a better understanding of what each application’s status is
• We will know we’re right when users can quickly tell what’s next on a given request
• We believe that showing users the status of their requests on the request details page
• Will result in users having a better understanding of the status of their current requests
• We will know we’re right when users can quickly tell which requests have which status

Questions we’re trying to answer:

• Which interaction model is more appealing? Which will help them locate the request that they’re working on? Understand the action they need to take? Understand the status?
• How important is knowing the status of a request?
• What kinds of filters at the top of the table would be valuable?
• Do people use both the buttons along the top and the others? What’s valuable in the title A, B, C, D area?
• What does everyone want for Approved PDFs? Formats to download?
• What buckets might you put requests into?
• What’s most valuable to who on the details page?
• How would people like to be notified if something’s changed before they last looked at it?
• How are people using documents? What are they doing with them? Or downloading?
• How would people go about pinning a request? Is that a valuable feature to have? How would they expect it work? * Where would they expect a pinned request to appear? Would they expect that if there’s a pinned request they get every notification about? Or is it just a placement or filter?

##Conversation Guide ###Introduction###

Thanks so much for agreeing to participate in our usability test. Today I’m going to show you some drawings of new Communicart features. I’ll:

• share my screen with you
• show you a potential new screen,
• give you a situation,
• ask where you would click and show you the screen that would result

We’re gathering feedback on advanced search, reporting and exporting sections. If we have time, we will also talk about the “my requests” screen.

As I ask you my questions, please say your thoughts aloud. Don’t hold back on any criticisms, concerns or ideas.

May I record our conversation?

###1. Advanced search, reporting and exporting

Advanced search:

• When you’re looking for a particular request in Communicart, what are you often in the midst of doing? What are you trying to accomplish? What other tools are you using?

Randomize order:

• If you want to find a pending request from a particular requestor [for ops people] or particular vendor [non ops people], what would you do?
• [If doesn’t come up previously] What fields might be most valuable to you?

Reporting / exporting:

• If you are looking for the same information repeatedly, where might you look to figure out ways of doing that faster? [Steer them towards reporting dashboard, if they don’t go there on the first try.]
• What reports might be most valuable to you?

Export and scheduling:

• If you wanted to export Communicart data to use in another application, what would you do?
• [If doesn’t come up previously] How would you like to receive recurring information?

###2. Interaction model

Interaction model 1:

This is a possible way Communicart might be organized in the future

• How would you go about finding a request that you’re working on?
• For this current request, who needs to take the next action? What do they need to do?

Interaction model 2:

This is a possible way Communicart might be organized in the future

• How would you go about finding a request that you’re working on?
• For this current request, who needs to take the next action? What are they doing?
• How would you go about finding all the approved requests?
• Where would you go to see spending by team?
• How might you want to organize your requests?
• When you click the download button, what do you expect to see?
• What kinds of filters at the top of the table would be valuable?

##Learnings

###Suggestions will apply to NCR users as well as 18F users:

1. People have existing, simple ways of searching for requests; they don’t notice the new advanced search bar. People scroll or use the browser’s search functionality to find a particular request. Suggestion: If we want people to use the advanced search, we may need to make it more prominent.

2. People don’t think of the “My Dashboard” page as a way to quickly recall frequent searches. When I asked people where they would go to repeat a search, they usually say they would look back in the advanced search area. Suggestion: Pursue previously-discussed “save this search” functionality in the advanced search.

3. People do think of the “My Dashboard” section as a place to find reports. When I asked people where they would find reports, they did go to the “My Dashboard” page. However, they often commented that it might be better named “Reports.” Suggestion: Rename the “My Dashboard” page to “Reports”

4. Neither interaction model was a clear winner. It took a few seconds for people to figure out the next step in the “Outlook style” model (model 1), but the hazard triangle in the status column was hard to understand in the “Excel style” model. Suggestion: Do more research on the interaction model than we had time to do.

###Findings that bear further research:

5. People think “reports” are file downloads. At the end of the report creation process, every participant expected to download a file or receive an email. None expected the tabular page that appeared. Suggestion: As most people who run complex reports just want to download data, consider making “Reports” all about exporting data. See the next finding.

6. There are two kinds of 18F users: people who want simple filters and people who want to export the data in sum to do advanced analysis in a spreadsheet. For the first kind of users, the advanced search is overkill — they simply need to filter by request status and, sometimes, request type. For second kind, the reporting interface is unnecessary — they simply want a dump of all the requests in a CSV or spreadsheet. Suggestion: Consider adding simple filters to the “My Requests” list and making “Reports” all about exporting data.

7. Many people liked the idea of having reports regularly emailed to them. Many would expect to receive a CSV attached to the email. Suggestion: Keep the ability to subscribe to reports, but make them.