Current UX Work

Week of 10/06/14

Manifest Builder progress

• Marketplace team has decided to move forward with integrating the Manifest Builder into Devhub before completing the Devhub overhaul project. Yay!
• Here is a prototype of the short-term Manifest Builder. This prototype works in the context of the existing Devhub, so it differs a bit from the prototype for the longer-term Devhub project.
• You can find details about the integration here.

Next Steps

• Finalize interfaces for outstanding fields. ETA week of 10/13/14.
• Write content for the App Manifest Checklist.
• Continue working on the longer-term project, specifically on Marketplace Config file and prototype.

Week of 09/22/14

Manifest Builder progress

• Met with Manifest Experts to decide which manifest fields are useful for advanced developers and which are useful for most developers. See the document here.
• Updated prototype of Manifest Builder. Changes reflect decisions made with Manifest Experts.
• Working through flows/pages for submitting and editing an app. See the in-progress documentation for Devhub.

Next Steps

• Review revised Devhub experience with core team.
• Work on Devhub Landing Page - now part of this project.
• Write Checklists for Manifest and Config files.
• Construct Marketplace Config file with developers, Manifest Experts, and review team.
• Design Marketplace Config Builder using what we've learned about the Manifest Builder.

Week of 08/25/14

Progress!

• Met with MDN team to move forward with re-writing instructions.
• Manifest Builder prototype - view it here. We will be running some tests with this prototype to see how it compares to the existing process of creating a Manifest from scratch.
• Working on UX design for the two concepts.

Week of 08/11/14

We've begun 3 areas of work:

• Working with MDN team to improve App Manifest instructions on MDN.
• Expanding on earlier work on Manifest Builder. Manifest Builder is key to the 2 new concepts, but also helps us in the immediate/short-term to combat submission failures.
  • See our thinking about the Manifest Builder here.
  • We are currently exploring interaction models that achieve specific goals we're focused on coming out of the research. We're also exploring best ways to collect the information for optional fields in the Manifest.
  • NOTE that these are WIP interaction models. They are not interaction design. So we're not ready for feedback about things like whether something is a dropdown or a checkbox.
• Design work for the 2 new concepts.

Final recommendations for the MDN here.

References to all our research here.

Week of 07/28/14

This week we begin design on our two concepts: remote repo and dropoff service. We'll base our design work on the findings from our recent user testing. We will also be working with developers to finalize the format and instructions for the Manifest and Config files.

We've completed testing around the Marketplace Config files. Topline findings:

1. Checklist. The Manifest was more difficult than the Config file for developers to produce. We believe that this is because the test instructions for the Config file were presented more explicitly as a "checklist" of stuff the developer needd to put in the Config file. Our recommendation is that we include checklists for both files on the MDN instructions. These checklists to be written as a list of questions, e.g., "When the app launches, what page should it start on? The URL to that page goes in the X field."
2. Required fields. A simple list of required fields at the top of the MDN instructions was very helpful for developers. They referred to it all the time. Some even copied it into their Config file to use as a "checklist."
3. JSON validation. Almost all developers made errors in their files that would result in JSON validation errors. MDN instructions should encourage (and link) to JSON validators before submitting to the Marketplace.
4. Sample file. As with the Manifest file testing, developers wanted a complete sample Config file to work with.
5. Navigation. The test MDN instruction page for the Config file did not include navigation (i.e., there was no table of contents side navigation). This forced developers to slow down to scan/read the content. Given the complexity and difficulty of the content, our reco is to remove the sidenav TOC from the MDN instructions for both files. The page format itself will then support the need to actually read the content. We'll test this hypothesis in our design work.

Config File Test Data + Findings

• Test #2 Observations
• Test #2 Raw Data

Week of 07/21/14

We've completed baseline testing for developer understanding of the current MDN App Manifest instructions. You can find more details in the locations below. Topline findings:

1. Finding strategies. As we would expect, some people skimmed the content looking for words they could pick out that matched their mental models, while others searched for the same thing. Less than 10% used the navigation (sidenav) to find content.
2. File name + format. Less than 10% named the file correctly as manifest.webapp. This is likely to be a major source of manifest validation failure.
3. Mental models. Some of the language we use does not match people's mental models. E.g., people who searched for "translate" were unable to find the locales section.
4. Sample content. Most people copy-pasted the MDN sample content. Many of them did not change the sample content after copying it. This is likely to be a major source of manifest validation failure.

Test Data + Findings

• Test #3 Observations
• Test #3 Raw Data
• Preliminary Recommendations Note: These are preliminary, not final. We used them to re-write the MDN instructions for our next test.

We are currently running a test with revised MDN instructions to determine if better instructions lead to better results. We're also beginning tests on the config file that's part of both new concepts. We'll post the results as they come in.

Week of 07/14/14

Both of our concepts require the developer to complete the App Manifest offline using his local environment and tools. It's critical that the instructions for completeing the App Manifest are correct, clear, and well written. The instructions must also be presented in ways that make it easy for readers (developers) to quickly find and understand information within the instructions.

We ran a series of tests to answer the following question:

1. What difference does content make? For this question, we'll rewrite the App Manifest instructions on App Center to see if clearer/more concise content improves developers' ability to create a succesful App Manifest.

Week of 06/23/14

Following team discussions and conversations with people at App Days, we've decided to pursue two concepts in more detail:

• Devhub as a concierge/dropoff service
• Devhub as a remote repository

In the next few weeks, we'll be doing some user testing to identify areas of focus for each concept.