Player Bug Reporter

Quality Assurance UX design for Ashes of Creation

While I was Senior Quality Analyst for the Systems / Economy Team, I spearheaded improvements to both Debug Tools QA utilized, as well proposing improvements to the Player Facing Bug Reporter.

Intrepid Studios
Quality Assurance Team

Time
2024 - 2026

Tools

  • Confluence for Documentation

  • Jira for Tasks and Bugs

  • Sentry for External Reports

  • Miro for Mockups

  • Testrail for Editor Testcases

Overview

During my time at Intrepid Studios, working on Ashes of Creation, one of the primary tools we provided players to report issues was an in-game Reporter.

This window gave our early access players the ability to report an issue, and we would receive these issues on our internal database manager.

As a member of QA, our job was to spot crucial flow frictions for users and their reporting.

I created two proposals based on player and QA issues with the Reporter.

Table of Contents

  • Reporting a Bug

  • Reporter window

  • Issues with Reporter

  • Proposal 1: Sizing and field changes

  • Proposal 2: “Put a bug on it” Initiative

REPORTING A BUG

In-Game Reporting followed these methods of opening the Reporter

  • Main Menu = Press Esc + click on “Report Issue” to open the Reporter

  • Chat = Press Enter + type “/bug” + press Enter to open the Reporter

External Methods of Reporting an issue

  • Forums = Less used, monitored primarily by community team

  • Discord = Many use this as QA/Community Teams had ability to Reply

Internal Developer Methods

  • Report in Slack Channels = Developer ONLY use, able to be alerted fast, but can clutter communication and cause panic!

  • Many Developers did not use the Reporter when playing, as Slack was faster

THE REPORTER WINDOW

In the Reporter window, we provide players the following fields:

  • Brief Description - Open Field, set character limit to summarize issue

  • Exploit - Toggle a Checkbox if the player perceived an issue as an Exploit

  • Type - List of QA curated Categories (World, Guilds, Storage, etc)

  • Sub-Type - Dropdown field, which would populate based on the selected Category

  • Report Details - Open Field, no Character Limit

  • Include Screenshot - Takes a screenshot ON SUBMIT

  • Submit Bug Report - Button

ISSUES WITH REPORTER

Lack of Context via Internal Database

  • While we had some crucial data for the player, the fields we provide are crude and obscure.

  • Internal terminology for “Type” and “Sub-Type” conflicts with both our in-game terminology and user terminology

  • The result is an external database which becomes CLUTTERED

  • Improper tagging from players provides clutter when we are trying to surface meaningful CLUSTERS

  • We also just lacked other methods to remove excess CLUTTER

Reporter Window Functional issues

  • In order to get to the Main Menu = players had to Press Esc + click on “Report Issue” to open the Reporter

    • Esc also functioned as Closing Windows and De-selecting in-game selections.

    • This was removing context as players would be unintentionally “wiping” their screen.

  • Dropdowns are a pain to cycle through for players, as the user does not know what Sub-types are available until after they select.

  • These Type, and Sub-types, may not include terminology or topic matching player issue context

  • Our screenshots only sent on Submission

  • We provided no way to change, remove, or take a new screenshot

  • Our Reporter window was much larger than it needs to be, consuming screen space for players

Chat Functionality is not surfaced to Players

  • Press Enter + type “/bug” + press Enter to open the Reporter has no indication of functionality for players who aren’t old school MMO' Players.

  • This is not great for New Users

How can we increase contextual accuracy in Reports?

P1: Sizing and field changes

Window

  • Reduce Size of Window to give back screen space to players

  • Provide a Keybind Shortcut for quick launch

  • Two Tabs:

    • Bug

    • Report Player

Fields

  • Replace standard dropdown with a Cycle Previous and Next Dropdown

    • similar to ones found when using Calendars for scheduling)

    • Adding Previous and Next as Buttons, this allows players to speed up their categorization

    • Dropdowns become problematic when you can’t cycle through the list, and then it covers anything below them

    • When cycled, there are Tags below update

    • Again, this allows users to search quickly

    • If we have the time, add Tooltips to give some more info

  • Information Button to lead to player to How to Guide

  • Screenshot taken at Report

  • Add ability to Take or Retake the Screenshot

On Submit

Include Account / Player Snapshot Data for Copy and Pasting to Load Player Configuration

  • Inventory, Gear, currency, etc json files that could be loaded by QA for issue replication

P2: “Put a bug on it”

The proposal above tackles improvements to the window sizing and field functionality to speed up reporting - this improvement is a 50% extreme, and 50% reasonable when it comes to providing both players a means to Report Issues as well as provide Developers with CONTEXT outside backend logging data.

Problem Statement:

  • How can we provide both players and developers with context to where the issue was encountered?

Proposed Solution:

  • Place a “Bug Button” on every primary interface of the game

  • When a bug is reported it does the following:

    • Takes a Screenshot (Which can be modified)

    • Tracks the Game Location (Main Menu, Character Creator, Map, Inventory, etc)

    • Player Account, Player Character coordinates and other in-game details, etc.

    • Automatically provides PLAYERS with context Category with Sub-Categories that are relevant to the location the bug was reported in (Can be changed if need be)

Who is this helping?

  • Our Developers

    • We improve coverage into areas of the game we do not yet provide access to the Reporter

    • This removes - some (not all) Categorization Cluttering in our Internal Report Database

    • Which helps improve Clustering game issues which are occurring

  • Our Players:

    • We remove the terminology and field friction which allow them to spend more thoughtful time on the actual issue, as well as get back their valuable play time

Conclusion

There are better backend data driven methods of gathering the data I proposed above, but this way it can be tied directly to the context of the player, which then helps QA reproduce the issue and clean up clutter and provide better clustering.

And honestly, it is my opinion that any game, be it in Indie, AAA, in Alpha, Beta, and even full released live service games could these.

I was heavily influenced by Sub-Nautica’s GDC Talk about their journey in EA getting reports using their in-game reporter.