Bug Triage
Bugs pour in. Show-stoppers, performance issues, art and design flaws, and a few recommendations top off a growing list of bugs that threaten to derail your plans. You know, like every day in game development. Can you fix them all? Should you fix them all? Which is the highest priority? What is the fix and who should fix it? These are the questions best handled by a bug triage process, like new patients showing up at an ER. Some should go to surgery; some can wait in the lobby; and some should just go home with an aspirin and sleep it off. But before we discuss the triage process, let's first go over the maladies and mayhem that can come from improperly triaged bugs.
Malady: Not a Bug!
This happens all the time, but who's the best person to respond to this? Give it to the developer and it wastes their time at a minimum. It could also annoy them and generate some unnecessary back-and-forth debate with QA. Worse case it steers them off course as they try to correct something that isn't truly a bug but perhaps a subjective misinterpretation or poorly run testcase by a tester.
Mayhem: Recommendations Posing as Bugs
Everyone's a critic. You see this a lot with art and design bugs. Testers run out of ammo for a weapon they like to use, and they throw it in as a bug. They don't like the way something looks or don't understand the LOD system and they'll write up an art issue. It could be they're right, but should the tester be art directing the game?
Malady: Tester Bugs
A bunch of testers in a death match get together, don't shoot at each other, take the time to each find the most performance-intensive grenade in the game and then all throw it at once at the same location, causing a 3 second drop to 5 FPS which the game recovers from fine. Now your producer sees this and decides to remove the grenade from the game because it can be a cert issue. True story! Thank you testers! You were so proud of your accomplishment, and your reward is a less fun game. These so-called "tester bugs" are what no player in their right mind would do and often can tolerate if they do occur. They should not be fixed.
Mayhem: Unverified Bugs Out of Left Field
Maybe these bugs are coming from your publisher or someone outside the core team. They could be on an old build, testing on unsupported hardware or playing over a VPN or similarly hampered network connection. These could be legit bugs, but it's doubtful if no one else is experiencing the issue. This deserves some further testing and verification before going to a developer.
Malady: Wrong Assignee and Wrong Priority
This probably happens more often than not with little harm. However, if there are a lot of other issues on peoples' plates, they may let it sit there for a while before realizing it isn't their bug. Equally harmful, the priority could be wrong so more important fixes may experience delays.
Rules for Bug Triage
To properly triage bugs, the first thing to establish are some bug process rules and then try to reinforce them with database workflow and field rules.
Recommended by LinkedIn
Setting up the Meeting
In a pinch, a producer, development director or embedded QA lead could just go ahead and assign a bug. As could a discipline lead if they are entering bugs for their team members to fix. But bugs that cross disciplines or come from QA under normal circumstances should ideally be triaged in a group setting.
You need to schedule a recurring bug triage meeting. Depending largely on the pace of bugs coming in, the triage could happen daily but no less than once or twice a week to remain relevant. If there are only a few bugs to go over, no one will complain about a short meeting.
Attendees should include:
It is NOT helpful to have many attendees. It can really slow down the meeting and waste peoples' time. One discipline representative is enough. Be mindful of other QA and developers attempting to infiltrate this meeting. While this meeting is important, it is not an all-hands situation. Nor does the pro of information access outweigh the con of distracting people from their work.
During the Meeting
The QA rep or producer can facilitate the meeting. They should bring up each bug, read the summary and description and show the screen shots and video attachments. Then ask, "Is this a bug?" then "Whose is it?"
The discipline leads should then say if it's their bug to categorize it, i.e. code, art or design. If clarifications or instructions are needed, the bug description and summary should be edited and further comments made. These changes can largely be dictated by the appropriate discipline lead to the meeting facilitator as they might acknowledge a bug but have specific ideas on what it is and how to fix it.
The facilitator should then also ask what the priority should be. Whether you are stack ranking or using the "A, B, C" approach, this is the time to decide where this bug fits in compared to all the other issues.
There may even be some bugs you decide not to fix. All the mayhem and malady issues previously mentioned could be cause to waive or similarly close a bug or kick it back to QA for more testing. Put in the notes the reasoning for retesting, closing or waiving a bug and per whom, so there's less debate coming back from QA. QA needs and deserves closure for the issues they bothered to type up.
Next Week
We will start looking at producing from the publishing perspective with our first money issue.
How many producers of learning video games are successful without a dedicated debugging programmer, and what is the job called?