A Crash Course in Software Quality Assurance

Years and years ago, I was once a tester at Electronic Arts Canada. As much as I did not enjoy working in their dungeon, making peanuts for an income, having 4 bottom level managers on power trips, and trying to navigate insidious office politics while being a sort of Aspergers'-ish physics graduate with a pretty high profile role in academia (for my age) already under my belt, I do have to say that the QA Tester training there was really excellent and left a lasting impact on my career, still to this day.

Did I need that life experience to get me here? Probably not.

But that said, I learned a lot and I've noticed over the years being an iOS Developer in Germany, what consistently is done poorly in pretty much every job I've had here is QA. There generally is no QA department and on many occasions I have found myself doing what I will present below, the Crash Course in Software QA. It made sense for me to finally write it down and share it, may it be of benefit to you.

So here you go!

A Crash Course in Quality Assurance

1. A Bug Report

A bug represents any observed behaviour in the app that does not correspond to its specification, or more plainly, where the app does not behave as you would expect, or simply does not offer a good user experience. These are all valid reasons to file a bug report.

Essentially a bug has 2 characteristics: it's severity and its reproducibility.

I) Reproducibility

Given the steps I would take in the application to try to reproduce the issue, how causal is that? i.e. 100% means it will happen every time you follow those steps with the given starting conditions (such as making sure it's on a specific device type, or OS version, or user credentials when logged in)

Generally bugs are 100% reproducible OR they are something less than that, which generally makes them harder to track. 100% reproducible bugs are the easiest for the developers to fix.

II) Severity

Severity has some pretty clear classifications. When the steps to reproduce lead to the bug happening, the result can be:

Critical: The app crashes, or freezes or becomes unresponsive.

Major: The app becomes unusable, or significant parts of the app become unavailable and essentially the only way to get around the issue is to restart the app, IF that's possible at all.

Minor: A cosmetic issue, mostly. Maybe text is drawn where you can't see it, or there appear to be 'visual glitches', or other layout problems.

Generally, critical and major bugs are must fix issues, and minors are should fix but can also be deferred to a later date / release version.

2. Writing Bug Reports

An understanding of Severity and Reproducibility are all you really need to write a good bug report. When you write a bug report, you want to include:

  • Build number it was found on, iOS Version, device type
  • Steps to reproduce (as much as you can / know, even if you're not sure if that's relevant.)
  • An estimate of reproducibility
  • A classification of severity

When writing the steps to reproduce, put yourself in the shoes of the person reading your bug report. Keep the steps short and in a bullet-point list. Describe what they should currently see on screen, then tell them what to interact with to get to the next step. (Over time, you and your developers will develop a short hand, such a "go to the XYZ screen", and not "After the app launches, (then a long series of steps to get them to the XYZ screen)".

Over time, as you gain experience with the software itself, and your co-workers, your bug reports will improve / be more efficient. That's normal. Also listen to the advice from the developers you are supporting as ultimately they are the consumers of your work.

3. Develop an End-user Test Suite

  • For any new release, you want to have a series of end-user scenarios you can have in a sort of checklist, that you will go through and verify. The goal is that we release software that we can confidently say is robust / stable.

  • In most cases, QA up-until-now has been very ad hoc. "Use the app and see what you see, report what you can". This usually does not find the edge cases at all.

  • This test suite will be some sort of document that gets added to, updated, modified, etc. as your time on a project continues. You're generally free to create that yourself if the company doesn't have anything else in place, and you should consider having such a thing in addition to your bug reporting tool, such as Jira, or similar. It's good to have your checklist all in one place, and you can even use Excel / Google Sheets for this purpose too.

4. Mentality

Your job is essentially:

i) To verify the app works as expected on all supported device types and OS versions.

ii) To not think in terms of "the happy path", but actually "how can I make this app fail?" ii) Again, your job is to find situations or scenarios that will fail and/or the developer hasn't considered with their implementation

iii) To support developers with any issue they are tracking / need more information on.

Your mentality should be one of a person who is enthusiastic about your role, and one where you enjoy helping people, and being part of a larger purpose. These mental characteristics go a very long way in terms of you being effective in your role, and other people wanting to work with you. In the end, your job is reducing the complexity of bugfixing for the developers, and you are ultimately assisting people. So your mentality will heavily influence if you are considered to be good at assisting the developers ensure their work is the best it can be.