This year at CodeStock I attended a talk by Tara Nicholson on the subject of quality assurance. Several of my colleagues wanted access to my notes on the talk and I thought this would be the best forum for those notes. The slides from Tara’s presentation can be found here.
The following are my notes:
“Software Quality is the degree to which written code meets product requirements, has value, and is maintainable.”
Where do bugs come from?
- Things change.
- Lack of understanding of requirements. (Tara stated that she thinks that this is the #1 source of bugs in software)
Think about software quality early. The cost at each stage of the development process increases exponentially.
- The agile testing quadrants. (Source: Lisa Crispin – http://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/)
- How? Create habits that routinely draw out things you have not thought of.
- Make use of quick cards – a card each member of the development team carries that has a list of the elements, integrations, and dependancies to think about related to QA during the development process.
- The “Grocery List” – A list of all features and requirements that were agreed upon by the business, development team, and QA team have agreed on. Note: some may not make it to production.
- Do you need a dedicated test team? It may not always be feasible, but it certainly is optimal.
Load and Performance Monitoring
- Server vs Page performance – can the server handle the load vs is the app performant from the users point of view
- Tools used at scripts
- Load Runner – runs load tests on the server in a virtual(non-production) environment.
- jMeter – back-end profiler
- Webpagetest – opensource front-end profiler.
- Speedcurve – for pay app from Webpagetest that digests the report from Webpagetest and make optimization suggestions.
- NewRelic – Infrastructure performance tester.
- What are the costs of supporting compatibility (what does it cost to run on IE8?)
- Must consider the platform (Win7, linux), carrier (Verison, AT&T), and client (Firefox, IE)
- Who owns the decision on compatibility? Do we have to support IE8 because the CEO’s son won’t use anything else?
- Plan early for compatibility and create a strategy to support it.
- If you decide not to support something, highly consider partial support. (i.e. works in IE8, but not optimally)
- Know your traffic/demographics – there is no reason to support Opera if no one visits your site using it.
- Automate tests – Scripps uses JBehave for BDD
- Small tests – unit tests that rarely change. <- most tests here.
- Medium tests – integration tests that change occasionally.
- Large tests – UI/Integration tests that change frequently. (i.e. user authentication works) <- fewest tests here
- Release Automation – Continuous integration, nuff said.
- Infrastructure Automation
- Automate content updates.
- Lisa Crispin – http://lisacrispin.com
- James Back – htte://www.satisfice.com/blog/
- Matt Heusser – http://xndev.com/creative-chaos/
- BBST – http://www.testingeducation.org/BBST/