Agile QA

Loading ...
jarmenia
Technology & Engineering
<p>A Practical Guide to implementing Agile 
QA process on Scrum Projects
Agile QA
Syed Rayhan
Co-founder, Code71, Inc.
Contact: srayhan@code71.com
Blog: http://blog.syedrayhan.com
Company: http://www.code71.com
Product: http://www.scrumpad.com
2
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
My Background
Expertise
Career

Iterative incremental development
 Technology planning and architecture
 On-shore/Off-shore  software development using 
Agile/Scrum
Interes ts
 Co-founder, Code71, Inc.
 13+ years of total experience
 Co-author of “Enterprise Java with UML”
 Cultural aspect of self-organizing team
 Scrum for projects delivered remotely
 Agile engineering practices
3
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section  2
Section  4
Section  3
A Case Study 
Holistic View of QA
Section  5
Introduction
Section  1
Q&A
Section  6
4
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
What to Expect 
Focus
Context
 Build common base of understanding
 Develop a set of gudielines- process, roles, and team composition
 Address typical questions asked
Key 
Takeaways
 How to perform QA on an Agile/Scrum project
 Agile/QA best practices
 Teams and organizations are adopting Agile/Scrum  
 Teams struggle with making the transition from waterfall to 
Agile/Scrum 
5
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section  2
Section  4
Section  3
A Case Study 
Holistic View of QA
Section  5
Introduction
Section  1
Q&A
Section  6
6
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
The challenges?
 Is QA part of the development team?  
 Can we fit QA in the same iteration as development?  
 How can we scale Agile QA?  
 Who does QA?  
 Does QA costs more in Agile as product seems to change from 
sprint to sprint?  
 Do we need “test plan”?  
 Are story acceptance tests enough? 
 
 When do we know testing is done?  
 Who defines test cases?  
 Do we need to track bugs?  
7
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
What is QA (Quality Assurance)? 
To ensure Software is working right
How?
Test, Test, Test
8
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Types of Testing?
Unit Testing 
Integration Testing 
Functional Testing 
System Testing 
Regression Testing 
Acceptance Testing 
Load Testing 
Smoke Testing 
Black 
Box
White 
Box
9
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Who Performs What?
Unit Testing 
Integration Testing 
System Testing 
Regression Testing 
Acceptance Testing 
Load Testing 
Smoke Testing 
What? 
Who? 
When? 
Automation? 
Developer 
Developer 
Tester 
Developer/Tester 
Client/Users 
Tester/Support Engineer 
Performance Engineer 
Coding 
Coding 
Test
Build/Test
Deployment/
Delivery
Deployment
Deployment
Always
Always
Possible
Possible
Possible
Possible
Always
10
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Right tools for right tests?
Unit Testing 
Integration Testing 
System Testing 
Regression Testing 
Acceptance Testing 
Load Testing 
Smoke Testing 
Test 
Tool 
NUnit, JUnit, Mock, DBUnit 
Unit test tools, HttpUnit, SoapUI, 
RESTClient 
JMeter, Httperf 
FIT, FitNesse 
Selenium, Fit, WET, Watir, WatiN 
Unit test tools, System test tools 
Regression test tools 
11
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
What is missing?
Assumptions
1. Right Spec
2. Right Design
Reality?
Measures
Spec review
Design review
Spec 
Design 
Code review
Code 
3. Right amount of Tests
Test Coverage
Test 
Assumptions are farther from truth
4. Right Tests
12
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Team Composition?
We will primarily focus on single-team model for our discussion
1
Dev
Test
2
3
1
2
3
iteration
region
Prod
1
2
…
1
Dev
Test
2
3
1
2
3
iteration
region
Prod
1
2
…
Out-of-Cycle
In-Cycle
2
1
13
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Team Dynamics
Product owners
Developers
backlog
Acceptance tests
Additional tests
Team review
Testers
14
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
QG#4
QG#3
QG#2
Quality Funnel
QG#1
QG#5
bugs
bugs
Backlog
Review
Design
Review
Unit & Int.
Test,
CI
Code
Review
System
Test
15
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section  2
Section  4
Section  3
A Case Study 
Holistic View of QA
Section  5
Introduction
Section  1
Q&A
Section  6
16
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Test Coverage
Tests coverage metrics can tell you what code is not tested 
“A measure of the proportion of a program 
exercised by a test suite, usually expressed as 
a percentage.”
• Function coverage
• Path coverage
• Statement coverage
Types of coverage
Definition
Measure
Usually expressed as a percentage
17
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Continuous Integration as the Glue
Monitor 
Source Code
Build
Regression
Test
Test 
Coverage
Report
Continuous Integration is Continuous QA
18
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Test Case Prioritization
high
high
low
low
1
2
Frequency of use
Risk of having bugs
3
4
hot
warm
normal
cold
19
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Inspect and Adapt through QA Lens
Five whys of root cause analysis
Prioritize bugs over stories
Log bugs found by testers
Quality Metrics
 Defect Rate           Bug count per iteration
 Defect Density      Bug count per module
                                   Bug count per function point
20
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
How to Scale?
Presentation
Business
Data
Service#1
Service#2
Service#3
Application#1
Application#2
SOA
21
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section  2
Section  4
Section  3
A Case Study 
Holistic View of QA
Section  5
Introduction
Section  1
Q&A
Section  6
22
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
A Case Study
Project
A large enterprise system that includes technologies like 
ASP.Net, BizTalk, Workflow, Scanning, SQL Server, Data 
Warehouse, and Mainframe 
Team
2 product owners, 1 scrum master, 1 architect, 5 developers, 
1 QA tester
Sprint
2 weeks
QA Process
Refine 
scope, 
acceptance 
test cases
Identify UI 
elements, 
test data
QA schedule
Write test scripts
test data
Day 1, 2
Day 3, 4
Day 5-8
Day 9
Day 10
Final
Test
&
Fix
Demo
&
Acceptanc
e Test
23
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section  2
Section  4
Section  3
A Case Study 
Holistic View of QA
Section  5
Introduction
Section  1
Q&A
Section  6
24
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Recap

“In-cycle” QA is critical to the success of a project  
 System testing is not the only “quality gate,” it includes all types of testing 
and reviews  
 Test automation is critical to “in-cycle” QA
 Target at least 90% test coverage

“Continuous Integration” is “Continuous QA”
 Prioritize test cases based on risk and frequency of usage
25
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Recap contd.
 All known bugs should be fixed first
 Right size story with well-thought out acceptance tests improves quality

Include all “Quality Gates” as part of definition of “Done” 
 Analyze each bug to understand where (Quality Gate) it should have been 
caught and improve (Inspect and adapt) 
 QA is not a designated person’s responsibility, it is a team’s responsibility 
(self-organizing team)
26
www.Code71.com
www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Q&A
“QA is not an act, but a habit”
Contact: srayhan@code71.com
Blog: http://blog.syedrayhan.com
Company: http://www.code71.com
Product: http://www.scrumpad.com
“QA is making sure right software works right”
</p>