You are here: Ultra Guide>Testing Web>SoftwareTestingISTQBNotes>LearningObjectivesISTQBFoundationAgile (07 Apr 2021, DanielZrymiak)Edit AttachTags

ISTQB Foundation Agile Learning Objectives


Learning Objectives for Agile Software Development

Summary Points

1.1 The Fundamentals of Agile Software Development

FA-1.1.1 (K1) Recall the basic concept of Agile software development based on the Agile Manifesto

Agile Manifesto has four statements of values
- Individuals and Interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan

Agile Manifesto has 12 principles:
- Highest priority to satisfy the customer through early and continuous delivery of valuable software
- Welcome changing requirements to harness change for competitive advantage
- Deliver working software frequently, to short timescales
- Business people and developers work together daily
- Build projects around motivated individuals, trusting them to get the job done
- Use face-to-face conversations as the most efficient and effective communication method
- Working software is the primary measure of progress.
- Agile processes promote sustainable development for sponsors, developers, and users to maintain a constant pace.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity - maximizing the work not done - is essential
- The best architectures, requirements, and designs emerge from self-organizing teams
- At regular intervals, teams reflect on how to be more effective; tuning and adjusting behavior accordingly.

FA-1.1.2 (K2) Understand the advantages of the whole-team approach

The whole-team approach involves everyone with the knowledge and skills necessary to ensure project success, representing customer and stakeholders.
Team size should be 3-9 people, ideally co-located, and functioning with daily stand-up meetings.
Benefits of the whole team approach include:
- Enhancing communication and collaboration
- Enabling various skill sets for the benefit of the project
- Making quality everyone's responsibility.

Testers work closely with developers and business representatives to ensure desired quality levels are achieved. (Power of 3)
Acceptance tests (business reps.), testing strategy and test automation approaches (dev) are jointly prepared.
Testing knowledge can be transferred and extended, and the whole team is involved in consultations of product features.

FA-1.1.3 (K2) Understand the benefits of early and frequent feedback

Agile projects have short iterations, necessitating early and continuous feedback from continuous integration.
From early and frequent feedback, Agile teams can incorporate new changes and focus on the features with the highest value or associated risk.
The capability of the team is transparent by iteration, showing obstacles and areas of potential improvement.

Benefits of early and frequent feedback include:
- Avoiding requirements misunderstandings, being more expensive to fix later in the development cycle.
- Clarifying customer feature requests, then making these available for customer use early.
– Discovering (via continuous integration), isolating, and resolving quality problems early.
- Providing Agile team with information on productivity and ability to deliver.
- Promoting consistent project momentum.

1.2 Aspects of Agile Approaches

FA-1.2.1 (K1) Recall Agile software development approaches

Principles include humanity, economics, mutual benefit, self-similarity, improvement, diversity, reflection, flow, opportunity, redundancy, failure, quality, baby steps, and accepted responsibility.\nXP has 13 primary practices: sit together, whole team, informative workspace, energized work, pair programming, stories, weekly cycle, quarterly cycle, slack, ten-minute build, continuous integration, test first programming, and incremental design."}'>Three Agile approaches: Extreme Programming (XP), Scrum, and Kanban.
XP, developed by Kent Beck, embraces communication, simplicity, feedback, courage, and respect. Principles include humanity, economics, mutual benefit, self-similarity, improvement, diversity, reflection, flow, opportunity, redundancy, failure, quality, baby steps, and accepted responsibility.
XP has 13 primary practices: sit together, whole team, informative workspace, energized work, pair programming, stories, weekly cycle, quarterly cycle, slack, ten-minute build, continuous integration, test first programming, and incremental design.

Also used to enforce meeting start and end times.\n- Transparency: team updates status on a daily basis at the daily scrum, making progress and results available to all interested parties.\n\nScrum defines three roles:\n- Scrum Master - ensures that Scrum practices and rules are implemented and followed. A coach\n- Product Owner - represents the customer and generates, maintains, and prioritizes the prooduct backlog.\n- Development Team - self-organized, cross-functional team which develops and tests the product.\n\nScrum does not dictate any software development techniques, and does not provide guidance on how testing has to be done in a Scrum project.\n"}'>Scrum is an Agile Management framework with the following:
- Sprint: 2-4 week iterations
- Product Increment: potentially releasable/shippable product
- Product Backlog: Prioritized list of planned product items
- Sprint Backlog: highest priority items from the product backlog to be realized within the sprint
- Definition of Done: appropriate criteria for sprint completion
- Timeboxing: tasks, requirements, and features the team expects to finish within the sprint. Also used to enforce meeting start and end times.
- Transparency: team updates status on a daily basis at the daily scrum, making progress and results available to all interested parties.

Scrum defines three roles:
- Scrum Master - ensures that Scrum practices and rules are implemented and followed. A coach
- Product Owner - represents the customer and generates, maintains, and prioritizes the prooduct backlog.
- Development Team - self-organized, cross-functional team which develops and tests the product.

Scrum does not dictate any software development techniques, and does not provide guidance on how testing has to be done in a Scrum project.

Tasks are symbolized by tickets moving from left to right across the board through the stations,.\n- Work-In-Progress Limit: Maximum number of tasks is permitted for a station or Kanban board. Workers pull tickets from a predecessor station.\n- Lead Time: Kanban optimizes the flow of tasks by minimizing the lead time for the complete value stream.\n\nTransparency is achieved by showing the content and progress of tasks. Tasks not yet scheduled are waiting on a backlog.\nIterations or sprints are optional in Kanban. Deliverables are released by item, rather than as part of a sprint. Timeboxing is optional."}'>Kanban is an approach which visualizes and optimizes the flow of work within a value-added chain, using three instruments:
- Kanban Board: each column shows a station of related activities (i.e. dev, testing). Tasks are symbolized by tickets moving from left to right across the board through the stations,.
- Work-In-Progress Limit: Maximum number of tasks is permitted for a station or Kanban board. Workers pull tickets from a predecessor station.
- Lead Time: Kanban optimizes the flow of tasks by minimizing the lead time for the complete value stream.

Transparency is achieved by showing the content and progress of tasks. Tasks not yet scheduled are waiting on a backlog.
Iterations or sprints are optional in Kanban. Deliverables are released by item, rather than as part of a sprint. Timeboxing is optional.

FA-1.2.2 (K3) Write testable user stories in collaboration with developers and business representatives

This is used for the product backlog\n- Conversation: Explains how the software will be used and spans release-planning to scheduled release.\n- Confirmation: Acceptance criteria are used to confirm that the story is done; and requires positive and negative tests. Defined acceptance criteria should be tested and shown to be satisfied."}'>Agile uses frequent informal reviews involving developers, testers, and business representatives to have collaborative user story creation.
User stories must address functional and non-functional characteristics, and include acceptance criteria.
Testers can identify missing details or non-functional requirements, and by confirming acceptance criteria with the business reps.

The INVEST technique ensures the User Stories are:
- Independent
- Negotiable
- Valuable
- Estimable
- Small
-Testable

The 3C concept applies 3 elements:
- Card: physical media describing the user story and its requirement, criticality, expected development, test duration, and acceptance criteria. This is used for the product backlog
- Conversation: Explains how the software will be used and spans release-planning to scheduled release.
- Confirmation: Acceptance criteria are used to confirm that the story is done; and requires positive and negative tests. Defined acceptance criteria should be tested and shown to be satisfied.

FA-1.2.3 (K2) Understand how retrospectives can be used as a mechanism for process improvement in Agile projects

Retrospectives can improve test effectiveness, test productivity, test case quality, team satisfaction, along with the testability of applications, user stories, features, or system interfaces. Root cause analysis can support continuous improvement.\n\nTesters are part of the team and bring unique perspectives on testing and non-testing activities. Testing occurs in each sprint and contributes to success.\n\nRetrospectives must occur in a professional environment characterized by mutual trust, and attributes are the same as for any other review."}'>Retrospectives are held at the end of each iteration to discuss what was successful, what could be improved, and how to incorporate improvements and retain successes in future iterations. Retrospectives can improve test effectiveness, test productivity, test case quality, team satisfaction, along with the testability of applications, user stories, features, or system interfaces. Root cause analysis can support continuous improvement.

Testers are part of the team and bring unique perspectives on testing and non-testing activities. Testing occurs in each sprint and contributes to success.

Retrospectives must occur in a professional environment characterized by mutual trust, and attributes are the same as for any other review.

FA-1.2.4 (K2) Understand the use and purpose of continuous integration

Configuration management, compilation, software build, deployment, and testing are wrapped into a single, automated, repeatable process. \n\nA continuous integrated process consists of the following automated activities:\n- Static code analysis\n- Compile: generate executable files\n- Unit tests: check code quality and coverage\n- Deploy: install build into test environment\n- Integration test\n- Report: post and report status of activities\n\nContinuous integration detects errors early and quickly, and allows for regular automated regression testing. Manual testing can focus on new new features, confirmation testing of defect fixes, and testing implemented changes.\n\n"}'>Continuous integration merges all changes and integrates all changed components regularly, at least once per day. Configuration management, compilation, software build, deployment, and testing are wrapped into a single, automated, repeatable process.

A continuous integrated process consists of the following automated activities:
- Static code analysis
- Compile: generate executable files
- Unit tests: check code quality and coverage
- Deploy: install build into test environment
- Integration test
- Report: post and report status of activities

Continuous integration detects errors early and quickly, and allows for regular automated regression testing. Manual testing can focus on new new features, confirmation testing of defect fixes, and testing implemented changes.

Continuous integration can provide the following benefits:
- Allows earlier detection and easier root cause analysis of integration problems and conflicting changes
- Gives the dev team regular feedback on whether the code is working
- Keeps the version of the software tested within a day of the version being developed
- Reduces regresson risk associated with refactoring due to rapid re-testing of the code base after changes
- Provides confidence that each day's dev work is based on a solid foundation
- Makes progress toward completion of the product increment visible
- Eliminaes schedule risks associated with big-bang integration
- Provides constant avaialbility of executable software throughout the sprint for testing, demonstration, or education purposes
- Reduces repetitive manual testing activiies
- Provides quick feedback on decisions made to improve quality and tests

Continuous integration also has risks and challenges:
- Continuous integration tools have to be introduced and maintained
- Continuous integration process must be defined and established
- Test automation requires additional resources and can be complex
- Thorough test coverage is essential to achieving automated testing advantages
- Teams sometimes over-rely on unit tests and perform too little system or acceptance testing

FA-1.2.5 (K1) Know the differences between iteration and release planning, and how a tester adds value in each of these activities

It defines and re-defines the product backlog and may involve refining larger user stories into a collection of smaller stories. Release plans are high-level.\nRelease planning provides the basis for a test approach and test plan spanning all iterations.\nBusiness reps establish and prioritize the user stories, supporting project and quallity risks and a high-level estimation. Testers are involved and add value by:\n- Defining testabls user stories, including acceptance criteria\n- Participating in project and quality risk analysis\n- Estimating testing effort associated with user stories\n- Defining the necessary test levels\n- Planning the testing for the release\n"}'>Release planning looks ahead to the release of a product. It defines and re-defines the product backlog and may involve refining larger user stories into a collection of smaller stories. Release plans are high-level.
Release planning provides the basis for a test approach and test plan spanning all iterations.
Business reps establish and prioritize the user stories, supporting project and quallity risks and a high-level estimation. Testers are involved and add value by:
- Defining testabls user stories, including acceptance criteria
- Participating in project and quality risk analysis
- Estimating testing effort associated with user stories
- Defining the necessary test levels
- Planning the testing for the release
Iteration planning looks ahead to the end of a single iteration and is concerned with the iteration backlog.\nIn iteration planning, the team selects user stories from the prioritized product backlog, elaborates the user stories, performs risk analysis, and estimates the work. Business reps clarify any questions.\nThe number of stories depends on team velocity and the estimated size of user stories. User stories are broken out into tasks to be carried out by appropriate team members,\n\nTesters are involved in iteration planning amd add value in the following activities:\n- Participating in detailed risk analysis\n- Determining testability\n- Creating acceptance tests\n- Breaking down user stories into tasks (particularly testing tasks)\n- Estimating testing effort for testing tasks\n- Identifying functional and non-functional aspects to be tested\n- Supporting and participating in test automation at multiple levels of testing\n"}'>After release planning is tone, Interactiom planning for the first iteration starts. Iteration planning looks ahead to the end of a single iteration and is concerned with the iteration backlog.
In iteration planning, the team selects user stories from the prioritized product backlog, elaborates the user stories, performs risk analysis, and estimates the work. Business reps clarify any questions.
The number of stories depends on team velocity and the estimated size of user stories. User stories are broken out into tasks to be carried out by appropriate team members,

Testers are involved in iteration planning amd add value in the following activities:
- Participating in detailed risk analysis
- Determining testability
- Creating acceptance tests
- Breaking down user stories into tasks (particularly testing tasks)
- Estimating testing effort for testing tasks
- Identifying functional and non-functional aspects to be tested
- Supporting and participating in test automation at multiple levels of testing
Iteration plans may change during an iteration.\nTesters must understand the big picture and have an adequate test basis and test oracle in each iteration for test development purposes. Changes must be embraced according to Agile principles."}'>Release plans may change with the project, triggered by internal factors (delivery capabilities, velocity, technical issues) and external factors (new markets and opportunities, new competitors, business threats). Iteration plans may change during an iteration.
Testers must understand the big picture and have an adequate test basis and test oracle in each iteration for test development purposes. Changes must be embraced according to Agile principles.
Test-related issues include:\n- Scope of testing, test goals, and reasons\n- Team members who will carry out the activities\n- Test environment and test data needed, when needed, and changes to the test environment\n- Timing, sequencing, dependencies, and prerequisites for functional and non-functional test activities, and dependencies on dev activities.\n- Project and quality risks to be addressed\n\nLarger team estimation effort should include consideration of the time and effort needed to complete the required test activities."}'>Release and iteration planning should address test planning. Test-related issues include:
- Scope of testing, test goals, and reasons
- Team members who will carry out the activities
- Test environment and test data needed, when needed, and changes to the test environment
- Timing, sequencing, dependencies, and prerequisites for functional and non-functional test activities, and dependencies on dev activities.
- Project and quality risks to be addressed

Larger team estimation effort should include consideration of the time and effort needed to complete the required test activities.

Learning Objectives for Fundamental Agile Testing Principles, Practices, and Processes

2.1 The Differences between Testing in Traditional and Agile Approaches

FA-2.1.1 (K2) Describe the differences between testing activities in Agile projects and non-Agile projects

Following release and iteration planning, selected user stories are developed, integrated, and tested throughout each iteration. Testing activities occur throughout the iteration, not just as a final activity.\n\nTesters, developers, and business stakeholders all have a role in testing. Hardening or stabilization iteractions can occur to resolve lingering defects and other forms of technical debt. No feature is considered done until it has been integrated and tested with the system."}'>One of the main differences between traditional and Agile lifecycles is the idea of very short iteratons, with each iteracton resulting in working software that delivers value. Following release and iteration planning, selected user stories are developed, integrated, and tested throughout each iteration. Testing activities occur throughout the iteration, not just as a final activity.

Testers, developers, and business stakeholders all have a role in testing. Hardening or stabilization iteractions can occur to resolve lingering defects and other forms of technical debt. No feature is considered done until it has been integrated and tested with the system.

Risk analysis influences development and estimation."}'>For risk-based testing, high-level risk analysis occurs during release testing (testers drive the analysis), and specific quality risks for iterations are identified and assessed in iteration planning. Risk analysis influences development and estimation.

This promotes collective ownership of quality. Test automation at all levels occurs, requiring testers to spend time creating, executing, monitoring, and maintaining automated tests and results. Manual testing on Agile projects is done using experience-based and defect-based techniques (i.e. software attacks, exploratory testing, error guessing)."}'>Collaboration is achieved through pairing, coaching, and sharing testing knowledge. This promotes collective ownership of quality. Test automation at all levels occurs, requiring testers to spend time creating, executing, monitoring, and maintaining automated tests and results. Manual testing on Agile projects is done using experience-based and defect-based techniques (i.e. software attacks, exploratory testing, error guessing).

Changes to existing features have testing implications, particularly regression testing. Automated testing helps to manage the test effort. The rate of change must consider the risks associated with such changes."}'>Since changes occur in Agile projects, lightweight work product documentation is preferred. Changes to existing features have testing implications, particularly regression testing. Automated testing helps to manage the test effort. The rate of change must consider the risks associated with such changes.

FA-2.1.2 (K2) Describe how development and testing activities are integrated in Agile projects

Project work products of immediate interest to Agile testers fall into three categories:
- Business oriented: describe what is needed (requirements) and how to use it (user documentation)
- Development - how the system is built: database entity-relationship diagrams, that actually implement the system (code), or evaluate individual pieces of code (automated unit tests).
- Test: how the system is tested (test strategies and plans), actually test the system (manual and automated tests), or present test results (test dashboards)

Focus less on producing documentation and have working software and automated tests that demonstrate conformance to requirements.
Balance is needed to increase efficiency with reduced documentation, but support business, testing, development, and maintenance activities.

Also test metrics from defect reports and test results logs. \nWhile a lightweight approach is preferred, regulated or safety-critical, complex prjects and products may require more formal specifications that support vertical and horizontal tracability reports."}'>Business Work Products: Epics, User Stories, Acceptance Criteria
Developer Work Products: Code, automated unit tests, Test-driven development
Tester Work Products: Automated tests, test plans, quality risk catalogs, manual tests, defect reports, test results logs. Also test metrics from defect reports and test results logs.
While a lightweight approach is preferred, regulated or safety-critical, complex prjects and products may require more formal specifications that support vertical and horizontal tracability reports.

FA-2.1.3 (K2) Describe the role of independent testing in Agile projects

Any given user story will progress sequentially through the following test activities:\n- Unit testing\n- Feature acceptance testing for verification (often automated) against acceptance criteria.\n- Feature acceptance testing for validation (frequently manual) for fitness for use and real feedback from business stakeholders.\nThere are also regression testing activities done in parallel.\n\nInternal alpha and external beta tests may occur at the close or completion of iterations or after a series of iterations. These can include:\n- user acceptance\n- operational acceptance\n- regulatory acceptance\n- contract acceptance"}'>In some Agile lifecycles, overlap of test levels occurs because changes to requirements, design, and code can happen at any point in an iteration. Any given user story will progress sequentially through the following test activities:
- Unit testing
- Feature acceptance testing for verification (often automated) against acceptance criteria.
- Feature acceptance testing for validation (frequently manual) for fitness for use and real feedback from business stakeholders.
There are also regression testing activities done in parallel.

Internal alpha and external beta tests may occur at the close or completion of iterations or after a series of iterations. These can include:
- user acceptance
- operational acceptance
- regulatory acceptance
- contract acceptance
Functional tests at integration and system levels can be automated with harnesses, open-source, or commercial tools; run as part of a continuous integration framework.\nAutomated tests (i.e. daily unit tests, weekly functional tests) confirm that the build is functioning and installable. Failed tests should have the underlying defects fixed at the next code check-in. Changes that break the build or cause the software to fail to install are detected quickly. This also helps to manage the regresson risk.\n\nUnit testing often has limited defect detection effectiveness so automated tests at the integration and system levels are also required."}'>Automated tools can be used for static analysis, unit testing, and code coverage. Functional tests at integration and system levels can be automated with harnesses, open-source, or commercial tools; run as part of a continuous integration framework.
Automated tests (i.e. daily unit tests, weekly functional tests) confirm that the build is functioning and installable. Failed tests should have the underlying defects fixed at the next code check-in. Changes that break the build or cause the software to fail to install are detected quickly. This also helps to manage the regresson risk.

Unit testing often has limited defect detection effectiveness so automated tests at the integration and system levels are also required.
In Agile teams, there is a risk that testers' positions within the team may come with a risk of lost independence and objective evaluation.\nIndependent and separate test teams may preserve indepdendence and providan objective, unbiased evaluation of the software. Problems with this approach include tiem pressures, lack of understanding of new features, and relationship issues with developers and business stakeholders."}">Independent testers are often more effective at finding defects. In Agile teams, there is a risk that testers' positions within the team may come with a risk of lost independence and objective evaluation.
Independent and separate test teams may preserve indepdendence and providan objective, unbiased evaluation of the software. Problems with this approach include tiem pressures, lack of understanding of new features, and relationship issues with developers and business stakeholders.
This maintains independence while gaining good product understanding and strong relationships with other team members. Specialized testers can wrok on long-term or iteration-independent activities (i.e. developing automated test tools, non-functional testing, test environment maintenance, system integration testing)."}'>Independent and separate test teams can be assigned on a long-term basis. This maintains independence while gaining good product understanding and strong relationships with other team members. Specialized testers can wrok on long-term or iteration-independent activities (i.e. developing automated test tools, non-functional testing, test environment maintenance, system integration testing).

2.2 Status of Testing in Agile Projects

FA-2.2.1 (K2) Describe the tools and techniques used to communicate the status of testing in an Agile project, including test progress and product quality

Testing tasks related to acceptance criteria defined ofor user stories. When tests are passed, the task moves to the done column. Blockers are identified and resolved through the awareness of the whole team.\n\nCustomer satisfacton surveys provide feedback whether the product meets customer expectations. Other metrics include pass/fail rates, defect discovery rates, confirmation and regression test results, defect density, defects found and fixed, requirements coverage, risk coverage, code coverage, and code churn. Metrics should not be used to reward, punish, or isolate any team members."}'>Testers in Agile use test automation results, progression of test tasks and stories on the Agile task boad, and burndown charts to record and show test progress and status. A burndown chart represents the amount of work left to be done against time allocated to the release or iteration.
Communication methods include wiki dashboards, dashboard-style emails, and standup meetings. Auto-generated results can be used for process improvement.

Progress is managed by moving tasks on the task board into columns (to do, work in progress, verify, done). Testing tasks related to acceptance criteria defined ofor user stories. When tests are passed, the task moves to the done column. Blockers are identified and resolved through the awareness of the whole team.

Customer satisfacton surveys provide feedback whether the product meets customer expectations. Other metrics include pass/fail rates, defect discovery rates, confirmation and regression test results, defect density, defects found and fixed, requirements coverage, risk coverage, code coverage, and code churn. Metrics should not be used to reward, punish, or isolate any team members.

FA-2.2.2 (K2) Describe the process of evolving tests across multiple iterations and explain why test automation is important to manage regression risk in Agile projects

In order to maintain velocity without incurring large amounts of technical debt, teams must invest in test automation as early as possible.\nTest assets (automated tests, manual test cases, test data, test artifacts) must be kept up-to-date with each iteration and maintained in a configuration management tool to ensure version control, ease of access by all team members, and support changes while preserving historic information.\n\nDue to a lack of time for complete repetition of tests, testers must allocate time in each iteration to review manual and automated test cases from previous and current iterations for inclusions in the regression test suite, and retire test cases that are no longer relevant. Teams need to automate as many tests as possible from previous and current iterations, to free testers to more thoroughly test new features and functions in the current iteration."}'>As each iteration completes, the product grows, and the scope of testing increases. In order to maintain velocity without incurring large amounts of technical debt, teams must invest in test automation as early as possible.
Test assets (automated tests, manual test cases, test data, test artifacts) must be kept up-to-date with each iteration and maintained in a configuration management tool to ensure version control, ease of access by all team members, and support changes while preserving historic information.

Due to a lack of time for complete repetition of tests, testers must allocate time in each iteration to review manual and automated test cases from previous and current iterations for inclusions in the regression test suite, and retire test cases that are no longer relevant. Teams need to automate as many tests as possible from previous and current iterations, to free testers to more thoroughly test new features and functions in the current iteration.
Shorter timeframes and constant changes will increase the impact of poor test design and implementation practices.\nTest automation permits rapid feedback on quality and a living document of ssystem functionality.\nAutomated unit tests provide immediate feedback on code and build quality, but not product quality.\nAutomated acceptance tests are run regularly as part of continuous integration full system build, and are rum at least daily. These test results provide feedback on product quality with respect to regression since the last build, but not overall product quality.\nAutomated tests can be run continuously against the system, to cover functionality and integration ponts."}'>Defining how the team designs, writes and stores test cases should occur during release planning. Shorter timeframes and constant changes will increase the impact of poor test design and implementation practices.
Test automation permits rapid feedback on quality and a living document of ssystem functionality.
Automated unit tests provide immediate feedback on code and build quality, but not product quality.
Automated acceptance tests are run regularly as part of continuous integration full system build, and are rum at least daily. These test results provide feedback on product quality with respect to regression since the last build, but not overall product quality.
Automated tests can be run continuously against the system, to cover functionality and integration ponts.
If the test failed due to a defect, the defect should be fixed prior to progressing with new features.\nOther testing tasks that could be automated include:\n- Test data generation\n- Loading test data into systems\n- Deployment of builds into test environments\n- Restoration of a test environment\n- Comparison of data outputs\nAutomation reduces overhead and allows the team to spend time developing and testing new features."}'>Automated regression tests are run as part of the daily main build in continuous integration environment, and when a new build is deployed to the test environment. If the test failed due to a defect, the defect should be fixed prior to progressing with new features.
Other testing tasks that could be automated include:
- Test data generation
- Loading test data into systems
- Deployment of builds into test environments
- Restoration of a test environment
- Comparison of data outputs
Automation reduces overhead and allows the team to spend time developing and testing new features.

2.3 Role and Skills of a Tester in an Agile Team

FA-2.3.1 (K2) Understand the skills (people, domain, and testing) of a tester in an Agile team

Testers should have good interpersonal skills and should:\n- Be positive and solution-oriented\n- Display critical, quality-oriented, and skeptical thinking about the product\n- Actively acquire information from stakeholders\n- Accurately evaluate and report test results, test progress, and product quality\n- Work effectively to define testable user stories, acceptance criteria, with customer representatives and stakeholders\n- Collaborate within the team, working in pairs\n- Respond to change quickly, including changing, adding, or improving test cases\n- Plan and organize work\n\nContinuous skills growth, including interpersonal skills growth, is essential for all testers."}'>

In addition to tester skills, Agile testers should be competent in test automation, test-driven development, acceptance test-driven development, white-box, black-box, and experience-based testing.
Agile methodologies depend on collaboration, communication, and interaction between team members and stakeholders. Testers should have good interpersonal skills and should:
- Be positive and solution-oriented
- Display critical, quality-oriented, and skeptical thinking about the product
- Actively acquire information from stakeholders
- Accurately evaluate and report test results, test progress, and product quality
- Work effectively to define testable user stories, acceptance criteria, with customer representatives and stakeholders
- Collaborate within the team, working in pairs
- Respond to change quickly, including changing, adding, or improving test cases
- Plan and organize work

Continuous skills growth, including interpersonal skills growth, is essential for all testers.

FA-2.3.2 (K2) Understand the role of a tester within an Agile team

The role of a tester includes generating and providing feedback on test status, test progress, product quality, and process quality.
Activities include:
- Understanding, implementing, and updating the test strategy
- Measuring and reporting test coverage across all applicable coverage dimensions
- Ensuring proper use of testing tools
- Configuring, using, and managing test environments and test data
- Reporting defects and working with the team to resolve them
- Coaching other team members in relevant aspects of testing
- Ensuring appropriate testing tasks are scheduled during release and iteration planning
- Actively collaborating with developers and business stakeholders to clarify requirements for testability, consistency, and completeness
- Participating proactively in team retrospectives, suggesting and implementing improvements.
Within an Agile team, each team member is responsible for quality and plays a role in performing test-related tasks.
Test-related organizational risks may include:
- Testers work so closely to developers that they may lose the appropriate tester mindset.
- Testers become tolerant of or silent about inefficient, ineffective, or low-quality practices within the team
- Testers cannot keep pace with incoming changes in time-constrained iterations
To mitigate these risks, variations may be considered to preserve tester independence.

Learning Objectives for Agile Testing Methods, Techniques, and Tools

3.1 Agile Testing Methods

FA-3.1.1 (K1) Recall the concepts of test-driven development, acceptance test-driven development, and behavior-driven development

Test-driven development (TDD) is used to develop code guided by automated test cases. The TDD process is:
- Add a test that captures the programmer's concept of the desired functioning of a small piece of code
- Run the test, which should fail without the code
- Write the code and run the test in a tight loop until the test passes.
- Refactor the code, ensuring it continues to pass the test
- Repeat the process for the next small piece of code, running previous tests as well as added tests.

Tests are primarliy unit-level and code-focused, but can also be written at integraton and system levels. Tests are automated and are used in continuous integration.

Acceptance Test-Driven Development defines acceptance criteria and tess during the creation of user stories. This is a collaborative approach that highlights how the software has to behave, and what participants need to do to ensure this behavior. Acceptance-test driven development creates reusable tests for regression testing, often for the continuous integration process. Tests, connnected to the data and service layers, can be executed at the system or acceptance levels. This allows for quick resolution of defects and determining if acceptance criteria are met for the feature.

Behavior-Driven Development (BDD) allows a developer to focus on testing the code based on the expected behavior of the software. Specific BDD frameworks can be used to define acceptance criteria based on:
- Given some initial context
- When an event occurs
- Then ensure some outcomes
BDD generates code that can be used by developers ot create test cases, and help developers collaborate with other stakeholders to define accurate unit tests based on business needs.

FA-3.1.2 (K1) Recall the concepts of the test pyramid

Typical test levels are from the base of the pyramid to the top: Unit, Integration, System, and Acceptance.
Usually unit and integration level tests (large number of tests) are automated and created using API tools. At the system and acceptance levels, the automated tests are created using GUI-based tools.
The test pyramid concept is based on the testing principle of early QA and testing, to eliminate defects as early as possible in the lifecycle.

FA-3.1.3 (K2) Summarize the testing quadrants and their relationships with testing levels and testing types

Testing quadrants align the test levels with appropriate test types, to ensure that all important test types and test levels are included in the development lifecycle.
In testing quadrants, tests can be business (user) or technology (developer) facing. Some tests confirm software behavior, others verify product, and can be manual, automated, or a combination.

The four quadrants (applicable to dynamic testing) are as follows:
Q1: Unit level, technology facing, supporting the developers. Unit tests should be automated and included in the continuous integration process.
Q2: System level, business facing, confirms product behavior. Functional tests, story tests, user experience prototypes, and simulations. These check the acceptance criteria and could be manual or automated. Useful when creating regression test suites.
Q3: System or User Acceptance Level, business facing, contains tests that critique the product using realistic scenarios and data. This quadrant contains exploratory testing, scenarios, process flows, usability testing, user acceptance testing, alpha and beta testing. Tests are manual and user-oriented.
Q4: System or operational acceptance level, technology facing, contains tests that critique the product. This quadrant contains performance, load, stress, scalabilty tests, security, maintainability, memory management, interoperabilty, data migration, infrastructure, and recovery testing. Tests are often automated.

FA-3.1.4 (K3) For a given Agile project, practice the role of a tester in a Scrum team

Teamwork is a fundamental principle in Agile, which emphasizes whole-team approach of developers, testers, and business representatives working together.
Organizational and behavioral best practices in Scrum teams include:
- Cross functional
- Self-organizing
- Co-located
- Collaborative
- Empowered: technical decisions made by the team as a whole, in collaboration with Product Owner and other teams as needed
- Committed
- Transparent: on Agile task board
- Credible
- Open to feedback: Retrospectives help teams learn from successes and failures
- Resilient
Sprint Zero is the first iteration where many preparation activitie take place to:
- Identify project scope (product backlog)
- Create an initial system architecture and high-level prototypes
- Plan, acquire, and install needed tools (for test, defect, test automation, continuous integration)
- Create an initial test strategy for all test levels addressing test scope, technical risks, test types, and coverage goals.
- Perform initial qualty risk analysis
- Define test metrics to measure the test process, progress of testing in the project, and product quality
- Specify the Definition of Done
- Create the task board
- Define when to continue or stop testing before delivering the system to the customer.

Sprint Zero sets the direction for what testing needs to achieve and how testing needs to achieve it throughout the sprints.
The objective in Agile projects is to deliver customer value on a continuous basis. The integration strategy should consider both design and testing.
It is important to identify all dependencies between underlying functions and features.
Test planning shouldstart during the release planning session and be updated during each sprint.
Sprint planning results in a set of tasks to put on the task board, where each task should have a length of one or two days of work.
Any testing issues should be tracked to keep a steady flow of testing
Many Agile practices may be useful for testers in a scrum team including:
- Pairing: two team members sit together to perform testing or other sprint tasks
- Incremental test design: Test cases and charters are gradually built from user stories and other test bases, starting with simple tests and moving toward complex ones.
- Mind mapping: Testers can use mind mapping to identify which test sessions to perform, to show test strategies, and to describe test data.

3.2 Assessing Quality Risks and Estimating Test Effort

FA-3.2.1 (K3) Assess quality risks within an Agile project

Risk identification, analysis, and risk mitigation strategies can be used by testers in Agile teams to help determine an acceptable number of test cases to execute (subject to constraints and compromises).
Risk is the possibility of a negative or undesirable outcome or event. The level of risk is found by assessing the likelihood of occurrence and the impact of the risk. Product quality is covered by qualty risks or product risks, while project risks affect project success.

In Agile projects, quality risk analysis takes place during Release Planning and Iteration Planning. Potential quality risks include incorrect calcuations in reports, slow response to user input, and difficulty in understanding screens and fields.
Tasks associated with higher risks should start earlier and involve more testing effort.

The Agile quality risk analysis process has the following steps:
1. Gather Agile team members together
2. List the backlog items for the current iteration (on a task board)
3. Identify the quality risks associated with each item, considering all relevant quality characteristics.
4. Assess each identified risk, which includes categorizing the risk and determining its level based on impact and likelihood of defects.
5. Determine the extent of testing proportional to the risk.
6. Select the appropriate test techniques to mitigate each rsk, based on the risk, the level of risk, and the relevant quality characteristic.

The tester designs, implements, and executes tests to mitigate the risks. This includes features, behaviors, quality characteristics, and attributes affecting customer, user, and stakeholder satisfaction.
Periodic adjustment of quality risk analysis is needed by identifying new risks, re-assessing the level of existing risks, and evaluating the effectiveness of risk mitigations.
Quality risks can be mitigated before test execution starts.

FA-3.2.2 (K3) Estimate testing effort based on iteration content and quality risks

During release planning, Agile teams estimate the effort to complete the release, which includes the testing effort. Plannng poker, with Fibonacci sequence, assigns story points to user stories. Estimates consider development and testing effort, complexity, and scope. Risk levels should also be considered.

3.3 Techniques in Agile Projects

FA-3.3.1 (K3) Interpret relevant information to support testing activities

Agile projects outline initial requirements as user stories in a prioritized backlog at the start of the project.
Non-functional requirements (usability, performance) are also important and can be connected to functional user stories or specified as unique user stories.
User stories serve as an important test basis. Other test bases include:
- Experience from previous projects
- Existing functions, features, and quality characteristics of the system
- Code, architecture, and design
- User profiles (context, system configurations, user behavior)
- Information on defects from existing and previous projects
- Categorization of defects in a defect taxonomy
- Applicable standards
- Quality risks

During each iteration, acceptance criteria is needed to test the following:
- Functional behavior: user actions as input operating under certain configurations.
- Quality characteristics: non-functional requirements (performance, reliability, usabilty)
- Scenarios (use cases): actions between an actor (user) and the system, in order to accomplish a specific goal or business task
- Business Rules: Acitivities performed under certain conditions defined by outside procedures and constraints
- External interfaces: User intefaces, systems
- Constraints: design or implementation constraint that restricts options for the developers
- Data definitions: format, data type, allowed values, default values for data items, complex data strucutures

Additional relevant information includes:
- How the system is supposed to work and be used
- System interfaces that can be used to test the system
- Whether current tool support is sufficient
- Whether the tester has enough knowledge and skill to perform the necessary tess

Testers will discouver the need for additional information (i.e. code coverage) and work collaboratively with the other Agile team members to obain that information. Definition of Done is critical in Agile projects.

Each test level (Unit, Integration, System) has its own Definition of Done, pertinent to its level.
Definition of Done for
- User Stories represent completion, specified and reviewed, and tasks to test and implement are identified and estimated by the team.
- Features (multiple user stories or epics) have complete design and code, with completed unit and integration tests. No major defects remain, and feature documentaton (user manuals, release notes, on-line help functions) are complete.
- Iteration has features ready and individually tested, with integration and documentation complete. Non-critical defects that cannot be fixed are added to the product backlog and prioritized.
- Releases: Coverage (new, changed, complexity, size); Quality (defect density, remaining defects, residual level of risk); Time (pre-determined release date reached); Cost (development, maintenance, lifecycle costs)

FA-3.3.2 (K2) Explain to business stakeholders how to define testable acceptance criteria

Acceptance test-driven development is a test-first approach. Test cases are created prior to implementing the user story. Test cases are created by the Agile team including the developer, tester, and business representative; and may be manual or automated.
At a specification workshop the user story is analyzed, discussed, and written by the Agile team who fixes incompleteness, ambiguities, and errors in the user story.

FA-3.3.3 (K3) Given a user story, write acceptance test-driven development test cases

The tests created are examples that describe specific characteristics and help the team implement the user story correctly. First, correct behavior is confirmed with positive tests. After, the negative path tests and non-functional attributes should be covered as well. Examples should not overlap, and should not exceed the scope of the documented user story.

FA-3.3.4 (K3) For both functional and non-functional behavior, write test cases using black box test design techniques based on given user stories

In Agile, testers can be created concurrently with developers' programming activities. Traditional black box design techniques (equivalence class, boundary value, decision tables, state transition) can be used to select values and create tests.
Non-functional requirements can be documented as user stories.

FA-3.3.5 (K3) Perform exploratory testing to support the testing of an Agile project

Exploratory testing is important doe to limited time and limited details within user stories. Exploratory testing should be combined with other experienced-based techniques as part of a reactive testing strategy (blended with analytical risk/requirements testing, model-based testing, and regression-averse testing.
In exploratory testing, test design and test execution occur at the same time, guided by a prepared test charter for a time-boxed testing session wich includes:
- Actor: intended user
- Purpose: particular objectives
- Setup: needed to start
- Priority: relative importance
- Reference: specifications, user story, risks, other sources
- Data
- Activities: what the actor wants to do with the system
- Oracle notes: how to evaluate the product
- Variations: alternative actions and evaluations
To manage exploratory testing, session-based test management can be used (60-120 minutes) and can include:
- Survey (learn how it works)
- Analysis (evaluate functionality or characteristics)
- Deep coverage (corner cases scenarios, interactions)

Quality of the tests depends on relevant questions:
- What is most iportant to find out about the system?
- In what way will the system fail?
- What happens if?
- What should happen when?
- Are customer needs, requirements, expectations fulfilled?
- Is the ssytem possible to install (and remove if necessary) in all supported upgrade paths?
A set of heuristics can be applied when testing to guide the testing and evaluate the results:
- Boundaries
- CRUD (Create, Read, Update, Delete)
- Configuration variations
- Interruptions (log off, shut down, reboot)
Tester should document the proces as much as possible, to include the following information:
- Test coverage
- Evaluation notes during testing
- Risk/strategy list
- Issues, questions, and anomalies: any unexpected behavior
- Actual behavior (that needs to be saved)

Information logged should be captured and summarized into some form of status management tools (test management, task management, task board) to make it easy for stakeholders to understand the current status for testing that was performed.

3.4 Tools in Agile Projects

FA-3.4.1 (K1) Recall different tools available to testers according to their purpose and to activities in Agile projects

Agile teams use physical story/ task boards to manage and track user stories, tests, and other tasks throughout the sprint.
Tools serve the following purposes:
- Record stories and their relevant development and test tasks
- Capture team members' estimates on tasks and calculate the effort required to implement a story, for iteration planning
- Associate development tasks and test tasks with the same story for a complete picture of implementation efforts.
- Aggregate developer and tester updates to the task status as work is completed, to automatically provide a calculated snapshot of the story, the iteration, and the overall release.
- Provide a visual representation (metrics, charts, dashboards) of the current state of each story, iteration, and release; allowing all stakeholders to quickly check status.
- Integrate with configuration management tools, which allo automated recording of code check-ins and builds against tasks, and automated status updates for tasks.
Agile teams use three additional types of tools to support communication and information sharing: wikis, instant messaging, and desktop sharing.
- Wikis allow teams to build and share an online knowledge base on various aspects of the project, influcing product features, tools and/or techniques for developing and testing; metrics, charts, and dahsboards on product status; and conversations between team members.
- Instant messaging, audio teleconferencing, and video chat permit real-time direct communications, stand up meetings for distributed team members, and reduction of cost constraints.
- Desktop sharing supports product demonstratons, code reviews, pairing, and capturing of product demonstrations to be posted to Wiki.
Software Build and Distribution Tools support continuous integration and build distribution.
Configuration Management Tools support source code and automated tests, along with manual tests and other test work products. Version control systems can be centralized or distributed to support the team size, structure, location, adn requirements to integrate with other tools.
Virtualization allows a single physical resource (server) to operate as many separate, smaller resources. Virtual machines or cloud instances support development and testing, enable more rapid provisioning, and capture defects and faults with snapshots.
Test Design, Implementation, and Execution Tools include:
- Test Design - mind maps
- Test Case Management Tools - task management
- Test Data Preparaton and Generation Tools - quick updates to test data
- Test Data Load Tools - Generaton or bulk-loading
- Automated Test Execution Tools - Commercial or Open-source to support test-first approaches like Behavior-Driven Development, Test-Driven Development, and Acceptance Test-Driven Development. Also keywords or natural language testing can be supported.
- Exploratory Test Tools - capture and log activities during an exploratory tests session to record actions taken and report defects to developers. This also supports adding the exploratory test to the regression test suite.
-- DanielZrymiak - 07 Apr 2021
Edit  | Attach | Print version | History: r1 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r1 - 07 Apr 2021, DanielZrymiak
© 2020 Ultranauts - 75 Broad Street, 2nd Floor, Suite 206, New York, NY 10004 - info@ultranauts.co