You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: documentation/quality-assurance/1_Test-Approach.md
+50-31Lines changed: 50 additions & 31 deletions
Original file line number
Diff line number
Diff line change
@@ -3,6 +3,7 @@
3
3
## Project Background
4
4
5
5
### Test Objectives
6
+
6
7
- Confirm that the service performs as per the defined User Needs, outlined during the discovery phase.
7
8
- Verify that the service meets the business and technical requirements set out by the DfE.
8
9
- Deliver a quality product by finding defects/bugs so that they can be fixed before delivering the service to the end user.
@@ -11,6 +12,7 @@
11
12
12
13
The product backlog and project sprints are managed within Azure DevOps using the scrum methodology.
13
14
User Stories in DevOps go through the following stages during the development process:
15
+
14
16
- To Do
15
17
- In Development
16
18
- In Review
@@ -19,38 +21,41 @@ User Stories in DevOps go through the following stages during the development pr
19
21
- Closed
20
22
21
23
Development of user stories is managed on a sprint board and goes through the following stages:
24
+
22
25
- Ready
23
26
- In Progress
24
27
- Complete
25
28
- Closed
26
29
Each status and Definition of Done act as control 'gates' for all work passing through this process.
27
30
28
31
### Definition of Done
29
-
30
-
- Development for the ticket is 'code complete.'
31
-
- Work items have been linked (use AB#).
32
-
- Your code builds clean without any errors or warnings.
33
-
- You have run all unit/integration tests and they pass.
34
-
- Your branch has been rebased onto main.
35
-
- You have tested by running locally.
36
-
- PR raised, reviewed & approved.
37
-
- Code squash merged into main.
38
-
- The functionality for the ticket has been tested and meets the acceptance criteria.
39
-
- Ticket status is updated on Azure DevOps.
40
-
- A sign-off is in place from the reporter and Product Owner (need to check if this is the expectation).
32
+
33
+
- Development for the ticket is 'code complete.'
34
+
- Work items have been linked (use AB#).
35
+
- Your code builds clean without any errors or warnings.
36
+
- You have run all unit/integration tests and they pass.
37
+
- Your branch has been rebased onto main.
38
+
- You have tested by running locally.
39
+
- PR raised, reviewed & approved.
40
+
- Code squash merged into main.
41
+
- The functionality for the ticket has been tested and meets the acceptance criteria.
42
+
- Ticket status is updated on Azure DevOps.
43
+
- A sign-off is in place from the reporter and Product Owner (need to check if this is the expectation).
41
44
42
45
## Test Engineer Roles and Responsibilities
43
46
44
47
As part of the project, the test engineer will be involved in the following activities:
45
-
- Involve in the refinement session - Identify functional and non-functional requirements from the tickets refinement session, add relevant scenarios, review acceptance criteria, and identify test data requirements.
46
-
- Review and add automated tests - Write automated feature/e2e tests ahead of/in step with the development of the service, maintain existing feature tests, and review unit and integration tests written by developers.
47
-
- Perform manual acceptance testing in Dev environment - Perform manual tests in the Dev environment to ensure that edge case scenarios and other scenarios that cannot be automated are tested.
48
-
- Perform ad-hoc exploratory testing in the Test Environment - Perform exploratory testing to ensure that all parts of the service integrate together correctly.
49
-
- Contribute to overall quality - Work towards contributing to the overall quality of the systems, process, and deliverables by ensuring each status of the ticket satisfies the pass criteria.
48
+
49
+
- Involve in the refinement session - Identify functional and non-functional requirements from the tickets refinement session, add relevant scenarios, review acceptance criteria, and identify test data requirements.
50
+
- Review and add automated tests - Write automated feature/e2e tests ahead of/in step with the development of the service, maintain existing feature tests, and review unit and integration tests written by developers.
51
+
- Perform manual acceptance testing in Dev environment - Perform manual tests in the Dev environment to ensure that edge case scenarios and other scenarios that cannot be automated are tested.
52
+
- Perform ad-hoc exploratory testing in the Test Environment - Perform exploratory testing to ensure that all parts of the service integrate together correctly.
53
+
- Contribute to overall quality - Work towards contributing to the overall quality of the systems, process, and deliverables by ensuring each status of the ticket satisfies the pass criteria.
50
54
51
55
## Testing Activities
52
56
53
57
### Overview
58
+
54
59
The following diagram shows the Agile Testing Quadrants outlining where each different type of testing sits, its order of implementation, and the associated categories.
@@ -62,66 +67,78 @@ The testing pyramid below shows the value of implementing different types of aut
62
67
### Automated Testing
63
68
64
69
#### Unit Testing
70
+
65
71
Unit Tests are developed by writing a test case to cover the requirement without any code supporting it. This test obviously fails. Then, the simplest code is written to pass the test. When another requirement comes along that alters the code, another test is written to satisfy this requirement, and the code is refactored as appropriate.
66
72
These tests are the responsibility of developers to write and maintain. Reviews of the unit test coverage by test engineers or other members of the team should be requested as appropriate.
Integration tests are used to ensure that parts of the code and frameworks work together in the correct way to create a working service. These tests build upon the quality assured during unit testing but are faster and lighter than end-to-end tests, allowing them to be run frequently and earlier in CI.
76
84
These tests are the responsibility of the developers to write and maintain in collaboration with the test engineers.
Feature and end-to-end tests simulate the behavior of an end user. These tests are the responsibility of test engineers to write and maintain in collaboration with developers.
87
97
88
98
##### Technology Selected
89
-
-[Playwright](https://playwright.dev/dotnet)
90
-
-[Xunit](https://github.com/xunit)
91
-
-[Specflow](https://specflow.org/)
99
+
100
+
-[Playwright](https://playwright.dev/dotnet)
101
+
-[Xunit](https://github.com/xunit)
102
+
-[Reqnroll](https://reqnroll.net/)
92
103
93
104
#### API Testing
105
+
94
106
API tests are designed to test that the correct responses are received from the API when requests are made to it. These tests are written by collaboration of dev and test engineer.
Automated accessibility testing will be carried out using Deque.axe.playwright library to generate feedback on whether the service meets high-level accessibility requirements in line with WCAG 2.2 AA. This will run as part of the pipeline in the later deployment steps from development to test and production.
UAT will be performed before release to ensure the system meets the requirements. This would also give assurance data has been successfully translated into the system.
112
127
113
128
##### Technology Selected
114
129
115
130
[will be added later on once we implement it]
116
131
117
132
#### Performance Testing
133
+
118
134
Performance testing is used to evaluate how the service is able to cope in terms of its stability, responsiveness, and usability when put under different simulated workloads. We yet have to implement it before going live.
119
135
120
136
##### Technology Selected
121
137
122
138
[will be added later on once we implement it]
123
139
124
140
#### Security Testing
141
+
125
142
Security scans help identify vulnerabilities and mitigate them that can lead to unauthorized access, data breaches, and theft of sensitive information. Further details can be found [here.](./3_Security-Testing.md)
126
143
127
144
### Manual Testing
@@ -141,7 +158,8 @@ This testing is carried out by test engineer(s) and/or developers when it is dee
141
158
### Testing Requirements
142
159
143
160
#### Supported Browsers
144
-
We will be focusing on Chrome browser on Windows and macOS and Edge on windows considering the insights we have from the previous system. The rest of the browsers shown below will be checked on adhoc basis.
161
+
162
+
We will be focusing on Chrome browser on Windows and macOS and Edge on windows considering the insights we have from the previous system. The rest of the browsers shown below will be checked on adhoc basis.
0 commit comments