Добавить юзера c обычной подпиской
Стань тестировщиком сегодня
Level 2
The role of testing in the developing process
Welcome to the new level, my young padawan! Before you learn new secrets about the power of testing, I would like to discuss the home task with you homework. On the following video,
Master Ki-Adi-Mundi will show you, how he tested the site from the practice task. He will share to you secrets of practical testing and tell, on what you should pay attention and what approach to apply
So, let's proceed to our topic - "the role of testing in the developing process". Perhaps, you will try to assure us that you are not ready yet, but already on the second level you will have to meet the forces of evil! As you can guess - the main forces of evil are programmers or developers. They create artful code that contains all of these horrible bugs.
Somebody may say, that in real life they are good-natured bearded
roly-poly persons, which would not hurt a fly. This is also true. But in the field of software development - they are our main enemies.
My young Padawan, now we go on a risky event: we will observe how the process is going on the Death Star (where programmers work) and compare it with our working process to be as efficient as possible. Since it is necessary to know an enemy better than a friend!
Testing and development stages
Уровень 2 Galaxy QA Academy, еще не доступен в мобильной версии, пожалуйста воспользуйтесь десктопной. Извините за неудобства.
Development process
Testing process
Test development. For ex: test-cases
Coding
Running tests (test cases, running automated tests)
Debugging tests (adaptation of test cases to changed functional)
Creating versions of the product that you are testing (build)
Analyse test results. Test reports
Product support
Product support
Further goes the organizational scheme of interactions of development and a software testing. About every stage you can learn more by clicking on the link, although most of these concepts you will meet on the following levels in more detail.
About testing stages in more detail
Testing of a software
Planning of the testing process
Test design
Running tests (testing cycles)
Tests debugging
System Testing
Stages of a static testing:
-
Requirements analysis - studies of specifications and functional system requirements. Obtaining data for planning of the testing process
-
Planning of the testing process - determination of the test business volumes, approach, resources and schedule of implementation of planned actions
-
Test design - defining the purpose of the testing, specifications of an input data, test architecture for organizing tests into groups
Stages of a dynamic testing:
-
Running tests (testing cycles) - a direct checking of designed tests, an analysis of various test cases
-
Tests debugging - reviewing and debugging test cases
-
System Testing - a functional verification, performance testing
-
Acceptance Testing: alpha-testing, beta-testing
-
Operation and support - check of the results, correction of defects
Acceptance Testing
Operation and
support
We have familiarized ourselves with the concept of testing and the development process, so to say - with a bird's-eye view. Now, it's time to come down and take a look at specific topics.
Let's move from simple to complex:
-
Requirements analysis - Level 8
-
Planning - Level 7
-
Test-case design - let's stop here for more details
We start to consider all of the items below now and will seamlessly continue to work on them in the next level. Let's start with the topic that will clarify what is Test.
Definitions of test and test-case
Test – «triplet» Input/State/Output
Test is a consequence of steps/actions, which translates a system form one condition to another
Test is described by ISO, where:
-
[I] – is an input data or an action
-
[S] – is a state of a system at which data will be input
-
[O] – is the expected output
Test set
-
A test set, implementing a complete business task for an automated functional test system
-
A test set includes, except directly test scenarios, test data and also the rules or their creation / generation
Test set – practical considerations
-
After the unification of tests in cases, no idle test should remain
-
When designing tests "from top to down", test-cases will be the part of a test set
-
It is recommended to implement exactly the «from top to bottom» approach for test case design
Test coverage – practical considerations
-
A number of tests doesn't define the quality of a test coverage
-
% of test coverage doesn't define a degree of confidence to the test results, if it is not 100%
-
100% coverage is almost impossible in the conditions of a real design
-
Advise: increase a limit of test coverage in the case of an error in a test area or component
Design of test scenarios – practical considerations
There are formal methodologies of a test scenarios development
-
Development methodology of test-cases based on usage scenarios
-
Development methodology of test-cases based on orthogonal defects classification
There are formal methodologies for assessing an amount of work required for the minimum test coverage
-
Design procedure of cyclomatic complexity based on McCabe metrics
Mixed methodologies - a combination of approaches
Classification of testing «completness» of a system
Unit level - module level
-
Testing of a code integrity on the level of logic modules
-
Performed by developers
-
Controlled by testing team using instruments for analysis of a code coverage by unit-tests (unit test coverage tools)
Comments: according to the TDD concept, a tester should claim from programmers a code coverage by unit-tests (elemental testing of each module). It is not always the responsibility of a tester. from the manual testing point of view, a module level is a testing of a freestanding entry form (login + password)
Integration level - a cross-module interaction
-
Testing of intermediate results of a system integration
-
Performed by developers and testers
Comments: from the manual testing point of view, this is a translation of your site entry form add a check of interaction between the form and a site page.
System level - a system level in general
Validation of a completely built system against stated requirements
Sub-levels of system testing:
-
Alpha Testing
Performed by a testing team within a developing team/organization
-
Beta Testing
Performed by a testing team in an environment of friendly customers
-
Acceptance Testing
Performed by a customer to determine whether a system will be accepted for operation
-
«Smoke testing». Performed by a testing team to determine whether a system will be accepted for testing. It is applied to determine whether a program works in general, and is it worth to begin a testing cycle.
Comments: From the manual testing point of view, it is a form testing throughout the project.
Example - examination of the user database, in which the new record must be displayed on your site through the final entry form after the user registration
Congratulations! Half of the second level is passed.
It's time to take a break and stretch your back, drink a cup of tea, take a deep breath and then dive again into the world of testing
Classification of testing by approch applied
Black box (Functional) Testing
-
Testing from the point of view of an end user
-
Often combined with the methods of "white box"
-
The most common test methods for user-oriented systems and applications
I think you are ready to move on, my friend. Good luck!
White box (Structural) Testing
-
Application analysis at the code level
-
There are manual-expert code testing and automated testing using static analysis tools (methods)
-
One of most «expensive» testing methods, requiring high level qualification
Grey box testing (серый ящик)
-
Mixed adaptive technique used by experienced testers or developers for debugging
-
Analysis of an application from the point of view of a performance by the end-user operations and the control of the results obtained from the use of knowledge of the application code and implementation of details of the application functions
Functional testing
Functional testing – the primary type of testing, which aims to test the functional requirements of the relevant software to its actual performance. The main objective of functional testing is to confirm that the developed software has all the functionality required by a customer
Depending on a goal, it can be performed:
• On the basis of the functional requirements specified in a specification. These test cases are created for testing. Their compilation takes into account the priority of the software functions that are necessary to cover tests. This way we can be sure that all functions of a developed product are working properly in different types of input data, their combinations, quantities and so on.
• On the basis of the business processes that should be provided by your application. In this case, we are not interested in the individual performance of the software functions as in the correctness of operations from the point of view of system usage scenarios. In this case, the test will be based on the system use cases.
• On the basis of common sense. Since, there is may be a bug in the project documentation, and even in the architect's mind may be a bug (something is forgotten, something is not taken into account), a tester should always be on an alert. Contradictions of a TS (Technical Specification) with common sense and experience should cause naturally reaction - there is a bug!
Non-funtional testing
In contrast to the functional testing, which purpose is checking an apparent working capacity of a program, non-functional testing may not be specified in requirements. This concept fits everything that affects the quality of a program, but is not applied to a business logic directly.
Non-functional requirements describe a product by:
-
Reliability - the ability of the system/software to perform its required functions under stated conditions for a specified period of time, or for a specified number of operations.
-
Performance - the degree to which a system or a component accomplishes its designated functions within given constraints regarding processing time and throughput rate.
-
Scalability - requirements for a horizontal or a vertical scaling of an application. Application must nicely and adequately change it slicing at adding or removing structural elements. The same applies to the application architecture
-
Security - protection of user data from hacking, external influences, non-authorized access and system errors
-
Installation testing – confirms that the application under test recovers from expected or unexpected events without loss of data or functionality
-
Usability testing – testing to determine whether the system/software meets the specified usability requirements
-
Configuration testing (or portability testing) – investigation of the program system performance under different software configurations
-
Failover and Recovery Testing – the activity of testing how well the software is able to recover from crashes, hardware failures and other similar problems
-
Stress testing – testing conducted to evaluate a system or component at or beyond the limits of its specified requirements to determine the load under which it fails and how. Don't be confused with performance testing, because in this case we are talking only about the peak load, rather than design load
-
Performance testing – the set of testing types, the purpose of which is to determine the performance, stability, resource consumption, and other quality attributes of the application in a variety of usage scenarios and workloads. Performance testing allows you to find potential vulnerabilities and weaknesses in order to prevent their harmful impact on the work of the program in terms of use. In this case we are talking only about the planned loads, described in the requirements for a product
-
Compatibility testing – the process of testing to understand if software is compatible with other elements of a system with which it should operate
Testing classification by challenge types
Verification – the process of evaluating a system or a component to determine its completeness and whether it works properly.
Validation – determination of the correctness of the products of software development with respect to the user needs and requirements.
The following table will help to identify the key differences between these concepts:
Usability and GUI tesdting
Usability and GUI are closely related, since they have common testing objects - interfaces. They are suitable to conduct simultaneously. To usability and GUI testing are related:
Expert analysis of usability
In this case, you pretend to be a cool designer and provide an expert analysis of your application in accordance with the objectives of a project, functional and non-functional requirements for a software. Expert analysis includes:
-
analysis of information architecture of an application
-
analysis of an interface and its components
-
analysis of functional correspondance of an interface
-
compliance of a project design to a corporate design
Example: you perform all kinds of subtle assessment, unified style applications, quality of pictures, extra lines and even pixels, scalability and expansibility of GUI application, grammatical errors, and so on.
Simulation of a user behavior
In this case, you take a role of the most primitive user and checks the behavior of applications by simulating its behavior. Your task is to forget the application and start using it from scratch. The task - to get an idea about the user impression as a whole. Find all the points that can spoil the mood of the user. Bug here will be all that is not obvious and it is not clear to a new user.
Example: waiting the result without preloader for more than three seconds - a bug. Or a link that is not underlined, wrong cursor shape at navigation - a usability bug. Or an absence of hints at a difficult registration process and so on.
Final jump into functional testing
Testing types, conclusion:
This is my favorite scheme. It is worth to take it to your heart close. Very often in interviews for testing, they provide a similar task - to test an object. It shows how flexible mind of tester is in terms of species and the test object. It does not matter what is in front of you, but it is important to understand the concept of the logical types of testing. View and memorize this scheme FOREVER, if a tester you decided to be.
Further are listed testing types which are sections or variations of functional testing. There is also a verification of the correctness of an application, but it has its own charecteristics...
Regression testing – a set of tests designed to detect defects in areas of applications that have already been tested. This is done not to definitively verify the absence of bugs, but to find and fix regression bugs, i.e. errors in what have worked fine before. Such errors are usually caused by the correction of other errors or adding new features, and in a very different place. After all, a program is like a Rubik's cube, you turn one face, and colors change throughout the belt.
Automated testing - the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions, without manual intervention. And we will learn it!
Negative testing - Testing a system or application using negative data. For example testing a password field that requires a minimum of 9 characters by entering a password of 6.
Testing strategy
Testing strategy you will have to apply on practice in the next task. But not all types yet, only yellow.
Testing types in a common strategy:
-
Functional testing
-
Negative testing
-
Business-cycles testing
-
Usability testing
-
Testing of user's interface
-
Testing of data and database integrity
-
Testing of security and managing access
-
Configuration testing
-
Installation testing
-
Tools
Did you understand the lesson? Answer the following questions
Practical task
It is time to apply the new knowledge into practice.
To begin with, practice with testing types.
First task is to choose an item from the list that you like and write one test for each type of testing. Do it in the same way as of a pencil testing, but in a table format (type of testing - test). Do the task as Google doc submit trough the form.
Second task - theoretical test. Check, how well you understood the material. Don't forget to log in the testing system.
Third task is even more practical. Try to use types of testing at testing this site. All bugs you will find also write in a separate Google doc and submit. I know you can do it!
Items for the first task:
1. Chair
2. Bottle of mineral water
3. Steppler
4. Prickler
5. Paper folder
6. Table lamp
7. Sprinkler
8. Ruler
9. Scotch tape
10. Felt pen
11. Screw driver
I'm so happy to see your success! You click levels as easy as shelling peas. I think it's already clear that become a tester is not so difficult. But it's only the beginning of the road and ahead of 20 more levels, each of them is unique in both content and design. Take a step further and discover new challenges by the power of testing.
I'm happy to check the result of your work after you toss to old master a couple of credits for a new lightsaber, because the old one already conk out...
You need to get at least 20,6 points for practical tasks from level 2 to go to level 3!
