Level 1. Defining Testing and Quality
Level 1
Defining Testing and Quality
Level 1 podcast
listen to the lesson as a podcast >
Dear student! Our course will help you better understand the goals of software testing in the context of existing project roles, the tasks that go with them, and the artifacts they produce.
And I say what you need is practice, practice, and more practice. Without real-world application, all that knowledge is worthless!
I will give you lots of tests and practical tasks, and I'll ask the Master to check everything thoroughly, thoroughly!
After the course you will be able to prepare, plan, and carry out the testing process on your own: set up test environments and test data sets, write defect descriptions, and produce testing reports.
You will also support the software development and project management process.
Too many clever words. Better explain in plain language what QA is.
As the ISO 9000:2005 standard tells us:
“Coordinated activities to direct and control an organization with regard to quality”, and a note adds that it “generally includes establishing a quality policy and quality objectives, quality planning, quality control, quality assurance and quality improvement”.
Hey, smarty-pants! And what about QC? I keep hearing about QA and QC, but what is QC and how is “that” different from QA?
Quality Control is the process of finding defects in a product so that they can be fixed. The goal of Quality Control is to maintain the quality of the product at a given moment in time. Quality Control is focused on the product currently being developed.
Quality Assurance includes Quality Control along with other processes that improve the quality of a company's work.
In other words, Quality Assurance guarantees that the process is set up correctly and gives a predictable result, while Quality Control guarantees that the product meets a given set of requirements.
Let's figure it out: what exactly is quality?
I think quality means the absence of bugs!
- A Mercedes is considered a high-quality car, yet this isn't about bugs.
- An Apple iPhone is considered a high-quality phone. Still, you couldn't call it “perfect”.
Then I think quality is customer satisfaction. Take that!
A good customer is always dissatisfied.
Then I'll put it this way: quality is meeting expectations. It does what it should and doesn't do what it shouldn't.
The ISO standard gives this definition: “Software quality is the capability of a software product to satisfy stated and implied needs when used under specified conditions.”
But I agree that the absence of bugs is a quality factor. By the way, let's talk about it in more detail:
about quality factors
A quality factor is a requirement for a program that is usually not spelled out in the contract with the customer, but is nevertheless desirable and raises the quality of the program.
R2D2, which quality factors do you know?
- Reliability — does the application work without failures, freezes, or unhandled exceptions;
- Maintainability — how hard it is to change the program to meet new requirements. It also means the program should be well documented, not too tangled, and have headroom for growth in resource usage (memory, CPU);
- Practicality — the purpose of the software should be clear from the program itself and its documentation;
- Efficiency — how sensibly the program uses resources (memory, CPU) while doing its job;
- Performance — does the application run at an acceptable speed when many users access it;
- Portability — how easily the program can be adapted to another environment: a different architecture, platform, operating system, or OS version;
- Functionality — does the application do what is required of it;
- Ease of use: how simple and convenient the program is to use. This mostly concerns the user interface;
- Security. This means protection of the software against hacking.
Holy moly! So many items. I think we'll meet these concepts again when we get to the types of testing.
You must be a little tired. When I get tired, I go and watch what the forces of evil are up to. It gives me new strength.
Yes! The horrors they commit give me strength to fight. Now I want to ask: where does Quality come from?
- The Quality of a Product is determined solely by the Quality of the Process used to develop it
- The Quality of the Process is determined solely by the Level of Development Culture in the Company
- The Quality of the Product is determined by the methodology and the approach to managing development / testing processes
Well, well. We keep walking around and around the subject. But we still haven't found out what testing is.
The incomparable Glenford Myers, in his book “Software Reliability” [Moscow: Mir, 1980], gave this definition:
Testing is the process of executing a program with the intent of finding errors. (a classic)
You old tin can! You give people a definition from 1980. Back then the internet was dial-up only. It's useless junk!
Old means experienced! It's still relevant!
I'll reprogram you into a toilet cleaner after definitions like that!
Validation in engineering or testing is the process of providing evidence that the requirements of a specific external user of a product or system are met.
I see you are tired of studying the fine points of testing, and have begun to use insults instead of wisdom.
You need to do an assignment. To restore your spiritual strength.
Imagine a form that validates an entered value.
Form requirement: if an integer value from 0 to 9 (inclusive) is entered, the form must return the value VALID
Is it something like this?
Correct. Don't rush; think carefully about how to test this form. Do you need to check every variant of valid and invalid input? For now you rely mostly on logic, but as you master the power of testing, you will start to see all the tests at a glance. If you happen to find bugs in the form, report them via the form at the end of the level, and you may earn bonus points.
I don't understand anything. How do I run the tests? This form is so strange and confusing. What am I supposed to do with it?
Each test, in this case, is a variant of the data entered into the form. For example, you enter the digit 3; it falls within the range from 0 to 9. The test passes if the form returned VALID. Then you enter 342, and the form does not return INVALID. In that case the test also passes. The point of the assignment is to come up with a minimal set of tests that lets you be 100% sure the form works correctly, covering all possible input–response pairs without trying every number in the world.
Now it's clearer. I should enter different data until I'm sure that I get the correct response in every case. Thank you, Master!
This is the answer to the Master's self-check question. When you are ready to check yourself, click the link.
Hooray! You did it. You're making your first progress. So, do you feel like a tester?
Good luck to you! As for the definition of testing, pay attention to this word:
Testing is the process of executing programs with the intent of finding errors.
From this, draw the conclusion that finding errors should not be the center of a tester's efforts.
I want to tell you more about errors. Errors and defects that make a program behave unexpectedly, and, as a result, the outcomes in testing, are called bugs.
Most bugs arise from mistakes made by the program's developers in its source code, behavior logic, or design. You will learn everything about bugs at level 5 of QA Academy.
In addition to this topic, I want to cite the definition coined by Paul Jorgensen:
Testing focuses on errors and failures. A test is the execution of actions on software in order to find errors or to demonstrate that it works.
Perhaps here I will agree with you. After work like that. And I'll give a generalized definition:
Testing is the process of verifying that the requirements stated for a product match the functionality actually implemented.
Advantages of this definition:
the focus of the testing process shifts toward verifying requirements
So, let's finally refine the definition:
Testing is the process of verifying that the requirements stated for a product match the functionality actually implemented, carried out by observing its operation in artificially created situations and on a limited set of tests chosen in a particular way according to the needs of the business task.
And don't forget common sense! The main axiom of the tester: everybody makes mistakes. Programmers, analysts, authors of requirements and documentation. Everyone needs to be checked!
So, the evolution of ideas about testing
| 1980 | The process of executing a program with the intent of finding errors. [G. Myers. Software Reliability. Moscow: Mir, 1980] |
|---|---|
| 1987 | The process of observing the execution of a program under specific conditions and, on that basis, evaluating some of its aspects. [ANSI/IEEE standard 610.12-1990: Glossary of SE Terminology. NY:IEEE, 1987] |
| 1990 | It is not an action. It is an intellectual discipline whose aim is to obtain reliable software without excessive effort spent on verifying it. [B. Beizer. Software Testing Techniques, Second Edition. NY:van Nostrand Reinhold, 1990] |
| 1999 | A technical investigation of a program to obtain information about its quality from the point of view of a particular group of stakeholders. [C. Kaner, 1999] |
| 2004 | Verification of the match between the actual behavior of a program and its expected behavior on a finite set of tests selected in a particular way. [IEEE Guide to Software Engineering Body of Knowledge, SWEBOK, 2004] |
Okay, so we've finally found out what testing is. I understood that the goals of testing are much broader than just finding defects! But I've accumulated some specific questions that all your theory gives no concrete answer to.
Suppose I found a certain number of errors. How do I tell whether N found bugs is a lot or a little? For example, 3 errors when testing the previous form?
An excellent question. It has troubled more than one generation of robots. They say that robot R1F1 burned out all his transistors searching for the answer. Think about it yourself.
Thought about it? Now check the answer.
It's very important to think for yourself before finding out what R2D2 thinks about it.
A question from the philosophy of testing: why do our users find bugs when we have spent so much time testing?!
I'll give you a minute to think it over yourself.
Then you can find out my opinion.
Still, I'd like to clarify: say I found my N bugs, let's say 82 bugs on the whole website. Can I calmly say the work is done and I've tested everything?
A simple question. You'll guess it yourself if you think for a minute.
Let's check the answers
Just think. If users will see the bugs anyway and testers won't catch 100% of them, why are testers needed at all? Why not leave it all to the users?
Ah, youth... No user wants to buy a software product with bugs. The era of Windows blue screens is long gone. The best-quality product wins the competition, even if it costs more. I will draw up a chart of defect-fixing costs for you. Then you'll understand that testers are needed at the very start of software creation. After all, with every stage more and more people are involved in fixing.
Taking this opportunity, I'd like to present some important facts about testing practice:
- Software testing takes from 30 to 50 percent of the total development cost (a classic metric)
- To test effectively means to solve project tasks within the process using integrable solutions
- Testing is not only a control process but also a powerful development tool that lets you reach a certain level of quality
Your task is to test effectively. Understood?!
Be polite, R2D2. You're talking to a future tester. The Master is very pleased with him. Now, let's sum up:
- The main goal of the testing process is to reach a certain level of quality of the Product or System under development
- The testing process can not only reveal and identify defects but also prevent them
- Testing becomes a process whose effectiveness radically affects the cost of the Product and its time to market
Did you understand what the first lesson was about? Seriously?
Great! Then here is your first theory test. If you pass it, the ship's backup generator will start. There will be enough energy to move on to the next level. But know that test results are added to the testers' rating only for users who bought a subscription. Note that you will have to register in the testing system.
To feel the knowledge of testing, you must use it in practice. I know it seems hard to you, but if you believe in yourself, you will find success.
Test the website and find the maximum number of defects. Send me the list through the form or by email
My dear Padawan, I hope today's adventure in the world of QA was interesting and exciting for you. I look forward to your assignment for review.
For a free review, repost our academy on social media. But let me clarify that a detailed analysis of the assignment is given only to premium subscribers or after paying for the review. After all, the Master's knowledge carries the highest price in this unstable world!