Software testing is one of the most critical aspects of the software development cycle. It helps us to build robust and efficient software applications. This article will introduce you to software testing, its importance in software development, and the different types of software testing. It will also help you understand how to perform unit testing in Python.
What is software testing?
When we create software applications, we want them to execute correctly and efficiently. Software testing is the process of evaluating a software system or application to identify defects, errors, or other issues that may impact its functionality, performance, or usability. We need software testing to ensure that the software application we are creating meets the requirements and specifications outlined by stakeholders. It also helps us ensure that the application functions as intended and is reliable and user-friendly.
We can perform software testing at different stages of the software development lifecycle. Based on that, the testing process is classified as unit testing, integration testing, system testing, acceptance testing, and others. Each stage of testing serves a different purpose and helps to ensure that the software system is thoroughly tested and meets the necessary quality standards.
However, the ultimate goal of testing a software application is to identify and fix defects and issues before the software application is released to end users. Using software testing, we also aim to ensure that the application is reliable, functional, and meets the needs of its users.
Importance of testing in software engineering
Software testing is a crucial aspect of software engineering. It plays a critical role in ensuring the quality, reliability, and safety of the software application. Testing helps us to identify defects, ensure functionality, improve user experience, ensure compatibility, and reduce risks, which are all important factors in the success of a software product.
To understand the importance of testing, let's consider a web application that allows users to make online payments. If the application is not thoroughly tested, it may contain defects that could lead to payment errors, security vulnerabilities, or user interface issues. This will result in frustrated users, lost revenue, or even legal liabilities. By testing the application thoroughly, we can identify these risks and address them before the application is released to end users.
Following are some of the reasons testing is important in software engineering.
- Testing helps us identify defects: Software testing helps us to identify defects and bugs in the software system before it is released to end users. Identifying and fixing defects early in the development process can save time and resources in the long run. It will also lead to good consumer experience and, hence, a successful product.
- We can ensure the desired functionality of the software: With software testing, we can ensure that the application functions as intended and meets the requirements and specifications outlined by stakeholders. This can help to avoid costly rework and negative impacts on end users.
- Testing helps us improve the user experience: Software testing helps us to identify usability issues and ensure that the software application is user-friendly. This can help to improve the overall user experience and increase user satisfaction.
- Testing helps us ensure compatibility: A software application is used on different types of operating systems and hardware devices. Testing helps us ensure that the software application is compatible with different hardware, software, and operating systems. This helps us ensure that the software system can be used by a wide range of users.
- Testing helps us reduce risks: Testing helps us to identify potential risks and vulnerabilities in the software system, such as data loss or system crashes. Identifying and addressing these risks early can help us to reduce the likelihood and impact of these issues occurring.
Different types of software testing
Various types of software testing can be used to evaluate the quality and reliability of a software application. The most common types of testing mechanisms include unit testing, integration testing, functional testing, system testing, acceptance testing, performance testing, and security testing. Let us discuss the basics of all these testing methods.
Unit testing
Unit testing is used to test the individual units or components of a software system to ensure that they are working correctly. In unit testing, we test each function or module of the software application in isolation from the rest of the application. This helps us ensure that each unit or component is working correctly and meets the required specifications. Software developers typically perform their own unit testing. We can perform unit testing in Python by using modules like PyTest and unittest.
Integration testing
Once the individual components of a software application are built and tested, we use integration testing to test the interactions between the components to ensure that they work together correctly. We usually perform integration testing after unit testing and before system testing.
System Testing
System testing involves testing the entire software application to ensure that it meets the requirements and specifications outlined by stakeholders. System testing includes different tests, such as functional testing, performance testing, and security testing, to evaluate the overall quality and reliability of the application.
- Functional testing focuses on testing the functionality of a software application against its specified requirements. We use functional testing to ensure that the software application behaves as expected and meets the user's requirements. Usually, functional testing is performed at different levels of the software testing life cycle, such as unit testing, integration testing, and system testing.
- In performance testing, we test the performance and scalability of a software application under different conditions, such as heavy loads or high traffic volumes. We can use performance testing to identify and mitigate performance bottlenecks and other issues that may impact the system's performance.
- We use security testing to identify potential security vulnerabilities in the software application. It helps us ensure that the application is secure against unauthorized access, data breaches, and other types of cyber attacks.
There are many other types of software testing. However, we are mostly concerned with unit testing as software developers. All other types of testing are done by dedicated software test engineers. Therefore, let us now discuss how to implement unit testing in Python.
Getting started with unit testing in Python
To perform unit testing in Python, we will use the unittest
module. The unittest
module is incorporated by default during Python installation. Hence, we can start working with this module without the need to install any additional software.
A glance at the methods in the unittest module
The unittest
module contains different functions for testing the output of functions. The following table contains some of the functions that we can use while implementing unit tests with the unittest
module in Python.
Method | Functionality |
---|---|
assertEqual(a, b, msg) | Checks if a==b. If not, the test case fails. The message takes a message that you want to print if the test case fails i.e. a!=b. |
assertNotEqual(a, b, msg) | Checks if a!=b. |
assertTrue(x, msg) | Checks if the Boolean expression x evaluates to True. |
assertFalse(x, msg) | Checks if the Boolean expression x is False. |
assertIsNone(x, msg) | Checks if x is None. |
assertIsNotNone(x, msg) | Checks if x is not None. |
assertIn(a, b, msg) | Here, a is an element and b is a container object like a list, set, or tuple. The function checks if a is present in b. |
assertNotIn(a, b, msg) | Checks if element a is not present in container object b. |
assertRaises(exc, fun, args, *kwargs) | The assertRaises() method is used to check if a particular function “fun” raises the exception “exc” if we pass the input arguments “args” and “*kwargs” to the function. |
assertGreater(a, b, msg) | Checks if a>b. |
assertGreaterEqual(a, b, msg) | Checks if a>=b |
assertLess(a, b, msg) | Checks if a<b. |
assertLessEqual(a, b, msg) | Checks if a<=b. |
assertItemsEqual(a, b, msg) | If a and b are two container objects like a list or tuple, the assertItemsEqual() method checks if a and b contain the same elements. |
assertIsInstance(a, b, msg) | Checks if a is an instance of type b. |
assertNotIsInstance(a, b, msg) | Checks if a is not an instance of type b. |
We can use all the methods defined in the above table to perform unit testing in Python. The msg
parameter in all the above methods is optional. We can pass a string message to the msg
parameter. The methods print the message whenever a test case fails. Now, let us create a module containing math functions to perform unit testing.
Create a Python module for unit testing
To perform unit testing in Python, we need a software module. We will define a test_math_functions module that contains five functions for performing mathematical operations. First, we will define a function is_not_a_number() to check whether an input is a number or not. The is_not_a_number() function takes an input argument and checks whether the input is not of the data types int or float. If yes, it returns True showing that the input is not a number. Otherwise, it returns False.
def is_not_a_number(x):
number_classes = [int, float]
if type(x) not in number_classes:
return True
return False
Next, we will define the add() function to add two numbers. The add() function takes two input arguments. It first checks whether both the inputs are numbers using the is_not_a_number() function. If yes, it adds the numbers and returns the result. If both the inputs are not numbers, the add() function raises a TypeError exception saying that both inputs should be numbers.
def add(a, b):
if is_not_a_number(a) or is_not_a_number(b):
raise TypeError("Both values must be numbers.")
return a + b
In a similar manner as the add() function, we will define the subtract(), multiply(), and divide() functions as shown below.
def subtract(a, b):
if is_not_a_number(a) or is_not_a_number(b):
raise TypeError("Both values must be numbers.")
return a - b
def multiply(a, b):
if is_not_a_number(a) or is_not_a_number(b):
raise TypeError("Both values must be numbers.")
return a * b
def divide(a, b):
if is_not_a_number(a) or is_not_a_number(b):
raise TypeError("Both values must be numbers.")
if b == 0:
raise ValueError("Cannot divide by zero")
return a / b
We will save all the above functions in a file test_math_functions.py
. You can download the file using this link.
Define test cases for each function in the module
Now that we have defined all the functions in the test_math_functions
module, we need to define test cases for each function to ensure that the functions work as expected. A test case is an action performed on a function to determine whether it works correctly. In the following subsections, we define the test cases for each function.
Test cases for the is_not_a_number() function
We will define the following test cases for the is_not_a_number()
function.
- It should return True for an integer input.
- It should return True for a floating point input.
- It should return False for a string input.
Test cases for the add() function
We will define the following test cases for the add()
function.
- It should return an integer for two integer inputs.
- It should return a floating point number for two floating point inputs.
- It should return a floating point number for an integer and a floating point input.
- It should raise a
TypeError
exception if any of the inputs to the function is not a number.
Test cases for the subtract() function
We will define the following test cases for the subtract()
function.
- It should return an integer for two integer inputs.
- It should return a floating point number for two floating point inputs.
- It should return a floating point number for an integer and a floating point input.
- It should raise a
TypeError
exception if any of the inputs to the function is not a number. - It should return a negative number if the second input argument is larger than the first input argument.
Test cases for the multiply() function
We will define the following test cases for the multiply() function.
- It should return an integer for two integer inputs.
- It should return a floating point number for two floating point inputs.
- It should return a floating point number for an integer and a floating point input.
- It should raise a
TypeError
exception if any of the inputs to the function is not a number. - It should return a negative number if one input argument is positive, and the other input is negative.
- It should return a positive value if both inputs are positive.
- It should return a positive value if both the inputs are negative.
- It should return 0 if any of the inputs is 0.
Test cases for the divide() function
We will define the following test cases for the divide() function.
- It should return a floating point number for any valid number inputs.
- It should raise a
TypeError
exception if any of the inputs to the function is not a number. - It should return a negative number if one input argument is positive while the other input is negative.
- It should return a positive value if both inputs are positive.
- It should return a positive value if both the inputs are negative.
- It should raise a
ValueError
exception if the second input argument to the function is 0.
Create a test environment for unit testing in Python
After defining test cases, we will create a test environment. Thus, we will use the TestCase
class defined in the unittest
module in Python. We will create a subclass of the TestCase
class and define methods inside the class to run test cases. In the subclass, we will define methods for testing the functions for performing math operations as shown below.
import unittest
class TestMathOps(unittest.TestCase):
def test_number_check(self):
#code for testing the test_number_check function
def test_addition(self):
#code for testing the add function
def test_subtraction(self):
#code for testing the subtract function
def test_multiplication(self):
#code for testing the multiply function
def test_division(self):
#code for testing the divide function
Here, we have defined the TestMathOps
class with placeholder functions to test each function in the test_math_functions
module that we created in the previous sections. Now, we will write test functions to execute test cases on all the functions in the test_math_functions
module.
Write test functions to run test cases
Once we have created the test environment using the TestCase
class defined in the unittest
module, we will write test functions for the test cases. Thus, we will use the methods defined in the unittest
module as shown in the table.
The test function for testing the is_not_a_number()
function is as follows:
def test_number_check(self):
# test case 1
self.assertFalse(test_math_functions.is_not_a_number(7))
# test case 2
self.assertFalse(test_math_functions.is_not_a_number(7.1))
# test case 3
self.assertTrue(test_math_functions.is_not_a_number("HoneyBadger"))
The test function for testing the add()
function is as follows:
def test_addition(self):
# test on basic operations
self.assertEqual(test_math_functions.add(2, 3), 5)
self.assertEqual(test_math_functions.add(-2, 3), 1)
self.assertEqual(test_math_functions.add(0, 0), 0)
# test case 1
self.assertIsInstance(test_math_functions.add(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.add(3.1, -5.7), float)
# test case 3
self.assertIsInstance(test_math_functions.add(3, 5.1), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.add, "Honeybadger", 1117)
The test function for testing the subtract()
function is as follows:
def test_subtraction(self):
# test on basic operations
self.assertEqual(test_math_functions.subtract(2, 3), -1)
self.assertEqual(test_math_functions.subtract(-2, 3), -5)
self.assertEqual(test_math_functions.subtract(0, 0), 0)
# test case 1
self.assertIsInstance(test_math_functions.subtract(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.subtract(3.1, -5.7), float)
# test case 3
self.assertIsInstance(test_math_functions.subtract(3, 5.1), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.subtract, "Honeybadger", 1117)
# test case 5
self.assertLess(test_math_functions.subtract(3, 5), 0)
The test function for testing the multiply()
function is as follows:
def test_multiplication(self):
# test case 1
self.assertIsInstance(test_math_functions.multiply(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.multiply(3.1, -5.7), float)
# test case 3
self.assertIsInstance(test_math_functions.multiply(3, 5.1), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.multiply, "Honeybadger", 1117)
# test case 5
self.assertLessEqual(test_math_functions.multiply(3, -5), 0)
# test case 6
self.assertGreaterEqual(test_math_functions.multiply(3, 5), 0)
# test case 7
self.assertGreaterEqual(test_math_functions.multiply(-3, -5), 0)
# test case 8
self.assertEqual(test_math_functions.multiply(3, 0), 0)
The test function for testing the divide()
function is as follows:
def test_division(self):
# test basic operations
self.assertEqual(test_math_functions.divide(6, 3), 2)
self.assertEqual(test_math_functions.divide(-6, 3), -2)
# test case 1
self.assertIsInstance(test_math_functions.divide(6, 3), float)
self.assertIsInstance(test_math_functions.divide(6, 2.5), float)
self.assertIsInstance(test_math_functions.multiply(9.3, 3.1), float)
# test case 2
self.assertRaises(TypeError, test_math_functions.divide, "Honeybadger", 1117)
# test case 3
self.assertLessEqual(test_math_functions.divide(9.1, -3.1), 0)
self.assertLessEqual(test_math_functions.divide(-9.1, 3.1), 0)
# test case 4
self.assertGreaterEqual(test_math_functions.divide(9.1, 3.1), 0)
# test case 5
self.assertGreaterEqual(test_math_functions.divide(-9.1, -3.1), 0)
# test case 6
self.assertRaises(ValueError, test_math_functions.divide,5, 0)
Run tests using the main() function in the unittest module
After defining all the functions, we can run them by calling the main()
function in the unittest
module as shown below. You can download the source code using this link.
import unittest
import test_math_functions
class TestMathOps(unittest.TestCase):
def test_number_check(self):
# test case 1
self.assertFalse(test_math_functions.is_not_a_number(7))
# test case 2
self.assertFalse(test_math_functions.is_not_a_number(7.1))
# test case 3
self.assertTrue(test_math_functions.is_not_a_number("HoneyBadger"))
def test_addition(self):
# test on basic operations
self.assertEqual(test_math_functions.add(2, 3), 5)
self.assertEqual(test_math_functions.add(-2, 3), 1)
self.assertEqual(test_math_functions.add(0, 0), 0)
# test case 1
self.assertIsInstance(test_math_functions.add(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.add(3.1, -5.7), float)
# test case 3
self.assertIsInstance(test_math_functions.add(3, 5.1), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.add, "Honeybadger", 1117)
def test_subtraction(self):
# test on basic operations
self.assertEqual(test_math_functions.subtract(2, 3), -1)
self.assertEqual(test_math_functions.subtract(-2, 3), -5)
self.assertEqual(test_math_functions.subtract(0, 0), 0)
# test case 1
self.assertIsInstance(test_math_functions.subtract(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.subtract(3.1, -5.7), float)
# test case 3
self.assertIsInstance(test_math_functions.subtract(3, 5.1), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.subtract, "Honeybadger", 1117)
# test case 5
self.assertLess(test_math_functions.subtract(3, 5), 0)
def test_multiplication(self):
# test case 1
self.assertIsInstance(test_math_functions.multiply(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.multiply(3.1, -5.7), float)
# test case 3
self.assertIsInstance(test_math_functions.multiply(3, 5.1), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.multiply, "Honeybadger", 1117)
# test case 5
self.assertLessEqual(test_math_functions.multiply(3, -5), 0)
# test case 6
self.assertGreaterEqual(test_math_functions.multiply(3, 5), 0)
# test case 7
self.assertGreaterEqual(test_math_functions.multiply(-3, -5), 0)
# test case 8
self.assertEqual(test_math_functions.multiply(3, 0), 0)
def test_division(self):
# test basic operations
self.assertEqual(test_math_functions.divide(6, 3), 2)
self.assertEqual(test_math_functions.divide(-6, 3), -2)
# test case 1
self.assertIsInstance(test_math_functions.divide(6, 3), float)
self.assertIsInstance(test_math_functions.divide(6, 2.5), float)
self.assertIsInstance(test_math_functions.multiply(9.3, 3.1), float)
# test case 2
self.assertRaises(TypeError, test_math_functions.divide, "Honeybadger", 1117)
# test case 3
self.assertLessEqual(test_math_functions.divide(9.1, -3.1), 0)
self.assertLessEqual(test_math_functions.divide(-9.1, 3.1), 0)
# test case 4
self.assertGreaterEqual(test_math_functions.divide(9.1, 3.1), 0)
# test case 5
self.assertGreaterEqual(test_math_functions.divide(-9.1, -3.1), 0)
# test case 6
self.assertRaises(ValueError, test_math_functions.divide,5, 0)
if __name__ == '__main__':
unittest.main()
Output:
.....
----------------------------------------------------------------------
Ran 5 tests in 0.000s
OK
In the above example, no tests have failed. Therefore, you will get an OK
status showing that all the test cases have passed.
If any of the test cases fails, the program will print a message in the output. To check this, let us change some values in the test cases so that they fail.
import unittest
import test_math_functions
class TestMathOps(unittest.TestCase):
def test_number_check(self):
# test case 1
self.assertFalse(test_math_functions.is_not_a_number(7))
# test case 2
self.assertFalse(test_math_functions.is_not_a_number(7.1))
# test case 3
self.assertTrue(test_math_functions.is_not_a_number("HoneyBadger"))
def test_addition(self):
# test on basic operations
self.assertEqual(test_math_functions.add(2, 3), 5)
self.assertEqual(test_math_functions.add(-2, 3), 1)
self.assertEqual(test_math_functions.add(0, 0), 0)
# test case 1
self.assertIsInstance(test_math_functions.add(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.add(3.1, -5.7), float)
# test case 3
self.assertIsInstance(test_math_functions.add(3, 5), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.add, "Honeybadger", 1117)
def test_subtraction(self):
# test on basic operations
self.assertEqual(test_math_functions.subtract(2, 3), -1)
self.assertEqual(test_math_functions.subtract(-2, 3), -5)
self.assertEqual(test_math_functions.subtract(0, 0), 0)
# test case 1
self.assertIsInstance(test_math_functions.subtract(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.subtract(3, 5), float)
# test case 3
self.assertIsInstance(test_math_functions.subtract(3, 5.1), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.subtract, "Honeybadger", 1117)
# test case 5
self.assertLess(test_math_functions.subtract(3, 5), 0)
def test_multiplication(self):
# test case 1
self.assertIsInstance(test_math_functions.multiply(3, 5), int)
# test case 2
self.assertIsInstance(test_math_functions.multiply(3.1, -5.7), float)
# test case 3
self.assertIsInstance(test_math_functions.multiply(3, 5.1), float)
# test case 4
self.assertRaises(TypeError, test_math_functions.multiply, "Honeybadger", 1117)
# test case 5
self.assertLessEqual(test_math_functions.multiply(3, -5), 0)
# test case 6
self.assertGreaterEqual(test_math_functions.multiply(-3, 5), 0)
# test case 7
self.assertGreaterEqual(test_math_functions.multiply(-3, -5), 0)
# test case 8
self.assertEqual(test_math_functions.multiply(3, 0), 0)
def test_division(self):
# test basic operations
self.assertEqual(test_math_functions.divide(6, 3), 2)
self.assertEqual(test_math_functions.divide(-6, 3), -2)
# test case 1
self.assertIsInstance(test_math_functions.divide(6, 3), float)
self.assertIsInstance(test_math_functions.divide(6, 2.5), float)
self.assertIsInstance(test_math_functions.multiply(9.3, 3.1), float)
# test case 2
self.assertRaises(TypeError, test_math_functions.divide, "Honeybadger", 1117)
# test case 3
self.assertLessEqual(test_math_functions.divide(9.1, -3.1), 0)
self.assertLessEqual(test_math_functions.divide(-9.1, 3.1), 0)
# test case 4
self.assertGreaterEqual(test_math_functions.divide(9.1, -3.1), 0)
# test case 5
self.assertGreaterEqual(test_math_functions.divide(-9.1, -3.1), 0)
# test case 6
self.assertRaises(ValueError, test_math_functions.divide,5, 0)
if __name__ == '__main__':
unittest.main()
Output:
FFF.F
======================================================================
FAIL: test_addition (__main__.TestMathOps)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/aditya1117/software testing in python/code_for_testing.py", line 24, in test_addition
self.assertIsInstance(test_math_functions.add(3, 5), float)
AssertionError: 8 is not an instance of <class 'float'>
======================================================================
FAIL: test_division (__main__.TestMathOps)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/aditya1117/software testing in python/code_for_testing.py", line 76, in test_division
self.assertGreaterEqual(test_math_functions.divide(9.1, -3.1), 0)
AssertionError: -2.9354838709677415 not greater than or equal to 0
======================================================================
FAIL: test_multiplication (__main__.TestMathOps)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/aditya1117/software testing in python/code_for_testing.py", line 56, in test_multiplication
self.assertGreaterEqual(test_math_functions.multiply(-3, 5), 0)
AssertionError: -15 not greater than or equal to 0
======================================================================
FAIL: test_subtraction (__main__.TestMathOps)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/aditya1117/software testing in python/code_for_testing.py", line 36, in test_subtraction
self.assertIsInstance(test_math_functions.subtract(3, 5), float)
AssertionError: -2 is not an instance of <class 'float'>
----------------------------------------------------------------------
Ran 5 tests in 0.001s
FAILED (failures=4)
In this example, you can observe that four test cases have failed. Hence, the program prints the message for each test case with the proper message. Finally, it also prints the FAILED
status with the number of failures. You can check the failed test cases and make changes so that all the test cases can be executed successfully.
Best practices for software testing in Python
Software testing is a critical process that ensures the robustness of the application. To improve the effectiveness and efficiency of software testing, we can use the following best practices.
- Write test cases first: Writing test cases before writing code helps ensure that the code meets the requirements and works as expected. This practice is known as test-driven development and is a widely used technique in software development.
- Use a testing framework: Python has several testing frameworks. These frameworks provide a structured way to organize and run tests, making it easier to write and maintain test cases. Therefore, you should perform testing with the help of a testing framework for better efficiency.
- Test each function and method separately: Isolating functions and methods for testing makes it easier to identify and fix issues. You should define tests to cover all possible code paths, including edge cases and error conditions. This will help you create a robust code that doesn’t run into errors often.
- Use mocking and stubbing: Mocking and stubbing are techniques for replacing dependencies with simulated objects. These techniques can help simplify testing and make it easier to isolate code for testing.
- Use code coverage tools: Code coverage tools measure how much of the code is being executed during testing. These tools can help identify untested or poorly tested code, which can help improve the overall quality of the code.
By following the above best practices, you can improve the quality and reliability of your code. It will also help you identify and fix issues earlier in the development process, and ultimately deliver a better product to end users.
Conclusion
In this article, we discussed the basics of software testing and its types. We also discussed how to perform unit testing in Python using the unittest module. I suggest downloading the code files and making changes to the code to observe how the functions behave. You can also write a different software module and implement test cases using the methods given in the unittest module. This will help you to better understand unit tests.
I hope you enjoyed reading this article. Stay tuned for more informative articles.
Happy learning!