Software QA: Stress and Pen Tests

5 min readJun 7, 2021

If you’ve ever developed a software that’s going to be distributed or used by multiple users, at some point you’d likely need to think about the performance and the security of the software. There is a concept in software engineering that specifically deals with the problems stated before, which is testing. But testing is just the big picture. You can check out my article on testing.

Testing is just the big picture and there are many types of tests including unit tests, integration tests, system tests and more. To address the problem above, we can use on what is called stress testing and penetration testing. So, let’s go more in-depth about what those two terms are.

Stress Testing

Stress testing is one type of test done to evaluate the stability, availability, and reliability of the software under development. The test is done by deliberately doing tasks beyond normal operational capacity, or even up to a breaking point, in order to observe the results. The results help developers determine the breaking points and/or up to what point the software fails to operate as intended.

Stress testing is very helpful if you intend to release the software publicly and be used by many users. Let’s say that the software is intended to be used by thousands of users making requests to the server at the same time. By stress testing, developers can evaluate the number of requests the server can handle and do actions based on the results gained.

Stress Test Demo

Enough theory crafting, let’s get into some stress testing demo. For this demo, The software being tested will be a Django REST Framework-based API. Locust which is a python-based GUI testing tool is a great tool for testing, so we will be using that in this demo. You can check out their documentation at https://docs.locust.io. Well then, let’s get started.

First of all, we need to install the tool first using pip globally or on a virtual environment:

pip install locust

Once that’s done, create a file in the root folder of the project named locustfile.py. Your project structure should look like this (just an illustration):

[projectname]/
├── [projectname]/
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
|
|── [App1]
|── locustfile.py
|── manage.py
└── requirements.txt

Once that’s done. Let’s edit the locustfile to specify what we are trying to stress test. For this demo, assume that we’ve created a POST request method for login, its path is /account/login and it will return a JSON Web Token (JWT). The token will be used to get the user object (basically user details) from the server) from the /account/user path and this is what we are going to stress test. The code should look something like this:

locustfile.py

Before running the test. Don’t forget to run the Django server:

python manage.py runserver

Once the server is run, we open the Locust GUI (make sure to run in the directory where you stored the locustfile.py):

locust

Open your browser and open up localhost:8089 which is where Locust is run. The page should look like below:

You’d see a form, just fill it accordingly to what you want. But in here, we’d test a total of 250 users with the spawn rate of 250/second. Fill in the host accordingly. Once done, then start the test by clicking the Start swarming button. To stop the test, simply click on the Stop button.

After running the test for about one minute, here’s the result.

The test shows that it can consistently do up to 211 requests per second which is lower than the expected 250 but close nonetheless. There is also some failures while trying to login with multiple users where the Server can’t handle more than a certain number of connections so it refused the connections made by 111 users.

One way to fix this problem is by using a load balancer to spread out the load handled by each server. Therefore, each server has balanced load and will reduce the chance of it refusing connections.

Penetration Testing

Penetration testing is a type of test to evaluate the security of the software, especially towards cyber attacks. More specifically, penetration testing is performed to identify vulnerabilities as well as strengths, which will result in a full risk assessment. The test is done by simulating cyber attacks to the software. If you’d like to know more about cyber attacks, Cisco made a really easy to understand article here.

Penetration Test Demo

Each framework has different ways of penetration testing. But for Django apps, the developers of Django have added a command that checks for software vulnerabilities. We can do it right at the root of the project, all you have to do is type in the terminal:

python manage.py check --deploy

That’s basically it. The output should look something like below:

path/to/project/project> python manage.py check --deploy
System check identified some issues:WARNINGS:
?: (security.W004) You have not set a value for the SECURE_HSTS_SECONDS setting. If your entire site is served only over SSL, you may want to consider setting a value and enabling 
HTTP Strict Transport Security. Be sure to read the documentation first; enabling HSTS carelessly can cause serious, irreversible problems.
?: (security.W008) Your SECURE_SSL_REDIRECT setting is not set to True. Unless your site should be available over both SSL and non-SSL connections, you may want to either set this 
setting True or configure a load balancer or reverse-proxy server to redirect all connections to HTTPS.
?: (security.W012) SESSION_COOKIE_SECURE is not set to True. Using a secure-only session cookie makes it more difficult for network traffic sniffers to hijack user sessions.       
?: (security.W016) You have 'django.middleware.csrf.CsrfViewMiddleware' in your MIDDLEWARE, but you have not set CSRF_COOKIE_SECURE to True. Using a secure-only CSRF cookie makes it more difficult for network traffic sniffers to steal the CSRF token.
?: (security.W018) You should not have DEBUG set to True in deployment.System check identified 5 issues (0 silenced).

As you can see above that Django has determined that there are some issues needed to be resolved in order to clear the software off of vulnerabilities. Make sure to solve all the problems stated by the check command before deploying to the production stage.

Final Thoughts

In short, the term stress testing is doing tasks to evaluate the overall performance of the software. Whereas penetration testing is doing tasks to evaluate the security of the software. By evaluating the performance and security of the software by performing both of the tests, we can improve the reliability, availability, stability, and security by mitigating cyber attacks.