What are among the finest practices for Python unit testing? Earlier than we break it down, let’s distinguish unit testing from integration testing:
- A unit take a look at verifies a unit of code in isolation.
- An integration take a look at verifies a number of items of code in conjunction.
Crucially, unit assessments mustn’t make community requests, modify database tables, or alter information on disk except there’s an apparent purpose for doing so. Unit assessments use mocked or stubbed dependencies, optionally verifying that they’re referred to as accurately, and make assertions on the outcomes of a single operate or module. Integration assessments, then again, use concrete dependencies and make assertions on the outcomes of a bigger system.
You would possibly disagree on the finer factors in relation to Python, however this distinction works properly for me. I received’t go over unit testing fundamentals, like learn how to run pytest or learn how to use the @patch decorator, assuming you’re acquainted already.
Keep away from Module-Stage Globals
Think about we have now a category outlined in validator.py:
import os
import fastjsonschema
import requests
schema_url = os.environ['SCHEMA_URL']
class Validator:
def __init__(self):
schema = requests.get(schema_url).json()
self.validator = fastjsonschema.compile(schema)
def validate(self, occasion: dict) -> dict:
return self.validator(occasion)
And we have now a module named my_lambda.py that makes use of the category:
validator = Validator()
def handle_event(occasion: dict, context: object) -> dict:
validator.validate(occasion)
return {'physique': 'Goodbye!', 'standing': 200}
And we wish to take a look at the module:
from unittest.mock import Mock, patch
from my_lambda import handle_event
@patch('my_lambda.Validator', autospec=True)
def test_handle_event(mock_validator: Mock):
outcome = handle_event({'message': 'Hi there!'}, None)
assert outcome == {'physique': 'Goodbye!', 'standing': 200}
We’re patching the Validator class and never the validator property as a result of the previous will enable us to forestall the community request when Validator() known as, whereas the latter would solely enable us to behave afterwards.
Because it’s written, this take a look at will fail throughout assortment when my_lambda.py is imported, earlier than the patch is even utilized:
test_my_lambda.py:3: in <module>
from my_lambda import handle_event
my_lambda.py:1: in <module>
from validator import Validator
validator.py:6: in <module>
schema_url = os.environ['SCHEMA_URL']
/usr/native/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Variations/3.9/lib/python3.9/os.py:679: in __getitem__
elevate KeyError(key) from None
E KeyError: 'SCHEMA_URL'
We might set `SCHEMA_URL` in our .env file, however that’s an additional step that may be required for each developer to run the take a look at. As an alternative, let’s attempt setting it proper earlier than the import:
import os
from unittest.mock import Mock, patch
os.environ['SCHEMA_URL'] = 'foo'
from my_lambda import handle_event
This may also fail throughout assortment:
test_my_lambda.py:7: in <module>
from my_lambda import handle_event
my_lambda.py:3: in <module>
validator = Validator()
validator.py:10: in __init__
schema = requests.get(schema_url).json()
...
E requests.exceptions.MissingSchema: Invalid URL 'foo': No scheme provided. Maybe you meant http://foo?
Although we’re patching the Validator class, it’s nonetheless attempting to make a community request as a result of Validator() known as when my_lambda.py is imported, earlier than the patch is even utilized. If I sound like a damaged document, it’s as a result of this may be complicated:
> Module-level globals are assigned when the module is imported, which makes them tough to patch.
The right approach to repair this downside is to get rid of the worldwide from my_lambda.py:
from validator import Validator
def handle_event(occasion: dict, context: object):
validator = Validator()
validator.validate(occasion)
Now the take a look at passes, nevertheless it’s type of ugly. Let’s take away one other international, this time from validator.py:
import os
import fastjsonschema
import requests
class Validator:
def __init__(self):
schema_url = os.environ['SCHEMA_URL']
schema = requests.get(schema_url).json()
self.validator = fastjsonschema.compile(schema)
def validate(self, occasion: dict) -> dict:
return self.validator(occasion)
This permits us to simplify the take a look at, restoring it to its unique type:
from unittest.mock import Mock, patch
from my_lambda import handle_event
@patch('my_lambda.Validator', autospec=True)
def test_handle_event(mock_validator: Mock):
outcome = handle_event({'message': 'Hi there!'}, None)
assert outcome == {'physique': 'Goodbye!', 'standing': 200}
Globals Aren’t At all times Dangerous
In case you should use a worldwide for some purpose, set it to None initially:
from typing import Non-obligatory
from validator import Validator
validator: Non-obligatory[Validator] = None
def handle_event(occasion: dict, context: object):
international validator
if not validator:
validator = Validator()
validator.validate(occasion)
The bottom line is to forestall any costly computation or community requests when the module is imported to facilitate patching.
Use Dependency Injection
Subsequent, let’s take a look at the category:
from validator import Validator
def test_validator_init():
outcome = Validator()
assert outcome.validator isn't None
This take a look at will fail except SCHEMA_URL is outlined within the atmosphere. As beforehand talked about, we might add it to our .env file, or we might set it instantly like we did earlier than with os.environ['SCHEMA_URL'] = 'foo'. It’s higher to make use of the @patch.dict decorator as a result of it can restore the unique worth after the take a look at exits:
import os
from unittest.mock import patch
from validator import Validator
@patch.dict(os.environ, {'SCHEMA_URL': 'foo'})
def test_validator_init():
outcome = Validator()
assert outcome.validator isn't None
Both method, this take a look at will fail as a result of it’s attempting to make a community request to an invalid URL:
validator.py:9: in __init__
schema = requests.get(schema_url).json()
...
E requests.exceptions.MissingSchema: Invalid URL 'foo': No scheme provided. Maybe you meant http://foo?
Once more, we wish to stop the community request. That is the place dependency injection, also referred to as inversion of management, is useful. Let’s refactor validator.py to simply accept a couple of arguments:
import os
from typing import Callable, Non-obligatory
import fastjsonschema
import requests
class Validator:
def __init__(
self,
validator: Non-obligatory[Callable[[dict], dict]] = None,
schema: Non-obligatory[dict] = None,
schema_url: Non-obligatory[str] = None,
):
if validator:
self.validator = validator
return
if schema:
self.validator = fastjsonschema.compile(schema)
return
if not schema_url:
schema_url = os.environ['SCHEMA_URL']
schema = requests.get(schema_url).json()
self.validator = fastjsonschema.compile(schema)
def validate(self, occasion: dict) -> dict:
return self.validator(occasion)
It’s value a second to know what’s taking place right here when Validator() known as:
- If referred to as with a `validator` argument, that argument is assigned to `self.validator`. Some other arguments are ignored.
- If referred to as with a `schema` argument, that argument is used to create `self.validator`.
- If referred to as with a `schema_url` argument, the schema is requested from the URL, the response is parsed and used to create `self.validator`.
- If referred to as with none arguments, the schema is requested from the URL specified within the atmosphere.
This permits us to simply stop the community request when operating the unit take a look at:
import os
from unittest.mock import Mock, patch
from validator import Validator
def test_validator_init_with_validator():
mock_validator = Mock()
outcome = Validator(validator=mock_validator)
assert outcome.validator == mock_validator
The draw back is we have to cowl the extra complexity within the constructor. Happily, it may be completed with some simple patching:
@patch('fastjsonschema.compile', autospec=True)
def test_validator_init_with_schema(mock_compile):
mock_validator = Mock()
mock_compile.return_value = mock_validator
outcome = Validator(schema={'sort': 'string'})
assert outcome.validator == mock_validator
@patch('fastjsonschema.compile', autospec=True)
@patch('requests.get', autospec=True)
def test_validator_init_with_schema_url(mock_get, mock_compile):
mock_schema = Mock()
mock_get.return_value.json.return_value = mock_schema
mock_validator = Mock()
mock_compile.return_value = mock_validator
outcome = Validator(schema_url="foo")
assert outcome.validator == mock_validator
mock_get.assert_called_once_with('foo')
@patch('fastjsonschema.compile', autospec=True)
@patch('requests.get', autospec=True)
@patch.dict(os.environ, {'SCHEMA_URL': 'bar'})
def test_validator_init_with_no_args(mock_get, mock_compile):
mock_schema = Mock()
mock_get.return_value.json.return_value = mock_schema
mock_validator = Mock()
mock_compile.return_value = mock_validator
outcome = Validator()
assert outcome.validator == mock_validator
mock_get.assert_called_once_with('bar')
def test_validator_validate():
mock_validator = Mock()
mock_validator.return_value = 42
validator = Validator(mock_validator)
outcome = validator.validate({})
assert outcome == 42
This final unit take a look at is dust easy as a result of we don’t actually care how fastjsonschema is applied. It has its personal unit assessments.