The ultimate guide to Python exception handling

Exceptions can occur for various reasons, such as invalid input, logical errors, file handling issues, network problems, or other exception conditions. Examples of exceptions in Python include ZeroDivisionError, TypeError, FileNotFoundError, and ValueError, among others. Exception handling in Python is a crucial aspect of writing robust and reliable code in Python.

This tutorial aims to provide a comprehensive understanding of Python exception handling techniques and provide examples of how those techniques can be used practically. By the end of the tutorial, you’ll know how to build reliable programs. Let’s get started!

What is an exception?

In Python, an exception is an event that occurs during the execution of a program and disrupts the normal flow of the program. It represents an error or an exception condition that the program encounters and cannot handle by itself. Even with a great testing suite, Python exceptions can happen and cause problems for users.

When an exception occurs, it is "raised" or "thrown" by the Python interpreter. The exception then propagates up the call stack, searching for an exception handler that can catch and handle the exception. If no suitable exception handler is found, the program terminates, and an error message is displayed.

Exceptions differ from syntax errors because syntax errors happen when Python translates your code to bytecode. Syntax errors prevent your program from running at all. Exceptions, on the other hand, are considered runtime errors, which can pop up during the normal execution of otherwise valid code. For example, dividing by zero doesn’t break Python’s syntax rules; it’s valid syntax. But it’s a mathematical impossibility, so it raises an exception. Specifically, it raises a ZeroDivisionError.

Another key thing to remember about exceptions is that they’re not necessarily catastrophic. They’re special Python objects that can be caught and handled if you can foresee them coming. With proper exception handling, your program has the opportunity to respond to unexpected situations like missing files or invalid user input more gracefully. Instead of crashing, your code can log the error, show a user-friendly message, or even use some predetermined fallback logic. This makes exception handling essential for writing robust applications that can withstand a wide range of real-world scenarios.

Here’s an example of a ZeroDivisionError exception being raised and handled using a try-except block:

try:
    result = 10 / 0  # Raises ZeroDivisionError
except ZeroDivisionError:
    print("Error: Division by zero!")

In this example, the code within the try block raises a ZeroDivisionError exception when attempting to divide by zero. The exception is caught by the except block, and the specified error message is printed, allowing the program to continue execution instead of abruptly terminating.

Nested try-except blocks

Nested try-except blocks provide a way to handle specific exceptions at different levels of code execution. This technique allows you to catch and handle exceptions more precisely based on the context in which they occur. Consider the following example:

try:
    # Outer try block
    try:
        # Inner try block
        file = open("nonexistent_file.txt", "r")
        content = file.read()
        file.close()
        print("File content:", content)
    except FileNotFoundError:
        print("Error: File not found!")
except:
    print("Error: Outer exception occurred!")

Output:

Error: File not found!

In this example, the inner Python try block attempts to open a file "nonexistent_file.txt" in read mode, which doesn’t exist and raises a FileNotFoundError. The exception is caught by the inner except block, which prints the error message "Error: File not found!".

Since the exception is handled within the inner except block, the outer except block is not executed. However, if the inner except block was not executed, the exception would propagate to the outer except block, and the code within the outer except block would be executed.

Using try-except-else-finally blocks

A powerful feature of Python’s exception-handling paradigm is the option to include else and finally clauses alongside try-except. This structure provides a clear separation of concerns for different parts of your code, making it more readable and robust:

def safe_divide(a, b):
    try:
        result = a / b
    except ZeroDivisionError:
        print("Please don't divide by zero")
    else:
        print("Thank you for dividing")
    finally:
        print("This happens all the time")

safe_divide(10, 0)
safe_divide(20, 2)

The try block is the primary block where you attempt to execute code that might throw an exception. In our example, that’s a division operation.

The except block catches any exceptions that occur within the try block. In this particular example, that’s a ZeroDivisionError.

The else block executes only if no exceptions are raised in the try block. It’s a good place for code that only runs once everything has succeeded.

The finally block executes regardless of whether an exception occurred; it is commonly used for cleanup actions like closing files, releasing locks, or cleaning up resources.

This structured flow helps you handle errors more precisely, keeps the “happy path” logic separate, and ensures that any necessary teardown always happens.

Handling multiple exceptions in a single except block

Sometimes, you might want to catch multiple types of exceptions, but you want to treat them the same. You can catch multiple exceptions with identical handling logic. Python lets you group them in a single except clause using parentheses. Check out this example:

def fetch_data(source):
    if source == "file":
        raise FileNotFoundError("File source missing!")
    elif source == "api":
        raise ConnectionError("Failed to connect to API!")
    else:
        return "Data fetched successfully!"

sources = ["file", "api", "database"]
for src in sources:
    try:
        data = fetch_data(src)
        print(data)
    except (FileNotFoundError, ConnectionError) as e:
        print("Recoverable error:", e)
    except Exception as ex:
        print("Unknown error:", ex)

In this snippet, you’ll notice a few things. First, both the FileNotFoundError and the ConnectionError are handled in the same way. Second, you’ll notice that the except Exception acts as a fallback if some other unanticipated exception type arises.

Grouping exceptions is particularly helpful when you know that your remediation steps for certain errors will be identical. This approach prevents repetitive code and keeps your error handling clear and concise.

Catching and re-raising exceptions

Catching and re-raising exceptions is a useful technique when you need to handle an exception at a specific level of code execution, perform certain actions, and then allow the exception to reproduce to higher levels for further handling. Let’s explore the example further and discuss its significance.

In the provided code snippet, the validate_age function takes an age parameter and checks if it is negative. If the age is negative, a ValueError is raised using the raise keyword. The exception is then caught by the except block that specifies ValueError as the exception type.

def validate_age(age):
    try:
        if age < 0:
            raise ValueError("Age cannot be negative!")
    except ValueError as ve:
        print("Error:", ve)
        raise  # Re-raise the exception

try:
    validate_age(-5)
except ValueError:
    print("Caught the re-raised exception!")

In this case, if the age provided to validate_age is -5, the condition if age < 0 is satisfied, and a ValueError is raised with the message "Age cannot be negative!".

The except block then catches the ValueError and prints the error message using print("Error:", ve). This step allows you to perform specific actions, such as logging the error or displaying a user-friendly error message.

After printing the error message, the raise statement is used to re-raise the caught exception. This re-raised exception propagates to a higher level of code execution, allowing it to be caught by an outer exception handler if present.

The output of this code snippet is:

Error: Age cannot be negative!
Caught the re-raised exception!

This example demonstrates the importance of catching and re-raising exceptions. By catching an exception, performing necessary actions, and re-raising it, you have more control over how the exception is handled at different levels of your code.

Handling exceptions in loops

Handling exceptions in loops is essential to ensure the smooth execution of code and prevent premature termination. By incorporating exception handling within loops, you can gracefully handle specific exceptions and continue the loop iteration. Let’s explore the example further to understand its significance.

numbers = [1, 2, 3, 0, 4, 5]

for num in numbers:
    try:
        result = 10 / num
        print(f"Result: {result}")
    except ZeroDivisionError:
        print("Error: Division by zero!")
    except Exception as e:
        print("Unknown Error:", e)

In this code snippet, we have a loop that iterates over a list of numbers. For each number, a division operation is performed by dividing 10 by the current number.

If the number is zero, a ZeroDivisionError is raised. The first except ZeroDivisionError block catches this exception and prints the error message "Error: Division by zero!". Additionally, there is a generic except Exception as e block to catch any other exceptions that may occur during the loop iteration.

The output of this code snippet is:

Result: 10.0
Result: 5.0
Result: 3.3333333333333335
Error: Division by zero!
Result: 2.5
Result: 2.0

As we can see from the output, the loop successfully performs the division operation for non-zero numbers and prints the results. When encountering a zero, the ZeroDivisionError is caught, and the error message is printed. The loop continues to execute, handling each iteration gracefully, even in the presence of exceptions.

Handling exceptions in asynchronous code (asyncio)

Handling exceptions in asynchronous code can be crucial to ensure the stability and proper functioning of asynchronous tasks. The asyncio library in Python provides tools and mechanisms to handle exceptions in async tasks effectively.

import asyncio

async def divide(a, b):
    return a / b

async def main():
    try:
        await divide(10, 0)
    except ZeroDivisionError:
        print("Error: Division by zero!")

asyncio.run(main())

In this code snippet, we have an asynchronous function named divide that performs a division operation on two numbers. The division is executed using the division operator /, which may raise a ZeroDivisionError if the denominator is zero.

Within this function, we use the try-except block to catch any ZeroDivisionError that may occur during the execution of the divide coroutine.

The output of this code snippet is:

Error: Division by zero!

As expected, the division by zero raises a ZeroDivisionError exception, which is caught by the except ZeroDivisionError block. The error message "Error: Division by zero!" is then printed. Handling exceptions in asynchronous code with asyncio involves using the try-except block within the context of an async task.

Developing custom exceptions

Python allows software developers to create custom Python exception classes to handle specific types of errors within their applications. Creating custom exception classes provides a novel way to organize and categorize errors, making code more maintainable and enabling more granular error handling. With these benefits in mind, let’s now explore the process of creating custom exception classes in Python.

To create a custom exception class, you need to define a new class that inherits from one of the built-in exception classes or the base Exception class. Here is a simple example:

import logging

class CustomException(Exception):
    def __init__(self, message):
        super().__init__(message)
        self.error_code = 1001


    def log_error(self):
        logger = logging.getLogger("custom_logger")
        logger.error(f"Custom Exception occurred: {self}")
def divide(a, b):
    if b == 0:
        raise CustomException("Division by zero is not allowed.")
    return a / b

try:
    result = divide(10, 0)
except CustomException as ce:
    print("Error code:", ce.error_code)
    ce.log_error()

Output:

Error code: 1001
Custom Exception occurred: Division by zero is not allowed.

In the code above, the CustomException class is defined, inheriting from the base Exception class. It has an additional attribute, error_code, which is set to 1001 in the constructor (__init__ method). The super().__init__(message) call initializes the base Exception class with the given error message.

The log_error method retrieves or creates a logger named "custom_logger" using logging.getLogger("custom_logger"). The logger.error method is used to log an error message indicating that a custom exception occurred. The {self} expression in the log message includes the string representation of the exception object.

The divide function performs division between two numbers. If the divisor (b) is zero, the CustomException is triggered with the error message "Division by zero is not allowed."

The division operation is attempted within a try-except block. If a CustomException is raised, it is caught using the except CustomException as ce block. The error code of the exception is printed (print("Error code:", ce.error_code)) to demonstrate access to the error_code attribute of the CustomException instance. Finally, the log_error method is invoked (ce.log_error()) to log the exception.

Best practices to keep in mind for Python exception handling

Python’s flexibility makes catching and raising exceptions straightforward, but using this carefully is essential. Poorly structured exception handling can make code messy and unpredictable. Following some established standards to keep your exception handling straightforward is in your best interest. Here are a few best practices to keep in mind:

Don’t use too broad of a catch

Avoid using overly broad except Exception: blocks unless it’s absolutely necessary. Broad catches can mask programming errors (like typos or bad logic) that should surface for debugging. Instead, handle specific exception types whenever possible.

Log and re-raise exceptions to keep track of them

If you’re catching an exception solely to log or perform housekeeping before shutting down, consider re-raising the same exception. This preserves the original traceback and makes debugging easier. If you catch an exception just to log it, you’ll probably have some unexpected application flow.

Separate exception recovery from business logic

Keep your “happy path” business logic decoupled from the fallback behavior. Exceptional cases should not be normal. This makes your code cleaner. Leverage the else clause on try blocks for code that should only run if no exception was raised.

Document expected failures

Good documentation or even simple comments across your codebase can go a long way toward clarifying which exceptions a function can raise. This helps your team (or even your future self) confidently write code without guesswork.

Logging exceptions to an error monitoring service

It’s inevitable that you’ll eventually come across errors and exceptions when developing software applications. Monitoring and managing these Python errors effectively is crucial for maintaining the stability and performance of an application. One effective approach is to log exceptions to an error-monitoring service. Such services provide valuable insights into the occurrence and impact of errors, enabling developers to identify and resolve issues promptly. By integrating an error-monitoring service into the development workflow, teams can enhance their debugging and troubleshooting capabilities, leading to improved user experience and overall application quality. Examples of some error monitoring services are Honeybadger, Sentry, Datadog, etc.

Integrating Honeybadger for error monitoring

Honeybadger is a powerful error-monitoring tool for Python applications. Integrating an error monitoring service like Honeybadger into your development workflow provides numerous benefits for effectively managing exceptions.

From real-time notifications and error grouping to rich diagnostics and trend analysis, Honeybadger equips you with the tools you need to quickly identify, investigate, and resolve errors and ultimately enhance the overall quality and reliability of your applications. To show off some of this cool functionality, let’s now explore some features and examples of integrating Honeybadger into your Python code.

To use Honeybadger you need to first get an API key, which you get by signing up for a free Honeybadger account. On the Honeybadger projects page, click on the “Create your first project” button, give your project a name, and finally click the “Settings” tab, and you will be able to access your API key there.

You will also need to install Honeybadger locally. You can do that with the following command:

pip install honeybadger

Customizing Error Notifications

Honeybadger allows you to customize error notifications based on your application’s needs. You can specify the severity level, tags, and additional context to be included in the error reports. Here’s an example:

from honeybadger import honeybadger

# Configure Honeybadger with your API key
honeybadger.configure(api_key='hbp_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX') # paste your API key

def divide(a, b):
    result = a / b
    return result        

# Example usage of the divide function
try:
    result = divide(10, 0)
except Exception as e:
    honeybadger.notify(e, context={'user_id': 123}, error_message='Division by zero is not allowed.') #'Division by zero is not allowed.' will show if `e` was not passed

In this example, honeybadger.notify() the context parameter provides additional contextual information, such as the user ID associated with the error. To manually send an error message to Honeybadger notification, you add the “error_message argument. The message in error_message will show up only when there is no error passed in the honeybadger.notify() method.

When you run the above code, you will receive in your email a message notifying you of the error, like the message that can be seen below.

Handling Python exceptions in the wild

Python exception handling is a critical aspect of writing robust and reliable Python code. By mastering various techniques, from nested try-except blocks to creating custom exceptions, you can enhance the reliability of your code. Additionally, error tracking tools like Honeybadger can monitor for exceptions, allowing you to proactively identify and address issues in your Python applications.