Using Exceptions Wisely

Raising an exception alters the flow of control in a program. The interpreter normally executes statements one after the other (with looping to provide repetition, and conditionals to allow decisions to be taken). When an exception is raised, however, an entirely different mechanism takes over. Precisely because it is exceptional, we tend to be less familiar with it, but knowing how exceptions are raised and handled can help you to program to focus on the main task, in confidence that when exceptional conditions do occur they will be handled appropriately. Knowing how, and when, to use exceptions, is a part of your development as a Python programmer.

Exceptions offer such programming convenience that we would likely be quite happy to pay a modest penalty in performance. The happy fact is, though, that when used judiciously exceptions can actually enhance your programs' performance as well as making them easier to read.

BaseException
 +-- SystemExit
 +-- KeyboardInterrupt
 +-- GeneratorExit
 +-- Exception
      +-- StopIteration
      +-- ArithmeticError
      |    +-- FloatingPointError
      |    +-- OverflowError
      |    +-- ZeroDivisionError
      +-- AssertionError
      +-- AttributeError
      +-- BufferError
      +-- EnvironmentError
      |    +-- IOError
      |    +-- OSError
      |         +-- WindowsError (Windows)
      |         +-- VMSError (VMS)
      +-- EOFError
      +-- ImportError
      +-- LookupError
      |    +-- IndexError
      |    +-- KeyError
      +-- MemoryError
      +-- NameError
      |    +-- UnboundLocalError
      +-- ReferenceError
      +-- RuntimeError
      |    +-- NotImplementedError
      +-- SyntaxError
      |    +-- IndentationError
      |         +-- TabError
      +-- SystemError
      +-- TypeError
      +-- ValueError
      |    +-- UnicodeError
      |         +-- UnicodeDecodeError
      |         +-- UnicodeEncodeError
      |         +-- UnicodeTranslateError
      +-- Warning
           +-- DeprecationWarning
           +-- PendingDeprecationWarning
           +-- RuntimeWarning
           +-- SyntaxWarning
           +-- UserWarning
           +-- FutureWarning
           +-- ImportWarning
           +-- UnicodeWarning
           +-- BytesWarning

>>> try:
...     raise ZeroDivisionError
... except ArithmeticError:
...     print("ArithmeticError")
... except ZeroDivisionError:
...     print("ZeroDivisionError")
...
ArithmeticError
>>> try:
...     raise ZeroDivisionError
... except ZeroDivisionError:
...     print("ZeroDivisionError")
... except ArithmeticError:
...     print("ArithmeticError")
...
ZeroDivisionError
>>>

try:
    raise ZeroDivisionError
except ArithmeticError:
    print("ArithmeticError")
except ZeroDivisionError:
    print("ZeroDivisionError")

ArithmeticError
try:
    raise ZeroDivisionError
except ZeroDivisionError:
    print("ZeroDivisionError")
except ArithmeticError:
    print("ArithmeticError")

ZeroDivisionError

If you want to create your own exceptions, simply subclass the built-in Exception class or one of its already existing subclasses. Then create instances as required to raise exceptions. You may want to include an __init__() method on your subclass. The standard Exception.__init__() saves the tuple of positional arguments to the args attribute, so you can either do the same yourself or call Exception.__init__() to do it on your behalf. Your exceptions may at some stage be passed to a piece of code that expects to find an args instance variable.

Here's an example of a user-defined exception.

>>> class LocalError(Exception):
...     def __init__(self, msg):
...         self.args = (msg, )
...         self.msg = msg
...     def __str__(self):
...         return self.msg
...
>>> try:
...     raise LocalError("Appropriate message")
... except LocalError as e:
...     print("Trapped", e)
...
Trapped Appropriate message
>>> raise LocalError
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    raise LocalError
TypeError: LocalError.__init__() missing 1 required positional argument: 'msg'
>>>

This exception class requires an argument when an instantiation call is made to create a new instance—without one, the __init__() method does not receive enough arguments. You can see this happening when the raise LocalError statement is executed at the end of the session: when you use a class to raise an exception, the interpreter attempts to create an instance of that exception by calling the class with no arguments. So the message you see has nothing to do with the exception you have tried to raise; it's reporting the interpreter's inability to create an exception instance because of an argument mismatch in the __init__() method.

Exception objects are generally simple—the most they normally do is establish attribute values that can be used by the handler to extract information about the exception. Since they are classes, it is possible to add complex logic in multiple methods, but this is normally not done. As usual in Python, simplicity is the order of the day.

Understanding the straightforward flow of control when an exception is raised in the try suite is relatively easy. It is less easy to appreciate what happens when exceptions occur in the except or finally suites. To look at that, define a function that raises exceptions in one of those three places, then see what it does under those circumstances.

class LocalError(Exception):

    def __init__(self, msg):
        self.args = (msg, )
        self.msg = msg

    def __str__(self):
        return self.msg

def fxfin(where):
    "Demonstrate exceptions in various places."
    try:
        if where == "try":
            raise LocalError("LocalError in try")
        raise ValueError("ValueError in try")
    except (ValueError, LocalError) as e:
        print("Caught", e)
        if where == "except":
            raise LocalError("LocalError in except")
        print("Exception not raised in except")
    finally:
        print("Running finalization")
        if where == "finally":
            raise LocalError("LocalError in finally")
        print("Exception not raised in finally")

for where in "try", "except", "finally":
    print("---- Exception in %s ----" % where)
    try:
        fxfin(where)
    except Exception as e:
        print("!!!", e, "raised")
    else:
        print("+++ No exception raised +++")

When you run the program you should see the following output:

---- Exception in try ----
Caught LocalError in try
Exception not raised in except
Running finalization
Exception not raised in finally
+++ No exception raised +++
---- Exception in except ----
Caught ValueError in try
Running finalization
Exception not raised in finally
!!! LocalError in except raised
---- Exception in finally ----
Caught ValueError in try
Exception not raised in except
Running finalization
!!! LocalError in finally raised

When the exception is raised in the try suite, everything is perfectly normal and comprehensible, and both the except and finally handlers run without interruption. By the time the finally suite runs the exception has already been fully handled. The except suite is always activated, but it can be so either by virtue of the parameter value or because of the final explicit exception. This means the except clause is more readable. With the "except" argument the handler raises a second exception. This terminates the except handler, but the finally handler still runs; once it is complete, the second exception is still raised from the function. When the exception is raised in the finally suite, the finally handler does not run to completion, and the exception is passed up to the surrounding code (so the traceback is produced because of an uncaught exception).

Note that when you see a traceback for the case where an exception is raised during the handling of an exception that a second exception occurred during the processing of the first. This information may be confusing to end users, but can be invaluable to a programmer.

Let's take a look at the bytecodes that the CPython 3.1 interpreter produces for a simple function with exception handling.

>>> import dis
>>> def fex1():
...     try:
...         a = 1
...     except KeyError:
...         b = 2
...
>>> dis.dis(fex1)
  2           0 SETUP_EXCEPT            10 (to 13)

  3           3 LOAD_CONST               1 (1)
              6 STORE_FAST               0 (a)
              9 POP_BLOCK
             10 JUMP_FORWARD            24 (to 37)

  4     >>   13 DUP_TOP
             14 LOAD_GLOBAL              0 (KeyError)
             17 COMPARE_OP              10 (exception match)
             20 POP_JUMP_IF_FALSE       36
             23 POP_TOP
             24 POP_TOP
             25 POP_TOP

  5          26 LOAD_CONST               2 (2)
             29 STORE_FAST               1 (b)
             32 POP_EXCEPT
             33 JUMP_FORWARD             1 (to 37)
        >>   36 END_FINALLY
        >>   37 LOAD_CONST               0 (None)
             40 RETURN_VALUE
>>>

The interpreter establishes an exception-handling context by pointing at location 13 as the place to go if an exception occurs (this is what the SETUP EXCEPT op code does). This is followed by the body of the try clause. If the try clause reaches the end, the POP_BLOCK opcode throws away the exception-handling context and the JUMP_FORWARD sends the interpreter off to perform the implicit return None that terminates every function.

If an exception is raised, however, control is transferred to location 13, where the interpreter attempts to match the exception to the except specifications. If a match is found (and after various housekeeping operations we will ignore), line 26 is where the except suite is performed, after which another JUMP_FORWARD again selects the implicit return None. If no match is found for the exception, the END_FINALLY ensures that the exception is re-raised to activate any surrounding exception-handling contexts.

The try/except blocks in your program can be nested lexically (that is, a try/except can be a part of the try suite of another try suite) or dynamically (that is, a try suite can call a function that activates one or more try/excepts). When a try block is nested dynamically, it will be deactivated by termination of the function even if the return statement is in the try suite or an except suite. The finally suite is always executed, even when the function returns from an unexpected place. An explicit return in the finally suite does not allow that suite to run to completion—instead the return is executed (overriding any return value that might have triggered the execution of the finally clause).

>>> def fdct1():
...     wdict = {}
...     for word in words:
...         if word not in wdict:
...             wdict[word] = 0
...         wdict[word] += 1
...
>>> def fdct2():
...     wdict = {}
...     for word in words:
...         try:
...             wdict[word] += 1
...         except KeyError:
...             wdict[word] = 1
...
>>> from timeit import timeit
>>> words = "the quick brown fox jumps over the lazy dog".split()
>>> timeit("fdct1()", "from __main__ import fdct1, words")
0.8012009589583613
>>> timeit("fdct2()", "from __main__ import fdct2, words")
1.33151116699446
>>> words = ["same"] * 9
>>> timeit("fdct1()", "from __main__ import fdct1, words")
0.5743699169834144
>>> timeit("fdct2()", "from __main__ import fdct2, words")
0.587544666021131
>>>