diff options
| author | Craig Jennings <c@cjennings.net> | 2024-04-07 13:41:34 -0500 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2024-04-07 13:41:34 -0500 |
| commit | 754bbf7a25a8dda49b5d08ef0d0443bbf5af0e36 (patch) | |
| tree | f1190704f78f04a2b0b4c977d20fe96a828377f1 /devdocs/python~3.12/library%2Fcontextlib.html | |
new repository
Diffstat (limited to 'devdocs/python~3.12/library%2Fcontextlib.html')
| -rw-r--r-- | devdocs/python~3.12/library%2Fcontextlib.html | 465 |
1 files changed, 465 insertions, 0 deletions
diff --git a/devdocs/python~3.12/library%2Fcontextlib.html b/devdocs/python~3.12/library%2Fcontextlib.html new file mode 100644 index 00000000..25f4c403 --- /dev/null +++ b/devdocs/python~3.12/library%2Fcontextlib.html @@ -0,0 +1,465 @@ + <span id="contextlib-utilities-for-with-statement-contexts"></span><h1>contextlib — Utilities for with-statement contexts</h1> <p><strong>Source code:</strong> <a class="reference external" href="https://github.com/python/cpython/tree/3.12/Lib/contextlib.py">Lib/contextlib.py</a></p> <p>This module provides utilities for common tasks involving the <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement. For more information see also <a class="reference internal" href="stdtypes#typecontextmanager"><span class="std std-ref">Context Manager Types</span></a> and <a class="reference internal" href="../reference/datamodel#context-managers"><span class="std std-ref">With Statement Context Managers</span></a>.</p> <section id="utilities"> <h2>Utilities</h2> <p>Functions and classes provided:</p> <dl class="py class"> <dt class="sig sig-object py" id="contextlib.AbstractContextManager"> +<code>class contextlib.AbstractContextManager</code> </dt> <dd> +<p>An <a class="reference internal" href="../glossary#term-abstract-base-class"><span class="xref std std-term">abstract base class</span></a> for classes that implement <a class="reference internal" href="../reference/datamodel#object.__enter__" title="object.__enter__"><code>object.__enter__()</code></a> and <a class="reference internal" href="../reference/datamodel#object.__exit__" title="object.__exit__"><code>object.__exit__()</code></a>. A default implementation for <a class="reference internal" href="../reference/datamodel#object.__enter__" title="object.__enter__"><code>object.__enter__()</code></a> is provided which returns <code>self</code> while <a class="reference internal" href="../reference/datamodel#object.__exit__" title="object.__exit__"><code>object.__exit__()</code></a> is an abstract method which by default returns <code>None</code>. See also the definition of <a class="reference internal" href="stdtypes#typecontextmanager"><span class="std std-ref">Context Manager Types</span></a>.</p> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.6.</span></p> </div> </dd> +</dl> <dl class="py class"> <dt class="sig sig-object py" id="contextlib.AbstractAsyncContextManager"> +<code>class contextlib.AbstractAsyncContextManager</code> </dt> <dd> +<p>An <a class="reference internal" href="../glossary#term-abstract-base-class"><span class="xref std std-term">abstract base class</span></a> for classes that implement <a class="reference internal" href="../reference/datamodel#object.__aenter__" title="object.__aenter__"><code>object.__aenter__()</code></a> and <a class="reference internal" href="../reference/datamodel#object.__aexit__" title="object.__aexit__"><code>object.__aexit__()</code></a>. A default implementation for <a class="reference internal" href="../reference/datamodel#object.__aenter__" title="object.__aenter__"><code>object.__aenter__()</code></a> is provided which returns <code>self</code> while <a class="reference internal" href="../reference/datamodel#object.__aexit__" title="object.__aexit__"><code>object.__aexit__()</code></a> is an abstract method which by default returns <code>None</code>. See also the definition of <a class="reference internal" href="../reference/datamodel#async-context-managers"><span class="std std-ref">Asynchronous Context Managers</span></a>.</p> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.7.</span></p> </div> </dd> +</dl> <dl class="py function"> <dt class="sig sig-object py" id="contextlib.contextmanager"> +<code>@contextlib.contextmanager</code> </dt> <dd> +<p>This function is a <a class="reference internal" href="../glossary#term-decorator"><span class="xref std std-term">decorator</span></a> that can be used to define a factory function for <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement context managers, without needing to create a class or separate <a class="reference internal" href="../reference/datamodel#object.__enter__" title="object.__enter__"><code>__enter__()</code></a> and <a class="reference internal" href="../reference/datamodel#object.__exit__" title="object.__exit__"><code>__exit__()</code></a> methods.</p> <p>While many objects natively support use in with statements, sometimes a resource needs to be managed that isn’t a context manager in its own right, and doesn’t implement a <code>close()</code> method for use with <code>contextlib.closing</code></p> <p>An abstract example would be the following to ensure correct resource management:</p> <pre data-language="python">from contextlib import contextmanager + +@contextmanager +def managed_resource(*args, **kwds): + # Code to acquire resource, e.g.: + resource = acquire_resource(*args, **kwds) + try: + yield resource + finally: + # Code to release resource, e.g.: + release_resource(resource) +</pre> <p>The function can then be used like this:</p> <pre data-language="python">>>> with managed_resource(timeout=3600) as resource: +... # Resource is released at the end of this block, +... # even if code in the block raises an exception +</pre> <p>The function being decorated must return a <a class="reference internal" href="../glossary#term-generator"><span class="xref std std-term">generator</span></a>-iterator when called. This iterator must yield exactly one value, which will be bound to the targets in the <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement’s <code>as</code> clause, if any.</p> <p>At the point where the generator yields, the block nested in the <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement is executed. The generator is then resumed after the block is exited. If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred. Thus, you can use a <a class="reference internal" href="../reference/compound_stmts#try"><code>try</code></a>…<a class="reference internal" href="../reference/compound_stmts#except"><code>except</code></a>…<a class="reference internal" href="../reference/compound_stmts#finally"><code>finally</code></a> statement to trap the error (if any), or ensure that some cleanup takes place. If an exception is trapped merely in order to log it or to perform some action (rather than to suppress it entirely), the generator must reraise that exception. Otherwise the generator context manager will indicate to the <code>with</code> statement that the exception has been handled, and execution will resume with the statement immediately following the <code>with</code> statement.</p> <p><a class="reference internal" href="#contextlib.contextmanager" title="contextlib.contextmanager"><code>contextmanager()</code></a> uses <a class="reference internal" href="#contextlib.ContextDecorator" title="contextlib.ContextDecorator"><code>ContextDecorator</code></a> so the context managers it creates can be used as decorators as well as in <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statements. When used as a decorator, a new generator instance is implicitly created on each function call (this allows the otherwise “one-shot” context managers created by <a class="reference internal" href="#contextlib.contextmanager" title="contextlib.contextmanager"><code>contextmanager()</code></a> to meet the requirement that context managers support multiple invocations in order to be used as decorators).</p> <div class="versionchanged"> <p><span class="versionmodified changed">Changed in version 3.2: </span>Use of <a class="reference internal" href="#contextlib.ContextDecorator" title="contextlib.ContextDecorator"><code>ContextDecorator</code></a>.</p> </div> </dd> +</dl> <dl class="py function"> <dt class="sig sig-object py" id="contextlib.asynccontextmanager"> +<code>@contextlib.asynccontextmanager</code> </dt> <dd> <p>Similar to <a class="reference internal" href="#contextlib.contextmanager" title="contextlib.contextmanager"><code>contextmanager()</code></a>, but creates an <a class="reference internal" href="../reference/datamodel#async-context-managers"><span class="std std-ref">asynchronous context manager</span></a>.</p> <p>This function is a <a class="reference internal" href="../glossary#term-decorator"><span class="xref std std-term">decorator</span></a> that can be used to define a factory function for <a class="reference internal" href="../reference/compound_stmts#async-with"><code>async with</code></a> statement asynchronous context managers, without needing to create a class or separate <a class="reference internal" href="../reference/datamodel#object.__aenter__" title="object.__aenter__"><code>__aenter__()</code></a> and <a class="reference internal" href="../reference/datamodel#object.__aexit__" title="object.__aexit__"><code>__aexit__()</code></a> methods. It must be applied to an <a class="reference internal" href="../glossary#term-asynchronous-generator"><span class="xref std std-term">asynchronous generator</span></a> function.</p> <p>A simple example:</p> <pre data-language="python">from contextlib import asynccontextmanager + +@asynccontextmanager +async def get_connection(): + conn = await acquire_db_connection() + try: + yield conn + finally: + await release_db_connection(conn) + +async def get_all_users(): + async with get_connection() as conn: + return conn.query('SELECT ...') +</pre> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.7.</span></p> </div> <p>Context managers defined with <a class="reference internal" href="#contextlib.asynccontextmanager" title="contextlib.asynccontextmanager"><code>asynccontextmanager()</code></a> can be used either as decorators or with <a class="reference internal" href="../reference/compound_stmts#async-with"><code>async with</code></a> statements:</p> <pre data-language="python">import time +from contextlib import asynccontextmanager + +@asynccontextmanager +async def timeit(): + now = time.monotonic() + try: + yield + finally: + print(f'it took {time.monotonic() - now}s to run') + +@timeit() +async def main(): + # ... async code ... +</pre> <p>When used as a decorator, a new generator instance is implicitly created on each function call. This allows the otherwise “one-shot” context managers created by <a class="reference internal" href="#contextlib.asynccontextmanager" title="contextlib.asynccontextmanager"><code>asynccontextmanager()</code></a> to meet the requirement that context managers support multiple invocations in order to be used as decorators.</p> <div class="versionchanged"> <p><span class="versionmodified changed">Changed in version 3.10: </span>Async context managers created with <a class="reference internal" href="#contextlib.asynccontextmanager" title="contextlib.asynccontextmanager"><code>asynccontextmanager()</code></a> can be used as decorators.</p> </div> </dd> +</dl> <dl class="py function"> <dt class="sig sig-object py" id="contextlib.closing"> +<code>contextlib.closing(thing)</code> </dt> <dd> +<p>Return a context manager that closes <em>thing</em> upon completion of the block. This is basically equivalent to:</p> <pre data-language="python">from contextlib import contextmanager + +@contextmanager +def closing(thing): + try: + yield thing + finally: + thing.close() +</pre> <p>And lets you write code like this:</p> <pre data-language="python">from contextlib import closing +from urllib.request import urlopen + +with closing(urlopen('https://www.python.org')) as page: + for line in page: + print(line) +</pre> <p>without needing to explicitly close <code>page</code>. Even if an error occurs, <code>page.close()</code> will be called when the <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> block is exited.</p> </dd> +</dl> <dl class="py function"> <dt class="sig sig-object py" id="contextlib.aclosing"> +<code>contextlib.aclosing(thing)</code> </dt> <dd> +<p>Return an async context manager that calls the <code>aclose()</code> method of <em>thing</em> upon completion of the block. This is basically equivalent to:</p> <pre data-language="python">from contextlib import asynccontextmanager + +@asynccontextmanager +async def aclosing(thing): + try: + yield thing + finally: + await thing.aclose() +</pre> <p>Significantly, <code>aclosing()</code> supports deterministic cleanup of async generators when they happen to exit early by <a class="reference internal" href="../reference/simple_stmts#break"><code>break</code></a> or an exception. For example:</p> <pre data-language="python">from contextlib import aclosing + +async with aclosing(my_generator()) as values: + async for value in values: + if value == 42: + break +</pre> <p>This pattern ensures that the generator’s async exit code is executed in the same context as its iterations (so that exceptions and context variables work as expected, and the exit code isn’t run after the lifetime of some task it depends on).</p> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.10.</span></p> </div> </dd> +</dl> <span class="target" id="simplifying-support-for-single-optional-context-managers"></span><dl class="py function"> <dt class="sig sig-object py" id="contextlib.nullcontext"> +<code>contextlib.nullcontext(enter_result=None)</code> </dt> <dd> +<p>Return a context manager that returns <em>enter_result</em> from <code>__enter__</code>, but otherwise does nothing. It is intended to be used as a stand-in for an optional context manager, for example:</p> <pre data-language="python">def myfunction(arg, ignore_exceptions=False): + if ignore_exceptions: + # Use suppress to ignore all exceptions. + cm = contextlib.suppress(Exception) + else: + # Do not ignore any exceptions, cm has no effect. + cm = contextlib.nullcontext() + with cm: + # Do something +</pre> <p>An example using <em>enter_result</em>:</p> <pre data-language="python">def process_file(file_or_path): + if isinstance(file_or_path, str): + # If string, open file + cm = open(file_or_path) + else: + # Caller is responsible for closing file + cm = nullcontext(file_or_path) + + with cm as file: + # Perform processing on the file +</pre> <p>It can also be used as a stand-in for <a class="reference internal" href="../reference/datamodel#async-context-managers"><span class="std std-ref">asynchronous context managers</span></a>:</p> <pre data-language="python">async def send_http(session=None): + if not session: + # If no http session, create it with aiohttp + cm = aiohttp.ClientSession() + else: + # Caller is responsible for closing the session + cm = nullcontext(session) + + async with cm as session: + # Send http requests with session +</pre> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.7.</span></p> </div> <div class="versionchanged"> <p><span class="versionmodified changed">Changed in version 3.10: </span><a class="reference internal" href="../glossary#term-asynchronous-context-manager"><span class="xref std std-term">asynchronous context manager</span></a> support was added.</p> </div> </dd> +</dl> <dl class="py function"> <dt class="sig sig-object py" id="contextlib.suppress"> +<code>contextlib.suppress(*exceptions)</code> </dt> <dd> +<p>Return a context manager that suppresses any of the specified exceptions if they occur in the body of a <code>with</code> statement and then resumes execution with the first statement following the end of the <code>with</code> statement.</p> <p>As with any other mechanism that completely suppresses exceptions, this context manager should be used only to cover very specific errors where silently continuing with program execution is known to be the right thing to do.</p> <p>For example:</p> <pre data-language="python">from contextlib import suppress + +with suppress(FileNotFoundError): + os.remove('somefile.tmp') + +with suppress(FileNotFoundError): + os.remove('someotherfile.tmp') +</pre> <p>This code is equivalent to:</p> <pre data-language="python">try: + os.remove('somefile.tmp') +except FileNotFoundError: + pass + +try: + os.remove('someotherfile.tmp') +except FileNotFoundError: + pass +</pre> <p>This context manager is <a class="reference internal" href="#reentrant-cms"><span class="std std-ref">reentrant</span></a>.</p> <p>If the code within the <code>with</code> block raises a <a class="reference internal" href="exceptions#BaseExceptionGroup" title="BaseExceptionGroup"><code>BaseExceptionGroup</code></a>, suppressed exceptions are removed from the group. If any exceptions in the group are not suppressed, a group containing them is re-raised.</p> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.4.</span></p> </div> <div class="versionchanged"> <p><span class="versionmodified changed">Changed in version 3.12: </span><code>suppress</code> now supports suppressing exceptions raised as part of an <a class="reference internal" href="exceptions#BaseExceptionGroup" title="BaseExceptionGroup"><code>BaseExceptionGroup</code></a>.</p> </div> </dd> +</dl> <dl class="py function"> <dt class="sig sig-object py" id="contextlib.redirect_stdout"> +<code>contextlib.redirect_stdout(new_target)</code> </dt> <dd> +<p>Context manager for temporarily redirecting <a class="reference internal" href="sys#sys.stdout" title="sys.stdout"><code>sys.stdout</code></a> to another file or file-like object.</p> <p>This tool adds flexibility to existing functions or classes whose output is hardwired to stdout.</p> <p>For example, the output of <a class="reference internal" href="functions#help" title="help"><code>help()</code></a> normally is sent to <em>sys.stdout</em>. You can capture that output in a string by redirecting the output to an <a class="reference internal" href="io#io.StringIO" title="io.StringIO"><code>io.StringIO</code></a> object. The replacement stream is returned from the <code>__enter__</code> method and so is available as the target of the <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement:</p> <pre data-language="python">with redirect_stdout(io.StringIO()) as f: + help(pow) +s = f.getvalue() +</pre> <p>To send the output of <a class="reference internal" href="functions#help" title="help"><code>help()</code></a> to a file on disk, redirect the output to a regular file:</p> <pre data-language="python">with open('help.txt', 'w') as f: + with redirect_stdout(f): + help(pow) +</pre> <p>To send the output of <a class="reference internal" href="functions#help" title="help"><code>help()</code></a> to <em>sys.stderr</em>:</p> <pre data-language="python">with redirect_stdout(sys.stderr): + help(pow) +</pre> <p>Note that the global side effect on <a class="reference internal" href="sys#sys.stdout" title="sys.stdout"><code>sys.stdout</code></a> means that this context manager is not suitable for use in library code and most threaded applications. It also has no effect on the output of subprocesses. However, it is still a useful approach for many utility scripts.</p> <p>This context manager is <a class="reference internal" href="#reentrant-cms"><span class="std std-ref">reentrant</span></a>.</p> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.4.</span></p> </div> </dd> +</dl> <dl class="py function"> <dt class="sig sig-object py" id="contextlib.redirect_stderr"> +<code>contextlib.redirect_stderr(new_target)</code> </dt> <dd> +<p>Similar to <a class="reference internal" href="#contextlib.redirect_stdout" title="contextlib.redirect_stdout"><code>redirect_stdout()</code></a> but redirecting <a class="reference internal" href="sys#sys.stderr" title="sys.stderr"><code>sys.stderr</code></a> to another file or file-like object.</p> <p>This context manager is <a class="reference internal" href="#reentrant-cms"><span class="std std-ref">reentrant</span></a>.</p> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.5.</span></p> </div> </dd> +</dl> <dl class="py function"> <dt class="sig sig-object py" id="contextlib.chdir"> +<code>contextlib.chdir(path)</code> </dt> <dd> +<p>Non parallel-safe context manager to change the current working directory. As this changes a global state, the working directory, it is not suitable for use in most threaded or async contexts. It is also not suitable for most non-linear code execution, like generators, where the program execution is temporarily relinquished – unless explicitly desired, you should not yield when this context manager is active.</p> <p>This is a simple wrapper around <a class="reference internal" href="os#os.chdir" title="os.chdir"><code>chdir()</code></a>, it changes the current working directory upon entering and restores the old one on exit.</p> <p>This context manager is <a class="reference internal" href="#reentrant-cms"><span class="std std-ref">reentrant</span></a>.</p> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.11.</span></p> </div> </dd> +</dl> <dl class="py class"> <dt class="sig sig-object py" id="contextlib.ContextDecorator"> +<code>class contextlib.ContextDecorator</code> </dt> <dd> +<p>A base class that enables a context manager to also be used as a decorator.</p> <p>Context managers inheriting from <code>ContextDecorator</code> have to implement <code>__enter__</code> and <code>__exit__</code> as normal. <code>__exit__</code> retains its optional exception handling even when used as a decorator.</p> <p><code>ContextDecorator</code> is used by <a class="reference internal" href="#contextlib.contextmanager" title="contextlib.contextmanager"><code>contextmanager()</code></a>, so you get this functionality automatically.</p> <p>Example of <code>ContextDecorator</code>:</p> <pre data-language="python">from contextlib import ContextDecorator + +class mycontext(ContextDecorator): + def __enter__(self): + print('Starting') + return self + + def __exit__(self, *exc): + print('Finishing') + return False +</pre> <p>The class can then be used like this:</p> <pre data-language="python">>>> @mycontext() +... def function(): +... print('The bit in the middle') +... +>>> function() +Starting +The bit in the middle +Finishing + +>>> with mycontext(): +... print('The bit in the middle') +... +Starting +The bit in the middle +Finishing +</pre> <p>This change is just syntactic sugar for any construct of the following form:</p> <pre data-language="python">def f(): + with cm(): + # Do stuff +</pre> <p><code>ContextDecorator</code> lets you instead write:</p> <pre data-language="python">@cm() +def f(): + # Do stuff +</pre> <p>It makes it clear that the <code>cm</code> applies to the whole function, rather than just a piece of it (and saving an indentation level is nice, too).</p> <p>Existing context managers that already have a base class can be extended by using <code>ContextDecorator</code> as a mixin class:</p> <pre data-language="python">from contextlib import ContextDecorator + +class mycontext(ContextBaseClass, ContextDecorator): + def __enter__(self): + return self + + def __exit__(self, *exc): + return False +</pre> <div class="admonition note"> <p class="admonition-title">Note</p> <p>As the decorated function must be able to be called multiple times, the underlying context manager must support use in multiple <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statements. If this is not the case, then the original construct with the explicit <code>with</code> statement inside the function should be used.</p> </div> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.2.</span></p> </div> </dd> +</dl> <dl class="py class"> <dt class="sig sig-object py" id="contextlib.AsyncContextDecorator"> +<code>class contextlib.AsyncContextDecorator</code> </dt> <dd> +<p>Similar to <a class="reference internal" href="#contextlib.ContextDecorator" title="contextlib.ContextDecorator"><code>ContextDecorator</code></a> but only for asynchronous functions.</p> <p>Example of <code>AsyncContextDecorator</code>:</p> <pre data-language="python">from asyncio import run +from contextlib import AsyncContextDecorator + +class mycontext(AsyncContextDecorator): + async def __aenter__(self): + print('Starting') + return self + + async def __aexit__(self, *exc): + print('Finishing') + return False +</pre> <p>The class can then be used like this:</p> <pre data-language="python">>>> @mycontext() +... async def function(): +... print('The bit in the middle') +... +>>> run(function()) +Starting +The bit in the middle +Finishing + +>>> async def function(): +... async with mycontext(): +... print('The bit in the middle') +... +>>> run(function()) +Starting +The bit in the middle +Finishing +</pre> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.10.</span></p> </div> </dd> +</dl> <dl class="py class"> <dt class="sig sig-object py" id="contextlib.ExitStack"> +<code>class contextlib.ExitStack</code> </dt> <dd> +<p>A context manager that is designed to make it easy to programmatically combine other context managers and cleanup functions, especially those that are optional or otherwise driven by input data.</p> <p>For example, a set of files may easily be handled in a single with statement as follows:</p> <pre data-language="python">with ExitStack() as stack: + files = [stack.enter_context(open(fname)) for fname in filenames] + # All opened files will automatically be closed at the end of + # the with statement, even if attempts to open files later + # in the list raise an exception +</pre> <p>The <a class="reference internal" href="../reference/datamodel#object.__enter__" title="object.__enter__"><code>__enter__()</code></a> method returns the <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a> instance, and performs no additional operations.</p> <p>Each instance maintains a stack of registered callbacks that are called in reverse order when the instance is closed (either explicitly or implicitly at the end of a <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement). Note that callbacks are <em>not</em> invoked implicitly when the context stack instance is garbage collected.</p> <p>This stack model is used so that context managers that acquire their resources in their <code>__init__</code> method (such as file objects) can be handled correctly.</p> <p>Since registered callbacks are invoked in the reverse order of registration, this ends up behaving as if multiple nested <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statements had been used with the registered set of callbacks. This even extends to exception handling - if an inner callback suppresses or replaces an exception, then outer callbacks will be passed arguments based on that updated state.</p> <p>This is a relatively low level API that takes care of the details of correctly unwinding the stack of exit callbacks. It provides a suitable foundation for higher level context managers that manipulate the exit stack in application specific ways.</p> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.3.</span></p> </div> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.ExitStack.enter_context"> +<code>enter_context(cm)</code> </dt> <dd> +<p>Enters a new context manager and adds its <a class="reference internal" href="../reference/datamodel#object.__exit__" title="object.__exit__"><code>__exit__()</code></a> method to the callback stack. The return value is the result of the context manager’s own <a class="reference internal" href="../reference/datamodel#object.__enter__" title="object.__enter__"><code>__enter__()</code></a> method.</p> <p>These context managers may suppress exceptions just as they normally would if used directly as part of a <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement.</p> <div class="versionchanged"> <p><span class="versionmodified changed">Changed in version 3.11: </span>Raises <a class="reference internal" href="exceptions#TypeError" title="TypeError"><code>TypeError</code></a> instead of <a class="reference internal" href="exceptions#AttributeError" title="AttributeError"><code>AttributeError</code></a> if <em>cm</em> is not a context manager.</p> </div> </dd> +</dl> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.ExitStack.push"> +<code>push(exit)</code> </dt> <dd> +<p>Adds a context manager’s <a class="reference internal" href="../reference/datamodel#object.__exit__" title="object.__exit__"><code>__exit__()</code></a> method to the callback stack.</p> <p>As <code>__enter__</code> is <em>not</em> invoked, this method can be used to cover part of an <a class="reference internal" href="../reference/datamodel#object.__enter__" title="object.__enter__"><code>__enter__()</code></a> implementation with a context manager’s own <a class="reference internal" href="../reference/datamodel#object.__exit__" title="object.__exit__"><code>__exit__()</code></a> method.</p> <p>If passed an object that is not a context manager, this method assumes it is a callback with the same signature as a context manager’s <a class="reference internal" href="../reference/datamodel#object.__exit__" title="object.__exit__"><code>__exit__()</code></a> method and adds it directly to the callback stack.</p> <p>By returning true values, these callbacks can suppress exceptions the same way context manager <a class="reference internal" href="../reference/datamodel#object.__exit__" title="object.__exit__"><code>__exit__()</code></a> methods can.</p> <p>The passed in object is returned from the function, allowing this method to be used as a function decorator.</p> </dd> +</dl> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.ExitStack.callback"> +<code>callback(callback, /, *args, **kwds)</code> </dt> <dd> +<p>Accepts an arbitrary callback function and arguments and adds it to the callback stack.</p> <p>Unlike the other methods, callbacks added this way cannot suppress exceptions (as they are never passed the exception details).</p> <p>The passed in callback is returned from the function, allowing this method to be used as a function decorator.</p> </dd> +</dl> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.ExitStack.pop_all"> +<code>pop_all()</code> </dt> <dd> +<p>Transfers the callback stack to a fresh <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a> instance and returns it. No callbacks are invoked by this operation - instead, they will now be invoked when the new stack is closed (either explicitly or implicitly at the end of a <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement).</p> <p>For example, a group of files can be opened as an “all or nothing” operation as follows:</p> <pre data-language="python">with ExitStack() as stack: + files = [stack.enter_context(open(fname)) for fname in filenames] + # Hold onto the close method, but don't call it yet. + close_files = stack.pop_all().close + # If opening any file fails, all previously opened files will be + # closed automatically. If all files are opened successfully, + # they will remain open even after the with statement ends. + # close_files() can then be invoked explicitly to close them all. +</pre> </dd> +</dl> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.ExitStack.close"> +<code>close()</code> </dt> <dd> +<p>Immediately unwinds the callback stack, invoking callbacks in the reverse order of registration. For any context managers and exit callbacks registered, the arguments passed in will indicate that no exception occurred.</p> </dd> +</dl> </dd> +</dl> <dl class="py class"> <dt class="sig sig-object py" id="contextlib.AsyncExitStack"> +<code>class contextlib.AsyncExitStack</code> </dt> <dd> +<p>An <a class="reference internal" href="../reference/datamodel#async-context-managers"><span class="std std-ref">asynchronous context manager</span></a>, similar to <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a>, that supports combining both synchronous and asynchronous context managers, as well as having coroutines for cleanup logic.</p> <p>The <a class="reference internal" href="#contextlib.ExitStack.close" title="contextlib.ExitStack.close"><code>close()</code></a> method is not implemented; <a class="reference internal" href="#contextlib.AsyncExitStack.aclose" title="contextlib.AsyncExitStack.aclose"><code>aclose()</code></a> must be used instead.</p> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.AsyncExitStack.enter_async_context"> +<code>coroutine enter_async_context(cm)</code> </dt> <dd> +<p>Similar to <a class="reference internal" href="#contextlib.ExitStack.enter_context" title="contextlib.ExitStack.enter_context"><code>ExitStack.enter_context()</code></a> but expects an asynchronous context manager.</p> <div class="versionchanged"> <p><span class="versionmodified changed">Changed in version 3.11: </span>Raises <a class="reference internal" href="exceptions#TypeError" title="TypeError"><code>TypeError</code></a> instead of <a class="reference internal" href="exceptions#AttributeError" title="AttributeError"><code>AttributeError</code></a> if <em>cm</em> is not an asynchronous context manager.</p> </div> </dd> +</dl> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.AsyncExitStack.push_async_exit"> +<code>push_async_exit(exit)</code> </dt> <dd> +<p>Similar to <a class="reference internal" href="#contextlib.ExitStack.push" title="contextlib.ExitStack.push"><code>ExitStack.push()</code></a> but expects either an asynchronous context manager or a coroutine function.</p> </dd> +</dl> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.AsyncExitStack.push_async_callback"> +<code>push_async_callback(callback, /, *args, **kwds)</code> </dt> <dd> +<p>Similar to <a class="reference internal" href="#contextlib.ExitStack.callback" title="contextlib.ExitStack.callback"><code>ExitStack.callback()</code></a> but expects a coroutine function.</p> </dd> +</dl> <dl class="py method"> <dt class="sig sig-object py" id="contextlib.AsyncExitStack.aclose"> +<code>coroutine aclose()</code> </dt> <dd> +<p>Similar to <a class="reference internal" href="#contextlib.ExitStack.close" title="contextlib.ExitStack.close"><code>ExitStack.close()</code></a> but properly handles awaitables.</p> </dd> +</dl> <p>Continuing the example for <a class="reference internal" href="#contextlib.asynccontextmanager" title="contextlib.asynccontextmanager"><code>asynccontextmanager()</code></a>:</p> <pre data-language="python">async with AsyncExitStack() as stack: + connections = [await stack.enter_async_context(get_connection()) + for i in range(5)] + # All opened connections will automatically be released at the end of + # the async with statement, even if attempts to open a connection + # later in the list raise an exception. +</pre> <div class="versionadded"> <p><span class="versionmodified added">New in version 3.7.</span></p> </div> </dd> +</dl> </section> <section id="examples-and-recipes"> <h2>Examples and Recipes</h2> <p>This section describes some examples and recipes for making effective use of the tools provided by <a class="reference internal" href="#module-contextlib" title="contextlib: Utilities for with-statement contexts."><code>contextlib</code></a>.</p> <section id="supporting-a-variable-number-of-context-managers"> <h3>Supporting a variable number of context managers</h3> <p>The primary use case for <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a> is the one given in the class documentation: supporting a variable number of context managers and other cleanup operations in a single <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement. The variability may come from the number of context managers needed being driven by user input (such as opening a user specified collection of files), or from some of the context managers being optional:</p> <pre data-language="python">with ExitStack() as stack: + for resource in resources: + stack.enter_context(resource) + if need_special_resource(): + special = acquire_special_resource() + stack.callback(release_special_resource, special) + # Perform operations that use the acquired resources +</pre> <p>As shown, <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a> also makes it quite easy to use <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statements to manage arbitrary resources that don’t natively support the context management protocol.</p> </section> <section id="catching-exceptions-from-enter-methods"> <h3>Catching exceptions from <code>__enter__</code> methods</h3> <p>It is occasionally desirable to catch exceptions from an <code>__enter__</code> method implementation, <em>without</em> inadvertently catching exceptions from the <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement body or the context manager’s <code>__exit__</code> method. By using <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a> the steps in the context management protocol can be separated slightly in order to allow this:</p> <pre data-language="python">stack = ExitStack() +try: + x = stack.enter_context(cm) +except Exception: + # handle __enter__ exception +else: + with stack: + # Handle normal case +</pre> <p>Actually needing to do this is likely to indicate that the underlying API should be providing a direct resource management interface for use with <a class="reference internal" href="../reference/compound_stmts#try"><code>try</code></a>/<a class="reference internal" href="../reference/compound_stmts#except"><code>except</code></a>/<a class="reference internal" href="../reference/compound_stmts#finally"><code>finally</code></a> statements, but not all APIs are well designed in that regard. When a context manager is the only resource management API provided, then <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a> can make it easier to handle various situations that can’t be handled directly in a <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement.</p> </section> <section id="cleaning-up-in-an-enter-implementation"> <h3>Cleaning up in an <code>__enter__</code> implementation</h3> <p>As noted in the documentation of <a class="reference internal" href="#contextlib.ExitStack.push" title="contextlib.ExitStack.push"><code>ExitStack.push()</code></a>, this method can be useful in cleaning up an already allocated resource if later steps in the <a class="reference internal" href="../reference/datamodel#object.__enter__" title="object.__enter__"><code>__enter__()</code></a> implementation fail.</p> <p>Here’s an example of doing this for a context manager that accepts resource acquisition and release functions, along with an optional validation function, and maps them to the context management protocol:</p> <pre data-language="python">from contextlib import contextmanager, AbstractContextManager, ExitStack + +class ResourceManager(AbstractContextManager): + + def __init__(self, acquire_resource, release_resource, check_resource_ok=None): + self.acquire_resource = acquire_resource + self.release_resource = release_resource + if check_resource_ok is None: + def check_resource_ok(resource): + return True + self.check_resource_ok = check_resource_ok + + @contextmanager + def _cleanup_on_error(self): + with ExitStack() as stack: + stack.push(self) + yield + # The validation check passed and didn't raise an exception + # Accordingly, we want to keep the resource, and pass it + # back to our caller + stack.pop_all() + + def __enter__(self): + resource = self.acquire_resource() + with self._cleanup_on_error(): + if not self.check_resource_ok(resource): + msg = "Failed validation for {!r}" + raise RuntimeError(msg.format(resource)) + return resource + + def __exit__(self, *exc_details): + # We don't need to duplicate any of our resource release logic + self.release_resource() +</pre> </section> <section id="replacing-any-use-of-try-finally-and-flag-variables"> <h3>Replacing any use of <code>try-finally</code> and flag variables</h3> <p>A pattern you will sometimes see is a <code>try-finally</code> statement with a flag variable to indicate whether or not the body of the <code>finally</code> clause should be executed. In its simplest form (that can’t already be handled just by using an <code>except</code> clause instead), it looks something like this:</p> <pre data-language="python">cleanup_needed = True +try: + result = perform_operation() + if result: + cleanup_needed = False +finally: + if cleanup_needed: + cleanup_resources() +</pre> <p>As with any <code>try</code> statement based code, this can cause problems for development and review, because the setup code and the cleanup code can end up being separated by arbitrarily long sections of code.</p> <p><a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a> makes it possible to instead register a callback for execution at the end of a <code>with</code> statement, and then later decide to skip executing that callback:</p> <pre data-language="python">from contextlib import ExitStack + +with ExitStack() as stack: + stack.callback(cleanup_resources) + result = perform_operation() + if result: + stack.pop_all() +</pre> <p>This allows the intended cleanup up behaviour to be made explicit up front, rather than requiring a separate flag variable.</p> <p>If a particular application uses this pattern a lot, it can be simplified even further by means of a small helper class:</p> <pre data-language="python">from contextlib import ExitStack + +class Callback(ExitStack): + def __init__(self, callback, /, *args, **kwds): + super().__init__() + self.callback(callback, *args, **kwds) + + def cancel(self): + self.pop_all() + +with Callback(cleanup_resources) as cb: + result = perform_operation() + if result: + cb.cancel() +</pre> <p>If the resource cleanup isn’t already neatly bundled into a standalone function, then it is still possible to use the decorator form of <a class="reference internal" href="#contextlib.ExitStack.callback" title="contextlib.ExitStack.callback"><code>ExitStack.callback()</code></a> to declare the resource cleanup in advance:</p> <pre data-language="python">from contextlib import ExitStack + +with ExitStack() as stack: + @stack.callback + def cleanup_resources(): + ... + result = perform_operation() + if result: + stack.pop_all() +</pre> <p>Due to the way the decorator protocol works, a callback function declared this way cannot take any parameters. Instead, any resources to be released must be accessed as closure variables.</p> </section> <section id="using-a-context-manager-as-a-function-decorator"> <h3>Using a context manager as a function decorator</h3> <p><a class="reference internal" href="#contextlib.ContextDecorator" title="contextlib.ContextDecorator"><code>ContextDecorator</code></a> makes it possible to use a context manager in both an ordinary <code>with</code> statement and also as a function decorator.</p> <p>For example, it is sometimes useful to wrap functions or groups of statements with a logger that can track the time of entry and time of exit. Rather than writing both a function decorator and a context manager for the task, inheriting from <a class="reference internal" href="#contextlib.ContextDecorator" title="contextlib.ContextDecorator"><code>ContextDecorator</code></a> provides both capabilities in a single definition:</p> <pre data-language="python">from contextlib import ContextDecorator +import logging + +logging.basicConfig(level=logging.INFO) + +class track_entry_and_exit(ContextDecorator): + def __init__(self, name): + self.name = name + + def __enter__(self): + logging.info('Entering: %s', self.name) + + def __exit__(self, exc_type, exc, exc_tb): + logging.info('Exiting: %s', self.name) +</pre> <p>Instances of this class can be used as both a context manager:</p> <pre data-language="python">with track_entry_and_exit('widget loader'): + print('Some time consuming activity goes here') + load_widget() +</pre> <p>And also as a function decorator:</p> <pre data-language="python">@track_entry_and_exit('widget loader') +def activity(): + print('Some time consuming activity goes here') + load_widget() +</pre> <p>Note that there is one additional limitation when using context managers as function decorators: there’s no way to access the return value of <a class="reference internal" href="../reference/datamodel#object.__enter__" title="object.__enter__"><code>__enter__()</code></a>. If that value is needed, then it is still necessary to use an explicit <code>with</code> statement.</p> <div class="admonition seealso"> <p class="admonition-title">See also</p> <dl class="simple"> <dt> +<span class="target" id="index-0"></span><a class="pep reference external" href="https://peps.python.org/pep-0343/"><strong>PEP 343</strong></a> - The “with” statement</dt> +<dd> +<p>The specification, background, and examples for the Python <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement.</p> </dd> </dl> </div> </section> </section> <section id="single-use-reusable-and-reentrant-context-managers"> <span id="single-use-reusable-and-reentrant-cms"></span><h2>Single use, reusable and reentrant context managers</h2> <p>Most context managers are written in a way that means they can only be used effectively in a <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement once. These single use context managers must be created afresh each time they’re used - attempting to use them a second time will trigger an exception or otherwise not work correctly.</p> <p>This common limitation means that it is generally advisable to create context managers directly in the header of the <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement where they are used (as shown in all of the usage examples above).</p> <p>Files are an example of effectively single use context managers, since the first <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statement will close the file, preventing any further IO operations using that file object.</p> <p>Context managers created using <a class="reference internal" href="#contextlib.contextmanager" title="contextlib.contextmanager"><code>contextmanager()</code></a> are also single use context managers, and will complain about the underlying generator failing to yield if an attempt is made to use them a second time:</p> <pre data-language="python">>>> from contextlib import contextmanager +>>> @contextmanager +... def singleuse(): +... print("Before") +... yield +... print("After") +... +>>> cm = singleuse() +>>> with cm: +... pass +... +Before +After +>>> with cm: +... pass +... +Traceback (most recent call last): + ... +RuntimeError: generator didn't yield +</pre> <section id="reentrant-context-managers"> <span id="reentrant-cms"></span><h3>Reentrant context managers</h3> <p>More sophisticated context managers may be “reentrant”. These context managers can not only be used in multiple <a class="reference internal" href="../reference/compound_stmts#with"><code>with</code></a> statements, but may also be used <em>inside</em> a <code>with</code> statement that is already using the same context manager.</p> <p><a class="reference internal" href="threading#threading.RLock" title="threading.RLock"><code>threading.RLock</code></a> is an example of a reentrant context manager, as are <a class="reference internal" href="#contextlib.suppress" title="contextlib.suppress"><code>suppress()</code></a>, <a class="reference internal" href="#contextlib.redirect_stdout" title="contextlib.redirect_stdout"><code>redirect_stdout()</code></a>, and <a class="reference internal" href="#contextlib.chdir" title="contextlib.chdir"><code>chdir()</code></a>. Here’s a very simple example of reentrant use:</p> <pre data-language="python">>>> from contextlib import redirect_stdout +>>> from io import StringIO +>>> stream = StringIO() +>>> write_to_stream = redirect_stdout(stream) +>>> with write_to_stream: +... print("This is written to the stream rather than stdout") +... with write_to_stream: +... print("This is also written to the stream") +... +>>> print("This is written directly to stdout") +This is written directly to stdout +>>> print(stream.getvalue()) +This is written to the stream rather than stdout +This is also written to the stream +</pre> <p>Real world examples of reentrancy are more likely to involve multiple functions calling each other and hence be far more complicated than this example.</p> <p>Note also that being reentrant is <em>not</em> the same thing as being thread safe. <a class="reference internal" href="#contextlib.redirect_stdout" title="contextlib.redirect_stdout"><code>redirect_stdout()</code></a>, for example, is definitely not thread safe, as it makes a global modification to the system state by binding <a class="reference internal" href="sys#sys.stdout" title="sys.stdout"><code>sys.stdout</code></a> to a different stream.</p> </section> <section id="reusable-context-managers"> <span id="reusable-cms"></span><h3>Reusable context managers</h3> <p>Distinct from both single use and reentrant context managers are “reusable” context managers (or, to be completely explicit, “reusable, but not reentrant” context managers, since reentrant context managers are also reusable). These context managers support being used multiple times, but will fail (or otherwise not work correctly) if the specific context manager instance has already been used in a containing with statement.</p> <p><a class="reference internal" href="threading#threading.Lock" title="threading.Lock"><code>threading.Lock</code></a> is an example of a reusable, but not reentrant, context manager (for a reentrant lock, it is necessary to use <a class="reference internal" href="threading#threading.RLock" title="threading.RLock"><code>threading.RLock</code></a> instead).</p> <p>Another example of a reusable, but not reentrant, context manager is <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a>, as it invokes <em>all</em> currently registered callbacks when leaving any with statement, regardless of where those callbacks were added:</p> <pre data-language="python">>>> from contextlib import ExitStack +>>> stack = ExitStack() +>>> with stack: +... stack.callback(print, "Callback: from first context") +... print("Leaving first context") +... +Leaving first context +Callback: from first context +>>> with stack: +... stack.callback(print, "Callback: from second context") +... print("Leaving second context") +... +Leaving second context +Callback: from second context +>>> with stack: +... stack.callback(print, "Callback: from outer context") +... with stack: +... stack.callback(print, "Callback: from inner context") +... print("Leaving inner context") +... print("Leaving outer context") +... +Leaving inner context +Callback: from inner context +Callback: from outer context +Leaving outer context +</pre> <p>As the output from the example shows, reusing a single stack object across multiple with statements works correctly, but attempting to nest them will cause the stack to be cleared at the end of the innermost with statement, which is unlikely to be desirable behaviour.</p> <p>Using separate <a class="reference internal" href="#contextlib.ExitStack" title="contextlib.ExitStack"><code>ExitStack</code></a> instances instead of reusing a single instance avoids that problem:</p> <pre data-language="python">>>> from contextlib import ExitStack +>>> with ExitStack() as outer_stack: +... outer_stack.callback(print, "Callback: from outer context") +... with ExitStack() as inner_stack: +... inner_stack.callback(print, "Callback: from inner context") +... print("Leaving inner context") +... print("Leaving outer context") +... +Leaving inner context +Callback: from inner context +Leaving outer context +Callback: from outer context +</pre> </section> </section> <div class="_attribution"> + <p class="_attribution-p"> + © 2001–2023 Python Software Foundation<br>Licensed under the PSF License.<br> + <a href="https://docs.python.org/3.12/library/contextlib.html" class="_attribution-link">https://docs.python.org/3.12/library/contextlib.html</a> + </p> +</div> |