Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES/1870.doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added documentation and examples for exception handling in background tasks created with ``asyncio.create_task()``, addressing silent failure patterns in high-throughput async systems -- by :user:`tsayin4`.
1 change: 1 addition & 0 deletions CONTRIBUTORS.txt
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ Günther Jena
Hans Adema
Harmon Y.
Harry Liu
Hilmi Tuna Sayın
Hiroshi Ogawa
Hrishikesh Paranjape
Hu Bo
Expand Down
50 changes: 50 additions & 0 deletions docs/client_advanced.rst
Original file line number Diff line number Diff line change
Expand Up @@ -841,6 +841,56 @@ In the end all you have to do is to close all sessions after the `yield` stateme
session_3.close(),
)

Exception Handling in Background Tasks
---------------------------------------

When using ``asyncio.create_task()`` with aiohttp requests, exceptions
raised during the request will not propagate to the spawning context:

.. code-block:: python

async def fetch_in_background():
async with aiohttp.ClientSession() as session:
# This exception will be logged but not propagated
resp = await session.get('http://invalid-url')

# Caller is unaware of the failure
asyncio.create_task(fetch_in_background())

This behavior can lead to silent failures in production systems. To
handle exceptions properly, retain the task reference and await it:

.. code-block:: python

async with aiohttp.ClientSession() as session:
task = asyncio.create_task(session.get('http://example.com'))
try:
resp = await task
except aiohttp.ClientError as e:
# Handle failure explicitly
logger.error(f"Background request failed: {e}")

For fire-and-forget patterns, use ``add_done_callback()`` to handle
exceptions without blocking:

.. code-block:: python

def handle_response(task: asyncio.Task):
try:
task.result()
except aiohttp.ClientError as e:
logger.error(f"Request failed: {e}")

task = asyncio.create_task(session.get('http://example.com'))
task.add_done_callback(handle_response)

.. warning::
When using ``asyncio.create_task()`` with aiohttp requests, ensure
that exceptions are explicitly handled. Background task failures
do not propagate to the spawning context by default and may lead
to silent failures in production systems.


Graceful Shutdown
-----------------

Expand Down
4 changes: 4 additions & 0 deletions docs/client_quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,10 @@ should be called in this case, e.g.::
# ...
await session.close()

.. warning::
When spawning requests as background tasks using ``asyncio.create_task()``,
exceptions must be explicitly handled. See :ref:`aiohttp-client-advanced`
for details on background task exception handling.

Passing Parameters In URLs
==========================
Expand Down
75 changes: 75 additions & 0 deletions examples/client_background_tasks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/usr/bin/env python3
"""
Background Task Exception Handling
===================================

Demonstrates proper exception handling for background HTTP requests.
"""

import asyncio
import logging

import aiohttp

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


async def unsafe_pattern():
"""Fire-and-forget without exception handling (unsafe)."""
async with aiohttp.ClientSession() as session:
# This will fail, but exception only logged to event loop
asyncio.create_task(session.get("http://invalid-nonexistent-domain-12345.com"))
logger.info("Task spawned, continuing without waiting...")


async def safe_pattern_callback():
"""Fire-and-forget with callback exception handling (safe)."""

def handle_result(task: asyncio.Task) -> None:
try:
resp = task.result()
logger.info(f"Background request completed with status {resp.status}")
resp.release() # Critical: release connection
except aiohttp.ClientError as e:
logger.error(f"Background request failed: {e}")

async with aiohttp.ClientSession() as session:
task = asyncio.create_task(session.get("http://httpbin.org/status/500"))
task.add_done_callback(handle_result)
await asyncio.sleep(2) # Wait for task to complete


async def safe_pattern_gather():
"""Multiple background tasks using asyncio.gather()."""
async with aiohttp.ClientSession() as session:
tasks = [
session.get("http://httpbin.org/status/200"),
session.get("http://httpbin.org/status/500"),
session.get("http://invalid-domain-12345.com"),
]

results = await asyncio.gather(*tasks, return_exceptions=True)

for i, result in enumerate(results):
if isinstance(result, Exception):
logger.error(f"Task {i} failed: {result}")
else:
logger.info(f"Task {i} succeeded: {result.status}")
result.release() # Critical: release connection


async def main():
logger.info("=== UNSAFE PATTERN ===")
await unsafe_pattern()
await asyncio.sleep(3) # Wait to see exception logged

logger.info("\n=== SAFE PATTERN (CALLBACK) ===")
await safe_pattern_callback()

logger.info("\n=== SAFE PATTERN (GATHER) ===")
await safe_pattern_gather()


if __name__ == "__main__":
asyncio.run(main())
Loading