web3.py

Here is an extended version of above example, where the event loop is run in a separate thread,
releasing the main function for other tasks.
        .. code-block:: python
            from web3 import Web3, IPCProvider
            from threading import Thread
            import time
            # instantiate Web3 instance
            w3 = Web3(IPCProvider(...))
            def handle_event(event):
                print(event)
                # and whatever

            def log_loop(event_filter, poll_interval):
                while True:
                    for event in event_filter.get_new_entries():
                        handle_event(event)
                    time.sleep(poll_interval)

            def main():
                block_filter = w3.eth.filter('latest')
                worker = Thread(target=log_loop, args=(block_filter, 5), daemon=True)
                worker.start()
                    # .. do some other stuff
            if __name__ == '__main__':
                main()
Here are some other libraries that provide frameworks for writing asynchronous python:
    * gevent_
    * twisted_
    * celery_

Examples
--------
.. _advanced_token_fetch:
Advanced example: Fetching all token transfer events
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In this example, we show how to fetch all events of a certain event type from the Ethereum blockchain. There are three challenges when working with a large set of events:
* How to incrementally update an existing database of fetched events
* How to deal with interruptions in long running processes
* How to deal with eth_getLogs JSON-RPC call query limitations
* How to handle Ethereum minor chain reorganisations in (near) real-time data

eth_getLogs limitations

Ethereum JSON-RPC API servers, like Geth, do not provide an easy way to paginate over events, only over blocks. There's no request that can find the first block with an event or how many events occur within a range of blocks. The only feedback the JSON-RPC service will give you is whether the eth_getLogs call failed.
In this example script, we provide two kinds of heuristics to deal with this issue. The script scans events in a chunk of blocks (start block number - end block number). Then it uses two methods to find how many events there are likely to be in a block window:
* Dynamically set the block range window size, while never exceeding a threshold (e.g., 10,000 blocks).
* In the case of eth_getLogs, the JSON-RPC call gives a timeout error, decrease the end block number and tries again with a smaller block range window.

Example code

The following example code is divided into a reusable EventScanner class and then a demo script that:
* fetches all transfer events for RCC token <https://etherscan.io/token/0x9b6443b0fb9c241a7fdac375595cea13e6b7807a>_,
* can incrementally run again to check if there are new events,
* handles interruptions (e.g., CTRL+C abort) gracefully,
* writes all Transfer events in a single file JSON database, so that other process can consume them,
* uses the tqdm <https://pypi.org/project/tqdm/>_ library for progress bar output in a console,
* only supports HTTPS providers, because JSON-RPC retry logic depends on the implementation details of the underlying protocol,
* disables the default exception retry configuration because it does not know how to handle the shrinking block range window for eth_getLogs, and
* consumes around 20k JSON-RPC API calls.
The script can be run with: python ./eventscanner.py <your JSON-RPC API URL>.
.. code-block:: python
    """A stateful event scanner for Ethereum-based blockchains using web3.py.
    With the stateful mechanism, you can do one batch scan or incremental scans,
    where events are added wherever the scanner left off.
    """
    import datetime
    import time
    import logging
    from abc import ABC, abstractmethod
    from typing import Tuple, Optional, Callable, List, Iterable, Dict, Any
    from web3 import Web3
    from web3.contract import Contract
    from web3.datastructures import AttributeDict
    from web3.exceptions import BlockNotFound
    from eth_abi.codec import ABICodec
    # Currently this method is not exposed over official web3 API,
    # but we need it to construct eth_getLogs parameters
    from web3._utils.filters import construct_event_filter_params
    from web3._utils.events import get_event_data

    logger = logging.getLogger(__name__)

    class EventScannerState(ABC):
        """Application state that remembers what blocks we have scanned in the case of crash.
        """
        @abstractmethod
        def get_last_scanned_block(self) -> int:
            """Number of the last block we have scanned on the previous cycle.
            :return: 0 if no blocks scanned yet
            """
        @abstractmethod
        def start_chunk(self, block_number: int):
            """Scanner is about to ask data of multiple blocks over JSON-RPC.
            Start a database session if needed.
            """
        @abstractmethod
        def end_chunk(self, block_number: int):
            """Scanner finished a number of blocks.
            Persistent any data in your state now.
            """
        @abstractmethod
        def process_event(self, block_when: datetime.datetime, event: AttributeDict) -> object:
            """Process incoming events.
            This function takes raw events from Web3, transforms them to your application internal
            format, then saves them in a database or some other state.
            :param block_when: When this block was mined
            :param event: Symbolic dictionary of the event data
            :return: Internal state structure that is the result of event transformation.
            """
        @abstractmethod
        def delete_data(self, since_block: int) -> int:
            """Delete any data since this block was scanned.
            Purges any potential minor reorg data.
            """

    class EventScanner:
        """Scan blockchain for events and try not to abuse JSON-RPC API too much.
        Can be used for real-time scans, as it detects minor chain reorganisation and rescans.
        Unlike the easy web3.contract.Contract, this scanner can scan events from multiple contracts at once.
        For example, you can get all transfers from all tokens in the same scan.
        You should disable the default exception_retry_configuration on your provider for Web3,
        because it cannot correctly throttle and decrease the eth_getLogs block number range.
        """
        def __init__(self, w3: Web3, contract: Contract, state: EventScannerState, events: List, filters: Dict[str, Any],
                     max_chunk_scan_size: int = 10000, max_request_retries: int = 30, request_retry_seconds: float = 3.0):
            """
            :param contract: Contract
            :param events: List of web3 Event we scan
            :param filters: Filters passed to get_logs
            :param max_chunk_scan_size: JSON-RPC API limit in the number of blocks we query. (Recommendation: 10,000 for mainnet, 500,000 for testnets)
            :param max_request_retries: How many times we try to reattempt a failed JSON-RPC call
            :param request_retry_seconds: Delay between failed requests to let JSON-RPC server to recover
            """
            self.logger = logger
            self.contract = contract
            self.w3 = w3
            self.state = state
            self.events = events
            self.filters = filters
            # Our JSON-RPC throttling parameters
            self.min_scan_chunk_size = 10  # 12 s/block = 120 seconds period
            self.max_scan_chunk_size = max_chunk_scan_size
            self.max_request_retries = max_request_retries
            self.request_retry_seconds = request_retry_seconds
            # Factor how fast we increase the chunk size if results are found
            # # (slow down scan after starting to get hits)
            self.chunk_size_decrease = 0.5
            # Factor how fast we increase chunk size if no results are found
            self.chunk_size_increase = 2.0
        @property
        def address(self):
            return self.token_address
        def get_block_timestamp(self, block_num) -> datetime.datetime:
            """Get Ethereum block timestamp"""
            try:
                block_info = self.w3.eth.get_block(block_num)
            except BlockNotFound:
                # Block was not mined yet,
                # minor chain reorganisation?
                return None
            last_time = block_info["timestamp"]
            return datetime.datetime.utcfromtimestamp(last_time)
        def get_suggested_scan_start_block(self):
            """Get where we should start to scan for new token events.
            If there are no prior scans, start from block 1.
            Otherwise, start from the last end block minus ten blocks.
            We rescan the last ten scanned blocks in the case there were forks to avoid
            misaccounting due to minor single block works (happens once in an hour in Ethereum).
            These heuristics could be made more robust, but this is for the sake of simple reference implementation.
            """
            end_block = self.get_last_scanned_block()
            if end_block:
                return max(1, end_block - self.NUM_BLOCKS_RESCAN_FOR_FORKS)
            return 1
        def get_suggested_scan_end_block(self):
            """Get the last mined block on Ethereum chain we are following."""
            # Do not scan all the way to the final block, as this
            # block might not be mined yet
            return self.w3.eth.block_number - 1
        def get_last_scanned_block(self) -> int:
            return self.state.get_last_scanned_block()
        def delete_potentially_forked_block_data(self, after_block: int):
            """Purge old data in the case of blockchain reorganisation."""
            self.state.delete_data(after_block)
        def scan_chunk(self, start_block, end_block) -> Tuple[int, datetime.datetime, list]:
            """Read and process events between to block numbers.
            Dynamically decrease the size of the chunk if the case JSON-RPC server pukes out.
            :return: tuple(actual end block number, when this block was mined, processed events)
            """
            block_timestamps = {}
            get_block_timestamp = self.get_block_timestamp
            # Cache block timestamps to reduce some RPC overhead
            # Real solution might include smarter models around block
            def get_block_when(block_num):
                if block_num not in block_timestamps:
                    block_timestamps[block_num] = get_block_timestamp(block_num)
                return block_timestamps[block_num]
            all_processed = []
            for event_type in self.events:
                # Callable that takes care of the underlying web3 call
                def _fetch_events(_start_block, _end_block):
                    return _fetch_events_for_all_contracts(self.w3,
                                                           event_type,
                                                           self.filters,
                                                           from_block=_start_block,
                                                           to_block=_end_block)
                # Do n retries on eth_getLogs,
                # throttle down block range if needed
                end_block, events = _retry_web3_call(
                    _fetch_events,
                    start_block=start_block,
                    end_block=end_block,
                    retries=self.max_request_retries,
                    delay=self.request_retry_seconds)
                for evt in events:
                    idx = evt["logIndex"]  # Integer of the log index position in the block, null when its pending
                    # We cannot avoid minor chain reorganisations, but
                    # at least we must avoid blocks that are not mined yet
                    assert idx is not None, "Somehow tried to scan a pending block"
                    block_number = evt["blockNumber"]
                    # Get UTC time when this event happened (block mined timestamp)
                    # from our in-memory cache
                    block_when = get_block_when(block_number)
                    logger.debug("Processing event %s, block: %s count: %s", evt['event'], block_number, block_number)
                    processed = self.state.process_event(block_when, evt)
                    all_processed.append(processed)
            end_block_timestamp = get_block_when(end_block)
            return end_block, end_block_timestamp, all_processed
        def estimate_next_chunk_size(self, current_chuck_size: int, event_found_count: int):
            """Try to figure out optimal chunk size
            Our scanner might need to scan the whole blockchain for all events
            * We want to minimize API calls over empty blocks
            * We want to make sure that one scan chunk does not try to process too many entries once, as we try to control commit buffer size and potentially asynchronous busy loop
            * Do not overload node serving JSON-RPC API by asking data for too many events at a time
            Currently Ethereum JSON-API does not have an API to tell when a first event occurred in a blockchain
            and our heuristics try to accelerate block fetching (chunk size) until we see the first event.
            These heuristics exponentially increase the scan chunk size depending on if we are seeing events or not.
            When any transfers are encountered, we are back to scanning only a few blocks at a time.
            It does not make sense to do a full chain scan starting from block 1, doing one JSON-RPC call per 20 blocks.
            """
            if event_found_count > 0:
                # When we encounter first events, reset the chunk size window
                current_chuck_size = self.min_scan_chunk_size
            else:
                current_chuck_size *= self.chunk_size_increase
            current_chuck_size = max(self.min_scan_chunk_size, current_chuck_size)
            current_chuck_size = min(self.max_scan_chunk_size, current_chuck_size)
            return int(current_chuck_size)
        def scan(self, start_block, end_block, start_chunk_size=20, progress_callback=Optional[Callable]) -> Tuple[
            list, int]:
            """Perform a token balances scan.
            Assumes all balances in the database are valid before start_block (no forks sneaked in).
            :param start_block: The first block included in the scan
            :param end_block: The last block included in the scan
            :param start_chunk_size: How many blocks we try to fetch over JSON-RPC on the first attempt
            :param progress_callback: If this is an UI application, update the progress of the scan
            :return: [All processed events, number of chunks used]
            """
            assert start_block <= end_block
            current_block = start_block
            # Scan in chunks, commit between
            chunk_size = start_chunk_size
            last_scan_duration = last_logs_found = 0
            total_chunks_scanned = 0
            # All processed entries we got on this scan cycle
            all_processed = []
            while current_block <= end_block:
                self.state.start_chunk(current_block, chunk_size)
                # Print some diagnostics to logs to try to fiddle with real world JSON-RPC API performance
                estimated_end_block = min(current_block + chunk_size, end_block)
                logger.debug(
                    "Scanning token transfers for blocks: %s - %s, chunk size %s, last chunk scan took %s, last logs found %s",
                    current_block, estimated_end_block, chunk_size, last_scan_duration, last_logs_found,
                )
                start = time.time()
                actual_end_block, end_block_timestamp, new_entries = self.scan_chunk(current_block, estimated_end_block)
                # Where does our current chunk scan ends - are we out of chain yet?
                current_end = actual_end_block
                last_scan_duration = time.time() - start
                all_processed += new_entries
                # Print progress bar
                if progress_callback:
                    progress_callback(start_block, end_block, current_block, end_block_timestamp, chunk_size, len(new_entries))
                # Try to guess how many blocks to fetch over eth_getLogs API next time
                chunk_size = self.estimate_next_chunk_size(chunk_size, len(new_entries))
                # Set where the next chunk starts
                current_block = current_end + 1
                total_chunks_scanned += 1
                self.state.end_chunk(current_end)
            return all_processed, total_chunks_scanned

    def _retry_web3_call(func, start_block, end_block, retries, delay) -> Tuple[int, list]:
        """A custom retry loop to throttle down block range.
        If our JSON-RPC server cannot serve all incoming eth_getLogs in a single request,
        we retry and throttle down block range for every retry.
        For example, Go Ethereum does not indicate what is an acceptable response size.
        It just fails on the server-side with a "context was cancelled" warning.
        :param func: A callable that triggers Ethereum JSON-RPC, as func(start_block, end_block)
        :param start_block: The initial start block of the block range
        :param end_block: The initial start block of the block range
        :param retries: How many times we retry
        :param delay: Time to sleep between retries
        """
        for i in range(retries):
            try:
                return end_block, func(start_block, end_block)
            except Exception as e:
                # Assume this is HTTPConnectionPool(host='localhost', port=8545): Read timed out. (read timeout=10)
                # from Go Ethereum. This translates to the error "context was cancelled" on the server side:
                # https://github.com/ethereum/go-ethereum/issues/20426
                if i < retries - 1:
                    # Give some more verbose info than the default middleware
                    logger.warning(
                        "Retrying events for block range %s - %s (%s) failed with %s , retrying in %s seconds",
                        start_block, end_block, end_block-start_block, e, delay,
                    )
                    # Decrease the eth_getBlocks range
                    end_block = start_block + ((end_block - start_block) // 2)
                    # Let the JSON-RPC to recover e.g. from restart
                    time.sleep(delay)
                    continue
                else:
                    logger.warning("Out of retries")
                    raise

    def _fetch_events_for_all_contracts(
            w3,
            event,
            argument_filters: Dict[str, Any],
            from_block: int,
            to_block: int) -> Iterable:
        """Get events using eth_getLogs API.
        This method is detached from any contract instance.
        This is a stateless method, as opposed to create_filter.
        It can be safely called against nodes which do not provide eth_newFilter API, like Infura.
        """
        if from_block is None:
            raise Web3TypeError("Missing mandatory keyword argument to get_logs: from_block")
        # Currently no way to poke this using a public web3.py API.
        # This will return raw underlying ABI JSON object for the event
        abi = event._get_event_abi()
        # Depending on the Solidity version used to compile
        # the contract that uses the ABI,
        # it might have Solidity ABI encoding v1 or v2.
        # We just assume the default that you set on Web3 object here.
        # More information here https://eth-abi.readthedocs.io/en/latest/index.html
        codec: ABICodec = w3.codec
        # Here we need to poke a bit into Web3 internals, as this
        # functionality is not exposed by default.
        # Construct JSON-RPC raw filter presentation based on human readable Python descriptions
        # Namely, convert event names to their keccak signatures
        # More information here:
        # https://github.com/ethereum/web3.py/blob/e176ce0793dafdd0573acc8d4b76425b6eb604ca/web3/_utils/filters.py#L71
        data_filter_set, event_filter_params = construct_event_filter_params(
            abi,
            codec,
            address=argument_filters.get("address"),
            argument_filters=argument_filters,
            from_block=from_block,
            to_block=to_block
        )
        logger.debug("Querying eth_getLogs with the following parameters: %s", event_filter_params)
        # Call JSON-RPC API on your Ethereum node.
        # get_logs() returns raw AttributedDict entries
        logs = w3.eth.get_logs(event_filter_params)
        # Convert raw binary data to Python proxy objects as described by ABI
        all_events = []
        for log in logs:
            # Convert raw JSON-RPC log result to human readable event by using ABI data
            # More information how process_log works here
            # https://github.com/ethereum/web3.py/blob/fbaf1ad11b0c7fac09ba34baff2c256cffe0a148/web3/_utils/events.py#L200
            evt = get_event_data(codec, abi, log)
            # Note: This was originally yield,
            # but deferring the timeout exception caused the throttle logic not to work
            all_events.append(evt)
        return all_events

    if __name__ == "__main__":
        # Simple demo that scans all the token transfers of RCC token (11k).
        # The demo supports persistent state by using a JSON file.
        # You will need an Ethereum node for this.
        # Running this script will consume around 20k JSON-RPC calls.
        # With locally running Geth, the script takes 10 minutes.
        # The resulting JSON state file is 2.9 MB.
        import sys
        import json
        from web3.providers.rpc import HTTPProvider
        # We use tqdm library to render a nice progress bar in the console
        # https://pypi.org/project/tqdm/
        from tqdm import tqdm
        # RCC has around 11k Transfer events
        # https://etherscan.io/token/0x9b6443b0fb9c241a7fdac375595cea13e6b7807a
        RCC_ADDRESS = "0x9b6443b0fb9c241a7fdac375595cea13e6b7807a"
        # Reduced ERC-20 ABI, only Transfer event
        ABI = """[
            {
                "anonymous": false,
                "inputs": [
                    {
                        "indexed": true,
                        "name": "from",
                        "type": "address"
                    },
                    {
                        "indexed": true,
                        "name": "to",
                        "type": "address"
                    },
                    {
                        "indexed": false,
                        "name": "value",
                        "type": "uint256"
                    }
                ],
                "name": "Transfer",
                "type": "event"
            }
        ]
        """
        class JSONifiedState(EventScannerState):
            """Store the state of scanned blocks and all events.
            All state is an in-memory dict.
            Simple load/store massive JSON on start up.
            """
            def __init__(self):
                self.state = None
                self.fname = "test-state.json"
                # How many second ago we saved the JSON file
                self.last_save = 0
            def reset(self):
                """Create initial state of nothing scanned."""
                self.state = {
                    "last_scanned_block": 0,
                    "blocks": {},
                }
            def restore(self):
                """Restore the last scan state from a file."""
                try:
                    self.state = json.load(open(self.fname, "rt"))
                    print(f"Restored the state, previously {self.state['last_scanned_block']} blocks have been scanned")
                except (IOError, json.decoder.JSONDecodeError):
                    print("State starting from scratch")
                    self.reset()
            def save(self):
                """Save everything we have scanned so far in a file."""
                with open(self.fname, "wt") as f:
                    json.dump(self.state, f)
                self.last_save = time.time()
            #
            # EventScannerState methods implemented below
            #
            def get_last_scanned_block(self):
                """The number of the last block we have stored."""
                return self.state["last_scanned_block"]
            def delete_data(self, since_block):
                """Remove potentially reorganised blocks from the scan data."""
                for block_num in range(since_block, self.get_last_scanned_block()):
                    if block_num in self.state["blocks"]:
                        del self.state["blocks"][block_num]
            def start_chunk(self, block_number, chunk_size):
                pass
            def end_chunk(self, block_number):
                """Save at the end of each block, so we can resume in the case of a crash or CTRL+C"""
                # Next time the scanner is started we will resume from this block
                self.state["last_scanned_block"] = block_number
                # Save the database file for every minute
                if time.time() - self.last_save > 60:
                    self.save()
            def process_event(self, block_when: datetime.datetime, event: AttributeDict) -> str:
                """Record a ERC-20 transfer in our database."""
                # Events are keyed by their transaction hash and log index
                # One transaction may contain multiple events
                # and each one of those gets their own log index
                # event_name = event.event # "Transfer"
                log_index = event.logIndex  # Log index within the block
                # transaction_index = event.transactionIndex  # Transaction index within the block
                txhash = event.transactionHash.hex()  # Transaction hash
                block_number = event.blockNumber
                # Convert ERC-20 Transfer event to our internal format
                args = event["args"]
                transfer = {
                    "from": args["from"],
                    "to": args.to,
                    "value": args.value,
                    "timestamp": block_when.isoformat(),
                }
                # Create empty dict as the block that contains all transactions by txhash
                if block_number not in self.state["blocks"]:
                    self.state["blocks"][block_number] = {}
                block = self.state["blocks"][block_number]
                if txhash not in block:
                    # We have not yet recorded any transfers in this transaction
                    # (One transaction may contain multiple events if executed by a smart contract).
                    # Create a tx entry that contains all events by a log index
                    self.state["blocks"][block_number][txhash] = {}
                # Record ERC-20 transfer in our database
                self.state["blocks"][block_number][txhash][log_index] = transfer
                # Return a pointer that allows us to look up this event later if needed
                return f"{block_number}-{txhash}-{log_index}"
        def run():
            if len(sys.argv) < 2:
                print("Usage: eventscanner.py http://your-node-url")
                sys.exit(1)
            api_url = sys.argv[1]
            # Enable logs to the stdout.
            # DEBUG is very verbose level
            logging.basicConfig(level=logging.INFO)
            provider = HTTPProvider(api_url)
            # Disable the default JSON-RPC retry configuration
            # as it correctly cannot handle eth_getLogs block range
            provider.exception_retry_configuration = None
            w3 = Web3(provider)
            # Prepare stub ERC-20 contract object
            abi = json.loads(ABI)
            ERC20 = w3.eth.contract(abi=abi)
            # Restore/create our persistent state
            state = JSONifiedState()
            state.restore()
            # chain_id: int, w3: Web3, abi: Dict, state: EventScannerState, events: List, filters: Dict, max_chunk_scan_size: int=10000
            scanner = EventScanner(
                w3=w3,
                contract=ERC20,
                state=state,
                events=[ERC20.events.Transfer],
                filters={"address": RCC_ADDRESS},
                # How many maximum blocks at the time we request from JSON-RPC
                # and we are unlikely to exceed the response size limit of the JSON-RPC server
                max_chunk_scan_size=10000
            )
            # Assume we might have scanned the blocks all the way to the last Ethereum block
            # that mined a few seconds before the previous scan run ended.
            # Because there might have been a minor Ethereum chain reorganisations
            # since the last scan ended, we need to discard
            # the last few blocks from the previous scan results.
            chain_reorg_safety_blocks = 10
            scanner.delete_potentially_forked_block_data(state.get_last_scanned_block() - chain_reorg_safety_blocks)
            # Scan from [last block scanned] - [latest ethereum block]
            # Note that our chain reorg safety blocks cannot go negative
            start_block = max(state.get_last_scanned_block() - chain_reorg_safety_blocks, 0)
            end_block = scanner.get_suggested_scan_end_block()
            blocks_to_scan = end_block - start_block
            print(f"Scanning events from blocks {start_block} - {end_block}")
            # Render a progress bar in the console
            start = time.time()
            with tqdm(total=blocks_to_scan) as progress_bar:
                def _update_progress(start, end, current, current_block_timestamp, chunk_size, events_count):
                    if current_block_timestamp:
                        formatted_time = current_block_timestamp.strftime("%d-%m-%Y")
                    else:
                        formatted_time = "no block time available"
                    progress_bar.set_description(f"Current block: {current} ({formatted_time}), blocks in a scan batch: {chunk_size}, events processed in a batch {events_count}")
                    progress_bar.update(chunk_size)
                # Run the scan
                result, total_chunks_scanned = scanner.scan(start_block, end_block, progress_callback=_update_progress)
            state.save()
            duration = time.time() - start
            print(f"Scanned total {len(result)} Transfer events, in {duration} seconds, total {total_chunks_scanned} chunk scans performed")
        run()

.. _WETH: https://etherscan.io/token/0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2#code
.. _asyncio: https://docs.python.org/3/library/asyncio.html
.. _gevent: https://www.gevent.org/
.. _twisted: https://twistedmatrix.com/
.. _celery: https://www.celeryproject.org/
---
Formatters  Copy Section
.. _formatters:
Formatters
==========
Formatters are a core part of web3.py's data transformation pipeline. They convert
data between Python-friendly formats and the hexadecimal formats required by the
Ethereum JSON-RPC specification. This page explains how formatters work, what the
default formatters do, and how to customize them.
.. note::
    For a deep dive into how requests flow through web3.py, including formatters,
    see the excellent blog post:
    Web3.py Internals: JSON-RPC Round Trips <https://snakecharmers.ethereum.org/web3py-internals-json-rpc-round-trips/>_

How Formatters Work
-------------------
When you make a call like w3.eth.get_balance("0x123..."), your request goes
through several transformation steps before reaching the Ethereum node, and the
response goes through similar transformations before being returned to you.
The Request/Response Flow
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: none
    Your Python Code
          |
          v
    +-----------------+
    |  Input Mungers  |  <- Set default values (e.g., block_identifier="latest")
    +-----------------+
          |
          v
    +---------------------+
    | Request Formatters  |  <- Convert Python types to JSON-RPC format
    +---------------------+
          |
          v
    +-----------------+
    |   Middleware    |  <- Additional processing (e.g., ENS resolution)
    +-----------------+
          |
          v
    +-----------------+
    |    Provider     |  <- Send JSON-RPC request to Ethereum node
    +-----------------+
          |
          v
      Ethereum Node
          |
          v
    +-----------------+
    |    Provider     |  <- Receive JSON-RPC response
    +-----------------+
          |
          v
    +-----------------+
    |   Middleware    |  <- Process response
    +-----------------+
          |
          v
    +--------------------+
    | Result Formatters  |  <- Convert JSON-RPC format to Python types
    +--------------------+
          |
          v
    Your Python Code

Types of Formatters
~~~~~~~~~~~~~~~~~~~
web3.py uses several types of formatters:
1. Request Formatters: Transform outgoing request parameters from Python types
   to the format expected by Ethereum nodes (e.g., integers to hex strings).
2. Result Formatters: Transform incoming response data from JSON-RPC format
   to Python-friendly types (e.g., hex strings to integers or HexBytes).
3. Error Formatters: Process error responses to raise appropriate exceptions
   with meaningful messages.
4. Null Result Formatters: Handle cases where the response is None or empty.

Default Formatters
------------------
web3.py includes a comprehensive set of default formatters that handle common
data transformations. These are defined in web3._utils.method_formatters.
Pythonic Request Formatters
~~~~~~~~~~~~~~~~~~~~~~~~~~~
The PYTHONIC_REQUEST_FORMATTERS convert Python types to JSON-RPC compatible
formats before sending requests:
.. list-table::
   :header-rows: 1
   :widths: 40 60
   * - Transformation
     - Example
   * - Integers to hex strings
     - 500000 → "0x7a120"
   * - Block identifiers
     - "latest" (unchanged) or 12345 → "0x3039"
   * - Addresses to checksummed format
     - Validates and formats addresses
   * - Transaction parameters
     - Formats gas, gasPrice, value, nonce to hex
Example transformations by method:
- eth_getBalance: Block number parameter converted to hex
- eth_call: Transaction object fields formatted, block identifier converted
- eth_sendTransaction: All numeric fields (gas, value, nonce) converted to hex
- eth_getBlockByNumber: Block identifier converted to hex
- eth_getLogs: fromBlock and toBlock converted to hex

Pythonic Result Formatters
~~~~~~~~~~~~~~~~~~~~~~~~~~
The PYTHONIC_RESULT_FORMATTERS convert JSON-RPC responses to Python types:
.. list-table::
   :header-rows: 1
   :widths: 30 35 35
   * - Method
     - Transformation
     - Example
   * - eth_getBalance
     - Hex to integer
     - "0x83a3c396d1a7b40" → 583760663573639744
   * - eth_blockNumber
     - Hex to integer
     - "0xf4240" → 1000000
   * - eth_getBlock
     - Full block formatting
     - Formats all block fields
   * - eth_getTransaction
     - Full transaction formatting
     - Formats all transaction fields
   * - eth_getTransactionReceipt
     - Receipt formatting
     - Formats logs, status, gas used
   * - eth_chainId
     - Hex to integer
     - "0x1" → 1
   * - eth_gasPrice
     - Hex to integer
     - "0x3b9aca00" → 1000000000

Block Result Formatters
~~~~~~~~~~~~~~~~~~~~~~~
Block responses are formatted with BLOCK_RESULT_FORMATTERS:
.. code-block:: python
    {
        "baseFeePerGas": to_integer_if_hex,
        "difficulty": to_integer_if_hex,
        "gasLimit": to_integer_if_hex,
        "gasUsed": to_integer_if_hex,
        "number": to_integer_if_hex,
        "size": to_integer_if_hex,
        "timestamp": to_integer_if_hex,
        "totalDifficulty": to_integer_if_hex,
        "hash": to_hexbytes(32),
        "parentHash": to_hexbytes(32),
        "miner": to_checksum_address,
        "transactions": [list of tx hashes or full tx objects],
        # ... and more
    }

Transaction Result Formatters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Transaction responses are formatted with TRANSACTION_RESULT_FORMATTERS:
.. code-block:: python
    {
        "blockNumber": to_integer_if_hex,
        "gas": to_integer_if_hex,
        "gasPrice": to_integer_if_hex,
        "nonce": to_integer_if_hex,
        "transactionIndex": to_integer_if_hex,
        "value": to_integer_if_hex,
        "v": to_integer_if_hex,
        "from": to_checksum_address,
        "to": to_checksum_address,
        "hash": to_hexbytes(32),
        "input": HexBytes,
        "r": to_hexbytes(32),
        "s": to_hexbytes(32),
        # ... and more
    }

Log Entry Formatters
~~~~~~~~~~~~~~~~~~~~
Log entries (events) are formatted with LOG_ENTRY_FORMATTERS:
.. code-block:: python
    {
        "blockNumber": to_integer_if_hex,
        "logIndex": to_integer_if_hex,
        "transactionIndex": to_integer_if_hex,
        "address": to_checksum_address,
        "blockHash": to_hexbytes(32),
        "transactionHash": to_hexbytes(32),
        "data": HexBytes,
        "topics": [list of HexBytes(32)],
    }

Using FormattingMiddlewareBuilder
---------------------------------
For custom formatting needs, you can use the FormattingMiddlewareBuilder class
to create middleware that applies your own formatters.
Basic Usage
~~~~~~~~~~~
.. code-block:: python
    from web3 import Web3
    from web3.middleware import FormattingMiddlewareBuilder
    w3 = Web3(Web3.HTTPProvider("http://localhost:8545"))
    # Create middleware with custom formatters
    custom_middleware = FormattingMiddlewareBuilder.build(
        request_formatters={
            "eth_myCustomMethod": lambda params: [p.upper() for p in params]
        },
        result_formatters={
            "eth_myCustomMethod": lambda result: result.lower()
        },
    )
    # Add to middleware stack
    w3.middleware_onion.add(custom_middleware)

Request Formatters Example
~~~~~~~~~~~~~~~~~~~~~~~~~~
Request formatters receive the parameters list and should return the transformed
parameters:
.. code-block:: python
    from web3.middleware import FormattingMiddlewareBuilder
    def my_request_formatter(params):
        """Transform request parameters before sending."""
        # params is a list of parameters for the RPC method
        address, block_id = params
        # Ensure address is lowercase
        return [address.lower(), block_id]
    custom_middleware = FormattingMiddlewareBuilder.build(
        request_formatters={
            "eth_getBalance": my_request_formatter,
        }
    )

Result Formatters Example
~~~~~~~~~~~~~~~~~~~~~~~~~
Result formatters receive the result value and should return the transformed result:
.. code-block:: python
    from web3.middleware import FormattingMiddlewareBuilder
    from decimal import Decimal
    def balance_to_ether(result):
        """Convert balance from wei to ether."""
        wei_balance = int(result, 16) if isinstance(result, str) else result
        return Decimal(wei_balance) / Decimal(1018)
    custom_middleware = FormattingMiddlewareBuilder.build(
        result_formatters={
            "eth_getBalance": balance_to_ether,
        }
    )
    w3.middleware_onion.add(custom_middleware)
    # Now get_balance returns Decimal in ether instead of int in wei
    balance = w3.eth.get_balance("0x...")  # Returns Decimal("1.5") instead of 1500000000000000000

Error Formatters Example
~~~~~~~~~~~~~~~~~~~~~~~~
Error formatters process error responses:
.. code-block:: python
    from web3.middleware import FormattingMiddlewareBuilder
    def custom_error_formatter(error):
        """Add custom error handling."""
        if "revert" in str(error.get("message", "")):
            error["message"] = f"Contract reverted: {error['message']}"
        return error
    custom_middleware = FormattingMiddlewareBuilder.build(
        error_formatters={
            "eth_call": custom_error_formatter,
        }
    )

Dynamic Formatters with Builders
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For formatters that need access to the Web3 instance or need to be built
dynamically per-request, use the sync_formatters_builder and
async_formatters_builder options:
.. code-block:: python
    from web3.middleware import FormattingMiddlewareBuilder
    def build_formatters(w3, method):
        """Build formatters dynamically based on method and w3 instance."""
        request_formatters = {}
        result_formatters = {}
        error_formatters = {}
        if method == "eth_getBalance":
            # Access w3 instance for dynamic behavior
            chain_id = w3.eth.chain_id
            if chain_id == 1:  # Mainnet
                result_formatters["eth_getBalance"] = lambda x: int(x, 16)
        return {
            "request_formatters": request_formatters,
            "result_formatters": result_formatters,
            "error_formatters": error_formatters,
        }
    # For async usage
    async def async_build_formatters(async_w3, method):
        """Async version of formatter builder."""
        chain_id = await async_w3.eth.chain_id
        # ... build formatters
        return {
            "request_formatters": {},
            "result_formatters": {},
            "error_formatters": {},
        }
    custom_middleware = FormattingMiddlewareBuilder.build(
        sync_formatters_builder=build_formatters,
        async_formatters_builder=async_build_formatters,
    )

Common Formatter Utilities
--------------------------
web3.py provides several utility functions for building formatters in
web3._utils.formatters and eth_utils:
.. code-block:: python
    from eth_utils import (
        to_checksum_address,
        to_hex,
        to_int,
    )
    from eth_utils.curried import (
        apply_formatter_at_index,
        apply_formatter_if,
        apply_formatter_to_array,
        apply_formatters_to_dict,
        apply_formatters_to_sequence,
    )
    from web3._utils.formatters import (
        apply_key_map,
        hex_to_integer,
        integer_to_hex,
    )
    # Apply formatter to specific index in params list
    formatter = apply_formatter_at_index(to_hex, 1)  # Format 2nd parameter
    # Apply formatter conditionally
    formatter = apply_formatter_if(
        lambda x: isinstance(x, int),  # condition
        to_hex  # formatter to apply if condition is True
    )
    # Apply formatters to each element of an array
    formatter = apply_formatter_to_array(to_checksum_address)
    # Apply different formatters to dict keys
    formatter = apply_formatters_to_dict({
        "gasPrice": to_hex,
        "value": to_hex,
        "from": to_checksum_address,
    })

Built-in Middleware Using Formatters
------------------------------------
Several built-in middleware use the formatting system:
PythonicMiddleware
~~~~~~~~~~~~~~~~~~
The PythonicMiddleware applies the default Pythonic formatters to convert
between Python types and JSON-RPC formats. This is included in the default
middleware stack.
.. code-block:: python
    from web3.middleware import PythonicMiddleware
    # This is equivalent to:
    PythonicMiddleware = FormattingMiddlewareBuilder.build(
        request_formatters=PYTHONIC_REQUEST_FORMATTERS,
        result_formatters=PYTHONIC_RESULT_FORMATTERS,
    )

ValidationMiddleware
~~~~~~~~~~~~~~~~~~~~
The ValidationMiddleware uses formatters to validate transaction parameters
like chainId before sending:
.. code-block:: python
    from web3.middleware import ValidationMiddleware
    # Included by default, validates transactions have correct chainId

EthereumTesterMiddleware
~~~~~~~~~~~~~~~~~~~~~~~~
The ethereum_tester_middleware uses formatters to transform data between
web3.py and the eth-tester backend:
.. code-block:: python
    from web3.providers.eth_tester.middleware import ethereum_tester_middleware

Best Practices
--------------
1. Order Matters: Request formatters are applied in order, and result formatters
   are applied in reverse order. Be mindful of the order when adding multiple
   formatting middleware.
2. Don't Duplicate: The default PythonicMiddleware handles most common
   transformations. Only add custom formatters for specific needs.
3. Test Your Formatters: Formatters can silently transform data incorrectly.
   Always test with real RPC calls.
4. Use Type Hints: When writing custom formatters, use type hints to catch
   errors early:
   .. code-block:: python
       from typing import Any
       def my_formatter(value: str) -> int:
           return int(value, 16)
5. Handle Edge Cases: Always handle None values and unexpected types:
   .. code-block:: python
       def safe_formatter(value):
           if value is None:
               return None
           return int(value, 16) if isinstance(value, str) else value

See Also
--------
- :ref:middleware_internals - How middleware works in web3.py
- :ref:Modifying_Middleware - How to add, remove, and configure middleware
- JSON-RPC Specification <https://www.jsonrpc.org/specification>_ - The underlying protocol
- Ethereum JSON-RPC API <https://ethereum.org/en/developers/docs/apis/json-rpc/>_ - Ethereum-specific methods
---
Gas Price  Copy Section
.. _Gas_Price:
Gas Price API
===============
.. warning::
    Gas price strategy is only supported for legacy transactions. The London fork
    introduced maxFeePerGas and maxPriorityFeePerGas transaction parameters
    which should be used over gasPrice whenever possible.
For Ethereum (legacy) transactions, gas price is a delicate property. For this reason,
Web3 includes an API for configuring it.
The Gas Price API allows you to define Web3's behaviour for populating the gas price.
This is done using a "Gas Price Strategy" - a method which takes the Web3 object and a
transaction dictionary and returns a gas price (denominated in wei).
Retrieving gas price
--------------------
To retrieve the gas price using the selected strategy simply call
:meth:~web3.eth.Eth.generate_gas_price
.. code-block:: python
    >>> web3.eth.generate_gas_price()
    20000000000
Creating a gas price strategy
-------------------------------
A gas price strategy is implemented as a python method with the following
signature:
.. code-block:: python
    def gas_price_strategy(web3, transaction_params=None):
    ...
The method must return a positive integer representing the gas price in wei.
To demonstrate, here is a rudimentary example of a gas price strategy that
returns a higher gas price when the value of the transaction is higher than
1 Ether.
.. code-block:: python
    from web3 import Web3
    def value_based_gas_price_strategy(web3, transaction_params):
        if transaction_params['value'] > Web3.to_wei(1, 'ether'):
            return Web3.to_wei(20, 'gwei')
        else:
            return Web3.to_wei(5, 'gwei')
Selecting the gas price strategy
--------------------------------
The gas price strategy can be set by calling :meth:~web3.eth.Eth.set_gas_price_strategy.
.. code-block:: python
    from web3 import Web3
    def value_based_gas_price_strategy(web3, transaction_params):
        ...
    w3 = Web3(...)
    w3.eth.set_gas_price_strategy(value_based_gas_price_strategy)
Available gas price strategies
------------------------------
.. py:module:: web3.gas_strategies.rpc
.. py:method:: rpc_gas_price_strategy(web3, transaction_params=None)
    Makes a call to the JSON-RPC eth_gasPrice
    method <https://ethereum.org/en/developers/docs/apis/json-rpc#eth_gasprice>_ which returns
    the gas price configured by the connected Ethereum node.
.. py:module:: web3.gas_strategies.time_based
.. py:method:: construct_time_based_gas_price_strategy(max_wait_seconds, sample_size=120, probability=98, weighted=False)
    Constructs a strategy which will compute a gas price such that the
    transaction will be mined within a number of seconds defined by
    max_wait_seconds with a probability defined by probability.  The
    gas price is computed by sampling sample_size of the most recently
    mined blocks. If weighted=True, the block time will be weighted towards
    more recently mined blocks.
    * max_wait_seconds The desired maximum number of seconds the
      transaction should take to mine.
    * sample_size The number of recent blocks to sample
    * probability An integer representation of the desired probability that
      the transaction will be mined within max_wait_seconds.  0 means 0%
      and 100 means 100%.
    The following ready to use versions of this strategy are available.
    * web3.gas_strategies.time_based.fast_gas_price_strategy: Transaction mined within 60 seconds.
    * web3.gas_strategies.time_based.medium_gas_price_strategy: Transaction mined within 5 minutes.
    * web3.gas_strategies.time_based.slow_gas_price_strategy: Transaction mined within 1 hour.
    * web3.gas_strategies.time_based.glacial_gas_price_strategy: Transaction mined within 24 hours.
    .. warning:: Due to the overhead of sampling the recent blocks it is
      recommended that a caching solution be used to reduce the amount of chain
      data that needs to be re-fetched for each request.
    .. code-block:: python
        from web3 import Web3
        from web3.gas_strategies.time_based import medium_gas_price_strategy
        w3 = Web3(...)
        w3.eth.set_gas_price_strategy(medium_gas_price_strategy)
        w3.provider.cache_allowed_requests = True
---
Index  Copy Section
.. meta::
   :description: Python Web3 SDK for Ethereum and EVM blockchains
gm
==
.. image:: _static/banner/banner-snek.jpg
   :alt: Banner Image
web3.py is a Python library for interacting with Ethereum.
It's commonly found in decentralized apps (dapps)_ to help with
sending transactions, interacting with smart contracts, reading
block data, and a variety of other use cases.
For project updates, follow @EthereumPython_ and sign up
for new post notifications on the blog_.

Getting Started
---------------
.. NOTE::
   👋 Brand new to Ethereum?
   0. Don't travel alone! Join the Ethereum Python Community Discord_.
   1. Read this blog post series_ for a gentle introduction to Ethereum blockchain concepts.
   2. The :ref:Overview page will give you a quick idea of what else web3.py can do.
   3. Try building a little something!
- Ready to code? → :ref:quickstart
- Quick tour? → :ref:overview
- Synchronous help? → Discord_
- Asynchronous help? → StackExchange_
- Report a bug? → Github_
- Want to help us? → :ref:Contribute <contributing>
- Looking for inspiration? → :ref:resources
.. include:: toc.rst

.. _decentralized apps (dapps): https://ethereum.org/en/dapps/
.. _@EthereumPython: https://twitter.com/EthereumPython
.. _blog: https://snakecharmers.ethereum.org/
.. _blog post series: https://snakecharmers.ethereum.org/a-developers-guide-to-ethereum-pt-1
.. _StackExchange: https://ethereum.stackexchange.com/questions/tagged/web3.py
.. _Discord: https://discord.gg/GHryRvPB84
.. _Github: https://github.com/ethereum/web3.py/issues
---
Internals  Copy Section
Web3 Internals
==============

.. warning:: This section of the documentation is for advanced users.  You should probably stay away from these APIs if you don't know what you are doing.
The Web3 library has multiple layers of abstraction between the public api
exposed by the web3 object and the backend or node that web3 is connecting to.
* Providers are responsible for the actual communication with the
  blockchain such as sending JSON-RPC requests over HTTP or an IPC socket.
* Middleware provide hooks for monitoring and modifying requests and
  responses to and from the provider.
* Managers provide thread safety and primitives to allow for asynchronous usage of web3.
Here are some common things you might want to do with these APIs.
* Redirect certain RPC requests to different providers such as sending all
  read operations to a provider backed by a remote node and all write operations
  to a local node that you control.
* Transparently intercept transactions sent over eth_sendTransaction, sign
  them locally, and then send them through eth_sendRawTransaction.
* Modify the response from an RPC request so that it is returned in different
  format such as converting all integer values to their hexadecimal
  representation.
* Validate the inputs to RPC requests

Request Lifecycle
-----------------
Each web3 RPC call passes through these layers in the following manner.
.. code-block:: none
                       *
                   | Request |    | Response |
                       *
                       |                ^
                       v                |
                 +-----------------------------+
                 |           Manager           |
                 +-----------------------------+
                       |                ^
                       v                |
                 +-----------------------------+
                 |         Middleware          |
                 +-----------------------------+
                       |                ^
                       v                |
                 +-----------------------------+
                 |          Provider           |
                 +-----------------------------+

You can visualize this relationship like an onion, with the Provider at the
center. The request originates from the Manager, outside of the onion, passing
down through each layer of the onion until it reaches the Provider at the
center. The Provider then handles the request, producing a response which will
then pass back out from the center of the onion, through each layer until it is
finally returned by the Manager.

Providers
---------
A provider is responsible for all direct blockchain interactions.  In most
cases this means interacting with the JSON-RPC server for an ethereum node over
HTTP or an IPC socket.  There is however nothing which requires providers to be
RPC based, allowing for providers designed for testing purposes which use an
in-memory EVM to fulfill requests.

Writing your own Provider
~~~~~~~~~~~~~~~~~~~~~~~~~
Writing your own provider requires implementing two required methods as well as
setting the middleware the provider should use.

.. py:method:: BaseProvider.make_request(method, params)
    Each provider class must implement this method.  This method should
    return a JSON object with either a 'result' key in the case of success,
    or an 'error' key in the case of failure.

    * method  This will be a string representing the JSON-RPC method that
      is being called such as 'eth_sendTransaction'.
    * params  This will be a list or other iterable of the parameters for
      the JSON-RPC method being called.

.. py:method:: BaseProvider.is_connected(show_traceback=False)
    This function should return True or False depending on whether the
    provider should be considered connected.  For example, an IPC socket
    based provider should return True if the socket is open and False
    if the socket is closed.
    If set to True, the optional show_traceback boolean will raise a
    ProviderConnectionError and provide information on why the provider should
    not be considered connected.

.. py:attribute:: BaseProvider.middleware
    This should be an iterable of middleware.
You can set a new list of middleware by assigning to provider.middleware,
with the first middleware that processes the request at the beginning of the list.

Provider Configurations
~~~~~~~~~~~~~~~~~~~~~~~
.. _request_caching:
Request Caching

/ Detailed source-code truncated for AI context efficiency. /

HTTPProvider and AsyncHTTPProvider instances retry certain requests by default
on exceptions. This can be configured via the exception_retry_configuration
property on the provider instance, which takes a
:class:~web3.providers.rpc.utils.ExceptionRetryConfiguration class as its value. The
retry mechanism employs an exponential backoff strategy, starting from the initial
value determined by the backoff_factor, and doubling the delay with each attempt,
up to the retries value. Below is an example showing the default options for the
retry configuration and how to override them.

.. py:class:: web3.providers.rpc.utils.ExceptionRetryConfiguration
    .. py:attribute:: errors
        A tuple of exceptions that the provider should retry on. The default is
        HTTPProvider: (ConnectionError, requests.HTTPError, requests.Timeout)
        and AsyncHTTPProvider: (aiohttp.ClientError, asyncio.TimeoutError).
    .. py:attribute:: retries
        The number of retries to attempt. The default is 5.
    .. py:attribute:: backoff_factor
        The initial delay multiplier, which doubles with each retry attempt. The default
        is 0.125.
    .. py:attribute:: method_allowlist
        A list of retryable methods. The default is an in-house list of deemed-safe-to-
        retry methods.
.. code-block:: python
    from web3 import Web3, HTTPProvider
    from web3.providers.rpc.utils import (
        REQUEST_RETRY_ALLOWLIST,
        ExceptionRetryConfiguration,
    )
    w3 = Web3(HTTPProvider(
        endpoint_uri="...",
        exception_retry_configuration=ExceptionRetryConfiguration(
            errors=DEFAULT_EXCEPTIONS,
            # number of retries to attempt
            retries=5,
            # initial delay multiplier, doubles with each retry attempt
            backoff_factor=0.125,
            # an in-house default list of retryable methods
            method_allowlist=REQUEST_RETRY_ALLOWLIST,
        ),
    ))
For the different http providers, DEFAULT_EXCEPTIONS is defined as:
- HTTPProvider: (ConnectionError, requests.HTTPError, requests.Timeout)
- AsyncHTTPProvider: (ConnectionError, aiohttp.ClientError, asyncio.TimeoutError)
Setting retry_configuration to None will disable retries on exceptions for the
provider instance.
.. code-block:: python
    from web3 import Web3, HTTPProvider
    w3 = Web3(HTTPProvider(endpoint_uri="...", retry_configuration=None)
Managers
--------
The Manager acts as a gatekeeper for the request/response lifecycle.  It is
unlikely that you will need to change the Manager as most functionality can be
implemented in the Middleware layer.
.. _internals__persistent_connection_providers:
Request Processing for Persistent Connection Providers
------------------------------------------------------
.. py:class:: web3.providers.persistent.request_processor.RequestProcessor
The RequestProcessor class is responsible for the storing and syncing up of
asynchronous requests to responses for a PersistentConnectionProvider. The
:class:~web3.providers.persistent.WebSocketProvider and the
:class:~web3.providers.persistent.AsyncIPCProvider are two persistent connection
providers. In order to send a request and receive a response to that same request,
PersistentConnectionProvider instances have to match request id values to
response id values coming back from the socket connection. Any provider that does
not adhere to the JSON-RPC 2.0 specification <https://www.jsonrpc.org/specification>_
in this way will not work with PersistentConnectionProvider instances. The specifics
of how the request processor handles this are outlined below.
Listening for Responses
~~~~~~~~~~~~~~~~~~~~~~~
Implementations of the PersistentConnectionProvider class have a message listener
background task that is called when the socket connection is established. This task
is responsible for listening for any and all messages coming in over the socket
connection and storing them in the RequestProcessor instance internal to the
PersistentConnectionProvider instance. The RequestProcessor instance is
responsible for storing the messages in the correct cache, either the one-to-one cache
or the one-to-many (subscriptions) queue, depending on whether the message has a
JSON-RPC id value or not.

One-To-One Requests
~~~~~~~~~~~~~~~~~~~
One-to-one requests can be summarized as any request that expects only one response
back. An example is using the eth module API to request the latest block number.
.. code-block:: python
    >>> async def ws_one_to_one_example():
    ...     async with AsyncWeb3(WebSocketProvider(f"ws://127.0.0.1:8546")) as w3:
    ...         # make a request and expect a single response returned on the same line
    ...         latest_block_num = await w3.eth.block_number
    >>> asyncio.run(ws_one_to_one_example())
With persistent socket connections, we have to call send() and asynchronously
receive responses via another means, generally by calling recv() or by iterating
on the socket connection for messages. As outlined above, the
PersistentConnectionProvider class has a message listener background task that
handles the receiving of messages.
Due to this asynchronous nature of sending and receiving, in order to make one-to-one
request-to-response calls work, we have to save the request information somewhere so
that, when the response is received, we can match it to the original request that was
made (i.e. the request with a matching id to the response that was received). The
stored request information is then used to process the response when it is received,
piping it through the response formatters and middleware internal to the web3.py
library.
In order to store the request information, the RequestProcessor class has an
internal RequestInformation cache. The RequestInformation class saves important
information about a request.
.. py:class:: web3._utils.caching.RequestInformation
    .. py:attribute:: method
        The name of the method - e.g. "eth_subscribe".
    .. py:attribute:: params
        The params used when the call was made - e.g. ("newPendingTransactions", True).
    .. py:attribute:: response_formatters
        The formatters that will be used to process the response.
    .. py:attribute:: middleware_response_processors
        Any middleware that processes responses that is present on the instance at the
        time of the request is appended here, in order, so the response may be piped
        through that logic when it comes in.
    .. py:attribute:: subscription_id
        If the request is an eth_subscribe request, rather than
        popping this information from the cache when the response to the subscription call
        comes in (i.e. the subscription id), we save the subscription id with the
        request information so that we can correctly process all subscription messages
        that come in with that subscription id. For one-to-one request-to-response
        calls, this value is always None.
One-to-one responses, those that include a JSON-RPC id in the response object, are
stored in an internal SimpleCache class, isolated from any one-to-many responses.
When the PersistentConnectionProvider is looking for a response internally, it will
expect the message listener task to store the response in this cache. Since the request
id is used in the cache key generation, it will then look for a cache key that matches
the response id with that of the request id. If the cache key is found, the response
is processed and returned to the user. If the cache key is not found, the operation will
time out and raise a TimeExhausted exception. This timeout can be configured by the
user when instantiating the PersistentConnectionProvider instance via the
response_timeout keyword argument.
One-To-Many Requests
~~~~~~~~~~~~~~~~~~~~
One-to-many requests can be summarized by any request that expects many responses as a
result of the initial request. The only current example is the eth_subscribe
request. The initial eth_subscribe request expects only one response, the
subscription id value, but it also expects to receive many eth_subscription
messages if and when the request is successful. For this reason, the original request
is considered a one-to-one request so that a subscription id can be returned to the
user on the same line. The many responses this call will produce can be handled in one
of a few ways.
The recommended way to handle one-to-many responses is to use the subscription manager
API. The subscription manager API is a public API on the AsyncWeb3 class, when
connected to a PersistentConnectionProvider instance, that allows the user to
subscribe to a subscription and handle the many responses asynchronously. The
subscription_manager instance is responsible for handling the many responses that
come in over the socket connection, as long as handlers are passed to each subscription
call. The subscription manager can also be used to unsubscribe from a subscription when
the user is done with it.
.. code-block:: python
    >>> async def new_heads_handler(
    ...     handler_context: NewHeadsSubscriptionContext,
    ... ) -> None:
    ...     result = handler_context.result
    ...     print(f"New block header: {result}\n")
    ...     if result["number"] > 1234567:
    ...         await handler_context.subscription.unsubscribe()
    >>> async def ws_subscription_example():
    ...     async with AsyncWeb3(WebSocketProvider(f"ws://127.0.0.1:8546")) as w3:
    ...         # Subscribe to new block headers and receive the subscription_id.
    ...         # A one-to-one call with a trigger for many responses
    ...         subscription_id = await w3.eth.subscribe("newHeads", handler=new_heads_handler)
    ...
    ...         # Handle the subscription messages asynchronously using the subscription
    ...         # manager. This will continue until no more subscriptions are present in
    ...         # the subscription manager, or indefinitely if the run_forever flag
    ...         # is set to True.
    ...         await w3.subscription_manager.handle_subscriptions(run_forever=False)
    >>> asyncio.run(ws_subscription_example())
The manager can also subscribe to many subscriptions at one time. The
EthSubscription classes, available via web3.utils.subscriptions, provide a
friendly API for managing subscriptions. Since each connection and provider instance
has its own message listener task and subscription manager instance, you can subscribe
to many subscriptions at once and handle the many responses that come in over the socket
connections via handlers. The handlers contain:
- async_w3: The AsyncWeb3 instance that the subscription was made on.
- subscription: The subscription instance that the handler is attached to.
- result: The response that came in over the socket connection for the subscription.
Subscriptions also accept a handler_context argument that can be used to pass
additional information to the handler when subscribing to a subscription. This can be
used to pass in an event object, for example, that can be used to parse a log event
when it comes in.

.. code-block:: python
    >>> from web3 import (
    ...     AsyncWeb3,
    ...     WebSocketProvider,
    ...     AsyncIPCProvider,
    ... )
    >>> from web3.utils.subscriptions import (
    ...     EthSubscription,
    ...     NewHeadsSubscription,
    ...     NewHeadsSubscriptionContext,
    ...     PendingTxSubscription,
    ...     PendingTxSubscriptionContext,
    ...     LogsSubscription,
    ...     LogsSubscriptionContext,
    ... )
    >>> async def new_heads_handler(
    ...     handler_context: NewHeadsSubscriptionContext,
    ... ) -> None:
    ...     header = handler_context.result
    ...     print(f"New block header: {header}\n")
    ...     if header["number"] > 1234567:
    ...         await handler_context.subscription.unsubscribe()
    >>> async def pending_txs_handler(
    ...     handler_context: PendingTxSubscriptionContext,
    ... ) -> None:
    ...     ...
    >>> async def log_handler(
    ...     handler_context: LogsSubscriptionContext,
    ... ) -> None:
    ...     log_receipt = handler_context.result
    ...     # event is now available in the handler context, because we pass it to in the
    ...     # handler_context when subscribing to the log
    ...     event_data = handler_context.transfer_event.process_log(log_receipt)
    ...     print(f"Log event data: {event_data}\n")
    >>> async def sub_manager():
    ...     local_w3 = await AsyncWeb3(AsyncIPCProvider(LOCAL_IPC))
    ...
    ...     # subscribe to many subscriptions via the subscription manager with handlers
    ...     weth_contract = local_w3.eth.contract(
    ...         address=local_w3.to_checksum_address("0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2"),
    ...         abi=WETH_ABI,
    ...     )
    ...     transfer_event = weth_contract.events.Transfer()
    ...     await local_w3.subscription_manager.subscribe(
    ...         [
    ...             NewHeadsSubscription(label="new-heads-mainnet", handler=new_heads_handler),
    ...             PendingTxSubscription(
    ...                 label="pending-tx-mainnet",  # optional label
    ...                 full_transactions=True,
    ...                 handler=pending_tx_handler,
    ...             ),
    ...             LogsSubscription(
    ...                 label="WETH transfers",  # optional label
    ...                 address=weth_contract.address,
    ...                 topics=[transfer_event.topic],
    ...                 handler=log_handler,
    ...                 # optional handler_context args to help parse a response
    ...                 handler_context={"transfer_event": transfer_event},
    ...             ),
    ...         ]
    ...     )
    ...
    ...     public_w3 = await AsyncWeb3(WebSocketProvider(PUBLIC_PROVIDER_WS))
    ...     # subscribe via eth_subscribe, with handler and label (optional)
    ...     await public_w3.eth.subscribe("public_newHeads", handler=pending_tx_handler, label="new-heads-public-ws")
    >>>     # This will handle all subscriptions until no more subscriptions are present
    ...     # in either subscription manager instance. If the run_forever flag is set
    ...     # to True on any manager instance, this will run indefinitely.
    >>>     await asyncio.gather(
    ...         public_w3.subscription_manager.handle_subscriptions(),
    ...         local_w3.subscription_manager.handle_subscriptions(),
    ...     )
    ...
    ...     # close the connections
    ...     await local_w3.provider.disconnect()
    ...     await public_w3.provider.disconnect()
    >>> asyncio.run(sub_manager())

The process_subscriptions() method on the
:class:~web3.providers.persistent.PersistentConnection class, the public API for
interacting with the active persistent socket connection, is also set up to receive
eth_subscription responses over an asynchronous iterator pattern. You can use this
method to listen for raw messages and process them as they come in.
.. code-block:: python
    >>> async def ws_subscription_example():
    ...     async with AsyncWeb3(WebSocketProvider(f"ws://127.0.0.1:8546")) as w3:
    ...         # Subscribe to new block headers and receive the subscription_id.
    ...         # A one-to-one call with a trigger for many responses
    ...         subscription_id = await w3.eth.subscribe("newHeads")
    ...
    ...         # Listen to the socket for the many responses utilizing the
    ...         # w3.socket PersistentConnection public API method
    ...         # process_subscriptions()
    ...         async for response in w3.socket.process_subscriptions():
    ...             # Receive only one-to-many responses here so that we don't
    ...             # accidentally return the response for a one-to-one request in this
    ...             # block
    ...
    ...             print(f"{response}\n")
    ...
    ...             if some_condition:
    ...                 # unsubscribe from new block headers, another one-to-one request
    ...                 is_unsubscribed = await w3.eth.unsubscribe(subscription_id)
    ...                 if is_unsubscribed:
    ...                     break
    >>> asyncio.run(ws_subscription_example())
One-to-many responses, those that do not include a JSON-RPC id in the response object,
are stored in an internal asyncio.Queue instance, isolated from any one-to-one
responses. When the PersistentConnectionProvider is looking for one-to-many
responses internally, it will expect the message listener task to store these messages
in this queue. Since the order of the messages is important, the queue is a FIFO queue.
The process_subscriptions() method on the PersistentConnection class is set up
to pop messages from this queue as FIFO over an asynchronous iterator pattern.
If the stream of messages from the socket is not being interrupted by any other
tasks, the queue will generally be in sync with the messages coming in over the
socket. That is, the message listener will put a message in the queue and the
process_subscriptions() method will pop that message from the queue and yield
control of the loop back to the listener. This will continue until the socket
connection is closed or the user unsubscribes from the subscription. If the stream of
messages lags a bit, or the provider is not consuming messages but has subscribed to
a subscription, this internal queue may fill up with messages until it reaches its max
size and then trigger a waiting asyncio.Event until the provider begins consuming
messages from the queue again. For this reason, it's important to begin consuming
messages from the queue, via the process_subscriptions() method, as soon as a
subscription is made.
---
Middleware  Copy Section
.. _middleware_internals:
Middleware
==========
Web3 is instantiated with layers of middleware by default. They sit between the public
Web3 methods and the :doc:providers, and are used to perform sanity checks, convert data
types, enable ENS support, and more. Each layer can modify the request and/or response.
While several middleware are enabled by default, others are available for optional use,
and you're free to create your own!
Each middleware layer gets invoked before the request reaches the provider, and then
processes the result after the provider returns, in reverse order. However, it is
possible for a middleware to return early from a call without the request ever getting
to the provider (or even reaching the middleware that are in deeper layers).

.. _Modifying_Middleware:
Configuring Middleware
-----------------------
Middleware can be added, removed, replaced, and cleared at runtime. To make that easier, you
can name the middleware for later reference.
Middleware Order
~~~~~~~~~~~~~~~~
Think of the middleware as being layered in an onion, where you initiate a web3.py request at
the outermost layer of the onion, and the Ethereum node (like geth) receives and responds
to the request inside the innermost layer of the onion. Here is a (simplified) diagram:
.. code-block:: none
                                         New request from web3.py
                                                     |
                                                     |
                                                     v

|
                                                     v
                                          Returned value in web3.py

The middleware are maintained in Web3.middleware_onion. See below for the API.
When specifying middleware in a list, or retrieving the list of middleware, they will
be returned in the order of outermost layer first and innermost layer last. In the above
example, that means that w3.middleware_onion.middleware would return the middleware
in the order of: [2, 1, 0].

.. _middleware_stack_api:
Middleware Stack API
~~~~~~~~~~~~~~~~~~~~
To add or remove items in different layers, use the following API:
.. py:method:: Web3.middleware_onion.add(middleware, name=None)
    Middleware will be added to the outermost layer. That means the new middleware will modify the
    request first, and the response last. You can optionally name it with any hashable object,
    typically a string.
    .. code-block:: python
        >>> w3 = Web3(...)
        >>> w3.middleware_onion.add(web3.middleware.GasPriceStrategyMiddleware)
        # or
        >>> w3.middleware_onion.add(web3.middleware.GasPriceStrategyMiddleware, 'gas_price_strategy')
.. py:method:: Web3.middleware_onion.inject(middleware, name=None, layer=None)
    Inject a named middleware to an arbitrary layer.
    The current implementation only supports injection at the innermost or
    outermost layers. Note that injecting to the outermost layer is equivalent to calling
    :meth:Web3.middleware_onion.add .
    .. code-block:: python
        # Either of these will put the gas_price_strategy middleware at the innermost layer
        >>> w3 = Web3(...)
        >>> w3.middleware_onion.inject(web3.middleware.GasPriceStrategyMiddleware, layer=0)
        # or
        >>> w3.middleware_onion.inject(web3.middleware.GasPriceStrategyMiddleware, 'gas_price_strategy', layer=0)
.. py:method:: Web3.middleware_onion.remove(middleware)
    Middleware will be removed from whatever layer it was in. If you added the middleware with
    a name, use the name to remove it. If you added the middleware as an object, use the object
    again later to remove it:
    .. code-block:: python
        >>> w3 = Web3(...)
        >>> w3.middleware_onion.remove(web3.middleware.GasPriceStrategyMiddleware)
        # or
        >>> w3.middleware_onion.remove('gas_price_strategy')
.. py:method:: Web3.middleware_onion.replace(old_middleware, new_middleware)
    Middleware will be replaced from whatever layer it was in. If the middleware was named, it will
    continue to have the same name. If it was un-named, then you will now reference it with the new
    middleware object.
    .. code-block:: python
        >>> from web3.middleware import GasPriceStrategyMiddleware, AttributeDictMiddleware
        >>> w3 = Web3(provider, middleware=[GasPriceStrategyMiddleware, AttributeDictMiddleware])
        >>> w3.middleware_onion.replace(GasPriceStrategyMiddleware, AttributeDictMiddleware)
        # this is now referenced by the new middleware object, so to remove it:
        >>> w3.middleware_onion.remove(AttributeDictMiddleware)
        # or, if it was named
        >>> w3.middleware_onion.replace('gas_price_strategy', AttributeDictMiddleware)
        # this is still referenced by the original name, so to remove it:
        >>> w3.middleware_onion.remove('gas_price_strategy')
.. py:method:: Web3.middleware_onion.clear()
    Empty all the middleware, including the default ones.
    .. code-block:: python
        >>> w3 = Web3(...)
        >>> w3.middleware_onion.clear()
        >>> assert len(w3.middleware_onion) == 0
.. py:attribute:: Web3.middleware_onion.middleware
    Return all the current middleware for the Web3 instance in the appropriate order for importing into a new
    Web3 instance.
    .. code-block:: python
        >>> w3_1 = Web3(...)
        # add uniquely named middleware:
        >>> w3_1.middleware_onion.add(web3.middleware.GasPriceStrategyMiddleware, 'test_middleware')
        # export middleware from first w3 instance
        >>> middleware = w3_1.middleware_onion.middleware
        # import into second instance
        >>> w3_2 = Web3(..., middleware=middleware)
        >>> assert w3_1.middleware_onion.middleware == w3_2.middleware_onion.middleware
        >>> assert w3_2.middleware_onion.get('test_middleware')

Instantiate with Custom Middleware
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Instead of working from the default list, you can specify a custom list of
middleware when initializing Web3:
.. code-block:: python
    Web3(middleware=[my_middleware1, my_middleware2])
.. warning::
  This will replace the default middleware. To keep the default functionality,
  either use middleware_onion.add() from above, or add the default middleware to
  your list of new middleware.

.. _default_middleware:
Default Middleware
------------------
The following middleware are included by default:
* gas_price_strategy
* ens_name_to_address
* attrdict
* validation
* gas_estimate
The defaults are defined in the get_default_middleware() method in web3/manager.py.
AttributeDict
~~~~~~~~~~~~~
.. py:class:: web3.middleware.AttributeDictMiddleware
    This middleware recursively converts any dictionary type in the result of a call
    to an AttributeDict. This enables dot-syntax access, like
    eth.get_block('latest').number in addition to
    eth.get_block('latest')['number'].
    .. note::
        Accessing a property via attribute breaks type hinting. For this reason, this
        feature is available as a middleware, which may be removed if desired.
ENS Name to Address Resolution
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. py:class:: web3.middleware.ENSNameToAddressMiddleware
    This middleware converts Ethereum Name Service (ENS) names into the
    address that the name points to. For example :meth:w3.eth.send_transaction <web3.eth.Eth.send_transaction> will
    accept .eth names in the 'from' and 'to' fields.
    .. note::
        This middleware only converts ENS names on chains where the proper ENS
        contracts are deployed to support this functionality. All other cases will
        result in a NameNotFound error.
Gas Price Strategy
~~~~~~~~~~~~~~~~~~
.. py:class:: web3.middleware.GasPriceStrategyMiddleware
  .. warning::
      Gas price strategy is only supported for legacy transactions. The London fork
      introduced maxFeePerGas and maxPriorityFeePerGas transaction parameters
      which should be used over gasPrice whenever possible.
  This adds a gasPrice to transactions if applicable and when a gas price strategy has
  been set. See :ref:Gas_Price for information about how gas price is derived.
Buffered Gas Estimate
~~~~~~~~~~~~~~~~~~~~~
.. py:class:: web3.middleware.BufferedGasEstimateMiddleware
    This adds a gas estimate to transactions if gas is not present in the transaction
    parameters. Sets gas to:
    min(w3.eth.estimate_gas + gas_buffer, gas_limit)
    where the gas_buffer default is 100,000
Validation
~~~~~~~~~~
.. py:class:: web3.middleware.ValidationMiddleware
    This middleware includes block and transaction validators which perform validations
    for transaction parameters.

Optional Middleware
-------------------
Web3 includes optional middleware for common use cases. Below is a list of available
middleware which are not enabled by default.
Stalecheck
~~~~~~~~~~~~
.. py:method:: web3.middleware.StalecheckMiddlewareBuilder
    This middleware checks how stale the blockchain is, and interrupts calls with a failure
    if the blockchain is too old.
    * allowable_delay is the length in seconds that the blockchain is allowed to be
      behind of time.time()
    Because this middleware takes an argument, you must create the middleware
    with a method call.
    .. code-block:: python
        two_day_stalecheck = StalecheckMiddlewareBuilder.build(60  60  24 * 2)
        web3.middleware_onion.add(two_day_stalecheck)
    If the latest block in the blockchain is older than 2 days in this example, then the
    middleware will raise a StaleBlockchain exception on every call except
    web3.eth.get_block().

.. _geth-poa:
Proof of Authority
~~~~~~~~~~~~~~~~~~
.. py:class:: web3.middleware.ExtraDataToPOAMiddleware
.. important::
    It is crucial that this middleware is injected at the 0th layer of the
    middleware onion, using
    w3.middleware_onion.inject(ExtraDataToPOAMiddleware, layer=0), to guarantee
    it is the first middleware to process the response and modify the extraData
    field. This ensures it processes the field before any other middleware attempts
    to validate it.
ExtraDataToPOAMiddleware is required to connect to geth --dev and may
also be needed for other EVM compatible blockchains like Polygon or BNB Chain
(Binance Smart Chain).
If the middleware is not injected at the 0th layer of the middleware onion, you may get
errors like the example below when interacting with your EVM node.
.. code-block:: shell
    web3.exceptions.ExtraDataLengthError: The field extraData is 97 bytes, but should be
    32.  It is quite likely that you are connected to a POA chain. Refer to
    http://web3py.readthedocs.io/en/stable/middleware.html#proof-of-authority
    for more details. The full extraData is: HexBytes('...')
The easiest way to connect to a default geth --dev instance which loads the
middleware is:
.. code-block:: python
    >>> from web3.auto.gethdev import w3
    # confirm that the connection succeeded
    >>> w3.client_version
    'Geth/v1.14.12-stable-4bb3c89d/linux-amd64/go1.22.4'
This example connects to a local geth --dev instance on Linux with a
unique IPC location and loads the middleware:
.. code-block:: python
    >>> from web3 import Web3, IPCProvider
    # connect to the IPC location started with 'geth --dev --datadir ~/mynode'
    >>> w3 = Web3(IPCProvider('~/mynode/geth.ipc'))
    >>> from web3.middleware import ExtraDataToPOAMiddleware
    # inject the poa compatibility middleware to the innermost layer (0th layer)
    >>> w3.middleware_onion.inject(ExtraDataToPOAMiddleware, layer=0)
    # confirm that the connection succeeded
    >>> w3.client_version
    'Geth/v1.14.12-stable-4bb3c89d/linux-amd64/go1.22.4'
Why is ExtraDataToPOAMiddleware necessary?
''''''''''''''''''''''''''''''''''''''''''''''
There is no strong community consensus on a single Proof-of-Authority (PoA) standard yet.
Some nodes have successful experiments running though. One is go-ethereum (geth),
which uses a prototype PoA for its development mode and the Goerli test network.
Unfortunately, it does deviate from the yellow paper specification, which constrains the
extraData field in each block to a maximum of 32-bytes. Geth is one such example
where PoA uses more than 32 bytes, so this middleware modifies the block data a bit
before returning it.
.. _local-filter:
Locally Managed Log and Block Filters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. py:method:: web3.middleware.LocalFilterMiddleware
This middleware provides an alternative to ethereum node managed filters. When used, Log and
Block filter logic are handled locally while using the same web3 filter api. Filter results are
retrieved using JSON-RPC endpoints that don't rely on server state.
.. doctest::
    >>> from web3 import Web3, EthereumTesterProvider
    >>> w3 = Web3(EthereumTesterProvider())
    >>> from web3.middleware import LocalFilterMiddleware
    >>> w3.middleware_onion.add(LocalFilterMiddleware)
.. code-block:: python
    #  Normal block and log filter apis behave as before.
    >>> block_filter = w3.eth.filter("latest")
    >>> log_filter = myContract.events.myEvent.build_filter().deploy()
Signing
~~~~~~~
.. py:method:: web3.middleware.SignAndSendRawMiddlewareBuilder
This middleware automatically captures transactions, signs them, and sends them as raw transactions.
The from field on the transaction, or w3.eth.default_account must be set to the address of the private key for
this middleware to have any effect.
The build method for this middleware builder takes a single argument:
   * private_key_or_account A single private key or a tuple, list or set of private keys.
      Keys can be in any of the following formats:
      * An eth_account.LocalAccount object
      * An eth_keys.PrivateKey object
      * A raw private key as a hex string or byte string
.. important::
    Since this middleware signs transactions, it must always run after any middleware
    that modifies the transaction. Therefore, it is recommended to inject the signing
    middleware at the 0th layer of the middleware onion using
    w3.middleware_onion.inject(SignAndSendRawMiddlewareBuilder.build(...), layer=0).
    Ensure that any transaction-modifying middleware exists in a higher layer within the
    onion so that it runs before the signing middleware.
.. note::
    If used with ExtraDataToPOAMiddleware, the injection order doesn't matter, as
    the extraData field isn't involved in transaction signing. The key is ensuring
    SignAndSendRawMiddlewareBuilder runs after any middleware that modifies the
    transaction.

.. code-block:: python
   >>> from web3 import Web3, EthereumTesterProvider
   >>> w3 = Web3(EthereumTesterProvider)
   >>> from web3.middleware import SignAndSendRawMiddlewareBuilder
   >>> from eth_account import Account
   >>> acct = Account.create('KEYSMASH FJAFJKLDSKF7JKFDJ 1530')
   >>> w3.middleware_onion.inject(SignAndSendRawMiddlewareBuilder.build(acct), layer=0)
   >>> w3.eth.default_account = acct.address
:ref:Hosted nodes<local_vs_hosted> (like Infura or Alchemy) only support signed
transactions. This often results in send_raw_transaction being used repeatedly.
Instead, we can automate this process with
SignAndSendRawMiddlewareBuilder.build(private_key_or_account).
.. code-block:: python
    >>> from web3 import Web3
    >>> w3 = Web3(Web3.HTTPProvider('HTTP_ENDPOINT'))
    >>> from web3.middleware import SignAndSendRawMiddlewareBuilder
    >>> from eth_account import Account
    >>> import os
    >>> acct = w3.eth.account.from_key(os.environ.get('PRIVATE_KEY'))
    >>> w3.middleware_onion.inject(SignAndSendRawMiddlewareBuilder.build(acct), layer=0)
    >>> w3.eth.default_account = acct.address
    >>> # use eth_sendTransaction to automatically sign and send the raw transaction
    >>> w3.eth.send_transaction(tx_dict)
    HexBytes('0x09511acf75918fd03de58141d2fd409af4fd6d3dce48eb3aa1656c8f3c2c5c21')
Similarly, with AsyncWeb3:
.. code-block:: python
    >>> from web3 import AsyncWeb3
    >>> async_w3 = AsyncWeb3(AsyncHTTPProvider('HTTP_ENDPOINT'))
    >>> from web3.middleware import SignAndSendRawMiddlewareBuilder
    >>> from eth_account import Account
    >>> import os
    >>> acct = async_w3.eth.account.from_key(os.environ.get('PRIVATE_KEY'))
    >>> async_w3.middleware_onion.inject(SignAndSendRawMiddlewareBuilder.build(acct), layer=0)
    >>> async_w3.eth.default_account = acct.address
    >>> # use eth_sendTransaction to automatically sign and send the raw transaction
    >>> await async_w3.eth.send_transaction(tx_dict)
    HexBytes('0x09511acf75918fd03de58141d2fd409af4fd6d3dce48eb3aa1656c8f3c2c5c21')
Now you can send a transaction from acct.address without having to build and sign each raw transaction.
When making use of this signing middleware, when sending dynamic fee transactions (recommended over legacy transactions),
the transaction type of 2 (or '0x2') is necessary. This is because transaction signing is validated based
on the transaction type parameter. This value defaults to '0x2' when maxFeePerGas and / or
maxPriorityFeePerGas are present as parameters in the transaction as these params imply a dynamic fee transaction.
Since these values effectively replace the legacy gasPrice value, do not set a gasPrice for dynamic fee transactions.
Doing so will lead to validation issues.
.. code-block:: python
   # dynamic fee transaction, introduced by EIP-1559:
   >>> dynamic_fee_transaction = {
   ...     'type': '0x2',  # optional - defaults to '0x2' when dynamic fee transaction params are present
   ...     'from': acct.address,  # optional if w3.eth.default_account was set with acct.address
   ...     'to': receiving_account_address,
   ...     'value': 22,
   ...     'maxFeePerGas': 2000000000,  # required for dynamic fee transactions
   ...     'maxPriorityFeePerGas': 1000000000,  # required for dynamic fee transactions
   ... }
   >>> w3.eth.send_transaction(dynamic_fee_transaction)
A legacy transaction still works in the same way as it did before EIP-1559 was introduced:
.. code-block:: python
   >>> legacy_transaction = {
   ...     'to': receiving_account_address,
   ...     'value': 22,
   ...     'gasPrice': 123456,  # optional - if not provided, gas_price_strategy (if exists) or eth_gasPrice is used
   ... }
   >>> w3.eth.send_transaction(legacy_transaction)

Creating Custom Middleware
--------------------------
To write your own middleware, create a class and extend from the base Web3Middleware
class, then override only the parts of the middleware that make sense for your use case.
.. note:: The Middleware API borrows from the Django middleware API introduced
          in version 1.10.0.
If all you need is to modify the params before a request is made, you can override
the request_processor method, make the necessary tweaks to the params, and pass the
arguments to the next element in the middleware stack. Need to do some processing on the
response? Override the response_processor method and return the modified response.
The pattern:
.. code-block:: python
    from web3.middleware import Web3Middleware
    class CustomMiddleware(Web3Middleware):
        def request_processor(self, method, params):
            # Pre-request processing goes here before passing to the next middleware.
            return (method, params)
        def response_processor(self, method, response):
            # Response processing goes here before passing to the next middleware.
            return response
        # If your provider is asynchronous, override the async methods instead:
        async def async_request_processor(self, method, params):
            # Pre-request processing goes here before passing to the next middleware.
            return (method, params)
        async def async_response_processor(self, method, response):
            # Response processing goes here before passing to the next middleware.
            return response

If you wish to prevent making a call under certain conditions, you can override the
wrap_make_request method. This allows for defining pre-request processing,
skipping or making the request under certain conditions, as well as response
processing before passing it to the next middleware.

.. code-block:: python
    from web3.middleware import Web3Middleware
    class CustomMiddleware(Web3Middleware):
        def wrap_make_request(self, make_request):
            def middleware(method, params):
                # pre-request processing goes here
                response = make_request(method, params)  # make the request
                # response processing goes here
                return response
            return middleware
        # If your provider is asynchronous, override the async method instead:
        async def async_wrap_make_request(self, make_request):
            async def middleware(method, params):
                # pre-request processing goes here
                response = await make_request(method, params)
                # response processing goes here
                return response
            return middleware

Custom middleware can be added to the stack via the class itself, using the
:ref:middleware_stack_api. The name kwarg is optional. For example:
.. code-block:: python
    from web3 import Web3
    from my_module import (
        CustomMiddleware,
    )
    w3 = Web3(HTTPProvider(endpoint_uri="..."))
    # add the middleware to the stack as the class
    w3.middleware_onion.add(CustomMiddleware, name="custom_middleware")
---
Migration  Copy Section
Migration Guide
===============
.. _migrating_v6_to_v7:
Migrating from v6 to v7
-----------------------
web3.py follows Semantic Versioning <http://semver.org>_, which means that
version 7 introduced backwards-incompatible changes. If you're upgrading from
web3.py v6 or earlier, you can expect to need to make some changes. Refer
to this guide for a summary of breaking changes when updating from v6 to
v7. If you are more than one major version behind, you should also review
the migration guides for the versions in between.

Provider Updates
~~~~~~~~~~~~~~~~

WebSocketProvider

WebsocketProviderV2, introduced in web3.py v6, has taken priority over the
legacy WebsocketProvider. The LegacyWebSocketProvider has been deprecated in
v7 and is slated for removal in the next major version of the library. In summary:
- WebsocketProvider -> LegacyWebSocketProvider (and deprecated)
- WebsocketProviderV2 -> WebSocketProvider
If migrating from WebSocketProviderV2 to WebSocketProvider, you can expect the
following changes:
- Instantiation no longer requires the persistent_websocket method:
  .. code-block:: python
      # WebsocketsProviderV2:
      AsyncWeb3.persistent_websocket(WebsocketProviderV2('...'))
      # WebSocketProvider:
      AsyncWeb3(WebSocketProvider('...'))
- Handling incoming subscription messages now occurs under a more flexible namespace:
  socket. The AsyncIPCProvider uses the same API to listen for messages via
  an IPC socket.
  .. code-block:: python
    # WebsocketsProviderV2:
    async for message in w3.ws.process_subscriptions():
      ...
    # WebSocketProvider:
    async for message in w3.socket.process_subscriptions():
      ...

AsyncIPCProvider (non-breaking feature)

An asynchronous IPC provider, AsyncIPCProvider, is newly available in v7.
This provider makes use of some of the same internals that the new WebSocketProvider
introduced, allowing it to also support eth_subscription.

EthereumTesterProvider

/ Detailed source-code truncated for AI context efficiency. /

abi_middleware is no longer necessary and has been removed. All of the functionality
of the abi_middleware was already handled by web3.py's ABI formatters. For additional
context: a bug in the ENS name-to-address middleware would override the formatters. Fixing
this bug has removed the need for the abi_middleware.

Caching Middleware

The following middleware have been removed:
- simple_cache_middleware
- latest_block_based_cache_middleware
- time_based_cache_middleware
All caching middleware has been removed in favor of a decorator/wrapper around the
make_request methods of providers with configuration options on the provider class.
The configuration options are outlined in the documentation in the
:ref:request_caching section.
If desired, the previous caching middleware can be re-created using the new class-based
middleware model overriding the wrap_make_request (or async_wrap_make_request)
method in the middleware class.

Result Generating Middleware

The following middleware have been removed:
- fixture_middleware
- result_generator_middleware
The fixture_middleware and result_generator_middleware which were used for
testing/mocking purposes have been removed. These have been replaced internally by the
RequestMocker class, utilized for testing via a request_mocker pytest fixture.

HTTP Retry Request Middleware

The http_retry_request_middleware has been removed in favor of a configuration
option on the HTTPProvider and AsyncHTTPProvider classes. The configuration
options are outlined in the documentation in the :ref:http_retry_requests section.

Normalize Request Parameters Middleware

The normalize_request_parameters middleware was not used anywhere internally and
has been removed.

Remaining camelCase -> snake_case Updates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following arguments have been renamed across the library from camelCase to
snake_case in all methods where they are passed in as a kwarg.
- fromBlock -> from_block
- toBlock -> to_block
- blockHash -> block_hash
Note that if a dictionary is used instead, say to a call such as eth_getLogs, the
keys in the dictionary should be camelCase. This is because the dictionary is passed
directly to the JSON-RPC request, where the keys are expected to be in camelCase.

Changes to Exception Handling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All Python standard library exceptions that were raised from within web3.py have
been replaced with custom Web3Exception classes. This change allows for better
control over exception handling, being able to distinguish between exceptions raised
by web3.py and those raised from elsewhere in a codebase. The following exceptions
have been replaced:
- AssertionError -> Web3AssertionError
- ValueError -> Web3ValueError
- TypeError -> Web3TypeError
- AttributeError -> Web3AttributeError
A new MethodNotSupported exception is now raised when a method is not supported by
web3.py. This allows a user to distinguish between when a method is not available on
the current provider, MethodUnavailable, and when a method is not supported by
web3.py under certain conditions, MethodNotSupported.
A MismatchedABI exception is now raised instead of a Web3ValidationError in
cases where an ABI is not compatible with the data being passed to it. This change
allows for more specific error handling when using certain ABI types.

JSON-RPC Error Handling

Rather than a ValueError being replaced with a Web3ValueError when a JSON-RPC
response comes back with an error object, a new Web3RPCError exception is
now raised to provide more distinction for JSON-RPC error responses. Some previously
existing exceptions now extend from this class since they too are related to JSON-RPC
errors:
- MethodUnavailable
- BlockNotFound
- TransactionNotFound
- TransactionIndexingInProgress

End of Support and Feature Removals
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Python 3.7 Support Dropped

Python 3.7 support has been dropped in favor of Python 3.8+. Python 3.7 is no longer
supported by the Python core team, and we want to focus our efforts on supporting
the latest versions of Python.

EthPM Module Removed

The EthPM module has been removed from the library. It was not widely used and has not
been functional since around October 2022. It was deprecated in v6 and has been
completely removed in v7.
Types in the
eth_typing.ethpm <https://github.com/ethereum/eth-typing/blob/ef9c2d566b7747bb6799214e2c89006b8cde4c36/eth_typing/ethpm.py>_
module have been deprecated and will be removed from eth-typing in the next major
release.

Geth Miner Namespace Removed

The geth.miner namespace, deprecated in v6, has been removed in v7.
The miner namespace was used for managing the concept of a miner in geth. This is
no longer a feature in geth and is planned for complete removal in the future, with
Ethereum having transitioned to proof-of-stake.

Geth Personal Namespace Removed

The geth.personal namespace, deprecated in v6, has been removed in v7.
The personal namespace was used for managing accounts and keys and was deprecated
in geth in v1.11.0. Geth has moved to using clef for account and key management.

ABI Types Removed

/ Detailed source-code truncated for AI context efficiency. /

In v5, some web3.py exceptions inherited from AttributeError, namely:
- NoABIFunctionsFound
- NoABIFound
- NoABIEventsFound
Others inherited from ValueError, namely:
- InvalidAddress
- NameNotFound
- LogTopicError
- InvalidEventABI
Now web3.py exceptions inherit from the same base Web3Exception.
As such, any code that was expecting a ValueError or an AttributeError from
web3.py must update to expecting one of the exceptions listed above, or
Web3Exception.
Similarly, exceptions raised in the EthPM and ENS modules inherit from the base
EthPMException and ENSException, respectively.
ValidationError

The Python dev tooling ecosystem is moving towards standardizing
ValidationError, so users know that they're catching the correct
ValidationError. The base ValidationError is imported from
eth_utils. However, we also wanted to empower users to catch all errors emitted
by a particular module. So we now have a Web3ValidationError, EthPMValidationError,
and an ENSValidationError that all inherit from the generic
eth_utils.exceptions.ValidationError.
Web3 class split into Web3 and AsyncWeb3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Web3 class previously contained both sync and async methods. We've separated
Web3 and AsyncWeb3 functionality to tighten up typing. For example:
.. code-block:: python
    from web3 import Web3, AsyncWeb3
    w3 = Web3(Web3.HTTPProvider(<provider.url>))
    async_w3 = AsyncWeb3(AsyncWeb3.AsyncHTTPProvider(<provider.url>))
dict to AttributeDict conversion moved to middleware
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Eth module data returned as key-value pairs was previously automatically converted to
an AttributeDict by result formatters, which could cause problems with typing. This
conversion has been moved to a default attrdict_middleware where it can be easily
removed if necessary. See the Eth module <web3.eth.html#web3.eth.Eth>_ docs for more detail.
Other Misc Changes
~~~~~~~~~~~~~~~~~~
- InfuraKeyNotFound exception has been changed to InfuraProjectIdNotFound
- SolidityError has been removed in favor of ContractLogicError
- When a method is unavailable from a node provider (i.e. a response error
  code of -32601 is returned), a MethodUnavailable error is
  now raised instead of ValueError
- Logs' data field was previously formatted with to_ascii_if_bytes, now formatted to HexBytes
- Receipts' type field was previously not formatted, now formatted with to_integer_if_hex
Removals
~~~~~~~~
- Removed unused IBAN module
- Removed WEB3_INFURA_API_KEY environment variable in favor of WEB3_INFURA_PROJECT_ID
- Removed Kovan auto provider
- Removed deprecated sha3 and soliditySha3 methods in favor of keccak and solidityKeccak
- Remove Parity Module and References

Other notable changes
~~~~~~~~~~~~~~~~~~~~~
- The ipfshttpclient library is now opt-in via a web3 install extra.
  This only affects the ethpm ipfs backends, which rely on the library.

.. _migrating_v4_to_v5:
Migrating from v4 to v5
-----------------------
Web3.py follows Semantic Versioning <http://semver.org>_, which means
that version 5 introduced backwards-incompatible changes. If your
project depends on Web3.py v4, then you'll probably need to make some changes.
Here are the most common required updates:
Python 3.5 no longer supported
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You will need to upgrade to either Python 3.6 or 3.7
eth-abi v1 no longer supported
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You will need to upgrade the eth-abi dependency to v2
Changes to base API
~~~~~~~~~~~~~~~~~~~
JSON-RPC Updates

In v4, JSON-RPC calls that looked up transactions or blocks and
didn't find them, returned None. Now if a transaction or
block is not found, a BlockNotFound or a TransactionNotFound
error will be thrown as appropriate. This applies to the
following web3 methods:
- :meth:~web3.eth.Eth.getTransaction will throw a TransactionNotFound error
- :meth:~web3.eth.Eth.getTransactionReceipt will throw a TransactionNotFound error
- :meth:~web3.eth.Eth.getTransactionByBlock will throw a TransactionNotFound error
- :meth:~web3.eth.Eth.getTransactionCount will throw a BlockNotFound error
- :meth:~web3.eth.Eth.getBlock will throw a BlockNotFound error
- :meth:~web3.eth.Eth.getUncleCount will throw a BlockNotFound error
- :meth:~web3.eth.Eth.getUncleByBlock will throw a BlockNotFound error
Removed Methods

- contract.buildTransaction was removed for contract.functions.buildTransaction.<method name>
- contract.deploy was removed for contract.constructor.transact
- contract.estimateGas was removed for contract.functions.<method name>.estimateGas
- contract.call was removed for contract.<functions/events>.<method name>.call
- contract.transact was removed for contract.<functions/events>.<method name>.transact
- contract.eventFilter was removed for contract.events.<event name>.createFilter
- middleware_stack was renamed to :meth:~Web3.middleware_onion
- web3.miner.hashrate was a duplicate of :meth:~web3.eth.Eth.hashrate and was removed.
- web3.version.network was a duplicate of :meth:~web3.net.Net.version and was removed.
- web3.providers.tester.EthereumTesterProvider and web3.providers.tester.TestRPCProvider have been removed for :meth:~web3.providers.eth_tester.EthereumTesterProvider
- web3.eth.enableUnauditedFeatures was removed
- web3.txpool was moved to :meth:~web3.geth.txpool
- web3.version.node was removed for web3.clientVersion
- web3.version.ethereum was removed for :meth:~web3.eth.Eth.protocolVersion
- Relocated personal RPC endpoints to reflect Parity and Geth implementations:
  - web3.personal.listAccounts was removed for :meth:~web3.geth.personal.listAccounts or :meth:~web3.parity.personal.listAccounts
  - web3.personal.importRawKey was removed for :meth:~web3.geth.personal.importRawKey or :meth:~web3.parity.personal.importRawKey
  - web3.personal.newAccount was removed for :meth:~web3.geth.personal.newAccount or :meth:~web3.parity.personal.newAccount
  - web3.personal.lockAccount was removed for :meth:~web3.geth.personal.lockAccount
  - web3.personal.unlockAccount was removed for :meth:~web3.geth.personal.unlockAccount or :meth:~web3.parity.personal.unlockAccount
  - web3.personal.sendTransaction was removed for :meth:~web3.geth.personal.sendTransaction or :meth:~web3.parity.personal.sendTransaction
- Relocated web3.admin module to web3.geth namespace
- Relocated web3.miner module to web3.geth namespace
Deprecated Methods

Expect the following methods to be removed in v6:
- web3.sha3 was deprecated for :meth:~Web3.keccak
- web3.soliditySha3 was deprecated for :meth:~Web3.solidityKeccak
- :meth:~web3.net.Net.chainId was deprecated for :meth:~web3.eth.Eth.chainId.
  Follow issue #1293 <https://github.com/ethereum/web3.py/issues/1293>_ for details
- web3.eth.getCompilers() was deprecated and will not be replaced
- :meth:~web3.eth.getTransactionFromBlock() was deprecated for :meth:~Web3.getTransactionByBlock
Deprecated ConciseContract and ImplicitContract

/ Detailed source-code truncated for AI context efficiency. /

In v4 Web3.toDecimal() is renamed: :meth:~Web3.toInt for improved clarity. It does not return a :class:decimal.Decimal, it returns an :class:int.

Removed Methods

- Web3.toUtf8 was removed for :meth:~Web3.toText.
- Web3.fromUtf8 was removed for :meth:~Web3.toHex.
- Web3.toAscii was removed for :meth:~Web3.toBytes.
- Web3.fromAscii was removed for :meth:~Web3.toHex.
- Web3.fromDecimal was removed for :meth:~Web3.toHex.
Provider Access
~~~~~~~~~~~~~~~~~
In v4, w3.currentProvider was removed, in favor of w3.providers.
Disambiguating String Inputs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There are a number of places where an arbitrary string input might be either
a byte-string that has been hex-encoded, or unicode characters in text.
These are named hexstr and text in Web3.py.
You specify which kind of :class:str you have by using the appropriate
keyword argument. See examples in :ref:overview_type_conversions.
In v3, some methods accepted a :class:str as the first positional argument.
In v4, you must pass strings as one of hexstr or text keyword arguments.
Notable methods that no longer accept ambiguous strings:
- :meth:~Web3.sha3
- :meth:~Web3.toBytes
Contracts
~~~~~~~~~
- When a contract returns the ABI type string, Web3.py v4 now returns a :class:str
  value by decoding the underlying bytes using UTF-8.
- When a contract returns the ABI type bytes (of any length),
  Web3.py v4 now returns a :class:bytes value
Personal API
~~~~~~~~~~~~
w3.personal.signAndSendTransaction is no longer available. Use
:meth:w3.personal.sendTransaction() <web3.personal.sendTransaction> instead.
---
Overview  Copy Section
.. _overview:
Overview
========
The purpose of this page is to give you a sense of everything web3.py can do
and to serve as a quick reference guide. You'll find a summary of each feature
with links to learn more.
Configuration
-------------
After installing web3.py (via pip install web3), you'll need to configure
a provider endpoint and any middleware you want to use beyond the defaults.

Providers
~~~~~~~~~
:doc:providers are how web3.py connects to a blockchain. The library comes with the
following built-in providers:
- :class:~web3.providers.rpc.HTTPProvider for connecting to http and https based JSON-RPC servers.
- :class:~web3.providers.ipc.IPCProvider for connecting to ipc socket based JSON-RPC servers.
- :class:~web3.providers.async_rpc.AsyncHTTPProvider for connecting to http and https based JSON-RPC servers asynchronously.
- :class:~web3.providers.persistent.AsyncIPCProvider for connecting to ipc socket based JSON-RPC servers asynchronously via a persistent connection.
- :class:~web3.providers.persistent.WebSocketProvider for connecting to websocket based JSON-RPC servers asynchronously via a persistent connection.
Examples

.. code-block:: python
   >>> from web3 import Web3, AsyncWeb3
   # IPCProvider:
   >>> w3 = Web3(Web3.IPCProvider('./path/to/filename.ipc'))
   >>> w3.is_connected()
   True
   # HTTPProvider:
   >>> w3 = Web3(Web3.HTTPProvider('http://127.0.0.1:8545'))
   >>> w3.is_connected()
   True
   # AsyncHTTPProvider:
   >>> w3 = AsyncWeb3(AsyncWeb3.AsyncHTTPProvider('http://127.0.0.1:8545'))
   >>> await w3.is_connected()
   True
   # -- Persistent Connection Providers -- #
   # WebSocketProvider:
   >>> w3 = await AsyncWeb3(AsyncWeb3.WebSocketProvider('ws://127.0.0.1:8546'))
   >>> await w3.is_connected()
   True
   # AsyncIPCProvider
   >>> w3 = await AsyncWeb3(AsyncWeb3.AsyncIPCProvider('./path/to/filename.ipc'))
   >>> await w3.is_connected()
   True

For more context, see the :doc:providers documentation.

Middleware
~~~~~~~~~~
Your web3.py instance may be further configured via :doc:middleware.
web3.py middleware is described using an onion metaphor, where each layer of
middleware may affect both the incoming request and outgoing response from your
provider. The documentation includes a :ref:visualization <Modifying_Middleware>
of this idea.
Several middleware are :ref:included by default <default_middleware>. You may add to
(:meth:add <Web3.middleware_onion.add>, :meth:inject <Web3.middleware_onion.inject>,
:meth:replace <Web3.middleware_onion.replace>) or disable
(:meth:remove <Web3.middleware_onion.remove>,
:meth:clear <Web3.middleware_onion.clear>) any of these middleware.

Accounts and Private Keys
-------------------------
Private keys are required to approve any transaction made on your behalf. The manner in
which your key is secured will determine how you create and send transactions in web3.py.
A local node, like Geth <https://geth.ethereum.org/>_, may manage your keys for you.
You can reference those keys using the :attr:web3.eth.accounts <web3.eth.Eth.accounts>
property.
A hosted node, like Infura <https://infura.io/>_, will have no knowledge of your keys.
In this case, you'll need to have your private key available locally for signing
transactions.
Full documentation on the distinction between keys can be found :ref:here <eth-account>.
The separate guide to :doc:transactions may also help clarify how to manage keys.

Base API
--------
The :ref:Web3 <web3_base> class includes a number of convenient utility functions:

Encoding and Decoding Helpers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- :meth:Web3.is_encodable() <web3.w3.is_encodable>
- :meth:Web3.to_bytes() <web3.Web3.to_bytes>
- :meth:Web3.to_hex() <web3.Web3.to_hex>
- :meth:Web3.to_int() <web3.Web3.to_int>
- :meth:Web3.to_json() <web3.Web3.to_json>
- :meth:Web3.to_text() <web3.Web3.to_text>

Address Helpers
~~~~~~~~~~~~~~~
- :meth:Web3.is_address() <web3.Web3.is_address>
- :meth:Web3.is_checksum_address() <web3.Web3.is_checksum_address>
- :meth:Web3.to_checksum_address() <web3.Web3.to_checksum_address>

Currency Conversions
~~~~~~~~~~~~~~~~~~~~
- :meth:Web3.from_wei() <web3.Web3.from_wei>
- :meth:Web3.to_wei() <web3.Web3.to_wei>

Cryptographic Hashing
~~~~~~~~~~~~~~~~~~~~~
- :meth:Web3.keccak() <web3.Web3.keccak>
- :meth:Web3.solidity_keccak() <web3.Web3.solidity_keccak>

web3.eth API
------------
The most commonly used APIs for interacting with Ethereum can be found under the
:ref:web3-eth namespace.

Fetching Data
~~~~~~~~~~~~~
Viewing account balances (:meth:get_balance <web3.eth.Eth.get_balance>), transactions
(:meth:get_transaction <web3.eth.Eth.get_transaction>), and block data
(:meth:get_block <web3.eth.Eth.get_block>) are some of the most common starting
points in web3.py.

API