How CPython Runs Your Code: Bytecode, the GIL, Reference Counting and the Import System

By 0xmote · Sep 11, 2026 · 14 min read · Updated Oct 15, 2026 Advanced tutorial

Key takeaways

Beyond syntax: how CPython compiles to bytecode and runs the eval loop, what the GIL implies for CPU vs I/O workloads, refcount plus generational GC, how import paths and caches interact, and production-grade logging, config, and observability.

This article is about how CPython executes code and what that means for running Python services. Unless noted, “Python” means the CPython reference implementation; PyPy, GraalPy, and others differ in their JIT, garbage collector, and threading model.

For syntax and library breadth, start with Python basics and the modules lesson.

CPython interpreter architecture

CPython compiles source code to bytecode before running it. Parsing and compilation happen once per module (and the result is cached in __pycache__/*.pyc), so the eval loop only has to walk a compact instruction stream.

End-to-end pipeline

Source text from a .py file or a string is tokenized and parsed into an abstract syntax tree (AST). The compiler turns the AST into a code object containing bytecode, and the eval loop in Python/ceval.c executes that bytecode inside a frame. Typing one line in the REPL goes through the same steps as running a file.

Code objects and frames

Defining a function produces a code object (types.CodeType) that holds the bytecode, constants, local variable names, and the table mapping instructions to line numbers. At call time, CPython creates a frame that points at that code object and holds the local variables and the operand stack for that particular call.

def add(a, b):
    return a + b

print(add.__code__.co_name)       # 'add'
print(add.__code__.co_varnames)   # ('a', 'b')
print(add.__code__.co_code[:10])  # first bytes of bytecode

Profilers, tracers, and debuggers work by walking this chain of frames, and a traceback is simply the frame chain recorded as an exception propagates through it.

Inspecting bytecode with `dis`

The dis module renders opcodes in a readable form.

import dis

def f(x):
    return x * 2

dis.dis(f)

On Python 3.11 and later you will see opcodes such as LOAD_FAST, LOAD_CONST, BINARY_OP, and RETURN_VALUE (exact names change between releases). CPython’s VM is a stack machine: LOAD_* instructions push operands, and BINARY_OP pops two values and pushes the result.

Object model and `PyObject`

At the C level every value is a PyObject*, and its type object (PyTypeObject) holds function pointers (“slots”) for operations such as addition, attribute access, and calling. When the interpreter executes a + b, it calls the left operand’s addition slot and falls back to the right operand’s reflected slot (__radd__) if the first returns NotImplemented.

The practical consequence is that each small Python operation costs an opcode dispatch plus a slot lookup, so a tight loop over millions of tiny operations is slow compared with the same work done in C. Moving the loop into NumPy, a C extension, or Cython removes that per-element overhead; a profiler should tell you whether that loop is actually where the time goes.

The eval loop and specialization

The eval loop fetches an opcode and jumps to the code that handles it. Since Python 3.11 (PEP 659), the interpreter is “specializing adaptive”: after an instruction has run a few times, it can be rewritten in place into a variant specialized for the types it has seen, for example a BINARY_OP that has only ever added two ints. That made 3.11 noticeably faster on typical pure-Python code without any change to user code.

PEP 523 lets a C extension install its own frame evaluation function. Application code rarely needs it, but it is the hook that some debuggers and JIT projects use to attach to the interpreter.

The Global Interpreter Lock (GIL)

What the GIL serializes

The GIL is a mutex that a thread must hold to execute Python bytecode in a standard CPython build. A process can have many threads, but only one of them runs Python code at any instant.

Why it exists

Every object carries a reference count that is modified constantly. With one global lock, those updates and most of the interpreter’s internal state need no further synchronization, and C extensions can assume they are not running concurrently with other Python code. Removing the lock means making reference counting thread-safe without making single-threaded code much slower, which is what the free-threaded build had to solve.

When the GIL is released

Blocking I/O calls release the GIL while they wait, and well-written C extensions (NumPy, hashlib, zlib, many database drivers) release it around long native computations. Other threads can run Python code during that time, which is why threads work well for I/O-bound concurrency even with the GIL.

Pure-Python CPU-bound code does not release the GIL voluntarily, so two such threads take turns instead of running in parallel and finish no sooner than one thread doing both jobs.

`threading` vs `multiprocessing`

Goal	Typical choice
Concurrent I/O (many sockets/clients)	`asyncio`, `threading`, or both
CPU-bound parallelism	`multiprocessing`, `ProcessPoolExecutor`, or task queues (Celery, RQ)
Native code releasing the GIL	Threads can scale for that section; depends on the library

Free-threaded builds

PEP 703 added an optional build without the GIL. It shipped as experimental in Python 3.13 and became an officially supported (but still optional, non-default) build in 3.14. Before using it in production, check that every C extension you depend on publishes free-threaded wheels, since an extension that is not marked compatible re-enables the GIL at import time, and measure single-threaded performance, which is lower than the default build.

Thread switching and `sys.setswitchinterval`

A thread holding the GIL is asked to release it after a switch interval, 5 ms by default (sys.getswitchinterval() returns 0.005), so that one CPU-bound thread cannot starve the others. This gives fairness between threads, not parallelism; lowering the interval increases switching overhead without using more cores.

Memory management and garbage collection

Reference counting

CPython’s primary mechanism is reference counting: every object tracks how many references point to it, and when the count drops to zero the object is freed immediately. For acyclic data this makes deallocation prompt and predictable, which is why a file object that goes out of scope is closed right away in CPython (other implementations do not promise that, so use with).

Cycles and the cyclic collector

Objects that reference each other keep each other’s counts above zero even when nothing else can reach them. A separate cycle detector periodically examines container objects and frees groups that are only reachable from each other. Through 3.13 it is generational: new objects start in generation 0 and move to older generations when they survive a collection, on the observation that most objects die young. Python 3.14 replaced this with an incremental collector that uses a young and an old generation, to shorten individual pauses.

import gc

class Node:
    def __init__(self):
        self.ref = None

a, b = Node(), Node()
a.ref, b.ref = b, a  # cycle

del a, b             # refcounts stay at 1; nothing is freed yet
print(gc.collect())  # number of unreachable objects found

`slots` and memory

Declaring __slots__ stores attributes in fixed slots instead of a per-instance __dict__, which saves memory when you create millions of instances. The cost is that you cannot add attributes that are not declared, and multiple inheritance from several classes with non-empty slots fails, so it is a choice for specific high-volume classes rather than a default.

pymalloc and small-object allocation

Objects of 512 bytes or less come from pymalloc, CPython’s own allocator, which carves pools out of larger arenas instead of calling malloc for each object. You do not configure it, but a workload that creates and discards huge numbers of small objects still pays in allocation work and cache misses, and arenas are only returned to the OS when completely empty (see the FAQ below).

`weakref` to avoid strong cycles

A weak reference does not increase the reference count. Caches (weakref.WeakValueDictionary), parent pointers in trees, and observer callbacks are common places to use one, so that the structure does not form a strong cycle in the first place and the object can be freed by reference counting alone.

Thresholds and the `gc` module

gc.get_threshold() and gc.set_threshold() control how often collections run. Change them only when profiling shows GC pauses or memory growth that matters, and measure the effect; some services call gc.freeze() after start-up so that long-lived objects created during initialization are excluded from future scans.

Import system and modules

The `sys.modules` cache

After the first successful import, a module object is stored in sys.modules. Later import statements for the same name return that object without re-running the module’s code, which is why module-level code executes once per process.

`sys.path` resolution order

For a top-level import, the path-based finder walks sys.path in order: the directory of the script being run (or the current directory for -c and the REPL), then entries from PYTHONPATH, then the standard library, then site-packages (where .pth files can add more, including editable installs). The first regular module or package found wins, which is how a file named json.py next to your script shadows the standard library. Namespace packages are the exception: portions without __init__.py are only used if no regular package of that name is found anywhere on the path. When the wrong code is imported, print module.__file__ and sys.path before changing anything else.

`importlib` for explicit control

importlib exposes the import machinery to Python code, which is useful for plugin systems and imports whose names are only known at runtime.

import importlib

m = importlib.import_module("json")
importlib.reload(m)  # re-executes the module's code in the same module object

reload re-runs the module’s code but does not update references other modules already hold to the old functions and classes, and objects created from old classes keep pointing at them. That makes it a development convenience, not something to use in a running service.

Packages and namespace packages

A directory with an __init__.py is a regular package. A directory without one can be part of a namespace package (PEP 420), whose portions may live in several directories on sys.path and are merged under one name. That is how separately installed distributions can share a top-level name such as google.cloud, and it also explains confusing cases where a stray directory without __init__.py is silently imported as an empty package.

Circular imports

When module A imports B and B imports A at module level, the second import finds a partially initialized module in sys.modules. import a succeeds but from a import name fails with ImportError: cannot import name ... (most likely due to a circular import) if name has not been defined yet. The durable fix is to break the cycle by moving shared code into a third module; importing inside a function delays the lookup and also works.

`importlib.metadata` for distributions

importlib.metadata reads metadata of installed distributions at runtime, such as version("requests") or the entry points a package registers. CLIs use it to print their own version, and plugin systems use entry points to discover installed plugins.

Production Python patterns

Logging, not `print`

The logging module gives you levels, per-module loggers, and handlers that can be reconfigured without touching the code that emits messages. Structured (JSON) output makes logs searchable by field in a log aggregator.

import logging
import json

class JsonFormatter(logging.Formatter):
    def format(self, record):
        payload = {
            "level": record.levelname,
            "msg": record.getMessage(),
            "logger": record.name,
        }
        # values passed via extra= become attributes on the record
        if hasattr(record, "user_id"):
            payload["user_id"] = record.user_id
        return json.dumps(payload, ensure_ascii=False)

h = logging.StreamHandler()
h.setFormatter(JsonFormatter())
logging.basicConfig(level=logging.INFO, handlers=[h])

logging.getLogger("app").info("user_login", extra={"user_id": 123})

A formatter only outputs what it puts in the payload. Keys passed through extra= are set as attributes on the LogRecord, so a custom formatter that does not read them drops them silently, a common reason fields “go missing” in JSON logs. Libraries such as python-json-logger or structlog handle this generically.

Configuration and secrets

Keep environment-specific values out of the code, as the 12-Factor methodology suggests: read them from environment variables or a secret store, and validate them at start-up with a typed settings class (for example Pydantic Settings) so a missing variable fails the deploy instead of the first request that needs it.

WSGI and ASGI deployment

Synchronous frameworks run under a WSGI server such as Gunicorn; async frameworks run under an ASGI server such as Uvicorn or Hypercorn (Gunicorn can also manage Uvicorn workers). Because of the GIL, CPU-bound request handling scales with the number of worker processes, not threads, so the worker count is usually tied to CPU cores and limited by memory per process.

Observability

Metrics (request rate, latency histograms, error rate, queue depth) show that something is wrong, distributed traces (OpenTelemetry) show which service or call is slow, and profiles show which code is slow. cProfile is built in; py-spy can sample a running production process without restarting it, and tracemalloc tracks where memory is allocated.

Reproducible dependencies and containers

Pin the full dependency tree with a lock file (uv, Poetry, or pip-tools) so that a rebuild installs exactly what was tested, and run containers as a non-root user with a read-only root filesystem where the application allows it.

Security basics

Pass subprocess an argument list instead of a shell string so user input cannot inject shell syntax. Never call pickle.loads on data from an untrusted source, since unpickling can execute arbitrary code. Terminate TLS consistently at one layer (the load balancer or ingress) and know which hops behind it are plain HTTP.

Frequently Asked Questions (FAQ)

Q. Why doesn’t my Python process’s memory usage go down after I delete a large data structure?

Deleting the last reference frees the objects inside CPython, but that does not mean the memory goes back to the operating system. Small objects come from pymalloc’s arenas, and an arena can only be released when every object in it is freed, so fragmentation often keeps memory reserved for reuse by the same process. Measure with tracemalloc to see whether Python objects are actually leaking. For one-off heavy jobs, running them in a separate process that exits is the reliable way to give the memory back.