Summary of Major Changes Between Python Versions

This post serves as a concise guide to the significant updates introduced in each new version of Python. It aims to assist in leveraging new features when updating your code base, or in implementing necessary safeguards for compatibility with previous versions.

The post is divided into two parts: the first part details the changes themselves, while the second part provides helpful tools, links, and resources for facilitating the process of upgrading code bases.

Versions

In this section, I have outlined significant updates to Python's syntax and its standard library, with the exception of the typing module, as I have primarily omitted modifications to modules. Changes to the C-API, byte-code, or other underlying components have not been covered.

For each segment, the end-of-life date (EOL) indicates when the Python Software Foundation will cease to offer security patches for a specific version.

Python 3.7 and earlier

This section is consolidated because, at the time of writing, all these versions have already reached their end-of-life (EOL) status. However, if you've been coding in Python for some time, you might not remember when these features were initially introduced.

• async and await (3.5+)
• matrix operator: a @ b (3.5+)
• type hints (3.5+)
• Formatted String Literals (aka f-strings) f"{something}" (3.6+)
• underscore in numeric literals 1_000_000 (3.6+)
• dictionaries are guaranteed insertion ordered (3.7+)
• contextvars (3.7+)
• dataclasses (3.7+)
• importlib.resources (3.7+)

Python 3.8 (EOL Oct 2024)

Assignment expressions

Also known as the Walrus operator

if (thing := get_thing()) is not None:
  do_something(thing)
else:
  raise Exception(f"Something is wrong with {thing}")        

Positional only parameters

def foo(a, b, /, c, d, *, e, f):
  # a, b: positional only
  # c, d: positional or keyword
  # e, f: keyword only        

Self documenting f-strings

# Before
f"user={user}"

# Now
f"{user=}"        

Importlib Metadata

import importlib.metadata
importlib.metadata.version("some-library")
# "2.3.4"
importlib.metadata.requires("some-library")
# ["thing==1.2.4", "other>=5"]
importlib.metadata.files("some-library")
# [...]        

Typing: TypedDict, Literal, Final, Protocol

• TypedDict - PEP 589
• Literal - PEP 586
• Final - PEP 591
• Protocol - PEP 544

Python 3.9 (EOL Oct 2025)

Typing: Builtin Generics

Can now use dict[...], list[...], set[...] etc instead of using typing.Dict, List, Set.

Remove Prefix/Suffix

Strings and related types now have the ability to utilize removeprefix and removesuffix methods for safer removal of substrings from the beginning or end. This approach is more secure than using string slicing, which necessitates accurate counting of the prefix length and updating the slice accordingly if the prefix changes.

if header.startswith("X-Forwarded-"):
  section = header.removeprefix("X-Forwarded-")        

Dict Union Operator (PEP 584)

combined_dict = dict_one | dict_two
updated_dict |= dict_three        

Annotations (PEP 593)

my_int: Annotated[int, SomeRange(0, 255)] = 0        

Zoneinfo (PEP 615)

IANA Time Zone Database is now part of standard library

import zoneinfo
some_zone = zoneinfo.ZoneInfo("Europe/Berlin")        

Python 3.10 (EOL Oct 2026)

Structural Pattern Matching (PEP 634, PEP 635, PEP 636)

See change log for more examples.

match command.split():
  case ["quit"]:
    print("Goodbye!")
    quit_game()
  case ["look"]:
    current_room.describe()
  case ["get", obj]:
    character.get(obj, current_room)
  case ["go", direction]:
    current_room = current_room.neighbor(direction)
  case [action]:
    ... # interpret single-verb action
  case [action, obj]:
    ... # interpret action, obj
  case _:
    ... # anything that didn't match        

Typing: Union using pipe

# Before
from typing import Optional, Union
thing: Optional[Union[str, list[str]]] = None

# Now
thing: str | list[str] | None = None        

Typing: ParamSpec (PEP 612)

Enables significantly improved transmission of typing data when utilizing Callable and other comparable types.

from typing import Awaitable, Callable, ParamSpec, TypeVar

P = ParamSpec("P")
R = TypeVar("R")

def add_logging(f: Callable[P, R]) -> Callable[P, Awaitable[R]]:
  async def inner(*args: P.args, **kwargs: P.kwargs) -> R:
    await log_to_database()
    return f(*args, **kwargs)
  return inner

@add_logging
def takes_int_str(x: int, y: str) -> int:
  return x + 7

await takes_int_str(1, "A") # Accepted
await takes_int_str("B", 2) # Correctly rejected by the type checker        

Typing: TypeAlias (PEP 613)

StrCache: TypeAlias = 'Cache[str]'  # a type alias
LOG_PREFIX = 'LOG[DEBUG]'  # a module constant        

Typing: TypeGuard (PEP 647)

_T = TypeVar("_T")

def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]:
  return len(val) == 2

def func(names: Tuple[str, ...]):
  if is_two_element_tuple(names):
    reveal_type(names)  # Tuple[str, str]
  else:
    reveal_type(names)  # Tuple[str, ...]        

Parenthesized Context Managers (PEP 617)

with (CtxManager() as example):
  ...

with (
  CtxManager1(), CtxManager2()
):
  ...

with (CtxManager1() as example, CtxManager2()):
  ...

with (CtxManager1(), CtxManager2() as example):
  ...

with (
  CtxManager1() as example1,
  CtxManager2() as example2,
):
  ...        

Dataclasses: slots, kw_only

Dataclass decorator now supports following:

• kw_only=True all parameters in __init__ will be marked keyword only.
• slots=True the generatred dataclass will use __slots__ for storing data.

Python 3.11 (EOL Oct 2027)

Tomllib

tomllib - Standard library TOML parser

Exception Groups (PEP 654)

PEP 654 introduces language features that enable a program to raise and handle multiple unrelated exceptions simultaneously. The builtin types ExceptionGroup and BaseExceptionGroup make it possible to group exceptions and raise them together, and the new except* syntax generalizes except to match subgroups of exception groups.

Enriching Exceptions with notes (PEP 678)

The add_note() method is added to BaseException. It can be used to enrich exceptions with context information that is not available at the time when the exception is raised. The added notes appear in the default traceback.

try:
  do_something()
except BaseException as e:
  e.add_note("this happened during do_something")
  raise        

Typing: Self (PEP 673)

class MyClass:
  @classmethod
  def from_hex(cls, s: str) -> Self:  # Self means instance of cls
    return cls(int(s, 16))
        
  def frobble(self, x: int) -> Self: # Self means this instance
    self.y >> x
    return self        

Typing: LiteralString (PEP 675)

The new LiteralString annotation may be used to indicate that a function parameter can be of any literal string type. This allows a function to accept arbitrary literal string types, as well as strings created from other literal strings. Type checkers can then enforce that sensitive functions, such as those that execute SQL statements or shell commands, are called only with static arguments, providing protection against injection attacks.

Typing: Marking TypedDict entries as [not] required (PEP 655)

# default is required
class Movie(TypedDict):
  title: str
  year: NotRequired[int]

# default is not-required
class Movie(TypedDict, total=False):
  title: Required[str]
  year: int        

Typing: Variadic Generics via TypeVarTuple (PEP 646)

PEP 484 previously introduced TypeVar, enabling creation of generics parameterised with a single type. PEP 646 adds TypeVarTuple, enabling parameterisation with an arbitrary number of types. In other words, a TypeVarTuple is a variadic type variable, enabling variadic generics.

This enables a wide variety of use cases. In particular, it allows the type of array-like structures in numerical computing libraries such as NumPy and TensorFlow to be parameterised with the array shape. Static type checkers will now be able to catch shape-related bugs in code that uses these libraries.

Typing: @dataclass_transform (PEP 681)

dataclass_transform may be used to decorate a class, metaclass, or a function that is itself a decorator. The presence of @dataclass_transform() tells a static type checker that the decorated object performs runtime “magic” that transforms a class, giving it dataclass-like behaviors.

# The create_model decorator is defined by a library.
@typing.dataclass_transform()
def create_model(cls: Type[T]) -> Type[T]:
  cls.__init__ = ...
  cls.__eq__ = ...
  cls.__ne__ = ...
  return cls

# The create_model decorator can now be used to create new model classes:
@create_model
class CustomerModel:
  id: int
  name: str        

Star unpacking expressions allowed in for statements:

This is officially supported syntax

for x in *a, *b:
  print(x)        

Python 3.12 (EOL Oct 2028)

Typing: Type Parameter Syntax (PEP 695)

Compact annotion of generic classes and functions

def max[T](args: Iterable[T]) -> T:
  ...

class list[T]:
  def __getitem__(self, index: int, /) -> T:
    ...

  def append(self, element: T) -> None:
    ...        

Ability to declare type aliases using type statement (generates TypeAliasType)

type Point = tuple[float, float]

# Type aliases can also be generic
type Point[T] = tuple[T, T]        

F-string changes (PEP 701)

Expression components inside f-strings can now be any valid Python expression, including strings reusing the same quote as the containing f-string, multi-line expressions, comments, backslashes, and unicode escape sequences.

Can re-use quotes (including nesting f-string statements

## Can re-use quotes
f"This is the playlist: {", ".join(songs)}"

f"{f"{f"{f"{f"{f"{1+1}"}"}"}"}"}" # '2'

## Multiline f-string with comments
f"This is the playlist: {", ".join([
  'Take me back to Eden',  # My, my, those eyes like fire
  'Alkaline',              # Not acid nor alkaline
  'Ascensionism'           # Take to the broken skies at last
])}"

## Backslashes / Unicode
f"This is the playlist: {"\n".join(songs)}"

f"This is the playlist: {"\N{BLACK HEART SUIT}".join(songs)}"        

Buffer protocol (PEP 688)

PEP 688 introduces a way to use the buffer protocol from Python code. Classes that implement the __buffer__() method are now usable as buffer types.

The new collections.abc.Buffer ABC provides a standard way to represent buffer objects, for example in type annotations. The new inspect.BufferFlags enum represents the flags that can be used to customize buffer creation.

Typing: Unpack for **kwargs typing (PEP 692)

from typing import TypedDict, Unpack

class Movie(TypedDict):
  name: str
  year: int

def foo(**kwargs: Unpack[Movie]):
  ...        

Typing: override decorator (PEP 698)

Ensure's that the method being overridden by a child class actually exists in a parent class.

from typing import override

class Base:
  def get_color(self) -> str:
    return "blue"

class GoodChild(Base):
  @override  # ok: overrides Base.get_color
  def get_color(self) -> str:
    return "yellow"

class BadChild(Base):
  @override  # type checker error: does not override Base.get_color
  def get_colour(self) -> str:
    return "red"        

Useful Things

Postponed Annotations (PEP 563)

In recent updates to Python, type annotations are initially processed as strings. This approach aids in avoiding circular imports, the necessity to enclose references in quotes before their declaration, and several other problems. Python 3.7 and later versions can utilize from __future__ import annotations to enable the interpreter to adopt this updated parsing method.

It's important to note that PEP 563, which introduced this feature, has been replaced by PEP 649. This newer proposal is scheduled for inclusion in Python version 3.13.

Typing Extensions

This library back-ports typing features so that they are available to type checkers inspecting older code bases.

import sys

if sys.version_info < (3, 10):
  from typing_extensions import TypeAlias
else:
  from typing import TypeAlias