Python Packaging: Build, Publish and Install Libraries

Python packaging has been modernized by PEP 517, 518, and 621 — replacing the legacy setup.py with a declarative pyproject.toml. This guide covers the complete packaging workflow: structuring a library, choosing a build backend, building wheel and sdist distributions, publishing to PyPI and private registries, managing versions, and automating the release process with GitHub Actions.

Table of Contents

Package Structure

my-library/
├── pyproject.toml         ← all package metadata + build config
├── README.md              ← shown on PyPI
├── LICENSE                ← e.g., MIT
├── CHANGELOG.md
├── src/
│   └── mylib/            ← src layout (recommended)
│       ├── __init__.py
│       ├── core.py
│       ├── utils.py
│       └── py.typed       ← marker file: package ships type stubs
└── tests/
    ├── conftest.py
    └── test_core.py
src layout: Placing your package under src/ prevents accidental imports from the project root during development. Tests always import from the installed package, not from the source tree — catching install-time issues early.

pyproject.toml: Build Backends

The [build-system] section in pyproject.toml declares which build backend to use. The three most common are Hatchling (modern, fast), setuptools (legacy compat), and Flit (minimal). Poetry uses its own backend automatically.

# Option 1: Hatchling (recommended for new projects)
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

# Option 2: setuptools (best for C extensions, complex builds)
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"

# Option 3: Flit (minimal — for pure Python libraries only)
[build-system]
requires = ["flit_core>=3.9"]
build-backend = "flit_core.buildapi"

Package Metadata

[project]
name = "techoral-utils"
version = "1.2.0"
description = "Utility library for Techoral applications"
readme = "README.md"
license = {text = "MIT"}
authors = [{name = "Avinash", email = "avinash@techoral.com"}]
maintainers = [{name = "Techoral Team", email = "info@techoral.com"}]
keywords = ["utilities", "python", "fastapi"]
classifiers = [
    "Development Status :: 5 - Production/Stable",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: Software Development :: Libraries",
    "Typing :: Typed",
]
requires-python = ">=3.11"
dependencies = [
    "pydantic>=2.0,<3",
    "httpx>=0.27",
]

[project.optional-dependencies]
redis = ["redis>=5.0"]
dev = ["pytest>=8", "ruff>=0.4", "mypy>=1.10"]

[project.urls]
Homepage = "https://techoral.com"
Repository = "https://github.com/deonash/techoral-utils"
Documentation = "https://techoral.com/docs"
"Bug Tracker" = "https://github.com/deonash/techoral-utils/issues"

[project.scripts]
techoral-cli = "techoral_utils.cli:main"

[project.entry-points."techoral.plugins"]
redis = "techoral_utils.redis_plugin:RedisPlugin"

[tool.hatch.build.targets.wheel]
packages = ["src/techoral_utils"]

[tool.hatch.version]
path = "src/techoral_utils/__init__.py"

Building Distributions

pip install build

# Build both wheel and sdist
python -m build

# Build only wheel (faster, no source tarball)
python -m build --wheel

# Output in dist/
# techoral_utils-1.2.0-py3-none-any.whl   ← wheel (binary distribution)
# techoral-utils-1.2.0.tar.gz              ← sdist (source distribution)

# Inspect wheel contents
python -m zipfile -l dist/techoral_utils-1.2.0-py3-none-any.whl

# Test install locally before publishing
pip install dist/techoral_utils-1.2.0-py3-none-any.whl
python -c "import techoral_utils; print(techoral_utils.__version__)"

Publishing to PyPI

pip install twine

# Validate distributions before upload
twine check dist/*

# Upload to Test PyPI first (always test here first!)
twine upload --repository testpypi dist/*
# Test install from Test PyPI:
pip install -i https://test.pypi.org/simple/ techoral-utils

# Upload to production PyPI
twine upload dist/*
# Prompts for username (__token__) and password (your API token)

# Or use environment variables (for CI)
export TWINE_USERNAME=__token__
export TWINE_PASSWORD=pypi-your-api-token-here
twine upload dist/*
API tokens: Never use your PyPI password directly. Create a scoped API token at pypi.org/manage/account/token/ — scope it to a single project for least privilege. Store it in your CI secrets, never in code.

Private Registries

# pip.conf — configure private registry (system-wide or per-user)
# ~/.config/pip/pip.conf (Linux/Mac) or %APPDATA%\pip\pip.ini (Windows)
[global]
extra-index-url = https://pypi.mycompany.com/simple/
trusted-host = pypi.mycompany.com

# Install from private registry
pip install techoral-internal --index-url https://pypi.mycompany.com/simple/

# Publish to private registry (e.g., JFrog Artifactory, AWS CodeArtifact)
twine upload \
  --repository-url https://pypi.mycompany.com/ \
  --username your-user \
  --password your-password \
  dist/*

# AWS CodeArtifact — get token dynamically
export CODEARTIFACT_TOKEN=$(aws codeartifact get-authorization-token \
  --domain mycompany --domain-owner 123456789012 \
  --query authorizationToken --output text)
twine upload \
  --repository-url https://mycompany-123456789012.d.codeartifact.us-east-1.amazonaws.com/pypi/my-repo/legacy/ \
  --username aws --password $CODEARTIFACT_TOKEN \
  dist/*

Semantic Versioning

# src/techoral_utils/__init__.py
__version__ = "1.2.0"

# Expose version in CLI
import importlib.metadata

def get_version() -> str:
    return importlib.metadata.version("techoral-utils")
# Bump version with bump2version (or use Poetry's version command)
pip install bump2version

# .bumpversion.cfg
[bumpversion]
current_version = 1.2.0
commit = True
tag = True

[bumpversion:file:src/techoral_utils/__init__.py]
[bumpversion:file:pyproject.toml]

# Bump commands
bump2version patch   # 1.2.0 → 1.2.1
bump2version minor   # 1.2.0 → 1.3.0
bump2version major   # 1.2.0 → 2.0.0

Automated Release with GitHub Actions

# .github/workflows/publish.yml
name: Publish to PyPI

on:
  push:
    tags:
      - "v*"  # triggers on v1.2.0, v2.0.0, etc.

jobs:
  publish:
    runs-on: ubuntu-latest
    permissions:
      id-token: write  # for OIDC trusted publishing

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install build tools
        run: pip install build twine

      - name: Build
        run: python -m build

      - name: Check
        run: twine check dist/*

      # Trusted publishing (recommended) — no API tokens needed
      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        # No credentials needed with trusted publishing configured on PyPI

Frequently Asked Questions

Wheel vs sdist — which should I publish?
Always publish both. Wheels install faster (no build step) and are the default for pip. sdists are the fallback when no compatible wheel exists (e.g., unusual CPU architecture). For pure Python packages, publish a universal wheel (py3-none-any).
What is trusted publishing?
Trusted publishing (OIDC) lets GitHub Actions publish to PyPI without an API token. Configure it on PyPI under your project's Publishing settings, link your GitHub repo and workflow name. The GitHub Action gets a short-lived token automatically. No secrets to rotate.
How do I include non-Python files (data files) in my package?
Add them to your package directory and declare them in pyproject.toml. With Hatchling: [tool.hatch.build] include = ["src/mylib/data/*.json"]. Access them at runtime with importlib.resources.files("mylib") / "data/config.json".

Read Next

Python Pytest Guide

Write comprehensive tests before publishing your library to PyPI.

Python Type Hints Guide

Add type annotations to your library for better IDE support and user experience.

Python Articles