Python GitHub Actions: CI/CD Pipeline for Python Apps

GitHub Actions is the most popular CI/CD platform for Python projects. With YAML workflow files committed alongside your code, every push and pull request can automatically run tests, lint code, build Docker images, and deploy to production. This guide covers a production-grade CI/CD setup: matrix testing across Python versions, dependency caching, code coverage, Docker builds, PyPI publishing with trusted publishing, and zero-downtime deployment.

Basic CI: Test on Every Push

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: testpass
          POSTGRES_DB: testdb
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432

      redis:
        image: redis:7
        options: --health-cmd "redis-cli ping"
        ports:
          - 6379:6379

    env:
      DATABASE_URL: postgresql://postgres:testpass@localhost:5432/testdb
      REDIS_URL: redis://localhost:6379/0
      SECRET_KEY: test-secret-key-not-for-prod

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: |
          pip install --upgrade pip
          pip install -e ".[dev,test]"

      - name: Run migrations
        run: alembic upgrade head

      - name: Run tests with coverage
        run: |
          pytest tests/ \
            --cov=myapp \
            --cov-report=xml \
            --cov-report=term-missing \
            --cov-fail-under=80 \
            -v

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          file: ./coverage.xml

Matrix Testing Across Python Versions

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false  # don't cancel others if one fails
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
        os: [ubuntu-latest, windows-latest, macos-latest]
        exclude:
          # Skip Windows + Python 3.13 (not yet stable)
          - os: windows-latest
            python-version: "3.13"

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Run tests
        run: pytest tests/ -x --tb=short

Dependency Caching

steps:
  - uses: actions/checkout@v4

  - uses: actions/setup-python@v5
    with:
      python-version: "3.12"
      cache: "pip"          # built-in pip cache (simple projects)

  # OR: manual cache for Poetry
  - name: Cache Poetry virtualenv
    uses: actions/cache@v4
    with:
      path: .venv
      key: venv-${{ runner.os }}-${{ hashFiles('poetry.lock') }}
      restore-keys: |
        venv-${{ runner.os }}-

  - name: Install with Poetry
    run: |
      pip install poetry
      poetry config virtualenvs.in-project true
      poetry install --no-root

  # Cache pre-commit hooks
  - name: Cache pre-commit
    uses: actions/cache@v4
    with:
      path: ~/.cache/pre-commit
      key: precommit-${{ hashFiles('.pre-commit-config.yaml') }}

  - name: Run pre-commit
    run: pre-commit run --all-files

Linting and Code Quality

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"

      - name: Install tools
        run: pip install ruff mypy

      - name: Ruff lint
        run: ruff check --output-format=github .

      - name: Ruff format check
        run: ruff format --check .

      - name: Type check with mypy
        run: mypy myapp/ --ignore-missing-imports --strict

      - name: Check for security issues
        run: |
          pip install bandit safety
          bandit -r myapp/ -ll
          safety check --full-report

Docker Build and Push

jobs:
  docker:
    runs-on: ubuntu-latest
    needs: [test, quality]  # only build if tests pass
    if: github.ref == 'refs/heads/main'

    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      # OR: login to GitHub Container Registry
      - name: Login to GHCR
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}  # auto-provided

      - name: Docker metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: |
            myorg/myapp
            ghcr.io/${{ github.repository }}
          tags: |
            type=ref,event=branch
            type=semver,pattern={{version}}
            type=sha,prefix=sha-

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha   # GitHub Actions cache
          cache-to: type=gha,mode=max
          platforms: linux/amd64,linux/arm64

PyPI Publishing

# .github/workflows/publish.yml
name: Publish to PyPI

on:
  push:
    tags:
      - "v*"

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install build twine
      - run: python -m build
      - run: twine check dist/*
      - uses: actions/upload-artifact@v4
        with:
          name: dist
          path: dist/

  publish:
    needs: build
    runs-on: ubuntu-latest
    environment: pypi        # requires manual approval
    permissions:
      id-token: write        # required for trusted publishing

    steps:
      - uses: actions/download-artifact@v4
        with:
          name: dist
          path: dist/

      # Trusted publishing — no API token needed
      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1

Deployment to AWS ECS

  deploy:
    needs: docker
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    environment: production

    steps:
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Download task definition
        run: |
          aws ecs describe-task-definition --task-definition myapp \
            --query taskDefinition > task-definition.json

      - name: Update ECS task definition image
        id: task-def
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: task-definition.json
          container-name: myapp
          image: ${{ steps.meta.outputs.tags }}

      - name: Deploy to ECS
        uses: aws-actions/amazon-ecs-deploy-task-definition@v1
        with:
          task-definition: ${{ steps.task-def.outputs.task-definition }}
          service: myapp-service
          cluster: production
          wait-for-service-stability: true  # wait for old tasks to stop

Managing Secrets

# Repository secrets: Settings → Secrets → Actions
# Reference in workflow:
env:
  DATABASE_URL: ${{ secrets.DATABASE_URL }}
  API_KEY: ${{ secrets.API_KEY }}

# Environment secrets (require approval gate)
jobs:
  deploy:
    environment: production  # uses secrets from 'production' environment
    steps:
      - run: deploy.sh
        env:
          DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}

# GITHUB_TOKEN — auto-generated, no setup needed
- name: Create GitHub Release
  uses: softprops/action-gh-release@v2
  with:
    files: dist/*.whl
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

# Dynamic secrets from AWS Secrets Manager
- name: Get secrets from AWS
  uses: aws-actions/aws-secretsmanager-get-secrets@v2
  with:
    secret-ids: |
      prod/myapp/database
      prod/myapp/api-keys
    parse-json-secrets: true

Frequently Asked Questions

How do I run only changed tests (test selection)?

Use pytest-testmon which tracks which tests are affected by code changes and only runs those. Install it and run pytest --testmon. On CI, store the .testmondata file in the Actions cache for persistence between runs.

How do I prevent deploys if tests fail?

Use needs: [test, quality] in your deploy job. GitHub Actions only runs a job if all its needs jobs succeed. Combine with branch protection rules to require the CI workflow to pass before merging pull requests.

How do I speed up slow GitHub Actions?

Cache everything: pip packages, Poetry virtualenvs, pre-commit hooks, Docker layers (use cache-from: type=gha). Split slow test suites with pytest-split across parallel jobs. Use larger runners (4-core) for CPU-intensive builds. Avoid running duplicate jobs — use workflow concurrency groups to cancel stale runs.