Skip to content

v0.2 Plan

v0.2 makes PyBetterleaks feel like a Python SDK instead of only a native loader. The release theme:

Programmatic config, async ergonomics, measured performance, and stronger release confidence without weakening the no-subprocess promise.

Release Goals

  • Keep v0.1 APIs stable: scan_text, scan_dir, and betterleaks_version.
  • Add typed Python config models that serialize to Betterleaks-compatible TOML.
  • Model upstream Betterleaks config concepts directly: prefilter, filter, validate, [extend], [[rules]], and [[rules.required]].
  • Add async wrappers with cooperative native cancellation.
  • Bridge validation env vars so Betterleaks Expr validators can read explicit allowlisted values from Python.
  • Add reproducible synthetic benchmarks.
  • Add release artifact checksums.
  • Improve wheel smoke tests to exercise typed config and async.
  • Keep musllinux/Alpine documented as unsupported.

Non-Goals

  • Do not replace Betterleaks config semantics with a separate Python rules engine.
  • Do not add install-time binary downloads.
  • Do not require Pydantic or other runtime Python dependencies.
  • Do not use runtime subprocesses to call the Betterleaks CLI.
  • Do not claim musllinux support without a clean loader proof.

Public API

from pybetterleaks import BetterleaksConfig, Expr, Rule, scan_text

config = BetterleaksConfig.with_defaults(
    rules=[
        Rule(
            id="internal-token",
            description="Internal service token",
            regex=r"INTERNAL_[A-Z0-9]{16}",
            keywords=["INTERNAL_"],
            filter=Expr('filter.containsAny(finding["secret"], ["_TEST_"])'),
        )
    ],
    disabled_rules=["generic-api-key"],
)

result = scan_text("INTERNAL_0123456789ABCDEF", config=config)

Scanner signatures:

def scan_text(
    text: str,
    *,
    config: BetterleaksConfig | None = None,
    config_path: str | PathLike[str] | None = None,
    validation: bool = False,
    validation_env_vars: Sequence[str] | None = None,
    redact: bool = True,
    timeout_seconds: float | None = None,
) -> ScanResult: ...

async def scan_text_async(...) -> ScanResult: ...
async def scan_dir_async(...) -> ScanResult: ...

Rules:

  • Passing both config and config_path raises ValueError.
  • Passing neither keeps v0.1 behavior.
  • config serializes to inline TOML and the bridge receives config_toml.
  • config_path remains available for user-owned Betterleaks TOML files.
  • Async cancellation calls BetterleaksCancel(request_id).

Implementation Milestones

1. Config Model And Serialization

  • Add python/pybetterleaks/config.py.
  • Export config types from pybetterleaks.__init__.
  • Use dataclasses and a narrow stdlib-only TOML serializer.
  • Add unit tests for:
  • modern Betterleaks TOML serialization
  • default config extension
  • duplicate rule rejection
  • inline config TOML request serialization
  • config and config_path exclusivity

2. Scanner Integration

  • Update request construction to accept config.
  • Serialize config into inline TOML and pass it through the bridge request as config_toml.
  • Add validation_env_vars to scan calls.
  • Add native smoke tests using a Python config object.

3. Async And Cancellation

  • Add scan_text_async and scan_dir_async.
  • Generate a request id per async scan.
  • Register request ids in the Go bridge with context.CancelFunc.
  • Add exported BetterleaksCancel.
  • Treat cancellation as best-effort because scans may finish before the cancel request arrives.

4. Validation Env Bridge

  • Copy only explicitly allowlisted env var values from Python into the request.
  • Temporarily mirror those values into Go process env during the scan.
  • Lock validation-env scans to prevent cross-scan env leakage.
  • Restore previous Go env values after the scan.

5. Benchmarks

  • Add benchmarks/ with synthetic fixtures.
  • Compare:
  • pybetterleaks.scan_text
  • pybetterleaks.scan_dir
  • Betterleaks CLI subprocess baseline, when --cli is requested
  • Keep README numbers empty until measurements are collected on release hardware.

6. Release Confidence

  • Generate SHA256 checksums for release artifacts.
  • Write checksums outside dist so PyPI publish sees only distributions.
  • Exercise typed config and async in scripts/wheel_smoke.py.
  • Keep Docker runtime E2E on a no-Go Python image.
  • Document musllinux/Alpine as unsupported until the loader path is fixed.

Musllinux Findings

Alpine currently fails when Python loads the Go shared library through ctypes:

initial-exec TLS resolves to dynamic definition

The same error reproduces with:

  • Go -buildmode=c-shared
  • Go -buildmode=c-archive linked into a musl shared object

Therefore v0.2 should keep musllinux visible as a blocker, but must not publish musllinux wheels until a clean loader path exists.

Test Plan

  • uv run pytest
  • uv run ruff check .
  • uv run mypy python
  • GOCACHE=/private/tmp/go-cache-pybetterleaks go test ./... from bridge/
  • GOCACHE=/private/tmp/go-cache-pybetterleaks go vet ./... from bridge/
  • uv run --group docs mkdocs build --strict
  • uv run python benchmarks/bench.py --rounds 1 --warmups 0
  • uv build --wheel
  • uv run python scripts/wheel_smoke.py
  • bash e2e/run.sh

Acceptance Criteria

v0.2 is ready when:

  • BetterleaksConfig can define at least one custom regex rule and scan with it through scan_text and scan_dir.
  • Existing v0.1 usage keeps working without changes.
  • Type checkers see the new public config and async symbols.
  • Docs include copy-pasteable config and async examples.
  • Benchmarks exist and README claims are based on measured output.
  • Release checksums are generated without contaminating PyPI upload inputs.
  • Normal wheels and Docker E2E pass.
  • Alpine/musllinux is explicitly documented as unsupported.