Provenance Records

Quick Start

# Install LineSpec
brew tap livecodelife/linespec
brew install linespec

# Or use go install
go install github.com/livecodelife/linespec/cmd/linespec@v1.3.0

# Create your first provenance record
linespec provenance create --title "Add user authentication"

# Validate all records
linespec provenance lint

# View the decision graph
linespec provenance graph

Core Concepts

What are Provenance Records?

Provenance Records are structured decision documents that capture:

• Intent - What we want to achieve and why
• Constraints - Rules that must be followed
• Scope - What files are affected (or explicitly forbidden)
• Relationships - How decisions connect (supersedes, related)
• Status - Where the decision is in its lifecycle

They provide a queryable history of architectural decisions that can be linted, graphed, and enforced at commit time.

ID Format

Records use the format: prov-YYYY-XXXXXXXX or prov-YYYY-XXXXXXXX-service-name

Where:

• YYYY is the four-digit year
• XXXXXXXX is 8 cryptographically random hex characters (lowercase)

The year prefix limits the collision probability per year. With 8 hex characters, there are 2^32 (4,294,967,296) possible combinations per year, making collisions extremely unlikely even with concurrent record creation.

Status Lifecycle

┌─────────┐    ┌─────────────┐    ┌────────────┐
│  open   │───▶│ implemented │───▶│ superseded │
└─────────┘    └─────────────┘    └────────────┘
                                     │
                                     ▼
                               ┌────────────┐
                               │ deprecated │
                               └────────────┘

• open - Decision is being discussed/developed
• implemented - Decision is complete and immutable
• superseded - Replaced by a newer record (use superseded_by)
• deprecated - No longer relevant

Schema Reference

Complete Example

id: prov-2026-001
title: "Protocol-level proxy interception for language-agnostic test evaluation"
status: implemented
created_at: "2026-03-12"
author: "caleb.cowen@gmail.com"

sealed_at_sha: "a3f92c1d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b"

intent: >
  LineSpec evaluates service behavior by intercepting traffic at the TCP/protocol
  layer rather than through application-level mocking...

constraints:
  - Interception must occur at the network/protocol layer
  - The system under test must not require modification
  - Proxy implementations must be protocol-correct

affected_scope:
  - pkg/proxy/**
  - pkg/registry/**
  - cmd/linespec/main.go

forbidden_scope:
  - "re:.*_test\\.go$"

supersedes: null
superseded_by: null
related:
  - prov-2026-002

associated_specs: []
associated_traces: []
monitors: []

tags:
  - architecture
  - proxy
  - core

Field Descriptions

Required Fields

Intent and Constraints

Scope

Graph Relationships

Proof of Completion

Metadata

CLI Commands

Create

Create a new provenance record:

# Interactive (opens editor)
linespec provenance create

# With pre-populated fields
linespec provenance create --title "Add caching layer" --tag architecture,performance

# For a specific service
linespec provenance create -i user-service --title "Add user auth"

# Skip editor
linespec provenance create --title "Quick fix" --no-edit

Options:

• --title "..." - Pre-populate title
• --supersedes prov-YYYY-XXXXXXXX - Link to older record
• --tag tag1,tag2 - Add tags
• --no-edit - Write without opening editor
• -i, --id-suffix name - Append service suffix
• -c, --config path - Use custom .linespec.yml

Lint

Validate provenance records:

# Lint all records
linespec provenance lint

# Lint specific record
linespec provenance lint --record prov-2026-001

# JSON output for CI
linespec provenance lint --format json

# Override enforcement level
linespec provenance lint --enforcement strict

Options:

• --record prov-YYYY-XXXXXXXX - Lint single record
• --enforcement level - none|warn|strict (default from config)
• --format format - human|json
• -c, --config path - Use custom config

Enforcement Levels:

• none - Don't enforce at all
• warn - Show warnings but allow (default)
• strict - Fail on any violation

Status

View record status:

# Overview of all records
linespec provenance status

# Detailed view of one record
linespec provenance status --record prov-2026-a1b2c3d4

# Filter by status
linespec provenance status --filter open
linespec provenance status --filter implemented

# Filter by tag
linespec provenance status --filter tag:architecture

# Save auto-populated scope
linespec provenance status --record prov-2026-a1b2c3d4 --save-scope

Graph

Render provenance graph:

# Full graph
linespec provenance graph

# Graph from specific record
linespec provenance graph --root prov-2026-a1b2c3d4

# Filter by status
linespec provenance graph --filter implemented

# JSON output
linespec provenance graph --format json

Options:

• --root prov-YYYY-XXXXXXXX - Start from specific record
• --filter status - Show only records with given status
• --format format - human|json|dot
• -c, --config path - Use custom config

Lock Scope

Lock scope to allowlist mode:

# Dry run first
linespec provenance lock-scope --record prov-2026-a1b2c3d4 --dry-run

# Lock scope (saves to file)
linespec provenance lock-scope --record prov-2026-a1b2c3d4

Options:

• --record prov-YYYY-XXXXXXXX - Required. The record to lock
• --dry-run - Print scope without writing
• -c, --config path - Use custom config

Deprecate

Mark record as deprecated:

linespec provenance deprecate --record prov-2026-a1b2c3d4 --reason "Replaced by new auth system"

Options:

• --record prov-YYYY-XXXXXXXX - Required. The record to deprecate
• --reason "..." - Deprecation reason
• -c, --config path - Use custom config

Search (Semantic)

Search provenance records by semantic similarity:

# Search with natural language query
linespec provenance search --query "authentication system"

# Limit results
linespec provenance search --query "database schema" --limit 10

Options:

• --query "text" - Required. Natural language search query
• --limit N - Maximum results to return (default: 5)
• -c, --config path - Use custom config

Requirements: Requires embedding configuration in .linespec.yml. Records must be indexed using linespec provenance index.

Audit (Semantic)

Audit recent changes against provenance history:

# Audit with description
linespec provenance audit --description "Added password validation"

# Audit with custom config
linespec provenance audit --description "Refactored user service" -c /path/to/.linespec.yml

Options:

• --description "text" - Required. Description of changes to audit
• -c, --config path - Use custom config

Output: Shows records with semantic similarity to your changes. Advisory only - always exits 0. Helps identify potential conflicts with prior decisions.

Index

Index provenance records for semantic search:

# Index all unindexed implemented records
linespec provenance index

# Dry run - show what would be indexed
linespec provenance index --dry-run

# Re-index all records (even if already indexed)
linespec provenance index --force

Options:

• --dry-run - Show what would be indexed without making API calls
• --force - Re-index even if embedding already exists
• -c, --config path - Use custom config

When to use: After enabling embedding configuration for the first time, to backfill historical records, or after changing embedding models.

Check

Check commits for violations:

# Check current HEAD
linespec provenance check

# Check specific commit
linespec provenance check --commit abc123

# Check commit range
linespec provenance check --range HEAD~5..HEAD

# Check against specific record only
linespec provenance check --record prov-2026-a1b2c3d4

# Check staged files (used by commit-msg hook)
linespec provenance check --staged --message-file .git/COMMIT_EDITMSG

Complete

Mark record as implemented:

# Normal completion
linespec provenance complete --record prov-2026-a1b2c3d4

# Force complete (skip check)
linespec provenance complete --record prov-2026-a1b2c3d4 --force

Options:

• --record prov-YYYY-XXXXXXXX - Required. The record to mark as implemented
• --force - Skip LineSpec existence check
• -c, --config path - Use custom config

Install Hooks

Install git hooks for automatic validation:

linespec provenance install-hooks

This creates both pre-commit and commit-msg hooks.

Configuration

Create a .linespec.yml file in your repository root:

provenance:
  # Directory containing provenance records (default: provenance)
  dir: provenance
  
  # Enforcement level: none|warn|strict (default: warn)
  enforcement: warn
  
  # Require provenance IDs in commit messages (default: false)
  commit_tag_required: false
  
  # Auto-populate affected_scope from git commits (default: true)
  auto_affected_scope: true
  
  # Additional directories to load records from (for monorepos)
  shared_repos:
    - examples/user-service/provenance
    - examples/todo-api/provenance

Semantic Search Configuration

Enable semantic search over provenance records using your choice of embedding provider:

Voyage AI (default)

provenance:
  embedding:
    provider: voyage                 # Embedding provider
    index_model: voyage-4-large     # Model for indexing (2048 dims)
    query_model: voyage-4-lite      # Model for queries (2048 dims)
    api_key: ${VOYAGE_API_KEY}      # API key from environment
    similarity_threshold: 0.50        # Minimum similarity (0.50-0.70 range)
    index_on_complete: true         # Auto-index on complete

OpenAI

provenance:
  embedding:
    provider: openai                 # Embedding provider
    index_model: text-embedding-3-small   # Model for indexing
    query_model: text-embedding-3-small   # Model for queries
    api_key: ${OPENAI_API_KEY}      # API key from environment
    similarity_threshold: 0.50        # Minimum similarity
    index_on_complete: true         # Auto-index on complete

Configuration Options

Git Integration

Two-Hook Strategy

The provenance system uses two git hooks that work together:

1. pre-commit hook: Runs first, lints modified provenance records for syntax/validity
2. commit-msg hook: Runs after you write your message, checks staged files against provenance scope

Commit Message Format

Reference provenance records in commit messages:

# Single record
git commit -m "Add user authentication [prov-2026-a1b2c3d4]"

# Multiple records
git commit -m "Fix auth and caching [prov-2026-a1b2c3d4] [prov-2026-deadbeef]"

# Service-specific
git commit -m "Update user service [prov-2026-a1b2c3d4-user-service]"

Implemented Record Enforcement

Once a provenance record is marked as implemented, it becomes immutable. The commit-msg hook will reject any commits tagged with an implemented record ID:

# This will FAIL - prov-2026-a1b2c3d4 is already implemented
git commit -m "Fix typo [prov-2026-a1b2c3d4]"
# Error: prov-2026-a1b2c3d4 is already implemented - cannot commit with this ID. 
#        Create a new record or supersede this one.

# Instead, create a new record or supersede:
linespec provenance create --title "Fix typo in auth" --supersedes prov-2026-a1b2c3d4
git commit -m "Fix typo [prov-2026-deadbeef]"

The only exception is the completion transition (when a record's own file changes from status: open to status: implemented), which is allowed.

Automated Embedding Indexing

When semantic search is enabled, a GitHub Action workflow automatically indexes newly completed provenance records when they are merged to the main branch. This ensures the embedding store stays up-to-date without manual intervention.

How it works:

1. When a record is marked implemented via linespec provenance complete, the embedding is generated locally (if index_on_complete: true)
2. When the record is merged to main, the GitHub Action runs and indexes any unindexed records
3. The action respects Voyage AI rate limits and handles batching automatically

provenance:
  embedding:
    provider: voyage
    index_model: voyage-4-large
    query_model: voyage-4-lite
    api_key: ${VOYAGE_API_KEY}
    similarity_threshold: 0.50
    index_on_complete: false  # Skip local embedding, let GitHub Action handle it

Required Secret: Add VOYAGE_API_KEY as a repository secret in GitHub (Settings → Secrets and variables → Actions).

Sealed at SHA and Stale Scope Warnings

What is sealed_at_sha?

When a provenance record is marked as implemented, the CLI automatically captures the current HEAD git SHA and stores it in the sealed_at_sha field. This field is:

• Immutable - Set once by the CLI, never modified after
• CLI-only - Never set manually
• Timestamp - Captures the exact moment a decision was "locked in"
• Only on implemented records - Open/superseded/deprecated records don't have this field

Example:

id: prov-2026-001
title: "Add user authentication"
status: implemented
created_at: "2026-03-12"
author: "user@example.com"

sealed_at_sha: "a3f92c1d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b"
#          ↑ Captured when `linespec provenance complete` was run

Stale Scope Warnings

The sealed_at_sha field enables stale scope warnings. When checking commits against provenance records:

1. If a commit touches files in a record's affected_scope
2. The CLI runs git diff <sealed_at_sha> HEAD on those files
3. If files haven't actually changed since the record was sealed, a warning is shown

Example warning:

⚠ Stale scope warnings in staged (non-blocking):

  • prov-2026-001 lists pkg/utils/helpers.go in affected_scope, but 
    file unchanged since record sealed at a3f92c1
    (File listed in affected_scope but unchanged since record sealed)

Key characteristics:

• Non-blocking - These are warnings, not errors
• Informational - Helps engineers make informed decisions
• Configurable - Controlled by the enforcement setting

Viewing the Sealed SHA

The sealed SHA is displayed in the status output for implemented records:

linespec provenance status --record prov-2026-001

Output:

prov-2026-001  ·  implemented
────────────────────────────────────────────────────────────

Title:        Add user authentication
Author:       user@example.com
Created:      2026-03-12
Sealed at:    a3f92c1        ← Shows the short SHA

Best Practices

Writing Good Provenance Records

• Start with Intent - Clearly state what you want to achieve and why
• Be Specific with Constraints - Write verifiable rules
• Use Appropriate Scope - Start in observed mode (empty affected_scope), then lock to allowlist
• Link Related Decisions - Use supersedes, superseded_by, and related
• Tag Thoughtfully - Use consistent tags for filtering

Scope Management

Observed Mode (empty affected_scope):

• Allows any file changes
• Good for early development
• Auto-populated from git history

Allowlist Mode (non-empty affected_scope):

• Only allows changes to listed files
• Good for mature decisions
• Prevents scope creep

Transition:

# After some commits, lock the scope
linespec provenance lock-scope --record prov-2026-001

Monorepo Strategy

For multiple services in one repo:

1. Root provenance/ - Shared architectural decisions
2. Service directories - Service-specific decisions
3. Use ID suffixes - prov-2026-a1b2c3d4-user-service

# .linespec.yml
provenance:
  dir: provenance
  shared_repos:
    - services/user-service/provenance
    - services/todo-api/provenance

Record Lifecycle

# 1. Create (status: open)
linespec provenance create --title "New feature"

# 2. Develop (make commits, scope auto-populates)
git commit -m "Implement feature [prov-2026-deadbeef]"

# 3. Lock scope (when feature is complete)
linespec provenance lock-scope --record prov-2026-deadbeef

# 4. Complete (status: implemented)
linespec provenance complete --record prov-2026-deadbeef

# 5. (Optional) Supersede later
linespec provenance create --title "Better approach" --supersedes prov-2026-deadbeef

Examples

Example 1: Simple Architecture Decision

id: prov-2026-a1b2c3d4
title: "Use PostgreSQL for primary data store"
status: implemented
created_at: "2026-03-15"
author: "dev@example.com"

intent: >
  After evaluating MySQL, PostgreSQL, and SQLite, we choose PostgreSQL
  for our primary data store. It provides better JSON support, more
  advanced indexing, and better handling of concurrent writes.

constraints:
  - All new tables must use PostgreSQL
  - Existing MySQL tables will be migrated gradually
  - Use connection pooling with minimum 10, maximum 100 connections

affected_scope:
  - pkg/db/**
  - migrations/**
  - config/database.yml

forbidden_scope:
  - "re:.*_test\\.go$"
  - vendor/**

tags:
  - architecture
  - database
  - postgresql

Example 2: Service-Specific Decision

id: prov-2026-deadbeef-user-service
title: "Implement JWT-based authentication"
status: open
created_at: "2026-03-16"
author: "auth-team@example.com"

intent: >
  The user service will implement JWT-based authentication to support
  stateless API access and microservice communication.

constraints:
  - JWT tokens must expire after 24 hours
  - Refresh tokens must expire after 30 days
  - Use RS256 algorithm with 2048-bit keys

affected_scope:
  - services/user-service/pkg/auth/**
  - services/user-service/handlers/auth.go

associated_specs:
  - path: services/user-service/specs/auth/login_success.linespec
    type: linespec
  - path: services/user-service/specs/auth/login_failure.linespec
    type: linespec

tags:
  - user-service
  - authentication
  - jwt

Example 3: Superseding an Old Decision

id: prov-2026-01234567
title: "Replace Redis caching with in-memory LRU"
status: implemented
created_at: "2026-03-17"
author: "perf-team@example.com"

intent: >
  After load testing, we found Redis adds unnecessary latency for our
  use case. An in-memory LRU cache provides better performance with
  simpler operations.

constraints:
  - Maximum cache size: 10,000 entries
  - Eviction policy: LRU
  - TTL: 5 minutes maximum

affected_scope:
  - pkg/cache/**
  - cmd/api/main.go

supersedes: prov-2026-a1b2c3d4
tags:
  - performance
  - caching
  - lru

Troubleshooting

Common Issues

• "Record not found" - Check that the record file exists in the provenance directory
• "File outside scope" - The file you're modifying isn't in affected_scope
• "Cycle detected in graph" - You have circular supersedes relationships
• Linter is slow - Reduce the number of shared_repos

Getting Help

• Open an issue: https://github.com/livecodelife/linespec/issues
• Check existing provenance records in the examples/
• Review the release plan: RELEASE_PLAN.md