Git for AI

Use version control to track AI code, prompts, configs, and deployment changes.

AI projects contain more than model code. They include notebooks, feature logic, prompts, evaluation suites, configurations, API services, infrastructure, documentation, and references to data and model artifacts. Git provides a reviewable history for the text-based parts of that system and connects changes to MLOps pipelines.

What Is Git?

Git is a distributed version-control system. It records snapshots of a repository, lets developers create parallel branches, compares changes, supports collaborative review, and preserves history that can be inspected or restored.

Instead of folders named Final, Final-New, and Final-Latest, Git identifies each committed state with a unique hash and records its authorship, timestamp, parent history, and message.

Why Git Matters in AI and MLOps

  • Tracks training, evaluation, inference, API, and data-processing code.
  • Versions prompts, schemas, tests, deployment manifests, and configuration templates.
  • Enables peer review and collaboration without overwriting another person's work.
  • Triggers continuous integration, model-training, packaging, and deployment pipelines.
  • Connects experiments and model artifacts to the code revision that produced them.
  • Supports investigation, rollback, audit history, and reproducible releases.
  • Makes documentation and operational runbooks evolve beside the implementation.

What Belongs in Git?

Usually TrackUsually Store Elsewhere
Source code and testsLarge raw or processed datasets
Prompt templates and output schemasTrained model weights and checkpoints
Small configuration templatesSecrets, tokens, passwords, and private keys
Environment and dependency definitionsLarge generated reports and experiment artifacts
Infrastructure and deployment filesCaches, logs, virtual environments, and build output
Documentation and runbooksPersonal, confidential, licensed, or restricted data
Dataset and model metadata referencesBinary artifacts that change frequently

Store large datasets and models in versioned object storage, artifact repositories, dataset-versioning systems, or model registries. Commit a stable identifier, checksum, manifest, or URI so code can locate the exact approved artifact.

The Basic Git Model

AreaMeaning
Working treeFiles currently checked out and being edited
Staging areaSelected changes prepared for the next commit
Local repositoryCommit history stored on the developer's machine
Remote repositoryShared repository used for collaboration and automation
BranchA movable name pointing to a line of commits
TagA named reference commonly used to mark a release

Basic Workflow

Terminal
git clone <repository-url>
cd <repository-directory>
git switch -c feature/add-drift-check

# Edit files and run tests
git status
git diff
git add src/drift.py tests/test_drift.py
git diff --staged
git commit -m "Add input drift validation"
git push -u origin feature/add-drift-check

A pull or merge request then lets teammates review the change and automated checks before it enters the protected main branch.

Common Commands

CommandPurpose
git initCreate a repository in the current directory
git clone URLCopy an existing repository and its history
git statusShow branch, staged, modified, and untracked files
git diffInspect unstaged changes
git add FILEStage selected changes
git commit -m MESSAGECreate a commit from staged changes
git logInspect commit history
git switch BRANCHChange branches
git fetchDownload remote references without merging
git pullFetch and integrate remote changes
git pushPublish local commits to a remote
git restore FILERestore working-tree content carefully

Review Before You Commit

Terminal
git status
git diff
git add path/to/intended-file.py
git diff --staged
git commit -m "Validate classification output schema"

Stage specific files or hunks rather than automatically adding everything. This reduces accidental inclusion of secrets, large artifacts, temporary files, or unrelated changes.

Good Commits

  • Represent one coherent change that can be understood and reviewed.
  • Include relevant tests, documentation, and configuration updates.
  • Avoid mixing formatting, refactoring, dependency changes, and new behavior without reason.
  • Use an imperative summary that states the result, such as Add model health endpoint.
  • Explain motivation, trade-offs, migration, or risk in the commit body when useful.
  • Do not commit generated outputs merely to make the working tree look clean.
Weak MessageStronger Message
updatesAdd schema validation for prediction requests
fix bugHandle empty feature vectors in batch inference
model changesSwitch fraud threshold to calibrated validation value
configSet canary traffic to five percent
final versionRelease inference API v1.4.0

Branches and Collaboration

Branches isolate work until it is ready to integrate. Keep branches short-lived and focused, synchronize regularly, and avoid allowing long-lived branches to diverge from the main line for weeks.

  • Protect the main branch from direct, unreviewed changes.
  • Require passing checks and appropriate reviewers before merge.
  • Use descriptive branch names such as feature/batch-inference or fix/token-redaction.
  • Delete merged branches to reduce clutter.
  • Choose merge, squash, or rebase conventions consistently across the team.
  • Do not rewrite history other people may already depend on without coordination.

Pull Requests and Code Review

Pull Request SectionWhat to Include
ProblemWhy the change is necessary
SolutionWhat changed and important design choices
EvidenceTests, evaluation results, screenshots, or benchmark links
AI impactModel, prompt, data, metric, and behavior changes
Operational impactDeployment, migration, monitoring, cost, and rollback
RiskSecurity, privacy, fairness, compatibility, and known limitations
Review guidanceFiles or decisions requiring special attention

Review AI changes for more than syntax. Ask whether the evaluation is valid, data use is authorized, leakage is prevented, metrics reflect the decision, model artifacts are traceable, and production safeguards are sufficient.

Handling Merge Conflicts

A conflict occurs when Git cannot combine changes automatically. Read both versions, understand the intended final behavior, edit the file, run tests and evaluations, then mark the resolution. Do not choose one side blindly in model configuration, prompt, dependency, or data-schema conflicts.

Terminal
git status
# Edit conflicted files and remove conflict markers
git add path/to/resolved-file
# Complete the merge or rebase according to the team's workflow
# Run tests and AI evaluations before pushing

.gitignore for AI Projects

GITIGNORE
# Python
__pycache__/
*.py[cod]
.venv/

# Local configuration and secrets
.env
.env.*
!.env.example
*.pem

# Notebooks
.ipynb_checkpoints/

# Data and model artifacts
data/raw/
data/processed/
artifacts/
checkpoints/
*.pkl
*.joblib
*.onnx
*.pt

# Logs and local tools
logs/
.cache/
.DS_Store

Adapt ignore rules to the repository. A file already committed remains tracked after it is added to .gitignore; remove it from tracking deliberately and verify no required shared artifact is lost.

Secrets Must Never Enter Git

  • Use environment variables only as runtime inputs, not as a substitute for secret storage.
  • Use an approved secret manager or CI/CD encrypted-secret facility.
  • Commit a documented .env.example containing names and safe placeholders only.
  • Run secret scanning before commits and in continuous integration.
  • Limit repository and pipeline permissions using least privilege.
  • If a secret is committed, rotate or revoke it immediately; deleting the latest file does not remove it from history.

Notebooks in Git

Jupyter notebooks are JSON documents containing code, text, metadata, and possibly large outputs. They can be difficult to review and merge. Clear unnecessary output, keep cells ordered and reproducible, avoid hidden state, and move reusable logic into tested Python modules.

  • Use notebooks for exploration and communication, not as the only production implementation.
  • Restart and run all cells before review to expose order dependencies.
  • Avoid embedding secrets or private data in cells and outputs.
  • Use notebook-aware diff or cleaning tools when adopted by the team.
  • Export critical figures or reports through reproducible scripts rather than manual editing.

Large Data and Model Files

Git stores file history and performs poorly when large binaries change repeatedly. Use an artifact or object store, model registry, dataset-versioning tool, or Git Large File Storage where appropriate. The repository should retain a stable reference and integrity information.

YAML
dataset:
  name: support_tickets
  version: 2026-07-20
  uri: s3://example-ml-data/support/2026-07-20/
  manifest_sha256: <approved-checksum>
model:
  registry_name: ticket_classifier
  version: 17

Repository Structure

Output
ai-project/
 README.md
 pyproject.toml
 .gitignore
 .env.example
 src/
    data/
    features/
    training/
    serving/
 tests/
 notebooks/
 prompts/
 evals/
 configs/
 deploy/
 docs/
 scripts/

The exact structure depends on the project. The important goal is to separate reusable code, experiments, tests, configuration, deployment, and documentation so responsibilities remain clear.

Configuration Management

  • Commit safe configuration defaults and schemas.
  • Inject environment-specific values during deployment.
  • Keep secrets outside the repository.
  • Version feature definitions, thresholds, prompt templates, and model-selection rules.
  • Validate configuration during CI and application startup.
  • Record the resolved production configuration or its immutable reference for traceability.

Dependency and Environment Reproducibility

Commit dependency declarations and lock files appropriate to the ecosystem, plus container or environment definitions when used. Review dependency updates, scan for vulnerabilities, and avoid unbounded versions that can change a build unexpectedly.

Git and Experiment Tracking

An experiment tracker should record the Git commit hash together with dataset version, configuration, environment, metrics, and artifacts. This connects a result to the source that produced it without storing large artifacts in Git.

Python
import subprocess

def current_git_commit() -> str:
    return subprocess.check_output(
        ["git", "rev-parse", "HEAD"], text=True
    ).strip()

run_metadata = {
    "git_commit": current_git_commit(),
    "dataset_version": "support-2026-07-20",
    "config": "configs/ticket_classifier.yaml",
}

Training on an uncommitted or dirty working tree reduces reproducibility. Record that state or require reviewed commits for official candidate models.

Prompts and Evaluations in Git

  • Store prompt templates as dedicated files rather than scattered string literals.
  • Version system instructions, examples, tool definitions, and output schemas together.
  • Keep evaluation inputs, expected behavior, rubrics, and scoring code under review.
  • Use synthetic or properly sanitized cases instead of committing sensitive production conversations.
  • Require evaluation results when a pull request changes AI behavior.
  • Tag releases so production outputs can be traced to a prompt and evaluation version.

Git in CI/CD and MLOps

Output
Pull request opened
-> Lint and unit tests
-> Data-contract and configuration checks
-> Security and secret scans
-> AI evaluation suite
-> Peer approval
-> Merge protected branch
-> Build immutable package or container
-> Train or register model candidate
-> Deploy to staging
-> Validate and approve
-> Progressive production release

The commit should identify the source of a build. Promote the same immutable artifact through environments instead of rebuilding different binaries from the same branch name.

Tags and Releases

Tags mark significant repository states such as application releases, model-serving changes, or prompt versions. A release should include notes about behavior, compatibility, migration, evaluation evidence, deployment, monitoring, and rollback.

Terminal
git tag -a v1.4.0 -m "Release inference API v1.4.0"
git push origin v1.4.0

Reverting and Rolling Back

A Git revert creates a new commit that reverses an earlier change while preserving shared history. A production rollback may also require restoring a model, prompt, feature schema, container, infrastructure configuration, and data migration. Git rollback alone is not automatically an operational rollback.

Repository Security

  • Require strong authentication and least-privilege repository roles.
  • Protect important branches and require reviews and status checks.
  • Use signed commits or tags when provenance requirements justify them.
  • Review third-party actions, hooks, dependencies, and CI permissions.
  • Pin or verify external automation dependencies where possible.
  • Prevent untrusted pull-request code from accessing production secrets.
  • Audit sensitive changes to deployment, permissions, data, and model-release configuration.

Common Mistakes

MistakeBetter Practice
git add . without reviewInspect status and staged diff; add intended changes
Commit secrets, data, or model binariesUse secret and artifact stores with safe references
Huge mixed commitsCreate small coherent commits with tests
Vague messagesDescribe the behavior or result changed
Long-lived branchesIntegrate small reviewed changes frequently
Notebook-only production logicMove reusable behavior into tested modules
Force-pushing shared historyCoordinate and preserve public history
Assuming Git equals full reproducibilityVersion data, artifacts, configuration, and environment too

Practical Exercise

Terminal
mkdir ai-git-practice
cd ai-git-practice
git init

# Create README.md, .gitignore, src/train.py, and tests/test_train.py
git status
git add README.md .gitignore src/train.py tests/test_train.py
git diff --staged
git commit -m "Create reproducible training project structure"

git switch -c feature/add-evaluation
# Add evaluation code and tests
git add src/evaluate.py tests/test_evaluate.py
git commit -m "Add classification evaluation metrics"
git log --oneline --decorate --graph --all

Extend the exercise with a remote repository, pull request, automated tests, a safe dataset manifest, and experiment metadata that records the commit hash.

Best Practices

  • Track text-based source, tests, prompts, schemas, configuration templates, deployment files, and documentation.
  • Store secrets, sensitive data, large datasets, and models in systems designed for them.
  • Review diffs and create small, meaningful commits.
  • Use short-lived branches, pull requests, protected main branches, and automated checks.
  • Connect every experiment, model, build, and deployment to an immutable commit and artifact version.
  • Keep notebooks reproducible and move production logic into tested modules.
  • Treat repository and CI/CD security as part of the ML supply chain.
  • Document releases and remember that model and data rollback extends beyond Git.