Commands Reference¶
Complete reference for Git DRS and related Git LFS commands.
Navigation: Getting Started → Commands Reference → Troubleshooting
Git DRS Commands¶
git drs init¶
Initialize Git DRS in a repository. Sets up Git LFS custom transfer hooks and configures .gitignore patterns.
Usage:
Options:
--transfers <n>: Number of concurrent transfers (default: 4)
Example:
What it does:
- Creates
.drs/directory structure - Configures Git LFS custom transfer agent
- Updates
.gitignoreto exclude DRS cache files - Stages
.gitignorechanges automatically
Note: Run this before adding remotes.
git drs remote¶
Manage DRS remote server configurations. Git DRS supports multiple remotes for working with development, staging, and production servers.
git drs remote add gen3 <name>¶
Add a Gen3 DRS server configuration.
Usage:
git drs remote add gen3 <remote-name> \
--url <server-url> \
--cred <credentials-file> \
--project <project-id> \
--bucket <bucket-name>
Options:
--url <url>: Gen3 server endpoint (required)--cred <file>: Path to credentials JSON file (required)--token <token>: Token for temporary access (alternative to --cred)--project <id>: Project ID in format<program>-<project>(required)--bucket <name>: S3 bucket name (required)
Examples:
# Add production remote
git drs remote add gen3 production \
--url https://calypr-public.ohsu.edu \
--cred /path/to/credentials.json \
--project my-project \
--bucket my-bucket
# Add staging remote
git drs remote add gen3 staging \
--url https://staging.calypr.ohsu.edu \
--cred /path/to/staging-credentials.json \
--project staging-project \
--bucket staging-bucket
Note: The first remote you add automatically becomes the default remote.
git drs remote add anvil <name>¶
Add an AnVIL/Terra DRS server configuration.
Note: AnVIL support is under active development. For production use, we recommend Gen3 workflows or version 0.2.2 for AnVIL functionality.
Usage:
Options:
--terraProject <id>: Terra/Google Cloud project ID (required)
Example:
git drs remote list¶
List all configured DRS remotes.
Usage:
Example Output:
* production gen3 https://calypr-public.ohsu.edu
staging gen3 https://staging.calypr.ohsu.edu
development gen3 https://dev.calypr.ohsu.edu
The * indicates the default remote used by all commands unless specified otherwise.
git drs remote set <name>¶
Set the default DRS remote for all operations.
Usage:
Examples:
# Switch to staging for testing
git drs remote set staging
# Switch back to production
git drs remote set production
# Verify change
git drs remote list
git drs fetch [remote-name]¶
Fetch DRS object metadata from remote server. Downloads metadata only, not actual files.
Usage:
# Fetch from default remote
git drs fetch
# Fetch from specific remote
git drs fetch staging
git drs fetch production
Note: fetch and push are commonly used together for cross-remote workflows. See git drs push below.
What it does:
- Identifies remote and project from configuration
- Transfers all DRS records for a given project from the server to the local
.drs/lfs/objects/directory
git drs push [remote-name]¶
Push local DRS objects to server. Uploads new files and registers metadata.
Usage:
# Push to default remote
git drs push
# Push to specific remote
git drs push staging
git drs push production
What it does:
- Checks local
.drs/lfs/objects/for DRS metadata - For each object, uploads file to bucket if file exists locally
- If file doesn't exist locally (metadata only), registers metadata without upload
- This enables cross-remote promotion workflows
Cross-Remote Promotion:
Transfer DRS records from one remote to another (eg staging to production) without re-uploading files:
# Fetch metadata from staging
git drs fetch staging
# Push metadata to production (no file upload since files don't exist locally)
git drs push production
This is useful when files are already in the production bucket with matching SHA256 hashes. It can also be used to reupload files given that the files are pulled to the repo first.
Note: fetch and push are commonly used together. fetch pulls metadata from one remote, push registers it to another.
git drs add-url¶
Add a file reference via S3 URL without copying the data.
Usage:
# Use default remote
git drs add-url s3://bucket/path/file --sha256 <hash>
# Use specific remote
git drs add-url s3://bucket/path/file --sha256 <hash> --remote staging
With AWS Credentials:
git drs add-url s3://bucket/path/file \
--sha256 <hash> \
--aws-access-key <key> \
--aws-secret-key <secret>
Options:
--sha256 <hash>: Required SHA256 hash of the file--remote <name>: Target remote (default: default_remote)--aws-access-key <key>: AWS access key--aws-secret-key <secret>: AWS secret key--endpoint <url>: Custom S3 endpoint--region <region>: AWS region
git drs create-cache¶
Create a cache from a manifest file (Terra/AnVIL).
git drs version¶
Display Git DRS version information.
Internal Commands¶
These commands are called automatically by Git hooks:
git drs precommit: Process staged files during commitgit drs transfer: Handle file transfers during push/pullgit drs transferref: Handle reference transfers (AnVIL/Terra)
Git LFS Commands¶
git lfs track¶
Manage file tracking patterns.
View Tracked Patterns:
Track New Pattern:
Untrack Pattern:
git lfs ls-files¶
List LFS-tracked files in the repository.
All Files:
Specific Pattern:
Output Format:
*prefix: File is localized (downloaded)-prefix: File is not localized- No prefix: File status unknown
git lfs pull¶
Download LFS-tracked files.
All Files:
Specific Files:
Multiple Patterns:
git lfs install¶
Configure Git LFS for the system or repository.
System-wide:
Repository-only:
The --skip-smudge option prevents automatic downloading of all LFS files during clone/checkout.
Standard Git Commands¶
Git DRS integrates with standard Git commands:
git add¶
Stage files for commit. LFS-tracked files are automatically processed.
git commit¶
Commit changes. Git DRS pre-commit hook runs automatically.
git push¶
Push commits to remote. Git DRS automatically uploads new files to DRS server.
git clone¶
Clone repository. Use with Git DRS initialization:
git clone <repo-url>
cd <repo-name>
git drs init
git drs remote add gen3 production --cred /path/to/credentials.json --url ... --project ... --bucket ...
Workflow Examples¶
Complete File Addition Workflow¶
# 1. Ensure file type is tracked
git lfs track "*.bam"
git add .gitattributes
# 2. Add your file
git add mydata.bam
# 3. Verify tracking
git lfs ls-files -I "mydata.bam"
# 4. Commit (creates DRS record)
git commit -m "Add analysis results"
# 5. Push (uploads to default DRS server)
git push
Selective File Download¶
# Check what's available
git lfs ls-files
# Download specific files
git lfs pull -I "results/*.txt"
git lfs pull -I "important-dataset.bam"
# Verify download
git lfs ls-files -I "results/*.txt"
Repository Setup from Scratch¶
# 1. Create and clone repo
git clone <new-repo-url>
cd <repo-name>
# 2. Initialize Git DRS
git drs init
# 3. Add DRS remote
git drs remote add gen3 production \
--url https://calypr-public.ohsu.edu \
--cred /path/to/credentials.json \
--project my-project \
--bucket my-bucket
# 4. Set up file tracking
git lfs track "*.bam"
git lfs track "*.vcf.gz"
git lfs track "data/**"
git add .gitattributes
git commit -m "Configure LFS tracking"
git push
# 5. Add data files
git add data/sample1.bam
git commit -m "Add sample data"
git push
Cross-Remote Promotion Workflow¶
# 1. Add multiple remotes
git drs remote add gen3 staging \
--url https://staging.calypr.ohsu.edu \
--cred /path/to/staging-credentials.json \
--project staging-project \
--bucket staging-bucket
git drs remote add gen3 production \
--url https://calypr-public.ohsu.edu \
--cred /path/to/prod-credentials.json \
--project prod-project \
--bucket prod-bucket
# 2. Fetch metadata from staging
git drs fetch staging
# 3. Push metadata to production (no re-upload)
git drs push production
Environment Variables¶
Git DRS respects these environment variables:
AWS_ACCESS_KEY_ID: AWS access key (for S3 operations)AWS_SECRET_ACCESS_KEY: AWS secret key (for S3 operations)GOOGLE_PROJECT: Google Cloud project ID (for AnVIL)WORKSPACE_BUCKET: Terra workspace bucket (for AnVIL)
Help and Documentation¶
Use --help with any command for detailed usage: