Gitleaks

What's new

• pre-commit scanning (great for pre-commit hooks!)
• whitelists per rule
• faster regex scanning
• more useful debug logging so you can check which regexes are performing poorly
• username/password authentication option
• access-token authentication as an option
• proper unit tests that don't rely on connecting to a git host (no more travis failing for pr's)

pre-commit scanning

This is a feature that has been requested for a while. I finally got around to it and I think it's one of the most useful features going forward as this shifts the secret detection as close as possible to the developer... if a developer attempts to commit keys they get a message stating they cannot commit due to a secret being found in their changes. Below is a demonstration

<p align="left"><img src="https://raw.githubusercontent.com/zricethezav/gifs/master/pre-commit-1.gif"></p>

Here's the pre-commit hook I added for what was seen above:

#!/bin/sh
gitleaksEnabled=$(git config --bool hooks.gitleaks)
cmd="gitleaks --verbose --redact --pretty"
if [ $gitleaksEnabled == "true" ]; then
    $cmd
    if [ $? -eq 1 ]; then
cat <<\EOF
Error: gitleaks has detected sensitive information in your changes.
If you know what you are doing you can disable this check using:

    git config hooks.gitleaks false

EOF
exit 1
    fi
fi

Better Rules:

Take a look at the examples for explanations on sample rules.

Password Authentication

Made possible by go-git, thanks! Supply a --username and --password to clone username/password

More useful debugging

In prior versions changing the log level to debug didn't really yield much benefit to the user trying to gain insight as to what gitleaks was doing. I added a --debug option that prints some useful information like clone, patch, and individual regex times. An example output with --debug set looks like:

DEBU[2019-11-12T20:53:50-05:00] -------------------------
DEBU[2019-11-12T20:53:50-05:00] | Times and Commit Counts|
DEBU[2019-11-12T20:53:50-05:00] -------------------------
totalAuditTime:  6 seconds 664 milliseconds 767 microseconds
totalPatchTime:  4 seconds 837 milliseconds 101 microseconds
totalCloneTime:  1 second 666 milliseconds 658 microseconds
totalCommits:  3038
DEBU[2019-11-12T20:53:50-05:00] --------------------------
DEBU[2019-11-12T20:53:50-05:00] | Individual Regex Times |
DEBU[2019-11-12T20:53:50-0h5:00] --------------------------
(?i)linkedin(.{0,20})?(?-i)['\"][0-9a-z]{12}['\"]........................................1 second 711 milliseconds 333 microseconds
xox[baprs]-([0-9a-zA-Z]{10,48})?.........................................................41 milliseconds 468 microseconds
AIza[0-9A-Za-z\\-_]{35}..................................................................46 milliseconds 233 microseconds
access_token\$production\$[0-9a-z]{16}\$[0-9a-f]{32}.....................................62 milliseconds 45 microseconds
sk_live_[0-9a-z]{32}.....................................................................51 milliseconds 633 microseconds
sq0atp-[0-9A-Za-z\-_]{22}................................................................59 milliseconds 711 microseconds
sq0csp-[0-9A-Za-z\\-_]{43}...............................................................53 milliseconds 940 microseconds
(?i)aws(.{0,20})?(?-i)['\"][0-9a-zA-Z\/+]{40}['\"].......................................1 second 777 milliseconds 181 microseconds
amzn\.mws\.[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}..................63 milliseconds 72 microseconds
(?i)(facebook|fb)(.{0,20})?(?-i)['\"][0-9a-f]{32}['\"]...................................2 seconds 277 milliseconds 274 microseconds
(?i)(facebook|fb)(.{0,20})?['\"][0-9]{13,17}['\"]........................................2 seconds 232 milliseconds 74 microseconds
(?i)twitter(.{0,20})?['\"][0-9a-z]{35,44}['\"]...........................................1 second 787 milliseconds 692 microseconds
(?i)twitter(.{0,20})?['\"][0-9a-z]{18,25}['\"]...........................................1 second 803 milliseconds 922 microseconds
(?i)linkedin(.{0,20})?['\"][0-9a-z]{16}['\"].............................................1 second 742 milliseconds 230 microseconds
(?i)(api_key|apikey|secret)(.{0,20})?['|"][0-9a-zA-Z]{16,45}['|"]........................4 seconds 615 milliseconds 233 microseconds
https://hooks.slack.com/services/T[a-zA-Z0-9_]{8}/B[a-zA-Z0-9_]{8}/[a-zA-Z0-9_]{24}......44 milliseconds 51 microseconds
(?i)stripe(.{0,20})?['\"][sk|rk]_live_[0-9a-zA-Z]{24}....................................2 seconds 477 milliseconds 561 microseconds
(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}........................104 milliseconds 256 microseconds
-----BEGIN EC PRIVATE KEY-----...........................................................38 milliseconds 285 microseconds
(?i)heroku(.{0,20})?['"][0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}['"].............1 second 736 milliseconds 486 microseconds
(?i)(mailchimp|mc)(.{0,20})?['"][0-9a-f]{32}-us[0-9]{1,2}['"]............................2 seconds 288 milliseconds 332 microseconds
(?i)(mailgun|mg)(.{0,20})?['"][0-9a-z]{32}['"]...........................................2 seconds 236 milliseconds 818 microseconds
(?i)github(.{0,20})?(?-i)['\"][0-9a-zA-Z]{35,40}['\"]....................................1 second 735 milliseconds 952 microseconds
(?i)twilio(.{0,20})?['\"][0-9a-f]{32}['\"]...............................................1 second 796 milliseconds 873 microseconds