GoldDigger： a powerful file sensitive information search tool

Most of the regular expressions used in this project come from d1vious's 【git-wild-hunt】project.

tool requirements

Python 3

tool installation

Since GoldDigger is developed based on Python 3, we first need to install and configure the Python 3 environment on our local device:

virtualenv -p python3 .

source bin/activate

python dig.py --help

tool download

Researchers can use the following command to clone the source code of this project to their local machine:

git clone https://github.com/ustayready/golddigger.git

tool usage

usage: dig.py [-h] [-e EXCLUDE] [-g GOLD] -d DIRECTORY [-r RECURSIVE] [-l LOG]

 

optional arguments:

-h, --help            show this help message and exit

-e EXCLUDE, --exclude EXCLUDE

JSON file containing extension exclusions

-g GOLD, --gold GOLD   JSON file containing the gold to search for

-d DIRECTORY, --directory DIRECTORY

Directory to search for gold

-r RECURSIVE, --recursive RECURSIVE

Search directory recursively?

-l LOG, --log LOG   Log file to save output

Parameter explanation

-h, --help   Show tool help information and exit

-e EXCLUDE, --exclude EXCLUDE

JSON file containing the extensions to be excluded

-g GOLD, --gold GOLD   JSON file containing the results to be searched

-d DIRECTORY, --directory DIRECTORY

Directory path to search for sensitive information

-r RECURSIVE, --recursive RECURSIVE

Whether to enable the file directory recursive search feature

-l LOG, --log LOG   Output the path of the result log file

Tool usage examples

GoldDigger will first recursively search all directories and attempt to match the content of the files based on the entries in the gold.json file. In addition, we can also use an exclusions.json file to exclude files or extensions that we do not want to search. This tool supports specifying the root directory for recursive search with the --directory parameter option.

The following is an example of the structure of the tool's output:

~/Engagements/CustomerName/data/randomfiles/

~/Engagements/CustomerName/data/randomfiles2/

~/Engagements/CustomerName/data/code/

The following commands can also be provided to parse all three reports:

~/Engagements/CustomerName/data/randomfiles/

~/Engagements/CustomerName/data/randomfiles2/

~/Engagements/CustomerName/data/code/

Run results

This tool will create a log file containing the scan results. Due to the nature of regular expressions, there may be many false positives. Nevertheless, this tool has been proven to increase productivity when processing thousands of files.