Detecting and Removing Secrets from Git History with Betterleaks and git-filter-repo
Have you ever wondered exactly what has entered your Git repository over time? Accidental commits of debug functions, incorrect comments, or copied files can usually be fixed with a follow-up commit. However, when sensitive data such as third-party service keys, integration credentials, database dumps, or entire /private/uploads directories are involved, simply committing a deletion does not remove the information from history.
Secrets committed to any repository, even a private one, remain accessible through git log or by checking out older commits. Anyone who can read the repository, clone it, fork it, or access connected CI/CD pipelines gains exposure. Contracts with clients frequently prohibit storing credentials or sensitive data in repositories for this reason, emphasizing the need for traceable password issuance and revocation processes.
The first step is to scan the repository thoroughly. The recommended tool is Betterleaks, which replaced earlier utilities such as gitleaks. It applies rule-based detection for patterns resembling AWS keys or GitHub tokens, uses BPE tokenization to reduce false positives, and automatically decodes Base64, hex, and URL-encoded content. Installation is available via Homebrew, Docker, or source build.
Running betterleaks git . --log-opts="--all" -v examines every commit, patch, and change across all branches. Results include details such as the rule ID, matched secret, author, commit message, and file path. Reports can be exported in JSON format for further processing. Manual review of findings is essential to distinguish real secrets from false positives.
After confirmation, git-filter-repo is used to rewrite history. Secrets extracted from the report are converted into replacement rules, and files matching dangerous extensions such as .sql, .env, or .bak are listed for removal. Before execution, secrets must be moved to external configuration files such as .env.local or settings.local.php so the application continues to function after cleanup.
The procedure is destructive and irreversible. A mirror backup must be created, all pull requests closed, and a clean clone prepared. The command git filter-repo --invert-paths --paths-from-file paths-to-remove.txt --replace-text secrets-to-replace.txt performs both file deletion and secret replacement in one pass. After verification with a new scan, aggressive garbage collection and a forced mirror push complete the process.
Every contributor and CI/CD pipeline must then perform a fresh clone rather than a pull or rebase, because commit hashes have changed. Even after these steps, old objects may survive in forks, backups, or caches, making secret rotation mandatory.
Related articles
Yandex Disk Data Center Incidents Leave Photos and Videos Inaccessible, Custom Recovery Tool Emerges
Following incidents at Yandex data centers in Sasovo and Vladimir, large numbers of photos and videos stored on Yandex Disk became unavailable for normal download. The company attributed the restrictions to load reduction, yet analysis indicates permanent loss of certain data clusters with no confirmed backups. Photos remain partially recoverable through lower-resolution S3 copies such as XXXL and XXL, while videos suffer from missing byte ranges that produce truncated or corrupted files. A researcher developed a specialized tool using mitmproxy to intercept browser requests and download available file variants in parallel streams. The utility also attempts block-by-block video retrieval to salvage whatever segments remain accessible. Users are warned that continued use of the tool may violate service terms and risk account suspension. The developer advises downloading data immediately rather than waiting for potential further degradation.
AT&T to Pay $177 Million to Settle Two 2024 Customer Data Breach Class Actions
AT&T has agreed to pay a total of $177 million to resolve two separate class-action lawsuits stemming from major customer data leaks that occurred in 2024. The settlement, approved on October 2 by Judge Sidney Fitzwater in the Northern District of Texas, allocates $149 million to the first breach and $28 million to the second, covering compensation, attorney fees, and administrative costs. The first incident, disclosed on March 30, 2024, exposed personal information of 73 million individuals, including names, addresses, phone numbers, dates of birth, account passwords, billing numbers, and Social Security numbers for some victims, with records dating from 2019 or earlier found on the dark web. The second breach originated from unauthorized access to AT&T’s Snowflake cloud environment between April 14 and 25, 2024, affecting nearly all mobile customers and revealing call metadata such as phone numbers, interaction counts, durations, and cell tower identifiers from May to October 2022 plus January 2023. No evidence of direct system intrusion was found in the first case, and the second exposure did not include Social Security numbers or call content. Individual claimants can receive up to $5,000 or $2,500 respectively for documented losses, with the claims deadline set for December 18, 2025. AT&T settled without admitting any wrongdoing or liability.
EY Confirms Data Breach Exposing Goldman Sachs and Man Group Client Details via Checkmarx Vulnerability
EY has confirmed a data breach involving client documents accessed through a third-party technology services management platform used to support its tax operations. The unauthorized access occurred via a vulnerability in Checkmarx software and lasted from March 28 to April 12, with detection only on April 23. Exposed information included names, addresses, tax identification numbers, email addresses, and financial details belonging to clients in Goldman Sachs wealth management and the Man Group investment fund. Neither Goldman Sachs nor Man Group systems were compromised, confirming the exposure originated solely from EY's environment. The leaked materials consisted of attachments from support tickets rather than core system data. EY has engaged independent security firms, notified regulators in California, Texas, Massachusetts, and Vermont, and offered credit monitoring to affected individuals.
ThreatMon Exposes SQL Server xp_cmdshell Abuse Stealing Credentials from Viva Aerobus
Researchers at ThreatMon discovered an exposed attacker infrastructure containing 17 attack tools and data stolen from airline Viva Aerobus. Attackers gained operating system access through the xp_cmdshell feature in SQL Server, which allows execution of system commands when enabled. They issued Windows and encoded PowerShell commands directly through database sessions. Data exfiltration occurred by reading files, splitting content into smaller chunks, converting to Base64, and returning results via normal SQL query responses to avoid network detection. Recovered materials included browser, Windows, and SQL credentials, source code, configuration files, database connection strings, OAuth, email, SFTP, and payment system references, plus connection history from SQL Server Management Studio. Activity took place between September 25 and 29, with no identified initial access vector or link to known malware families.