[INS-285] Fix custom detectors line number reporting to match the full regex instead of capture group

[mustansir14]

Description:

Problem

Currently for custom detectors, line number reporting works by looking for the Raw value from the detector result in the chunk. This works for most cases, except this specific one: When the provided regex contains a capture group, and the raw value is also present before the actual regex match.

This causes an incorrect line number to be reported.

An example for better explanation:

Regexp: secret *= *"([^"\r\n]+)"

Chunk:

"mysecret"
// some code
// some code
secret = "mysecret"

In this example, the regex matches on Line 4, but the Raw value is also present on Line 1, which causes Line 1 to be reported.

Solution

We have a primarySecret field in the detector's result. It's primary purpose is to indicate the value to determine correct line numbers when multiple regexes are provided. However, we can also use it to solve this problem. by setting to the full match value instead of the Raw value. This would result in correct line numbers to be reported.

Implementing this also requires primary_regex_name to be set in the detector config. I've added logic to ensure it is always set.

Checklist:

• Tests passing (make test-community)?
• Lint passing (make lint this requires golangci-lint)?

[kashifkhan0771]

Since we now always set the primary regex ourselves, is it still necessary to expose this as a user-configurable option and document it?

We should properly document this behavior in code, clearly explaining how and why the primary regex is always used for full matches.

We should explain this in the README as well: going forward, the custom detector line number will point to the full match of the primary regex or, if none is explicitly set, the first regex. This is important because a match may span multiple lines.

[mustansir14]

The user may want to set a regex other than the first one as the primary regex, so I think this option should still be exposed.

Totally agree with the documentation part! I'll do that

[kashifkhan0771]

Add a detailed comment here why are we using fullMatch instead of secret here.

[kashifkhan0771]

Write a test case for a match which span multiple lines.

[kashifkhan0771]

We set the full regex match as the primary secret value.
Reasoning:
The engine calculates the line number using the match. When a primary secret is set, it uses that value instead of the raw secret.
While the secret match itself is sufficient to calculate the line number, the same group match could appear elsewhere in the data.
To avoid ambiguity, we store the full regex match as the primary secret value.
This primary secret value is used only for identifying the exact line number and is not used anywhere else.

Example:
Full regex match: secret = ABC123
Secret (raw): ABC123

In this case, the primary secret value stores the full string `secret = ABC123`,
allowing the engine to pinpoint the exact location and avoid matching redundant occurrences of `ABC123` in the data.

[kashifkhan0771]

We need to write a test case in the engine too.
Location: https://github.com/trufflesecurity/trufflehog/blob/main/pkg/engine/engine_test.go#L166

[kashifkhan0771]

The test case should include the full regex match appearing across multiple lines, while the actual secret exists on only one line. The primary secret value will span multiple lines and should resolve to the line number where the match starts.

I want to confirm how the primary secret value is processed when it contains multi-line values - specifically, how the engine determines and reports the starting line.

[mustansir14]

Added

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}