Running a security check every week—or after every staging deployment—does not make it a continuous pentest by itself. Frequency is useful only when the test is safe to repeat, stable enough to compare, and explicit about what it did not cover.
The operating envelope matters as much as the scanner.
Begin With a Repeatable Profile
An automated baseline should have known behavior. If templates update during each run, a result can change because the target changed, because the scanner changed, or both.
Rorix pins its bundled HTTP-header templates with the scanner image and executes them with a narrow local runner that has no runtime update mechanism. Each report can therefore identify the catalog that actually ran.
Pinned checks do not mean frozen coverage forever. They make coverage changes deliberate: update the profile, validate it, version it, and let the next report show that the test definition changed.
Bound the Ways a Test Can Affect the Target
Safe repetition requires explicit limits. Rorix's current automated profile excludes destructive tests, denial-of-service payloads, credential attacks, write methods, private-network targets, and unbounded crawling.
Its optional OpenAPI testing is intentionally narrow:
- only
GETandHEADoperations; - no more than 50 operations;
- no more than 100 requests;
- a baseline request and at most one bounded query-boundary case per operation;
- no credentials, destructive payloads, or external references.
Request concurrency, response size, redirects, and total run duration are also bounded. These controls do more than reduce risk. They make duration and target load predictable enough for scheduling and CI.
Report Coverage, Not Just Findings
A result with zero findings can mean “the tested checks observed no issue.” It can also mean the OpenAPI document failed to load, the run timed out, or half the operations were skipped.
Those outcomes must not look identical.
A useful report records:
- which engines started and completed;
- which operations were tested;
- which operations were skipped and why;
- how many requests were sent;
- whether any limit or timeout ended coverage early.
Incomplete coverage is a result, not an implementation detail.
Track Findings as a Lifecycle
The value of repetition is change detection. A stable identity for each finding lets the platform distinguish:
- new — first observed in the current lifecycle;
- recurring — still present in consecutive relevant runs;
- fixed — absent after complete comparable coverage;
- regressed — returned after being resolved.
This is more useful than accumulating independent reports. Operators can focus on movement instead of rereading the same missing-header finding every week.
A finding should only be marked fixed when the relevant engine completed. Missing evidence from an incomplete run is not proof of remediation.
Introduce Enforcement After the Baseline
Start scheduled and CI runs in advisory mode. Review the initial findings, confirm which checks are stable for the target, and classify accepted risks and false positives.
Then enforce on new or regressed findings at the chosen severity threshold. This keeps an unrelated change from inheriting responsibility for every historical observation while still preventing the baseline from getting worse.
Rorix CI follows this model: accepted-risk and false-positive findings do not block, and enforcement can be enabled only after a stored target is verified and the baseline is understood.
Keep Automated and Human Testing Distinct
A safe automated profile is good at repeatable HTTP hardening checks and bounded API observations. It does not replace creative abuse-case exploration, authenticated business-logic review, complex authorization testing, or the judgment of a human penetration tester.
The distinction should remain visible in the product and the report. Calling a narrow baseline “comprehensive” creates the wrong confidence; calling it continuous makes sense when its scope and cadence are explicit.
Assign an Operator
Someone must own target verification, schedules, incomplete runs, profile changes, accepted-risk reviews, and scanner failures. Automation without an operator tends to produce either silent gaps or a backlog nobody trusts.
The Rorix Pentesting guide documents the current checks, safeguards, schedules, and CI model. The Pentesting API reference covers automation against stored targets.
Continuous testing is not “run more attacks.” It is a measured loop: verify scope, execute a bounded profile, preserve coverage, reconcile change, and decide what happens next.