Continuous Penetration Testing: Why a Yearly Pentest Leaves Vibe Coded Apps Exposed

Most software is penetration tested once a year, and plenty of applications are never tested at all. A vibe coded app can change more in a single week than a traditional codebase changed across that whole year, which leaves a wide gap between how fast your code moves and how rarely anyone checks it for security holes, and continuous penetration testing exists to close that gap.

Usually sold as PTaaS (penetration testing as a service), continuous penetration testing runs security tests against your live application on a schedule instead of once a year, so when code ships the tests run again and fresh vulnerabilities surface while they are still cheap to fix. For an application that changes daily, that cadence matters far more than the depth of any single test.

What continuous penetration testing actually is

A traditional penetration test is a snapshot: a tester spends a week or two attacking your application, writes a report and hands it over, and that report is accurate the day it lands before it slowly goes stale, because every deploy that follows it is ground nobody has checked.

Continuous penetration testing changes the rhythm, so instead of one deep test a year your application is tested on a repeating schedule, daily in the case of the platform we are building. What makes it continuous is that the tests run on a schedule, so the window between a vulnerability shipping and someone noticing it shrinks from a year down to a day. The external part means the testing happens against the deployed application from the public internet, the same vantage point an attacker has, with nothing to install and no source code to hand over. The agentic part means AI agents drive the testing, so rather than a static scanner firing a fixed checklist, the agents probe the app, read what comes back and chase a lead the way a human tester would, at a frequency no human could sustain every day.

The yearly pentest was built for software that barely moved

Annual testing made sense when a release happened twice a year and a change request took a fortnight, and most security and compliance frameworks still assume that rhythm, which is how the annual pentest became the box you ticked for your insurer, your enterprise customer or your auditor.

Modern delivery broke that assumption, and vibe coding broke it harder, because when an AI agent can ship a feature in an afternoon, a pentest from January tells you nothing about the code that went live in March, and you end up paying for a snapshot of an application that no longer exists.

Why vibe coded apps need this most

Vibe coded apps carry two traits that make continuous testing close to essential rather than optional. The first is that they change constantly, because the whole point of building with Lovable, Bolt, Cursor or Claude Code is speed, so features land daily and each one is a fresh chance to ship a security hole that an annual test will never see. The second is that they start from a weaker baseline, because Veracode's 2025 GenAI Code Security Report found that 45% of AI generated code shipped with a known security vulnerability, and we see the same pattern in the apps we audit, where access controls are missing, keys are exposed and input goes unvalidated. Our field notes on vibe coded app security gather that data in one place, and the conclusion is hard to dodge: when the starting point is riskier and the change rate is higher, testing once a year leaves most of the year unchecked.

What agentic testing adds that a scanner does not

Plain vulnerability scanners have been around for decades, and they are fast, cheap and shallow, because they fire a fixed list of checks, flag anything that matches a signature and bury the few real findings under a pile of false positives, with no way to reason about how your application actually behaves.

Agentic testing sits between a blunt scanner and a human pentester, because an agent can read a response, form a theory, test it and then follow the result into the next theory, which lets it find the broken access control that only shows up when you chain three requests together, the kind of flaw a signature scanner walks straight past, and it can do that every day.

The limits matter too, so here is the honest version: agentic testing does not replace a deep human audit, and we will not pretend that it does, because a skilled tester still finds things automation misses, especially the business logic flaws that are specific to your product. The sensible setup is layered, with automation giving you daily coverage and catching regressions early while a human goes deep on a schedule, and you want both rather than either on its own.

Introducing Argus

We are building Argus, a continuous penetration testing platform for applications and vibe coded apps. Argus is not live yet, so what follows describes what it is designed to do rather than results we are claiming.

Argus is built to:

• Run scheduled external vulnerability tests against your live application every day rather than once a year.
• Use AI agents that probe the app the way an attacker would and chase what they find, rather than firing a static checklist.
• Map findings to the OWASP Top 10 and the OWASP Top 10 for LLM Applications, so the output speaks the language your auditor and insurer expect.
• Watch for the failure modes vibe coded apps ship most often, including exposed API keys, missing authentication, server side request forgery and database access that is far too broad.
• Process and store your data in Australia, with reports written in plain English rather than CVSS soup.
• Tell you the moment something new appears, instead of in a PDF eleven months from now.

Argus opens for early access soon, so if you run an app you would rather test every day than once a year, tell us about it and we will let you know the moment it is ready.

How Argus fits alongside a deeper audit

Argus is the continuous layer rather than a replacement for everything else we do, and the way we think about it is a sequence:

• The free vibe code scanner is the surface check that takes about fifteen seconds, where you paste a URL, see the obvious issues and commit to nothing.
• The vibe code audit is the deep dive, where a human engineer takes your app apart and hands back a prioritised fix list.
• Managed AI app security wraps ongoing protection, monitoring and an Essential Eight uplift around your application across the year.
• Argus is the daily check, the continuous external testing that runs in the background and flags new exposure as it appears.

Most businesses start with a deeper audit to clear the backlog, then keep Argus running so the app never quietly drifts back into the dark.

Continuous testing and your Australian obligations

The compliance case is getting harder to ignore, because the ACSC Essential Eight is built around patching and monitoring on a tight cycle that an annual test cannot evidence. Under the Privacy Act and the Notifiable Data Breaches scheme, a breach you could have caught with regular testing is a much harder conversation with the OAIC than one you genuinely could not have seen coming, and buyers who ask you to prove cyber maturity, whether through SMB1001 or a procurement questionnaire, increasingly want evidence of ongoing testing rather than a certificate from a year ago. Continuous testing turns security from an annual event into a background process, and for a business shipping code most weeks that is the only cadence that matches reality.

The short version

Annual penetration testing was designed for software that sat still, and vibe coded apps do the opposite, so continuous external agentic testing closes the gap between how fast you ship and how fast anyone checks what you shipped, at a price and cadence a small business can actually live with. Argus is how we are bringing that to Australian businesses and the apps they build with AI, and it is launching soon, so to get a look when it opens you can register your interest, or run the free scanner on your app today to see where you stand right now.