Software audit: what to review before fixing, buying or rewriting

1. Start with the decision, scope and evidence sources

Define what the organization must decide: accept a delivery, invest, renew a vendor, explain slowness, reduce failures, prepare a migration or improve security. The same application requires different samples and depth depending on the question. Agreeing to “review the code” does not identify users, risks, modules or closure criteria.

List repositories, branches, artifacts, environments, databases, services, integrations, documentation, tickets, metrics, incidents and people available. Record dates, versions, access and limitations. Triangulate documents and interviews with artifacts and behavior; no source is a complete truth by itself.

• Decision, owner, date and consequences of acting or not acting.
• Users, flows and data with the greatest impact or uncertainty.
• Components, repositories, versions and environments included.
• Period of metrics, incidents, changes and activity reviewed.
• Samples, exclusions and access or evidence limitations.

Promising to review “all the code” hides a more important question: what evidence is needed to make the decision?

2. Model the product, architecture and real dependencies

Start with the outcome a user needs and follow the path through interface, services, data, queues, files, third parties and infrastructure. Document components, boundaries, contracts, owners and important decisions. Compare diagrams with deployments, configuration, traffic and repositories to find aspirational architecture.

ISO/IEC 25010:2023 provides a quality model that can help formulate assessment objectives; it does not replace specific requirements. Select relevant attributes—such as functional suitability, reliability, security, efficiency and maintainability—and define how they will be observed in the product context.

• Value proposition, users, flows and critical outcomes.
• Components, trust boundaries, data and integration contracts.
• Internal, external, operational and human dependencies.
• Known architecture decisions and inherited constraints.
• Quality attributes and expected evidence for each.

A current diagram is a testable hypothesis, not a substitute for the deployed system.

3. Reproduce symptoms and separate production signals

Define concrete examples: flow, user, volume, data, time, environment, expected and observed result. Do not blame a language or database without measurement. Decompose latency, errors and load across client, network, application, queries, storage, queues and integrations. Use tracing or profiling only within safe limits.

Google’s SRE monitoring chapter distinguishes symptoms from causes and black-box from white-box monitoring. Review whether signals show user experience and internal state, whether alerts require human action and whether changes correlate with variations. An average can hide long tails and segment-specific failures.

• Reproducible scenario, input, load, state and expected result.
• Latency percentiles, error rate, volume and saturation.
• Queries, dependencies, retries, blocking and background jobs.
• Correlation between version, deployment, incident and behavior.
• Limitations of the environment or data used for measurement.

4. Assess code and tests as the ability to change safely

Sample components according to criticality, change frequency, incidents and coupling. Review structure, responsibilities, duplication, complexity, error handling, concurrency, conventions and architecture consistency. Metrics guide questions; a generic threshold does not prove debt or quality by itself.

Connect tests to risks and behavior. Distinguish unit, integration, contract, system, regression, migration and performance tests. Check stability, data, isolation, duration, false results and pipeline use. High percentage coverage can execute lines without verifying important decisions.

• Criteria for selecting repositories, modules, classes or changes.
• Responsibilities, coupling, duplication and exception handling.
• Recurring defects and areas where fixes introduce new failures.
• Tests connected to critical flows, risks and contracts.
• Ease of understanding, changing, deploying and rolling back.

Maintainability appears in real changes: time, dependency, defects, tests and rollback ability.

5. Review data, integrations and migrations as product concerns

Map entities, keys, constraints, indexes, critical queries, growth, retention and access. Look for business rules duplicated across code and database, impossible states, manual reconciliations and silent failures. Verify how an inconsistency is detected and corrected without improvising edits in production.

For each API, file, queue or provider, document contract, authentication, limits, timeout, retry, idempotency, ordering, duplicates and observability. Review forward migrations and rollback, backup, compatibility and validation. A “stable” integration may depend on one person reprocessing failures.

• Ownership, classification, origin, destination and data lifecycle.
• Constraints and invariants code and storage must preserve.
• Critical queries, indexes, volume, archiving and operational windows.
• Contracts, errors, retries, idempotency and integration reconciliation.
• Migration, rollback, backup and integrity verification.

6. Assess security and supply chain according to risk

NIST SSDF organizes secure-development practices that can fit an existing lifecycle; OWASP SAMM helps assess and improve assurance activities. Examine how requirements are defined, software is protected, releases are produced and vulnerabilities are addressed. Do not turn maturity into a contest to maximize every practice.

For web applications, OWASP ASVS can supply verifiable requirements; OWASP SCVS covers components. Connect identity, authorization, inputs, secrets, logs, dependencies, builds and provenance to architecture and threats. A scanner does not understand business logic, data, compensating controls or practical exploitability.

• Requirements and threats relevant to functions, roles and data.
• Authentication, authorization, session, inputs, secrets and logging.
• Component inventory, versions, provenance and update process.
• Protection of repositories, pipelines, artifacts and environments.
• Intake, prioritization, remediation and learning from vulnerabilities.

A dependency alert is not automatically urgent risk; confirm version, use, exposure and impact first.

7. Verify delivery, operations, observability and recovery

Follow a change from requirement and commit through build, test, artifact, approval, deployment and verification. Review environment separation, configuration, secrets, migrations, feature flags and rollback. Identify manual steps, indispensable people and differences between versioned and running state.

Connect metrics, logs, traces, alerts, runbooks, incidents, backup and restore to critical flows. Confirm who responds, what information they receive and how learning occurs. Reliability is not proven because the system was available yesterday; it needs objectives, measurement, capacity, recovery and change discipline.

• Reproducible build, identifiable artifact and release provenance.
• Tests, approvals, gradual deployment, verification and rollback.
• Configuration and secrets separated from code and environments.
• User-oriented signals, actionable alerts and diagnosis.
• Backup, restoration, continuity and incident history.

8. Group causes and compare options before building the roadmap

Avoid hundreds of isolated observations. Group symptoms that share a cause: unclear boundaries, weak data model, missing contracts, manual process or concentrated knowledge. For each finding, document evidence, scope, impact, confidence, dependency and limitation. Separate deliberate debt from decisions that no longer fit.

The SEI treats technical debt as an architecture and management concern, not merely code cleanup. Compare no change, containment, documentation, monitoring, refactoring, encapsulation, gradual replacement, migration and rewrite. Prioritize by value and risk reduction, assign an owner and define the test that will demonstrate improvement.

• Grouping cause, evidence, components and observed effects.
• Confidence, uncertainty, limitation and additional evidence needed.
• Options with relative cost, dependency, risk and reversibility.
• Immediate actions, stabilization, modernization and sustained improvement.
• Owner, date, indicator and verifiable closure criterion.

A rewrite changes the type of risk; it does not remove it. Rules, data, integrations and knowledge still need to be recovered.