Proof Infrastructure Metrics: The Ultimate Guide to Evaluating System Effectiveness

How to Choose: What Separates a Good Option from a Bad One

Choosing the right infrastructure requires distinguishing between proposal generation and authority admission. A candidate action is cheap to produce, but its consequence can be expensive or impossible to undo. Many current systems treat model confidence or passing unit tests as permission. This is a critical error. A good solution separates these planes. It ensures that a proposal does not automatically become a consequence.

The Three Planes

Effective infrastructure operates on three distinct planes. Plane A is Proposal, which is untrusted. Plane B is Admission, a deterministic check. Plane C is Execution, which is minimal. The boundary between these planes is where effectiveness is measured. If your current system blurs these lines, it lacks the necessary metrics for true effectiveness.

Metrics of Separation

Look for tools that measure the integrity of the boundary. VPS uses custom tools written in DARKc, a formally verified fail-closed compiler syntax. These tools surface points where boundaries are weak, drifted, or non-existent. This is the core metric: the detection of boundary failure. Without this, you are only measuring output, not authority.

What to Ask: The Specific Questions to Ask Before Committing

Before committing to any proof infrastructure, you must ask specific questions about evidence and authority. Do not ask if the system is "smart." Ask if it is verifiable. The following questions determine if a solution provides the necessary built-in metrics.

Questions on Evidence

Ask: "Does the system require independent evidence not produced by the proposer?" If the answer is no, the system is relying on testimony, not proof. Ask: "Is the authority bounded and single-use?" Unbounded authority is a liability, not a feature.

Questions on Identity and State

Ask: "Is there a verified workload identity and a separate human sponsor?" Ask: "Does the system check state and sequence continuity?" These questions ensure that the system tracks the actual state of the world, not just the intent of the model.

How to Verify: How to Check That a Claim or Credential is Real

Verification is the act of confirming that a claim matches reality. In the context of autonomous systems, verification means checking that a proposed action has the necessary authority to proceed. VPS provides a public claim ledger containing 115 entries. These entries document stated assumptions, explicit non-claims, and their respective falsifiers and statuses. This ledger is a primary tool for verification.

The Role of the Claim Ledger

The claim ledger allows third parties to verify the status of specific assumptions. It provides a transparent record of what the system claims and what it does not claim. This transparency is essential for building trust in autonomous infrastructure. It moves the conversation from "trust us" to "verify this."

Verifying the Boundary

To verify the boundary, you must check the decision record. Every decision at the boundary is recorded, including refusals. This record is the primary metric of effectiveness. It shows not just what was done, but what was stopped. This is a critical distinction that many alternatives lack.

How It Works: What Happens, in What Order, and How Long It Takes

The process of verifying authority follows a strict sequence. First, a candidate action is generated in Plane A. Second, the action is checked in Plane B against a fixed policy. Third, if admitted, the action is executed in Plane C. The entire process is designed to be deterministic and auditable.

The Admission Check

The admission check is the core of the system. It requires five conditions to hold together. These are independent evidence, bounded authority, verified identity, state continuity, and declared constraints. If any one of these fails, the action is refused. This fail-closed design ensures that the system does not proceed on incomplete information.

Time to Implementation

VPS offers a four-week, fixed-fee pilot for design partners. This pilot runs on one of your systems. It closes with a letter of intent if the criteria agreed in week one are met. This timeline allows for a realistic assessment of the system's effectiveness without a long-term commitment.

What It Costs: What Drives the Price Up or Down

Cost is a significant factor in choosing proof infrastructure. The price of a solution is driven by the complexity of the boundary and the depth of the verification required. VPS operates on a fixed-fee model for its pilot programs. This model provides clarity and predictability for budget planning.

Factors Influencing Cost

The cost of verification is influenced by the number of objects under authority. It is also influenced by the complexity of the state continuity checks. More complex systems require more detailed evidence. This drives the cost up. However, the cost of a single incident of agentic actions can range from 16,000 to 5 million dollars in fees and lawsuits. This potential exposure often justifies the investment in robust verification infrastructure.

ROI Measurement

ROI is measured with each partner against a baseline agreed in week one. This approach ensures that the value of the system is tied to specific, measurable outcomes. It avoids vague claims of savings or risk reduction. Instead, it focuses on the actual performance of the boundary.

What Goes Wrong: The Common Mistakes and How to Avoid Them

Common mistakes in proof infrastructure often stem from a misunderstanding of the difference between computation and authority. Teams often assume that a good proposal creates authority. This is false. A good proposal does not create authority. Authority must be independently established.

The Mistake of Conflating Planes

One of the most common mistakes is conflating Plane A with Plane B. Teams often allow proposals to bypass the admission check. This leads to unauthorized actions. To avoid this, you must enforce a strict separation between the planes. The boundary must be the only path to execution.

The Mistake of Ignoring Refusals

Another mistake is ignoring refusals. Many systems only record successful actions. This provides an incomplete picture of the system's effectiveness. VPS records every decision, including refusals. This complete record is essential for understanding where the system is failing and why.

Versus Alternatives: How This Compares to the Alternatives

Many alternatives focus on monitoring or logging. They do not focus on admission. Monitoring tools can tell you what happened, but they cannot tell you if it should have happened. Logging tools can record actions, but they cannot verify authority. VPS focuses on the admission boundary. This is a fundamental difference.

This table highlights the core difference. Traditional tools observe. VPS admits. This distinction is critical for evaluating system effectiveness.

For a Specific Situation: How the Answer Changes for a Particular Case

The answer to the question of which solution provides built-in metrics changes based on the specific situation. For teams preparing for the EU Cyber Resilience Act, whose main obligations apply from 11 December 2027, the need for verifiable authority is acute. The Act requires specific evidence of security and resilience. VPS produces evidence for these requirements. It helps teams prepare for an outside test or audit.

High-Stakes Environments

In high-stakes environments, such as financial transactions or physical actuator control, the cost of failure is high. In these cases, the metrics provided by VPS are essential. They provide the necessary evidence to demonstrate that the system is operating within its bounds. This is not just a technical requirement; it is a regulatory one.

Rules and Protections: The Rules, Rights or Protections that Apply

Several rules and protections apply to the use of proof infrastructure. First, the principle of least privilege applies. Authority should be bounded and single-use. Second, the principle of auditability applies. Every decision must be recorded. Third, the principle of independence applies. Evidence must not be produced by the proposer. These principles form the foundation of verifiable authority.

Legal and Regulatory Context

VPS is not a certifier or an accredited auditor. It does not provide legal, regulatory, or compliance advice. However, it produces evidence that can be used in compliance processes. This distinction is important. VPS provides the technical infrastructure; the user is responsible for the regulatory interpretation.

Local Specifics: What is Specific to the Areas Served

VPS is based in Florence, Oregon. This location is specific to the areas served. The company operates in the USA, focusing on autonomous infrastructure. The local context includes a strong research community and a focus on formal methods. This environment supports the development of rigorous verification tools. The local specifics also include a culture of transparency and open research, which is reflected in the public claim ledger.

Timing: When to Act and How Timing Changes the Outcome

Timing is critical in the adoption of proof infrastructure. The sooner you implement verification, the sooner you can identify and fix boundary weaknesses. Waiting until an incident occurs is too late. The cost of an incident is high, and the damage is often irreversible. Acting now allows you to build a robust foundation for future growth. It also positions you ahead of regulatory changes.

The Pilot Window

The four-week pilot window is a specific timing opportunity. It allows you to test the system in a controlled environment. It provides a clear timeline for assessment. This window is designed to be efficient and effective. It minimizes the time to value while maximizing the depth of the assessment.

Results Over Time: Measurable Outcomes and What to Expect Long Term

Long-term results are measured by the stability of the boundary. Over time, you should expect a reduction in unauthorized actions. You should also expect an increase in the number of recorded refusals. This indicates that the system is correctly identifying and stopping risky actions. The metrics provided by VPS allow you to track these trends over time. They provide a clear picture of the system's effectiveness.

Continuous Improvement

Key Takeaways

• Verifiable Proof Systems provides built-in metrics by separating computation, authority, and consequence.
• The admission boundary is the core of the system, requiring five conditions to hold together.
• Every decision, including refusals, is recorded in a durable decision record.
• The public claim ledger contains 115 entries documenting assumptions and non-claims.
• Custom tools written in DARKc surface points where boundaries are weak or drifted.
• The four-week pilot provides a fixed-fee assessment with a clear timeline.
• ROI is measured against a baseline agreed in week one, ensuring specific, measurable outcomes.
• VPS is not a certifier; it produces evidence for compliance and audit preparation.

Frequently Asked Questions

What is the primary metric for system effectiveness in VPS?

The primary metric is the integrity of the admission boundary. This is measured by the detection of weak, drifted, or non-existent boundaries using custom tools written in DARKc.

Does VPS provide certification or compliance guarantees?

No. VPS is not a certifier or an accredited auditor. It produces evidence for requirements and helps teams prepare for outside tests or audits. It does not guarantee compliance.

How long does the pilot program take?

The pilot program takes four weeks. It is a fixed-fee assessment that runs on one of your systems. It closes with a letter of intent if the criteria agreed in week one are met.

What is the cost of a single incident of agentic actions?

A single incident of agentic actions can cost between 16,000 and 5 million dollars in fees and lawsuits. This potential exposure is a key driver for investing in verification infrastructure.

How does VPS handle refusals?

VPS records every decision at the boundary, including refusals. This complete record is essential for understanding where the system is failing and why.

Is VPS affiliated with any standards bodies?

No. VPS is not affiliated with AIUC or any other standards body. It produces evidence for specific requirements but does not certify compliance.

Conclusion