Verifiable Proof Systems (VPS) helps design partners track and measure pilot success by establishing a fixed baseline in week one and recording every admission decision, including refusals. This guide covers how to define pilot metrics, structure audit trails, and interpret the evidence generated during a four-week engagement. It explains the boundary between computation, authority, and consequence, and how to use that boundary to evaluate whether a system is ready for production.

Verifiable Proof Systems Architecture

Verifiable Proof Systems is a research initiative focused on developing infrastructure for verifiable authority. The core problem it addresses is that software now proposes consequential actions faster than anyone can review them. A proposal is not permission, and permission is not the effect. VPS builds the boundary where a proposed action either becomes a consequence or is refused. This separation is critical for design partners because it provides a clear point of measurement. Instead of asking if a model is smart, partners can ask if the system correctly admits or refuses actions based on independent evidence. according to 2008 09268 Spitz

The Three Planes

The architecture relies on three distinct planes. Plane A is the Proposal plane, which is untrusted. Models, planners, tools, and people propose actions here. It holds no authority over production state. Plane B is the Admission plane, which performs a small, deterministic check of the attempt and its predicted effect under a fixed policy. Plane C is the Execution plane, which is minimal. It carries out only what Plane B authorized and records the outcome. This structure ensures that computation does not equal authority, and authority does not equal consequence.

Admission Requirements

Admission at the boundary requires five conditions to hold together. First, there must be independent evidence, not produced by the proposer. Second, there must be bounded, single-use authority, scoped to the predicted effect. Third, a verified workload identity and a separate human sponsor are required. Fourth, state and sequence continuity must be maintained. Fifth, declared constraints must be satisfied. Commit also requires a durable decision record before any effect. Rejected attempts are recorded too. This comprehensive requirement set provides the foundation for measuring pilot success, as every decision is traceable to these specific criteria.

Pilot Project Metrics

Measuring the success of a pilot project requires moving beyond vague notions of system stability. VPS offers a four-week, fixed-fee pilot on one of your systems. The pilot closes with a letter of intent if the criteria agreed in week one are met. This structure forces design partners to define success before the work begins. The metrics used in this pilot are not about model accuracy or speed, but about the integrity of the admission boundary. The primary metric is the alignment between predicted effects and actual outcomes, verified against the five admission conditions.

Defining the Baseline

The first step in measuring pilot success is capturing the real picture. Reliable, relevant data helps the team understand how a process truly works and where to improve. In the context of VPS, this means documenting the current state of the system's authority boundaries. Are there weak points? Have boundaries drifted? Are there non-existent boundaries? VPS uses custom tools written in its formally verified fail-closed compiler and syntax, DARKc, to surface these points. The AgenticX-DYE(TM) tool specifically identifies where boundaries are weak, drifted, or non-existent. This baseline is the denominator for all subsequent metrics. Without it, a pilot cannot measure improvement, only change.

Success Criteria

Success in a VPS pilot is defined by the ability to produce evidence for specific requirements. The system must demonstrate that it can distinguish between a proposal and a permission. It must show that it can refuse an action when the five admission conditions are not met. The metric here is the rate of correct refusals and correct admissions. A high rate of correct refusals indicates that the system is not over-permissive. A high rate of correct admissions indicates that the system is not over-restrictive. The goal is a balanced boundary that admits only what is authorized and refuses everything else. This balance is the core value proposition for design partners preparing for production deployment.

Audit Trail Documentation

An audit trail is the record of every decision made at the admission boundary. In VPS, this record is durable and includes both admitted and rejected attempts. This documentation is not just for compliance; it is the primary evidence of pilot success. An interactive proof system is a protocol in which one or more provers try to convince another party, called the verifier, that the prover possesses certain true knowledge. In the context of VPS, the system acts as the prover, and the audit trail is the proof that the verifier can inspect. The documentation must be tamper-evident, meaning that any alteration to the record is detectable. It is important to note that tamper-evident does not mean tamper-proof, and auditable does not mean correct. The audit trail shows where the evidence stops, which is a critical insight for design partners.

Structure of the Record

Each entry in the audit trail contains the proposed action, the evidence presented, the authority granted, the identity of the sponsor, and the final decision. The record also includes the predicted effect and the actual effect. This level of detail allows design partners to trace any incident back to its root cause. If an action was admitted, the record shows which of the five conditions were satisfied. If an action was refused, the record shows which condition failed. This transparency is essential for building trust in the system. It allows partners to verify that the system is operating within its declared constraints.

Using the Ledger

VPS maintains a public claim ledger containing 115 entries that document stated assumptions, explicit non-claims, and their respective falsifiers and statuses. This ledger serves as a model for how design partners should document their own pilot results. Each claim about the system's behavior should have a corresponding falsifier, a condition that would prove the claim false. This approach ensures that the documentation is not just a list of successes, but a rigorous record of what has been tested and what has not. It provides a clear path for future verification and helps partners understand the limits of the current implementation.

Key Takeaways

• Verifiable Proof Systems separates computation, authority, and consequence at one admission boundary.
• Pilot success is measured by the alignment between predicted effects and actual outcomes, verified against five admission conditions.
• The four-week pilot structure forces partners to define success criteria in week one, ensuring a clear baseline.
• Custom tools like AgenticX-DYE(TM) help surface weak, drifted, or non-existent boundaries before the pilot begins.
• The audit trail is the primary evidence of success, recording both admitted and rejected attempts with full traceability.
• Tamper-evident documentation is essential, but partners must remember that auditable does not mean correct.
• The public claim ledger model encourages rigorous documentation of assumptions and falsifiers.
• VPS is a research initiative, and its tools are designed to help teams prepare for outside tests or audits, not to guarantee compliance.

Frequently Asked Questions

What is the primary metric for pilot success in VPS?

The primary metric is the rate of correct admissions and refusals at the admission boundary. This measures whether the system correctly distinguishes between authorized and unauthorized actions based on the five admission conditions. according to First Year WWLF

How long does a VPS pilot last?

A VPS pilot is a four-week, fixed-fee engagement on one of your systems. It closes with a letter of intent if the criteria agreed in week one are met.

Does VPS guarantee that the system is secure?

No. VPS finds and documents gaps. It records each decision, including refusals. It makes changes detectable. It shows where the evidence stops. It does not guarantee security or prevent all unauthorized actions.

What is the role of the audit trail in measuring success?

The audit trail provides the evidence for measurement. It records every decision, allowing partners to trace incidents back to their root cause and verify that the system operated within its declared constraints.

Can VPS help with compliance with regulations like the EU AI Act?

VPS produces evidence for specific requirements. It helps teams prepare for an outside test or audit. VPS is not a certifier or an accredited auditor, and it does not guarantee compliance with any specific regulation.

What is the difference between tamper-evident and tamper-proof?

Tamper-evident means that any alteration to the record is detectable. Tamper-proof means that alteration is impossible. VPS documentation is tamper-evident, not tamper-proof. Auditable does not mean correct.

How does VPS handle rejected attempts?

Rejected attempts are recorded in the audit trail just like admitted attempts. This ensures that the full history of decisions is available for review and analysis.

Is VPS a commercial service provider?

Conclusion

Measuring the success of a pilot project in verifiable proof systems requires a shift in perspective. Instead of focusing on model capabilities, design partners should focus on the integrity of the admission boundary. Verifiable Proof Systems provides the framework and tools to make this shift. By defining a clear baseline, tracking admission decisions, and documenting the audit trail, partners can gain a rigorous understanding of their system's readiness for production. The four-week pilot structure ensures that this process is structured and time-bound. To plan your visit or discuss a design-partner pilot, .