evoproc.validators

validators.py

Validator helpers for global-state procedures.

Design overview

• Global state: Steps may read any variable produced by earlier steps or given in the problem. There is no step-to-step pass-through requirement.

• Step 1 rule: The first step must take exactly one input: problem_text. (No other inputs will be provided externally.)

• Final step rule: The last step must output exactly final_answer (a description, not the computed value).

• Safety checks: Every declared input must be resolvable from prior outputs (or problem_text), variables should not be silently redefined, and unused outputs are flagged to keep the procedure minimal and readable.

All functions return a list of Diagnostic objects that can be fed into a repair loop. Use validate_procedure_structured() to run the default suite.

Module Attributes

Action	machine-usable repair hints for downstream auto-fix prompts.
Severity	fatal means the procedure is not runnable without regeneration or structural rewrite; repairable means a small, local JSON edit should suffice.
Validator	Callable signature for a validator function.
DEFAULT_VALIDATORS	Default validator suite for global-state procedures with a strict Step 1 and final step.

Functions

validate_final_step_output(p)	Require the final step to output exactly ["final_answer"].
validate_first_step_inputs(p)	Enforce the Step 1 input contract: it must be exactly ["problem_text"].
validate_inputs_resolvable_from_prior(p)	Ensure every step input is available from global state when the step runs.
validate_no_redefine_existing_vars(p)	Discourage shadowing: warn if a step re-uses an already-produced variable name.
validate_procedure_structured(p[, validators])	Run a composed set of validators and return de-duplicated diagnostics.
validate_unused_outputs(p)	Flag dead variables: outputs never consumed by any later step.

Classes

Diagnostic

Structured validator finding.

evoproc.validators.Action

machine-usable repair hints for downstream auto-fix prompts.

Type:

Action

alias of Literal[‘PATCH_LOCALLY’, ‘REWRITE_FIRST_STEP’, ‘ADD_FINAL_STEP’, ‘EXTEND_PROCEDURE_TO_FINAL’, ‘ADD_MISSING_PRODUCER’]

evoproc.validators.Severity

fatal means the procedure is not runnable without regeneration or structural rewrite; repairable means a small, local JSON edit should suffice.

Type:

Severity

alias of Literal[‘repairable’, ‘fatal’]

class evoproc.validators.Diagnostic

Bases: TypedDict

Structured validator finding.

Keys

severity:

Either "fatal" or "repairable".

action:

A short, machine-usable hint for an auto-repair prompt (see Action).

message:

Human-readable description of the issue.

details:

Machine-targeted payload (e.g., step ids, variable names) that an auto-repair routine can use to patch the JSON.

severity: Literal['repairable', 'fatal']

action: Literal['PATCH_LOCALLY', 'REWRITE_FIRST_STEP', 'ADD_FINAL_STEP', 'EXTEND_PROCEDURE_TO_FINAL', 'ADD_MISSING_PRODUCER']

message: str

details: Dict[str, Any]

evoproc.validators.validate_first_step_inputs(p)

Enforce the Step 1 input contract: it must be exactly ["problem_text"].

Rationale

In the global-state design, the only external input is the problem text. All additional information must be produced by earlier steps (which, for Step 1, means there are none), so Step 1 cannot declare any other inputs.

type p:

Dict[str, Any]

param p:

Procedure JSON (already parsed to Python dict).

returns:

fatal if Step 1 inputs differ from exactly ["problem_text"], otherwise empty.

rtype:

List[Diagnostic]

Parameters:

p (Dict[str, Any])

Return type:

List[Diagnostic]

evoproc.validators.validate_final_step_output(p)

Require the final step to output exactly ["final_answer"].

Notes

final_answer is terminal and consumed outside the procedure. The final step should describe the final answer (no numeric computation here).

Parameters:

p (Dict[str, Any]) – Procedure JSON (already parsed to Python dict).

Returns:

fatal if the last step’s outputs are not exactly ["final_answer"].

Return type:

List[Diagnostic]

evoproc.validators.validate_inputs_resolvable_from_prior(p)

Ensure every step input is available from global state when the step runs.

Rule

For each step i and input variable v:

• Either v == "problem_text", or

• some prior step k < i produced v in its outputs.

type p:

Dict[str, Any]

param p:

Procedure JSON (already parsed to Python dict).

returns:

fatal diagnostics for inputs that cannot be traced to problem_text or some earlier step’s outputs.

rtype:

List[Diagnostic]

Parameters:

p (Dict[str, Any])

Return type:

List[Diagnostic]

evoproc.validators.validate_no_redefine_existing_vars(p)

Discourage shadowing: warn if a step re-uses an already-produced variable name.

Rationale

In a global state, silently overwriting prior variables can confuse both the LLM and debuggers. Prefer distinct names or explicitly mark transformations (e.g., normalized_total instead of re-using total).

type p:

Dict[str, Any]

param p:

Procedure JSON (already parsed to Python dict).

returns:

repairable diagnostics suggesting renames for redefined variables.

rtype:

List[Diagnostic]

Parameters:

p (Dict[str, Any])

Return type:

List[Diagnostic]

evoproc.validators.validate_unused_outputs(p)

Flag dead variables: outputs never consumed by any later step.

Exemptions

• final_answer (terminal output)

• problem_text (not an output; guarded for safety)

Algorithm

Walk backwards through steps, accumulating the set of inputs that appear at or after each position. Any output at index i that never appears in a later step’s inputs is considered unused.

type p:

Dict[str, Any]

param p:

Procedure JSON (already parsed to Python dict).

returns:

repairable diagnostics suggesting removal of unused outputs per step.

rtype:

List[Diagnostic]

Parameters:

p (Dict[str, Any])

Return type:

List[Diagnostic]

evoproc.validators.Validator

Callable signature for a validator function.

alias of Callable[[Dict[str, Any]], List[Diagnostic]]

evoproc.validators.DEFAULT_VALIDATORS: List[Callable[[Dict[str, Any]], List[Diagnostic]]] = [<function validate_first_step_inputs>, <function validate_final_step_output>, <function validate_inputs_resolvable_from_prior>, <function validate_no_redefine_existing_vars>, <function validate_unused_outputs>]

Default validator suite for global-state procedures with a strict Step 1 and final step.

evoproc.validators.validate_procedure_structured(p, validators=None)

Run a composed set of validators and return de-duplicated diagnostics.

Pipeline:

1. Coerce p into a Procedure via Pydantic (schema check). - If this fails, return fatal diagnostics and short-circuit.

2. Dump the Procedure back to a normalized JSON dict.

3. Run the semantic validators on that normalized dict.

Parameters:

• p (Dict[str, Any]) – Procedure JSON to validate (already parsed to Python dict).

• validators (Optional[List[Callable[[Dict[str, Any]], List[Diagnostic]]]]) – Optional custom list of validator callables. If omitted, uses DEFAULT_VALIDATORS.

Returns:

De-duplicated diagnostics suitable for a repair loop.

Return type:

List[Diagnostic]