evoproc_procedures.pipelines

Functions

run_full_procedure_structured(idx, question, ...)

Create → repair → execute a structured procedure for one task.

evoproc_procedures.pipelines.run_full_procedure_structured(idx, question, *, model, query_fn, run_steps_fn, answer_schema, seed=1234, print_diagnostics=False)

Create → repair → execute a structured procedure for one task.

Steps

1. Build a creation prompt (global-state constraints) for question.

2. Ask the LLM (via query_fn) to emit a Procedure JSON that validates Procedure.model_json_schema().

3. Iteratively repair with repair_procedure_structured(…) until it validates.

4. Execute the procedure with run_steps_fn to obtain the final answer/state.

type idx:

int

param idx:

An index for logging (e.g., dataset row).

type question:

str

param question:

Natural-language task text (fed as problem_text).

type model:

str

param model:

Backend model name (passed through to query_fn).

type query_fn:

Callable[[str, str, Optional[Dict[str, Any]], Optional[int]], str]

param query_fn:

Callable (prompt, model, fmt, seed) -> str. Provide your backend.

type run_steps_fn:

Callable[..., Dict[str, Any]]

param run_steps_fn:

Your executor. Two common signatures:

1. run_steps_fn(proc, question, answer_schema, model, print_bool=False) -> state

2. run_steps_fn(proc, *, inputs: dict, answer_schema: dict, model: str) -> state

This wrapper will try (a) first, then (b).

type answer_schema:

Dict[str, Any]

param answer_schema:

JSON Schema for the final answer object (e.g., GSM/ARC).

type seed:

Optional[int]

param seed:

Optional seed for determinism (if backend supports it).

type print_diagnostics:

bool

param print_diagnostics:

If True, prints repair diagnostics each iteration.

returns:

Validated procedure JSON and the execution state (should include the final answer).

rtype:

(procedure_json, state_dict)

raises ValueError:

If the model response cannot be parsed as JSON.

raises RuntimeError:

If the procedure cannot be repaired to pass validation.

Parameters:

• idx (int)

• question (str)

• model (str)

• query_fn (Callable[[str, str, Dict[str, Any] | None, int | None], str])

• run_steps_fn (Callable[[...], Dict[str, Any]])

• answer_schema (Dict[str, Any])

• seed (int | None)

• print_diagnostics (bool)

Return type:

Tuple[Dict[str, Any], Dict[str, Any]]