evoproc_procedures.schemas

Answer schemas for final LLM outputs.

These JSON Schemas constrain the structure of the final answer object the LLM should return. They are suitable both for:

  1. passing as fmt/response_format to your LLM call, and

  2. local validation via jsonschema.

Guidelines

  • Keep answers compact and deterministic for downstream comparison.

  • Prefer including a bounded confidence when available (0..1).

  • For numeric tasks, return both a human final_answer string (traceable) and a

machine final_answer_numerical for exact comparison.

Module Attributes

general_answer_schema

Minimal answer container; useful for quick smoke tests or free-form tasks.

bool_answer_schema

Boolean answer with both a natural-language answer and a strict bool.

GSM_answer_schema

GSM-style numeric answer:

ARC_answer_schema

ARC-style multiple choice:

Functions

get_schema(name)

Get a schema by short name.

list_schemas()

Return the available schema keys (general, bool, gsm, arc).

validate_answer(payload, schema, *[, ...])

Validate a payload against a JSON Schema; return an error message or None.
evoproc_procedures.schemas.general_answer_schema: Final[Dict[str, Any]] = {'$schema': 'https://json-schema.org/draft/2020-12/schema', 'additionalProperties': False, 'description': 'Minimal answer envelope with only a string answer.', 'properties': {'final_answer': {'description': 'Final answer text.', 'type': 'string'}}, 'required': ['final_answer'], 'title': 'GeneralAnswer', 'type': 'object'}

Minimal answer container; useful for quick smoke tests or free-form tasks.

Structure:

  • answer (str): Human-readable final answer text.

evoproc_procedures.schemas.bool_answer_schema: Final[Dict[str, Any]] = {'$schema': 'https://json-schema.org/draft/2020-12/schema', 'additionalProperties': False, 'description': 'Answer with both a textual rationale and a strict boolean.', 'properties': {'confidence': {'description': 'Optional model confidence for the boolean (0..1).', 'maximum': 1, 'minimum': 0, 'type': 'number'}, 'final_answer': {'description': "Final answer text (e.g., 'yes'/'no' with rationale).", 'maxLength': 1000, 'type': 'string'}, 'final_answer_bool': {'description': 'Machine-usable boolean answer.', 'type': 'boolean'}}, 'required': ['final_answer', 'final_answer_bool'], 'title': 'BooleanAnswer', 'type': 'object'}

Boolean answer with both a natural-language answer and a strict bool.

Structure:

  • answer (str): Human-readable justification or ‘yes’/’no’ text.

  • answer_bool (bool): Machine-usable boolean.

Tip

If you want the string and bool to agree, enforce it in code or add a post-check (schema can’t easily express cross-field string-to-bool logic).

evoproc_procedures.schemas.GSM_answer_schema: Final[Dict[str, Any]] = {'$schema': 'https://json-schema.org/draft/2020-12/schema', 'additionalProperties': False, 'description': 'Natural-language answer plus a strict numeric extraction.', 'properties': {'confidence': {'description': 'Optional model confidence (0..1).', 'maximum': 1, 'minimum': 0, 'type': 'number'}, 'final_answer': {'description': 'Final answer statement (end with the result).', 'maxLength': 1000, 'type': 'string'}, 'final_answer_numerical': {'description': 'Final numeric value (finite).', 'type': 'number'}, 'units': {'description': "Optional units for the numeric value (e.g., 'USD', 'km').", 'type': 'string'}}, 'required': ['final_answer', 'final_answer_numerical'], 'title': 'GSMNumericAnswer', 'type': 'object'}

GSM-style numeric answer:

Structure:

  • answer (str): Full reasoning / final statement in natural language.

  • answer_numerical (number): Final numeric value for exact comparison.

  • confidence (number, optional): 0..1 calibrated confidence.

Notes

  • Keep numbers finite; avoid ‘NaN’/’Infinity’ (validator checks in code).

  • If units matter, add an optional ‘units’ enum (e.g., ‘USD’, ‘km’, …).

evoproc_procedures.schemas.ARC_answer_schema: Final[Dict[str, Any]] = {'$schema': 'https://json-schema.org/draft/2020-12/schema', 'additionalProperties': False, 'description': 'Multiple-choice answer constrained to a single letter.', 'properties': {'choice_rationale': {'description': 'Optional brief explanation of the choice.', 'maxLength': 1000, 'type': 'string'}, 'confidence': {'description': 'Optional model confidence (0..1).', 'maximum': 1, 'minimum': 0, 'type': 'number'}, 'final_answer': {'description': 'Chosen option label.', 'enum': ['A', 'B', 'C', 'D'], 'type': 'string'}}, 'required': ['final_answer'], 'title': 'ARCChoiceAnswer', 'type': 'object'}

ARC-style multiple choice:

Structure:

  • answer (str enum): One of ‘A’, ‘B’, ‘C’, ‘D’.

  • confidence (number, optional): 0..1 confidence.

  • choice_rationale (string, optional): Brief justification for the choice.

evoproc_procedures.schemas.get_schema(name)[source]

Get a schema by short name.

Parameters:

name (str) – One of general, bool, gsm, arc.

Raises:

KeyError – If the name is unknown.

Return type:

Dict[str, Any]

evoproc_procedures.schemas.list_schemas()[source]

Return the available schema keys (general, bool, gsm, arc).

Return type:

Iterable[str]

evoproc_procedures.schemas.validate_answer(payload, schema, *, check_finite_number=True)[source]

Validate a payload against a JSON Schema; return an error message or None.

This is a convenience wrapper for quick tests. For richer error reporting, catch jsonschema.ValidationError yourself and format as needed.

Parameters:

  • payload (Dict[str, Any]) – Parsed JSON object from model output.

  • schema (Dict[str, Any]) – The JSON Schema dict to validate against.

  • check_finite_number (bool) – If True, reject non-finite numbers in numeric fields.

Return type:

Optional[str]

Returns:

None if valid, otherwise a short error message.