Change AI engine to execute tools in Java instead of on frontend (#6116)

# Description of Changes
Redesign AI engine so that it autogenerates the `tool_models.py` file
from the OpenAPI spec so the Python has access to the Java API
parameters and the full list of Java tools that it can run. CI ensures
that whenever someone modifies a tool endpoint that the AI enigne tool
models get updated as well (the dev gets told to run `task
engine:tool-models`).

There's loads of advantages to having the Java be the one that actually
executes the tools, rather than the frontend as it was previously set up
to theoretically use:
- The AI gets much better descriptions of the params from the API docs
- It'll be usable headless in the future so a Java daemon could run to
execute ops on files in a folder without the need for the UI to run
- The Java already has all the logic it needs to execute the tools 
- We don't need to parse the TypeScript to find the API (which is hard
because the TS wasn't designed to be computer-read to extract the API)

I've also hooked up the prototype frontend to ensure it's working
properly, and have built it in a way that all the tool names can be
translated properly, which was always an issue with previous prototypes
of this.

---------

Co-authored-by: Anthony Stirling <[email protected]>
Co-authored-by: EthanHealy01 <[email protected]>
This commit is contained in:
James Brunton
2026-04-20 15:57:11 +01:00
committed by GitHub
co-authored by Anthony Stirling EthanHealy01
parent cc9650e7a3
commit e5767ed58b
45 changed files with 3565 additions and 1285 deletions
+2 -2
View File
@@ -4,7 +4,7 @@ from typing import Annotated, Literal
from pydantic import Field
from stirling.models import ApiModel, OperationId
from stirling.models import ApiModel, ToolEndpoint
from .common import StepKind, ToolOperationStep
@@ -13,7 +13,7 @@ class AiToolAgentStep(ApiModel):
kind: Literal[StepKind.AI_TOOL] = StepKind.AI_TOOL
title: str
description: str
tool: OperationId
tool: ToolEndpoint
instruction: str
+2 -2
View File
@@ -5,7 +5,7 @@ from typing import Literal, assert_never
from pydantic import Field, model_validator
from stirling.models import OPERATIONS, ApiModel, OperationId
from stirling.models import OPERATIONS, ApiModel, ToolEndpoint
from stirling.models.agent_tool_models import AGENT_OPERATIONS, AgentToolId, AnyParamModel, AnyToolId
@@ -110,7 +110,7 @@ class ToolOperationStep(ApiModel):
def validate_tool_parameter_pairing(self) -> ToolOperationStep:
if isinstance(self.tool, AgentToolId):
expected_type = AGENT_OPERATIONS[self.tool]
elif isinstance(self.tool, OperationId):
elif isinstance(self.tool, ToolEndpoint):
expected_type = OPERATIONS[self.tool]
else:
assert_never(self.tool)
+3 -3
View File
@@ -4,7 +4,7 @@ from typing import Annotated, Any, Literal
from pydantic import Field
from stirling.models import ApiModel, OperationId, ParamToolModel
from stirling.models import ApiModel, ParamToolModel, ToolEndpoint
from .agent_specs import AgentSpec
from .common import WorkflowOutcome
@@ -12,7 +12,7 @@ from .common import WorkflowOutcome
class ExecutionStepResult(ApiModel):
step_index: int
tool: OperationId | None = None
tool: ToolEndpoint | None = None
success: bool
output_summary: str | None = None
output_data: dict[str, Any] = Field(default_factory=dict)
@@ -33,7 +33,7 @@ class AgentExecutionRequest(ApiModel):
class ToolCallExecutionAction(ApiModel):
outcome: Literal[WorkflowOutcome.TOOL_CALL] = WorkflowOutcome.TOOL_CALL
tool: OperationId
tool: ToolEndpoint
parameters: ParamToolModel
rationale: str | None = None