ML Agents and Tool Use

An agent is a system that uses a model to decide actions over time, often calling tools, reading state, writing state, and observing results. The model call is only one component. The system boundary includes tool schemas, permissions, memory, retry behavior, approvals, and rollback.

Command Examples

python - <<'PY'
tool = {"name": "search_docs", "input_schema": {"type": "object"}}
print(tool["name"])
PY

Example output and meaning:

The important check is whether every tool has a narrow schema, clear permission boundary, idempotency story, and observable audit trail.

Agent Loop

flowchart TB
  Goal[Goal and constraints] --> State[State and memory]
  State --> Plan[Model plans next step]
  Plan --> Policy{Tool allowed and input valid?}
  Policy -->|No| Refuse[Refuse, ask, or escalate]
  Policy -->|Yes| Tool[Execute tool with deterministic auth]
  Tool --> Observe[Observation and audit event]
  Observe --> Decide{Done, continue, or rollback?}
  Decide -->|Continue| State
  Decide -->|Rollback| Compensate[Compensating action / human review]
  Decide -->|Done| Final[Final response and committed state]

Agents fail when any step is underspecified or trusted too broadly.

Tool Design

Good tools are narrow and explicit:

• typed inputs,
• clear read/write behavior,
• bounded side effects,
• idempotency keys for writes,
• safe retries,
• audit logs,
• timeouts,
• rate limits,
• permission checks outside the model.

Do not rely on the model to enforce authorization. The tool host must enforce it.

Memory and State

Agent memory can be useful, but every memory type has risk.

Guardrails

Guardrails should be layered:

1. restrict tool availability,
2. validate tool inputs,
3. require human approval for high-impact actions,
4. sandbox execution,
5. log every tool call,
6. run policy checks on outputs,
7. make rollback possible.

Evaluation

Agent evals must include trajectories, not only final answers.

Measure:

• task success,
• tool selection accuracy,
• invalid tool calls,
• unnecessary actions,
• recovery from tool errors,
• permission violations,
• cost and latency,
• state consistency after failure.

Agent safety test cases:

Agent Production Gaps

Agents are production workflows with probabilistic planning. Treat every tool call and state transition as an auditable operation.

Study Cards

References

• ReAct: Synergizing Reasoning and Acting in Language Models
• Toolformer
• OpenAI function calling guide
• NIST AI Risk Management Framework