ReAct, For Real: Building Deterministic Tool-Using Agents with LangGraph

(feat. a Safe AST Calculator)

A practical guide to wiring a reason + act loop that’s auditable, stable, and observable. We use LangGraph for orchestration and a hardened AST calculator for math, no evals, no vibes.

Why this pattern

• Determinism over vibes. The calculator is pure Python AST (whitelisted ops), so results are exact and testable.

• Controlled loop. First model step is forced to call the tool; the second cannot call tools and must answer. No infinite ping-pong.

• Observability. Streamed console output shows precisely when the LLM runs vs. when the tool runs, with token usage.

---

Architecture at a glance

┌──────────────┐     (tools_condition)      ┌────────────┐
│   agent LLM  │ ─────────────────────────► │  ToolNode  │
│ (forced 1st) │                            │ calculator │
└──────┬───────┘                            └─────┬──────┘
       │                                          │ ToolMessage
       │ no tool calls → END                      ▼
       │                                    ┌──────────────┐
       └─────────────────────────────────── │  agent LLM   │
                                            │ (free finish)│
                                            └──────┬───────┘
                                                   │ AiMessage (final)
                                                   ▼ END

Key primitives:

• MessagesState: appends messages, preserving the OpenAI tool-calling protocol.

• ToolNode: executes declared tools.

• tools_condition: routes to tools only when the assistant produced tool calls.

• One-tool-call policy: first LLM is bound to the calculator; the finisher LLM is tool-free.

---

The safe calculator (snippet)

Goal: evaluate math deterministically, with a tiny NL preprocessor.

# Whitelisted ops only
_ALLOWED_OPS = {ast.Add: op.add, ast.Sub: op.sub, ast.Mult: op.mul,
                ast.Div: op.truediv, ast.Pow: op.pow, ast.Mod: op.mod,
                ast.UAdd: op.pos, ast.USub: op.neg}

def _preprocess(s: str) -> str:
    s = s.strip().lower()
    s = re.sub(r"\b(please|thanks|thank you|what\s*is|what's|calculate|compute|equals?)\b", "", s)
    s = re.sub(r"\bthe\b", "", s)
    s = re.sub(r"\bplus\b", "+", s); s = re.sub(r"\bminus\b", "-", s)
    s = re.sub(r"\btimes\b", "*", s); s = re.sub(r"\bdivided by\b", "/", s)
    s = re.sub(r"\b(?:the\s+)?square root of\b", "sqrt(", s)
    s = re.sub(r"√\s*", "sqrt(", s)
    s = re.sub(r"sqrt\(\s*([0-9\.]+)\s*\)?", r"sqrt(\1)", s)
    s = re.sub(r"(\d+(?:\.\d+)?)\s*%\s*of\s*([0-9\.]+)", r"(\1/100*\2)", s)
    s = re.sub(r"(\d+(?:\.\d+)?)\s*%", r"(\1/100)", s)
    return s.replace("^", "**").strip()

def _eval_ast(node: ast.AST) -> float:
    if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
        return float(node.value)
    if isinstance(node, ast.UnaryOp) and type(node.op) in _ALLOWED_OPS:
        return _ALLOWED_OPS[type(node.op)](_eval_ast(node.operand))
    if isinstance(node, ast.BinOp) and type(node.op) in _ALLOWED_OPS:
        return _ALLOWED_OPS[type(node.op)](_eval_ast(node.left), _eval_ast(node.right))
    if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "sqrt" and len(node.args) == 1:
        return math.sqrt(_eval_ast(node.args[0]))
    if isinstance(node, ast.Expr):
        return _eval_ast(node.value)
    raise ValueError("Unsupported syntax.")

Tool wrapper:

@tool("calculator_tool")
def calculator_tool(expression: str) -> str:
    try:
        pre = _preprocess(expression)
        value = _eval_ast(ast.parse(pre, mode="eval").body)
        return str(float(value))
    except Exception as e:
        return f"Error: {e}"

Why this is safe: Only numeric literals and whitelisted operators/functions are allowed; no names, attrs, or imports can ever execute.

---

LangGraph wiring (snippet)

Bind two LLM phases: forced first (must call calculator) and free finish (no tools bound).

llm_forced_first = ChatOpenAI(...).bind_tools(
    [calculator_tool],
    tool_choice={"type":"function","function":{"name":"calculator_tool"}}
)
llm_free = ChatOpenAI(...)

def _has_calc_tool_call(msgs) -> bool:
    return any(isinstance(m, ToolMessage) and getattr(m,"name","")=="calculator_tool" for m in msgs)

def agent_node(state: MessagesState):
    first_phase = not _has_calc_tool_call(state["messages"])
    llm = llm_forced_first if first_phase else llm_free
    ai = llm.invoke([SystemMessage(content=SYSTEM_RULE), *state["messages"]],
                    config={"tags": ["agent_llm", "forced_first" if first_phase else "free_finish"]})
    return {"messages": [ai]}

tool_node = ToolNode([calculator_tool])

g = StateGraph(MessagesState)
g.add_node("agent", agent_node)
g.add_node("tools", tool_node)
g.add_conditional_edges("agent", tools_condition, {"tools": "tools", "__end__": END})
g.add_edge("tools", "agent")
g.set_entry_point("agent")
graph = g.compile()

Why it ends cleanly: After tools run, the graph goes back to the agent once to produce the final answer; tools_condition routes to END if no tool calls are present.

---

Observability: know exactly when the LLM is used

Add a tiny callback to print LLM start/end and token usage; gate with DEBUG_AGENT=1.

class ConsoleLLMHandler(BaseCallbackHandler):
    def on_llm_start(self, serialized, prompts, **kw):
        if os.getenv("DEBUG_AGENT")=="1":
            print(f"[LLM START] {serialized.get('name','LLM')} | tags={kw.get('tags',[])}", file=sys.stderr)
    def on_llm_end(self, result, **kw):
        if os.getenv("DEBUG_AGENT")=="1":
            print(f"[LLM END] tags={kw.get('tags',[])} | usage={(result.llm_output or {}).get('token_usage',{})}", file=sys.stderr)

handler = ConsoleLLMHandler()
llm_forced_first = ChatOpenAI(..., callbacks=[handler]).bind_tools(...)
llm_free = ChatOpenAI(..., callbacks=[handler])

Optional: DEBUG_CALC=1 prints NL → normalized math expression for the tool.

---

Running it

export OPENAI_API_KEY=sk-...
# telemetry on:
export DEBUG_AGENT=1
export DEBUG_CALC=1

python calculator_agent.py "Calculate 12% of 255 plus the square root of 244"

Sample (trimmed) output:

================================ Human Message =================================

Calculate 12% of 255 plus the square root of 244

[LLM START] ChatOpenAI | tags=['seq:step:1', 'agent_llm', 'forced_first']
[LLM END] tags=['seq:step:1', 'agent_llm', 'forced_first'] | usage={'completion_tokens': 14, 'prompt_tokens': 161, 'total_tokens': 175, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}
================================== Ai Message ==================================
Tool Calls:
  calculator_tool (call_134y7dqSU2udc2rr0OTNHKQs)
  Args:
    expression: 12% of 255 plus sqrt(244)

================================= Tool Message =================================
Name: calculator_tool

46.2204993518133

[LLM START] ChatOpenAI | tags=['seq:step:1', 'agent_llm', 'free_finish']
[LLM END] tags=['seq:step:1', 'agent_llm', 'free_finish'] | usage={'completion_tokens': 23, 'prompt_tokens': 121, 'total_tokens': 144, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}
================================== Ai Message ==================================

The result of 12% of 255 plus the square root of 244 is approximately 46.22.

---

Design choices: pros & cons

Pros

• Deterministic math; no LLM hallucinations for arithmetic.

• Single tool call guarantees predictable latency and flow.

• Great DX: streaming, token accounting, and NL conveniences (% of, √, word operators).

Cons

• Narrow by design (sqrt, basic ops). Extend via whitelist if you need log/exp/pi/e.

• Requires an LLM key for orchestration (the math itself is local).

• NL preprocessor is conservative to keep parsing predictable.

---

Code & assets

• Source: $\mathrm{<a\; href="https://github.com/JordiCorbilla/langgraph-cookbook">https://github.com/JordiCorbilla/langgraph-cookbook</a>}$

---

Snippet appendix

System rule (keeps the agent honest)

SYSTEM_RULE = (
  "You are a math assistant. For ANY numeric computation, "
  "call `calculator_tool` exactly once, then return the final answer. "
  "Do not call tools again after you have the numeric result."
)

Smoke test idea

# Expect: ≥2 AI messages (forced+final), exactly 1 ToolMessage, final AI has no tool_calls