Lesson 1 of 136

Checking lesson access...

Video and written lesson · 48 min

The Whole System in One Hour

One model call, one tool and one trace, end to end on the starter repository.

PreviewCaptionsSoon

You are viewing a free preview lesson.

Situation

You have just become the owner of the Order Support Assistant, which answers an online shop's customers about their orders. Today it does one thing: a customer asks where an order is, a language model asks for an order lookup, the application performs it, and the model writes the answer. Before the foundation lessons explain each piece, you run the whole path once and read its trace.

Concept

The assistant has four parts, and the course keeps them apart from the first line of code:

  • The application is ordinary Java. It holds the customer's identity, the order data and every rule about who may see what.
  • The model (gemma4:e2b, running locally in Ollama) reads text and proposes an answer or a tool call: a request to run one named operation with arguments.
  • The tool is application code the model may ask for: here get_order_status, which reads one order.
  • The trace records each model call with its tokens, each tool call with its arguments and outcome, and how long each took.

The split carries the principle the course rests on: the model proposes, the application decides. The model can ask for any order number; whether the customer may see it is decided in the tool's Java code, with the customer id the application already holds, never one the model supplies. Model APIs and frameworks will change; that split will not.

Mechanism

One question takes at most two model calls and one round of tool calls:

Drawing diagram…

Each call to the model is a model.call span and the tool run a tool.call span, all under one assistant.request span. If the model answers without a tool, there is no second call.

Decision Point

The shop has three order operations: read an order's status, cancel an order, and change its delivery address. Which ones should the model be able to call today?

Worked Decision

Options: (a) all three, so the assistant is useful at once; (b) only the status lookup; (c) none, answering from a database query the application runs before calling the model.

We choose (b). A status lookup is read-only: a wrong call gives a wrong or empty answer, which is recoverable. Cancelling is a side effect: a wrong call loses a sale, and a retry could cancel twice; it needs validation, idempotency and perhaps a person's approval first (Modules 3 and 6). Option (c) is cheaper and fully deterministic, and right if every question names an order number; Module 2 asks that question properly. Rejected: (a), because it gives the assistant authority we cannot yet limit.

Your Decision

Apply the same reasoning to "change the delivery address". Should the model get that tool now, later, or never? Name what would have to be true first.

Implementation

You need Java 21, Docker and Ollama. Pull the model once (about 4.6 GB) and start the local trace viewer, Jaeger, from the project folder:

bash
ollama pull gemma4:e2b
docker compose up -d

The Project

The starter project is a plain Java application built with Gradle (Spring Boot arrives in Module 6); the wrapper's properties pin the Gradle version:

<!-- file: settings.gradle -->
groovy
rootProject.name = 'order-support-assistant'
<!-- file: gradle/wrapper/gradle-wrapper.properties -->
properties
distributionUrl=https\://services.gradle.org/distributions/gradle-9.8.2-bin.zip
<!-- file: build.gradle -->
groovy
plugins {
    id 'application'
}

group = 'com.tarkev'
version = '0.0.1-SNAPSHOT'

java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(21)
    }
}

repositories {
    mavenCentral()
}

dependencies {
    implementation platform('io.opentelemetry:opentelemetry-bom:1.67.0')
    implementation 'io.opentelemetry:opentelemetry-sdk'
    implementation 'io.opentelemetry:opentelemetry-exporter-otlp'
    implementation 'tools.jackson.core:jackson-databind:3.2.4'
    testImplementation platform('org.junit:junit-bom:6.1.3')
    testImplementation 'org.junit.jupiter:junit-jupiter'
    testImplementation 'io.opentelemetry:opentelemetry-sdk-testing'
    testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}

application {
    mainClass = providers.gradleProperty('main').orElse('com.tarkev.ordersupport.Main')
}

tasks.named('test') {
    useJUnitPlatform()
}
<!-- file: compose.yaml -->
yaml
services:
  jaeger:
    image: jaegertracing/jaeger:2.22.0
    ports:
      - "16686:16686"
      - "4318:4318"

Jaeger receives traces on port 4318 and shows them at http://localhost:16686. ./gradlew run starts Main; -Pmain=<class> runs another class, which later lessons use for experiments.

Orders and the Tool

The order store is an in-memory stand-in for the shop's order service, with two customers:

java
package com.tarkev.ordersupport.orders;

import java.time.LocalDate;

public record Order(String id, String customerId, String status, LocalDate expectedDelivery) {
}
java
package com.tarkev.ordersupport.orders;

import java.time.LocalDate;
import java.util.Map;
import java.util.Optional;

public final class InMemoryOrders {
    private final Map<String, Order> orders = Map.of(
            "1042", new Order("1042", "C-100", "SHIPPED", LocalDate.of(2026, 10, 27)),
            "1043", new Order("1043", "C-100", "PROCESSING", LocalDate.of(2026, 11, 3)),
            "2001", new Order("2001", "C-200", "DELIVERED", LocalDate.of(2026, 10, 20)));

    public Optional<Order> find(String orderId) {
        return Optional.ofNullable(orders.get(orderId));
    }
}

A tool has two halves: a specification the model reads (name, description, a JSON schema for the arguments) and the code the application runs. The customer id comes from the caller, not from the model's arguments, and an order that belongs to someone else gets the same reply as one that does not exist, so the model cannot learn that it exists:

java
package com.tarkev.ordersupport.tools;

import java.util.Map;

public record ToolSpec(String name, String description, Map<String, Object> parameters) {
}
java
package com.tarkev.ordersupport.tools;

import com.tarkev.ordersupport.orders.InMemoryOrders;
import java.util.List;
import java.util.Map;

public final class OrderStatusTool {
    public static final ToolSpec SPEC = new ToolSpec("get_order_status",
            "Look up the status and expected delivery date of one of the customer's orders.",
            Map.of("type", "object",
                    "properties", Map.of("orderId", Map.of("type", "string")),
                    "required", List.of("orderId")));

    public record Result(String outcome, String text) {
    }

    private final InMemoryOrders orders;

    public OrderStatusTool(InMemoryOrders orders) {
        this.orders = orders;
    }

    public Result execute(String customerId, Map<String, Object> arguments) {
        String orderId = String.valueOf(arguments.get("orderId"));
        var order = orders.find(orderId);
        if (order.isEmpty() || !order.get().customerId().equals(customerId)) {
            String outcome = order.isEmpty() ? "not_found" : "not_authorized";
            return new Result(outcome, "No order " + orderId + " was found for this customer.");
        }
        return new Result("found", "Order %s: status %s, expected delivery %s."
                .formatted(orderId, order.get().status(), order.get().expectedDelivery()));
    }
}

The Model Client

The application reaches the model through a small interface, so tests can script it:

java
package com.tarkev.ordersupport.model;

import java.util.Map;

public record ToolCall(String name, Map<String, Object> arguments) {
}
java
package com.tarkev.ordersupport.model;

import java.util.List;

public record Message(String role, String content, List<ToolCall> toolCalls, String toolName) {
    public static Message system(String text) {
        return new Message("system", text, List.of(), null);
    }

    public static Message user(String text) {
        return new Message("user", text, List.of(), null);
    }

    public static Message toolResult(String toolName, String text) {
        return new Message("tool", text, List.of(), toolName);
    }
}
java
package com.tarkev.ordersupport.model;

import com.tarkev.ordersupport.tools.ToolSpec;
import java.util.List;

public interface ModelClient {
    record Reply(Message message, String model, int promptTokens, int outputTokens) {
    }

    Reply chat(List<Message> messages, List<ToolSpec> tools);
}

The Ollama implementation sends one HTTP request to /api/chat. "think": false turns off the model's hidden reasoning, which would add tokens to every call, and temperature 0 makes runs easier to compare. Module 2 takes the request apart:

java
package com.tarkev.ordersupport.model;

import com.tarkev.ordersupport.tools.ToolSpec;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.json.JsonMapper;

public final class OllamaModelClient implements ModelClient {
    private static final TypeReference<Map<String, Object>> ARGUMENTS = new TypeReference<>() {
    };
    private final HttpClient http = HttpClient.newHttpClient();
    private final JsonMapper json = JsonMapper.builder().build();
    private final URI chatUri;
    private final String model;

    public OllamaModelClient(String baseUrl, String model) {
        this.chatUri = URI.create(baseUrl + "/api/chat");
        this.model = model;
    }

    @Override
    public Reply chat(List<Message> messages, List<ToolSpec> tools) {
        var body = Map.of("model", model, "stream", false, "think", false,
                "options", Map.of("temperature", 0),
                "messages", messages.stream().map(OllamaModelClient::toOllama).toList(),
                "tools", tools.stream().map(tool -> Map.of("type", "function", "function", tool)).toList());
        var request = HttpRequest.newBuilder(chatUri)
                .header("Content-Type", "application/json")
                .timeout(Duration.ofSeconds(120))
                .POST(HttpRequest.BodyPublishers.ofString(json.writeValueAsString(body)))
                .build();
        JsonNode reply = json.readTree(send(request));
        JsonNode message = reply.path("message");
        var calls = new ArrayList<ToolCall>();
        for (JsonNode call : message.path("tool_calls")) {
            JsonNode function = call.path("function");
            calls.add(new ToolCall(function.path("name").asString(),
                    json.convertValue(function.path("arguments"), ARGUMENTS)));
        }
        return new Reply(new Message("assistant", message.path("content").asString(), calls, null),
                model, reply.path("prompt_eval_count").asInt(), reply.path("eval_count").asInt());
    }

    private String send(HttpRequest request) {
        try {
            var response = http.send(request, HttpResponse.BodyHandlers.ofString());
            if (response.statusCode() != 200) {
                throw new IOException("Ollama returned " + response.statusCode() + ": " + response.body());
            }
            return response.body();
        } catch (IOException e) {
            throw new UncheckedIOException("Model call to " + chatUri + " failed", e);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new IllegalStateException("Interrupted while calling the model", e);
        }
    }

    private static Map<String, Object> toOllama(Message message) {
        var out = new HashMap<String, Object>(Map.of("role", message.role(), "content", message.content()));
        if (message.toolName() != null) {
            out.put("tool_name", message.toolName());
        }
        if (!message.toolCalls().isEmpty()) {
            out.put("tool_calls", message.toolCalls().stream().map(call -> Map.of("function",
                    Map.of("name", call.name(), "arguments", call.arguments()))).toList());
        }
        return out;
    }
}

The Assistant and Its Trace

The assistant puts the parts together. It sends the instructions, the question and the tool list; if the model asks for tools, it runs them and calls the model once more. Every step is a span with the attributes you will read in Jaeger:

java
package com.tarkev.ordersupport;

import com.tarkev.ordersupport.model.Message;
import com.tarkev.ordersupport.model.ModelClient;
import com.tarkev.ordersupport.model.ToolCall;
import com.tarkev.ordersupport.tools.OrderStatusTool;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.context.Scope;
import java.util.ArrayList;
import java.util.List;

public final class OrderSupportAssistant {
    static final String INSTRUCTIONS = """
            You answer one customer's questions about their orders in an online shop.
            Use the get_order_status tool to look up an order. Never guess a status or a date.
            If the question is not about an order's status, say you can only help with order status.
            """;

    public record Answer(String text, String traceId) {
    }

    private final ModelClient model;
    private final OrderStatusTool orderStatus;
    private final Tracer tracer;

    public OrderSupportAssistant(ModelClient model, OrderStatusTool orderStatus, Tracer tracer) {
        this.model = model;
        this.orderStatus = orderStatus;
        this.tracer = tracer;
    }

    public Answer answer(String customerId, String question) {
        Span span = tracer.spanBuilder("assistant.request").setAttribute("customer.id", customerId).startSpan();
        try (Scope ignored = span.makeCurrent()) {
            var messages = new ArrayList<>(List.of(Message.system(INSTRUCTIONS), Message.user(question)));
            ModelClient.Reply reply = callModel(messages);
            if (!reply.message().toolCalls().isEmpty()) {
                messages.add(reply.message());
                for (ToolCall call : reply.message().toolCalls()) {
                    messages.add(Message.toolResult(call.name(), runTool(customerId, call)));
                }
                reply = callModel(messages);
            }
            return new Answer(reply.message().content(), span.getSpanContext().getTraceId());
        } finally {
            span.end();
        }
    }

    private ModelClient.Reply callModel(List<Message> messages) {
        Span span = tracer.spanBuilder("model.call").startSpan();
        try (Scope ignored = span.makeCurrent()) {
            ModelClient.Reply reply = model.chat(messages, List.of(OrderStatusTool.SPEC));
            span.setAttribute("model.name", reply.model());
            span.setAttribute("model.prompt_tokens", reply.promptTokens());
            span.setAttribute("model.output_tokens", reply.outputTokens());
            span.setAttribute("model.tool_calls", reply.message().toolCalls().size());
            return reply;
        } catch (RuntimeException e) {
            span.recordException(e);
            span.setStatus(StatusCode.ERROR);
            throw e;
        } finally {
            span.end();
        }
    }

    private String runTool(String customerId, ToolCall call) {
        Span span = tracer.spanBuilder("tool.call").setAttribute("tool.name", call.name())
                .setAttribute("tool.arguments", call.arguments().toString()).startSpan();
        try (Scope ignored = span.makeCurrent()) {
            if (!call.name().equals(OrderStatusTool.SPEC.name())) {
                span.setAttribute("tool.outcome", "unknown_tool");
                return "There is no tool named " + call.name() + ".";
            }
            OrderStatusTool.Result result = orderStatus.execute(customerId, call.arguments());
            span.setAttribute("tool.outcome", result.outcome());
            return result.text();
        } finally {
            span.end();
        }
    }
}

Tracing is set up once, sending spans to Jaeger over OTLP. Main acts for customer C-100, as if that customer had logged in:

java
package com.tarkev.ordersupport;

import io.opentelemetry.exporter.otlp.http.trace.OtlpHttpSpanExporter;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.export.BatchSpanProcessor;

public final class Tracing {
    private Tracing() {
    }

    public static OpenTelemetrySdk start(String serviceName) {
        var exporter = OtlpHttpSpanExporter.builder()
                .setEndpoint(System.getenv().getOrDefault("OTLP_TRACES_ENDPOINT",
                        "http://localhost:4318/v1/traces"))
                .build();
        var tracerProvider = SdkTracerProvider.builder()
                .setResource(Resource.getDefault().toBuilder().put("service.name", serviceName).build())
                .addSpanProcessor(BatchSpanProcessor.builder(exporter).build())
                .build();
        return OpenTelemetrySdk.builder().setTracerProvider(tracerProvider).build();
    }
}
java
package com.tarkev.ordersupport;

import com.tarkev.ordersupport.model.OllamaModelClient;
import com.tarkev.ordersupport.orders.InMemoryOrders;
import com.tarkev.ordersupport.tools.OrderStatusTool;

public final class Main {
    public static void main(String[] args) {
        try (var openTelemetry = Tracing.start("order-support-assistant")) {
            var model = new OllamaModelClient(env("OLLAMA_BASE_URL", "http://localhost:11434"),
                    env("OLLAMA_MODEL", "gemma4:e2b"));
            var assistant = new OrderSupportAssistant(model, new OrderStatusTool(new InMemoryOrders()),
                    openTelemetry.getTracer("order-support"));
            var answer = assistant.answer("C-100", String.join(" ", args));
            System.out.println(answer.text());
            System.out.println("trace " + answer.traceId());
        }
    }

    static String env(String name, String fallback) {
        String value = System.getenv(name);
        return value == null || value.isBlank() ? fallback : value;
    }
}

Running It

bash
./gradlew run --args="Where is my order 1042?"
<!-- captured: ollama 0.33.2, gemma4:e2b, 2026-10-24 -->
text
Your order 1042 has been **SHIPPED** and the expected delivery date is **2026-10-27**.
trace 028c7bbd30419080399e3157a90120d9

Open http://localhost:16686, choose the service order-support-assistant and open the trace with that id. Jaeger draws it as a timeline; these are its spans and the attributes the code set:

<!-- captured: ollama 0.33.2, gemma4:e2b, 2026-10-24 -->
text
assistant.request    7796 ms  customer.id=C-100
  model.call         7089 ms  model.name=gemma4:e2b  model.prompt_tokens=142  model.output_tokens=21  model.tool_calls=1
  tool.call             1 ms  tool.name=get_order_status  tool.arguments={orderId=1042}  tool.outcome=found
  model.call          701 ms  model.name=gemma4:e2b  model.prompt_tokens=203  model.output_tokens=33  model.tool_calls=0

The first model call took seven seconds because Ollama was loading the model; once loaded, the same call takes about 0.6 seconds, so a first run is no measure of latency. The model asked for one tool call with orderId 1042, the tool found the order, and the second call, 61 prompt tokens longer because it carries the tool result, wrote the answer. The tool took one millisecond: the time and the tokens are in the model calls. The ** in the answer is Markdown, which the shop's chat window may not render: a question for Module 2.

The Seven Artifacts

Owning the system means keeping, beside the code, the documents a real team keeps. The starter project ships them as templates under docs/ and one evaluation case under evals/; each is started in the lesson named below and updated from then on:

ArtifactFileStarted in
Architecture docdocs/architecture.md2.10
ADR logdocs/adr/2.10
Failure-mode and threat registerdocs/failure-register.md3.06
Runbookdocs/runbook.mdModule 6
Evaluation setevals/cases/3.08
Cost and latency ledgerdocs/cost-ledger.csv2.10
Postmortemsdocs/postmortems/8.09
<!-- file: docs/architecture.md -->
markdown
# Order Support Assistant: Architecture

## Components
## Model and application boundary
## State
## Data flows
## Capacity notes
<!-- file: docs/adr/0000-template.md -->
markdown
# ADR NNNN: Title

Status: proposed | accepted | superseded by NNNN · Date:

## Context
## Options
## Decision
## Consequences
## Rejected alternative and why
<!-- file: docs/failure-register.md -->
markdown
# Failure-Mode and Threat Register

| Failure or threat | Detection (trace field, metric, log) | Limit | Recovery | Since |
| --- | --- | --- | --- | --- |
<!-- file: docs/runbook.md -->
markdown
# Runbook

## How to tell the assistant is unhealthy
## What to check first
## What to roll back, and how
## Who approves what
<!-- file: docs/postmortems/template.md -->
markdown
# Postmortem: Title

Date: · Trace ids:

## What happened
## Timeline (from traces)
## Root cause (operational or behavioral)
## What we changed (code or configuration)
## Added to the eval set or runbook
<!-- file: docs/cost-ledger.csv -->
text
date,request_type,model,model_calls,tool_calls,prompt_tokens,output_tokens,duration_ms,notes

An evaluation case states an input and what a correct run does. This one is today's question; Module 3 writes a dozen more and Module 11 runs them:

<!-- file: evals/cases/order-status-1042.json -->
json
{
  "id": "order-status-1042",
  "customerId": "C-100",
  "input": "Where is my order 1042?",
  "expect": {
    "toolCalls": [{ "name": "get_order_status", "arguments": { "orderId": "1042" } }],
    "answerMentions": ["shipped", "2026-10-27"]
  }
}

A Test That Needs No Model

The test scripts the model and records spans in memory, so it runs in milliseconds with the same result every time. It also checks the authority rule: customer C-100 learns nothing about order 2001, whatever the model asks for:

java
package com.tarkev.ordersupport;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;

import com.tarkev.ordersupport.model.Message;
import com.tarkev.ordersupport.model.ModelClient;
import com.tarkev.ordersupport.model.ToolCall;
import com.tarkev.ordersupport.orders.InMemoryOrders;
import com.tarkev.ordersupport.tools.OrderStatusTool;
import com.tarkev.ordersupport.tools.ToolSpec;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.sdk.testing.exporter.InMemorySpanExporter;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.data.SpanData;
import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;

class OrderSupportAssistantTest {
    private final InMemorySpanExporter spans = InMemorySpanExporter.create();
    private final ScriptedModel model = new ScriptedModel();
    private final OrderSupportAssistant assistant = new OrderSupportAssistant(model,
            new OrderStatusTool(new InMemoryOrders()),
            SdkTracerProvider.builder().addSpanProcessor(SimpleSpanProcessor.create(spans)).build().get("test"));

    @Test
    void looksUpTheOrderThroughTheTool() {
        model.replies.add(toolCall("1042"));
        model.replies.add(text("Your order 1042 has shipped."));

        var answer = assistant.answer("C-100", "Where is my order 1042?");

        assertEquals("Your order 1042 has shipped.", answer.text());
        assertEquals("Order 1042: status SHIPPED, expected delivery 2026-10-27.", model.lastToolResult());
        assertEquals(List.of("model.call", "tool.call", "model.call", "assistant.request"),
                spans.getFinishedSpanItems().stream().map(SpanData::getName).toList());
    }

    @Test
    void neverShowsAnotherCustomersOrder() {
        model.replies.add(toolCall("2001"));
        model.replies.add(text("I could not find that order."));

        assistant.answer("C-100", "Where is order 2001?");

        assertFalse(model.lastToolResult().contains("DELIVERED"));
        assertEquals("not_authorized", spans.getFinishedSpanItems().get(1).getAttributes()
                .get(AttributeKey.stringKey("tool.outcome")));
    }

    private static ModelClient.Reply toolCall(String orderId) {
        var call = new ToolCall("get_order_status", Map.of("orderId", orderId));
        return new ModelClient.Reply(new Message("assistant", "", List.of(call), null), "scripted", 0, 0);
    }

    private static ModelClient.Reply text(String content) {
        return new ModelClient.Reply(new Message("assistant", content, List.of(), null), "scripted", 0, 0);
    }

    static final class ScriptedModel implements ModelClient {
        final ArrayDeque<Reply> replies = new ArrayDeque<>();
        final List<List<Message>> received = new ArrayList<>();

        @Override
        public Reply chat(List<Message> messages, List<ToolSpec> tools) {
            received.add(List.copyOf(messages));
            return replies.removeFirst();
        }

        String lastToolResult() {
            return received.getLast().getLast().content();
        }
    }
}
bash
./gradlew test

What Can Fail, and How We Would Know

FailureDetectionLimitRecovery
Ollama is not running or the model is not pulledmodel.call span with status ERROR and a ConnectException (or HTTP 404) event; no answer printed120-second request timeoutStart Ollama, ollama pull gemma4:e2b; Module 6 adds a fallback answer
The model answers without calling the tool and invents a statusOne model.call with model.tool_calls = 0 and no tool.call span, on a question that names an orderNone yet: instructions only ask it not to guessModule 2 validates output; Module 11 adds eval cases that catch it
The model asks for another customer's ordertool.call with tool.outcome = not_authorizedEnforced in OrderStatusTool, not in the promptNothing to recover: the tool returned no data
The model asks for a tool that does not existtool.call with tool.outcome = unknown_toolThe application only runs tools it knowsThe model gets an error text and answers once more
Jaeger is downA warning from the OTLP exporter in the console; the answer still printsTracing never blocks an answerdocker compose up -d; the lost trace is gone

Owning It

You now own this system, and owning it starts with being able to show what it did. Your first duty is a habit: every wrong answer you report comes with its trace id. The trace shows the model's tool calls, their arguments and outcomes, and each call's tokens, so nobody argues from memory about what the model "probably" did. From Module 6 this is the runbook's first rule.

Exercise

Model answers: lesson 1.02.

Design

Write your decision on the delivery-address tool in five or six sentences: the options, your choice, what must be true before the model gets it, and the alternative you rejected. Then add two failures that tool could bring to the table above, each with the span attribute that would reveal it.

Build

  1. Run ./gradlew test and then the assistant with three questions: "Where is my order 1042?", "Where is order 2001?" and "Can I return a jacket?".
  2. For each, find the trace in Jaeger and write down: how many model.call spans it has, whether a tool.call appears and with which tool.outcome, and the prompt and output tokens of each call.
  3. One of the three questions should not need the tool. Did the model call it anyway? What would you change, and in which part of the system: the instructions, the tool description or the Java code?

Check Yourself

Question 1

The model asks get_order_status for order 2001 on behalf of customer C-100. Why does the tool reply exactly as it would for an order that does not exist?

Question 2

A question that names an order produces a trace with one model.call and no tool.call. What has probably happened, and why can the current system not stop it?

Question 3

Why does the test use a scripted model instead of gemma4:e2b, and what does it still not tell you about the real system?

Think first

Coming soon

Checkpoint questions that ask you to commit to an answer before the lesson shows it.