Skip to main content
The replay URL (shown in the dashboard as Target URL (replay_url)) is the endpoint Kindred calls to re-run a session when you use the Test Reproducibility feature. You set it on the Reproducibility page and optionally save it per agent.

​
What the replay URL is

  • Reproducibility page — In the web app, go to Reproducibility (or open it from Behavior Search via Test Reproducibility for a session). The page URL can include sessionId and agentId: /reproducibility?sessionId=...&agentId=...
  • Target URL (replay_url) — A single field on that page where you enter the base URL of your agent’s replay endpoint (e.g. https://your-agent.example.com/replay or an ngrok URL). Kindred will send replay payloads to this endpoint when you click Run.
Your replay endpoint must accept POST requests with a JSON body that includes the original session’s messages or user input and an is_replay flag. Kindred runs that request multiple times (N runs) and compares results to check determinism.

​
How to get to the Reproducibility page

  1. From Behavior Search — Search for sessions, select a session, then use Test Reproducibility. This opens the Reproducibility page with sessionId (and usually agentId) pre-filled.
  2. Direct URL — Go to /reproducibility and optionally add query params: ?sessionId=...&agentId=...
  3. App nav — Use the main navigation link to Reproducibility and then paste or choose a session ID (and agent ID if needed).

​
Set and save the Target URL (replay_url)

  1. On the Reproducibility page, ensure a session is selected (and agent ID is set or derived).
  2. In the Target URL (replay_url) field, enter the full base URL of your replay endpoint, e.g.:
    • https://your-agent.example.com/replay
    • https://abc123.ngrok.io (if your server listens on / or /replay)
  3. Click Save to store this URL for the current agent so it’s pre-filled next time.
If you don’t set a target URL, the dashboard will warn you and Run will be disabled until a URL is entered.

​
Run reproducibility

  1. Set Target URL (replay_url) as above (and save if desired).
  2. Choose N runs (e.g. 1–20). Kindred will call your replay endpoint N times with the same payload.
  3. Click Run. Kindred sends the replay payload to your endpoint N times and compares responses.
  4. Review the results table for pass/fail and any divergence between runs.

​
Replay endpoint contract

Your server should accept POST requests (e.g. at / or /replay) with a JSON body. Two common shapes:
  • LLM path — { "messages": [...], "model": "gpt-4o-mini", "is_replay": true }
  • User path — { "input": "user message", "is_replay": true }
The exact schema may depend on your agent; Kindred sends the same structure it derives from the original session. Your endpoint should run the agent (or LLM call) and return the response so Kindred can compare across runs.

​
Example: replay test server

The Kindred repo includes a small replay-server that accepts replay payloads and forwards them to OpenAI. You can use it to try reproducibility locally:
  1. Start the replay server (e.g. on port 5001) and set OPENAI_API_KEY.
  2. Expose it with a tunnel (e.g. ngrok http 5001).
  3. In the Reproducibility page, set Target URL (replay_url) to the ngrok URL (e.g. https://abc123.ngrok.io).
  4. Select a session that has replay context and click Run.
That server accepts POST / or POST /replay with the replay payload and returns the model response so Kindred can compare N runs.

​
When a session is not replayable

If the session has no replay context (e.g. no user or LLM request to replay, or logs not found for your account), the UI will show that the session is not replayable and Run will be disabled. Use a session that was recorded with the tracer and that has the required context.

​
Next steps