Trace Forwarding

Configure OpenTelemetry (OTEL) endpoints to stream OTLP trace data from Elementum to your observability and AI evaluation platforms.

Overview

Trace Forwarding lets your organization push OpenTelemetry (OTEL) trace data from Elementum environments to external platforms — observability tools, AI evaluation suites, or any system that accepts the OpenTelemetry Protocol (OTLP) over gRPC or HTTP. You can configure multiple endpoints per channel to route traces to different teams or platforms in parallel.

OTEL vs. OTLP. OpenTelemetry (OTEL) is the open standard for generating and collecting trace, metric, and log data. The OpenTelemetry Protocol (OTLP) is the wire format OTEL uses to transmit that data over gRPC or HTTP. Elementum produces OTEL trace data and exports it using OTLP, so your endpoints must accept OTLP.

Go to Organization Settings > Platform > Trace Forwarding. Only Organization Administrators can view and modify these settings.

Trace channels

Two independent channels are available:

Channel	What it forwards	Default protocol	Example destinations
General Traces	Operational telemetry from all Elementum activities	gRPC	Datadog, Grafana, Jaeger, Honeycomb
GenAI Traces	Detailed LLM and agent spans from AI-powered operations	HTTP	Weave (W&B), LangSmith, Arize

Quick start: add your first endpoint

1. Go to Organization Settings > Platform > Trace Forwarding.
2. Click Add General Endpoint or Add GenAI Endpoint depending on the type of traces you want to forward.
3. Enter a Name, your Endpoint URL, and confirm the Protocol matches what your destination platform expects.
4. Select an Authorization Type and fill in the required credentials.
5. Optionally, add Resource Attributes to tag every span with metadata such as environment or team.
6. Click Save & Test. A green Message delivered result confirms the endpoint is reachable.
7. Click Enable to activate the endpoint.
8. Go to Organization Settings > Platform > Environments, click Configure on an environment card, then click Edit Environment, select your endpoint under Trace Forwarding, and click Save Changes.

Add or edit an endpoint

Click Add General Endpoint or Add GenAI Endpoint to open the endpoint dialog. To edit an existing endpoint, click the pencil icon in its row. Both actions open the same dialog.

Connection

Field	Required	Details
Name	Yes	A human-readable label used to identify the endpoint in the list.
Endpoint URL	Yes	The full URL of your OTLP receiver. Must be a valid HTTP or HTTPS URL.
Protocol	Yes	gRPC (binary, efficient) or HTTP (REST-based). Defaults to gRPC for General Traces and HTTP for GenAI Traces.

Authentication

Select one of four options from the Authorization Type dropdown:

Type	Behavior
None	No authentication headers are sent. Use only for internal or open endpoints.
Bearer Token	Sends an Authorization: <scheme> <token> header. Requires a Bearer Token; Bearer Scheme is optional and defaults to Bearer.
Basic Auth	Sends an Authorization: Basic <base64> header. Requires Username and Password.
Custom Headers	Sends one or more arbitrary request headers. Add as many Header Key / Header Value pairs as needed.

Secret values (tokens, passwords, header values) are encrypted at rest. When you reopen an endpoint for editing, existing secrets are masked as ****. Click Overwrite to replace a secret, or leave it masked to keep the current value.

Optional fields

• Additional headers — Extra HTTP headers attached to every trace export request beyond those required for authentication. Click + Add Header to add key-value pairs.
• Resource attributes — Key-value pairs added to the OTLP resource on every exported span. Use these to tag trace data with environment, team, or deployment metadata that your external platform can filter on (e.g. environment = production, team = ai-platform).

Click Save & Test to save the endpoint and verify connectivity immediately. General Trace endpoints also offer a Save option that skips the connection test. GenAI endpoints require Save & Test — connectivity must be verified before the endpoint is saved.

Manage endpoints

Test a connection

A connection test sends a sample OTLP trace to the configured endpoint. You can run a test two ways:

• Click Save & Test when creating or editing an endpoint.
• Click the send icon (Test Connection) in the endpoint list row.

The test times out after 30 seconds. A result modal shows latency, HTTP status code, trace ID, and span ID on success, or an error message and guidance on failure. From the modal you can Edit Connection, toggle Enable / Disable, or Cancel to close.

Assign to an environment

After configuring and enabling an endpoint, assign it to one or more environments. Traces are only forwarded from environments where an endpoint is assigned.

1. Go to Organization Settings > Platform > Environments.
2. Find the environment you want to forward traces from and click Configure on its card, then click Edit Environment.
3. In the Trace Forwarding section, select your configured endpoint from the list.
4. Click Save Changes.

Repeat for each environment you want to forward traces from.

Delete an endpoint

1. Click the trash icon in the endpoint list row.
2. Confirm the deletion in the dialog that appears.

Continuing an existing trace

Include a W3C traceparent header on a request to signal trace propagation for the traces Elementum exports. The exported spans are tied back to the information you provide, placed under the same trace using the span ID you supply as their parent, so in your platform the request appears as a single connected trace with Elementum’s spans nested under your calling span. This requires a Trace Forwarding endpoint that is enabled and assigned to the environment; without one, there is nowhere to export the spans. If no traceparent is sent, the exported spans are not tied to an external trace.

Add the header

Set the traceparent header on the request your system sends to Elementum. If your calling application is instrumented with OpenTelemetry, its HTTP client adds this header automatically. Otherwise, add it manually, populated from the calling span in your tracing system.

traceparent: 00-<trace-id>-<parent-span-id>-01

Field	Length	Value
trace-id	32 hex	The overall trace ID. Elementum’s spans adopt this.
parent-span-id	16 hex	The span in your system that Elementum’s spans attach under.

The leading 00 (version) and trailing 01 (flags) are fixed parts of the W3C format; you normally leave them as shown, and if your caller uses OpenTelemetry they are set automatically.

Example: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

You may also send a tracestate header; Elementum preserves it.

Verify

Open the trace by the trace-id you sent. You should see your originating span with Elementum’s spans nested beneath it as one connected trace.

Propagation is opt-in per request. Without a traceparent, exported spans are not tied to an external trace.

Troubleshooting

Test returns 401 or 403

The credentials are invalid or expired. Re-enter the token or password and ensure the API key has the correct scopes.

Test returns 404

The endpoint URL is likely incorrect. Double-check the URL and, for HTTP endpoints, confirm the path includes /v1/traces or the equivalent path required by your platform.

Test times out after 30 seconds

A network or firewall issue is preventing the connection. Verify the endpoint host is reachable from the Elementum backend and check any applicable firewall rules.

Test passes but no data appears in the platform

Check that the endpoint is Enabled and assigned to the environment you are testing from. A passing test only confirms connectivity, not that traces are being produced — trigger an AI operation in Elementum to generate spans.

GenAI attribute reference

View OpenTelemetry semantic conventions used in Elementum AI spans

LLM call spans

These attributes appear on spans representing a single call to a language model. Span names follow the pattern chat <model>.

Attribute	Type	Description
gen_ai.system	string	The LLM provider (e.g. aws.bedrock, openai, anthropic).
gen_ai.request.model	string	The model name that was requested.
gen_ai.response.model	string	The model name that actually responded.
gen_ai.usage.input_tokens	integer	Number of tokens in the prompt.
gen_ai.usage.output_tokens	integer	Number of tokens in the completion.
gen_ai.prompt	string (JSON)	Input messages as a JSON array of {role, content} objects.
gen_ai.completion	string (JSON)	Output messages as a JSON array of {role, content} objects.
gen_ai.operation.name	string	Always "chat" for LLM call spans.
gen_ai.response.finish_reasons	string[]	Why the model stopped generating (e.g. ["stop"]).
gen_ai.response.id	string	The provider’s unique response identifier.

Tool execution spans

These attributes appear on spans representing an agent executing a tool. Span names follow the pattern execute_tool <tool_name>.

Attribute	Type	Description
gen_ai.tool.name	string	The function name of the tool that was called.
gen_ai.tool.call.id	string	Links this execution back to the LLM’s tool call request.
gen_ai.operation.name	string	Always "execute_tool" for tool execution spans.

Span tree structure

Traces from Elementum AI operations are organized into a span hierarchy:

Interaction type	Span structure
Single LLM call	root → chat
Tool-calling agent (one round)	root → chat → tool → chat
Multi-step agent (multiple rounds)	root → chat → tool → chat → tool → chat → …