LangSmith Upgrade and Integration#

Dosu now uses LangSmith version 0.4.59 for enhanced observability, tracing, and evaluation. This upgrade improves compatibility with OpenTelemetry (otel) and Pydantic AI, enabling more robust tracing and validation workflows for your AI and agent applications.

Key Benefits#

Improved OpenTelemetry Support:
- LangSmith 0.4.59 offers better integration with OpenTelemetry, allowing you to collect distributed traces and metrics for your workflows.
- To enable distributed tracing, install LangSmith with the otel extra and configure your preferred exporters.
Enhanced Pydantic AI Integration:
- The new version supports both Pydantic 1.x and 2.x, ensuring compatibility with modern validation patterns and Pydantic AI workflows.
- This enables seamless validation of LLM and agent outputs using Pydantic models.
Expanded Extras and Dependencies:
- Additional extras such as otel, pytest, and vcr are available for advanced tracing, testing, and integration scenarios.

How to Use LangSmith Tracing in Dosu#

LangSmith is integrated into Dosu to provide tracing and evaluation for LLM calls, agent workflows, and asynchronous events. You can leverage these features as follows:

Tracing Functions and Workflows#

Use the traceable decorator to enable tracing for your functions and workflows. This allows you to capture execution details, propagate context, and filter sensitive data.

from langsmith import traceable

@traceable(name="my_function", run_type="chain", exclude_params=["password"])
def my_function(...):
    ...

Validating Outputs with Pydantic#

Integrate Pydantic models for output validation in your LLM and agent calls. This ensures structured and validated responses, taking advantage of improved Pydantic AI support.

from pydantic import BaseModel

class OutputModel(BaseModel):
    ...

@traceable(name="llm_call", run_type="llm")
def llm_call(...):
    ...
    return OutputModel.parse_obj(response)

Enabling OpenTelemetry Tracing#

To enable distributed tracing with OpenTelemetry:

Install LangSmith with the otel extra:
```
pip install "langsmith[otel]"
```
Configure your OpenTelemetry exporters as needed. Refer to the LangSmith documentation for setup details.

Best Practices#

Use the traceable decorator for all critical functions and workflows to maximize observability.
Validate LLM and agent outputs with Pydantic models for robust data handling.
Review the LangSmith documentation for advanced tracing and evaluation patterns.

Additional Resources#

No migration steps are required for existing Dosu users. For advanced usage or troubleshooting, consult the official LangSmith changelog and guides.