LLM Integration#

The cyber-doctor project integrates with large language models through a factory pattern that routes requests to two distinct client families:

1. OpenAI-compatible clients (OurAPI / LLMclientgeneric) — for all text-based tasks
2. ZhipuAI SDK clients — for multimodal tasks (image generation, image description, video generation)

The entry point for all client construction is Clientfactory.

Client Architecture#

Clientfactory
├── get_client() → OurAPI (text generation, RAG, KG, search, PPT, Docx)
└── get_special_client() → ZhipuAI clients (image/video tasks)

LLMclientbase 1 defines the abstract interface. It initializes an openai.OpenAI instance using three environment variables 2:

LLMclientgeneric 3 provides the concrete implementation with three callable methods:

• chat_with_ai(prompt) — single-turn, non-streaming; used for structured generation (PPT, Docx)
• chat_with_ai_stream(prompt, history) — multi-turn streaming; primary method for interactive chat
• chat_using_messages(messages) — accepts a pre-built message list; used when callers perform their own prompt construction

All text calls use top_p=0.7, temperature=0.95, max_tokens=1024 4.

OurAPI 5 is a thin subclass of LLMclientgeneric with no additional logic — it exists to allow provider-specific overrides in future.

ZhipuAI Multimodal Clients#

Three ZhipuAI client instances are module-level singletons defined in client/zhipuAPI/client.py, each backed by a separate API key env var:

Clientfactory.get_special_client(client_type) selects the right instance based on userPurposeType 6.

How Each Modality Is Called#

All multimodal API calls are in qa/function_tool.py:

• Image generation — client.images.generations(model=..., prompt=...)
• Image description — client.chat.completions.create() with a multimodal message payload (text + image_url)
• Video generation — async two-step: client.videos.generations() returns a job ID; the caller polls client.videos.retrieve_videos_result(id=...) until task_status is complete, then reads video_result[0].url

Routing by User Intent#

userPurposeType is the central enum mapping user intent to the correct client and handler. Text-centric types (text, RAG, KnowledgeGraph, PPT, Docx, InternetSearch, DeepResearch) all go through Clientfactory().get_client() → OurAPI. Multimodal types (ImageGeneration, ImageDescribe, Video) go through Clientfactory.get_special_client() → the relevant ZhipuAI singleton.

Key Files#