Xiaohongshu URL Handling

Xiaohongshu URL Handling#

The XHS class in source/application/app.py is the central entry point for all URL processing. It defines a set of compiled regex patterns as class-level constants, uses them to validate and classify incoming URLs, and resolves xhslink.com short links to full URLs before any data extraction occurs.

URL Pattern Constants#

Nine regex patterns are compiled at class definition time :

Constant	Pattern	Matches
`LINK_XHS`	`(?:https?://)?www\.xiaohongshu\.com/explore/\S+`	XHS explore-page links
`LINK_RN`	`(?:https?://)?www\.rednote\.com/explore/\S+`	RedNote explore-page links
`USER_XHS`	`(?:https?://)?www\.xiaohongshu\.com/user/profile/[a-z0-9]+/\S+`	XHS user-profile links
`USER_RN`	`(?:https?://)?www\.rednote\.com/user/profile/[a-z0-9]+/\S+`	RedNote user-profile links
`SHARE_XHS`	`(?:https?://)?www\.xiaohongshu\.com/discovery/item/\S+`	XHS share/discovery links
`SHARE_RN`	`(?:https?://)?www\.rednote\.com/discovery/item/\S+`	RedNote share/discovery links
`SHORT`	`(?:https?://)?xhslink\.com/[^\s"<>\\^`{	}，。；！？、【】《》]+`

All patterns accept both http:// and https:// schemes, or no scheme at all .

Link Extraction Pipeline: `extract_links`#

extract_links(url) is the primary URL processing method. It accepts a raw string (which may contain multiple space-separated tokens, e.g. a share message with embedded links) and returns a list of validated full URLs.

Processing order per token :

Short-link detection — If SHORT matches, call Html.request_url(url, content=False) to follow the redirect and obtain the final URL. The resolved URL then passes through the remaining steps below.
SHARE_XHS / SHARE_RN — /discovery/item/ links; highest priority after short-link resolution.
LINK_XHS / LINK_RN — /explore/ links.
USER_XHS / USER_RN — /user/profile/ links.

Only the first matching pattern's result is appended. If no pattern matches, the token is silently dropped.

Short-Link Resolution: `Html.request_url`#

Short links are resolved in source/application/request.py by calling Html.request_url(url, content=False):

When content=False, the method returns str(response.url) — the final URL after all HTTP redirects — rather than the response body .
The underlying httpx.AsyncClient is configured with follow_redirects=True, so the full redirect chain is resolved transparently.
If no http scheme is present, https:// is prepended automatically before the request .
On HTTPError, the method logs the error and returns "", which causes the token to be skipped downstream .

The method is decorated with @retry for resilience against transient network failures .

ID Extraction: `extract_id`#

After URL validation, extract_id(links) parses the post or user ID from each validated URL using ID or ID_USER. The ID is the path segment immediately following explore/, item/, or the user hex-string in /user/profile/, up to the ? query separator.

Entry Points#

extract_links is called from all downstream consumers:

extract() — batch extraction API
extract_cli() — CLI mode
monitor() — clipboard monitor (continuously polls clipboard for new links)
handle() (REST API) — /xhs/detail POST endpoint
deal_detail_mcp() — MCP server tools (get_detail_data, download_detail)

Accepted link formats, per the MCP server instructions :

https://www.xiaohongshu.com/explore/...
https://www.xiaohongshu.com/discovery/item/...
https://xhslink.com/...