Xiaohongshu Anti-Crawling#
Xiaohongshu (ε°ηΊ’δΉ¦) enforces several overlapping access control layers that cause extraction failures in XHS-Downloader:
xsec_tokenrequirement β notes and images cannot be fetched without a valid, server-issued security token in the URL.- Geographic IP restrictions β the platform blocks non-mainland-China IPs, including VPNs and proxies.
- Rate limiting / CAPTCHA β excessive request frequency triggers temporary bans or CAPTCHA challenges.
- URL domain changes β share links migrated from
.comto.cnin July 2026, breaking hardcoded regex patterns.
The project maintainer treats these as platform-level risk controls and does not attempt to circumvent them . For users affected by geographic or token restrictions, the Tampermonkey browser userscript is the primary recommended alternative .
xsec_token Requirement#
xsec_token is a server-generated security token Xiaohongshu requires as a URL query parameter to access note and image content . URLs without it (e.g., https://www.xiaohongshu.com/explore/<note_id>) now fail to retrieve content; the correct format is:
https://www.xiaohongshu.com/explore/<note_id>?xsec_token=<token>
Key properties :
- The token is tied to the requesting account, date, and source β it is not a fixed value.
- It cannot be derived from a
note_idalone. - Tokens obtained at collection time must be reused at download time; swapping tokens causes failures .
In the codebase, the token is not parsed separately β it is passed through as part of the full URL string to request_url() without decomposition.
How to Obtain xsec_token#
| Source | Method |
|---|---|
| Search API | https://edith.xiaohongshu.com/api/sns/web/v1/search/notes returns xsec_token per result |
| Author profile API | Paginate the author's notes with user_id + cursor; each item includes its paired xsec_token |
| Tampermonkey userscript | Automatically extracts from unsafeWindow.__INITIAL_STATE__ in user notes, album feeds, search results, and recommendation feeds |
| Share link | Copying a share link from the XHS app/browser includes xsec_token directly β the simplest approach |
Geographic IP Restrictions#
Xiaohongshu enforces mainland China ISP IP requirements at the platform level. Users outside China β including those using VPNs or proxies β encounter CAPTCHA and "failed to retrieve data" errors regardless of whether they provide a valid cookie .
This is a server-side enforcement mechanism, not a bug in XHS-Downloader. The tool has no built-in proxy or geo-bypass capability.
If you can browse the works normally in the browser, you can consider using the Tampermonkey user script. β maintainer
Rate Limiting and CAPTCHA#
Rapid sequential requests trigger Xiaohongshu's risk control, resulting in temporary bans (typically ~24 hours) or CAPTCHA challenges . Bans can persist even for normal browser sessions if the account was flagged via API access .
XHS-Downloader mitigates this with a randomized inter-request delay using a lognormal distribution β averaging 6 seconds with a 0.5-second floor β called after every successful HTTP response :
# source/module/tools.py
def get_wait_time(avg_delay=6.0, sigma=0.6) -> float:
mu = log(avg_delay) - (sigma**2 / 2)
return max(0.5, lognormvariate(mu, sigma))
Users running from source code can tune avg_delay and sigma directly. The compiled executable does not support this change .
HTTP errors (including rate-limit responses) are caught at request_url() and logged, returning an empty string for upstream handling.
Domain Change: .com β .cn (July 2026)#
In July 2026, Xiaohongshu began issuing share links with a .cn domain suffix instead of .com . XHS-Downloader's URL matching patterns in source/application/app.py are hardcoded to .com only:
LINK_XHS = compile(r"(?:https?://)?www\.xiaohongshu\.com/explore/\S+")
SHARE_XHS = compile(r"(?:https?://)?www\.xiaohongshu\.com/discovery/item/\S+")
SHORT = compile(r"(?:https?://)?xhslink\.com/[^\s\"<>\\^`{|}οΌγοΌοΌοΌγγγγγ]+")
When the app's extract_links() method encounters a .cn URL, it silently discards it, producing a "ζεε°ηΊ’δΉ¦δ½ειΎζ₯ε€±θ΄₯" (failed to extract link) warning with no content retrieved .
Fix: Update the patterns to (?:com|cn) . The maintainer has directed users to use the develop branch while this is resolved .
Users can manually work around this by replacing
.cnwith.comin the shared URL before inputting it.
Workarounds and Mitigations#
| Issue | Workaround |
|---|---|
Missing xsec_token | Copy share link directly from XHS app (token is included); or use Tampermonkey script |
xsec_token needed for batch | Use search API or author profile API β both return tokens alongside note IDs |
| Outside China / VPN blocked | Use the Tampermonkey userscript while browsing normally in a browser |
| Rate limit / CAPTCHA | Wait ~24 hours; reduce request frequency by editing get_wait_time() in source/module/tools.py |
.cn domain links failing | Manually replace .cn with .com, or switch to the develop branch |
| Old/inaccessible content | No workaround; some content remains inaccessible due to server-side restrictions |
Cookie Configuration#
A valid browser cookie improves access (required for higher video quality and some content) but does not bypass geographic IP restrictions . See the Video Quality Selection article for cookie setup steps.