Compare commits
11 commits
9ab8e11672
...
3467472fb7
| Author | SHA1 | Date | |
|---|---|---|---|
|
3467472fb7 | ||
|
42784ffc83 | ||
|
5be483808f | ||
|
d5d9f462b3 | ||
|
|
0538230d46 | ||
|
|
3b484a9d0d | ||
|
9f72c26ad6 | ||
|
|
2db97eaee8 | ||
|
|
8c98fb7032 | ||
|
56f4ce44a2 | ||
|
|
ab87be43af |
15 changed files with 1456 additions and 246 deletions
|
|
@ -22,5 +22,8 @@ RUN pip install --no-cache-dir -r requirements.txt
|
|||
# 确保启动脚本可执行
|
||||
RUN chmod +x start.sh
|
||||
|
||||
# 暴露端口
|
||||
EXPOSE 8456
|
||||
|
||||
# 设置容器启动命令
|
||||
CMD ["./start.sh"]
|
||||
|
|
|
|||
327
README.en.md
327
README.en.md
|
|
@ -7,87 +7,90 @@
|
|||
|
||||
[English](./README.en.md)\|[Simplified Chinese](./README.md)
|
||||
|
||||
🚀 "Douyin_TikTok_Download_API" is a high-performance asynchronous out-of-the-box[Tik Tok](https://www.douyin.com)\|[Tiktok](https://www.tiktok.com)\|[Biliable](https://www.bilibili.com)Data crawling tool, supports API calls, online batch analysis and download.
|
||||
🚀"Douyin_TikTok_Download_API" is a high-performance asynchronous API that can be used out of the box[Tik Tok](https://www.douyin.com)\|[Tiktok](https://www.tiktok.com)\|[Biliable](https://www.bilibili.com)Data crawling tool supports API calling, online batch analysis and downloading.
|
||||
|
||||
[](LICENSE)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/releases/latest)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/stargazers)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/network/members)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues)[](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues?q=is%3Aissue+is%3Aclosed)<br>[](https://pypi.org/project/douyin-tiktok-scraper/)[](https://pypi.org/project/douyin-tiktok-scraper/#files)[](https://pypi.org/project/douyin-tiktok-scraper/)[](https://pypi.org/project/douyin-tiktok-scraper/)<br>[](https://api.douyin.wtf/docs)[](https://api.tikhub.io/docs)<br>[](https://afdian.net/@evil0ctal)[](https://ko-fi.com/evil0ctal)[](https://www.patreon.com/evil0ctal)
|
||||
|
||||
</div>
|
||||
|
||||
## Sponsors
|
||||
## Sponsor
|
||||
|
||||
These sponsors have paid to place them here,**Doinan_tics_download_api**The project will always be free and open source. If you wish to be a sponsor of this project, please check out my[GitHub Sponsor Page](https://github.com/sponsors/evil0ctal)。
|
||||
These sponsors have paid to be placed here,**Doinan_tics_download_api**The project will always be free and open source. If you would like to become a sponsor of this project, please check out my[GitHub Sponsor Page](https://github.com/sponsors/evil0ctal)。
|
||||
|
||||
<div align="center">
|
||||
<a href="https://www.tikhub.io/" target="_blank">
|
||||
<img src="https://tikhub.io/wp-content/uploads/2024/11/Main-Logo.webp" width="100" alt="TikHub.io - Global Social Data & API Marketplace">
|
||||
</a>
|
||||
<div>
|
||||
<h2><b>TikHub.io</b></h2>
|
||||
<p>Your Ultimate Social Media Data & API Marketplace</p>
|
||||
<p>
|
||||
Professional data solutions for Douyin, Xiaohongshu, TikTok, Instagram, YouTube,
|
||||
Twitter, and more.<br>
|
||||
Real-time Data | Flexible APIs | Seamless Integration | Competitive Pricing with Discounts
|
||||
</p>
|
||||
<p>
|
||||
<b>Discover TikHub.io Marketplace</b><br>
|
||||
Buy and sell custom APIs, services, and social media solutions.<br>
|
||||
Join a thriving ecosystem of developers, businesses, and content creators.
|
||||
</p>
|
||||
<p><em>Trusted by leading global influencer marketing and social media intelligence platforms</em></p>
|
||||
</div>
|
||||
</div>
|
||||
<p align="center">
|
||||
<a href="https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad">
|
||||
<img style="border-radius:20px" width="845" height="845" alt="TikHub IO_Banner zh" src="https://github.com/user-attachments/assets/18ce4674-83b3-4312-a5d8-a45cf7cef7b2">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
## 👻 Introduction
|
||||
[Tickubbub](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)Provides more than 700 endpoints that can be used to obtain and analyze data from 14+ social media platforms - including videos, users, comments, stores, products, trends, etc., complete all data access and analysis in one stop.
|
||||
|
||||
> 🚨If you want to use a private server to run this project, please refer to:[Deployment preparations](./README.md#%EF%B8%8F%E9%83%A8%E7%BD%B2%E5%89%8D%E7%9A%84%E5%87%86%E5%A4%87%E5%B7%A5%E4%BD%9C%E8%AF%B7%E4%BB%94%E7%BB%86%E9%98%85%E8%AF%BB),[Docker deployment](./README.md#%E9%83%A8%E7%BD%B2%E6%96%B9%E5%BC%8F%E4%BA%8C-docker),[One-click deployment](./README.md#%E9%83%A8%E7%BD%B2%E6%96%B9%E5%BC%8F%E4%B8%80-linux)
|
||||
By checking in every day, you can get free quota. You can use my registration invitation link:[https://user.tikhub.io/users/signup?referral_code=1wRL8eQk](https://user.tikhub.io/users/signup?referral_code=1wRL8eQk&utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)or invitation code:`1wRL8eQk`, you can get it by registering and recharging`$2`Quota.
|
||||
|
||||
This project is based on[Pydebio](https://github.com/pywebio/PyWebIO)，[Fasting](https://fastapi.tiangolo.com/)，[HTTPX](https://www.python-httpx.org/), fast asynchronous[Tik Tok](https://www.douyin.com/)/[Tiktok](https://www.tiktok.com/)Data crawling tool, and online batch analysis and downloading of watermark-free videos or picture albums through the web, data crawling API, iOS shortcuts without watermark download and other functions. You can deploy or transform this project yourself to achieve more functions, or you can call it directly in your project[scraper.py](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/Stable/scraper.py)Or install an existing one[pip package](https://pypi.org/project/douyin-tiktok-scraper/)As a parsing library, easy to crawl data, etc....
|
||||
[Tickubbub](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)The following services are provided:
|
||||
|
||||
- Rich data interface
|
||||
- Get free quota by signing in every day
|
||||
- High-quality API services
|
||||
- Official website:[https://tikhub.io/](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)
|
||||
- GitHub address:<https://github.com/TikHubIO/>
|
||||
|
||||
## 👻Introduction
|
||||
|
||||
> 🚨If you need to use a private server to run this project, please refer to:[Deployment preparations](./README.md#%EF%B8%8F%E9%83%A8%E7%BD%B2%E5%89%8D%E7%9A%84%E5%87%86%E5%A4%87%E5%B7%A5%E4%BD%9C%E8%AF%B7%E4%BB%94%E7%BB%86%E9%98%85%E8%AF%BB),[Docker deployment](./README.md#%E9%83%A8%E7%BD%B2%E6%96%B9%E5%BC%8F%E4%BA%8C-docker),[One-click deployment](./README.md#%E9%83%A8%E7%BD%B2%E6%96%B9%E5%BC%8F%E4%B8%80-linux)
|
||||
|
||||
This project is based on[PyWebIO](https://github.com/pywebio/PyWebIO)，[speedy](https://fastapi.tiangolo.com/)，[HTTPX](https://www.python-httpx.org/), fast and asynchronous[Tik Tok](https://www.douyin.com/)/[Tiktok](https://www.tiktok.com/)Data crawling tool, and realizes online batch parsing and downloading of videos or photo albums without watermarks, data crawling API, and iOS shortcut commands without watermark downloads through the Web. You can deploy or modify this project yourself to achieve more functions, or you can call it directly in your project[scraper.py](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/Stable/scraper.py)or install an existing[pip package](https://pypi.org/project/douyin-tiktok-scraper/)As a parsing library, it is easy to crawl data, etc.....
|
||||
|
||||
_Some simple application scenarios:_
|
||||
|
||||
_Download videos that are prohibited from being downloaded, perform data analysis, and download without watermark on iOS (with[iOS's shortcut command APP](https://apps.apple.com/cn/app/%E5%BF%AB%E6%8D%B7%E6%8C%87%E4%BB%A4/id915249334)In conjunction with this project API, it can realize in-app download or read clipboard download, etc...._
|
||||
_Download prohibited videos, perform data analysis, download without watermark on iOS (with[Shortcut command APP that comes with iOS](https://apps.apple.com/cn/app/%E5%BF%AB%E6%8D%B7%E6%8C%87%E4%BB%A4/id915249334)Cooperate with the API of this project to achieve in-app downloads or read clipboard downloads), etc....._
|
||||
|
||||
## 🔊 V4 version notes
|
||||
|
||||
- If you are interested in writing this project, please add WeChat`Evil0ctal`Note: Github project reconstruction, everyone can communicate and learn from each other in the group, and do not allow advertisements or illegal things to be made purely friends and technical communication.
|
||||
- This project uses`X-Bogus`Algorithm and`A_Bogus`The algorithm requests TikTok and TikTok's Web API.
|
||||
- Due to Douyin's risk control, please go to**Get the Douyin website cookies in the browser and replace them in config.yaml.**
|
||||
- Please read the document below before asking for an issue, and most solutions to the problem will be included in the document.
|
||||
- This project is completely free, but please follow it when using it:[Apache-2.0 license](https://github.com/Evil0ctal/Douyin_TikTok_Download_API?tab=Apache-2.0-1-ov-file#readme)
|
||||
- If you are interested in writing this project together, please add us on WeChat`Evil0ctal`Note: Github project reconstruction, everyone can communicate and learn from each other in the group. Advertising and illegal things are not allowed. It is purely for making friends and technical exchanges.
|
||||
- This project uses`X-Bogus`Algorithms and`A_Bogus`The algorithm requests the Web API of Douyin and TikTok.
|
||||
- Due to Douyin's risk control, after deploying this project, please**Obtain the cookie of Douyin website in the browser and replace it in config.yaml.**
|
||||
- Please read the document below before raising an issue. Solutions to most problems will be included in the document.
|
||||
- This project is completely free, but when using it, please comply with:[Apache-2.0 license](https://github.com/Evil0ctal/Douyin_TikTok_Download_API?tab=Apache-2.0-1-ov-file#readme)
|
||||
|
||||
## 🔖TikHub.io API
|
||||
|
||||
[TikHub.io](https://api.tikhub.io/)It is an API platform that provides various public data interfaces including Douyin and TikTok. If you want to support it[Doinan_tics_download_api](https://github.com/Evil0ctal/Douyin_TikTok_Download_API)We strongly recommend that you choose the project development[TikHub.io](https://api.tikhub.io/)。
|
||||
[TikHub.io](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)Provides more than 700 endpoints that can be used to obtain and analyze data from 14+ social media platforms - including videos, users, comments, stores, products, trends, etc., complete all data access and analysis in one stop.
|
||||
|
||||
If you want to support[Doinan_tics_download_api](https://github.com/Evil0ctal/Douyin_TikTok_Download_API)For project development, we strongly recommend that you choose[TikHub.io](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)。
|
||||
|
||||
#### Features:
|
||||
|
||||
> 📦 Out of the box
|
||||
> 📦 Ready to use right out of the box
|
||||
|
||||
Simplify the usage process and quickly carry out development work using the encapsulated SDK. All API interfaces are designed according to the RESTful architecture and are described and documented using the OpenAPI specification, accompanied by example parameters to ensure that calls are easier.
|
||||
Simplify the use process and use the packaged SDK to quickly carry out development work. All API interfaces are designed based on RESTful architecture and are described and documented using OpenAPI specifications, with sample parameters included to ensure easier calling.
|
||||
|
||||
> 💰 Cost Advantage
|
||||
> 💰 Cost advantage
|
||||
|
||||
There is no preset package limit, no monthly usage threshold, all consumption is billed instantly based on the actual usage, and is billed step by step based on the user's daily request volume. At the same time, you can check in in the user's background through daily check-in, and these free amounts will not expire.
|
||||
There are no preset package restrictions and no monthly usage thresholds. All consumption is billed immediately based on actual usage, and tiered billing is performed based on the user's daily requests. At the same time, free quotas can be obtained in the user backend through daily check-ins, and these free quotas will not expire.
|
||||
|
||||
> ⚡️ Quick support
|
||||
> ⚡️ Fast support
|
||||
|
||||
We have a huge Discord community server where administrators and other users will quickly reply to you to help you quickly resolve current issues.
|
||||
We have a large Discord community server, where administrators and other users will quickly reply to you and help you quickly solve current problems.
|
||||
|
||||
> 🎉 Embrace open source
|
||||
> 🎉Embrace open source
|
||||
|
||||
Some of the source code of TikHub will be open sourced on Github and will sponsor some open source projects.
|
||||
Part of TikHub's source code will be open sourced on Github, and it will sponsor authors of some open source projects.
|
||||
|
||||
#### Link:
|
||||
#### Registration and use:
|
||||
|
||||
- Githubub:[TIKHOB GITUB](https://github.com/TikHubIO)
|
||||
- Discord:[Tachub](https://discord.com/invite/aMEAS8Xsvz)
|
||||
- Register:[TikHub singnup](https://beta-web.tikhub.io/en-us/users/signup)
|
||||
- API Docs:[TickHub API Docs](https://api.tikhub.io/)
|
||||
By checking in every day, you can get free quota. You can use my registration invitation link:[https://user.tikhub.io/users/signup?referral_code=1wRL8eQk](https://user.tikhub.io/users/signup?referral_code=1wRL8eQk&utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)or invitation code:`1wRL8eQk`, you can get it by registering and recharging`$2`Quota.
|
||||
|
||||
## 🖥 Demo site: I am very fragile... Please do not press test (·•᷄ࡇ•᷅ )
|
||||
#### Related links:
|
||||
|
||||
> 😾The online download function of the demo site has been turned off, and Douyin's parsing and API services cannot be guaranteed for availability on the Demo site due to cookies.
|
||||
- Official website:[https://tikhub.io/](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)
|
||||
- API documentation:<https://api.tikhub.io/docs>
|
||||
- Githubub:<https://github.com/TikHubIO/>
|
||||
- Discord：<https://discord.com/invite/aMEAS8Xsvz>
|
||||
|
||||
## 🖥Demo site: I am very vulnerable...please do not stress test (·•᷄ࡇ•᷅ )
|
||||
|
||||
> 😾The online download function of the demo site has been turned off, and due to cookie reasons, Douyin's parsing and API services cannot guarantee availability on the Demo site.
|
||||
|
||||
🍔Web APP:<https://douyin.wtf/>
|
||||
|
||||
|
|
@ -95,33 +98,33 @@ Some of the source code of TikHub will be open sourced on Github and will sponso
|
|||
|
||||
🌭tikub APU Docuration:<https://api.tikhub.io/docs>
|
||||
|
||||
💾 iOS Shortcut:[Shortcut release](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/discussions/104?sort=top)
|
||||
💾iOS Shortcut (shortcut command):[Shortcut release](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/discussions/104?sort=top)
|
||||
|
||||
📦️Desktop downloader (recommended warehouse):
|
||||
📦️Desktop downloader (recommended by warehouse):
|
||||
|
||||
- [Johnserf-Seed/Tiktokdownload](https://github.com/Johnserf-Seed/TikTokDownload)
|
||||
- [HFrost0/bilix](https://github.com/HFrost0/bilix)
|
||||
- [Tairraos/TikDown - \[Updated to be\]](https://github.com/Tairraos/TikDown/)
|
||||
- [Tairraos/TikDown - \[needs update\]](https://github.com/Tairraos/TikDown/)
|
||||
|
||||
## ⚗️Technology Stack
|
||||
## ⚗️Technology stack
|
||||
|
||||
- [/app/web](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/app/web)-[Pydebio](https://www.pyweb.io/)
|
||||
- [/app/api](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/app/api)-[Fasting](https://fastapi.tiangolo.com/)
|
||||
- [/app/web](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/app/web)-[PyWebIO](https://www.pyweb.io/)
|
||||
- [/app/api](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/app/api)-[speedy](https://fastapi.tiangolo.com/)
|
||||
- [/crawlers](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/crawlers)-[HTTPX](https://www.python-httpx.org/)
|
||||
|
||||
> **_/crawlers_**
|
||||
|
||||
- Submit requests to APIs of different platforms and retrieve data, and return dictionary (dict) after processing, supports asynchronousness.
|
||||
- Submit requests to APIs on different platforms and retrieve data. After processing, a dictionary (dict) is returned, and asynchronous support is supported.
|
||||
|
||||
> **_/app/api_**
|
||||
|
||||
- Obtain the request parameters and use`Crawlers`After processing data, the related classes return in JSON form, download videos, and implement fast calls with iOS shortcuts, and support asynchronous.
|
||||
- Get request parameters and use`Crawlers`The related classes process the data and return it in JSON form, download the video, and cooperate with iOS shortcut commands to achieve fast calling and support asynchronous.
|
||||
|
||||
> **_/app/web_**
|
||||
|
||||
- use`PyWebIO`A simple web program created, process the value entered on the web page and use it`Crawlers`The related class processing interface outputs related data on the web page.
|
||||
- use`PyWebIO`A simple web program created to process the values entered on the web page and then use them`Crawlers`The related class processing interface outputs related data on the web page.
|
||||
|
||||
**_Most of the parameters of the above files can be in the corresponding`config.yaml`Make modifications in_**
|
||||
**_Most of the parameters of the above files can be found in the corresponding`config.yaml`Modify in_**
|
||||
|
||||
## 💡Project file structure
|
||||
|
||||
|
|
@ -144,81 +147,81 @@ Some of the source code of TikHub will be open sourced on Github and will sponso
|
|||
│ └─web
|
||||
└─utils
|
||||
|
||||
## ✨Support functions:
|
||||
## ✨Supported functions:
|
||||
|
||||
- Batch analysis on the web side (supports Douyin/TikTok hybrid analysis)
|
||||
- Download videos or albums online.
|
||||
- Production[pip package](https://pypi.org/project/douyin-tiktok-scraper/)Easy and quick import of your project
|
||||
- [iOS shortcuts to quickly call API](https://apps.apple.com/cn/app/%E5%BF%AB%E6%8D%B7%E6%8C%87%E4%BB%A4/id915249334)Implement watermark-free videos/pictures in-app download
|
||||
- Complete API documentation ([Demo/Demo](https://api.douyin.wtf/docs))
|
||||
- Rich API interfaces:
|
||||
- TikTok web version API
|
||||
- Batch parsing on the web page (supports Douyin/TikTok mixed parsing)
|
||||
- Download videos or photo albums online.
|
||||
- make[pip package](https://pypi.org/project/douyin-tiktok-scraper/)Conveniently and quickly import your projects
|
||||
- [iOS shortcut commands to quickly call API](https://apps.apple.com/cn/app/%E5%BF%AB%E6%8D%B7%E6%8C%87%E4%BB%A4/id915249334)Achieve in-app download of watermark-free videos/photo albums
|
||||
- Complete API documentation ([Demo/Demonstration](https://api.douyin.wtf/docs))
|
||||
- Rich API interface:
|
||||
- Douyin web version API
|
||||
|
||||
- [x] Video data analysis
|
||||
- [x] Obtain user's homepage work data
|
||||
- [x] Obtain data on the user's homepage liked works
|
||||
- [x] Obtain data on the user's homepage collection of works
|
||||
- [x] Get user homepage work data
|
||||
- [x] Obtain the data of works liked by the user's homepage
|
||||
- [x] Obtain the data of collected works on the user's homepage
|
||||
- [x] Get user homepage information
|
||||
- [x] Obtain user compiled works data
|
||||
- [x] Obtain user live streaming data
|
||||
- [x] Get live streaming data for the specified user
|
||||
- [x] Get the ranking of gift-giving users in the live broadcast room
|
||||
- [x] Get individual video comment data
|
||||
- [x] Get comments and response data for specified videos
|
||||
- [x] Get user collection work data
|
||||
- [x] Get user live stream data
|
||||
- [x] Get the live streaming data of a specified user
|
||||
- [x] Get the ranking of users who give gifts in the live broadcast room
|
||||
- [x] Get single video comment data
|
||||
- [x] Get the comment reply data of the specified video
|
||||
- [x] Generate msToken
|
||||
- [x] Generate verification_fp
|
||||
- [x] Generate verify_fp
|
||||
- [x] Generate s_v_web_id
|
||||
- [x] Generate X-Bogus parameters using interface URL
|
||||
- [x] Generate A_Bogus parameters using interface URL
|
||||
- [x] Extract a single user id
|
||||
- [x] Extract list user id
|
||||
- [x] Extract individual works id
|
||||
- [x] Extract a single work id
|
||||
- [x] Extract list work id
|
||||
- [x] Extract list live broadcast room number
|
||||
- [x] Extract list live broadcast room number
|
||||
- [x] Extract live broadcast room number from list
|
||||
- [x] Extract live broadcast room number from list
|
||||
- TikTok web version API
|
||||
|
||||
- [x] Video data analysis
|
||||
- [x] Obtain user's homepage work data
|
||||
- [x] Obtain data on the user's homepage liked works
|
||||
- [x] Get user homepage work data
|
||||
- [x] Obtain the data of works liked by the user's homepage
|
||||
- [x] Get user homepage information
|
||||
- [x] Get the user's homepage fan data
|
||||
- [x] Get user's homepage follow data
|
||||
- [x] Obtain data on the collection of works by users on the homepage
|
||||
- [x] Get search data for users' homepage
|
||||
- [x] Get user home page fan data
|
||||
- [x] Get user homepage follow data
|
||||
- [x] Get user homepage collection work data
|
||||
- [x] 获取用户主页搜藏数据
|
||||
- [x] Get user homepage playlist data
|
||||
- [x] Get individual video comment data
|
||||
- [x] Get comments and response data for specified videos
|
||||
- [x] Get single video comment data
|
||||
- [x] Get the comment reply data of the specified video
|
||||
- [x] Generate msToken
|
||||
- [x] Generate ttwid
|
||||
- [x] Generate X-Bogus parameters using interface URL
|
||||
- [x] Extract individual user sec_user_id
|
||||
- [x] Extract a single user sec_user_id
|
||||
- [x] Extract list user sec_user_id
|
||||
- [x] Extract individual works id
|
||||
- [x] Extract a single work id
|
||||
- [x] Extract list work id
|
||||
- [x] Get user unique_id
|
||||
- [x] Get the list unique_id
|
||||
- [x] Get list unique_id
|
||||
- Bilibili web version API
|
||||
- [x] Get individual video details
|
||||
- [x] Get the video streaming address
|
||||
- [x] Obtain data on video works published by users
|
||||
- [x] Get all user favorites information
|
||||
- [x] Get video data in the specified favorites
|
||||
- [x] Get information about the specified user
|
||||
- [x] Get video stream address
|
||||
- [x] Obtain user-published video work data
|
||||
- [x] Get all favorites information of the user
|
||||
- [x] Get video data in specified favorites
|
||||
- [x] Get information about a specified user
|
||||
- [x] Get comprehensive popular video information
|
||||
- [x] Get comments for the specified video
|
||||
- [x] Get a reply to the specified comment under the video
|
||||
- [x] Get the specified user dynamics
|
||||
- [x] Get real-time video barrage
|
||||
- [x] Get information about the specified live broadcast room
|
||||
- [x] Get live video streaming
|
||||
- [x] Get the anchor who is currently broadcasting in the specified partition
|
||||
- [x] Get a list of all live partitions
|
||||
- [x] Obtain video score information through bv number
|
||||
- [x] Get comments for specified video
|
||||
- [x] Get the reply to the specified comment under the video
|
||||
- [x] Get the specified user's updates
|
||||
- [x] Get real-time video barrages
|
||||
- [x] Get specified live broadcast room information
|
||||
- [x] Get live room video stream
|
||||
- [x] Get the anchors who are live broadcasting in the specified partition
|
||||
- [x] Get a list of all live broadcast partitions
|
||||
- [x] Obtain video sub-p information through bv number
|
||||
|
||||
* * *
|
||||
|
||||
## 📦 Call the parsing library (deprecated and needs to be updated):
|
||||
## 📦Call the parsing library (obsolete and needs to be updated):
|
||||
|
||||
> 💡PIPI ：<https://pypi.org/project/douyin-tiktok-scraper/>
|
||||
|
||||
|
|
@ -241,45 +244,45 @@ asyncio.run(hybrid_parsing(url=input("Paste Douyin/TikTok/Bilibili share URL her
|
|||
|
||||
## 🗺️Supported submission formats:
|
||||
|
||||
> 💡 Tip: Includes but is not limited to the following examples. If you encounter link resolution failure, please enable a new one.[issue](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues)
|
||||
> 💡Tip: Including but not limited to the following examples, if you encounter link parsing failure, please open a new one[issue](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues)
|
||||
|
||||
- TikTok Sharing Password (Copy within the APP)
|
||||
- Douyin sharing password (copy in APP)
|
||||
|
||||
```text
|
||||
7.43 pda:/ 让你在几秒钟之内记住我 https://v.douyin.com/L5pbfdP/ 复制此链接，打开Dou音搜索，直接观看视频！
|
||||
```
|
||||
|
||||
- TikTok short URL (copy within the APP)
|
||||
- Douyin short URL (copy within APP)
|
||||
|
||||
```text
|
||||
https://v.douyin.com/L4FJNR3/
|
||||
```
|
||||
|
||||
- Douyin Normal URL (web version copy)
|
||||
- Douyin normal URL (copy from web version)
|
||||
|
||||
```text
|
||||
https://www.douyin.com/video/6914948781100338440
|
||||
```
|
||||
|
||||
- TikTok Discovery Page URL (APP Copy)
|
||||
- Douyin discovery page URL (APP copy)
|
||||
|
||||
```text
|
||||
https://www.douyin.com/discover?modal_id=7069543727328398622
|
||||
```
|
||||
|
||||
- TikTok short URL (copy within the APP)
|
||||
- TikTok short URL (copy within APP)
|
||||
|
||||
```text
|
||||
https://www.tiktok.com/t/ZTR9nDNWq/
|
||||
```
|
||||
|
||||
- TikTok normal website address (web version copy)
|
||||
- TikTok normal URL (copy from web version)
|
||||
|
||||
```text
|
||||
https://www.tiktok.com/@evil0ctal/video/7156033831819037994
|
||||
```
|
||||
|
||||
- TikTok batch URL (no need to use matching separation)
|
||||
- Douyin/TikTok batch URL (no need to use matching separation)
|
||||
|
||||
```text
|
||||
https://v.douyin.com/L4NpDJ6/
|
||||
|
|
@ -290,7 +293,7 @@ https://www.tiktok.com/t/ZTR9nDNWq/
|
|||
https://www.tiktok.com/@evil0ctal/video/7156033831819037994
|
||||
```
|
||||
|
||||
## 🛰️API Documentation
|
||||
## 🛰️API documentation
|
||||
|
||||
**_API documentation:_**
|
||||
|
||||
|
|
@ -298,71 +301,71 @@ local:<http://localhost/docs>
|
|||
|
||||
Online:<https://api.douyin.wtf/docs>
|
||||
|
||||
**_API Demo:_**
|
||||
**_API demo:_**
|
||||
|
||||
- Crawl video data (TikTok or Douyin mixed analysis)`https://api.douyin.wtf/api/hybrid/video_data?url=[视频链接/Video URL]&minimal=false`
|
||||
- Download video/picture album (TikTok or Douyin mixed analysis)`https://api.douyin.wtf/api/download?url=[视频链接/Video URL]&prefix=true&with_watermark=false`
|
||||
- Crawl video data (TikTok or Douyin hybrid analysis)`https://api.douyin.wtf/api/hybrid/video_data?url=[视频链接/Video URL]&minimal=false`
|
||||
- Download videos/photo albums (TikTok or Douyin hybrid analysis)`https://api.douyin.wtf/api/download?url=[视频链接/Video URL]&prefix=true&with_watermark=false`
|
||||
|
||||
**_For more demonstrations, please check the document content..._**
|
||||
**_For more demonstrations, please see the documentation..._**
|
||||
|
||||
## ⚠️Preparation before deployment (please read carefully):
|
||||
## ⚠️Preparation work before deployment (please read carefully):
|
||||
|
||||
- You need to solve the risk control problem of crawler cookies by yourself, otherwise the interface may be unavailable. After modifying the configuration file, you need to restart the service before it takes effect. It is best to use cookies from the account you have logged in.
|
||||
- Douyin web cookies (acquire and replace cookies in the following configuration files):
|
||||
- You need to solve the problem of crawler cookie risk control by yourself, otherwise the interface may become unusable. After modifying the configuration file, you need to restart the service for it to take effect, and it is best to use cookies from accounts that you have already logged in to.
|
||||
- Douyin web cookie (obtain and replace the cookie in the configuration file below):
|
||||
- <https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/30e56e5a7f97f87d60b1045befb1f6db147f8590/crawlers/douyin/web/config.yaml#L7>
|
||||
- TikTok web cookies (acquire and replace cookies in the following configuration files):
|
||||
- TikTok web-side cookies (obtain and replace the cookies in the configuration file below):
|
||||
- <https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/30e56e5a7f97f87d60b1045befb1f6db147f8590/crawlers/tiktok/web/config.yaml#L6>
|
||||
- I turned off the online download function of the demonstration site. Someone downloaded a huge video and it crashed directly on my server. You can right-click to save the video on the web parsing result page...
|
||||
- The cookies on the demo site are my own and are not guaranteed to be valid for a long time. They only serve as a demonstration. If you deploy it yourself, please get the cookies yourself.
|
||||
- HTTP 403 error will occur if you need to access the video link returned by TikTok Web API. Please use the API in this project`/api/download`The interface downloads TikTok videos. This interface has been manually closed in the demonstration site, and you need to deploy this project yourself.
|
||||
- There is one here**Video tutorial**You can refer to:**_<https://www.bilibili.com/video/BV1vE421j7NR/>_**
|
||||
- I turned off the online download function of the demo site. The video someone downloaded was so huge that it crashed the server. You can right-click on the web page parsing results page to save the video...
|
||||
- The cookies of the demo site are my own and are not guaranteed to be valid for a long time. They only serve as a demonstration. If you deploy it yourself, please obtain the cookies yourself.
|
||||
- If you need to directly access the video link returned by TikTok Web API, an HTTP 403 error will occur. Please use the API in this project.`/api/download`The interface downloads TikTok videos. This interface has been manually closed in the demo site, and you need to deploy this project by yourself.
|
||||
- here is one**Video tutorial**You can refer to:**_<https://www.bilibili.com/video/BV1vE421j7NR/>_**
|
||||
|
||||
## 💻Deployment (Method 1 Linux)
|
||||
|
||||
> 💡Tip: It is best to deploy this project to a server in the United States, otherwise strange bugs may occur.
|
||||
> 💡Tips: It is best to deploy this project to a server in the United States, otherwise strange BUGs may appear.
|
||||
|
||||
Recommended to use[DigitalOcean](https://www.digitalocean.com/)server, because it can be free.
|
||||
Recommended for everyone to use[Digitalocean](https://www.digitalocean.com/)server, because you can have sex for free.
|
||||
|
||||
Sign up with my invitation link and you can get a credit of $200, and I can get a reward of $25 when you spend $25 on it.
|
||||
Use my invitation link to sign up and you can get a $200 credit, and when you spend $25 on it, I can also get a $25 reward.
|
||||
|
||||
My invitation link:
|
||||
|
||||
<https://m.do.co/c/9f72a27dec35>
|
||||
|
||||
> Use scripts to deploy this project in one click
|
||||
> Use script to deploy this project with one click
|
||||
|
||||
- This project provides one-click deployment scripts to quickly deploy the project on the server.
|
||||
- The script was tested on Ubuntu 20.04 LTS, and other systems may have problems. If there are any problems, please solve them yourself.
|
||||
- Download using wget command[install.sh](https://raw.githubusercontent.com/Evil0ctal/Douyin_TikTok_Download_API/main/bash/install.sh)Go to the server and run
|
||||
- This project provides a one-click deployment script that can quickly deploy this project on the server.
|
||||
- The script was tested on Ubuntu 20.04 LTS. Other systems may have problems. If there are any problems, please solve them yourself.
|
||||
- Download using wget command[install.sh](https://raw.githubusercontent.com/Evil0ctal/Douyin_TikTok_Download_API/main/bash/install.sh)to the server and run
|
||||
|
||||
|
||||
wget -O install.sh https://raw.githubusercontent.com/Evil0ctal/Douyin_TikTok_Download_API/main/bash/install.sh && sudo bash install.sh
|
||||
|
||||
> Turn on/stop service
|
||||
> Start/stop service
|
||||
|
||||
- Use the following command to control the operation or stop of the service:
|
||||
- Use the following commands to control running or stopping the service:
|
||||
- `sudo systemctl start Douyin_TikTok_Download_API.service`
|
||||
- `sudo systemctl stop Douyin_TikTok_Download_API.service`
|
||||
|
||||
> Turn on/off automatically
|
||||
> Turn on/off automatic operation at startup
|
||||
|
||||
- Use the following command to set the service to automatically run on or cancel the automatic run on:
|
||||
- Use the following commands to set the service to run automatically at boot or cancel automatic run at boot:
|
||||
- `sudo systemctl enable Douyin_TikTok_Download_API.service`
|
||||
- `sudo systemctl disable Douyin_TikTok_Download_API.service`
|
||||
|
||||
> Update the project
|
||||
> Update project
|
||||
|
||||
- When the project is updated, make sure that the update script is executed in the virtual environment and update all dependencies. Enter the project bash directory and run update.sh:
|
||||
- When the project is updated, ensure that the update script is executed in the virtual environment and all dependencies are updated. Enter the project bash directory and run update.sh:
|
||||
- `cd /www/wwwroot/Douyin_TikTok_Download_API/bash && sudo bash update.sh`
|
||||
|
||||
## 💽Deployment (Method 2 Docker)
|
||||
|
||||
> 💡 Tip: Docker deployment is the easiest way to deploy, suitable for users who are not familiar with Linux. This method is suitable for ensuring environmental consistency, isolation and quick settings.
|
||||
> Please use a server that can access Douyin or TikTok normally, otherwise strange bugs may occur.
|
||||
> 💡Tip: Docker deployment is the simplest deployment method and is suitable for users who are not familiar with Linux. This method is suitable for ensuring environment consistency, isolation and quick setup.
|
||||
> Please use a server that can normally access Douyin or TikTok, otherwise strange BUG may occur.
|
||||
|
||||
### Preparation
|
||||
|
||||
Before you begin, make sure your system has Docker installed. If Docker is not installed, you can[Docker official website](https://www.docker.com/products/docker-desktop/)Download and install.
|
||||
Before you begin, make sure Docker is installed on your system. If you haven't installed Docker yet, you can install it from[Docker official website](https://www.docker.com/products/docker-desktop/)Download and install.
|
||||
|
||||
### Step 1: Pull the Docker image
|
||||
|
||||
|
|
@ -372,40 +375,40 @@ First, pull the latest Douyin_TikTok_Download_API image from Docker Hub.
|
|||
docker pull evil0ctal/douyin_tiktok_download_api:latest
|
||||
```
|
||||
|
||||
If necessary, you can replace it`latest`Tags for the specific version you need to deploy.
|
||||
Can be replaced if needed`latest`Label the specific version you need to deploy.
|
||||
|
||||
### Step 2: Run the Docker container
|
||||
|
||||
After pulling the image, you can start a container from this image. The following are the commands to run the container, including the basic configuration:
|
||||
After pulling the image, you can start a container from this image. Here are the commands to run the container, including basic configuration:
|
||||
|
||||
```bash
|
||||
docker run -d --name douyin_tiktok_api -p 80:80 evil0ctal/douyin_tiktok_download_api
|
||||
```
|
||||
|
||||
Each part of this command works as follows:
|
||||
Each part of this command does the following:
|
||||
|
||||
- `-d`: Run containers in the background (separated mode).
|
||||
- `-d`: Run the container in the background (detached mode).
|
||||
- `--name douyin_tiktok_api `: Name the container`douyin_tiktok_api `。
|
||||
- `-p 80:80`: Map port 80 on the host to port 80 of the container. Adjust the port number according to your configuration or port availability.
|
||||
- `evil0ctal/douyin_tiktok_download_api`: The name of the Docker image to be used.
|
||||
- `-p 80:80`：将主机上的80端口映射到容器的80端口。根据您的配置或端口可用性调整端口号。
|
||||
- `evil0ctal/douyin_tiktok_download_api`: The name of the Docker image to use.
|
||||
|
||||
### Step 3: Verify that the container is running
|
||||
### Step 3: Verify the container is running
|
||||
|
||||
Use the following command to check if your container is running:
|
||||
Check if your container is running using the following command:
|
||||
|
||||
```bash
|
||||
docker ps
|
||||
```
|
||||
|
||||
This lists all active containers. Find`douyin_tiktok_api `to confirm its normal operation.
|
||||
This will list all active containers. Find`douyin_tiktok_api `to confirm that it is functioning properly.
|
||||
|
||||
### Step 4: Access the application
|
||||
### Step 4: Access the App
|
||||
|
||||
After the container runs, you should be able to pass`http://localhost`Or the API client access Douyin_TikTok_Download_API. If you have a different port configured or accessed from a remote location, adjust the URL.
|
||||
Once the container is running, you should be able to pass`http://localhost`Or API client access Douyin_TikTok_Download_API. Adjust the URL if a different port is configured or accessed from a remote location.
|
||||
|
||||
### Optional: Custom Docker commands
|
||||
|
||||
For more advanced deployments, you may want to customize Docker commands, including environment variables, volume mounts for persistent data, or other Docker parameters. Here is an example:
|
||||
For more advanced deployments, you may wish to customize Docker commands to include environment variables, volume mounts for persistent data, or other Docker parameters. Here is an example:
|
||||
|
||||
```bash
|
||||
docker run -d --name douyin_tiktok_api -p 80:80 \
|
||||
|
|
@ -414,12 +417,12 @@ docker run -d --name douyin_tiktok_api -p 80:80 \
|
|||
evil0ctal/douyin_tiktok_download_api
|
||||
```
|
||||
|
||||
- `-v /path/to/your/data:/data`: Turn on the host`/path/to/your/data`The directory mounted to the container`/data`Directory, used to persist or share data.
|
||||
- `-e MY_ENV_VAR=my_value`: Set environment variables in the container`MY_ENV_VAR`, its value is`my_value`。
|
||||
- `-v /path/to/your/data:/data`: Change the`/path/to/your/data`Directory mounted to the container`/data`Directory for persisting or sharing data.
|
||||
- `-e MY_ENV_VAR=my_value`: Set environment variables within the container`MY_ENV_VAR`, whose value is`my_value`。
|
||||
|
||||
### Configuration file modification
|
||||
|
||||
Most of the configurations of the project can be found in the following directories`config.yaml`Modify the file:
|
||||
Most of the configuration of the project can be found in the following directories:`config.yaml`File modification:
|
||||
|
||||
- `/crawlers/douyin/web/config.yaml`
|
||||
- `/crawlers/tiktok/web/config.yaml`
|
||||
|
|
@ -427,7 +430,7 @@ Most of the configurations of the project can be found in the following director
|
|||
|
||||
### Step 5: Stop and remove the container
|
||||
|
||||
When you need to stop and remove the container, use the following command:
|
||||
When you need to stop and remove a container, use the following commands:
|
||||
|
||||
```bash
|
||||
# Stop
|
||||
|
|
@ -439,17 +442,17 @@ docker rm douyin_tiktok_api
|
|||
|
||||
## 📸Screenshot
|
||||
|
||||
**_API speed test (compare the official API)_**
|
||||
**_API speed test (compared to official API)_**
|
||||
|
||||
<details><summary>🔎点击展开截图</summary>
|
||||
|
||||
TikTok official API:
|
||||
Douyin official API:
|
||||
|
||||
This project API:
|
||||
API of this project:
|
||||
|
||||
TikTok official API:
|
||||
|
||||
This project API:
|
||||
API of this project:
|
||||
|
||||
</details>
|
||||
<hr>
|
||||
|
|
|
|||
57
README.md
57
README.md
|
|
@ -35,26 +35,23 @@
|
|||
|
||||
这些赞助商已付费放置在这里，**Douyin_TikTok_Download_API** 项目将永远免费且开源。如果您希望成为该项目的赞助商，请查看我的 [GitHub 赞助商页面](https://github.com/sponsors/evil0ctal)。
|
||||
|
||||
<div align="center">
|
||||
<a href="https://www.tikhub.io/" target="_blank">
|
||||
<img src="https://tikhub.io/wp-content/uploads/2024/11/Main-Logo.webp" width="100" alt="TikHub.io - Global Social Data & API Marketplace">
|
||||
</a>
|
||||
<div>
|
||||
<h2><b>TikHub.io</b></h2>
|
||||
<p>Your Ultimate Social Media Data & API Marketplace</p>
|
||||
<p>
|
||||
Professional data solutions for Douyin, Xiaohongshu, TikTok, Instagram, YouTube,
|
||||
Twitter, and more.<br>
|
||||
Real-time Data | Flexible APIs | Seamless Integration | Competitive Pricing with Discounts
|
||||
</p>
|
||||
<p>
|
||||
<b>Discover TikHub.io Marketplace</b><br>
|
||||
Buy and sell custom APIs, services, and social media solutions.<br>
|
||||
Join a thriving ecosystem of developers, businesses, and content creators.
|
||||
</p>
|
||||
<p><em>Trusted by leading global influencer marketing and social media intelligence platforms</em></p>
|
||||
</div>
|
||||
</div>
|
||||
<p align="center">
|
||||
<a href="https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad">
|
||||
<img style="border-radius:20px" width="845" height="845" alt="TikHub IO_Banner zh" src="https://github.com/user-attachments/assets/18ce4674-83b3-4312-a5d8-a45cf7cef7b2">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
[TikHub](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad) 提供超过 700 个端点，可用于从 14+ 个社交媒体平台获取与分析数据 —— 包括视频、用户、评论、商店、商品与趋势等，一站式完成所有数据访问与分析。
|
||||
|
||||
通过每日签到，可以获取免费额度。可以使用我的注册邀请链接：[https://user.tikhub.io/users/signup?referral_code=1wRL8eQk](https://user.tikhub.io/users/signup?referral_code=1wRL8eQk&utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad) 或 邀请码：`1wRL8eQk`，注册并充值即可获得`$2`额度。
|
||||
|
||||
[TikHub](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad) 提供以下服务：
|
||||
|
||||
- 丰富的数据接口
|
||||
- 每日签到免费获取额度
|
||||
- 高质量的API服务
|
||||
- 官网：[https://tikhub.io/](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)
|
||||
- GitHub地址：[https://github.com/TikHubIO/](https://github.com/TikHubIO/)
|
||||
|
||||
|
||||
## 👻介绍
|
||||
|
|
@ -78,7 +75,9 @@
|
|||
|
||||
## 🔖TikHub.io API
|
||||
|
||||
[TikHub.io](https://api.tikhub.io/)是一个API平台，提供包括Douyin、TikTok在内的各种公开数据接口，如果您想支持 [Douyin_TikTok_Download_API](https://github.com/Evil0ctal/Douyin_TikTok_Download_API) 项目的开发，我们强烈建议您选择[TikHub.io](https://api.tikhub.io/)。
|
||||
[TikHub.io](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad) 提供超过 700 个端点，可用于从 14+ 个社交媒体平台获取与分析数据 —— 包括视频、用户、评论、商店、商品与趋势等，一站式完成所有数据访问与分析。
|
||||
|
||||
如果您想支持 [Douyin_TikTok_Download_API](https://github.com/Evil0ctal/Douyin_TikTok_Download_API) 项目的开发，我们强烈建议您选择 [TikHub.io](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)。
|
||||
|
||||
#### 特点：
|
||||
|
||||
|
|
@ -88,7 +87,7 @@
|
|||
|
||||
> 💰 成本优势
|
||||
|
||||
不预设套餐限制，没有月度使用门槛，所有消费按实际使用量即时计费，并且根据用户每日的请求量进行阶梯式计费，同时可以通过每日签到在用户后台进行签到获取免费的额度，并且这些免费额度不会过期。
|
||||
不预设套餐限制，没有月度使用门槛，所有消费按实际使用量即时计费，并且根据用户每日的请求量进行阶梯式计费，同时可以通过每日签到在用户后台获取免费的额度，并且这些免费额度不会过期。
|
||||
|
||||
> ⚡️ 快速支持
|
||||
|
||||
|
|
@ -98,12 +97,16 @@
|
|||
|
||||
TikHub的部分源代码会开源在Github上，并且会赞助一些开源项目的作者。
|
||||
|
||||
#### 链接：
|
||||
#### 注册与使用：
|
||||
|
||||
- Github: [TikHub Github](https://github.com/TikHubIO)
|
||||
- Discord: [TikHub Discord](https://discord.com/invite/aMEAS8Xsvz)
|
||||
- Register: [TikHub signup](https://beta-web.tikhub.io/en-us/users/signup)
|
||||
- API Docs: [TikHub API Docs](https://api.tikhub.io/)
|
||||
通过每日签到，可以获取免费额度。可以使用我的注册邀请链接：[https://user.tikhub.io/users/signup?referral_code=1wRL8eQk](https://user.tikhub.io/users/signup?referral_code=1wRL8eQk&utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad) 或 邀请码：`1wRL8eQk`，注册并充值即可获得`$2`额度。
|
||||
|
||||
#### 相关链接：
|
||||
|
||||
- 官网：[https://tikhub.io/](https://tikhub.io/?utm_source=github.com/Evil0ctal/Douyin_TikTok_Download_API&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad)
|
||||
- API 文档：[https://api.tikhub.io/docs](https://api.tikhub.io/docs)
|
||||
- GitHub：[https://github.com/TikHubIO/](https://github.com/TikHubIO/)
|
||||
- Discord：[https://discord.com/invite/aMEAS8Xsvz](https://discord.com/invite/aMEAS8Xsvz)
|
||||
|
||||
## 🖥演示站点: 我很脆弱...请勿压测(·•᷄ࡇ•᷅ ）
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
import os
|
||||
import zipfile
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
import aiofiles
|
||||
import httpx
|
||||
|
|
@ -48,42 +50,102 @@ async def fetch_data_stream(url: str, request:Request , headers: dict = None, fi
|
|||
await out_file.write(chunk)
|
||||
return True
|
||||
|
||||
@router.get("/download", summary="在线下载抖音|TikTok视频/图片/Online download Douyin|TikTok video/image")
|
||||
async def merge_bilibili_video_audio(video_url: str, audio_url: str, request: Request, output_path: str, headers: dict) -> bool:
|
||||
"""
|
||||
下载并合并 Bilibili 的视频流和音频流
|
||||
"""
|
||||
try:
|
||||
# 创建临时文件
|
||||
with tempfile.NamedTemporaryFile(suffix='.m4v', delete=False) as video_temp:
|
||||
video_temp_path = video_temp.name
|
||||
with tempfile.NamedTemporaryFile(suffix='.m4a', delete=False) as audio_temp:
|
||||
audio_temp_path = audio_temp.name
|
||||
|
||||
# 下载视频流
|
||||
video_success = await fetch_data_stream(video_url, request, headers=headers, file_path=video_temp_path)
|
||||
# 下载音频流
|
||||
audio_success = await fetch_data_stream(audio_url, request, headers=headers, file_path=audio_temp_path)
|
||||
|
||||
if not video_success or not audio_success:
|
||||
print("Failed to download video or audio stream")
|
||||
return False
|
||||
|
||||
# 使用 FFmpeg 合并视频和音频
|
||||
ffmpeg_cmd = [
|
||||
'ffmpeg', '-y', # -y 覆盖输出文件
|
||||
'-i', video_temp_path, # 视频输入
|
||||
'-i', audio_temp_path, # 音频输入
|
||||
'-c:v', 'copy', # 复制视频编码，不重新编码
|
||||
'-c:a', 'copy', # 复制音频编码，不重新编码（保持原始质量）
|
||||
'-f', 'mp4', # 确保输出格式为MP4
|
||||
output_path
|
||||
]
|
||||
|
||||
print(f"FFmpeg command: {' '.join(ffmpeg_cmd)}")
|
||||
result = subprocess.run(ffmpeg_cmd, capture_output=True, text=True)
|
||||
print(f"FFmpeg return code: {result.returncode}")
|
||||
if result.stderr:
|
||||
print(f"FFmpeg stderr: {result.stderr}")
|
||||
if result.stdout:
|
||||
print(f"FFmpeg stdout: {result.stdout}")
|
||||
|
||||
# 清理临时文件
|
||||
try:
|
||||
os.unlink(video_temp_path)
|
||||
os.unlink(audio_temp_path)
|
||||
except:
|
||||
pass
|
||||
|
||||
return result.returncode == 0
|
||||
|
||||
except Exception as e:
|
||||
# 清理临时文件
|
||||
try:
|
||||
os.unlink(video_temp_path)
|
||||
os.unlink(audio_temp_path)
|
||||
except:
|
||||
pass
|
||||
print(f"Error merging video and audio: {e}")
|
||||
return False
|
||||
|
||||
@router.get("/download", summary="在线下载抖音|TikTok|Bilibili视频/图片/Online download Douyin|TikTok|Bilibili video/image")
|
||||
async def download_file_hybrid(request: Request,
|
||||
url: str = Query(
|
||||
example="https://www.douyin.com/video/7372484719365098803",
|
||||
description="视频或图片的URL地址，也支持抖音|TikTok的分享链接，例如：https://v.douyin.com/e4J8Q7A/"),
|
||||
description="视频或图片的URL地址，支持抖音|TikTok|Bilibili的分享链接，例如：https://v.douyin.com/e4J8Q7A/ 或 https://www.bilibili.com/video/BV1xxxxxxxxx"),
|
||||
prefix: bool = True,
|
||||
with_watermark: bool = False):
|
||||
"""
|
||||
# [中文]
|
||||
### 用途:
|
||||
- 在线下载抖音|TikTok 无水印或有水印的视频/图片
|
||||
- 在线下载抖音|TikTok|Bilibili 无水印或有水印的视频/图片
|
||||
- 通过传入的视频URL参数，获取对应的视频或图片数据，然后下载到本地。
|
||||
- 如果你在尝试直接访问TikTok单一视频接口的JSON数据中的视频播放地址时遇到HTTP403错误，那么你可以使用此接口来下载视频。
|
||||
- Bilibili视频会自动合并视频流和音频流，确保下载的视频有声音。
|
||||
- 这个接口会占用一定的服务器资源，所以在Demo站点是默认关闭的，你可以在本地部署后调用此接口。
|
||||
### 参数:
|
||||
- url: 视频或图片的URL地址，也支持抖音|TikTok的分享链接，例如：https://v.douyin.com/e4J8Q7A/。
|
||||
- url: 视频或图片的URL地址，支持抖音|TikTok|Bilibili的分享链接，例如：https://v.douyin.com/e4J8Q7A/ 或 https://www.bilibili.com/video/BV1xxxxxxxxx
|
||||
- prefix: 下载文件的前缀，默认为True，可以在配置文件中修改。
|
||||
- with_watermark: 是否下载带水印的视频或图片，默认为False。
|
||||
- with_watermark: 是否下载带水印的视频或图片，默认为False。(注意：Bilibili没有水印概念)
|
||||
### 返回:
|
||||
- 返回下载的视频或图片文件响应。
|
||||
|
||||
# [English]
|
||||
### Purpose:
|
||||
- Download Douyin|TikTok video/image with or without watermark online.
|
||||
- Download Douyin|TikTok|Bilibili video/image with or without watermark online.
|
||||
- By passing the video URL parameter, get the corresponding video or image data, and then download it to the local.
|
||||
- If you encounter an HTTP403 error when trying to access the video playback address in the JSON data of the TikTok single video interface directly, you can use this interface to download the video.
|
||||
- Bilibili videos will automatically merge video and audio streams to ensure downloaded videos have sound.
|
||||
- This interface will occupy a certain amount of server resources, so it is disabled by default on the Demo site, you can call this interface after deploying it locally.
|
||||
### Parameters:
|
||||
- url: The URL address of the video or image, also supports Douyin|TikTok sharing links, for example: https://v.douyin.com/e4J8Q7A/.
|
||||
- url: The URL address of the video or image, supports Douyin|TikTok|Bilibili sharing links, for example: https://v.douyin.com/e4J8Q7A/ or https://www.bilibili.com/video/BV1xxxxxxxxx
|
||||
- prefix: The prefix of the downloaded file, the default is True, and can be modified in the configuration file.
|
||||
- with_watermark: Whether to download videos or images with watermarks, the default is False.
|
||||
- with_watermark: Whether to download videos or images with watermarks, the default is False. (Note: Bilibili has no watermark concept)
|
||||
### Returns:
|
||||
- Return the response of the downloaded video or image file.
|
||||
|
||||
# [示例/Example]
|
||||
url: https://www.douyin.com/video/7372484719365098803
|
||||
url: https://www.bilibili.com/video/BV1U5efz2Egn
|
||||
"""
|
||||
# 是否开启此端点/Whether to enable this endpoint
|
||||
if not config["API"]["Download_Switch"]:
|
||||
|
|
@ -103,7 +165,7 @@ async def download_file_hybrid(request: Request,
|
|||
try:
|
||||
data_type = data.get('type')
|
||||
platform = data.get('platform')
|
||||
aweme_id = data.get('aweme_id')
|
||||
video_id = data.get('video_id') # 改为使用video_id
|
||||
file_prefix = config.get("API").get("Download_File_Prefix") if prefix else ''
|
||||
download_path = os.path.join(config.get("API").get("Download_Path"), f"{platform}_{data_type}")
|
||||
|
||||
|
|
@ -112,25 +174,48 @@ async def download_file_hybrid(request: Request,
|
|||
|
||||
# 下载视频文件/Download video file
|
||||
if data_type == 'video':
|
||||
file_name = f"{file_prefix}{platform}_{aweme_id}.mp4" if not with_watermark else f"{file_prefix}{platform}_{aweme_id}_watermark.mp4"
|
||||
url = data.get('video_data').get('nwm_video_url_HQ') if not with_watermark else data.get('video_data').get(
|
||||
'wm_video_url_HQ')
|
||||
file_name = f"{file_prefix}{platform}_{video_id}.mp4" if not with_watermark else f"{file_prefix}{platform}_{video_id}_watermark.mp4"
|
||||
file_path = os.path.join(download_path, file_name)
|
||||
|
||||
# 判断文件是否存在，存在就直接返回
|
||||
if os.path.exists(file_path):
|
||||
return FileResponse(path=file_path, media_type='video/mp4', filename=file_name)
|
||||
|
||||
# 获取视频文件
|
||||
__headers = await HybridCrawler.TikTokWebCrawler.get_tiktok_headers() if platform == 'tiktok' else await HybridCrawler.DouyinWebCrawler.get_douyin_headers()
|
||||
# response = await fetch_data(url, headers=__headers)
|
||||
# 获取对应平台的headers
|
||||
if platform == 'tiktok':
|
||||
__headers = await HybridCrawler.TikTokWebCrawler.get_tiktok_headers()
|
||||
elif platform == 'bilibili':
|
||||
__headers = await HybridCrawler.BilibiliWebCrawler.get_bilibili_headers()
|
||||
else: # douyin
|
||||
__headers = await HybridCrawler.DouyinWebCrawler.get_douyin_headers()
|
||||
|
||||
success = await fetch_data_stream(url, request, headers=__headers, file_path=file_path)
|
||||
if not success:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="An error occurred while fetching data"
|
||||
)
|
||||
# Bilibili 特殊处理：音视频分离
|
||||
if platform == 'bilibili':
|
||||
video_data = data.get('video_data', {})
|
||||
video_url = video_data.get('nwm_video_url_HQ') if not with_watermark else video_data.get('wm_video_url_HQ')
|
||||
audio_url = video_data.get('audio_url')
|
||||
if not video_url or not audio_url:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="Failed to get video or audio URL from Bilibili"
|
||||
)
|
||||
|
||||
# 使用专门的函数合并音视频
|
||||
success = await merge_bilibili_video_audio(video_url, audio_url, request, file_path, __headers.get('headers'))
|
||||
if not success:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="Failed to merge Bilibili video and audio streams"
|
||||
)
|
||||
else:
|
||||
# 其他平台的常规处理
|
||||
url = data.get('video_data').get('nwm_video_url_HQ') if not with_watermark else data.get('video_data').get('wm_video_url_HQ')
|
||||
success = await fetch_data_stream(url, request, headers=__headers, file_path=file_path)
|
||||
if not success:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="An error occurred while fetching data"
|
||||
)
|
||||
|
||||
# # 保存文件
|
||||
# async with aiofiles.open(file_path, 'wb') as out_file:
|
||||
|
|
@ -142,7 +227,7 @@ async def download_file_hybrid(request: Request,
|
|||
# 下载图片文件/Download image file
|
||||
elif data_type == 'image':
|
||||
# 压缩文件属性/Compress file properties
|
||||
zip_file_name = f"{file_prefix}{platform}_{aweme_id}_images.zip" if not with_watermark else f"{file_prefix}{platform}_{aweme_id}_images_watermark.zip"
|
||||
zip_file_name = f"{file_prefix}{platform}_{video_id}_images.zip" if not with_watermark else f"{file_prefix}{platform}_{video_id}_images_watermark.zip"
|
||||
zip_file_path = os.path.join(download_path, zip_file_name)
|
||||
|
||||
# 判断文件是否存在，存在就直接返回、
|
||||
|
|
@ -159,7 +244,7 @@ async def download_file_hybrid(request: Request,
|
|||
index = int(urls.index(url))
|
||||
content_type = response.headers.get('content-type')
|
||||
file_format = content_type.split('/')[1]
|
||||
file_name = f"{file_prefix}{platform}_{aweme_id}_{index + 1}.{file_format}" if not with_watermark else f"{file_prefix}{platform}_{aweme_id}_{index + 1}_watermark.{file_format}"
|
||||
file_name = f"{file_prefix}{platform}_{video_id}_{index + 1}.{file_format}" if not with_watermark else f"{file_prefix}{platform}_{video_id}_{index + 1}_watermark.{file_format}"
|
||||
file_path = os.path.join(download_path, file_name)
|
||||
image_file_list.append(file_path)
|
||||
|
||||
|
|
|
|||
|
|
@ -51,3 +51,67 @@ async def hybrid_parsing_single_video(request: Request,
|
|||
params=dict(request.query_params),
|
||||
)
|
||||
raise HTTPException(status_code=status_code, detail=detail.dict())
|
||||
|
||||
# 更新Cookie
|
||||
@router.post("/update_cookie",
|
||||
response_model=ResponseModel,
|
||||
summary="更新Cookie/Update Cookie")
|
||||
async def update_cookie_api(request: Request,
|
||||
service: str = Body(example="douyin", description="服务名称/Service name"),
|
||||
cookie: str = Body(example="YOUR_NEW_COOKIE", description="新的Cookie值/New Cookie value")):
|
||||
"""
|
||||
# [中文]
|
||||
### 用途:
|
||||
- 更新指定服务的Cookie
|
||||
### 参数:
|
||||
- service: 服务名称 (如: douyin_web)
|
||||
- cookie: 新的Cookie值
|
||||
### 返回:
|
||||
- 更新结果
|
||||
|
||||
# [English]
|
||||
### Purpose:
|
||||
- Update Cookie for specified service
|
||||
### Parameters:
|
||||
- service: Service name (e.g.: douyin_web)
|
||||
- cookie: New Cookie value
|
||||
### Return:
|
||||
- Update result
|
||||
|
||||
# [示例/Example]
|
||||
service = "douyin_web"
|
||||
cookie = "YOUR_NEW_COOKIE"
|
||||
"""
|
||||
try:
|
||||
if service == "douyin":
|
||||
from crawlers.douyin.web.web_crawler import DouyinWebCrawler
|
||||
douyin_crawler = DouyinWebCrawler()
|
||||
await douyin_crawler.update_cookie(cookie)
|
||||
return ResponseModel(code=200,
|
||||
router=request.url.path,
|
||||
data={"message": f"Cookie for {service} updated successfully"})
|
||||
elif service == "tiktok":
|
||||
# 这里可以添加TikTok的cookie更新逻辑
|
||||
# from crawlers.tiktok.web.web_crawler import TikTokWebCrawler
|
||||
# tiktok_crawler = TikTokWebCrawler()
|
||||
# await tiktok_crawler.update_cookie(cookie)
|
||||
return ResponseModel(code=200,
|
||||
router=request.url.path,
|
||||
data={"message": f"Cookie for {service} will be updated (not implemented yet)"})
|
||||
elif service == "bilibili":
|
||||
# 这里可以添加Bilibili的cookie更新逻辑
|
||||
# from crawlers.bilibili.web.web_crawler import BilibiliWebCrawler
|
||||
# bilibili_crawler = BilibiliWebCrawler()
|
||||
# await bilibili_crawler.update_cookie(cookie)
|
||||
return ResponseModel(code=200,
|
||||
router=request.url.path,
|
||||
data={"message": f"Cookie for {service} will be updated (not implemented yet)"})
|
||||
else:
|
||||
raise ValueError(f"Service '{service}' is not supported. Supported services: douyin, tiktok, bilibili")
|
||||
except Exception as e:
|
||||
status_code = 400
|
||||
detail = ErrorResponseModel(code=status_code,
|
||||
router=request.url.path,
|
||||
params=dict(request.query_params),
|
||||
)
|
||||
raise HTTPException(status_code=status_code, detail=detail.dict())
|
||||
171
chrome-cookie-sniffer/README.md
Normal file
171
chrome-cookie-sniffer/README.md
Normal file
|
|
@ -0,0 +1,171 @@
|
|||
# Chrome Cookie Sniffer
|
||||
|
||||
一个用于自动嗅探和提取网站Cookie的Chrome扩展程序。支持抖音等主流平台，具备智能去重、时间控制和Webhook回调等功能。
|
||||
|
||||
## 功能特性
|
||||
|
||||
- 🎯 **智能Cookie抓取** - 自动拦截POST/GET请求中的Cookie
|
||||
- ⏱️ **防重复机制** - 5分钟内不重复抓取相同服务
|
||||
- 🔄 **内容去重** - 只有Cookie内容变化时才保存
|
||||
- 🎨 **现代化界面** - Card列表展示，状态一目了然
|
||||
- 🔗 **Webhook回调** - Cookie更新时自动推送到指定地址
|
||||
- 📋 **一键复制** - 快速复制Cookie到剪贴板
|
||||
- 🗂️ **数据管理** - 支持导出、清理和单独删除
|
||||
- 🔧 **调试友好** - 内置Webhook测试功能
|
||||
|
||||
## 支持的网站
|
||||
|
||||
- 🎵 **抖音** (douyin.com)
|
||||
- 🚀 **扩展性** - 架构支持轻松添加更多平台
|
||||
|
||||
## 安装方法
|
||||
|
||||
### 1. 下载源码
|
||||
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
# 或直接下载ZIP文件并解压
|
||||
```
|
||||
|
||||
### 2. 在Chrome中加载扩展
|
||||
|
||||
1. **打开Chrome扩展管理页面**
|
||||
- 方法一：地址栏输入 `chrome://extensions/`
|
||||
- 方法二：菜单 → 更多工具 → 扩展程序
|
||||
|
||||
2. **启用开发者模式**
|
||||
- 在扩展管理页面右上角，开启"开发者模式"开关
|
||||
|
||||
3. **加载解压的扩展程序**
|
||||
- 点击"加载已解压的扩展程序"按钮
|
||||
- 选择 `chrome-cookie-sniffer` 文件夹
|
||||
- 确认加载
|
||||
|
||||
4. **验证安装**
|
||||
- 扩展列表中出现"Cookie Sniffer"
|
||||
- 浏览器工具栏出现扩展图标
|
||||
- 状态显示为"已启用"
|
||||
|
||||
### 3. 权限确认
|
||||
|
||||
安装时Chrome会请求以下权限：
|
||||
- `webRequest` - 拦截网络请求
|
||||
- `storage` - 本地数据存储
|
||||
- `cookies` - 读取Cookie信息
|
||||
- `activeTab` - 当前标签页访问
|
||||
- `host_permissions` - 访问douyin.com域名
|
||||
|
||||
## 使用方法
|
||||
|
||||
### 基础使用
|
||||
|
||||
1. **访问目标网站** - 打开抖音等支持的网站
|
||||
2. **触发请求** - 正常浏览，触发POST/GET请求
|
||||
3. **查看结果** - 点击扩展图标查看抓取的Cookie
|
||||
|
||||
### 配置Webhook
|
||||
|
||||
1. **打开扩展弹窗**
|
||||
2. **输入Webhook地址** - 在顶部输入框填入回调URL
|
||||
3. **测试连接** - 点击"🔧 测试"按钮验证
|
||||
4. **自动回调** - Cookie更新时自动POST到指定地址
|
||||
|
||||
### Webhook数据格式
|
||||
|
||||
```json
|
||||
{
|
||||
"service": "douyin",
|
||||
"cookie": "具体的Cookie字符串",
|
||||
"timestamp": "2025-08-29T12:34:56.789Z"
|
||||
}
|
||||
```
|
||||
|
||||
测试时会额外包含：
|
||||
```json
|
||||
{
|
||||
"test": true,
|
||||
"message": "这是一个测试回调..."
|
||||
}
|
||||
```
|
||||
|
||||
### 数据管理
|
||||
|
||||
- **📋 复制Cookie** - 点击卡片中的复制按钮
|
||||
- **🗑️ 删除数据** - 删除单个服务的Cookie
|
||||
- **🔄 刷新** - 手动刷新数据显示
|
||||
- **📤 导出** - 导出所有数据为JSON文件
|
||||
- **🧹 清空** - 清空所有Cookie数据
|
||||
|
||||
## 调试指南
|
||||
|
||||
### 查看日志
|
||||
|
||||
1. **打开扩展管理页面** (`chrome://extensions/`)
|
||||
2. **找到Cookie Sniffer扩展**
|
||||
3. **点击"服务工作进程"** - 查看蓝色链接
|
||||
4. **查看控制台输出** - 所有日志都在这里
|
||||
|
||||
### 常见问题
|
||||
|
||||
**Q: 扩展不工作？**
|
||||
- 检查是否启用开发者模式
|
||||
- 确认权限已正确授予
|
||||
- 查看service worker是否正在运行
|
||||
|
||||
**Q: 没有抓取到Cookie？**
|
||||
- 确认访问的是支持的网站
|
||||
- 检查是否触发了POST/GET请求
|
||||
- 查看service worker控制台日志
|
||||
|
||||
**Q: Webhook测试失败？**
|
||||
- 检查URL格式是否正确
|
||||
- 确认服务器支持跨域请求
|
||||
- 验证服务器是否正常响应
|
||||
|
||||
### 开发者选项
|
||||
|
||||
修改 `background.js` 中的 `SERVICES` 配置来添加新网站：
|
||||
|
||||
```javascript
|
||||
const SERVICES = {
|
||||
douyin: {
|
||||
name: 'douyin',
|
||||
displayName: '抖音',
|
||||
domains: ['douyin.com'],
|
||||
cookieDomain: '.douyin.com'
|
||||
},
|
||||
// 添加新服务
|
||||
bilibili: {
|
||||
name: 'bilibili',
|
||||
displayName: 'B站',
|
||||
domains: ['bilibili.com'],
|
||||
cookieDomain: '.bilibili.com'
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
## 文件结构
|
||||
|
||||
```
|
||||
chrome-cookie-sniffer/
|
||||
├── manifest.json # 扩展配置文件
|
||||
├── background.js # 后台服务脚本
|
||||
├── popup.html # 弹窗界面
|
||||
├── popup.js # 弹窗逻辑
|
||||
└── README.md # 说明文档
|
||||
```
|
||||
|
||||
## 注意事项
|
||||
|
||||
- ⚠️ **仅用于合法用途** - 请遵守网站服务条款
|
||||
- 🔒 **数据安全** - Cookie数据存储在本地，不会上传
|
||||
- 🔄 **定期更新** - 网站更新可能影响抓取效果
|
||||
- 📱 **Chrome限制** - 部分网站可能有反爬虫机制
|
||||
|
||||
## 开源协议
|
||||
|
||||
本项目遵循 MIT 开源协议。
|
||||
|
||||
## 贡献指南
|
||||
|
||||
欢迎提交Issue和Pull Request来改进这个项目！
|
||||
177
chrome-cookie-sniffer/background.js
Normal file
177
chrome-cookie-sniffer/background.js
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
// 启动时记录
|
||||
console.log('Cookie Sniffer service worker 已启动');
|
||||
|
||||
// 服务配置
|
||||
const SERVICES = {
|
||||
douyin: {
|
||||
name: 'douyin',
|
||||
displayName: '抖音',
|
||||
domains: ['douyin.com'],
|
||||
cookieDomain: '.douyin.com'
|
||||
}
|
||||
};
|
||||
|
||||
// 获取服务名称
|
||||
function getServiceFromUrl(url) {
|
||||
for (const [key, service] of Object.entries(SERVICES)) {
|
||||
if (service.domains.some(domain => url.includes(domain))) {
|
||||
return service;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// 检查是否在5分钟内抓取过
|
||||
async function shouldSkipCapture(serviceName) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.storage.local.get([`lastCapture_${serviceName}`], function(result) {
|
||||
const lastTime = result[`lastCapture_${serviceName}`];
|
||||
if (!lastTime) {
|
||||
resolve(false);
|
||||
return;
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const fiveMinutes = 5 * 60 * 1000;
|
||||
const shouldSkip = (now - lastTime) < fiveMinutes;
|
||||
|
||||
if (shouldSkip) {
|
||||
console.log(`${serviceName}: 5分钟内已抓取过，跳过`);
|
||||
}
|
||||
resolve(shouldSkip);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// 检查Cookie是否有变化
|
||||
async function isCookieChanged(serviceName, newCookie) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.storage.local.get([`cookieData_${serviceName}`], function(result) {
|
||||
const existingData = result[`cookieData_${serviceName}`];
|
||||
if (!existingData || existingData.cookie !== newCookie) {
|
||||
resolve(true);
|
||||
} else {
|
||||
console.log(`${serviceName}: Cookie内容无变化，跳过`);
|
||||
resolve(false);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// 保存Cookie数据
|
||||
async function saveCookieData(serviceName, url, cookie, source = 'headers') {
|
||||
const cookieData = {
|
||||
service: serviceName,
|
||||
url: url,
|
||||
timestamp: Date.now(),
|
||||
lastUpdate: new Date().toISOString(),
|
||||
cookie: cookie,
|
||||
source: source
|
||||
};
|
||||
|
||||
// 保存服务数据
|
||||
chrome.storage.local.set({
|
||||
[`cookieData_${serviceName}`]: cookieData,
|
||||
[`lastCapture_${serviceName}`]: Date.now()
|
||||
});
|
||||
|
||||
// 触发Webhook回调
|
||||
await sendWebhook(serviceName, cookie);
|
||||
|
||||
console.log(`${serviceName}: Cookie已保存`);
|
||||
}
|
||||
|
||||
// Webhook回调
|
||||
async function sendWebhook(serviceName, cookie) {
|
||||
chrome.storage.local.get(['webhookUrl'], function(result) {
|
||||
const webhookUrl = result.webhookUrl;
|
||||
if (webhookUrl && webhookUrl.trim()) {
|
||||
const payload = {
|
||||
service: serviceName,
|
||||
cookie: cookie,
|
||||
timestamp: new Date().toISOString()
|
||||
};
|
||||
|
||||
fetch(webhookUrl, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify(payload)
|
||||
}).then(response => {
|
||||
console.log(`Webhook回调成功: ${serviceName}`, response.status);
|
||||
}).catch(error => {
|
||||
console.error(`Webhook回调失败: ${serviceName}`, error);
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
chrome.webRequest.onBeforeSendHeaders.addListener(
|
||||
async function(details) {
|
||||
const service = getServiceFromUrl(details.url);
|
||||
if (!service) return;
|
||||
|
||||
console.log(`请求拦截: ${service.displayName}`, details.url, details.method);
|
||||
|
||||
if (details.method === "POST" || details.method === "GET") {
|
||||
// 检查5分钟限制
|
||||
if (await shouldSkipCapture(service.name)) {
|
||||
return;
|
||||
}
|
||||
|
||||
let cookieFound = false;
|
||||
|
||||
// 尝试从请求头获取Cookie
|
||||
if (details.requestHeaders) {
|
||||
for (let header of details.requestHeaders) {
|
||||
if (header.name.toLowerCase() === "cookie") {
|
||||
console.log(`从请求头捕获到Cookie: ${service.displayName}`);
|
||||
|
||||
// 检查Cookie是否有变化
|
||||
if (await isCookieChanged(service.name, header.value)) {
|
||||
await saveCookieData(service.name, details.url, header.value, 'headers');
|
||||
}
|
||||
|
||||
cookieFound = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 如果请求头没有Cookie，使用cookies API备用方案
|
||||
if (!cookieFound) {
|
||||
chrome.cookies.getAll({domain: service.cookieDomain}, async function(cookies) {
|
||||
if (cookies && cookies.length > 0) {
|
||||
console.log(`通过cookies API获取到: ${service.displayName}`, cookies.length, '个cookie');
|
||||
const cookieString = cookies.map(c => `${c.name}=${c.value}`).join('; ');
|
||||
|
||||
// 检查Cookie是否有变化
|
||||
if (await isCookieChanged(service.name, cookieString)) {
|
||||
await saveCookieData(service.name, details.url, cookieString, 'cookies_api');
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
},
|
||||
{ urls: ["https://*.douyin.com/*", "https://douyin.com/*"] },
|
||||
["requestHeaders", "extraHeaders"]
|
||||
);
|
||||
|
||||
// 添加存储变化监听
|
||||
chrome.storage.onChanged.addListener((changes, areaName) => {
|
||||
if (areaName === 'local') {
|
||||
// 监听服务数据变化
|
||||
Object.keys(changes).forEach(key => {
|
||||
if (key.startsWith('cookieData_')) {
|
||||
const serviceName = key.replace('cookieData_', '');
|
||||
const serviceConfig = SERVICES[serviceName];
|
||||
if (serviceConfig && changes[key].newValue) {
|
||||
console.log(`${serviceConfig.displayName} Cookie数据已更新`);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
24
chrome-cookie-sniffer/manifest.json
Normal file
24
chrome-cookie-sniffer/manifest.json
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
{
|
||||
"manifest_version": 3,
|
||||
"name": "Cookie Sniffer",
|
||||
"version": "1.0",
|
||||
"description": "监听并获取指定网站的请求 Cookie",
|
||||
"permissions": [
|
||||
"webRequest",
|
||||
"storage",
|
||||
"activeTab",
|
||||
"cookies"
|
||||
],
|
||||
"host_permissions": [
|
||||
"https://*.douyin.com/*",
|
||||
"https://douyin.com/*"
|
||||
],
|
||||
"background": {
|
||||
"service_worker": "background.js"
|
||||
},
|
||||
"action": {
|
||||
"default_popup": "popup.html",
|
||||
"default_title": "Cookie Sniffer"
|
||||
}
|
||||
}
|
||||
|
||||
178
chrome-cookie-sniffer/popup.html
Normal file
178
chrome-cookie-sniffer/popup.html
Normal file
|
|
@ -0,0 +1,178 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<style>
|
||||
body {
|
||||
width: 400px;
|
||||
padding: 15px;
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
margin: 0;
|
||||
background: #f8f9fa;
|
||||
}
|
||||
|
||||
.header {
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
|
||||
.header h3 {
|
||||
margin: 0 0 15px 0;
|
||||
color: #2c3e50;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.webhook-config {
|
||||
background: white;
|
||||
border-radius: 8px;
|
||||
padding: 12px;
|
||||
margin-bottom: 15px;
|
||||
border: 1px solid #e1e8ed;
|
||||
}
|
||||
|
||||
.webhook-config label {
|
||||
display: block;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
margin-bottom: 5px;
|
||||
}
|
||||
|
||||
.webhook-config input {
|
||||
width: 100%;
|
||||
padding: 8px;
|
||||
border: 1px solid #ddd;
|
||||
border-radius: 4px;
|
||||
font-size: 13px;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
.controls {
|
||||
text-align: center;
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
|
||||
.btn {
|
||||
padding: 6px 12px;
|
||||
margin: 0 3px;
|
||||
border: none;
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
transition: background-color 0.2s;
|
||||
}
|
||||
|
||||
.btn-primary { background: #007bff; color: white; }
|
||||
.btn-primary:hover { background: #0056b3; }
|
||||
.btn-danger { background: #dc3545; color: white; }
|
||||
.btn-danger:hover { background: #c82333; }
|
||||
.btn-success { background: #28a745; color: white; }
|
||||
.btn-success:hover { background: #1e7e34; }
|
||||
|
||||
.service-card {
|
||||
background: white;
|
||||
border-radius: 8px;
|
||||
margin-bottom: 12px;
|
||||
border: 1px solid #e1e8ed;
|
||||
overflow: hidden;
|
||||
transition: box-shadow 0.2s;
|
||||
}
|
||||
|
||||
.service-card:hover {
|
||||
box-shadow: 0 2px 8px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
.card-header {
|
||||
padding: 12px 15px;
|
||||
border-bottom: 1px solid #e1e8ed;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.service-name {
|
||||
font-weight: 600;
|
||||
color: #2c3e50;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.service-status {
|
||||
font-size: 11px;
|
||||
padding: 2px 6px;
|
||||
border-radius: 10px;
|
||||
color: white;
|
||||
}
|
||||
|
||||
.status-active { background: #28a745; }
|
||||
.status-inactive { background: #6c757d; }
|
||||
|
||||
.card-body {
|
||||
padding: 12px 15px;
|
||||
}
|
||||
|
||||
.last-update {
|
||||
color: #666;
|
||||
font-size: 12px;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
.actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.btn-sm {
|
||||
padding: 4px 8px;
|
||||
font-size: 11px;
|
||||
}
|
||||
|
||||
.empty-state {
|
||||
text-align: center;
|
||||
color: #6c757d;
|
||||
font-style: italic;
|
||||
padding: 40px 20px;
|
||||
}
|
||||
|
||||
.status-info {
|
||||
background: white;
|
||||
border-radius: 8px;
|
||||
padding: 10px;
|
||||
margin-bottom: 15px;
|
||||
border-left: 4px solid #17a2b8;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<h3>Cookie Sniffer</h3>
|
||||
|
||||
<div class="webhook-config">
|
||||
<label>Webhook回调地址</label>
|
||||
<div style="display: flex; gap: 8px; align-items: center;">
|
||||
<input type="url" id="webhookUrl" placeholder="https://your-server.com/webhook" style="flex: 1;">
|
||||
<button class="btn btn-sm" id="testWebhook" style="background: #17a2b8; color: white; white-space: nowrap;" disabled>
|
||||
🔧 测试
|
||||
</button>
|
||||
</div>
|
||||
<div id="webhookStatus" style="font-size: 11px; color: #666; margin-top: 4px; min-height: 14px;"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="controls">
|
||||
<button class="btn btn-primary" id="refresh">刷新</button>
|
||||
<button class="btn btn-danger" id="clear">清空所有</button>
|
||||
<button class="btn btn-success" id="export">导出JSON</button>
|
||||
</div>
|
||||
|
||||
<div id="statusInfo" class="status-info" style="display: none;"></div>
|
||||
|
||||
<div id="serviceCards"></div>
|
||||
|
||||
<div id="emptyState" class="empty-state" style="display: none;">
|
||||
暂未抓取到任何Cookie数据<br>
|
||||
请访问相关网站触发请求
|
||||
</div>
|
||||
|
||||
<script src="popup.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
292
chrome-cookie-sniffer/popup.js
Normal file
292
chrome-cookie-sniffer/popup.js
Normal file
|
|
@ -0,0 +1,292 @@
|
|||
document.addEventListener('DOMContentLoaded', function() {
|
||||
const refreshBtn = document.getElementById('refresh');
|
||||
const clearBtn = document.getElementById('clear');
|
||||
const exportBtn = document.getElementById('export');
|
||||
const webhookInput = document.getElementById('webhookUrl');
|
||||
const testWebhookBtn = document.getElementById('testWebhook');
|
||||
const webhookStatus = document.getElementById('webhookStatus');
|
||||
const statusInfo = document.getElementById('statusInfo');
|
||||
const serviceCards = document.getElementById('serviceCards');
|
||||
const emptyState = document.getElementById('emptyState');
|
||||
|
||||
// 服务配置
|
||||
const SERVICES = {
|
||||
douyin: { name: 'douyin', displayName: '抖音', icon: '🎵' }
|
||||
};
|
||||
|
||||
// 加载Webhook配置
|
||||
function loadWebhookConfig() {
|
||||
chrome.storage.local.get(['webhookUrl'], function(result) {
|
||||
if (result.webhookUrl) {
|
||||
webhookInput.value = result.webhookUrl;
|
||||
}
|
||||
updateTestButtonState();
|
||||
});
|
||||
}
|
||||
|
||||
// 保存Webhook配置
|
||||
function saveWebhookConfig() {
|
||||
const url = webhookInput.value.trim();
|
||||
chrome.storage.local.set({ webhookUrl: url });
|
||||
showStatusInfo('Webhook地址已保存');
|
||||
updateTestButtonState();
|
||||
}
|
||||
|
||||
// 更新测试按钮状态
|
||||
function updateTestButtonState() {
|
||||
const url = webhookInput.value.trim();
|
||||
testWebhookBtn.disabled = !url || !isValidUrl(url);
|
||||
}
|
||||
|
||||
// 验证URL格式
|
||||
function isValidUrl(string) {
|
||||
try {
|
||||
new URL(string);
|
||||
return string.startsWith('http://') || string.startsWith('https://');
|
||||
} catch (_) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// 测试Webhook回调
|
||||
async function testWebhook() {
|
||||
const url = webhookInput.value.trim();
|
||||
if (!url) {
|
||||
webhookStatus.textContent = '请先输入Webhook地址';
|
||||
webhookStatus.style.color = '#dc3545';
|
||||
return;
|
||||
}
|
||||
|
||||
testWebhookBtn.disabled = true;
|
||||
testWebhookBtn.textContent = '⏳ 测试中...';
|
||||
webhookStatus.textContent = '正在发送测试请求...';
|
||||
webhookStatus.style.color = '#17a2b8';
|
||||
|
||||
// 获取现有数据或创建测试数据
|
||||
chrome.storage.local.get(['cookieData_douyin'], async function(result) {
|
||||
let testData;
|
||||
|
||||
if (result.cookieData_douyin) {
|
||||
// 使用现有数据
|
||||
testData = {
|
||||
service: 'douyin',
|
||||
cookie: result.cookieData_douyin.cookie,
|
||||
timestamp: new Date().toISOString(),
|
||||
test: true,
|
||||
message: '这是一个测试回调，使用了真实的Cookie数据'
|
||||
};
|
||||
} else {
|
||||
// 使用模拟数据
|
||||
testData = {
|
||||
service: 'douyin',
|
||||
cookie: 'test_cookie=test_value; another_cookie=another_value',
|
||||
timestamp: new Date().toISOString(),
|
||||
test: true,
|
||||
message: '这是一个测试回调，使用了模拟Cookie数据'
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify(testData)
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
webhookStatus.textContent = `✅ 测试成功 (${response.status})`;
|
||||
webhookStatus.style.color = '#28a745';
|
||||
} else {
|
||||
webhookStatus.textContent = `❌ 服务器错误 (${response.status})`;
|
||||
webhookStatus.style.color = '#dc3545';
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Webhook测试失败:', error);
|
||||
if (error.name === 'TypeError' && error.message.includes('fetch')) {
|
||||
webhookStatus.textContent = '❌ 网络错误或跨域限制';
|
||||
} else {
|
||||
webhookStatus.textContent = `❌ 请求失败: ${error.message}`;
|
||||
}
|
||||
webhookStatus.style.color = '#dc3545';
|
||||
} finally {
|
||||
testWebhookBtn.disabled = false;
|
||||
testWebhookBtn.textContent = '🔧 测试';
|
||||
updateTestButtonState();
|
||||
|
||||
// 5秒后清除状态信息
|
||||
setTimeout(() => {
|
||||
webhookStatus.textContent = '';
|
||||
}, 5000);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 显示状态信息
|
||||
function showStatusInfo(message) {
|
||||
statusInfo.textContent = message;
|
||||
statusInfo.style.display = 'block';
|
||||
setTimeout(() => {
|
||||
statusInfo.style.display = 'none';
|
||||
}, 3000);
|
||||
}
|
||||
|
||||
// 加载服务数据
|
||||
function loadServiceData() {
|
||||
const serviceKeys = Object.keys(SERVICES).map(service => `cookieData_${service}`);
|
||||
chrome.storage.local.get(serviceKeys, function(result) {
|
||||
const hasData = Object.keys(result).length > 0;
|
||||
|
||||
if (!hasData) {
|
||||
serviceCards.innerHTML = '';
|
||||
emptyState.style.display = 'block';
|
||||
return;
|
||||
}
|
||||
|
||||
emptyState.style.display = 'none';
|
||||
serviceCards.innerHTML = '';
|
||||
|
||||
Object.keys(SERVICES).forEach(serviceKey => {
|
||||
const service = SERVICES[serviceKey];
|
||||
const data = result[`cookieData_${serviceKey}`];
|
||||
|
||||
if (data) {
|
||||
createServiceCard(service, data);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// 创建服务卡片
|
||||
function createServiceCard(service, data) {
|
||||
const card = document.createElement('div');
|
||||
card.className = 'service-card';
|
||||
|
||||
const isRecent = Date.now() - data.timestamp < 5 * 60 * 1000; // 5分钟内
|
||||
const lastUpdate = new Date(data.lastUpdate).toLocaleString();
|
||||
|
||||
card.innerHTML = `
|
||||
<div class="card-header">
|
||||
<div class="service-name">${service.icon} ${service.displayName}</div>
|
||||
<div class="service-status ${isRecent ? 'status-active' : 'status-inactive'}">
|
||||
${isRecent ? '活跃' : '休眠'}
|
||||
</div>
|
||||
</div>
|
||||
<div class="card-body">
|
||||
<div class="last-update">上次更新: ${lastUpdate}</div>
|
||||
<div class="actions">
|
||||
<button class="btn btn-primary btn-sm copy-btn" data-service="${service.name}">
|
||||
📋 复制Cookie
|
||||
</button>
|
||||
<button class="btn btn-danger btn-sm delete-btn" data-service="${service.name}">
|
||||
🗑️ 删除
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
serviceCards.appendChild(card);
|
||||
}
|
||||
|
||||
// 复制Cookie到剪贴板
|
||||
async function copyCookie(serviceName) {
|
||||
chrome.storage.local.get([`cookieData_${serviceName}`], async function(result) {
|
||||
const data = result[`cookieData_${serviceName}`];
|
||||
if (data && data.cookie) {
|
||||
try {
|
||||
await navigator.clipboard.writeText(data.cookie);
|
||||
showStatusInfo(`${SERVICES[serviceName].displayName} Cookie已复制到剪贴板`);
|
||||
} catch (err) {
|
||||
// 备用方案
|
||||
const textarea = document.createElement('textarea');
|
||||
textarea.value = data.cookie;
|
||||
document.body.appendChild(textarea);
|
||||
textarea.select();
|
||||
document.execCommand('copy');
|
||||
document.body.removeChild(textarea);
|
||||
showStatusInfo(`${SERVICES[serviceName].displayName} Cookie已复制到剪贴板`);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 删除服务数据
|
||||
function deleteService(serviceName) {
|
||||
if (confirm(`确定要删除 ${SERVICES[serviceName].displayName} 的Cookie数据吗？`)) {
|
||||
chrome.storage.local.remove([
|
||||
`cookieData_${serviceName}`,
|
||||
`lastCapture_${serviceName}`
|
||||
], function() {
|
||||
loadServiceData();
|
||||
showStatusInfo(`${SERVICES[serviceName].displayName} 数据已删除`);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// 清空所有数据
|
||||
function clearAllData() {
|
||||
if (confirm('确定要清空所有Cookie数据吗？')) {
|
||||
const keysToRemove = [];
|
||||
Object.keys(SERVICES).forEach(service => {
|
||||
keysToRemove.push(`cookieData_${service}`);
|
||||
keysToRemove.push(`lastCapture_${service}`);
|
||||
});
|
||||
|
||||
chrome.storage.local.remove(keysToRemove, function() {
|
||||
loadServiceData();
|
||||
showStatusInfo('所有数据已清空');
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// 导出数据
|
||||
function exportData() {
|
||||
const serviceKeys = Object.keys(SERVICES).map(service => `cookieData_${service}`);
|
||||
chrome.storage.local.get(serviceKeys, function(result) {
|
||||
const exportData = {};
|
||||
|
||||
Object.keys(result).forEach(key => {
|
||||
const serviceName = key.replace('cookieData_', '');
|
||||
exportData[serviceName] = result[key];
|
||||
});
|
||||
|
||||
const blob = new Blob([JSON.stringify(exportData, null, 2)], {type: 'application/json'});
|
||||
const url = URL.createObjectURL(blob);
|
||||
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = `cookie-sniffer-${new Date().toISOString().slice(0,10)}.json`;
|
||||
a.click();
|
||||
|
||||
URL.revokeObjectURL(url);
|
||||
showStatusInfo('数据已导出');
|
||||
});
|
||||
}
|
||||
|
||||
// 事件绑定
|
||||
refreshBtn.addEventListener('click', loadServiceData);
|
||||
clearBtn.addEventListener('click', clearAllData);
|
||||
exportBtn.addEventListener('click', exportData);
|
||||
webhookInput.addEventListener('blur', saveWebhookConfig);
|
||||
webhookInput.addEventListener('input', updateTestButtonState);
|
||||
testWebhookBtn.addEventListener('click', testWebhook);
|
||||
|
||||
// 代理点击事件
|
||||
serviceCards.addEventListener('click', function(e) {
|
||||
if (e.target.classList.contains('copy-btn')) {
|
||||
const serviceName = e.target.getAttribute('data-service');
|
||||
copyCookie(serviceName);
|
||||
} else if (e.target.classList.contains('delete-btn')) {
|
||||
const serviceName = e.target.getAttribute('data-service');
|
||||
deleteService(serviceName);
|
||||
}
|
||||
});
|
||||
|
||||
// 初始化
|
||||
loadWebhookConfig();
|
||||
loadServiceData();
|
||||
|
||||
// 自动刷新（每30秒）
|
||||
setInterval(loadServiceData, 30000);
|
||||
});
|
||||
|
|
@ -25,7 +25,7 @@ Web:
|
|||
API:
|
||||
# Network Configuration
|
||||
Host_IP: 0.0.0.0 # default IP | 默认IP
|
||||
Host_Port: 80 # default port is 80 | 默认端口为80
|
||||
Host_Port: 8456 # default port is 8456 | 默认端口为8456
|
||||
Docs_URL: /docs # API documentation URL | API文档URL
|
||||
Redoc_URL: /redoc # API documentation URL | API文档URL
|
||||
|
||||
|
|
|
|||
|
|
@ -348,6 +348,26 @@ class DouyinWebCrawler:
|
|||
# 对于URL列表
|
||||
return await WebCastIdFetcher.get_all_webcast_id(urls)
|
||||
|
||||
async def update_cookie(self, cookie: str):
|
||||
"""
|
||||
更新指定服务的Cookie
|
||||
|
||||
Args:
|
||||
service: 服务名称 (如: douyin_web)
|
||||
cookie: 新的Cookie值
|
||||
"""
|
||||
global config
|
||||
service = "douyin"
|
||||
print('DouyinWebCrawler before update', config["TokenManager"][service]["headers"]["Cookie"])
|
||||
print('DouyinWebCrawler to update', cookie)
|
||||
# 1. 更新内存中的配置（立即生效）
|
||||
config["TokenManager"][service]["headers"]["Cookie"] = cookie
|
||||
print('DouyinWebCrawler cookie updated', config["TokenManager"][service]["headers"]["Cookie"])
|
||||
# 2. 写入配置文件（持久化）
|
||||
config_path = f"{path}/config.yaml"
|
||||
with open(config_path, 'w', encoding='utf-8') as file:
|
||||
yaml.dump(config, file, default_flow_style=False, allow_unicode=True, indent=2)
|
||||
|
||||
async def main(self):
|
||||
"""-------------------------------------------------------handler接口列表-------------------------------------------------------"""
|
||||
|
||||
|
|
|
|||
|
|
@ -32,10 +32,13 @@
|
|||
# ==============================================================================
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
import httpx
|
||||
|
||||
from crawlers.douyin.web.web_crawler import DouyinWebCrawler # 导入抖音Web爬虫
|
||||
from crawlers.tiktok.web.web_crawler import TikTokWebCrawler # 导入TikTok Web爬虫
|
||||
from crawlers.tiktok.app.app_crawler import TikTokAPPCrawler # 导入TikTok App爬虫
|
||||
from crawlers.bilibili.web.web_crawler import BilibiliWebCrawler # 导入Bilibili Web爬虫
|
||||
|
||||
|
||||
class HybridCrawler:
|
||||
|
|
@ -43,6 +46,25 @@ class HybridCrawler:
|
|||
self.DouyinWebCrawler = DouyinWebCrawler()
|
||||
self.TikTokWebCrawler = TikTokWebCrawler()
|
||||
self.TikTokAPPCrawler = TikTokAPPCrawler()
|
||||
self.BilibiliWebCrawler = BilibiliWebCrawler()
|
||||
|
||||
async def get_bilibili_bv_id(self, url: str) -> str:
|
||||
"""
|
||||
从 Bilibili URL 中提取 BV 号，支持短链重定向
|
||||
"""
|
||||
# 如果是 b23.tv 短链，需要重定向获取真实URL
|
||||
if "b23.tv" in url:
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.head(url, follow_redirects=True)
|
||||
url = str(response.url)
|
||||
|
||||
# 从URL中提取BV号
|
||||
bv_pattern = r'(?:video\/|\/)(BV[A-Za-z0-9]+)'
|
||||
match = re.search(bv_pattern, url)
|
||||
if match:
|
||||
return match.group(1)
|
||||
else:
|
||||
raise ValueError(f"Cannot extract BV ID from URL: {url}")
|
||||
|
||||
async def hybrid_parsing_single_video(self, url: str, minimal: bool = False):
|
||||
# 解析抖音视频/Parse Douyin video
|
||||
|
|
@ -65,6 +87,14 @@ class HybridCrawler:
|
|||
data = await self.TikTokAPPCrawler.fetch_one_video(aweme_id)
|
||||
# $.imagePost exists if aweme_type is photo
|
||||
aweme_type = data.get("aweme_type")
|
||||
# 解析Bilibili视频/Parse Bilibili video
|
||||
elif "bilibili" in url or "b23.tv" in url:
|
||||
platform = "bilibili"
|
||||
aweme_id = await self.get_bilibili_bv_id(url) # BV号作为统一的video_id
|
||||
response = await self.BilibiliWebCrawler.fetch_one_video(aweme_id)
|
||||
data = response.get('data', {}) # 提取data部分
|
||||
# Bilibili只有视频类型，aweme_type设为0(video)
|
||||
aweme_type = 0
|
||||
else:
|
||||
raise ValueError("hybrid_parsing_single_video: Cannot judge the video source from the URL.")
|
||||
|
||||
|
|
@ -103,27 +133,44 @@ class HybridCrawler:
|
|||
and then use the .update() method to update the data
|
||||
"""
|
||||
|
||||
result_data = {
|
||||
'type': url_type,
|
||||
'platform': platform,
|
||||
'aweme_id': aweme_id,
|
||||
'desc': data.get("desc"),
|
||||
'create_time': data.get("create_time"),
|
||||
'author': data.get("author"),
|
||||
'music': data.get("music"),
|
||||
'statistics': data.get("statistics"),
|
||||
'cover_data': {
|
||||
'cover': data.get("video").get("cover"),
|
||||
'origin_cover': data.get("video").get("origin_cover"),
|
||||
'dynamic_cover': data.get("video").get("dynamic_cover")
|
||||
},
|
||||
'hashtags': data.get('text_extra'),
|
||||
}
|
||||
# 根据平台适配字段映射
|
||||
if platform == 'bilibili':
|
||||
result_data = {
|
||||
'type': url_type,
|
||||
'platform': platform,
|
||||
'video_id': aweme_id,
|
||||
'desc': data.get("title"), # Bilibili使用title
|
||||
'create_time': data.get("pubdate"), # Bilibili使用pubdate
|
||||
'author': data.get("owner"), # Bilibili使用owner
|
||||
'music': None, # Bilibili没有音乐信息
|
||||
'statistics': data.get("stat"), # Bilibili使用stat
|
||||
'cover_data': {}, # 将在各平台处理中填充
|
||||
'hashtags': None, # Bilibili没有hashtags概念
|
||||
}
|
||||
else:
|
||||
result_data = {
|
||||
'type': url_type,
|
||||
'platform': platform,
|
||||
'video_id': aweme_id, # 统一使用video_id字段，内容可能是aweme_id或bv_id
|
||||
'desc': data.get("desc"),
|
||||
'create_time': data.get("create_time"),
|
||||
'author': data.get("author"),
|
||||
'music': data.get("music"),
|
||||
'statistics': data.get("statistics"),
|
||||
'cover_data': {}, # 将在各平台处理中填充
|
||||
'hashtags': data.get('text_extra'),
|
||||
}
|
||||
# 创建一个空变量，稍后使用.update()方法更新数据/Create an empty variable and use the .update() method to update the data
|
||||
api_data = None
|
||||
# 判断链接类型并处理数据/Judge link type and process data
|
||||
# 抖音数据处理/Douyin data processing
|
||||
if platform == 'douyin':
|
||||
# 填充封面数据
|
||||
result_data['cover_data'] = {
|
||||
'cover': data.get("video", {}).get("cover"),
|
||||
'origin_cover': data.get("video", {}).get("origin_cover"),
|
||||
'dynamic_cover': data.get("video", {}).get("dynamic_cover")
|
||||
}
|
||||
# 抖音视频数据处理/Douyin video data processing
|
||||
if url_type == 'video':
|
||||
# 将信息储存在字典中/Store information in a dictionary
|
||||
|
|
@ -160,6 +207,12 @@ class HybridCrawler:
|
|||
}
|
||||
# TikTok数据处理/TikTok data processing
|
||||
elif platform == 'tiktok':
|
||||
# 填充封面数据
|
||||
result_data['cover_data'] = {
|
||||
'cover': data.get("video", {}).get("cover"),
|
||||
'origin_cover': data.get("video", {}).get("origin_cover"),
|
||||
'dynamic_cover': data.get("video", {}).get("dynamic_cover")
|
||||
}
|
||||
# TikTok视频数据处理/TikTok video data processing
|
||||
if url_type == 'video':
|
||||
# 将信息储存在字典中/Store information in a dictionary
|
||||
|
|
@ -198,6 +251,50 @@ class HybridCrawler:
|
|||
'watermark_image_list': watermark_image_list
|
||||
}
|
||||
}
|
||||
# Bilibili数据处理/Bilibili data processing
|
||||
elif platform == 'bilibili':
|
||||
# 填充封面数据
|
||||
result_data['cover_data'] = {
|
||||
'cover': data.get("pic"), # Bilibili使用pic作为封面
|
||||
'origin_cover': data.get("pic"),
|
||||
'dynamic_cover': data.get("pic")
|
||||
}
|
||||
# Bilibili只有视频，直接处理视频数据
|
||||
if url_type == 'video':
|
||||
# 获取视频播放地址需要额外调用API
|
||||
cid = data.get('cid') # 获取cid
|
||||
if cid:
|
||||
# 获取播放链接，cid需要转换为字符串
|
||||
playurl_data = await self.BilibiliWebCrawler.fetch_video_playurl(aweme_id, str(cid))
|
||||
# 从播放数据中提取URL
|
||||
dash = playurl_data.get('data', {}).get('dash', {})
|
||||
video_list = dash.get('video', [])
|
||||
audio_list = dash.get('audio', [])
|
||||
|
||||
# 选择最高质量的视频流
|
||||
video_url = video_list[0].get('baseUrl') if video_list else None
|
||||
audio_url = audio_list[0].get('baseUrl') if audio_list else None
|
||||
|
||||
api_data = {
|
||||
'video_data': {
|
||||
'wm_video_url': video_url,
|
||||
'wm_video_url_HQ': video_url,
|
||||
'nwm_video_url': video_url, # Bilibili没有水印概念
|
||||
'nwm_video_url_HQ': video_url,
|
||||
'audio_url': audio_url, # Bilibili音视频分离
|
||||
'cid': cid, # 保存cid供后续使用
|
||||
}
|
||||
}
|
||||
else:
|
||||
api_data = {
|
||||
'video_data': {
|
||||
'wm_video_url': None,
|
||||
'wm_video_url_HQ': None,
|
||||
'nwm_video_url': None,
|
||||
'nwm_video_url_HQ': None,
|
||||
'error': 'Failed to get cid for video playback'
|
||||
}
|
||||
}
|
||||
# 更新数据/Update data
|
||||
result_data.update(api_data)
|
||||
return result_data
|
||||
|
|
|
|||
68
deploy.ps1
Normal file
68
deploy.ps1
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
<#
|
||||
.SYNOPSIS
|
||||
Deploys the Douyin_TikTok_Download_API to a local Docker image or pushes to a remote registry.
|
||||
|
||||
.DESCRIPTION
|
||||
This script builds the Docker image and optionally pushes it to a registry or saves it as a tar file for manual transfer.
|
||||
|
||||
.PARAMETER Push
|
||||
If specified, pushes the image to the registry defined in $Registry.
|
||||
|
||||
.PARAMETER Save
|
||||
If specified, saves the image to a .tar file for manual transfer.
|
||||
|
||||
.EXAMPLE
|
||||
.\deploy.ps1 -Save
|
||||
Builds the image and saves it to douyin_tiktok_download_api.tar
|
||||
|
||||
.EXAMPLE
|
||||
.\deploy.ps1 -Push
|
||||
Builds the image and pushes it to your private registry.
|
||||
#>
|
||||
|
||||
param (
|
||||
[Switch]$Push,
|
||||
[Switch]$Save
|
||||
)
|
||||
|
||||
$ImageName = "douyin_tiktok_download_api"
|
||||
# Update this with your actual registry if you use one, e.g., git.khoavo.myds.me/vndangkhoa/douyin-api
|
||||
$Registry = "git.khoavo.myds.me/vndangkhoa/douyin-api"
|
||||
$Tag = "latest"
|
||||
|
||||
Write-Host "Building Docker image..." -ForegroundColor Cyan
|
||||
docker-compose build
|
||||
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Error "Build failed!"
|
||||
exit 1
|
||||
}
|
||||
|
||||
Write-Host "Build successful!" -ForegroundColor Green
|
||||
|
||||
if ($Save) {
|
||||
Write-Host "Saving image to ${ImageName}.tar..." -ForegroundColor Cyan
|
||||
docker save -o ${ImageName}.tar ${ImageName}:${Tag}
|
||||
Write-Host "Image saved to ${PWD}\${ImageName}.tar" -ForegroundColor Green
|
||||
Write-Host "You can now upload this file to your Synology NAS and load it via Docker."
|
||||
}
|
||||
|
||||
if ($Push) {
|
||||
$RemoteImage = "${Registry}:${Tag}"
|
||||
Write-Host "Tagging image as ${RemoteImage}..." -ForegroundColor Cyan
|
||||
docker tag ${ImageName}:${Tag} ${RemoteImage}
|
||||
|
||||
Write-Host "Pushing image to ${Registry}..." -ForegroundColor Cyan
|
||||
docker push ${RemoteImage}
|
||||
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Error "Push failed! Make sure you are logged in to the registry."
|
||||
exit 1
|
||||
}
|
||||
|
||||
Write-Host "Push successful!" -ForegroundColor Green
|
||||
}
|
||||
|
||||
if (-not $Push -and -not $Save) {
|
||||
Write-Host "Build complete. Use -Save to create a transferable file or -Push to push to a registry." -ForegroundColor Yellow
|
||||
}
|
||||
|
|
@ -1,17 +1,42 @@
|
|||
version: "3.9" # Docker Compose 文件版本
|
||||
version: "3.9"
|
||||
|
||||
services: # 定义服务列表
|
||||
douyin_tiktok_download_api: # 服务名称
|
||||
image: evil0ctal/douyin_tiktok_download_api # 使用的 Docker 镜像
|
||||
network_mode: host # 使用主机网络模式
|
||||
container_name: douyin_tiktok_download_api # 容器名称
|
||||
restart: always # 容器退出后总是重启
|
||||
volumes: # 挂载卷配置
|
||||
- ./douyin_tiktok_download_api/douyin_web/config.yaml:/app/crawlers/douyin/web/config.yaml
|
||||
- ./douyin_tiktok_download_api/tiktok_web/config.yaml:/app/crawlers/tiktok/web/config.yaml
|
||||
- ./douyin_tiktok_download_api/tiktok_app/config.yaml:/app/crawlers/tiktok/app/config.yaml
|
||||
environment: # 环境变量配置
|
||||
TZ: Asia/Shanghai # 设置时区为亚洲/上海
|
||||
PUID: 1026 # 设置容器内部的用户 ID
|
||||
PGID: 100 # 设置容器内部的用户组 ID
|
||||
privileged: true # 设置特权模式以便容器内部可以执行特权操作
|
||||
services:
|
||||
douyin_tiktok_download_api:
|
||||
# Build from the current directory
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
image: git.khoavo.myds.me/vndangkhoa/kv-tiktok-download:v1
|
||||
container_name: douyin_tiktok_download_api
|
||||
restart: always
|
||||
|
||||
# Use bridge mode with explicit port mapping for better NAS compatibility
|
||||
ports:
|
||||
- "${HOST_PORT:-8456}:8456"
|
||||
|
||||
volumes:
|
||||
# Mount configuration file
|
||||
- ./config.yaml:/app/config.yaml
|
||||
# Mount download directory to persist data
|
||||
- ./download:/app/download
|
||||
# (Optional) Mount log directory if the app writes logs to a specific file
|
||||
# - ./logs:/app/logs
|
||||
|
||||
environment:
|
||||
- TZ=${TZ:-Asia/Shanghai}
|
||||
- PUID=${PUID:-1026}
|
||||
- PGID=${PGID:-100}
|
||||
# You can add other environment variables here if the app supports them
|
||||
|
||||
# Set user/group to avoid permission issues if the container supports PUID/PGID internally
|
||||
# Note: If the app doesn't natively support PUID/PGID,
|
||||
# we might need to rely on the user running the container to set permissions on the volumes.
|
||||
# However, LinuxServer.io style containers use these env vars.
|
||||
# This specific app might not use them in its entrypoint, but it doesn't hurt to have them available.
|
||||
|
||||
# Resource limits (optional, good for NAS)
|
||||
# deploy:
|
||||
# resources:
|
||||
# limits:
|
||||
# cpus: '0.50'
|
||||
# memory: 512M
|
||||
|
|
|
|||
Loading…
Reference in a new issue