## File: README.md
QD for Python3
操作说明
==========
请参阅 **[使用指南](https://qd-today.github.io/qd/zh_CN/)**
更新日志
===========
详见 **[CHANGELOG.md](./CHANGELOG.md)**
维护项目精力有限, 仅保证对 Chrome 浏览器的支持。如果测试了其他浏览器可以 Pull Request。
许可
===========
[MIT](https://fastly.jsdelivr.net/gh/qd-today/qd@master/LICENSE) 许可协议
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fqd-today%2Fqd?ref=badge_large)
致谢
===========
## Contributors ✨
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
| [**a76yyyy**](http://www.a76yyyy.cn)[🎨](#design-a76yyyy) [💻](https://github.com/qd-today/qd/commits?author=a76yyyy) [🚧](#maintenance-a76yyyy) [📖](https://github.com/qd-today/qd/commits?author=a76yyyy) | [**Roy Binux**](http://binux.me/)[🎨](#design-Binux) [💻](https://github.com/qd-today/qd/commits?author=Binux) [🚧](#maintenance-Binux) [📖](https://github.com/qd-today/qd/commits?author=Binux) | [**AragonSnow**](https://github.com/AragonSnow)[💻](https://github.com/qd-today/qd/commits?author=AragonSnow) [🎨](#design-AragonSnow) [🚧](#maintenance-AragonSnow) | [**Mark**](https://www.quchao.net)[🎨](#design-Mark-1215) [📝](#blog-Mark-1215) [📖](https://github.com/qd-today/qd/commits?author=Mark-1215) [🚧](#maintenance-Mark-1215) [💻](https://github.com/qd-today/qd/commits?author=Mark-1215) | [**pidan**](https://github.com/cdpidan)[🎨](#design-cdpidan) [💻](https://github.com/qd-today/qd/commits?author=cdpidan) | [**buzhibujue**](https://buzhibujue.cf)[💻](https://github.com/qd-today/qd/commits?author=buzhibujuelb) | [**billypon**](https://github.com/billypon)[💻](https://github.com/qd-today/qd/commits?author=billypon) | [**acooler15**](http://www.lingyan8.com)[💻](https://github.com/qd-today/qd/commits?author=acooler15) [🚧](#maintenance-acooler15) |
| --- | --- | --- | --- | --- | --- | --- | --- |
| [**shxyke**](https://github.com/aa889788)[💻](https://github.com/qd-today/qd/commits?author=aa889788) | [**xiaoxiao**](https://github.com/gxitm)[💻](https://github.com/qd-today/qd/commits?author=gxitm) | [**Casper**](https://blog.hicasper.com)[💻](https://github.com/qd-today/qd/commits?author=caspershw) | [**旋子**](https://github.com/ckx000)[💻](https://github.com/qd-today/qd/commits?author=ckx000) | [**chen8945**](https://github.com/chen8945)[💻](https://github.com/qd-today/qd/commits?author=chen8945) | [**seiuneko**](https://github.com/seiuneko)[💻](https://github.com/qd-today/qd/commits?author=seiuneko) | [**powersee**](https://github.com/powersee)[💻](https://github.com/qd-today/qd/commits?author=powersee) | [**Cirn09**](https://github.com/Cirn09)[💻](https://github.com/qd-today/qd/commits?author=Cirn09) |
| [**Ovler**](https://github.com/Ovler-Young)[💻](https://github.com/qd-today/qd/commits?author=Ovler-Young) | [**An**](https://www.lshell.com)[💻](https://github.com/qd-today/qd/commits?author=Ansen) | [**Siman8**](https://github.com/Siman8)[💻](https://github.com/qd-today/qd/commits?author=Siman8) | [**Uiolee**](https://github.com/uiolee)[💻](https://github.com/qd-today/qd/commits?author=uiolee) | [**OwnerCM**](https://github.com/OwnerCM)[💻](https://github.com/qd-today/qd/commits?author=OwnerCM) | [**jionkitten**](https://github.com/jionkitten)[💻](https://github.com/qd-today/qd/commits?author=jionkitten) | [**Patrick-Ze**](https://github.com/Patrick-Ze)[💻](https://github.com/qd-today/qd/commits?author=Patrick-Ze) | |
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
Stargazers over time
===========
[](https://starchart.cc/qd-today/qd)
---
## File: web/docs/guide/deployment.md
# Deployment
## Docker Container Deployment
Docker Container Deployment is the easiest way to deploy QD.
> Please always remember to back up your database before updating or redeploying.
### Container
**DockerHub URL** : [https://hub.docker.com/r/qdtoday/qd](https://hub.docker.com/r/qdtoday/qd)
> Tag meaning:
>
> - `latest`: Latest Release version
> - `lite-latest`: Latest Release version without OCR related functions
> - `ja3-latest`: Integrated curl-impersonate to solve the problem that ja3 fingerprint is identified as curl, does not support http3 and Quic connection
> - `20xxxxxx`: Specify Release version, version number is represented by Release release date
> - `dev`: Latest development version, synchronized with the latest source code, no stability guarantee
### Deploy Method
#### 1. Docker Compose (Recommend)
``` sh
# Create and switch to the QD directory.
mkdir -p $(pwd)/qd/config && cd $(pwd)/qd
# Download docker-compose.yml
wget https://fastly.jsdelivr.net/gh/qd-today/qd@master/docker-compose.yml
# Modify the configuration environment variables according to the requirements and configuration description
vi ./docker-compose.yml
# Execute Docker Compose command
docker-compose up -d
```
> See [Configuration](#configuration-environment-variables) below for configuration description
>
> If you don't need `OCR` or `hard disk space is not larger than 600M`, please use **`qdtoday/qd:lite-latest`** image, **this image only removes OCR related functions, other than the mainline version to keep consistent**.
>
> **Please don't use AliCloud image source to pull Docker container, it will not pull the latest image.**
#### 2. 1Panel Deployment
2.1. Create a new application in 1Panel
> The QD framework name may be different in different 1Panel versions, but they are all `QD` icons.
2.2. Configure related settings
> If you need to set environment variables, please click `Edit Compose file`
>
> See [Configuration](#configuration-environment-variables) below for configuration description
>
> If you don't need `OCR` or `hard disk space is not larger than 600M`, please use **`qdtoday/qd:lite-latest`** image, **this image only removes OCR related functions, other than the mainline version to keep consistent**.
2.3. Click `Confirm` to install QD via 1Panel
#### 3. Docker Run
``` sh
docker run -d --name qd -p 8923:80 -v $(pwd)/qd/config:/usr/src/app/config qdtoday/qd
```
Try this command if you cannot connect to the external network inside the container:
``` sh
# Create container using Host network mode, port: 8923
docker run -d --name qd --env PORT=8923 --net=host -v $(pwd)/qd/config:/usr/src/app/config qdtoday/qd
```
> Please note that after creating a container with this command, please change the api request of `http://localhost/` form in the template to `api://` or `http://localhost:8923/` manually in order to complete the related API request properly.
>
> **Do not run both old and new versions of QD framework, or map different running QD container databases to the same file.**
## Source Code Deployment
1. **Version >= python3.9**
``` sh
# Please cd to the root of the framework source code first
pip3 install -r requirements.txt
```
2. **Modify the configuration**
``` sh
# Execute the following command to copy the configuration file
# Modifying the content of the local_config.py file is not affected by updating the source code through git
cp config.py local_config.py
# Modify the configuration environment variables according to the requirements and configuration description
vi local_config.py
```
3. **Run**
``` sh
python ./run.py
```
4. **Access**
``` sh
# Access the web page
http://localhost:8923/
```
> If you are using the source code deployment method, please change the api request of `http://localhost/` form in the template to `api://` or `http://localhost:8923/` manually in order to complete the related API request properly.
>
> Templates need to be published to be displayed in "Public Templates", and you need admin rights to approve them in "My Publish Requests".
## Configure administrators
``` sh
python ./chrole.py your@email.address admin
```
> The first registered user is the administrator by default, you need to log out and then login to get full administrator rights
## Configuration Environment Variables
|variable name|required|default value|description|
|:-: | :-: | :-: | :-:|
|BIND|No|0.0.0.0|Listening address|
|PORT|No|8923|Listening port|
|QD_DEBUG|No|False|Whether to enable Debug mode|
|WORKER_METHOD|No|Queue|Task timing execution method, The default is Queue, optional Queue or Batch, Batch mode is the old version of timing task execution method, the performance is weak, **Recommended only when Queue timed execution mode fails**|
|MULTI_PROCESS|No|False|(Experimental) Whether to enable multi-process mode, invalid on Windows platform|
|AUTO_RELOAD|No|False|Whether to enable automatic hot reload, invalid when MULTI_PROCESS=True|
|STATIC_URL_PREFIX|No|`/static/`|Static file URL prefix|
|DOMAIN|No|''|Specify the access domain name, **(recommended modification)**, otherwise the function of resetting password by email is not valid|
|AES_KEY|No|binux|AES encryption key, **(Modification strongly recommended)**|
|COOKIE_SECRET|No|binux|cookie encryption key, **(Modification strongly recommended)**|
|COOKIE_DAY|No|5|The number of days the cookie is kept in the client|
|DB_TYPE|No|sqlite3|Set to 'mysql' when MySQL is required|
|JAWSDB_MARIA_URL|No|''|When you need to use MySQL, set to `mysql://username:password@hostname:port/database_name?auth_plugin=`|
|QD_SQL_ECHO|No|False|Whether to enable the log output of SQLAlchmey, the default is False, When set to True, the SQL statement will be output on the console, allow to set to debug to enable debug mode|
|QD_SQL_LOGGING_NAME|No|QD.sql_engine|SQLAlchmey log name, default is 'QD.sql_engine'|
|QD_SQL_LOGGING_LEVEL|No|Warning|SQLAlchmey log level, default is 'Warning'|
|QD_SQL_ECHO_POOL|No|True|Whether to enable SQLAlchmey's connection pool log output, the default is True, allow setting to debug to enable debug mode|
|QD_SQL_LOGGING_POOL_NAME|No|QD.sql_pool|SQLAlchmey connection pool log name, the default is 'QD.sql_pool'|
|QD_SQL_LOGGING_POOL_LEVEL|No|Warning|SQLAlchmey connection pool log level, default is 'Warning'|
|QD_SQL_POOL_SIZE|No|10|SQLAlchmey connection pool size, default is 10|
|QD_SQL_MAX_OVERFLOW|No|50|SQLAlchmey connection pool maximum overflow, the default is 50|
|QD_SQL_POOL_PRE_PING|No|True|Whether to ping before the connection pool gets a connection, the default is True|
|QD_SQL_POOL_RECYCLE|No|3600|SQLAlchmey connection pool recovery time, the default is 3600|
|QD_SQL_POOL_TIMEOUT|No|60|SQLAlchmey connection pool timeout, the default is 60|
|QD_SQL_POOL_USE_LIFO|No|True|SQLAlchmey whether to use LIFO algorithm, the default is True|
|REDISCLOUD_URL|No|''|When you need to use Redis or RedisCloud, set to |
|REDIS_DB_INDEX|No|1|The default is 1|
|QD_EVIL|No|500|(Only when the Redis connection is enabled)Score = number of operation failures (such as login, verification, test, etc.) * corresponding penalty pointsWhen the score reaches the upper limit of evil, it will be automatically banned until the next hour cycle|
|EVIL_PASS_LAN_IP|No|True|Whether to turn off the evil restriction of local private IP address users and Localhost_API requests|
|TRACEBACK_PRINT|No|False|Whether to enable to print Exception's TraceBack information in the console log|
|PUSH_PIC_URL|No|[push_pic.png](https://fastly.jsdelivr.net/gh/qd-today/qd@master/web/static/img/push_pic.png)|The default is [push_pic.png](https ://fastly.jsdelivr.net/gh/qd-today/qd@master/web/static/img/push_pic.png)|
|PUSH_BATCH_SW|No|True|Whether to allow periodic push of QD task logs, the default is True|
|MAIL_SMTP|No|""|Email SMTP server|
|MAIL_PORT|No|465|Email SMTP server port|
|MAIL_SSL|No|True|Whether to use SSL when connecting to the email server, the default is True|
|MAIL_STARTTLS|No|False|Whether to use TLS when connecting to the email server, the default is False|
|MAIL_USER|No|""|Email username|
|MAIL_PASSWORD|No|""|Email password|
|MAIL_FROM|No|MAIL_USER|The Email used when sending, the default is the same as MAIL_USER|
|MAIL_DOMAIN_HTTPS|No|False|Whether to use HTTPS for email domain name. Not the framework itself HTTPS configuration. If you need HTTPS, please use an external reverse proxy|
|PROXIES|No|""|Global proxy domain name list, separated by "\|"|
|PROXY_DIRECT_MODE|No|""|Global proxy blacklist mode, not enabled by default "url" is URL matching mode; "regexp" is regular expression matching mode|
|PROXY_DIRECT|No|""|Global proxy blacklist matching rules|
|NEW_TASK_DELAY|No|1|Preparation time after new task, default is 1 second|
|TASK_WHILE_LOOP_TIMEOUT|No|900|Maximum runtime of a single While loop during a task run, the default is 15 minutes|
|TASK_REQUEST_LIMIT|No|1500|Maximum number of requests per task run, the default is 1500|
|USE_PYCURL|No|True|Whether to enable Pycurl module|
|ALLOW_RETRY|No|True|When some requests in the Pycurl environment may cause Request errors, automatically modify the conflict settings and resend the request|
|DNS_SERVER|No|""|Use specified DNS for resolution via Curl (only supports Pycurl environment), such as 8.8.8.8|
|CURL_ENCODING|No|True|Whether to allow to use Curl for Encoding operation|
|CURL_CONTENT_LENGTH|No|True|Whether to allow Curl to use custom Content-Length request in Headers|
|NOT_RETRY_CODE|No|[See configuration for details](https://github.com/qd-today/qd/blob/master/config.py)...|[See configuration for details](https://github.com/qd-today/qd/blob/master/config.py)...|
|EMPTY_RETRY|No|True|[See configuration for details](https://github.com/qd-today/qd/blob/master/config.py)...|
|USER0ISADMIN|No|True|The first registered user is an administrator, False to close|
|NOTEPAD_LIMIT|No|20|The maximum number of notepads within the same user, the default is 20|
|EXTRA_ONNX_NAME|No|""|Customize the ONNX file name in the config directory(do not fill in the ".onnx" suffix)Separate multiple onnx file names with "\|"|
|EXTRA_CHARSETS_NAME|No|""|Custom ONNX in the config directory corresponds to the custom charsets.json file name(do not fill in the ".json" suffix)Multiple json file names are separated by "\|"|
|WS_PING_INTERVAL|No|5|WebSocket ping interval, the default is 5 seconds|
|WS_PING_TIMEOUT|No|30|WebSocket ping timeout, the default is 30 seconds|
|WS_MAX_MESSAGE_SIZE|No|10485760|WebSocket maximum message size, the default is 10485760 bytes|
|WS_MAX_QUEUE_SIZE|No|100|WebSocket maximum queue size, the default is 100|
|WS_MAX_CONNECTIONS_SUBSCRIBE|No|30|WebSocket subscribe page maximum number of connections, the default is 30|
|SUBSCRIBE_ACCELERATE_URL|No|qd-ph|Subscribe page acceleration URL, the default is qd-ph, [See configuration for details](https://github.com/qd-today/qd/blob/master/config.py)...|
> For details, please refer to [config.py](https://github.com/qd-today/qd/blob/master/config.py)
---
## File: web/docs/guide/faq.md
# FAQ
## how to backup and restore the database?
QD uses **sqlite3** as the framework database by default, and the `database.db` file is saved in `config` directory. When deploying with a Docker container, you can use the `docker cp` command to back up the database file, and then use the `docker cp` command to restore the database file in the new container.
```sh
# database backup
docker cp container_name:/usr/src/app/config/database.db .
# Database recovery
docker cp database.db container_name:/usr/src/app/config/
```
## how to configure the email server in Docker?
```sh
docker run -d --name qd -p 8923:80 -v $(pwd)/qd/config:/usr/src/app/config --env MAIL_SMTP=$STMP_Server_ --env MAIL_PORT=$Mailbox_server_port --env MAIL_USER=$Username --env MAIL_PASSWORD=$Password --env DOMAIN=$Domain qdtoday/qd
```
## how to use MySQL in Docker?
```sh
docker run -d --name qd -p 8923:80 -v $(pwd)/qd/config:/usr/src/app/config --ENV DB_TYPE=mysql --ENV JAWSDB_MARIA_URL=mysql://$username:$password@$hostname:$port/$database_name?auth_plugin= qdtoday/qd
```
## how to build a Docker image by myself?
Please refer to the build file [Dockerfile](https://github.com/qd-today/qd/blob/master/Dockerfile) of this image.
## How to view the API and Jinja2 template variables supported by the current framework?
Please access the home page of the framework, and then click the `Common API/Filter` button in the upper left corner to view the API and Jinja2 template variables supported by the current framework.
## how to submit a bug issue?
Please enable `Debug` mode after encountering a problem, and then submit detailed error information to [Issue](https://github.com/qd-today/qd/issues).
## Which requests are necessary for QD?
Empirically, the following requests are necessary:
- login page
- POST to login page
- Issue a request for username, password
- The page that caused the page to jump after sending
- The first page before and after page turning
## Will my username and password be revealed?
In order to help users initiate requests, user names and passwords still need to be recorded. This can only rely on the self-discipline of server maintainers to ensure the security of back-end data. But in the framework design, each user is encrypted with a secure key when storing. Encrypting user data with a key can ensure that user data cannot be decrypted only by obtaining the database. (Encrypted user data includes templates uploaded by users, variables set by users for tasks, etc.)
If you are still worried, you can build the QD framework by yourself, download the template and run it on your own server.
## Prompt `PermissionError: [Errno 1] Operation not permitted`
1. If it is an ARM32 Debian system, please check whether the `Docker` version is less than `20.10.0`, and whether the `libseccomp` version is less than `2.4.4`. If so, please upgrade the above components.
Update `libseccomp` reference operation:
```sh
# Get signing keys to verify the new packages, otherwise they will not install
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC 648ACFD622F3D138
# Add the Buster backport repository to apt sources.list
echo 'deb http://httpredir.debian.org/debian buster-backports main contrib non-free' | sudo tee -a /etc/apt/sources.list.d/debian-backports.list
sudo apt update
sudo apt install libseccomp2 -t buster-backports
```
> Source:
>
> -
> -
2. Check if the `/usr/src/app` directory inside the container is mapped to the outside of the container.
> Note that the framework only needs to map the `/usr/src/app/config` directory.
## Prompt warning message: `Connect Redis falied: Error 10061`
QD uses `redis` as a flow limiting tool. If the `redis` service is not installed, the framework will prompt the following warning message.
```sh
[W xxxxxx xx:xx:xx QD.RedisDB redisdb:28] Connect Redis falied: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
```
However, `redis` is not required in this framework, if you don't need to use the `flow-limiting` feature, you can ignore the warning message.
> It is recommended to use `Docker compose` to deploy the QD framework, and the Docker compose configuration already installs the redis container by default.
## Prompt warning message: `Import PyCurl module falied`
QD uses the `pycurl` module to send HTTP proxy requests. If the `pycurl` module is not installed, the framework will prompt the following warning message:
```sh
[W xxxxxx xx:xx:xx QD.Http.Fetcher fetcher:34] Import PyCurl module falied: No module named 'pycurl'
```
However, `pycurl` is not required in this framework, if you don't need to use the `proxy` function, you can ignore the warning message.
> The `pycurl` module is cumbersome to install on the Windows system, and the `libcurl` library needs to be installed. For the specific installation method, please refer to [pycurl official documentation](http://pycurl.io/docs/latest/install.html).
>
> It is recommended to use a container or linux system to deploy the QD framework. The docker container has a pre-installed Curl environment, and the pycurl module is installed by default
## How to Register Notification Tools
You can register different notification tools to receive notifications when specific events (such as failed check-ins) occur.
Please refer to [Pusher](/toolbox/pusher) for details.
## Subscribe updating page prompts undefined error
- [issue#423](https://github.com/qd-today/qd/issues/423)
> The subscribe updating web page prompts an error code of undefined, or the console shows WebSocket connection failed but does not show the reason for the error
Please check if the "reverse proxy" configuration is correct, refer to [Nginx reverse proxy WebSocket service connection error](https://blog.csdn.net/tiven_/article/details/126126442)
> Reference configuration is as follows:
>
> ```Nginx
> server {
> listen 80;
> # Modify server_name by yourself
> server_name qd.example.com;
> location / {
> proxy_pass http://ip:port;
>
> # Set WebSocket Configuration
> proxy_http_version 1.1;
> proxy_set_header Upgrade $http_upgrade;
> proxy_set_header Connection "upgrade";
>
> # Set other optional configuration
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Forwarded-Proto $scheme;
> }
> }
> ```
## Error Code: 4006
> Error message: "Update failed, reason: Cannot connect to host xxx.xxx:443 ssl:False"
Cause: Unable to connect to Github or GitHub acceleration source.
Solution 1: Use a proxy
Solution 2: Change the GitHub acceleration source
Add/modify the environment variable in the container: `SUBSCRIBE_ACCELERATE_URL=https://xxx.xxx/https://raw.githubusercontent.com/`
Replace `https://xxx.xxx/` with an available acceleration source. If you can't find an acceleration source, you can refer to or for published acceleration sources.
---
## File: web/docs/guide/how-to-use.md
# How to Use?
## Har Obtain
### 1. What is HAR?
HAR:
### 2. Packet capture
#### 2.1. Chrome or Edge
1. Press `F12`, `Ctrl + Shift + I`, or from the Chrome menu select `More tools` > `Developer tools`.
2. From the panel that opens at the bottom of your screen, select the `Network tab`.
3. Make sure the `Record` button in the upper left corner of the `Network tab` is shown in **red**.
4. If it's grey, click it once to start recording.
5. Check the box next to `Preserve log`.
6. Click the `Clear` button to clear out any existing logs from the `Network tab`.
7. Now try to reproduce the task you were trying to do.
8. Once you have reproduced the task, right-click anywhere on the grid of network requests.
9. Select `Save as HAR with Content`.
10. Save the file to your computer.
#### 2.2. Firefox
1. Press `F12` (or Go to `Tools` > `Web Developer` > `Network`).
2. Now try to reproduce the task you were trying to do.
3. Right-click on the loaded results.
4. Select Save all as har.
#### 2.3. Windows - Fiddler
1. Open Fiddler, open the `Tools` menu, select `Options`.
2. Select the `HTTPS tab`, check `Capture HTTPS Connects`.
3. Select the `Connections tab`, check `Decrypt HTTPS traffic`.
4. Now try to reproduce the task you were trying to do.
5. export to HAR format - please select HTTPArchive as the export method.
#### 2.4. IOS - Stream
1. Open Stream.
2. Before capture the HTTPS request, you need to install the CA certificate, `setting` > `General` > `About` > `Certificate Trust Settings` to trust the CA certificate.
3. Click the start packet capture button, the phone will automatically pop up the VPN configuration window, and then select Allow.
4. Now try to reproduce the task you were trying to do.
5. On the app page, click Stop Capture to end this capture.
6. export the HAR file.
### 3. Community HAR
1. Click the `Community Template` button to the right of `my template`.
2. Update the repository to get the latest HAR file list.
3. Select the HAR file you want to use and click the `Subscribe` button to subscribe to the template.
4. Modify the template according to your needs.
5. Then jump to `step 5` of [3. Edit the template](#edit-the-template) to continue editing.
## Upload the HAR file
1. Access and login to QD framework.
2. Click the `+` button to the right of `my template`.
3. Upload the HAR file you just saved.
4. Click `upload` to continue.
## Edit the template
::: v-pre
1. Replace the username, password, cookie, header and other parts that change according to the user with a template similar to `{{ username }}`. (templates support **jinja2 syntax**)
2. Use the test panel in request editing to test whether the template is returned correctly, use the correct and wrong user names to test。
3. Fill in `success/failure assertion`, which helps to detect sign-in failures and template failures.
4. When some data from the previous request is needed in the request, variable extraction is used to extract the data through regularization and save it in the environment variable. Use `ab(\w+)cd`, the group selector, to select part of it.
5. Use `__log__` to extract task logs.
6. When all request edits are complete, use `Test` next to the Download button to test the overall.
7. The template being edited will be automatically saved in the browser cache, so don't worry about losing it.
8. Click the `Save` button to save the template.
9. Click the `Download` button to download the template.
:::
## Create scheduled task
1. Click the `+` button to the right of `my task`.
2. Select the template you just created.
3. Fill in the task variables, task interval, task group and task execution time.
4. Click the `Test` button to test the task.
5. Click the Save button to save the task.
---
## File: web/docs/guide/update.md
# Update
> Please always remember to back up your database before updating or redeploying.
>
> After the update, please restart the container or clear the browser cache.
## Source Code Deployment Update
``` sh
# First cd to the directory of source code, execute the command and restart the process
wget https://gitee.com/qd-today/qd/raw/master/update.sh -O ./update.sh && \
sh ./update.sh
```
## Docker Compose Deployment Update
``` sh
# First cd to the directory of docker-compose.yml, execute the command and restart the container
docker compose pull && \
docker compose up -d
```
## Docker Container Deployment Update
``` sh
# Enter the container background first, restart the container after executing the command
wget https://gitee.com/qd-today/qd/raw/master/update.sh -O /usr/src/app/update.sh && \
sh /usr/src/app/update.sh
```
## Forcibly synchronize the latest source code
``` sh
# First cd to the root directory of code, execute the command and restart the process
# docker exec -it "container name" /bin/sh
wget https://gitee.com/qd-today/qd/raw/master/update.sh -O ./update.sh && \
sh ./update.sh -f
```
---
## File: web/docs/guide/what-is-qd.md
# What is QD?
QD is a web framework for HTTP timed task automation base on HAR Editor and Tornado Server.
## Features
- **Har-based**: By simply uploading Har obtained through packet capture, you can create HTTP task template required by the framework.
- **Tornado Server**: Use Tornado as a server to asynchronously respond to front-end and initiate HTTP requests.
- **API & Plugin Support**: Various APIs and filters are built in for template creation, and custom plugins will be provided later.
- **Open source**: QD is an open source project based on the MIT license.
## How to deploy
Please refer: [Deployment](deployment)
## How to use
Please refer: [How to use](how-to-use)
## How to Update
Please refer: [Update](update)
## Discussion
- Github: [Issue](https://github.com/qd-today/qd/issues)
- Github: [Discussion](https://github.com/qd-today/qd/discussions)
---
## File: web/docs/toolbox/pusher.md
# Pusher
## Push Registration
The QD framework provides various [push methods](#push methods), and you can register different push tools in `Toolbox`->`Push Registration` to push notifications to you when specific events occur (e.g. timed task execution failure).
::: tip
The parameters filled in when registering the push are separated and connected by `;`. If the parameter value is empty, please be sure to keep the `;` after the parameter position, otherwise it may cause parameter parsing error.
:::
### Push Registration Test
After registering the push method in `Toolbox`->`Push Registration`, you can click the `Test` button to test whether the push method is available.
If the push method is available, you will receive a push message, otherwise you will be prompted that the push failed.
::: tip
When testing the push registration, please make sure that the following conditions are met:
- The correct parameters are filled in;
- The `email` and `password` are filled in with the user's email and password of the QD framework.
:::
### Push Registration Former value
After registering the push method in `Toolbox`->`Push Registration`, you can click the `Former value` button to view the former value of the push registration.
::: tip
When viewing the former value of the push registration, please make sure that the `email` and `password` are filled in with the user's email and password of the QD framework.
:::
## Push Methods
The QD framework provides the following push methods:
### E-mail Push
E-mail push does not need to set parameters in `Toolbox`->`Push Registration`, you need to configure the following parameters in the environment variables:
Variable name|Required|Default|Description
:-: | :-: | :-: | :-:
MAIL_SMTP|True|""|Email SMTP server
MAIL_PORT|False|465|Email SMTP server port
MAIL_SSL|False|True|Whether to use SSL
MAIL_STARTTLS|False|False|Whether to use TLS
MAIL_USER|True|""|Email username
MAIL_PASSWORD|True|""|Email password
MAIL_FROM|False|MAIL_USER|The Email used when sending, the default is the same as MAIL_USER
MAIL_DOMAIN_HTTPS|False|False|Whether to use HTTPS for email domain name. Not the framework itself HTTPS configuration. If you need HTTPS, please use an external reverse proxy
If you are using the following email, refer to the SMTP enable method and configuration method below to get your SMTP server address and port.
Email|SMTP enable method|SMTP configuration method|Other instructions
:-: | :-: | :-: | :-:
Tencent Enterprise Mail|[How to enable Tencent Enterprise Mail POP/SMTP/IMAP service?](https://open.work.weixin.qq.com/help2/pc/19886)|[Common mail client software settings](https://open.work.weixin.qq.com/help2/pc/19885)|[How do members bind/associate WeChat and enable secure login to get client-specific passwords?](https://open.work.weixin.qq.com/help2/pc/19902)
QQ Mail|[How to enable QQ Mail POP3/SMTP/IMAP service?](https://service.mail.qq.com/detail/0/428)|[How to turn on POP3/SMTP/IMAP?](https://service.mail.qq.com/detail/0/310)|[Why do I need to set a separate password to enable POP3/SMTP/IMAP?](https://service.mail.qq.com/detail/0/86)
Netease Enterprise Mail|-|[Enterprise Mail POP, SMTP, IMAP server address settings.](https://qiye.163.com/help/client-profile.html)|[What is a client authorization code and how do I use it?](https://qiye.163.com/help/af988e.html)
Netease Mail|[What is POP3, SMTP and IMAP?](https://help.mail.163.com/faqDetail.do?code=d7a5dc8471cd0c0e8b4b8f4f8e49998b374173cfe9171305fa1ce630d7f67ac21b87735d7227c217)|[How to enable client protocol?](https://help.mail.163.com/faqDetail.do?code=d7a5dc8471cd0c0e8b4b8f4f8e49998b374173cfe9171305fa1ce630d7f67ac2a5feb28b66796d3b)|-
Gmail|-|[How to use POP3/SMTP/IMAP service?](https://support.google.com/mail/answer/7104828?hl=en)|[Sign in with app passwords](https://support.google.com/accounts/answer/185833?hl=en)
Outlook|-|[POP, IMAP and SMTP settings](https://support.microsoft.com/en-us/office/pop-imap-and-smtp-settings-8361e398-8af4-4e97-b147-6c6c4ac95353)|[Use app passwords with apps that don't support two-step verification](https://support.microsoft.com/en-us/account-billing/using-app-passwords-with-apps-that-don-t-support-two-step-verification-5896ed9b-4263-e681-128a-a6f2979a7944)
::: tip MailGun
If you have configured MailGun, please configure the following parameters in the environment variables:
Variable name|Required|Default|Description
:-: | :-: | :-: | :-:
MAILGUN_KEY|True|""|MailGun API Key
MAILGUN_DOMAIN|True|DOMAIN|MailGun Domain, The default is the value of the DOMAIN in the environment variables, Please configure the value of DOMAIN in the environment variables, and set the corresponding Domain in the MailGun console, otherwise MailGun cannot be used
:::
### Bark Push
Bark push needs to set parameters in `Toolbox`->`Push Registration`:
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
BarkUrl|True|""|Bark push address, The format is `https://api.day.app/push_key`, The push_key can be obtained in the Bark client, If you are using a self-built Bark service, please replace `https://api.day.app/` with your Bark service address. For example: `http://bark.example.com/push_key`
### Server Chan Push
Server Chan push needs to set parameters in `Toolbox`->`Push Registration`:
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
skey|True|""|Server Chan push sendkey, Can be obtained in [Server Chan](https://sct.ftqq.com/sendkey)
### Telegram Bot Push
Telegram Bot push needs to set parameters in `Toolbox`->`Push Registration`:
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
TG_TOKEN|True|""|Telegram Bot Token, Can be obtained in [BotFather](https://t.me/BotFather), Should be a combination of Bot ID and corresponding Key, but not including `bot`, That is, the token form: `1111111111:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA`
TG_USERID|True|""|Telegram Chat ID, Can be obtained in [Telegram API](https://api.telegram.org/bot1111111111:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/getUpdates), The `chat_id` field in the Telegram API, such as `222222222`
TG_HOST|False|""|Telegram API Host, Can be a domain name or IP address, For example, `tg.mydomain.com`, You can also add the `http://` or `https://` prefix, If left blank, the default value `api.telegram.org` is used
PROXY_URL|False|""|Proxy address, The format is `scheme://username:password@host:port`, For example, `http://user:password@host:port`, If left blank, the Proxy is not used
PUSH_PIC_URL|False|""|Custom push picture address, If left blank, the environment variable `PUSH_PIC_URL` value is used
::: details Example
Assume that you have created a Telegram Bot API with a custom domain name:
```https://tg.mydomain.com/bot1111111111:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/sendMessage?chat_id=222222222&text=HelloWorld```
The above request will send a `HelloWorld` message to the chat `222222222`, then when registering the Telegram Bot as a push method:
- `TG_TOKEN` is `1111111111:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA`
- `TG_USERID` is `222222222`
- `TG_HOST` is `tg.mydomain.com`
Therefore, the final form is as follows:
``` Text
1111111111:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA;222222222;tg.mydomain.com
```
:::
### DingTalk Push
DingTalk push needs to set parameters in `Toolbox`->`Push Registration`:
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
DINGDING_TOKEN|True|""|DingTalk push Token, Can be obtained in [Custom Robot Access](https://open.dingtalk.com/document/robots/custom-robot-access), If you set the `IP address range` in `Security Settings`, Please add the IP address of the QD server to `IP address range`, Otherwise, you will not be able to receive push messages; If you set the `Custom Keywords` in `Security Settings`, Please add `QD`/`Push`/`Test` to `Custom Keywords`, Otherwise, you will not be able to receive push messages; Please do not enable `Signature` in `Security Settings`, QD framework does not support DingTalk Signature push for the time being.
PUSH_PIC_URL|False|""|Custom push picture address, If left blank, the environment variable `PUSH_PIC_URL` value is used
### WXPusher Push
WXPusher push needs to set parameters in `Toolbox`->`Push Registration`:
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
APPTOKEN|True|""|WXPusher push Token, Can be obtained in [WXPusher](https://wxpusher.zjiecode.com/docs/#/?id=%e8%8e%b7%e5%8f%96apptoken)
WxPusher_UID|True|""|WXPusher push UID, Can be obtained in [WXPusher](https://wxpusher.zjiecode.com/docs/#/?id=%e8%8e%b7%e5%8f%96uid)
### WeCom Application Push
WeCom application push needs to set parameters in `Toolbox`->`Push Registration`:
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
CorpID|True|""|WeCom CorpID, Can be obtained in [WeCom](https://developer.work.weixin.qq.com/document/path/98728)
AgentID|True|""|WeCom application AgentID, Can be obtained in [WeCom](https://developer.work.weixin.qq.com/document/path/90665#agentid)
AgentSecret|True|""|WeCom application Secret, Can be obtained in [WeCom](https://developer.work.weixin.qq.com/document/path/90665#secret)
PUSH_PIC_URL_or_Media_id|False|""|Custom push picture address or Media_id, Media_id can be obtained through [WeCom](https://developer.work.weixin.qq.com/document/path/90253#10112) interface, If left blank, the environment variable `PUSH_PIC_URL` value is used
QYWX_PROXY_HOST|False|""|WeCom Host, Can be a domain name or IP address, For example, `qywx.mydomain.com`, You can also add the `http://` or `https://` prefix, If left blank, the default value `https://qyapi.weixin.qq.com/` is used
::: tip QYWX_PROXY_HOST
If you use Nginx to proxy the WeCom application push, the following is an example of Nginx configuration:
``` Nginx
server {
listen 443 ssl;
server_name qywx.mydomain.com;
ssl_certificate /etc/nginx/ssl/qywx.mydomain.com.crt;
ssl_certificate_key /etc/nginx/ssl/qywx.mydomain.com.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
ssl_prefer_server_ciphers on;
location / {
proxy_pass https://qyapi.weixin.qq.com/;
}
}
```
:::
### WeCom Webhook Push
WeCom Webhook push needs to set parameters in `Toolbox`->`Push Registration`:
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
QYWX_WebHook_Key|True|""|WeCom Webhook Key, Can be obtained in [WeCom](https://work.weixin.qq.com/api/doc/90000/90136/91770)
### Custom Push
Custom push supports `GET` and `POST` push methods, using `{log}` and `{t}` to represent the log and title to be replaced.
Custom push needs to set parameters in `Toolbox`->`Push Registration`:
#### Custom Get Push
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
URL|True|""|Custom Get push address, For example, `https://example.com/push?log={log}&t={t}`
GET_Header|False|""|Custom Get push Header, Use json format (double quotes), the format is `{ "key1": "value1", "key2": "value2" }`, If left blank, the Header is not set
#### Custom Post Push
Parameter name|Required|Default|Description
:-: | :-: | :-: | :-:
URL|True|""|Custom Post push address, For example, `https://example.com/push`
POST_Header|False|""|Custom Post push Header, Use json format (double quotes), the format is `{ "key1": "value1", "key2": "value2" }`, For example, `{ "key1": "{log}", "key2": "{t}" }`, If left blank, the Header is not set
POST_Data|False|""|Custom Post push Body, Use json format (double quotes), For example, `{ "key1": "{log}", "key2": "{t}" }`, If left blank, the Body is not set
## Push Settings
After registering the push method in `Toolbox`->`Push Registration`, you can set the trigger conditions of the push method in `Toolbox`->`Push Settings`.
In `Push Settings`, you can set the push switch of each task, the task result push channel, the task result notification selection, and the task result batch push.
### Task Result Push Channel
The task result push channel is used to set the task result push channel. The task result push channel includes the following:
- [Bark Push](#bark-push)
- [Server Chan Push](#server-chan-push)
- [WXPusher Push](#wxpusher-push)
- [E-mail Push](#e-mail-push)
- [WeCom Application Push](#wecom-application-push)
- [Telegram Bot Push](#telegram-bot-push)
- [DingTalk Push](#dingtalk-push)
- [WeCom Webhook Push](#wecom-webhook-push)
- [Custom Push](#custom-push)
### Task Result Notification Selection
The task result notification selection is used to set when to push the task result. The task result notification selection includes the following:
- Manual execution success notification
- Manual execution failure notification
- Automatic execution success notification
- Automatic execution failure notification
`Notify after automatic error` can be set to push notifications after automatic execution fails several times. For example, if set to `3`, the notification will be pushed after automatic execution fails 3 times.
### Task Result Batch Push
The task result batch push is used to set the batch push of the task result. When `Enable batch push` is turned on, the task result within the specified time interval before the current push time will be batch pushed according to the `Batch push time setting` and `Batch push time interval` when the task result is pushed.
- `Batch push time setting`: The initial batch push time setting, for example, set to `12:00:00`, then a batch push will be made at `12:00:00` on the same day.
- `Batch push time interval`: Set how many seconds to push the task result once. The default is `86400` seconds, that is, the task result within 86400 seconds before the current push time will be batch pushed every day.
---
## File: web/docs/zh_CN/guide/deployment.md
# 部署
## Docker 容器部署
Docker 容器部署是部署 QD 的最简单方式。
> 操作前请一定要记得备份数据库!!!
### 容器
**DockerHub 网址**：[https://hub.docker.com/r/qdtoday/qd](https://hub.docker.com/r/qdtoday/qd)
> Tag 含义:
>
> - `latest`: 最新 Release 版本
> - `lite-latest`: 去除 OCR 相关功能的最新 Release 版本
> - `ja3-latest`: 集成 curl-impersonate 解决 ja3 指纹被识别为 curl 的问题, 不支持 http3 和 Quic 连接
> - `20xxxxxx`: 指定 Release 版本, 版本号表示为 Release 发布日期
> - `dev`: 最新开发版, 同步最新源码, 不保证稳定性
### 部署方法
#### 1. Docker Compose（推荐）
``` sh
# 创建并切换到 QD 目录。
mkdir -p $(pwd)/qd/config && cd $(pwd)/qd
# 下载 docker-compose.yml
wget https://fastly.jsdelivr.net/gh/qd-today/qd@master/docker-compose.yml
# 根据需求和配置说明修改配置环境变量
vi ./docker-compose.yml
# 执行 Docker Compose 命令
docker-compose up -d
```
> 配置描述见下文 [Configuration](#配置环境变量)
>
> 如不需要`OCR功能`或者`硬盘空间不大于600M`, 请使用 **`qdtoday/qd:lite-latest`** 镜像, **该镜像仅去除了OCR相关功能, 其他与主线版本保持一致**。
>
> **请勿使用 阿里云镜像源 拉取 Docker 容器, 会导致无法拉取最新镜像**
#### 2. 1Panel 部署
2.1. 在 1Panel 中创建一个新的应用
> 不同 1Panel 版本显示 QD 框架名称可能不同, 但均为 `QD` 图标.
2.2. 配置相关设置
> 如需设置环境变量请点击 `编辑 Compose 文件`
>
> 配置描述见下文 [Configuration](#配置环境变量)
>
> 如不需要`OCR功能`或者`硬盘空间不大于600M`, 请使用 **`qdtoday/qd:lite-latest`** 镜像, **该镜像仅去除了OCR相关功能, 其他与主线版本保持一致**。
2.3. 点击 `确认` 即可通过 1Panel 安装 QD
#### 3. Docker 运行
``` sh
docker run -d --name qd -p 8923:80 -v $(pwd)/qd/config:/usr/src/app/config qdtoday/qd
```
容器内部无法连通外部网络时尝试该命令:
``` sh
# 使用 Host 网络模式创建容器, 端口号: 8923
docker run -d --name qd --env PORT=8923 --net=host -v $(pwd)/qd/config:/usr/src/app/config qdtoday/qd
```
> 注意: 使用该命令创建容器后, 请将模板里 `http://localhost/` 形式的 api 请求, 手动改成 `api://` 或 `http://localhost:8923/` 后, 才能正常完成相关API请求。
>
> **请勿同时运行新旧版 QD 框架, 或将不同运行中的 QD 容器数据库映射为同一文件。**
## 源码部署
1. **Version >= python3.9**
``` sh
# 请先cd到框架源码根目录
pip3 install -r requirements.txt
```
2. **修改相关设置**
``` sh
# 执行以下命令复制配置文件
# 修改 local_config.py 文件的内容不受通过 git 更新源码的影响
cp config.py local_config.py
# 根据需求和配置说明修改配置文件或环境变量值
vi local_config.py
```
3. **启动**
``` sh
python ./run.py
```
4. **访问**
``` sh
# 访问网页
http://localhost:8923/
```
> 如果您使用的是源码部署方式，请手动将模板中 `http://localhost/` 形式的 api 请求改为 `api://` 或 `http://localhost:8923/` ，以便正确完成相关API 请求。
>
> 模板需要发布才会在「公开模板」中展示, 你需要管理员权限在「我的发布请求」中审批通过。
## 设置管理员
``` sh
python ./chrole.py your@email.address admin
```
> 首位注册用户默认为管理员, 需要先登出再登陆后才能获得完整管理员权限
## 配置环境变量
变量名|是否必须|默认值|说明
:-: | :-: | :-: | :-:
BIND|否|0.0.0.0|监听地址
PORT|否|8923|监听端口
QD_DEBUG|否|False|是否启用Debug模式
WORKER_METHOD|否|Queue|任务定时执行方式, 默认为 Queue, 可选 Queue 或 Batch, Batch 模式为旧版定时任务执行方式, 性能较弱, **建议仅当 Queue 定时执行模式失效时使用**
MULTI_PROCESS|否|False|(实验性)是否启用多进程模式, Windows平台无效
AUTO_RELOAD|否|False|是否启用自动热加载, MULTI_PROCESS=True时无效
STATIC_URL_PREFIX|否|`/static/`|静态文件URL前缀
DOMAIN|否|''|指定访问域名, **(建议修改)**, 否则通过邮件重置密码及邮箱推送等功能无效
AES_KEY|否|binux|AES加密密钥, **(强烈建议修改)**
COOKIE_SECRET|否|binux|cookie加密密钥, **(强烈建议修改)**
COOKIE_DAY|否|5|Cookie在客户端中保留的天数
DB_TYPE|否|sqlite3|需要使用MySQL时设置为'mysql'
JAWSDB_MARIA_URL|否|''|需要使用MySQL时, 设置为 (mysql://用户名:密码@hostname:port/数据库名?auth_plugin=)
QD_SQL_ECHO|否|False|是否启用 SQLAlchmey 的日志输出, 默认为 False, 设置为 True 时, 会在控制台输出 SQL 语句, 允许设置为 debug 以启用 debug 模式
QD_SQL_LOGGING_NAME|否|QD.sql_engine|SQLAlchmey 日志名称, 默认为 'QD.sql_engine'
QD_SQL_LOGGING_LEVEL|否|Warning|SQLAlchmey 日志级别, 默认为 'Warning'
QD_SQL_ECHO_POOL|否|True|是否启用 SQLAlchmey 的连接池日志输出, 默认为 True, 允许设置为 debug 以启用 debug 模式
QD_SQL_LOGGING_POOL_NAME|否|QD.sql_pool|SQLAlchmey 连接池日志名称, 默认为 'QD.sql_pool'
QD_SQL_LOGGING_POOL_LEVEL|否|Warning|SQLAlchmey 连接池日志级别, 默认为 'Warning'
QD_SQL_POOL_SIZE|否|10|SQLAlchmey 连接池大小, 默认为 10
QD_SQL_MAX_OVERFLOW|否|50|SQLAlchmey 连接池最大溢出, 默认为 50
QD_SQL_POOL_PRE_PING|否|True|是否在连接池获取连接前, 先ping一下, 默认为 True
QD_SQL_POOL_RECYCLE|否|3600|SQLAlchmey 连接池回收时间, 默认为 3600
QD_SQL_POOL_TIMEOUT|否|60|SQLAlchmey 连接池超时时间, 默认为 60
QD_SQL_POOL_USE_LIFO|否|True|SQLAlchmey 是否使用 LIFO 算法, 默认为 True
REDISCLOUD_URL|否|''|需要使用Redis或RedisCloud时, 设置为
REDIS_DB_INDEX|否|1|默认为1
QD_EVIL|否|500|(限Redis连接已开启)登录用户或IP在1小时内 分数 = 操作失败(如登录, 验证, 测试等操作)次数 * 相应惩罚分值 分数达到evil上限后自动封禁直至下一小时周期
EVIL_PASS_LAN_IP|否|True|是否关闭本机私有IP地址用户及Localhost_API请求的evil限制
TRACEBACK_PRINT|否|False|是否启用在控制台日志中打印Exception的TraceBack信息
PUSH_PIC_URL|否|[push_pic.png](https://fastly.jsdelivr.net/gh/qd-today/qd@master/web/static/img/push_pic.png)|默认为[push_pic.png](https://fastly.jsdelivr.net/gh/qd-today/qd@master/web/static/img/push_pic.png)
PUSH_BATCH_SW|否|True|是否允许开启定期推送 QD 任务日志, 默认为True
MAIL_SMTP|否|""|邮箱SMTP服务器
MAIL_PORT|否|465|邮箱SMTP服务器端口
MAIL_SSL|否|True|是否启用邮箱SSL, 默认为True
MAIL_STARTTLS|否|False|是否启用邮箱STARTTLS, 默认为False
MAIL_USER|否|""|邮箱用户名
MAIL_PASSWORD|否|""|邮箱密码
MAIL_FROM|否|MAIL_USER|发送时使用的邮箱，默认与MAIL_USER相同
MAIL_DOMAIN_HTTPS|否|False|发送的邮件链接启用HTTPS, 非框架前端使用HTTPS, 如果前端需要HTTPS, 请使用反向代理.
PROXIES|否|""|全局代理域名列表,用"|"分隔
PROXY_DIRECT_MODE|否|""|全局代理黑名单模式,默认不启用 "url"为网址匹配模式;"regexp"为正则表达式匹配模式
PROXY_DIRECT|否|""|全局代理黑名单匹配规则
NEW_TASK_DELAY|否|1|新建任务后准备时间, 单位为秒, 默认为1秒
TASK_WHILE_LOOP_TIMEOUT|否|900|任务运行中单个 While 循环最大运行时间, 单位为秒, 默认为 15 分钟
TASK_REQUEST_LIMIT|否|1500|任务运行中单个任务最大请求次数, 默认为 1500 次
USE_PYCURL|否|True|是否启用Pycurl模组
ALLOW_RETRY|否|True|在Pycurl环境下部分请求可能导致Request错误时, 自动修改冲突设置并重发请求
DNS_SERVER|否|""|通过Curl使用指定DNS进行解析(仅支持Pycurl环境), 如 8.8.8.8
CURL_ENCODING|否|True|是否允许使用Curl进行Encoding操作
CURL_CONTENT_LENGTH|否|True|是否允许Curl使用Headers中自定义Content-Length请求
NOT_RETRY_CODE|否|[详见配置](https://github.com/qd-today/qd/blob/master/config.py)...|[详见配置](https://github.com/qd-today/qd/blob/master/config.py)...
EMPTY_RETRY|否|True|[详见配置](https://github.com/qd-today/qd/blob/master/config.py)...
USER0ISADMIN|否|True|第一个注册用户为管理员，False关闭
NOTEPAD_LIMIT|否|20|单个用户拥有记事本最大数量, 默认为 20
EXTRA_ONNX_NAME|否|""|config目录下自定义ONNX文件名(不填 ".onnx" 后缀)多个onnx文件名用"\|"分隔
EXTRA_CHARSETS_NAME|否|""|config目录下自定义ONNX对应自定义charsets.json文件名(不填 ".json" 后缀)多个json文件名用"\|"分隔
WS_PING_INTERVAL|No|5|WebSocket ping间隔, 单位为秒, 默认为 5s
WS_PING_TIMEOUT|No|30|WebSocket ping超时时间, 单位为秒, 默认为 30s
WS_MAX_MESSAGE_SIZE|No|10485760|WebSocket 单次接收最大消息大小, 默认为 10MB
WS_MAX_QUEUE_SIZE|No|100|WebSocket 最大消息队列大小, 默认为 100
WS_MAX_CONNECTIONS_SUBSCRIBE|No|30|WebSocket 公共模板更新页面最大连接数, 默认为 30
SUBSCRIBE_ACCELERATE_URL|No|jsdelivr_cdn|订阅加速方式或地址, 用于加速公共模板更新, 仅适用于 GitHub. [详见配置](https://github.com/qd-today/qd/blob/master/config.py)...
> 详细信息请查阅[config.py](https://github.com/qd-today/qd/blob/master/config.py)
---
## File: web/docs/zh_CN/guide/faq.md
# 常见问题
## 如何备份和恢复数据库?
QD 默认使用 **sqlite3** 作为框架数据库，`database.db` 文件保存在 `config` 目录下。使用 Docker 容器部署时，可以使用 `docker cp` 命令备份数据库文件，然后使用 `docker cp` 命令在新容器中恢复数据库文件。
```sh
# 数据库备份
docker cp container_name:/usr/src/app/config/database.db .
# 数据库恢复
docker cp database.db container_name:/usr/src/app/config/
```
## 如何在 Docker 中配置邮箱服务器?
```sh
docker run -d --name qd -p 8923:80 -v $(pwd)/qd/config:/usr/src/app/config --env MAIL_SMTP=STMP服务器 --env MAIL_PORT=邮箱服务器端口 --env MAIL_USER=用户名 --env MAIL_PASSWORD=密码 --env DOMAIN=域名 qdtoday/qd
```
## 如何在 Docker 中使用 MySQL?
```sh
docker run -d --name qd -p 8923:80 -v $(pwd)/qd/config:/usr/src/app/config --ENV DB_TYPE=mysql --ENV JAWSDB_MARIA_URL=mysql://用户名:密码@hostname:port/数据库名 qdtoday/qd
```
## 如何自己搭建 Docker 镜像?
请参考此镜像的构建文件 [Dockerfile](https://github.com/qd-today/qd/blob/master/Dockerfile)。
## 如何查看当前框架支持的 API 和 Jinja2 模板变量?
请进入框架首页，然后点击左上角的 `常用 API/过滤器` 按钮，可以查看当前框架支持的 API 和 Jinja2 模板变量。
## 如何提交 bug 问题?
请在遇到问题后开启 `Debug` 模式，然后将详细的错误日志提交至 [Issue](https://github.com/qd-today/qd/issues)。
## QD 模板一般需要哪些请求?
根据经验，以下请求是必要的：
- 登录页面
- 发布到登录页面
- 发起 用户名、密码 请求
- 发送后导致页面跳转的页面
- 翻页前后的第一个网页
## 我的用户名和密码会被泄露吗?
为了帮助用户发起请求，终究需要记录用户名和密码。这只能靠服务器维护人员的自律来保证后端数据的安全。但在框架设计中，每个用户在存储时都使用安全密钥进行加密。使用密钥对用户数据进行加密，可以保证仅获取数据库就无法解密用户数据。（加密的用户数据包括用户上传的模板、用户为任务设置的变量等）
如果还是不放心，可以自己搭建 QD 框架，下载模板在自己的服务器上运行。
## 提示错误信息 `PermissionError: [Errno 1] Operation not permitted`?
1. 如果是 ARM32 Debian 系统, 请检查 `Docker` 版本是否小于 `20.10.0`, 且 `libseccomp` 版本是否小于 `2.4.4`, 如果是的话, 请升级以上组件.
更新 `libseccomp` 参考操作:
```sh
# Get signing keys to verify the new packages, otherwise they will not install
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC 648ACFD622F3D138
# Add the Buster backport repository to apt sources.list
echo 'deb http://httpredir.debian.org/debian buster-backports main contrib non-free' | sudo tee -a /etc/apt/sources.list.d/debian-backports.list
sudo apt update
sudo apt install libseccomp2 -t buster-backports
```
> 来源于:
>
> - [https://github.com/Taxel/PlexTraktSync/pull/474](https://github.com/Taxel/PlexTraktSync/pull/474)
> - [https://stackoverflow.com/questions/70195968/dockerfile-raspberry-pi-python-pip-install-permissionerror-errno-1-operation](https://stackoverflow.com/questions/70195968/dockerfile-raspberry-pi-python-pip-install-permissionerror-errno-1-operation)
2. 请检查是否将容器内的 `/usr/src/app` 目录映射至容器外部.
> 请注意框架仅需映射 `/usr/src/app/config` 目录即可.
## 提示警告信息: `Connect Redis falied: Error 10061`
QD 使用 `redis` 作为限流工具，如果没有安装 `redis` 服务，框架会提示以下警告信息。
```sh
[W xxxxxx xx:xx:xx QD.RedisDB redisdb:28] Connect Redis falied: Error 10061 connecting to localhost:6379. 由于目标计算机积极拒绝，无法连接。
```
然而，`redis` 在框架中并不是必须的，如果你不需要使用 `限流` 功能，可以忽略该警告信息。
> 建议使用 `Docker compose` 部署 QD 框架, Docker compose 配置已默认安装 redis 容器。
## 提示警告信息: `Import PyCurl module falied`
QD 使用 `pycurl` 模块来发送 HTTP Proxy 请求。如果没有安装 `pycurl` 模块，框架会提示以下警告信息。
```sh
[W xxxxxx xx:xx:xx QD.Http.Fetcher fetcher:34] Import PyCurl module falied: No module named 'pycurl'
```
然而，`pycurl` 在框架中并不是必须的，如果你不需要使用 `Proxy` 功能，可以忽略该警告信息。
> `pycurl` 模块在 Windows 系统上安装比较麻烦，需要安装 `libcurl` 库，具体安装方法请参考 [pycurl 官方文档](http://pycurl.io/docs/latest/install.html)。
>
> 建议使用容器或 linux 系统部署 QD 框架, Docker 容器已预装 Curl 环境, 默认安装 pycurl 模组。
## 如何注册推送方式
你可以在 `工具箱`->`推送注册`中注册不同的推送工具，以便在发生特定事件（例如定时任务执行失败）时向你推送通知
请参考 [推送工具](/zh_CN/toolbox/pusher)
## 公共模板更新页面提示错误代码为 undefined
- [issue#423](https://github.com/qd-today/qd/issues/423)
> 公共模板更新页面提示错误代码为 undefined, 或者控制台显示 WebSocket 连接 failed 但不显示错误原因
请检查反向代理相关配置是否正确, 参考 [Nginx 反向代理 WebSocket 服务连接报错](https://blog.csdn.net/tiven_/article/details/126126442)
> 参考配置如下:
>
> ```Nginx
> server {
> listen 80;
> # 自行修改 server_name
> server_name qd.example.com;
> location / {
> proxy_pass http://ip:port;
>
> # WebSocket 关键配置 开始
> proxy_http_version 1.1;
> proxy_set_header Upgrade $http_upgrade;
> proxy_set_header Connection "upgrade";
> # WebSocket 关键配置 结束
>
> # 其他可选配置 开始
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Forwarded-Proto $scheme;
> # 其他可选配置 结束
> }
> }
> ```
## 错误代码：4006
> 提示错误信息为："更新失败，原因：Cannot connect to host xxx.xxx:443 ssl:False"
报错原因：Github 或 GitHub 加速源无法连接。
解决方法 1：使用代理
解决方法 2：更换 Github 加速源
容器的环境变量中增加/修改 `SUBSCRIBE_ACCELERATE_URL=https://xxx.xxx/https://raw.githubusercontent.com/`
`https://xxx.xxx/` 替换为可用的加速源，找不到加速源的可以参考 或 中发布的加速源
---
## File: web/docs/zh_CN/guide/how-to-use.md
# 如何使用?
## 获取 HAR
### 1. 什么是 HAR?
HAR：
### 2. 抓包
#### 2.1. Chrome 或 Edge
1. 按 `F12` 、 `Ctrl + Shift + I` ，或从 Chrome 菜单中选择 `更多工具` > `开发者工具` 。
2. 从屏幕底部或右侧打开的面板中，选择 `网络` 选项卡。
3. 确保 `网络` 选项卡左上角的 `录制` 按钮显示为 **红色**。
4. 如果它是灰色的，单击一下开始录制。
5. 选中 `保留日志` 旁边的框。
6. 单击 `清除` 按钮以清除 `网络` 选项卡中的所有现有日志。
7. 现在尝试重现您计划发起的 HTTP 请求。
8. 重现任务后，右键单击网络请求网格上的任意位置。
9. 选择 `另存为带内容的 HAR`。
10. 将文件保存到您的计算机。
#### 2.2. 火狐
1. 按 `F12` （或转到 `Tools` > `Web Developer` > `Network` ）。
2. 现在尝试重现您计划发起的 HTTP 请求。
3. 右键单击加载的结果。
4. 选择全部另存为 har。
#### 2.3. Windows - Fiddler
1. 打开 Fiddler，打开 `Tools` 菜单，选择 `Options`。
2. 选择 `HTTPS 选项卡` ，勾选 `Capture HTTPS Connects` 。
3. 选择 `连接` 选项卡，选中 `解密 HTTPS 流量` 。
4. 现在尝试重现您计划发起的 HTTP 请求。
5. 导出为 HAR 格式 - 请选择 HTTPArchive 作为导出方式。
#### 2.4. IOS - Stream
1. 打开 Stream。
2. 抓取HTTPS请求前，需要安装CA证书， `setting` > `General` > `About` > `Certificate Trust Settings` 信任CA证书。
3. 点击开始抓包按钮，手机会自动弹出VPN配置窗口，然后选择允许。
4. 现在尝试重现您计划发起的 HTTP 请求。
5. 在应用页面，点击停止抓拍结束本次抓拍。
6. 导出HAR文件。
### 3. 公共模板
1. 点击 `我的模板` 右侧的 `社区模板` 按钮。
2. 更新存储库以获取最新的 HAR 文件列表。
3. 选择你要使用的HAR文件，点击 `Subscribe` 按钮订阅模板。
4. 根据需要修改模板。
5. 然后跳到 [3.编辑模板](#编辑模板) 继续编辑。
## 上传 HAR 文件
1. 访问并登录 QD 首页。
2. 点击 `我的模板` 右侧的 `+` 按钮。
3. 上传刚刚保存的 HAR 文件。
4. 点击 `上传` 继续。
## 编辑模板
::: v-pre
1. 将用户名、密码、cookie、header 等根据用户变化的部分替换成类似 `{{ username }}` 的模板。（模板支持 **jinja2 语法**）
2. 在请求编辑中使用测试面板测试模板是否正确返回，使用正确和错误的用户名进行测试。
3. 填写 `成功/失败断言`，这有助于检测模板任务是否运行失败。
4. 当请求中需要上一次请求的一些数据时，使用变量抽取，通过正则化抽取数据，保存在环境变量中。使用组选择器 `ab(\w+)cd` 选择其中的一部分。
5. 使用 `__log__` 提取任务日志。
6. 当所有请求编辑完成后，使用下载按钮旁边的 `测试` 来测试整体。
7. 正在编辑的模板会自动保存在浏览器缓存中，不用担心丢失。
8. 单击 `保存` 按钮保存模板。
9. 单击 `下载` 按钮下载模板。
:::
## 创建定时任务
1. 点击 `我的任务` 右侧的 `+` 按钮。
2. 选择刚刚创建的模板。
3. 填写任务变量，任务间隔、任务组和任务执行时间。
4. 单击 `测试` 按钮测试任务。
5. 单击保存按钮保存任务。