lux Documentation

Fast and simple video download library and CLI tool written in Go

31,326 stars Go Go 1 file · ~9,660 tokens #bilibili#crawler#download#downloader#go#golang#iqiyi#qq
RAW Doc

1. Project Overview & Quickstart (iawia002/lux)

Lux

👾 Lux is a fast and simple video downloader built with Go.

Installation

Prerequisites

The following dependencies are required and must be installed separately.

Note: FFmpeg does not affect the download, only affects the final file merge.

Install via `go install`

To install Lux, use go install, or download the binary file from Releases page.

bash
$ go install github.com/iawia002/lux@latest

Homebrew (macOS only)

For macOS users, you can install lux via:

bash
$ brew install lux

Arch Linux

For Arch Users AUR package is available.

Void Linux

For Void linux users, you can install lux via:

text
$ xbps-install -S lux

[Scoop](https://scoop.sh/) on Windows

sh
$ scoop install lux

[Chocolatey](https://chocolatey.org/) on Windows

text
$ choco install lux

[Cask](https://github.com/axetroy/cask.rs) on Windows/macOS/Linux

sh
$ cask install github.com/iawia002/lux

Getting Started

Usage:

text
lux [OPTIONS] URL [URL...]

Download a video

console
$ lux "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

 Site:      YouTube youtube.com
 Title:     Rick Astley - Never Gonna Give You Up (Video)
 Type:      video
 Stream:
     [248]  -------------------
     Quality:         1080p video/webm; codecs="vp9"
     Size:            63.93 MiB (67038963 Bytes)
     # download with: lux -f 248 ...

 41.88 MiB / 63.93 MiB [=================>-------------]  65.51% 4.22 MiB/s 00m05s

The -i option displays all available quality of video without downloading.

console
$ lux -i "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

 Site:      YouTube youtube.com
 Title:     Rick Astley - Never Gonna Give You Up (Video)
 Type:      video
 Streams:   # All available quality
     [248]  -------------------
     Quality:         1080p video/webm; codecs="vp9"
     Size:            49.29 MiB (51687554 Bytes)
     # download with: lux -f 248 ...

     [137]  -------------------
     Quality:         1080p video/mp4; codecs="avc1.640028"
     Size:            43.45 MiB (45564306 Bytes)
     # download with: lux -f 137 ...

     [398]  -------------------
     Quality:         720p video/mp4; codecs="av01.0.05M.08"
     Size:            37.12 MiB (38926432 Bytes)
     # download with: lux -f 398 ...

     [136]  -------------------
     Quality:         720p video/mp4; codecs="avc1.4d401f"
     Size:            31.34 MiB (32867324 Bytes)
     # download with: lux -f 136 ...

     [247]  -------------------
     Quality:         720p video/webm; codecs="vp9"
     Size:            31.03 MiB (32536181 Bytes)
     # download with: lux -f 247 ...

Use lux -f stream "URL" to download a specific stream listed in the output of -i option.

Download anything else

If Lux is provided the URL of a specific resource, then it will be downloaded directly:

console
$ lux "https://img9.bcyimg.com/drawer/15294/post/1799t/1f5a87801a0711e898b12b640777720f.jpg"

lux doesn't support this URL right now, but it will try to download it directly

 Site:      Universal
 Title:     1f5a87801a0711e898b12b640777720f
 Type:      image/jpeg
 Stream:
     [default]  -------------------
     Size:            1.00 MiB (1051042 Bytes)
     # download with: lux -f default "URL"

 1.00 MiB / 1.00 MiB [===================================] 100.00% 1.21 MiB/s 0s

Download playlist

The -p option downloads an entire playlist instead of a single video.

console
$ lux -i -p "https://www.bilibili.com/bangumi/play/ep198061"

 Site:      哔哩哔哩 bilibili.com
 Title:     Doctor X 第四季:第一集
 Type:      video
 Streams:   # All available quality
     [default]  -------------------
     Quality:         高清 1080P
     Size:            845.66 MiB (886738354 Bytes)
     # download with: lux -f default "URL"


 Site:      哔哩哔哩 bilibili.com
 Title:     Doctor X 第四季:第二集
 Type:      video
 Streams:   # All available quality
     [default]  -------------------
     Quality:         高清 1080P
     Size:            930.71 MiB (975919195 Bytes)
     # download with: lux -f default "URL"

......

You can use the -start, -end or -items option to specify the download range of the list:

text
-start
    	Playlist video to start at (default 1)
-end
    	Playlist video to end at
-items
    	Playlist video items to download. Separated by commas like: 1,5,6,8-10

For bilibili playlists only:

text
-eto
  File name of each bilibili episode doesn't include the playlist title

Multiple inputs

You can also download multiple URLs at once:

console
$ lux -i "https://www.bilibili.com/video/av21877586" "https://www.bilibili.com/video/av21990740"

 Site:      哔哩哔哩 bilibili.com
 Title:     【莓机会了】甜到虐哭的13集单集MAD「我现在什么都不想干,更不想看14集」
 Type:      video
 Streams:   # All available quality
     [default]  -------------------
     Quality:         高清 1080P
     Size:            51.88 MiB (54403767 Bytes)
     # download with: lux -f default "URL"


 Site:      哔哩哔哩 bilibili.com
 Title:     【莓救了】甜到虐哭!!!国家队单集MAD-当熟悉的bgm响起,眼泪从脸颊滑下
 Type:      video
 Streams:   # All available quality
     [default]  -------------------
     Quality:         高清 1080P
     Size:            77.63 MiB (81404093 Bytes)
     # download with: lux -f default "URL"

These URLs will be downloaded one by one.

You can also use the -F option to read URLs from file:

console
$ lux -F ~/Desktop/u.txt

 Site:      微博 weibo.com
 Title:     在Google,我们设计什么? via@阑夕
 Type:      video
 Stream:
     [default]  -------------------
     Size:            19.19 MiB (20118196 Bytes)
     # download with: lux -f default "URL"

 19.19 MiB / 19.19 MiB [=================================] 100.00% 9.69 MiB/s 1s

......

You can use the -start, -end or -items option to specify the download range of the list:

text
-start
    	File line to start at (default 1)
-end
    	File line to end at
-items
    	File lines to download. Separated by commas like: 1,5,6,8-10

Resume a download

Ctrl+C interrupts a download.

A temporary .download file is kept in the output directory. If lux is ran with the same arguments, then the download progress will resume from the last session.

Auto retry

lux will auto retry when the download failed, you can specify the retry times by -retry option (default is 100).

Cookies

Cookies can be provided to lux with the -c option if they are required for accessing the video.

Cookies can be the following format or Netscape Cookie format:

console
name=value; name2=value2; ...

Cookies can be a string or a text file, supply cookies in one of the two following ways.

As a string:

console
$ lux -c "name=value; name2=value2" "https://www.bilibili.com/video/av20203945"

As a text file:

console
$ lux -c cookies.txt "https://www.bilibili.com/video/av20203945"

Proxy

You can set the HTTP/SOCKS5 proxy using environment variables:

console
$ HTTP_PROXY="http://127.0.0.1:1087/" lux -i "https://www.youtube.com/watch?v=Gnbch2osEeo"
console
$ HTTP_PROXY="socks5://127.0.0.1:1080/" lux -i "https://www.youtube.com/watch?v=Gnbch2osEeo"

Multi-Thread

Use --multi-thread or -m multiple threads to download single video.

Use --thread or -n option to set the number of download threads(default is 10).

Note: If the video has multi fragment, the number of actual download threads will increase.

For example:

  • If -n is set to 10, and the video has 2 fragments, then 20 threads will actually be used.
  • If the video has 20 fragments, only 10 fragments are downloaded in the same time, the actual threads count is 100.

Special Tips: Use too many threads in mgtv download will cause HTTP 403 error, we recommend setting the number of threads to 1.

bilibili

You can just use av or ep number to download bilibili's video:

console
$ lux -i ep198381 av21877586

 Site:      哔哩哔哩 bilibili.com
 Title:     狐妖小红娘:第79话 南国公主的吃货本色
 Type:      video
 Streams:   # All available quality
     [default]  -------------------
     Quality:         高清 1080P
     Size:            485.23 MiB (508798478 Bytes)
     # download with: lux -f default "URL"


 Site:      哔哩哔哩 bilibili.com
 Title:     【莓机会了】甜到虐哭的13集单集MAD「我现在什么都不想干,更不想看14集」
 Type:      video
 Streams:   # All available quality
     [default]  -------------------
     Quality:         高清 1080P
     Size:            51.88 MiB (54403767 Bytes)
     # download with: lux -f default "URL"

Use specified Referrer

A Referrer can be used for the request with the -r option:

console
$ lux -r "https://www.bilibili.com/video/av20383055/" "http://cn-scnc1-dx.acgvideo.com/"

Specify the output path and name

The -o option sets the path, and -O option sets the name of the downloaded file:

console
$ lux -o ../ -O "hello" "https://example.com"

Debug Mode

The -d option outputs network request messages:

console
$ lux -i -d "http://www.bilibili.com/video/av20088587"

URL:         http://www.bilibili.com/video/av20088587
Method:      GET
Headers:     http.Header{
    "Referer":         {"http://www.bilibili.com/video/av20088587"},
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
}
Status Code: 200

URL:         https://interface.bilibili.com/v2/playurl?appkey=84956560bc028eb7&cid=32782944&otype=json&qn=116&quality=116&type=&sign=fb2e3f261fec398652f96d358517e535
Method:      GET
Headers:     http.Header{
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"},
    "Referer":         {"https://interface.bilibili.com/v2/playurl?appkey=84956560bc028eb7&cid=32782944&otype=json&qn=116&quality=116&type=&sign=fb2e3f261fec398652f96d358517e535"},
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
}
Status Code: 200

 Site:      哔哩哔哩 bilibili.com
 Title:     燃油动力的遥控奥迪R8跑赛道
 Type:      video
 Streams:   # All available quality
     [default]  -------------------
     Quality:         高清 1080P
     Size:            64.38 MiB (67504795 Bytes)
     # download with: lux -f default "URL"

Reuse extracted data

The -j option will print the extracted data in JSON format.

console
$ lux -j "https://www.bilibili.com/video/av20203945"

{
    "site": "哔哩哔哩 bilibili.com",
    "title": "【2018拜年祭单品】相遇day by day",
    "type": "video",
    "streams": {
        "15": {
            "urls": [
                {
                    "url": "...",
                    "size": 18355205,
                    "ext": "flv"
                }
            ],
            "quality": "流畅 360P",
            "size": 18355205
        },
        "32": {
            "urls": [
                {
                    "url": "...",
                    "size": 40058632,
                    "ext": "flv"
                }
            ],
            "quality": "清晰 480P",
            "size": 40058632
        },
        "64": {
            "urls": [
                {
                    "url": "...",
                    "size": 82691087,
                    "ext": "flv"
                }
            ],
            "quality": "高清 720P",
            "size": 82691087
        },
        "80": {
            "urls": [
                {
                    "url": "...",
                    "size": 121735559,
                    "ext": "flv"
                }
            ],
            "quality": "高清 1080P",
            "size": 121735559
        }
    }
}

Options

text
-i	Information only
  -F string
    	URLs file path
  -d	Debug mode
  -j	Print extracted data
  -s	Minimum outputs
  -v	Show version

Download:

text
-f string
    	Select specific stream to download
  -p	Download playlist
  -n int
    	The number of download thread (only works for multiple-parts video) (default 10)
  -c string
    	Cookie
  -r string
    	Use specified Referrer
  -cs int
    	HTTP chunk size for downloading (in MB) (default 1)

Network:

text
-retry int
    	How many times to retry when the download failed (default 10)

Playlist:

text
-start int
    	Playlist video to start at (default 1)
  -end int
    	Playlist video to end at
  -items string
    	Playlist video items to download. Separated by commas like: 1,5,6,8-10

Filesystem:

text
-o string
    	Specify the output path
  -O string
    	Specify the output file name

Subtitle:

text
-C	Download subtitles
  -C -items en,zh
    	Download specific languages (YouTube only)
  -C -items en,zh -embed 
    	Embed subtitles into the video (YouTube only)

Youku:

text
-ccode string
    	Youku ccode (default "0502")
  -ckey string
    	Youku ckey (default "7B19C0AB12633B22E7FE81271162026020570708D6CC189E4924503C49D243A0DE6CD84A766832C2C99898FC5ED31F3709BB3CDD82C96492E721BDD381735026")
  -password string
    	Youku password

aria2:

Note: If you use aria2 to download, you need to merge the multi-part videos yourself.

text
-aria2
    	Use Aria2 RPC to download
  -aria2addr string
    	Aria2 Address (default "localhost:6800")
  -aria2method string
    	Aria2 Method (default "http")
  -aria2token string
    	Aria2 RPC Token

Supported Sites

Site URL 🎬 Videos 🌁 Images 🔊 Audio 📚 Playlist 🍪 VIP adaptation Build Status
抖音 https://www.douyin.com ✓ ✓
哔哩哔哩 https://www.bilibili.com ✓ ✓ ✓
半次元 https://bcy.net ✓
pixivision https://www.pixivision.net ✓
优酷 https://www.youku.com ✓ ✓
YouTube https://www.youtube.com ✓ ✓
西瓜视频(头条) https://m.toutiao.com, https://v.ixigua.com, https://www.ixigua.com ✓
爱奇艺 https://www.iqiyi.com ✓
新片场 https://www.xinpianchang.com ✓
芒果 TV https://www.mgtv.com ✓
糖豆广场舞 https://www.tangdou.com ✓
Tumblr https://www.tumblr.com ✓ ✓
Vimeo https://vimeo.com ✓
Facebook https://facebook.com ✓
斗鱼视频 https://v.douyu.com ✓
秒拍 https://www.miaopai.com ✓
微博 https://weibo.com ✓
Instagram https://www.instagram.com ✓ ✓
Threads https://www.threads.net ✓ ✓
Twitter https://twitter.com ✓
腾讯视频 https://v.qq.com ✓
网易云音乐 https://music.163.com ✓
音悦台 https://yinyuetai.com ✓
极客时间 https://time.geekbang.org ✓
Pornhub https://pornhub.com ✓
XVIDEOS https://xvideos.com ✓
聯合新聞網 https://udn.com ✓
TikTok https://www.tiktok.com ✓
Pinterest https://www.pinterest.com ✓
好看视频 https://haokan.baidu.com ✓
AcFun https://www.acfun.cn ✓ ✓
Eporner https://eporner.com ✓
StreamTape https://streamtape.com ✓
虎扑 https://hupu.com ✓
虎牙视频 https://v.huya.com ✓
喜马拉雅 https://www.ximalaya.com ✓
快手 https://www.kuaishou.com ✓
Reddit https://www.reddit.com ✓ ✓
VKontakte https://vk.com ✓
知乎 https://zhihu.com ✓
Rumble https://rumble.com ✓
小红书 https://xiaohongshu.com ✓
Zing MP3 https://zingmp3.vn ✓ ✓
Bitchute https://www.bitchute.com ✓
Odysee https://odysee.com ✓ ✓

Known issues

优酷

优酷的 ccode 经常变化导致 lux 不可用,如果你知道有新的可用的 ccode,可以直接使用 lux -ccode ... 而不用等待 lux 更新(当然,也欢迎你给我们提一个 Pull request 来更新默认的 ccode)

最好是每次下载都附带登录过的 Cookie 以避免部分 ccode 的问题

西瓜/头条视频

西瓜/头条视频必须带 Cookie 才能下载成功,西瓜和头条可共用西瓜视频的 Cookie,Cookie 的有效期可能较短,下载失败就更新 Cookie 尝试:

text
$ lux -c "msToken=yoEh0-qLUq4obZ8Sfxsem_CxCo9R3NM6ViTrWaRcM1...; ttwid=1%7C..." "https://m.toutiao.com/is/iYbTfJ79/"

Contributing

Lux is an open source project and built on the top of open-source projects. Check out the Contributing Guide to get started.

Authors

Code with ❤️ by iawia002 and lovely contributors

Similar projects

License

MIT

Copyright (c) 2018-present, iawia002

2. Official Technical Reference & Guides (iawia002/website)

The Kubernetes documentation

This repository contains the assets required to build the Kubernetes website and documentation. We're glad that you want to contribute!

Using this repository

You can run the website locally using Hugo (Extended version), or you can run it in a container runtime. We strongly recommend using the container runtime, as it gives deployment consistency with the live website.

Prerequisites

To use this repository, you need the following installed locally:

[!NOTE]
Make sure to install the Hugo extended version specified by the HUGO_VERSION environment variable in the netlify.toml file.

Before you start, install the dependencies. Clone the repository and navigate to the directory:

bash
git clone https://github.com/kubernetes/website.git
cd website

The Kubernetes website uses the Docsy Hugo theme,
which can be installed via npm. You can also download a pre-configured
development container image that includes Hugo and Docsy. Additionally, a Git
submodule is used for tools that generate the reference documentation.

Windows

powershell
# fetch submodule dependencies
git submodule update --init --recursive --depth 1

Linux / other Unix

bash
# fetch submodule dependencies
make module-init

Running the website using a container

To build the site in a container, run the following:

bash
# You can set $CONTAINER_ENGINE to the name of any Docker-like container tool
make container-serve

If you see errors, it probably means that the hugo container did not have enough computing resources available. To solve it, increase the amount of allowed CPU and memory usage for Docker on your machine (MacOS and Windows).

Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh.

Running the website locally using Hugo

To install dependencies, deploy and test the site locally, run:

  • For macOS and Linux

    bash
    npm ci
    make serve
  • For Windows (PowerShell)

    powershell
    npm ci
    hugo.exe server --buildFuture --environment development

This will start the local Hugo server on port 1313. Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh.

Building the API reference pages

The API reference pages located in content/en/docs/reference/kubernetes-api are built from the Swagger specification, also known as OpenAPI specification, using https://github.com/kubernetes-sigs/reference-docs/tree/master/gen-resourcesdocs.

To update the reference pages for a new Kubernetes release follow these steps:

  1. Pull in the api-ref-generator submodule:

    bash
    git submodule update --init --recursive --depth 1
  2. Update the Swagger specification:

    bash
    curl 'https://raw.githubusercontent.com/kubernetes/kubernetes/master/api/openapi-spec/swagger.json' > api-ref-assets/api/swagger.json
  3. In api-ref-assets/config/, adapt the files toc.yaml and fields.yaml to reflect the changes of the new release.

  4. Next, build the pages:

    bash
    make api-reference

    You can test the results locally by building and serving the site from a container:

    bash
    make container-serve

    In a web browser, go to http://localhost:1313/docs/reference/kubernetes-api/ to view the API reference.

  5. When all changes of the new contract are reflected into the configuration files toc.yaml and fields.yaml, create a Pull Request with the newly generated API reference pages.

Troubleshooting

error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version

Hugo is shipped in two set of binaries for technical reasons. The current website runs based on the Hugo Extended version only. In the release page look for archives with extended in the name. To confirm, run hugo version and look for the word extended.

Troubleshooting macOS for too many open files

If you run make serve on macOS and receive the following error:

bash
ERROR 2020/08/01 19:09:18 Error: listen tcp 127.0.0.1:1313: socket: too many open files
make: *** [serve] Error 1

Try checking the current limit for open files:

launchctl limit maxfiles

Then run the following commands (adapted from https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c):

shell
#!/bin/sh

# These are the original gist links, linking to my gists now.
# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxfiles.plist
# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxproc.plist

curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxfiles.plist
curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxproc.plist

sudo mv limit.maxfiles.plist /Library/LaunchDaemons
sudo mv limit.maxproc.plist /Library/LaunchDaemons

sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
sudo chown root:wheel /Library/LaunchDaemons/limit.maxproc.plist

sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist

This works for Catalina as well as Mojave macOS.

Get involved with SIG Docs

Learn more about SIG Docs Kubernetes community and meetings on the community page.

You can also reach the maintainers of this project at:

Contributing to the docs

You can click the Fork button in the upper-right area of the screen to create a copy of this repository in your GitHub account. This copy is called a fork. Make any changes you want in your fork, and when you are ready to send those changes to us, go to your fork and create a new pull request to let us know about it.

Once your pull request is created, a Kubernetes reviewer will take responsibility for providing clear, actionable feedback. As the owner of the pull request, it is your responsibility to modify your pull request to address the feedback that has been provided to you by the Kubernetes reviewer.

Also, note that you may end up having more than one Kubernetes reviewer provide you feedback or you may end up getting feedback from a Kubernetes reviewer that is different than the one initially assigned to provide you feedback.

Furthermore, in some cases, one of your reviewers might ask for a technical review from a Kubernetes tech reviewer when needed. Reviewers will do their best to provide feedback in a timely fashion but response time can vary based on circumstances.

For more information about contributing to the Kubernetes documentation, see:

New contributor ambassadors

If you need help at any point when contributing, the New Contributor Ambassadors are a good point of contact. These are SIG Docs approvers whose responsibilities include mentoring new contributors and helping them through their first few pull requests. The best place to contact the New Contributors Ambassadors would be on the Kubernetes Slack. Current New Contributors Ambassadors for SIG Docs:

Name Slack GitHub
Sreeram Venkitesh @sreeram.venkitesh @sreeram-venkitesh

Localization READMEs

Language Language
Bengali Korean
Chinese Polish
French Portuguese
German Russian
Hindi Spanish
Indonesian Ukrainian
Italian Vietnamese
Japanese

Code of conduct

Participation in the Kubernetes community is governed by the CNCF Code of Conduct.

Thank you

Kubernetes thrives on community participation, and we appreciate your contributions to our website and our documentation!