1. Project Overview & Quickstart (future-architect/vuls)
File: README.md
Vuls: VULnerability Scanner
Vulnerability scanner for Linux/FreeBSD, agent-less, written in Go.
Twitter: @vuls_en
Abstract
For a system administrator, having to perform security vulnerability analysis and software update on a daily basis can be a burden.
To avoid downtime in a production environment, it is common for a system administrator to choose not to use the automatic update option provided by the package manager and to perform update manually.
This leads to the following problems.
- The system administrator will have to constantly watch out for any new vulnerabilities in NVD (National Vulnerability Database) or similar databases.
- It might be impossible for the system administrator to monitor all the software if there are a large number of software packages installed in the server.
- It is expensive to perform analysis to determine the servers affected by new vulnerabilities. The possibility of overlooking a server or two during analysis is there.
Vuls is a tool created to solve the problems listed above. It has the following characteristics.
- Informs users of the vulnerabilities that are related to the system.
- Informs users of the servers that are affected.
- Vulnerability detection is done automatically to prevent any oversight.
- A report is generated on a regular basis using CRON or other methods. to manage vulnerability.
Main Features
Scan for any vulnerabilities in Linux/FreeBSD/Windows/macOS
Supports major Linux/FreeBSD/Windows/macOS
- Alpine, Amazon Linux, CentOS, AlmaLinux, Rocky Linux, Debian, Oracle Linux, Raspbian, RHEL, openSUSE, openSUSE Leap, SUSE Enterprise Linux, Fedora, and Ubuntu
- FreeBSD
- Windows
- macOS
- Cloud, on-premise, Running Docker Container
High-quality scan
Vulnerability Database
OVAL
Security Advisory
Commands(yum, zypper, pkg-audit)
- RHSA / ALAS / ELSA / FreeBSD-SA
- Changelog
PoC, Exploit
CERT
KEV
- CISA(Cybersecurity & Infrastructure Security Agency): Known Exploited Vulnerabilities Catalog
- VulnCheck: VulnCheck KEV
Cyber Threat Intelligence(MITRE ATT&CK and CAPEC)
Libraries
WordPress
Scan mode
- Scan without root privilege, no dependencies
- Almost no load on the scan target server
- Offline mode scan with no internet access. (CentOS, Alma Linux, Rocky Linux, Debian, Oracle Linux, Red Hat, Fedora, and Ubuntu)
- Scan with root privilege
- Almost no load on the scan target server
- Detect processes affected by update using yum-ps (Amazon Linux, CentOS, Alma Linux, Rocky Linux, Oracle Linux, Fedora, and RedHat)
- Detect processes which updated before but not restarting yet using checkrestart of debian-goodies (Debian and Ubuntu)
- Offline mode scan with no internet access. (CentOS, Alma Linux, Rocky Linux, Debian, Oracle Linux, Red Hat, Fedora, and Ubuntu)
[Remote, Local scan mode, Server mode](https://vuls.io/docs/en/architecture-remote-local.html)
- User is required to only set up one machine that is connected to other target servers via SSH
- If you don't want the central Vuls server to connect to each server by SSH, you can use Vuls in the Local Scan mode.
- First, start Vuls in server mode and listen as an HTTP server.
- Next, issue a command on the scan target server to collect software information. Then send the result to Vuls Server via HTTP. You receive the scan results as JSON format.
- No SSH needed, No Scanner needed. Only issuing Linux commands directory on the scan target server.
**Dynamic** Analysis
- It is possible to acquire the state of the server by connecting via SSH and executing the command.
- Vuls warns when the scan target server was updated the kernel etc. but not restarting it.
Scan vulnerabilities of non-OS-packages
- Libraries of programming language
- Self-compiled software
- Network Devices
Vuls has some options to detect the vulnerabilities
- Lockfile based Scan
- GitHub Integration
- Common Platform Enumeration (CPE) based Scan
- OWASP Dependency Check Integration
Scan WordPress core, themes, plugins
MISC
- Nondestructive testing
- Pre-authorization is NOT necessary before scanning on AWS
- Vuls works well with Continuous Integration since tests can be run every day. This allows you to find vulnerabilities very quickly.
- Auto-generation of configuration file template
- Auto-detection of servers set using CIDR, generate configuration file template
- Email and Slack notification is possible (supports Japanese language)
- Scan result is viewable on accessory software, TUI Viewer in a terminal or Web UI (VulsRepo).
What Vuls Doesn't Do
- Vuls doesn't update the vulnerable packages.
Document
For more information such as Installation, Tutorial, Usage, visit vuls.io
日本語翻訳ドキュメント
Authors
kotakanbe (@kotakanbe) created vuls and these fine people have contributed.
Contribute
see vulsdoc
Sponsors
License
Please see LICENSE.
File: contrib/future-vuls/README.md
future-vuls
Main Features
future-vuls upload- upload vuls results json to future-vuls
future-vuls discoverExplores hosts within the CIDR range using the ping command
Describes the information including CPEs on the found hosts in a toml-formatted file
Executes snmp2cpe(https://github.com/future-architect/vuls/pull/1625) to active hosts to obtain CPE,
Commands running internallysnmp2cpe v2c {IPAddr} public | snmp2cpe convert
Structure of toml-formatted file
[server.{ip}]
ip = {IpAddr}
server_name = ""
uuid = {UUID}
cpe_uris = []
fvuls_sync = falsefuture-vuls add-cpe- Create pseudo server to Fvuls to obtain uuid and Upload CPE information on the specified(FvulsSync is true and UUID is obtained) hosts to Fvuls
- Fvuls_Sync must be rewritten to true to designate it as the target of the command
future-vuls discoverfuture-vuls add-cpe
These two commands are used to manage the CPE of network devices, and by executing the commands in the order from the top, you can manage the CPE of each device in Fvuls
toml file after command execution
["192.168.0.10"]
ip = "192.168.0.10"
server_name = "192.168.0.10"
uuid = "e811e2b1-9463-d682-7c79-a4ab37de28cf"
cpe_uris = ["cpe:2.3:h:fortinet:fortigate-50e:-:*:*:*:*:*:*:*", "cpe:2.3:o:fortinet:fortios:5.4.6:*:*:*:*:*:*:*"]
fvuls_sync = trueInstallation
git clone https://github.com/future-architect/vuls.git
cd vuls
make build-future-vulsCommand Reference
./future-vuls -h
Usage:
future-vuls [command]
Available Commands:
add-cpe Create a pseudo server in Fvuls and register CPE. Default outputFile is ./discover_list.toml
completion Generate the autocompletion script for the specified shell
discover discover hosts with CIDR range. Run snmp2cpe on active host to get CPE. Default outputFile is ./discover_list.toml
help Help about any command
upload Upload to FutureVuls
version Show version
Flags:
-h, --help help for future-vuls
Use "future-vuls [command] --help" for more information about a command.Subcommands
./future-vuls upload -h
Upload to FutureVuls
Usage:
future-vuls upload [flags]
Flags:
--config string config file (default is $HOME/.cobra.yaml)
-g, --group-id int future vuls group id, ENV: VULS_GROUP_ID
-h, --help help for upload
-s, --stdin input from stdin. ENV: VULS_STDIN
-t, --token string future vuls token
--url string future vuls upload url
--uuid string server uuid. ENV: VULS_SERVER_UUID./future-vuls discover -h
discover hosts with CIDR range. Run snmp2cpe on active host to get CPE. Default outputFile is ./discover_list.toml
Usage:
future-vuls discover --cidr <CIDR_RANGE> --output <OUTPUT_FILE> [flags]
Examples:
future-vuls discover --cidr 192.168.0.0/24 --output discover_list.toml
Flags:
--cidr string cidr range
--community string snmp community name. default: public
-h, --help help for discover
--output string output file
--snmp-version string snmp version v1,v2c and v3. default: v2c./future-vuls add-cpe -h
Create a pseudo server in Fvuls and register CPE. Default outputFile is ./discover_list.toml
Usage:
future-vuls add-cpe --token <VULS_TOKEN> --output <OUTPUT_FILE> [flags]
Examples:
future-vuls add-cpe --token <VULS_TOKEN>
Flags:
-h, --help help for add-cpe
--http-proxy string proxy url
--output string output file
-t, --token string future vuls token ENV: VULS_TOKENUsage
future-vuls upload
cat results.json | future-vuls upload --stdin --token xxxx --url https://xxxx --group-id 1 --uuid xxxxfuture-vuls discover
./future-vuls discover --cidr 192.168.0.1/24
Discovering 192.168.0.1/24...
192.168.0.1: Execute snmp2cpe...
failed to execute snmp2cpe. err: failed to execute snmp2cpe. err: exit status 1
192.168.0.2: Execute snmp2cpe...
failed to execute snmp2cpe. err: failed to execute snmp2cpe. err: exit status 1
192.168.0.4: Execute snmp2cpe...
failed to execute snmp2cpe. err: failed to execute snmp2cpe. err: exit status 1
192.168.0.5: Execute snmp2cpe...
failed to execute snmp2cpe. err: failed to execute snmp2cpe. err: exit status 1
192.168.0.6: Execute snmp2cpe...
New network device found 192.168.0.6
wrote to discover_list.tomlfuture-vuls add-cpe
./future-vuls add-cpe --token fvgr-686b92af-5216-11ee-a241-0a58a9feac02
Creating 1 pseudo server...
192.168.0.6: Created FutureVuls pseudo server ce024b45-1c59-5b86-1a67-e78a40dfec01
wrote to discover_list.toml
Uploading 1 server's CPE...
192.168.0.6: Uploaded CPE cpe:2.3:h:fortinet:fortigate-50e:-:*:*:*:*:*:*:*
192.168.0.6: Uploaded CPE cpe:2.3:o:fortinet:fortios:5.4.6:*:*:*:*:*:*:*File: contrib/snmp2cpe/README.md
snmp2cpe
Main Features
- Estimate hardware and OS CPE from SNMP reply of network devices
Installation
$ git clone https://github.com/future-architect/vuls.git
$ make build-snmp2cpeCommand Reference
$ snmp2cpe help
snmp2cpe: SNMP reply To CPE
Usage:
snmp2cpe [command]
Available Commands:
completion Generate the autocompletion script for the specified shell
convert snmpget reply to CPE
help Help about any command
v1 snmpget with SNMPv1
v2c snmpget with SNMPv2c
v3 snmpget with SNMPv3
version Print the version
Flags:
-h, --help help for snmp2cpe
Use "snmp2cpe [command] --help" for more information about a command.Usage
$ snmp2cpe v2c --debug 192.168.1.99 public
2023/03/28 14:16:54 DEBUG: .1.3.6.1.2.1.1.1.0 ->
2023/03/28 14:16:54 DEBUG: .1.3.6.1.2.1.47.1.1.1.1.12.1 -> Fortinet
2023/03/28 14:16:54 DEBUG: .1.3.6.1.2.1.47.1.1.1.1.7.1 -> FGT_50E
2023/03/28 14:16:54 DEBUG: .1.3.6.1.2.1.47.1.1.1.1.10.1 -> FortiGate-50E v5.4.6,build1165b1165,171018 (GA)
{"192.168.1.99":{"entPhysicalTables":{"1":{"entPhysicalMfgName":"Fortinet","entPhysicalName":"FGT_50E","entPhysicalSoftwareRev":"FortiGate-50E v5.4.6,build1165b1165,171018 (GA)"}}}}
$ snmp2cpe v2c 192.168.1.99 public | snmp2cpe convert
{"192.168.1.99":["cpe:2.3:h:fortinet:fortigate-50e:-:*:*:*:*:*:*:*","cpe:2.3:o:fortinet:fortios:5.4.6:*:*:*:*:*:*:*"]}File: contrib/trivy/README.md
trivy-to-vuls
Main Features
- convert trivy's results json to vuls's report json
Installation
git clone https://github.com/future-architect/vuls.git
make build-trivy-to-vulsCommand Reference
Parse trivy json to vuls results
Usage:
trivy-to-vuls parse [flags]
Flags:
-h, --help help for parse
-s, --stdin input from stdin
-d, --trivy-json-dir string trivy json dir (default "./")
-f, --trivy-json-file-name string trivy json file name (default "results.json")Usage
- use trivy output
trivy -q image -f=json python:3.4-alpine | trivy-to-vuls parse --stdinFile: setup/docker/README.md
See Vulsdoc
2. Official Technical Reference & Guides (future-architect/future-architect.github.io)
service stack
- ホスティング : Github Pages
- 静的サイトジェネレータ:Hexo
- 分析:Google Analytics
ユーザプロフィールを追加する方法
- _profile.yml に追加
setup
SNSカウントを表示させるために、Facebook開発者用のトークンが必要
- Facebook開発者キーが必要
- アクセストークンを取得
- https://graph.facebook.com/oauth/access_token?client_id=xxx&client_secret=xxx&grant_type=client_credentials
- 次のようなJSONが取得できるので、このaccess tokenの値を取得
{
"access_token": "xxx|xxx",
"token_type": "bearer"
}
- アクセストークンが正常か確認
https://graph.facebook.com?id=https%3A%2F%2Ffuture-architect.github.io%2Farticles%2F20210313%2F&fields=engagement&access_token=xxx|xxx - 環境変数に設定
- export FB_TOKEN=xxx|xxx
Google Analytics
人気の記事を表示させるためGOOGLE_APPLICATION_CREDENTIALSが必要
set GOOGLE_APPLICATION_CREDENTIALS キーパス
View local server
$ cd ../
$ git clone --depth 1 https://github.com/future-architect/future-architect.github.io
$ cd future-architect.github.io
$ npm install
$ node_modules\.bin\hexo server
--> http://localhost:4000 にページが表示されたら完了Deploy
main ブランチにマージトリガーでGitHub Actionsによりデプロイされる。
SNSカウントの更新
GitHub Actions経由で実行されるため、通常開発者によるSNSカウントの更新は不要。
# インストール
go install github.com/ma91n/snssharecount/cmd/snssharecount@latest
go install github.com/ma91n/snssharecount/cmd/ga@latest
go install github.com/ma91n/snssharecount/cmd/pv@latest実行
# 実行
set http_proxy=
set https_proxy=
snssharecount > temp.json
mv temp.json sns_count_cache.json
ga > ga_cache.json
pv > ga4_pv.jsonMermaid 図のSVGキャッシュ
記事中の ```mermaid コードフェンスは、ビルド時に source/_mermaid/<ソースのSHA-256>.svg の
キャッシュがあればインラインSVGとして展開される(ブラウザ側の mermaid.js 読み込みが不要になる)。
キャッシュが無い図は従来どおりブラウザでレンダリングされるため、SVG未生成でも記事は壊れない。
図を追加・編集したらローカルでキャッシュを更新してコミットする。Docker が必要
(レンダラはセルフホストの kroki。未起動なら自動で docker compose 起動する)。
make mermaid
# 停止する場合
docker compose -f mermaid_svg.compose.yml down画像圧縮
pngquantをインストールする
# ディレクトリ横断
pngquant */*.png --skip-if-larger --ext .png --force
# 特定ディレクトリ配下
pngquant *.png --skip-if-larger --ext .png --forcejpegoptimをインストールする
https://github.com/XhmikosR/jpegoptim-windows/releases/
# jpegoptimに再帰的オプションがなかったためbatファイルでラップした
# 月次で実行する
jpegoptimall.battextlint
Installation
> npm install textlint --global
> npm install textlint-rule-preset-ja-technical-writing --global
> npm install textlint-rule-spellcheck-tech-word --globalLint
> textlint source/_postsLint with lint-staged(staged files only)
git add source/_posts
npx lint-staged