{"owner":"lm-sys","repo":"FastChat","hasSkills":true,"totalSkillsCount":6,"totalTokensCount":2693,"categories":["command-prompt"],"hasMcp":false,"mcpConfig":null,"found":["docs/commands/leaderboard.md","docs/commands/data_cleaning.md","docs/commands/local_cluster.md","docs/commands/conv_release.md","docs/commands/webserver.md","docs/commands/pypi.md"],"skills":{"docs/commands/leaderboard.md":"### Get logs\n```\ngsutil -m rsync -r gs://fastchat_logs ~/fastchat_logs/\n```\n\n### Clean battle data\n```\ncd ~/FastChat/fastchat/serve/monitor\npython3 clean_battle_data.py\n```\n\n### Run Elo analysis\n```\npython3 elo_analysis.py --clean-battle-file clean_battle_20230905.json\n```\n\n### Copy files to HF space\n1. update plots\n```\nscp atlas:/data/lmzheng/FastChat/fastchat/serve/monitor/elo_results_20230905.pkl .\n```\n\n2. update table\n```\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/raw/main/leaderboard_table_20230905.csv\n```\n\n### Update files on webserver\n```\nDATE=20231002\n\nrm -rf elo_results.pkl leaderboard_table.csv\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/resolve/main/elo_results_$DATE.pkl\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/resolve/main/leaderboard_table_$DATE.csv\nln -s leaderboard_table_$DATE.csv leaderboard_table.csv\nln -s elo_results_$DATE.pkl elo_results.pkl\n```\n","docs/commands/data_cleaning.md":"## Data cleaning\n\n## Requirements\n```\npip3 install bs4 markdownify\npip3 install polyglot pyicu pycld2\n```\n\n## Steps\n```\n# Convert html to markdown\npython3 -m fastchat.data.clean_sharegpt --in sharegpt_html.json --out sharegpt_clean.json\n\n# Keep or remove specific languages\npython3 -m fastchat.data.optional_clean --in sharegpt_clean.json --out sharegpt_clean_lang.json --skip-lang SOME_LANGUAGE_CODE\n\n# Split long conversations\npython3 -m fastchat.data.split_long_conversation --in sharegpt_clean_lang.json --out sharegpt_clean_lang_split.json --model-name /home/ubuntu/model_weights/llama-7b/\n```\n","docs/commands/local_cluster.md":"### Local GPU cluster\nnode-01\n```\npython3 -m fastchat.serve.controller --host 0.0.0.0 --port 10002\n\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-13b-v1.5 --model-name vicuna-13b --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-13b-v1.5 --model-name vicuna-13b --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker-address http://$(hostname):31001\n\nCUDA_VISIBLE_DEVICES=2,3 ray start --head\npython3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-33b-v1.3 --model-name vicuna-33b --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --num-gpus 2\n```\n\nnode-02\n```\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-13b-chat-hf --model-name llama-2-13b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-13b-chat-hf --model-name llama-2-13b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker-address http://$(hostname):31001 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=2 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-7b-chat-hf --model-name llama-2-7b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=3 python3 -m fastchat.serve.vllm_worker --model-path WizardLM/WizardLM-13B-V1.1 --model-name wizardlm-13b  --controller http://node-01:10002 --host 0.0.0.0 --port 31003 --worker-address http://$(hostname):31003\n```\n\nnode-03\n```\npython3 -m fastchat.serve.vllm_worker --model-path mosaicml/mpt-30b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000 --num-gpus 2\npython3 -m fastchat.serve.vllm_worker --model-path timdettmers/guanaco-33b-merged --model-name guanaco-33b  --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --num-gpus 2 --tokenizer hf-internal-testing/llama-tokenizer\n```\n\nnode-04\n```\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.multi_model_worker --model-path ~/model_weights/RWKV-4-Raven-14B-v12-Eng98%25-Other2%25-20230523-ctx8192.pth --model-name RWKV-4-Raven-14B --model-path lmsys/fastchat-t5-3b-v1.0 --model-name fastchat-t5-3b --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker http://$(hostname):31000 --limit 4\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.multi_model_worker --model-path OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5 --model-name oasst-pythia-12b --model-path mosaicml/mpt-7b-chat --model-name mpt-7b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker http://$(hostname):31001 --limit 4\nCUDA_VISIBLE_DEVICES=2 python3 -m fastchat.serve.multi_model_worker --model-path lmsys/vicuna-7b-v1.5 --model-name vicuna-7b --model-path THUDM/chatglm-6b --model-name chatglm-6b --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker http://$(hostname):31002 --limit 4\nCUDA_VISIBLE_DEVICES=3 python3 -m fastchat.serve.vllm_worker --model-path ~/model_weights/alpaca-13b  --controller http://node-01:10002 --host 0.0.0.0 --port 31003 --worker-address http://$(hostname):31003\n```\n\ntest\n```\npython3 -m fastchat.serve.test_message --model vicuna-13b --controller http://localhost:10002\n```\n","docs/commands/conv_release.md":"## Chatbot Arena Conversations\n\n1. Gather battles\n```\npython3 clean_battle_data.py --max-num 10 --mode conv_release\n```\n\n2. Tag OpenAI moderation\n```\npython3 tag_openai_moderation.py --in clean_battle_conv_20230814.json\n```\n\n3. Clean PII\n\n4. Filter additional blocked words\n\n```\npython3 filter_bad_conv.py --in clean_battle_conv_20230630_tagged_v1_pii.json\n```\n\n5. Add additional toxicity tag\n\n\n## All Conversations\n\n1. Gather chats\n```\npython3 clean_chat_data.py\n```\n\n2. Sample\n```\npython3 conv_release_scripts/sample.py\n```\n\n\n## Prompt distribution\n\n","docs/commands/webserver.md":"### Install\n```\nsudo apt update\nsudo apt install tmux htop\n\nwget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh\nbash Anaconda3-2022.10-Linux-x86_64.sh\n\nconda create -n fastchat python=3.9\nconda activate fastchat\n\ngit clone https://github.com/lm-sys/FastChat.git\ncd FastChat\npip3 install -e .\n```\n\n\n### Launch servers\n```\ncd fastchat_logs/controller\npython3 -m fastchat.serve.controller --host 0.0.0.0 --port 21001\npython3 -m fastchat.serve.register_worker --controller http://localhost:21001 --worker-name https://\npython3 -m fastchat.serve.test_message --model vicuna-13b --controller http://localhost:21001\n\ncd fastchat_logs/server0\n\npython3 -m fastchat.serve.huggingface_api_worker --model-info-file ~/elo_results/register_hf_api_models.json\n\nexport OPENAI_API_KEY=\nexport ANTHROPIC_API_KEY=\nexport GCP_PROJECT_ID=\n\npython3 -m fastchat.serve.gradio_web_server_multi --controller http://localhost:21001 --concurrency 50 --add-chatgpt --add-claude --add-palm --elo ~/elo_results/elo_results.pkl --leaderboard-table-file ~/elo_results/leaderboard_table.csv --register ~/elo_results/register_oai_models.json --show-terms\n\npython3 backup_logs.py\n```\n\n\n### Check the launch time\n```\nfor i in $(seq 0 11); do cat fastchat_logs/server$i/gradio_web_server.log | grep \"Running on local URL\" | tail -n 1; done\n```\n\n\n### Increase the limit of max open files\nOne process (do not need reboot)\n```\nsudo prlimit --nofile=1048576:1048576 --pid=$id\n\nfor id in $(ps -ef | grep gradio_web_server | awk '{print $2}'); do echo $id; prlimit --nofile=1048576:1048576 --pid=$id; done\n```\n\nSystem (need reboot): Add the lines below to `/etc/security/limits.conf`\n```\n* hard nofile 65535\n* soft nofile 65535\n```\n\n\n### Gradio edit  (3.35.2)\n1. gtag and canvas\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/templates/frontend/index.html\n```\n\n```\n<!-- Google tag (gtag.js) -->\n<script async src=\"https://www.googletagmanager.com/gtag/js?id=G-K6D24EE9ED\"></script><script>\n  window.dataLayer = window.dataLayer || [];\n  function gtag(){dataLayer.push(arguments);}\n  gtag('js', new Date());\n  gtag('config', 'G-K6D24EE9ED');\n  window.__gradio_mode__ = \"app\";\n</script>\n<script src=\"https://cdnjs.cloudflare.com/ajax/libs/html2canvas/1.4.1/html2canvas.min.js\"></script>\n```\n\n2. deprecation warnings\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/deprecation.py\n```\n\n```\ndef check_deprecated_parameters(\n```\n\n3. Loading\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/templates/frontend/assets/index-188ef5e8.js\n```\n\n```\n%s/\"Loading...\"/\"Loading...(Please refresh if it takes more than 30 seconds)\"/g\n```\n","docs/commands/pypi.md":"### Requirement\n```\npython3 -m pip install twine\npython3 -m pip install --upgrade pip\npip3 install build\n```\n\n### Upload\n```\nbash scripts/upload_pypi.sh\n```\n"},"files":{"docs/commands/leaderboard.md":"### Get logs\n```\ngsutil -m rsync -r gs://fastchat_logs ~/fastchat_logs/\n```\n\n### Clean battle data\n```\ncd ~/FastChat/fastchat/serve/monitor\npython3 clean_battle_data.py\n```\n\n### Run Elo analysis\n```\npython3 elo_analysis.py --clean-battle-file clean_battle_20230905.json\n```\n\n### Copy files to HF space\n1. update plots\n```\nscp atlas:/data/lmzheng/FastChat/fastchat/serve/monitor/elo_results_20230905.pkl .\n```\n\n2. update table\n```\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/raw/main/leaderboard_table_20230905.csv\n```\n\n### Update files on webserver\n```\nDATE=20231002\n\nrm -rf elo_results.pkl leaderboard_table.csv\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/resolve/main/elo_results_$DATE.pkl\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/resolve/main/leaderboard_table_$DATE.csv\nln -s leaderboard_table_$DATE.csv leaderboard_table.csv\nln -s elo_results_$DATE.pkl elo_results.pkl\n```\n","docs/commands/data_cleaning.md":"## Data cleaning\n\n## Requirements\n```\npip3 install bs4 markdownify\npip3 install polyglot pyicu pycld2\n```\n\n## Steps\n```\n# Convert html to markdown\npython3 -m fastchat.data.clean_sharegpt --in sharegpt_html.json --out sharegpt_clean.json\n\n# Keep or remove specific languages\npython3 -m fastchat.data.optional_clean --in sharegpt_clean.json --out sharegpt_clean_lang.json --skip-lang SOME_LANGUAGE_CODE\n\n# Split long conversations\npython3 -m fastchat.data.split_long_conversation --in sharegpt_clean_lang.json --out sharegpt_clean_lang_split.json --model-name /home/ubuntu/model_weights/llama-7b/\n```\n","docs/commands/local_cluster.md":"### Local GPU cluster\nnode-01\n```\npython3 -m fastchat.serve.controller --host 0.0.0.0 --port 10002\n\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-13b-v1.5 --model-name vicuna-13b --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-13b-v1.5 --model-name vicuna-13b --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker-address http://$(hostname):31001\n\nCUDA_VISIBLE_DEVICES=2,3 ray start --head\npython3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-33b-v1.3 --model-name vicuna-33b --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --num-gpus 2\n```\n\nnode-02\n```\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-13b-chat-hf --model-name llama-2-13b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-13b-chat-hf --model-name llama-2-13b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker-address http://$(hostname):31001 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=2 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-7b-chat-hf --model-name llama-2-7b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=3 python3 -m fastchat.serve.vllm_worker --model-path WizardLM/WizardLM-13B-V1.1 --model-name wizardlm-13b  --controller http://node-01:10002 --host 0.0.0.0 --port 31003 --worker-address http://$(hostname):31003\n```\n\nnode-03\n```\npython3 -m fastchat.serve.vllm_worker --model-path mosaicml/mpt-30b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000 --num-gpus 2\npython3 -m fastchat.serve.vllm_worker --model-path timdettmers/guanaco-33b-merged --model-name guanaco-33b  --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --num-gpus 2 --tokenizer hf-internal-testing/llama-tokenizer\n```\n\nnode-04\n```\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.multi_model_worker --model-path ~/model_weights/RWKV-4-Raven-14B-v12-Eng98%25-Other2%25-20230523-ctx8192.pth --model-name RWKV-4-Raven-14B --model-path lmsys/fastchat-t5-3b-v1.0 --model-name fastchat-t5-3b --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker http://$(hostname):31000 --limit 4\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.multi_model_worker --model-path OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5 --model-name oasst-pythia-12b --model-path mosaicml/mpt-7b-chat --model-name mpt-7b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker http://$(hostname):31001 --limit 4\nCUDA_VISIBLE_DEVICES=2 python3 -m fastchat.serve.multi_model_worker --model-path lmsys/vicuna-7b-v1.5 --model-name vicuna-7b --model-path THUDM/chatglm-6b --model-name chatglm-6b --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker http://$(hostname):31002 --limit 4\nCUDA_VISIBLE_DEVICES=3 python3 -m fastchat.serve.vllm_worker --model-path ~/model_weights/alpaca-13b  --controller http://node-01:10002 --host 0.0.0.0 --port 31003 --worker-address http://$(hostname):31003\n```\n\ntest\n```\npython3 -m fastchat.serve.test_message --model vicuna-13b --controller http://localhost:10002\n```\n","docs/commands/conv_release.md":"## Chatbot Arena Conversations\n\n1. Gather battles\n```\npython3 clean_battle_data.py --max-num 10 --mode conv_release\n```\n\n2. Tag OpenAI moderation\n```\npython3 tag_openai_moderation.py --in clean_battle_conv_20230814.json\n```\n\n3. Clean PII\n\n4. Filter additional blocked words\n\n```\npython3 filter_bad_conv.py --in clean_battle_conv_20230630_tagged_v1_pii.json\n```\n\n5. Add additional toxicity tag\n\n\n## All Conversations\n\n1. Gather chats\n```\npython3 clean_chat_data.py\n```\n\n2. Sample\n```\npython3 conv_release_scripts/sample.py\n```\n\n\n## Prompt distribution\n\n","docs/commands/webserver.md":"### Install\n```\nsudo apt update\nsudo apt install tmux htop\n\nwget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh\nbash Anaconda3-2022.10-Linux-x86_64.sh\n\nconda create -n fastchat python=3.9\nconda activate fastchat\n\ngit clone https://github.com/lm-sys/FastChat.git\ncd FastChat\npip3 install -e .\n```\n\n\n### Launch servers\n```\ncd fastchat_logs/controller\npython3 -m fastchat.serve.controller --host 0.0.0.0 --port 21001\npython3 -m fastchat.serve.register_worker --controller http://localhost:21001 --worker-name https://\npython3 -m fastchat.serve.test_message --model vicuna-13b --controller http://localhost:21001\n\ncd fastchat_logs/server0\n\npython3 -m fastchat.serve.huggingface_api_worker --model-info-file ~/elo_results/register_hf_api_models.json\n\nexport OPENAI_API_KEY=\nexport ANTHROPIC_API_KEY=\nexport GCP_PROJECT_ID=\n\npython3 -m fastchat.serve.gradio_web_server_multi --controller http://localhost:21001 --concurrency 50 --add-chatgpt --add-claude --add-palm --elo ~/elo_results/elo_results.pkl --leaderboard-table-file ~/elo_results/leaderboard_table.csv --register ~/elo_results/register_oai_models.json --show-terms\n\npython3 backup_logs.py\n```\n\n\n### Check the launch time\n```\nfor i in $(seq 0 11); do cat fastchat_logs/server$i/gradio_web_server.log | grep \"Running on local URL\" | tail -n 1; done\n```\n\n\n### Increase the limit of max open files\nOne process (do not need reboot)\n```\nsudo prlimit --nofile=1048576:1048576 --pid=$id\n\nfor id in $(ps -ef | grep gradio_web_server | awk '{print $2}'); do echo $id; prlimit --nofile=1048576:1048576 --pid=$id; done\n```\n\nSystem (need reboot): Add the lines below to `/etc/security/limits.conf`\n```\n* hard nofile 65535\n* soft nofile 65535\n```\n\n\n### Gradio edit  (3.35.2)\n1. gtag and canvas\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/templates/frontend/index.html\n```\n\n```\n<!-- Google tag (gtag.js) -->\n<script async src=\"https://www.googletagmanager.com/gtag/js?id=G-K6D24EE9ED\"></script><script>\n  window.dataLayer = window.dataLayer || [];\n  function gtag(){dataLayer.push(arguments);}\n  gtag('js', new Date());\n  gtag('config', 'G-K6D24EE9ED');\n  window.__gradio_mode__ = \"app\";\n</script>\n<script src=\"https://cdnjs.cloudflare.com/ajax/libs/html2canvas/1.4.1/html2canvas.min.js\"></script>\n```\n\n2. deprecation warnings\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/deprecation.py\n```\n\n```\ndef check_deprecated_parameters(\n```\n\n3. Loading\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/templates/frontend/assets/index-188ef5e8.js\n```\n\n```\n%s/\"Loading...\"/\"Loading...(Please refresh if it takes more than 30 seconds)\"/g\n```\n","docs/commands/pypi.md":"### Requirement\n```\npython3 -m pip install twine\npython3 -m pip install --upgrade pip\npip3 install build\n```\n\n### Upload\n```\nbash scripts/upload_pypi.sh\n```\n"},"items":[{"name":"conv_release.md","path":"docs/commands/conv_release.md","rawUrl":"https://raw.githubusercontent.com/lm-sys/FastChat/HEAD/docs/commands/conv_release.md","title":"Command: /conv_release","category":"command-prompt","format":"markdown","content":"## Chatbot Arena Conversations\n\n1. Gather battles\n```\npython3 clean_battle_data.py --max-num 10 --mode conv_release\n```\n\n2. Tag OpenAI moderation\n```\npython3 tag_openai_moderation.py --in clean_battle_conv_20230814.json\n```\n\n3. Clean PII\n\n4. Filter additional blocked words\n\n```\npython3 filter_bad_conv.py --in clean_battle_conv_20230630_tagged_v1_pii.json\n```\n\n5. Add additional toxicity tag\n\n\n## All Conversations\n\n1. Gather chats\n```\npython3 clean_chat_data.py\n```\n\n2. Sample\n```\npython3 conv_release_scripts/sample.py\n```\n\n\n## Prompt distribution\n\n","isInternal":false,"tokens":154,"sizeBytes":552},{"name":"data_cleaning.md","path":"docs/commands/data_cleaning.md","rawUrl":"https://raw.githubusercontent.com/lm-sys/FastChat/HEAD/docs/commands/data_cleaning.md","title":"Command: /data_cleaning","category":"command-prompt","format":"markdown","content":"## Data cleaning\n\n## Requirements\n```\npip3 install bs4 markdownify\npip3 install polyglot pyicu pycld2\n```\n\n## Steps\n```\n# Convert html to markdown\npython3 -m fastchat.data.clean_sharegpt --in sharegpt_html.json --out sharegpt_clean.json\n\n# Keep or remove specific languages\npython3 -m fastchat.data.optional_clean --in sharegpt_clean.json --out sharegpt_clean_lang.json --skip-lang SOME_LANGUAGE_CODE\n\n# Split long conversations\npython3 -m fastchat.data.split_long_conversation --in sharegpt_clean_lang.json --out sharegpt_clean_lang_split.json --model-name /home/ubuntu/model_weights/llama-7b/\n```\n","isInternal":false,"tokens":158,"sizeBytes":599},{"name":"leaderboard.md","path":"docs/commands/leaderboard.md","rawUrl":"https://raw.githubusercontent.com/lm-sys/FastChat/HEAD/docs/commands/leaderboard.md","title":"Command: /leaderboard","category":"command-prompt","format":"markdown","content":"### Get logs\n```\ngsutil -m rsync -r gs://fastchat_logs ~/fastchat_logs/\n```\n\n### Clean battle data\n```\ncd ~/FastChat/fastchat/serve/monitor\npython3 clean_battle_data.py\n```\n\n### Run Elo analysis\n```\npython3 elo_analysis.py --clean-battle-file clean_battle_20230905.json\n```\n\n### Copy files to HF space\n1. update plots\n```\nscp atlas:/data/lmzheng/FastChat/fastchat/serve/monitor/elo_results_20230905.pkl .\n```\n\n2. update table\n```\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/raw/main/leaderboard_table_20230905.csv\n```\n\n### Update files on webserver\n```\nDATE=20231002\n\nrm -rf elo_results.pkl leaderboard_table.csv\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/resolve/main/elo_results_$DATE.pkl\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/resolve/main/leaderboard_table_$DATE.csv\nln -s leaderboard_table_$DATE.csv leaderboard_table.csv\nln -s elo_results_$DATE.pkl elo_results.pkl\n```\n","isInternal":false,"tokens":274,"sizeBytes":950},{"name":"local_cluster.md","path":"docs/commands/local_cluster.md","rawUrl":"https://raw.githubusercontent.com/lm-sys/FastChat/HEAD/docs/commands/local_cluster.md","title":"Command: /local_cluster","category":"command-prompt","format":"markdown","content":"### Local GPU cluster\nnode-01\n```\npython3 -m fastchat.serve.controller --host 0.0.0.0 --port 10002\n\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-13b-v1.5 --model-name vicuna-13b --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-13b-v1.5 --model-name vicuna-13b --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker-address http://$(hostname):31001\n\nCUDA_VISIBLE_DEVICES=2,3 ray start --head\npython3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-33b-v1.3 --model-name vicuna-33b --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --num-gpus 2\n```\n\nnode-02\n```\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-13b-chat-hf --model-name llama-2-13b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-13b-chat-hf --model-name llama-2-13b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker-address http://$(hostname):31001 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=2 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-7b-chat-hf --model-name llama-2-7b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=3 python3 -m fastchat.serve.vllm_worker --model-path WizardLM/WizardLM-13B-V1.1 --model-name wizardlm-13b  --controller http://node-01:10002 --host 0.0.0.0 --port 31003 --worker-address http://$(hostname):31003\n```\n\nnode-03\n```\npython3 -m fastchat.serve.vllm_worker --model-path mosaicml/mpt-30b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000 --num-gpus 2\npython3 -m fastchat.serve.vllm_worker --model-path timdettmers/guanaco-33b-merged --model-name guanaco-33b  --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --num-gpus 2 --tokenizer hf-internal-testing/llama-tokenizer\n```\n\nnode-04\n```\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.multi_model_worker --model-path ~/model_weights/RWKV-4-Raven-14B-v12-Eng98%25-Other2%25-20230523-ctx8192.pth --model-name RWKV-4-Raven-14B --model-path lmsys/fastchat-t5-3b-v1.0 --model-name fastchat-t5-3b --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker http://$(hostname):31000 --limit 4\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.multi_model_worker --model-path OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5 --model-name oasst-pythia-12b --model-path mosaicml/mpt-7b-chat --model-name mpt-7b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker http://$(hostname):31001 --limit 4\nCUDA_VISIBLE_DEVICES=2 python3 -m fastchat.serve.multi_model_worker --model-path lmsys/vicuna-7b-v1.5 --model-name vicuna-7b --model-path THUDM/chatglm-6b --model-name chatglm-6b --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker http://$(hostname):31002 --limit 4\nCUDA_VISIBLE_DEVICES=3 python3 -m fastchat.serve.vllm_worker --model-path ~/model_weights/alpaca-13b  --controller http://node-01:10002 --host 0.0.0.0 --port 31003 --worker-address http://$(hostname):31003\n```\n\ntest\n```\npython3 -m fastchat.serve.test_message --model vicuna-13b --controller http://localhost:10002\n```\n","isInternal":false,"tokens":1264,"sizeBytes":3679},{"name":"pypi.md","path":"docs/commands/pypi.md","rawUrl":"https://raw.githubusercontent.com/lm-sys/FastChat/HEAD/docs/commands/pypi.md","title":"Command: /pypi","category":"command-prompt","format":"markdown","content":"### Requirement\n```\npython3 -m pip install twine\npython3 -m pip install --upgrade pip\npip3 install build\n```\n\n### Upload\n```\nbash scripts/upload_pypi.sh\n```\n","isInternal":false,"tokens":45,"sizeBytes":157},{"name":"webserver.md","path":"docs/commands/webserver.md","rawUrl":"https://raw.githubusercontent.com/lm-sys/FastChat/HEAD/docs/commands/webserver.md","title":"Command: /webserver","category":"command-prompt","format":"markdown","content":"### Install\n```\nsudo apt update\nsudo apt install tmux htop\n\nwget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh\nbash Anaconda3-2022.10-Linux-x86_64.sh\n\nconda create -n fastchat python=3.9\nconda activate fastchat\n\ngit clone https://github.com/lm-sys/FastChat.git\ncd FastChat\npip3 install -e .\n```\n\n\n### Launch servers\n```\ncd fastchat_logs/controller\npython3 -m fastchat.serve.controller --host 0.0.0.0 --port 21001\npython3 -m fastchat.serve.register_worker --controller http://localhost:21001 --worker-name https://\npython3 -m fastchat.serve.test_message --model vicuna-13b --controller http://localhost:21001\n\ncd fastchat_logs/server0\n\npython3 -m fastchat.serve.huggingface_api_worker --model-info-file ~/elo_results/register_hf_api_models.json\n\nexport OPENAI_API_KEY=\nexport ANTHROPIC_API_KEY=\nexport GCP_PROJECT_ID=\n\npython3 -m fastchat.serve.gradio_web_server_multi --controller http://localhost:21001 --concurrency 50 --add-chatgpt --add-claude --add-palm --elo ~/elo_results/elo_results.pkl --leaderboard-table-file ~/elo_results/leaderboard_table.csv --register ~/elo_results/register_oai_models.json --show-terms\n\npython3 backup_logs.py\n```\n\n\n### Check the launch time\n```\nfor i in $(seq 0 11); do cat fastchat_logs/server$i/gradio_web_server.log | grep \"Running on local URL\" | tail -n 1; done\n```\n\n\n### Increase the limit of max open files\nOne process (do not need reboot)\n```\nsudo prlimit --nofile=1048576:1048576 --pid=$id\n\nfor id in $(ps -ef | grep gradio_web_server | awk '{print $2}'); do echo $id; prlimit --nofile=1048576:1048576 --pid=$id; done\n```\n\nSystem (need reboot): Add the lines below to `/etc/security/limits.conf`\n```\n* hard nofile 65535\n* soft nofile 65535\n```\n\n\n### Gradio edit  (3.35.2)\n1. gtag and canvas\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/templates/frontend/index.html\n```\n\n```\n<!-- Google tag (gtag.js) -->\n<script async src=\"https://www.googletagmanager.com/gtag/js?id=G-K6D24EE9ED\"></script><script>\n  window.dataLayer = window.dataLayer || [];\n  function gtag(){dataLayer.push(arguments);}\n  gtag('js', new Date());\n  gtag('config', 'G-K6D24EE9ED');\n  window.__gradio_mode__ = \"app\";\n</script>\n<script src=\"https://cdnjs.cloudflare.com/ajax/libs/html2canvas/1.4.1/html2canvas.min.js\"></script>\n```\n\n2. deprecation warnings\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/deprecation.py\n```\n\n```\ndef check_deprecated_parameters(\n```\n\n3. Loading\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/templates/frontend/assets/index-188ef5e8.js\n```\n\n```\n%s/\"Loading...\"/\"Loading...(Please refresh if it takes more than 30 seconds)\"/g\n```\n","isInternal":false,"tokens":798,"sizeBytes":2694}],"systemPromptSnippet":"<yakaai_skills repo=\"lm-sys/FastChat\">\n<!-- File: docs/commands/conv_release.md (Tokens: ~154 | Category: command-prompt) -->\n## Chatbot Arena Conversations\n\n1. Gather battles\n```\npython3 clean_battle_data.py --max-num 10 --mode conv_release\n```\n\n2. Tag OpenAI moderation\n```\npython3 tag_openai_moderation.py --in clean_battle_conv_20230814.json\n```\n\n3. Clean PII\n\n4. Filter additional blocked words\n\n```\npython3 filter_bad_conv.py --in clean_battle_conv_20230630_tagged_v1_pii.json\n```\n\n5. Add additional toxicity tag\n\n\n## All Conversations\n\n1. Gather chats\n```\npython3 clean_chat_data.py\n```\n\n2. Sample\n```\npython3 conv_release_scripts/sample.py\n```\n\n\n## Prompt distribution\n\n<!-- File: docs/commands/data_cleaning.md (Tokens: ~158 | Category: command-prompt) -->\n## Data cleaning\n\n## Requirements\n```\npip3 install bs4 markdownify\npip3 install polyglot pyicu pycld2\n```\n\n## Steps\n```\n# Convert html to markdown\npython3 -m fastchat.data.clean_sharegpt --in sharegpt_html.json --out sharegpt_clean.json\n\n# Keep or remove specific languages\npython3 -m fastchat.data.optional_clean --in sharegpt_clean.json --out sharegpt_clean_lang.json --skip-lang SOME_LANGUAGE_CODE\n\n# Split long conversations\npython3 -m fastchat.data.split_long_conversation --in sharegpt_clean_lang.json --out sharegpt_clean_lang_split.json --model-name /home/ubuntu/model_weights/llama-7b/\n```\n\n<!-- File: docs/commands/leaderboard.md (Tokens: ~274 | Category: command-prompt) -->\n### Get logs\n```\ngsutil -m rsync -r gs://fastchat_logs ~/fastchat_logs/\n```\n\n### Clean battle data\n```\ncd ~/FastChat/fastchat/serve/monitor\npython3 clean_battle_data.py\n```\n\n### Run Elo analysis\n```\npython3 elo_analysis.py --clean-battle-file clean_battle_20230905.json\n```\n\n### Copy files to HF space\n1. update plots\n```\nscp atlas:/data/lmzheng/FastChat/fastchat/serve/monitor/elo_results_20230905.pkl .\n```\n\n2. update table\n```\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/raw/main/leaderboard_table_20230905.csv\n```\n\n### Update files on webserver\n```\nDATE=20231002\n\nrm -rf elo_results.pkl leaderboard_table.csv\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/resolve/main/elo_results_$DATE.pkl\nwget https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard/resolve/main/leaderboard_table_$DATE.csv\nln -s leaderboard_table_$DATE.csv leaderboard_table.csv\nln -s elo_results_$DATE.pkl elo_results.pkl\n```\n\n<!-- File: docs/commands/local_cluster.md (Tokens: ~1264 | Category: command-prompt) -->\n### Local GPU cluster\nnode-01\n```\npython3 -m fastchat.serve.controller --host 0.0.0.0 --port 10002\n\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-13b-v1.5 --model-name vicuna-13b --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-13b-v1.5 --model-name vicuna-13b --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker-address http://$(hostname):31001\n\nCUDA_VISIBLE_DEVICES=2,3 ray start --head\npython3 -m fastchat.serve.vllm_worker --model-path lmsys/vicuna-33b-v1.3 --model-name vicuna-33b --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --num-gpus 2\n```\n\nnode-02\n```\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-13b-chat-hf --model-name llama-2-13b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-13b-chat-hf --model-name llama-2-13b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker-address http://$(hostname):31001 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=2 python3 -m fastchat.serve.vllm_worker --model-path meta-llama/Llama-2-7b-chat-hf --model-name llama-2-7b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --tokenizer meta-llama/Llama-2-7b-chat-hf\nCUDA_VISIBLE_DEVICES=3 python3 -m fastchat.serve.vllm_worker --model-path WizardLM/WizardLM-13B-V1.1 --model-name wizardlm-13b  --controller http://node-01:10002 --host 0.0.0.0 --port 31003 --worker-address http://$(hostname):31003\n```\n\nnode-03\n```\npython3 -m fastchat.serve.vllm_worker --model-path mosaicml/mpt-30b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker-address http://$(hostname):31000 --num-gpus 2\npython3 -m fastchat.serve.vllm_worker --model-path timdettmers/guanaco-33b-merged --model-name guanaco-33b  --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker-address http://$(hostname):31002 --num-gpus 2 --tokenizer hf-internal-testing/llama-tokenizer\n```\n\nnode-04\n```\nCUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.multi_model_worker --model-path ~/model_weights/RWKV-4-Raven-14B-v12-Eng98%25-Other2%25-20230523-ctx8192.pth --model-name RWKV-4-Raven-14B --model-path lmsys/fastchat-t5-3b-v1.0 --model-name fastchat-t5-3b --controller http://node-01:10002 --host 0.0.0.0 --port 31000 --worker http://$(hostname):31000 --limit 4\nCUDA_VISIBLE_DEVICES=1 python3 -m fastchat.serve.multi_model_worker --model-path OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5 --model-name oasst-pythia-12b --model-path mosaicml/mpt-7b-chat --model-name mpt-7b-chat --controller http://node-01:10002 --host 0.0.0.0 --port 31001 --worker http://$(hostname):31001 --limit 4\nCUDA_VISIBLE_DEVICES=2 python3 -m fastchat.serve.multi_model_worker --model-path lmsys/vicuna-7b-v1.5 --model-name vicuna-7b --model-path THUDM/chatglm-6b --model-name chatglm-6b --controller http://node-01:10002 --host 0.0.0.0 --port 31002 --worker http://$(hostname):31002 --limit 4\nCUDA_VISIBLE_DEVICES=3 python3 -m fastchat.serve.vllm_worker --model-path ~/model_weights/alpaca-13b  --controller http://node-01:10002 --host 0.0.0.0 --port 31003 --worker-address http://$(hostname):31003\n```\n\ntest\n```\npython3 -m fastchat.serve.test_message --model vicuna-13b --controller http://localhost:10002\n```\n\n<!-- File: docs/commands/pypi.md (Tokens: ~45 | Category: command-prompt) -->\n### Requirement\n```\npython3 -m pip install twine\npython3 -m pip install --upgrade pip\npip3 install build\n```\n\n### Upload\n```\nbash scripts/upload_pypi.sh\n```\n\n<!-- File: docs/commands/webserver.md (Tokens: ~798 | Category: command-prompt) -->\n### Install\n```\nsudo apt update\nsudo apt install tmux htop\n\nwget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh\nbash Anaconda3-2022.10-Linux-x86_64.sh\n\nconda create -n fastchat python=3.9\nconda activate fastchat\n\ngit clone https://github.com/lm-sys/FastChat.git\ncd FastChat\npip3 install -e .\n```\n\n\n### Launch servers\n```\ncd fastchat_logs/controller\npython3 -m fastchat.serve.controller --host 0.0.0.0 --port 21001\npython3 -m fastchat.serve.register_worker --controller http://localhost:21001 --worker-name https://\npython3 -m fastchat.serve.test_message --model vicuna-13b --controller http://localhost:21001\n\ncd fastchat_logs/server0\n\npython3 -m fastchat.serve.huggingface_api_worker --model-info-file ~/elo_results/register_hf_api_models.json\n\nexport OPENAI_API_KEY=\nexport ANTHROPIC_API_KEY=\nexport GCP_PROJECT_ID=\n\npython3 -m fastchat.serve.gradio_web_server_multi --controller http://localhost:21001 --concurrency 50 --add-chatgpt --add-claude --add-palm --elo ~/elo_results/elo_results.pkl --leaderboard-table-file ~/elo_results/leaderboard_table.csv --register ~/elo_results/register_oai_models.json --show-terms\n\npython3 backup_logs.py\n```\n\n\n### Check the launch time\n```\nfor i in $(seq 0 11); do cat fastchat_logs/server$i/gradio_web_server.log | grep \"Running on local URL\" | tail -n 1; done\n```\n\n\n### Increase the limit of max open files\nOne process (do not need reboot)\n```\nsudo prlimit --nofile=1048576:1048576 --pid=$id\n\nfor id in $(ps -ef | grep gradio_web_server | awk '{print $2}'); do echo $id; prlimit --nofile=1048576:1048576 --pid=$id; done\n```\n\nSystem (need reboot): Add the lines below to `/etc/security/limits.conf`\n```\n* hard nofile 65535\n* soft nofile 65535\n```\n\n\n### Gradio edit  (3.35.2)\n1. gtag and canvas\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/templates/frontend/index.html\n```\n\n```\n<!-- Google tag (gtag.js) -->\n<script async src=\"https://www.googletagmanager.com/gtag/js?id=G-K6D24EE9ED\"></script><script>\n  window.dataLayer = window.dataLayer || [];\n  function gtag(){dataLayer.push(arguments);}\n  gtag('js', new Date());\n  gtag('config', 'G-K6D24EE9ED');\n  window.__gradio_mode__ = \"app\";\n</script>\n<script src=\"https://cdnjs.cloudflare.com/ajax/libs/html2canvas/1.4.1/html2canvas.min.js\"></script>\n```\n\n2. deprecation warnings\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/deprecation.py\n```\n\n```\ndef check_deprecated_parameters(\n```\n\n3. Loading\n```\nvim /home/vicuna/anaconda3/envs/fastchat/lib/python3.9/site-packages/gradio/templates/frontend/assets/index-188ef5e8.js\n```\n\n```\n%s/\"Loading...\"/\"Loading...(Please refresh if it takes more than 30 seconds)\"/g\n```\n</yakaai_skills>"}