大模型微调实战指南：LoRA vs QLoRA vs 全量微调

# 大模型微调实战指南：LoRA vs QLoRA vs 全量微调 本教程将手把手带你完成三种主流微调方法的实操，从环境搭建到模型部署，每一步都有详细说明。 ## 一、微调方法对比 | 方法 | 显存需求 | 训练速度 | 效果 | 适用场景 | |------|---------|---------|------|---------| | 全量微调 | 80GB+ | 慢 | 最好 | 数据充足、有GPU集群 | | LoRA | 16-24GB | 快 | 好 | 个人开发者、小数据集 | | QLoRA | 8-12GB | 中等 | 较好 | 消费级显卡、预算有限 | **简单理解：** - **全量微调**：改模型所有参数，像重新学一遍 - **LoRA**：只加一小层外挂参数，原模型不动 - **QLoRA**：先把模型压缩到4bit，再加LoRA ## 二、环境准备 ### 2.1 硬件要求 - **最低配置**：RTX 3090 / RTX 4080（24GB显存） - **推荐配置**：RTX 4090 / A100（40GB+显存） - **磁盘空间**：至少 50GB（模型权重 + 数据集） ### 2.2 软件环境 Looking in indexes: https://download.pytorch.org/whl/cu121 Collecting torch==2.1.0 Downloading https://download-r2.pytorch.org/whl/cu121/torch-2.1.0%2Bcu121-cp311-cp311-linux_x86_64.whl (2200.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 GB 548.7 kB/s eta 0:00:00 Collecting filelock (from torch==2.1.0) Downloading filelock-3.25.2-py3-none-any.whl.metadata (2.0 kB) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.11/site-packages (from torch==2.1.0) (4.15.0) Collecting sympy (from torch==2.1.0) Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB) Collecting networkx (from torch==2.1.0) Downloading networkx-3.6.1-py3-none-any.whl.metadata (6.8 kB) Requirement already satisfied: jinja2 in /usr/lib/python3.11/site-packages (from torch==2.1.0) (3.1.4) Collecting fsspec (from torch==2.1.0) Downloading fsspec-2026.2.0-py3-none-any.whl.metadata (10 kB) Collecting triton==2.1.0 (from torch==2.1.0) Downloading https://download-r2.pytorch.org/whl/triton-2.1.0-0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (89.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 MB 1.9 MB/s eta 0:00:00 Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib64/python3.11/site-packages (from jinja2->torch==2.1.0) (2.1.3) Collecting mpmath<1.4,>=1.1.0 (from sympy->torch==2.1.0) Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) Downloading filelock-3.25.2-py3-none-any.whl (26 kB) Looking in indexes: https://mirrors.tencent.com/pypi/simple Collecting transformers==4.36.0 Downloading https://mirrors.tencent.com/pypi/packages/0f/12/d8e27a190ca67811f81deea3183b528d9169f10b74d827e0b9211520ecfa/transformers-4.36.0-py3-none-any.whl (8.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.2/8.2 MB 42.9 MB/s eta 0:00:00 Collecting peft==0.7.0 Downloading https://mirrors.tencent.com/pypi/packages/82/cc/bf022d6bc3996a5939c3ee39bde2b0e1f8bf6cea6ef9c9cdaf1639586237/peft-0.7.0-py3-none-any.whl (168 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.3/168.3 kB 387.7 kB/s eta 0:00:00 Collecting accelerate==0.25.0 Downloading https://mirrors.tencent.com/pypi/packages/f7/fc/c55e5a2da345c9a24aa2e1e0f60eb2ca290b6a41be82da03a6d4baec4f99/accelerate-0.25.0-py3-none-any.whl (265 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 265.7/265.7 kB 19.5 MB/s eta 0:00:00 Collecting filelock (from transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/81/47/dd9a212ef6e343a6857485ffe25bba537304f1913bdbed446a23f7f592e1/filelock-3.29.0-py3-none-any.whl (39 kB) Collecting huggingface-hub<1.0,>=0.19.3 (from transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/a8/af/48ac8483240de756d2438c380746e7130d1c6f75802ef22f3c6d49982787/huggingface_hub-0.36.2-py3-none-any.whl (566 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 566.4/566.4 kB 29.9 MB/s eta 0:00:00 Collecting numpy>=1.17 (from transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/cf/c5/9fcb7e0e69cef59cf10c746b84f7d58b08bc66a6b7d459783c5a4f6101a6/numpy-2.4.4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.9/16.9 MB 51.6 MB/s eta 0:00:00 Collecting packaging>=20.0 (from transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl (100 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.2/100.2 kB 9.7 MB/s eta 0:00:00 Requirement already satisfied: pyyaml>=5.1 in /usr/lib64/python3.11/site-packages (from transformers==4.36.0) (6.0.1) Collecting regex!=2019.12.17 (from transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/01/ea/4c8d306e9c36ac22417336b1e02e7b358152c34dc379673f2d331143725f/regex-2026.4.4-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (799 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 799.8/799.8 kB 5.8 MB/s eta 0:00:00 Requirement already satisfied: requests in /usr/lib/python3.11/site-packages (from transformers==4.36.0) (2.32.3) Collecting tokenizers<0.19,>=0.14 (from transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/15/0b/c09b2c0dc688c82adadaa0d5080983de3ce920f4a5cbadb7eaa5302ad251/tokenizers-0.15.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 39.1 MB/s eta 0:00:00 Collecting safetensors>=0.3.1 (from transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/a0/60/429e9b1cb3fc651937727befe258ea24122d9663e4d5709a48c9cbfceecb/safetensors-0.7.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (507 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 507.2/507.2 kB 3.1 MB/s eta 0:00:00 Collecting tqdm>=4.27 (from transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/16/e1/3079a9ff9b8e11b846c6ac5c8b5bfb7ff225eee721825310c91b3b50304f/tqdm-4.67.3-py3-none-any.whl (78 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.4/78.4 kB 6.1 MB/s eta 0:00:00 Collecting psutil (from peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/b5/70/5d8df3b09e25bce090399cf48e452d25c935ab72dad19406c77f4e828045/psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl (155 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.6/155.6 kB 1.8 MB/s eta 0:00:00 Collecting torch>=1.13.0 (from peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/8c/8b/d7be22fbec9ffee6cff31a39f8750d4b3a65d349a286cf4aec74c2375662/torch-2.11.0-cp311-cp311-manylinux_2_28_x86_64.whl (530.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 530.6/530.6 MB 4.7 MB/s eta 0:00:00 Collecting fsspec>=2023.5.0 (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/d5/1f/5f4a3cd9e4440e9d9bc78ad0a91a1c8d46b4d429d5239ebe6793c9fe5c41/fsspec-2026.3.0-py3-none-any.whl (202 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 202.6/202.6 kB 13.5 MB/s eta 0:00:00 Collecting hf-xet<2.0.0,>=1.1.3 (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0) Downloading https://mirrors.tencent.com/pypi/packages/df/9a/a24b26dc8a65f0ecc0fe5be981a19e61e7ca963b85e062c083f3a9100529/hf_xet-1.4.3-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 19.4 MB/s eta 0:00:00 Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0) (4.15.0) Requirement already satisfied: setuptools<82 in /usr/lib/python3.11/site-packages (from torch>=1.13.0->peft==0.7.0) (68.0.0) Collecting sympy>=1.13.3 (from torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl (6.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 57.3 MB/s eta 0:00:00 Collecting networkx>=2.5.1 (from torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/9e/c9/b2622292ea83fbb4ec318f5b9ab867d0a28ab43c5717bb85b0a5f6b3b0a4/networkx-3.6.1-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 9.8 MB/s eta 0:00:00 Requirement already satisfied: jinja2 in /usr/lib/python3.11/site-packages (from torch>=1.13.0->peft==0.7.0) (3.1.4) Collecting cuda-toolkit==13.0.2 (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/57/b2/453099f5f3b698d7d0eab38916aac44c7f76229f451709e2eb9db6615dcd/cuda_toolkit-13.0.2-py2.py3-none-any.whl (2.4 kB) Collecting cuda-bindings<14,>=13.0.3 (from torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/e9/94/2748597f47bb1600cd466b20cab4159f1530a3a33fe7f70fee199b3abb9e/cuda_bindings-13.2.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (6.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 21.1 MB/s eta 0:00:00 Collecting nvidia-cudnn-cu13==9.19.0.56 (from torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/a3/22/0b4b932655d17a6da1b92fa92ab12844b053bb2ac2475e179ba6f043da1e/nvidia_cudnn_cu13-9.19.0.56-py3-none-manylinux_2_27_x86_64.whl (366.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 366.1/366.1 MB 7.2 MB/s eta 0:00:00 Collecting nvidia-cusparselt-cu13==0.8.0 (from torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/fd/53/43b0d71f4e702fa9733f8b4571fdca50a8813f1e450b656c239beff12315/nvidia_cusparselt_cu13-0.8.0-py3-none-manylinux2014_x86_64.whl (169.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.9/169.9 MB 15.2 MB/s eta 0:00:00 Collecting nvidia-nccl-cu13==2.28.9 (from torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/b0/b4/878fefaad5b2bcc6fcf8d474a25e3e3774bc5133e4b58adff4d0bca238bc/nvidia_nccl_cu13-2.28.9-py3-none-manylinux_2_18_x86_64.whl (196.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.5/196.5 MB 14.5 MB/s eta 0:00:00 Collecting nvidia-nvshmem-cu13==3.4.5 (from torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/3c/35/a9bf80a609e74e3b000fef598933235c908fcefcef9026042b8e6dfde2a9/nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (60.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.4/60.4 MB 50.0 MB/s eta 0:00:00 Collecting triton==3.6.0 (from torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/e0/12/b05ba554d2c623bffa59922b94b0775673de251f468a9609bc9e45de95e9/triton-3.6.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (188.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 188.2/188.2 MB 16.1 MB/s eta 0:00:00 Collecting nvidia-cublas==13.1.0.3.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/e7/44/423ac00af4dd95a5aeb27207e2c0d9b7118702149bf4704c3ddb55bb7429/nvidia_cublas-13.1.0.3-py3-none-manylinux_2_27_x86_64.whl (423.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 423.1/423.1 MB 6.1 MB/s eta 0:00:00 Collecting nvidia-cuda-runtime==13.0.96.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/2e/24/d1558f3b68b1d26e706813b1d10aa1d785e4698c425af8db8edc3dced472/nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (2.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 41.6 MB/s eta 0:00:00 Collecting nvidia-cufft==12.0.0.61.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/a8/2f/7b57e29836ea8714f81e9898409196f47d772d5ddedddf1592eadb8ab743/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (214.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 214.1/214.1 MB 11.9 MB/s eta 0:00:00 Collecting nvidia-cufile==1.15.1.6.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/3f/70/4f193de89a48b71714e74602ee14d04e4019ad36a5a9f20c425776e72cd6/nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 27.2 MB/s eta 0:00:00 Collecting nvidia-cuda-cupti==13.0.85.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/33/6d/737d164b4837a9bbd202f5ae3078975f0525a55730fe871d8ed4e3b952b0/nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl (10.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 66.4 MB/s eta 0:00:00 Collecting nvidia-curand==10.4.0.35.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/a5/9f/be0a41ca4a4917abf5cb9ae0daff1a6060cc5de950aec0396de9f3b52bc5/nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl (59.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.5/59.5 MB 51.8 MB/s eta 0:00:00 Collecting nvidia-cusolver==12.0.4.66.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/5f/67/cba3777620cdacb99102da4042883709c41c709f4b6323c10781a9c3aa34/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl (200.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.9/200.9 MB 13.6 MB/s eta 0:00:00 Collecting nvidia-cusparse==12.6.3.3.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/fa/18/623c77619c31d62efd55302939756966f3ecc8d724a14dab2b75f1508850/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (145.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 145.9/145.9 MB 11.6 MB/s eta 0:00:00 Collecting nvidia-nvjitlink==13.0.88.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/56/7a/123e033aaff487c77107195fa5a2b8686795ca537935a24efae476c41f05/nvidia_nvjitlink-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (40.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.7/40.7 MB 55.0 MB/s eta 0:00:00 Collecting nvidia-cuda-nvrtc==13.0.88.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/c3/68/483a78f5e8f31b08fb1bb671559968c0ca3a065ac7acabfc7cee55214fd6/nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (90.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.2/90.2 MB 50.2 MB/s eta 0:00:00 Collecting nvidia-nvtx==13.0.85.* (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/c2/f3/d86c845465a2723ad7e1e5c36dcd75ddb82898b3f53be47ebd429fb2fa5d/nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (148 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 148.0/148.0 kB 1.7 MB/s eta 0:00:00 Requirement already satisfied: charset-normalizer<4,>=2 in /usr/lib/python3.11/site-packages (from requests->transformers==4.36.0) (3.2.0) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.11/site-packages (from requests->transformers==4.36.0) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3.11/site-packages (from requests->transformers==4.36.0) (1.26.19) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3.11/site-packages (from requests->transformers==4.36.0) (2023.7.22) Collecting cuda-pathfinder~=1.1 (from cuda-bindings<14,>=13.0.3->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/11/d0/c177e29701cf1d3008d7d2b16b5fc626592ce13bd535f8795c5f57187e0e/cuda_pathfinder-1.5.4-py3-none-any.whl (51 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.7/51.7 kB 5.8 MB/s eta 0:00:00 Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch>=1.13.0->peft==0.7.0) Downloading https://mirrors.tencent.com/pypi/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl (536 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 3.8 MB/s eta 0:00:00 Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib64/python3.11/site-packages (from jinja2->torch>=1.13.0->peft==0.7.0) (2.1.3) Installing collected packages: nvidia-cusparselt-cu13, mpmath, cuda-toolkit, triton, tqdm, sympy, safetensors, regex, psutil, packaging, nvidia-nvtx, nvidia-nvshmem-cu13, nvidia-nvjitlink, nvidia-nccl-cu13, nvidia-curand, nvidia-cufile, nvidia-cuda-runtime, nvidia-cuda-nvrtc, nvidia-cuda-cupti, nvidia-cublas, numpy, networkx, hf-xet, fsspec, filelock, cuda-pathfinder, nvidia-cusparse, nvidia-cufft, nvidia-cudnn-cu13, huggingface-hub, cuda-bindings, tokenizers, nvidia-cusolver, transformers, torch, accelerate, peft Successfully installed accelerate-0.25.0 cuda-bindings-13.2.0 cuda-pathfinder-1.5.4 cuda-toolkit-13.0.2 filelock-3.29.0 fsspec-2026.3.0 hf-xet-1.4.3 huggingface-hub-0.36.2 mpmath-1.3.0 networkx-3.6.1 numpy-2.4.4 nvidia-cublas-13.1.0.3 nvidia-cuda-cupti-13.0.85 nvidia-cuda-nvrtc-13.0.88 nvidia-cuda-runtime-13.0.96 nvidia-cudnn-cu13-9.19.0.56 nvidia-cufft-12.0.0.61 nvidia-cufile-1.15.1.6 nvidia-curand-10.4.0.35 nvidia-cusolver-12.0.4.66 nvidia-cusparse-12.6.3.3 nvidia-cusparselt-cu13-0.8.0 nvidia-nccl-cu13-2.28.9 nvidia-nvjitlink-13.0.88 nvidia-nvshmem-cu13-3.4.5 nvidia-nvtx-13.0.85 packaging-26.2 peft-0.7.0 psutil-7.2.2 regex-2026.4.4 safetensors-0.7.0 sympy-1.14.0 tokenizers-0.15.2 torch-2.11.0 tqdm-4.67.3 transformers-4.36.0 triton-3.6.0 Looking in indexes: https://mirrors.tencent.com/pypi/simple Collecting datasets==2.15.0 Downloading https://mirrors.tencent.com/pypi/packages/e2/cf/db41e572d7ed958e8679018f8190438ef700aeb501b62da9e1eed9e4d69a/datasets-2.15.0-py3-none-any.whl (521 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 521.2/521.2 kB 3.5 MB/s eta 0:00:00 Collecting bitsandbytes==0.41.3 Downloading https://mirrors.tencent.com/pypi/packages/1b/db/1a3c0d3542484806c273e8027a328b12be69c1042bb9e134efe93ddf9b50/bitsandbytes-0.41.3-py3-none-any.whl (92.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.6/92.6 MB 27.1 MB/s eta 0:00:00 Requirement already satisfied: numpy>=1.17 in /usr/local/lib64/python3.11/site-packages (from datasets==2.15.0) (2.4.4) Collecting pyarrow>=8.0.0 (from datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/f3/27/99c42abe8e21b44f4917f62631f3aa31404882a2c41d8a4cd5c110e13d52/pyarrow-24.0.0-cp311-cp311-manylinux_2_28_x86_64.whl (48.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.8/48.8 MB 8.9 MB/s eta 0:00:00 Collecting pyarrow-hotfix (from datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/2e/c3/94ade4906a2f88bc935772f59c934013b4205e773bcb4239db114a6da136/pyarrow_hotfix-0.7-py3-none-any.whl (7.9 kB) Collecting dill<0.3.8,>=0.3.0 (from datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/f5/3a/74a29b11cf2cdfcd6ba89c0cecd70b37cd1ba7b77978ce611eb7a146a832/dill-0.3.7-py3-none-any.whl (115 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 115.3/115.3 kB 1.2 MB/s eta 0:00:00 Collecting pandas (from datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/20/17/ec40d981705654853726e7ac9aea9ddbb4a5d9cf54d8472222f4f3de06c2/pandas-3.0.2-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (11.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 68.1 MB/s eta 0:00:00 Requirement already satisfied: requests>=2.19.0 in /usr/lib/python3.11/site-packages (from datasets==2.15.0) (2.32.3) Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.11/site-packages (from datasets==2.15.0) (4.67.3) Collecting xxhash (from datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/f7/d0/3c91e4e6a05ca4d7df8e39ec3a75b713609258ec84705ab34be6430826a1/xxhash-3.7.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (193 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 193.9/193.9 kB 2.2 MB/s eta 0:00:00 Collecting multiprocess (from datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/86/c2/dec9722dc3474c164a0b6bcd9a7ed7da542c98af8cabce05374abab35edd/multiprocess-0.70.19-py311-none-any.whl (144 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 144.5/144.5 kB 9.3 MB/s eta 0:00:00 Collecting fsspec<=2023.10.0,>=2023.1.0 (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/e8/f6/3eccfb530aac90ad1301c582da228e4763f19e719ac8200752a4841b0b2d/fsspec-2023.10.0-py3-none-any.whl (166 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.4/166.4 kB 1.7 MB/s eta 0:00:00 Collecting aiohttp (from datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/7e/a5/0521aa32c1ddf3aa1e71dcc466be0b7db2771907a13f18cddaa45967d97b/aiohttp-3.13.5-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (1.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 31.9 MB/s eta 0:00:00 Requirement already satisfied: huggingface-hub>=0.18.0 in /usr/local/lib/python3.11/site-packages (from datasets==2.15.0) (0.36.2) Requirement already satisfied: packaging in /usr/local/lib/python3.11/site-packages (from datasets==2.15.0) (26.2) Requirement already satisfied: pyyaml>=5.1 in /usr/lib64/python3.11/site-packages (from datasets==2.15.0) (6.0.1) Collecting aiohappyeyeballs>=2.5.0 (from aiohttp->datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/0f/15/5bf3b99495fb160b63f95972b81750f18f7f4e02ad051373b669d17d44f2/aiohappyeyeballs-2.6.1-py3-none-any.whl (15 kB) Collecting aiosignal>=1.4.0 (from aiohttp->datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl (7.5 kB) Requirement already satisfied: attrs>=17.3.0 in /usr/lib/python3.11/site-packages (from aiohttp->datasets==2.15.0) (23.1.0) Collecting frozenlist>=1.1.1 (from aiohttp->datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/11/b1/71a477adc7c36e5fb628245dfbdea2166feae310757dea848d02bd0689fd/frozenlist-1.8.0-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (231 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 231.1/231.1 kB 2.0 MB/s eta 0:00:00 Collecting multidict<7.0,>=4.5 (from aiohttp->datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/5a/56/21b27c560c13822ed93133f08aa6372c53a8e067f11fbed37b4adcdac922/multidict-6.7.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (246 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 246.3/246.3 kB 12.2 MB/s eta 0:00:00 Collecting propcache>=0.2.0 (from aiohttp->datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/52/6a/57f43e054fb3d3a56ac9fc532bc684fc6169a26c75c353e65425b3e56eef/propcache-0.4.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (210 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 210.0/210.0 kB 2.1 MB/s eta 0:00:00 Collecting yarl<2.0,>=1.17.0 (from aiohttp->datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/9a/64/c53487d9f4968045b8afa51aed7ca44f58b2589e772f32745f3744476c82/yarl-1.23.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (102 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.7/102.7 kB 6.6 MB/s eta 0:00:00 Requirement already satisfied: filelock in /usr/local/lib/python3.11/site-packages (from huggingface-hub>=0.18.0->datasets==2.15.0) (3.29.0) Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /usr/local/lib64/python3.11/site-packages (from huggingface-hub>=0.18.0->datasets==2.15.0) (1.4.3) Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/site-packages (from huggingface-hub>=0.18.0->datasets==2.15.0) (4.15.0) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/lib/python3.11/site-packages (from requests>=2.19.0->datasets==2.15.0) (3.2.0) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.11/site-packages (from requests>=2.19.0->datasets==2.15.0) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3.11/site-packages (from requests>=2.19.0->datasets==2.15.0) (1.26.19) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3.11/site-packages (from requests>=2.19.0->datasets==2.15.0) (2023.7.22) INFO: pip is looking at multiple versions of multiprocess to determine which version is compatible with other requirements. This could take a while. Collecting multiprocess (from datasets==2.15.0) Downloading https://mirrors.tencent.com/pypi/packages/4b/88/9039f2fed1012ef584751d4ceff9ab4a51e5ae264898f0b7cbf44340a859/multiprocess-0.70.18-py311-none-any.whl (144 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 144.5/144.5 kB 10.1 MB/s eta 0:00:00 Downloading https://mirrors.tencent.com/pypi/packages/b2/07/8cbb75d6cfbe8712d8f7f6a5615f083c6e710ab916b748fbb20373ddb142/multiprocess-0.70.17-py311-none-any.whl (144 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 144.3/144.3 kB 1.7 MB/s eta 0:00:00 Downloading https://mirrors.tencent.com/pypi/packages/50/15/b56e50e8debaf439f44befec5b2af11db85f6e0f344c3113ae0be0593a91/multiprocess-0.70.16-py311-none-any.whl (143 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.5/143.5 kB 7.6 MB/s eta 0:00:00 Downloading https://mirrors.tencent.com/pypi/packages/e7/41/96ac938770ba6e7d5ae1d8c9cafebac54b413549042c6260f0d0a6ec6622/multiprocess-0.70.15-py311-none-any.whl (135 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.4/135.4 kB 9.1 MB/s eta 0:00:00 Requirement already satisfied: python-dateutil>=2.8.2 in /usr/lib/python3.11/site-packages (from pandas->datasets==2.15.0) (2.8.2) Requirement already satisfied: six>=1.5 in /usr/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->datasets==2.15.0) (1.16.0) Installing collected packages: bitsandbytes, xxhash, pyarrow-hotfix, pyarrow, propcache, multidict, fsspec, frozenlist, dill, aiohappyeyeballs, yarl, pandas, multiprocess, aiosignal, aiohttp, datasets Attempting uninstall: fsspec Found existing installation: fsspec 2026.3.0 Uninstalling fsspec-2026.3.0: Successfully uninstalled fsspec-2026.3.0 Successfully installed aiohappyeyeballs-2.6.1 aiohttp-3.13.5 aiosignal-1.4.0 bitsandbytes-0.41.3 datasets-2.15.0 dill-0.3.7 frozenlist-1.8.0 fsspec-2023.10.0 multidict-6.7.1 multiprocess-0.70.15 pandas-3.0.2 propcache-0.4.1 pyarrow-24.0.0 pyarrow-hotfix-0.7 xxhash-3.7.0 yarl-1.23.0 Looking in indexes: https://mirrors.tencent.com/pypi/simple Collecting trl==0.7.4 Downloading https://mirrors.tencent.com/pypi/packages/0d/44/c406c3cf5981bddb16ff72acb5ca235888db4073d868cf51bd143bef3aad/trl-0.7.4-py3-none-any.whl (133 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.9/133.9 kB 5.9 MB/s eta 0:00:00 Collecting wandb Downloading https://mirrors.tencent.com/pypi/packages/7b/e9/b4bf8f3509dcea1cec52233a38991459654635b5a8e6a494eb912e1b9cfb/wandb-0.26.1-py3-none-manylinux_2_28_x86_64.whl (27.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 27.2/27.2 MB 9.8 MB/s eta 0:00:00 Collecting scipy Downloading https://mirrors.tencent.com/pypi/packages/09/7d/af933f0f6e0767995b4e2d705a0665e454d1c19402aa7e895de3951ebb04/scipy-1.17.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (35.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 35.3/35.3 MB 75.3 MB/s eta 0:00:00 Requirement already satisfied: torch>=1.4.0 in /usr/local/lib64/python3.11/site-packages (from trl==0.7.4) (2.11.0) Requirement already satisfied: transformers>=4.18.0 in /usr/local/lib/python3.11/site-packages (from trl==0.7.4) (4.36.0) Requirement already satisfied: numpy>=1.18.2 in /usr/local/lib64/python3.11/site-packages (from trl==0.7.4) (2.4.4) Requirement already satisfied: accelerate in /usr/local/lib/python3.11/site-packages (from trl==0.7.4) (0.25.0) Requirement already satisfied: datasets in /usr/local/lib/python3.11/site-packages (from trl==0.7.4) (2.15.0) Collecting tyro>=0.5.11 (from trl==0.7.4) Downloading https://mirrors.tencent.com/pypi/packages/93/4f/c43a0a8f0c66fd40a1d6cc47332a5a1d1043e9b331f7070ea701b91a7598/tyro-1.0.13-py3-none-any.whl (185 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 185.2/185.2 kB 4.9 MB/s eta 0:00:00 Collecting click>=8.0.1 (from wandb) Downloading https://mirrors.tencent.com/pypi/packages/ae/44/c1221527f6a71a01ec6fbad7fa78f1d50dfa02217385cf0fa3eec7087d59/click-8.3.3-py3-none-any.whl (110 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110.5/110.5 kB 4.8 MB/s eta 0:00:00 Collecting gitpython!=3.1.29,>=1.0.0 (from wandb) Downloading https://mirrors.tencent.com/pypi/packages/fd/6f/b842bfa6f21d6f87c57f9abf7194225e55279d96d869775e19e9f7236fc5/gitpython-3.1.49-py3-none-any.whl (212 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 212.2/212.2 kB 4.9 MB/s eta 0:00:00 Requirement already satisfied: packaging in /usr/local/lib/python3.11/site-packages (from wandb) (26.2) Collecting platformdirs (from wandb) Downloading https://mirrors.tencent.com/pypi/packages/75/a6/a0a304dc33b49145b21f4808d763822111e67d1c3a32b524a1baf947b6e1/platformdirs-4.9.6-py3-none-any.whl (21 kB) Collecting protobuf!=5.28.0,!=5.29.0,<8,>4.21.0 (from wandb) Downloading https://mirrors.tencent.com/pypi/packages/53/1b/3b431694a4dc6d37b9f653f0c64b0a0d9ec074ee810710c0c3da21d67ba7/protobuf-7.34.1-cp310-abi3-manylinux2014_x86_64.whl (324 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 324.3/324.3 kB 6.3 MB/s eta 0:00:00 Collecting pydantic<3 (from wandb) Downloading https://mirrors.tencent.com/pypi/packages/f3/0a/fd7d723f8f8153418fb40cf9c940e82004fce7e987026b08a68a36dd3fe7/pydantic-2.13.3-py3-none-any.whl (471 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 472.0/472.0 kB 26.9 MB/s eta 0:00:00 Requirement already satisfied: pyyaml in /usr/lib64/python3.11/site-packages (from wandb) (6.0.1) Requirement already satisfied: requests<3,>=2.0.0 in /usr/lib/python3.11/site-packages (from wandb) (2.32.3) Collecting sentry-sdk>=2.0.0 (from wandb) Downloading https://mirrors.tencent.com/pypi/packages/fa/eb/d875669993b762556ae8b2efd86219943b4c0864d22204d622a9aee3052b/sentry_sdk-2.58.0-py2.py3-none-any.whl (460 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 460.9/460.9 kB 6.3 MB/s eta 0:00:00 Requirement already satisfied: typing-extensions<5,>=4.8 in /usr/local/lib/python3.11/site-packages (from wandb) (4.15.0) Collecting gitdb<5,>=4.0.1 (from gitpython!=3.1.29,>=1.0.0->wandb) Downloading https://mirrors.tencent.com/pypi/packages/a0/61/5c78b91c3143ed5c14207f463aecfc8f9dbb5092fb2869baf37c273b2705/gitdb-4.0.12-py3-none-any.whl (62 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 6.6 MB/s eta 0:00:00 Collecting annotated-types>=0.6.0 (from pydantic<3->wandb) Downloading https://mirrors.tencent.com/pypi/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl (13 kB) Collecting pydantic-core==2.46.3 (from pydantic<3->wandb) Downloading https://mirrors.tencent.com/pypi/packages/0a/44/93f489d16fb63fbd41c670441536541f6e8cfa1e5a69f40bc9c5d30d8c90/pydantic_core-2.46.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 44.6 MB/s eta 0:00:00 Collecting typing-inspection>=0.4.2 (from pydantic<3->wandb) Downloading https://mirrors.tencent.com/pypi/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl (14 kB) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/lib/python3.11/site-packages (from requests<3,>=2.0.0->wandb) (3.2.0) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.11/site-packages (from requests<3,>=2.0.0->wandb) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3.11/site-packages (from requests<3,>=2.0.0->wandb) (1.26.19) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3.11/site-packages (from requests<3,>=2.0.0->wandb) (2023.7.22) Requirement already satisfied: filelock in /usr/local/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (3.29.0) Requirement already satisfied: setuptools<82 in /usr/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (68.0.0) Requirement already satisfied: sympy>=1.13.3 in /usr/local/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (1.14.0) Requirement already satisfied: networkx>=2.5.1 in /usr/local/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (3.6.1) Requirement already satisfied: jinja2 in /usr/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (3.1.4) Requirement already satisfied: fsspec>=0.8.5 in /usr/local/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (2023.10.0) Requirement already satisfied: cuda-toolkit==13.0.2 in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (13.0.2) Requirement already satisfied: cuda-bindings<14,>=13.0.3 in /usr/local/lib64/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (13.2.0) Requirement already satisfied: nvidia-cudnn-cu13==9.19.0.56 in /usr/local/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (9.19.0.56) Requirement already satisfied: nvidia-cusparselt-cu13==0.8.0 in /usr/local/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (0.8.0) Requirement already satisfied: nvidia-nccl-cu13==2.28.9 in /usr/local/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (2.28.9) Requirement already satisfied: nvidia-nvshmem-cu13==3.4.5 in /usr/local/lib/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (3.4.5) Requirement already satisfied: triton==3.6.0 in /usr/local/lib64/python3.11/site-packages (from torch>=1.4.0->trl==0.7.4) (3.6.0) Requirement already satisfied: nvidia-cublas==13.1.0.3.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (13.1.0.3) Requirement already satisfied: nvidia-cuda-runtime==13.0.96.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (13.0.96) Requirement already satisfied: nvidia-cufft==12.0.0.61.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (12.0.0.61) Requirement already satisfied: nvidia-cufile==1.15.1.6.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (1.15.1.6) Requirement already satisfied: nvidia-cuda-cupti==13.0.85.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (13.0.85) Requirement already satisfied: nvidia-curand==10.4.0.35.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (10.4.0.35) Requirement already satisfied: nvidia-cusolver==12.0.4.66.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (12.0.4.66) Requirement already satisfied: nvidia-cusparse==12.6.3.3.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (12.6.3.3) Requirement already satisfied: nvidia-nvjitlink==13.0.88.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (13.0.88) Requirement already satisfied: nvidia-cuda-nvrtc==13.0.88.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (13.0.88) Requirement already satisfied: nvidia-nvtx==13.0.85.* in /usr/local/lib/python3.11/site-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==13.0.2; platform_system == "Linux"->torch>=1.4.0->trl==0.7.4) (13.0.85) Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /usr/local/lib/python3.11/site-packages (from transformers>=4.18.0->trl==0.7.4) (0.36.2) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib64/python3.11/site-packages (from transformers>=4.18.0->trl==0.7.4) (2026.4.4) Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib64/python3.11/site-packages (from transformers>=4.18.0->trl==0.7.4) (0.15.2) Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib64/python3.11/site-packages (from transformers>=4.18.0->trl==0.7.4) (0.7.0) Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.11/site-packages (from transformers>=4.18.0->trl==0.7.4) (4.67.3) Collecting docstring-parser>=0.15 (from tyro>=0.5.11->trl==0.7.4) Downloading https://mirrors.tencent.com/pypi/packages/a7/5f/ed01f9a3cdffbd5a008556fc7b2a08ddb1cc6ace7effa7340604b1d16699/docstring_parser-0.18.0-py3-none-any.whl (22 kB) Collecting typeguard>=4.0.0 (from tyro>=0.5.11->trl==0.7.4) Downloading https://mirrors.tencent.com/pypi/packages/91/88/b55b3117287a8540b76dbdd87733808d4d01c8067a3b339408c250bb3600/typeguard-4.5.1-py3-none-any.whl (36 kB) Requirement already satisfied: psutil in /usr/local/lib64/python3.11/site-packages (from accelerate->trl==0.7.4) (7.2.2) Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib64/python3.11/site-packages (from datasets->trl==0.7.4) (24.0.0) Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.11/site-packages (from datasets->trl==0.7.4) (0.7) Requirement already satisfied: dill<0.3.8,>=0.3.0 in /usr/local/lib/python3.11/site-packages (from datasets->trl==0.7.4) (0.3.7) Requirement already satisfied: pandas in /usr/local/lib64/python3.11/site-packages (from datasets->trl==0.7.4) (3.0.2) Requirement already satisfied: xxhash in /usr/local/lib64/python3.11/site-packages (from datasets->trl==0.7.4) (3.7.0) Requirement already satisfied: multiprocess in /usr/local/lib/python3.11/site-packages (from datasets->trl==0.7.4) (0.70.15) Requirement already satisfied: aiohttp in /usr/local/lib64/python3.11/site-packages (from datasets->trl==0.7.4) (3.13.5) Requirement already satisfied: cuda-pathfinder~=1.1 in /usr/local/lib/python3.11/site-packages (from cuda-bindings<14,>=13.0.3->torch>=1.4.0->trl==0.7.4) (1.5.4) Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /usr/local/lib/python3.11/site-packages (from aiohttp->datasets->trl==0.7.4) (2.6.1) Requirement already satisfied: aiosignal>=1.4.0 in /usr/local/lib/python3.11/site-packages (from aiohttp->datasets->trl==0.7.4) (1.4.0) Requirement already satisfied: attrs>=17.3.0 in /usr/lib/python3.11/site-packages (from aiohttp->datasets->trl==0.7.4) (23.1.0) Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib64/python3.11/site-packages (from aiohttp->datasets->trl==0.7.4) (1.8.0) Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib64/python3.11/site-packages (from aiohttp->datasets->trl==0.7.4) (6.7.1) Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib64/python3.11/site-packages (from aiohttp->datasets->trl==0.7.4) (0.4.1) Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib64/python3.11/site-packages (from aiohttp->datasets->trl==0.7.4) (1.23.0) Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython!=3.1.29,>=1.0.0->wandb) Downloading https://mirrors.tencent.com/pypi/packages/c1/d4/59e74daffcb57a07668852eeeb6035af9f32cbfd7a1d2511f17d2fe6a738/smmap-5.0.3-py3-none-any.whl (24 kB) Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /usr/local/lib64/python3.11/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers>=4.18.0->trl==0.7.4) (1.4.3) Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/site-packages (from sympy>=1.13.3->torch>=1.4.0->trl==0.7.4) (1.3.0) Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib64/python3.11/site-packages (from jinja2->torch>=1.4.0->trl==0.7.4) (2.1.3) Requirement already satisfied: python-dateutil>=2.8.2 in /usr/lib/python3.11/site-packages (from pandas->datasets->trl==0.7.4) (2.8.2) Requirement already satisfied: six>=1.5 in /usr/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->datasets->trl==0.7.4) (1.16.0) Installing collected packages: typing-inspection, typeguard, smmap, sentry-sdk, scipy, pydantic-core, protobuf, platformdirs, docstring-parser, click, annotated-types, tyro, pydantic, gitdb, gitpython, wandb, trl Successfully installed annotated-types-0.7.0 click-8.3.3 docstring-parser-0.18.0 gitdb-4.0.12 gitpython-3.1.49 platformdirs-4.9.6 protobuf-7.34.1 pydantic-2.13.3 pydantic-core-2.46.3 scipy-1.17.1 sentry-sdk-2.58.0 smmap-5.0.3 trl-0.7.4 typeguard-4.5.1 typing-inspection-0.4.2 tyro-1.0.13 wandb-0.26.1 ### 2.3 下载基础模型 ## 三、准备训练数据 ### 3.1 数据格式 推荐使用 Alpaca 格式的 JSON 数据： ### 3.2 加载和处理数据 **数据量建议：** - LoRA微调：500-5000条高质量数据即可 - 全量微调：建议 10000 条以上 - 数据质量 > 数据数量 ## 四、LoRA 微调（推荐新手） ### 4.1 配置 LoRA 参数 **参数调优技巧：** - ：简单任务（翻译、摘要） - ：中等任务（对话、问答） - ：复杂任务（代码生成、推理） ### 4.2 开始训练 ### 4.3 保存和加载 LoRA 权重 ## 五、QLoRA 微调（显存最少） **QLoRA 的核心优势：7B模型只需要约8GB显存，RTX 3060就能跑！** ## 六、效果测试 ## 七、常见问题 **Q1: 显存不够怎么办？** - 降低 到 1-2 - 增大 - 使用 QLoRA 而不是 LoRA - 开启 **Q2: 训练 loss 不下降？** - 检查学习率（LoRA 建议 1e-4 ~ 3e-4） - 检查数据质量（格式是否正确） - 增加训练轮数 **Q3: 如何选择基础模型？** - 中文场景：Qwen2、ChatGLM4、Baichuan2 - 英文场景：Llama-3、Mistral - 代码场景：DeepSeek-Coder、CodeLlama ## 总结 | 你的情况 | 推荐方案 | |---------|---------| | 刚入门、消费级显卡 | QLoRA + 7B模型 | | 有一定经验、24GB显存 | LoRA + 7B/13B模型 | | 公司级、有A100集群 | 全量微调 + 70B模型 | > 📌 关注AI导航，获取更多AI实战教程。如有疑问可在评论区留言！ *最后更新：2026年4月*