Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

trying to get this working with text-generation-webui #154

Open
caterpillarpants opened this issue Oct 27, 2023 · 9 comments
Open

trying to get this working with text-generation-webui #154

caterpillarpants opened this issue Oct 27, 2023 · 9 comments

Comments

@caterpillarpants
Copy link

I cannot compile the wheels. whether I use winglian or old_compatible, I end up with this error message.
` RuntimeError: Error compiling objects for extension
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for alpaca-lora-4bit
Running setup.py clean for alpaca-lora-4bit
Failed to build alpaca-lora-4bit
Installing collected packages: alpaca-lora-4bit
Running setup.py install for alpaca-lora-4bit ... error
error: subprocess-exited-with-error

× Running setup.py install for alpaca-lora-4bit did not run successfully.
│ exit code: 1
╰─> [125 lines of output]
`

then it resumes installing and later gives this error.
` RuntimeError: Error compiling objects for extension
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> alpaca-lora-4bit
`

I had gotten this to work in the past, but have since had to reinstall text-generation-webui, at which time my "monkey-patch" stopped working. Any ideas to get it to work? I need the 4bit support since I have limited vram.
@johnsmith0031
Copy link
Owner

Can you provide some more logs about the error?

@caterpillarpants
Copy link
Author

This is my attempt to install winglian

`ana@durga:/text-generation-webui-main/repositories/alpaca_lora_4bit$ pip uninstall alpaca_lora_4bit
WARNING: Skipping alpaca_lora_4bit as it is not installed.
ana@durga:/text-generation-webui-main/repositories/alpaca_lora_4bit$ git checkout winglian-setup_pip
Already on 'winglian-setup_pip'
Your branch is up to date with 'origin/winglian-setup_pip'.
ana@durga:~/text-generation-webui-main/repositories/alpaca_lora_4bit$ pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit
Preparing metadata (setup.py) ... done
Requirement already satisfied: accelerate>=0.22.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.24.0)
Requirement already satisfied: colorama in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.4.6)
Requirement already satisfied: datasets in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (2.10.1)
Requirement already satisfied: einops in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.6.1)
Requirement already satisfied: packaging in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (23.2)
Requirement already satisfied: peft>=0.5.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.5.0)
Requirement already satisfied: pyzmq in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (25.1.0)
Requirement already satisfied: safetensors in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.3.1)
Requirement already satisfied: sentencepiece in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.1.99)
Requirement already satisfied: torch in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (2.0.1+cu118)
Requirement already satisfied: transformers>=4.31.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (4.35.0.dev0)
Requirement already satisfied: wandb in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.15.4)
Requirement already satisfied: huggingface-hub in /home/ana/.local/lib/python3.10/site-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (0.17.3)
Requirement already satisfied: psutil in /usr/lib/python3/dist-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (5.9.0)
Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (5.4.1)
Requirement already satisfied: numpy>=1.17 in /home/ana/.local/lib/python3.10/site-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (1.24.0)
Requirement already satisfied: tqdm in /home/ana/.local/lib/python3.10/site-packages (from peft>=0.5.0->alpaca-lora-4bit==0.1.3) (4.65.0)
Requirement already satisfied: filelock in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.12.4)
Requirement already satisfied: jinja2 in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.1.2)
Requirement already satisfied: triton==2.0.0 in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (2.0.0)
Requirement already satisfied: sympy in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (1.11.1)
Requirement already satisfied: typing-extensions in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (4.5.0)
Requirement already satisfied: networkx in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.1)
Requirement already satisfied: lit in /home/ana/.local/lib/python3.10/site-packages (from triton==2.0.0->torch->alpaca-lora-4bit==0.1.3) (16.0.2)
Requirement already satisfied: cmake in /home/ana/.local/lib/python3.10/site-packages (from triton==2.0.0->torch->alpaca-lora-4bit==0.1.3) (3.26.3)
Requirement already satisfied: regex!=2019.12.17 in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2023.3.23)
Requirement already satisfied: tokenizers<0.15,>=0.14 in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (0.14.1)
Requirement already satisfied: requests in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2.31.0)
Requirement already satisfied: multiprocess in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.70.14)
Requirement already satisfied: pyarrow>=6.0.0 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (11.0.0)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (2023.4.0)
Requirement already satisfied: aiohttp in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (3.8.4)
Requirement already satisfied: pandas in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (2.0.3)
Requirement already satisfied: responses<0.19 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.18.0)
Requirement already satisfied: xxhash in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (3.2.0)
Requirement already satisfied: dill<0.3.7,>=0.3.0 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.3.6)
Requirement already satisfied: GitPython!=3.1.29,>=1.0.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (3.1.31)
Requirement already satisfied: appdirs>=1.4.3 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.4.4)
Requirement already satisfied: Click!=8.0.0,>=7.0 in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (8.0.3)
Requirement already satisfied: sentry-sdk>=1.0.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.26.0)
Requirement already satisfied: docker-pycreds>=0.4.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (0.4.0)
Requirement already satisfied: pathtools in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (0.1.2)
Requirement already satisfied: setproctitle in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.2.2)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (59.6.0)
Requirement already satisfied: protobuf!=4.21.0,<5,>=3.19.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (3.20.2)
Requirement already satisfied: six>=1.4.0 in /usr/lib/python3/dist-packages (from docker-pycreds>=0.4.0->wandb->alpaca-lora-4bit==0.1.3) (1.16.0)
Requirement already satisfied: attrs>=17.3.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (23.1.0)
Requirement already satisfied: aiosignal>=1.1.2 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.3.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (6.0.4)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (3.1.0)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (4.0.2)
Requirement already satisfied: frozenlist>=1.1.1 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.3.3)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.9.1)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/ana/.local/lib/python3.10/site-packages (from GitPython!=3.1.29,>=1.0.0->wandb->alpaca-lora-4bit==0.1.3) (4.0.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ana/.local/lib/python3.10/site-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2020.6.20)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (3.3)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ana/.local/lib/python3.10/site-packages (from jinja2->torch->alpaca-lora-4bit==0.1.3) (2.1.3)
Requirement already satisfied: tzdata>=2022.1 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2023.3)
Requirement already satisfied: pytz>=2020.1 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2023.3)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2.8.2)
Requirement already satisfied: mpmath>=0.19 in /home/ana/.local/lib/python3.10/site-packages (from sympy->torch->alpaca-lora-4bit==0.1.3) (1.3.0)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/ana/.local/lib/python3.10/site-packages (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->alpaca-lora-4bit==0.1.3) (5.0.0)
Building wheels for collected packages: alpaca-lora-4bit
Building wheel for alpaca-lora-4bit (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [92 lines of output]
running bdist_wheel
running build
running build_py
package init file 'src/alpaca_lora_4bit/quant_cuda/init.py' not found (or not a regular file)
running build_ext
/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building 'alpaca_lora_4bit.quant_cuda' extension
Emitting ninja build file /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
FAILED: /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o
/usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
/home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
detected during:
instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator==(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]"
(61): here
instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator!=(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]"
/home/ana/.local/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /home/ana/.local/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here
  
  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/usr/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/setup.py", line 18, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 529, in build_extension
      objects = self.compiler.compile(sources,
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for alpaca-lora-4bit
Running setup.py clean for alpaca-lora-4bit
Failed to build alpaca-lora-4bit
Installing collected packages: alpaca-lora-4bit
Running setup.py install for alpaca-lora-4bit ... error
error: subprocess-exited-with-error

× Running setup.py install for alpaca-lora-4bit did not run successfully.
│ exit code: 1
╰─> [125 lines of output]
running install
/usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.10
creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/Finetune4bConfig.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/autograd_4bit.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/model_attn_mlp_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/matmul_utils_4bit.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/arg_parser.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/train_data.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/models.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/custom_autotune.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/triton_utils.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/gradient_checkpointing.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/amp_wrapper.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/llama_flash_attn_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/peft_tuners_lora_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/gptq_for_llala_lora_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/llama_attn_hijack_xformers.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server
copying src/alpaca_lora_4bit/server/server.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server
copying src/alpaca_lora_4bit/server/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server
package init file 'src/alpaca_lora_4bit/quant_cuda/init.py' not found (or not a regular file)
running build_ext
/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building 'alpaca_lora_4bit.quant_cuda' extension
creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10
creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src
creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit
creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda
Emitting ninja build file /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] c++ -MMD -MF /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda.cpp -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
[2/2] /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
FAILED: /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o
/usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
/home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
detected during:
instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator==(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]"
(61): here
instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator!=(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]"
/home/ana/.local/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /home/ana/.local/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here
  
  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/usr/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/setup.py", line 18, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/usr/lib/python3.10/distutils/command/install.py", line 619, in run
      self.run_command('build')
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 529, in build_extension
      objects = self.compiler.compile(sources,
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> alpaca-lora-4bit

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
`

@caterpillarpants
Copy link
Author

And here is my attempt to install the old_compatible.

`ana@durga:/text-generation-webui-main/repositories/alpaca_lora_4bit$ pip uninstall alpaca_lora_4bit
WARNING: Skipping alpaca_lora_4bit as it is not installed.
ana@durga:/text-generation-webui-main/repositories/alpaca_lora_4bit$ git checkout old_compatible
Already on 'old_compatible'
Your branch is up to date with 'origin/old_compatible'.
ana@durga:~/text-generation-webui-main/repositories/alpaca_lora_4bit$ pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit
Preparing metadata (setup.py) ... done
Requirement already satisfied: accelerate>=0.22.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.24.0)
Requirement already satisfied: colorama in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.4.6)
Requirement already satisfied: datasets in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (2.10.1)
Requirement already satisfied: einops in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.6.1)
Requirement already satisfied: packaging in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (23.2)
Requirement already satisfied: peft>=0.5.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.5.0)
Requirement already satisfied: pyzmq in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (25.1.0)
Requirement already satisfied: safetensors in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.3.1)
Requirement already satisfied: sentencepiece in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.1.99)
Requirement already satisfied: torch in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (2.0.1+cu118)
Requirement already satisfied: transformers>=4.31.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (4.35.0.dev0)
Requirement already satisfied: wandb in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.15.4)
Requirement already satisfied: numpy>=1.17 in /home/ana/.local/lib/python3.10/site-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (1.24.0)
Requirement already satisfied: psutil in /usr/lib/python3/dist-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (5.9.0)
Requirement already satisfied: huggingface-hub in /home/ana/.local/lib/python3.10/site-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (0.17.3)
Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (5.4.1)
Requirement already satisfied: tqdm in /home/ana/.local/lib/python3.10/site-packages (from peft>=0.5.0->alpaca-lora-4bit==0.1.3) (4.65.0)
Requirement already satisfied: jinja2 in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.1.2)
Requirement already satisfied: triton==2.0.0 in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (2.0.0)
Requirement already satisfied: filelock in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.12.4)
Requirement already satisfied: sympy in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (1.11.1)
Requirement already satisfied: networkx in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.1)
Requirement already satisfied: typing-extensions in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (4.5.0)
Requirement already satisfied: lit in /home/ana/.local/lib/python3.10/site-packages (from triton==2.0.0->torch->alpaca-lora-4bit==0.1.3) (16.0.2)
Requirement already satisfied: cmake in /home/ana/.local/lib/python3.10/site-packages (from triton==2.0.0->torch->alpaca-lora-4bit==0.1.3) (3.26.3)
Requirement already satisfied: tokenizers<0.15,>=0.14 in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (0.14.1)
Requirement already satisfied: regex!=2019.12.17 in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2023.3.23)
Requirement already satisfied: requests in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2.31.0)
Requirement already satisfied: pandas in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (2.0.3)
Requirement already satisfied: xxhash in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (3.2.0)
Requirement already satisfied: responses<0.19 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.18.0)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (2023.4.0)
Requirement already satisfied: dill<0.3.7,>=0.3.0 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.3.6)
Requirement already satisfied: multiprocess in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.70.14)
Requirement already satisfied: pyarrow>=6.0.0 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (11.0.0)
Requirement already satisfied: aiohttp in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (3.8.4)
Requirement already satisfied: sentry-sdk>=1.0.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.26.0)
Requirement already satisfied: docker-pycreds>=0.4.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (0.4.0)
Requirement already satisfied: pathtools in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (0.1.2)
Requirement already satisfied: Click!=8.0.0,>=7.0 in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (8.0.3)
Requirement already satisfied: protobuf!=4.21.0,<5,>=3.19.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (3.20.2)
Requirement already satisfied: GitPython!=3.1.29,>=1.0.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (3.1.31)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (59.6.0)
Requirement already satisfied: appdirs>=1.4.3 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.4.4)
Requirement already satisfied: setproctitle in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.2.2)
Requirement already satisfied: six>=1.4.0 in /usr/lib/python3/dist-packages (from docker-pycreds>=0.4.0->wandb->alpaca-lora-4bit==0.1.3) (1.16.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (6.0.4)
Requirement already satisfied: aiosignal>=1.1.2 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.3.1)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (4.0.2)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.9.1)
Requirement already satisfied: frozenlist>=1.1.1 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.3.3)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (3.1.0)
Requirement already satisfied: attrs>=17.3.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (23.1.0)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/ana/.local/lib/python3.10/site-packages (from GitPython!=3.1.29,>=1.0.0->wandb->alpaca-lora-4bit==0.1.3) (4.0.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ana/.local/lib/python3.10/site-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (1.26.16)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2020.6.20)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ana/.local/lib/python3.10/site-packages (from jinja2->torch->alpaca-lora-4bit==0.1.3) (2.1.3)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2.8.2)
Requirement already satisfied: tzdata>=2022.1 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2023.3)
Requirement already satisfied: pytz>=2020.1 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2023.3)
Requirement already satisfied: mpmath>=0.19 in /home/ana/.local/lib/python3.10/site-packages (from sympy->torch->alpaca-lora-4bit==0.1.3) (1.3.0)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/ana/.local/lib/python3.10/site-packages (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->alpaca-lora-4bit==0.1.3) (5.0.0)
Building wheels for collected packages: alpaca-lora-4bit
Building wheel for alpaca-lora-4bit (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [93 lines of output]
running bdist_wheel
running build
running build_py
package init file 'src/alpaca_lora_4bit/quant_cuda/init.py' not found (or not a regular file)
running build_ext
/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building 'alpaca_lora_4bit.quant_cuda' extension
Emitting ninja build file /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] c++ -MMD -MF /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda.cpp -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
[2/2] /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
FAILED: /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o
/usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
/home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
detected during:
instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator==(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]"
(61): here
instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator!=(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]"
/home/ana/.local/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /home/ana/.local/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here
  
  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/usr/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/setup.py", line 18, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 529, in build_extension
      objects = self.compiler.compile(sources,
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for alpaca-lora-4bit
Running setup.py clean for alpaca-lora-4bit
Failed to build alpaca-lora-4bit
Installing collected packages: alpaca-lora-4bit
Running setup.py install for alpaca-lora-4bit ... error
error: subprocess-exited-with-error

× Running setup.py install for alpaca-lora-4bit did not run successfully.
│ exit code: 1
╰─> [125 lines of output]
running install
/usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.10
creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/Finetune4bConfig.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/autograd_4bit.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/model_attn_mlp_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/matmul_utils_4bit.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/arg_parser.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/train_data.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/models.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/custom_autotune.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/triton_utils.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/gradient_checkpointing.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
copying src/alpaca_lora_4bit/amp_wrapper.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit
creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/llama_flash_attn_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/peft_tuners_lora_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/gptq_for_llala_lora_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
copying src/alpaca_lora_4bit/monkeypatch/llama_attn_hijack_xformers.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch
creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server
copying src/alpaca_lora_4bit/server/server.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server
copying src/alpaca_lora_4bit/server/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server
package init file 'src/alpaca_lora_4bit/quant_cuda/init.py' not found (or not a regular file)
running build_ext
/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building 'alpaca_lora_4bit.quant_cuda' extension
creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10
creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src
creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit
creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda
Emitting ninja build file /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] c++ -MMD -MF /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda.cpp -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
[2/2] /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"' '-DPYBIND11_STDLIB="libstdcpp"' '-DPYBIND11_BUILD_ABI="cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
FAILED: /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o
/usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17
/home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
detected during:
instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator==(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]"
(61): here
instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator!=(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]"
/home/ana/.local/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /home/ana/.local/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here
  
  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/usr/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/setup.py", line 18, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/usr/lib/python3.10/distutils/command/install.py", line 619, in run
      self.run_command('build')
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 529, in build_extension
      objects = self.compiler.compile(sources,
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> alpaca-lora-4bit

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
`

@caterpillarpants
Copy link
Author

caterpillarpants commented Oct 27, 2023 •
edited
Loading

I guess I cannot find a .log file. so I hope the above is useful. If there is a specific log file somewhere please tell me where it is and I can post it for you.

I have to admit. I'm kinda giddy getting tech support from a guy who made 4bit training possible for llm's. you are pretty awesome, sir.

@johnsmith0031
Copy link
Owner

I found the similar issue here...
NVIDIA/nccl#650
Maybe the cause is the compatibility of cuda and std c++

@caterpillarpants
Copy link
Author

Well, I am running gcc 12. I installed gcc 10 via sudo apt install gcc-10, but I don't think it's default or active. sudo update-alternatives isn't working initially. Frankly, I'm a bit over my head. I have to go to bed soon, so I'll have to take this up tomorrow. I hope to try the bisqwit solution. So update tomorrow!

@caterpillarpants
Copy link
Author

well, I have nothing to update. I am now on gcc 10 and it's not working. I also have nvcc in the path, but it still can't find it.

here is my current error:
` error: [Errno 2] No such file or directory: '/home/ana/text-generation-webui-main/installer_files/env/bin/nvcc'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for alpaca-lora-4bit
Running setup.py clean for alpaca-lora-4bit
Failed to build alpaca-lora-4bit
ERROR: Could not build wheels for alpaca-lora-4bit, which is required to install pyproject.toml-based projects
`

I had a similar problem trying to deep deepspeed to compile. My nvcc was at /usr/bin and pip or alpacalora wasn't looking there, so I put it in the path with : export PATH=/usr/bin:$PATH
this fixed it for deepspeed, but isn't working here.

@johnsmith0031
Copy link
Owner

try putting this after the activation of virtual env? The virtual env would overwrite the environment variables.

@caterpillarpants
Copy link
Author

OK, so I've solved this nvcc issue, but have a new error. I am using text-generation-webui which has a new cmd_linux.sh command which starts a terminal and the I thought we were to do our text-gen terminal stuff with this.I assumed it started the environment, but really I have no idea. Anyway...

If I compile while using the cmd_linus.sh shell, I get the nvcc error.

If I compile without the cmd_linux.sh.shell, that it to say, in the normal terminal, I do not get the nvcc error. Instead I get these errors:
Building wheel for alpaca-lora-4bit (setup.py) ... error

File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error compiling objects for extension

File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error compiling objects for extension

There is additional output, but I didn't want to clutter the post. if you want a verbose output I can provide it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@johnsmith0031 @caterpillarpants