machine-learning/llama-cpp

LLM inference in C/C++.

Versions

Slot Version Repository Platforms
0 scm tombriden ~amd64 arm? armv7? armv8? x86?

Metadata

Homepage https://github.com/ggml-org/llama.cpp
Summary LLM inference in C/C++
Dependencies (
    (
        build:
        vulkan? (
            sys-libs/shaderc
            sys-libs/spirv-headers
            sys-libs/vulkan-headers
        )
        build+run:
        providers:libressl? (
            dev-libs/libressl:=
        )
        providers:openssl? (
            dev-libs/openssl:=
        )
        cuda? (
            dev-util/nvidia-cuda-toolkit
        )
        openmp? (
            sys-libs/libgomp:=
        )
        vulkan? (
            sys-libs/vulkan-loader
        )
    )
    (
        (
            build+fetch:
            sys-apps/util-linux[>=2.13_pre2]
            dev-scm/git[>=1.6.0][curl]
            net-misc/curl[ssl(+)]
        )
    )
    (
        build:
        sys-devel/cmake[>=3.29.0]
    )
)
Downloads
Licences MIT

Choices

OPTIONS

cuda Enable the CUDA backend for NVIDIA GPUs
openmp Multi-threading via OpenMP
vulkan Enable the Vulkan backend for GPU acceleration

providers

libressl Use LibreSSL as the SSL provider
openssl Use OpenSSL as the SSL provider