Feature Request: Option to specify base URL for server mode

[vlasky]

I've been testing the use of Nginx as a proxy to serve llamafile under a subdirectory.

i.e. to be able to access the llamafile server via a URL like this:

https://mydomain.com/llamafile/

Llamafile is being launched via systemd under Linux.

I am using a rewrite rule in Nginx to remove the llamafile/ subdirectory in the URL before passing the request to llamafile.

Although I am successfully able to open the index page, it doesn't load correctly because the HTML contains Javascript code that imports the following scripts from the following absolute URLs:

/index.js
/completion.js
/json-schema-to-grammar.mjs

This is a result of the following HTML lines in the index page:

<script type="module">
    import {
      html, h, signal, effect, computed, render, useSignal, useEffect, useRef, Component
    } from '/index.js';

    import { llama } from '/completion.js';
    import { SchemaConverter } from '/json-schema-to-grammar.mjs';

At the moment, llamafile does not have a command line option to specify a base URL or URL subdirectory prefix.

The --path command line option cannot be used for this purpose.

It would be great if a base URL/URL subdirectory prefix command line option could be added to llamafile.

[clyfish]

ggml-org/llama.cpp@0642b22#diff-250e89dd8f0b29364f4245b54b512cde3e6d898245953aa7d7da6e61708bde21
@mofosyne We can sync with llama.cpp upstream to fix this issue.

[clyfish]

@vlasky Before sync with llama.cpp, we can modify the frontend files and build llamafile ourselves.

sed -i "s# from '/# from './#g" llama.cpp/server/public/index.html
sed -i 's# fetch("/# fetch("./#g' llama.cpp/server/public/completion.js

[vlasky]

I have submitted a PR for this feature:

#597

[vlasky]

I have submitted a PR for this same functionality in llamafiler as well:

#604

[vlasky]

These enhancements have been released in llamafile v0.8.15