LLMs for Tourism Recommendation Experiments

LLM4TRS is an experimental setup for evaluating LLMs recommendation and inference abilities with tourism data.

In this setting, we explore which data input and information can be leveraged by a model to improve performance on the task of rating prediction. We look at the following scenarios

• Zero-shot using only user names and item titles
• Few-Shot using a user's rating and review history
• Content-based using information from the dataset, such as item name, location, amenitities
• Collaborative using information on similar users or friends of a user
• Category-based using categories from the Yelp platform

Usage

Download the Yelp dataset

Download the Yelp dataset, unzip it and move the JSON files to data/raw_data/.

Install required packages

pip install -r requirements.txt

Download the models

The models are pulled from the ollama library and run locally. They must be downloaded before the program execution. We used the following models

• gemma2:9b
• gemma2:27b
• zephyr
• llama3.1:8b
• llama3.1:70b

Preprocessing

Run the preprocessing script that aggregates the users data, their restaurant reviews and restaurant information.

python3 preprocess.py

Run Evaluations:

In order to run the experiments on the evaluation set, use the following command:

python3 run.py -m gemma2 -all

The results of each run is logged in evaluation.log.

License

MIT