The LLM for our analysis was run over resources provided by Kisski, part of Gwdg Göttingen. We utilize seeds to ensure replicability.
Note that the seed feature is stil in its beta. For more information on seeds, see OpenAI's documentation.
To reproduce our results:
-
Request an API key from Kisski here by clicking on "Buchen" (Booking).
-
You will be redirected to an AcademicID login. If you are a student or researcher from a German University you will be able to log in with your university credentials or create a new account and aqcuire the API key.
-
Add you API key to the script:
LLM/api_predict_num_statements.pyorLLM/api_predict_statement_spans.py, depending on the prompt you want.
from openai import OpenAI
import pandas as pd
# API configuration
api_key = "YOUR_API_KEY" # Replace with your own API key- Execute the script and check your results!
We also experimented with Llama for subcases from the annotation guidelines with manual prompting over Huggingchat. You can find the prompts used and an example answer in LLM/prompts_subtasks.md