Parameter settings for post-processing models¶
(Default values are in ~/search/Post-processing.yaml)
The benchmark provides several arguments for describing:
Basic setting of the parameters
See below for the details:
Post-processing required parameters¶
model (str): The name of the pre-processing model, which automatically loads the configs from search/properties/models/<model_name>.yaml.model_save_dir (str): The save path of model.tmp_dir (str): The temperature path of files.mode (str): Train or test mode.best_model_list (list): The best model path list for test mode.
Supervised and unsupervised model train parameters¶
device (str): used device (cpu or gpu) to run the codes.epoch (int): training epochs.batch_size (int): training batch size.learning_rate (float): learning rate of optimizing the models.dropout (float): The dropout for training.loss (str): The loss type for training.lambda (float): The hyperparameter lambda in PM2 and xQuAD for balance relevance and diversity.
LLMs-based model parameters¶
prompts_dir (str): The path of LLMs-based model’s prompts, default as promptsllm_rerank.txt.model_name (str): The name of LLMs-based backbone model, default as gpt-4o.api_url (str): The api url of LLM API.api_key (str): The api key of LLM API.max_new_tokens (int): The max new tokens for LLMs-based model, default as 1024.temperature (float): The temperature for LLM generation.top_p (float): The top_p for LLM generation.
Evaluation parameters¶
eval_step (int): How many epochs apart should the model be evaluated on the validation set.eval_batch_size (int): The batch size used to conduct evaluation.
Log set ups¶
log_name (str): The running log name, which will create a new dictionary ~searchloglog_nameand store the test results and all parameters into it.