Parameter settings for post-processing models¶
(Default values are in ~/search/Post-processing.yaml)
The benchmark provides several arguments for describing:
Basic setting of the parameters
See below for the details:
Post-processing required parameters¶
model (str)
: The name of the pre-processing model, which automatically loads the configs from search/properties/models/<model_name>.yaml.model_save_dir (str)
: The save path of model.tmp_dir (str)
: The temperature path of files.mode (str)
: Train or test mode.best_model_list (list)
: The best model path list for test mode.
Supervised and unsupervised model train parameters¶
device (str)
: used device (cpu or gpu) to run the codes.epoch (int)
: training epochs.batch_size (int)
: training batch size.learning_rate (float)
: learning rate of optimizing the models.dropout (float)
: The dropout for training.loss (str)
: The loss type for training.lambda (float)
: The hyperparameter lambda in PM2 and xQuAD for balance relevance and diversity.
LLMs-based model parameters¶
prompts_dir (str)
: The path of LLMs-based model’s prompts, default as promptsllm_rerank.txt.model_name (str)
: The name of LLMs-based backbone model, default as gpt-4o.api_url (str)
: The api url of LLM API.api_key (str)
: The api key of LLM API.max_new_tokens (int)
: The max new tokens for LLMs-based model, default as 1024.temperature (float)
: The temperature for LLM generation.top_p (float)
: The top_p for LLM generation.
Evaluation parameters¶
eval_step (int)
: How many epochs apart should the model be evaluated on the validation set.eval_batch_size (int)
: The batch size used to conduct evaluation.
Log set ups¶
log_name (str)
: The running log name, which will create a new dictionary ~searchloglog_nameand store the test results and all parameters into it.