---

HomePage |

Introduction

FairDiverse is a unified, comprehensive and efficient benchmark toolkit for fairnes-aware and diversity-aware IR models. It aims to help the researchers to reproduce and develop IR models.

In the lastest release, our library includes 30+ algorithms covering four major categories:

• Pre-processing models

• In-processing models

• Post-processing models

• IR base models

We design a unified pipelines.

For the usage, we use following steps:

The utilized parameters in each config files can be found in following docs:

Parameter Descriptions

Recommendation Parameters

• Parameter settings data pre-processing
  • Required parameters
• Parameter settings for evaluations
  • Evaluation required parameters
• Parameter settings for your new dataset
  • Required parameters
• Parameter settings for in-processing models
  • In-processing required parameters
• Parameter settings for post-processing models
  • Post-processing required parameters

Search Parameters

• Parameter settings for dataset
  • Dataset
  • Column definition
  • Train-Test split
• Parameter settings data pre-processing
  • Required parameters
• Parameter settings for pre-processing models
  • Pre-processing required parameters
  • Ranklib parameters
  • Evaluation parameters
• Parameter settings for post-processing models
  • Post-processing required parameters
  • Supervised and unsupervised model train parameters
  • LLMs-based model parameters
  • Evaluation parameters
  • Log set ups

Custom Your Models (APIs)

For the develop your own recommendation model, you can use following steps:

Recommendation develop APIs

• reranker
  • RecReRanker
• trainer
  • RecTrainer
• Abstract_Ranker
  • Abstract_Regularizer
  • Abstract_Reweigher
  • Abstract_Sampler
• Abstract_Reranker
  • Abstract_Reranker

Recommendation other APIs

• recommendation.evaluator
  • Abstract_Evaluator
  • CTR_Evaluator
  • LLM_Evaluator
  • Ranking_Evaluator
• recommendation.llm_rec
  • LLMRecommender
• recommendation.metric
  • AUC_score()
  • Entropy()
  • Gini()
  • HR()
  • MMF()
  • MRR()
  • MinMaxRatio()
  • NDCG()
  • dcg()
  • mask_utility()
  • reconstruct_utility()
• recommendation.process_data
• sampler
  • AbstractDataset
  • PairWiseDataset
  • PointWiseDataset
  • RankingTestDataset
  • SequentialDataset
• group_utils
  • Build_Adjecent_Matrix()
  • Init_Group_AdjcentMatrix()
  • convert_keys_values_to_int()
  • get_cos_similar_torch()
  • get_iid2text()
  • load_json()

Search develop APIs

• trainer_preprocessing_ranker
  • RankerTrainer
    • RankerTrainer.load_configs()
    • RankerTrainer.train()
• fair_model
  • PreprocessingFairnessIntervention
    • PreprocessingFairnessIntervention.fit()
    • PreprocessingFairnessIntervention.fit_transform()
    • PreprocessingFairnessIntervention.transform()
• postprocessing_model.base

Search other APIs

• fairness_evaluator
  • compute_cumulative_exposer()
  • compute_igf_ratio()
  • compute_individual_fairness()
  • evaluate()
  • evaluate_qid()
  • evaluate_runs()
  • get_quotas_count()
  • get_sort_df()
• utils
  • compute_dynamic_values_to_code()
  • load_model_data()
  • process_data_input()
  • process_data_output()
  • save_model_data()
• ranker
  • Ranker
    • Ranker.evaluate()
    • Ranker.evaluate_run()
    • Ranker.predict()
    • Ranker.train()
• loss
  • list_pairwise_loss()
  • ndcg_loss()
• process_dataset
  • calculate_best_rank()
  • data_process()
  • data_process_worker()
  • generate_qd()
  • get_doc_judge()
  • get_docs_dict()
  • get_query_dict()
  • get_query_suggestion()
  • get_stand_best_metric()
• utils.utils
  • get_metrics_20()
  • get_rel_feat()
  • load_embedding()
  • pkl_load()
  • pkl_save()
  • read_rel_feat()
  • remove_duplicate()
  • restore_doc_ids()
  • split_list()
• api_llm
  • ApiException
  • ApiProxy
  • LMAgent
• div_type
  • div_dataset
    • div_dataset.get_listpair_train_data()
    • div_dataset.get_listpairs()
  • div_query
    • div_query.add_docs()
    • div_query.add_docs_rel_score()
    • div_query.add_query_suggestion()
    • div_query.get_alpha_DCG()
    • div_query.get_best_rank()
    • div_query.get_test_alpha_nDCG()
    • div_query.set_std_metric()
  • subtopic

The Team

FairDiverse is developed and maintained by RUC, UvA.

Here is the list of our lead developers in each development phase.

Time	Version	Lead Developers
Nov. 2024 ~ Feb. 2025	v0.0.1	Chen Xu, Zhirui Deng, Clara Rus

License

FairDiverse uses MIT License.