fairness_evaluator

fairdiverse.search.fairness_evaluator.compute_cumulative_exposer(df_top, ki)

Compute the cumulative exposure for the top-k items in the ranking.

:param df_toppandas.DataFrame

The top-k items in the ranking.

:param kiint

The number of top-k items.

:returnfloat

The cumulative exposure score.

fairdiverse.search.fairness_evaluator.compute_igf_ratio(top_k_IDS, _orig_df, _orig_sort_col)

Compute the IGF (Item Group Fairness) ratio for the top-k items.

:param top_k_IDSlist

A list of IDs representing the top-k items.

:param _orig_dfpandas.DataFrame

The original DataFrame containing all items.

:param _orig_sort_colstr

The column name to sort the original DataFrame by.

:returnfloat

The IGF ratio.

fairdiverse.search.fairness_evaluator.compute_individual_fairness(data, ranking)

Compute the individual fairness score for the ranking.

:param datapandas.DataFrame

The data to evaluate fairness.

:param rankingstr

The ranking column to use for evaluation.

:returnfloat

The individual fairness score between 0 and 1.

fairdiverse.search.fairness_evaluator.evaluate(data, query_col, s_attribute, eval_path, args_eval)

Evaluate the given data by calculating metrics for each query and sensitive attribute.

:param datapandas.DataFrame

The data containing the query identifiers and sensitive attribute.

:param query_colstr

The column name for the query identifiers in the data.

:param s_attributestr

The sensitive attribute column name.

:param eval_pathstr

Path where evaluation results will be saved.

:param args_evaldict

A dictionary containing evaluation configurations, including metrics and other settings.

:returnNone

This function saves the evaluation results as CSV files.

fairdiverse.search.fairness_evaluator.evaluate_qid(df, eval_measure, s_attribute, sensitive_groups, args_eval)

Evaluate the data for a single query ID, calculating the specified metrics.

:param dfpandas.DataFrame

The data for a single query ID.

:param eval_measurestr

The evaluation metric to calculate.

:param s_attributestr

The sensitive attribute column name.

:param sensitive_groupslist

A list of sensitive attribute groups.

:param args_evaldict

A dictionary containing evaluation configurations, including rankings and k list.

:returnpandas.DataFrame

A DataFrame containing the evaluation results.

fairdiverse.search.fairness_evaluator.evaluate_runs(args_eval, qids, eval_path, runs, split)

Evaluate the runs by calculating metrics and saving the results in CSV files.

:param args_evaldict

A dictionary containing evaluation configurations, including metrics and other settings.

:param qidslist

A list of unique query identifiers (QIDs) to evaluate.

:param eval_pathstr

Path where evaluation results will be saved.

:param runsint

Number of runs to evaluate.

:param splitstr

The data split to evaluate (e.g., “train”, “test”).

:returnNone

This function saves the evaluation results as CSV files.

fairdiverse.search.fairness_evaluator.get_quotas_count(_df, s_attribute, sensitive_groups)

Calculate the quota count for each sensitive group in the data.

:param _dfpandas.DataFrame

The data to calculate the quotas for.

:param s_attributestr

The sensitive attribute column name.

:param sensitive_groupslist

A list of sensitive attribute groups.

:returndict

A dictionary with the sensitive group names as keys and their corresponding quota counts as values.

fairdiverse.search.fairness_evaluator.get_sort_df(_sort_col, _df, _k)

Sort the DataFrame by the specified column and return the top-k rows.

:param _sort_colstr

The column to sort the data by.

:param _dfpandas.DataFrame

The DataFrame to be sorted.

:param _kint

The number of top items to return.

:returnpandas.DataFrame

The top-k sorted rows from the DataFrame.