semantic_compare

record_evaluation(result_df, query_id, result, message)

Updates the result DataFrame with the evaluation outcome.

Parameters:

  • result_df (pandas.DataFrame) – DataFrame containing query results.

  • query_id (str) – Identifier of the query being evaluated.

  • result (str) – The evaluation result (“TRUE”, “FALSE”, or “UNDETERMINED”).

  • message (str) – A message explaining the evaluation result.

Returns:

The updated result DataFrame.

Return type:

pandas.DataFrame

compare_gold_to_generated(gold, generated, database_name, db_type='ms-sql', db_list_file='./local/db_info.json')

Compares gold standard and generated SQL queries for semantic equivalence.

Parameters:

  • gold (str) – The gold standard SQL query.

  • generated (str) – The generated SQL query.

  • database_name (str) – The name of the database to query.

  • db_type (str, optional) – The type of the database (“ms-sql”, “sqlite”, or “tsql”).

  • db_list_file (str, optional) – Path to the database connection information JSON file.

Returns:

A dictionary containing the equivalence result and reason.

Return type:

dict

do_batch_compare(database_name, result_file)

Performs a batch comparison of generated SQL queries against gold standards.

Parameters:

  • database_name (str) – The name of the database to query.

  • result_file (str) – Path to the Excel file containing queries and results.