A Summary of Our Paper Dr.Spider

This blog post provides a summary of our paper “Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness,”.

Paper Overview

Dr.Spider is a comprehensive diagnostic evaluation benchmark designed to assess the robustness of Text-to-SQL models. The benchmark includes various perturbation types that test different aspects of model robustness, helping researchers understand where their models succeed and where they fail.

Read the Full Post

For a detailed explanation of our work, including methodology, results, and insights, please read the full blog post on Medium:

Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness

Resources

• Paper: arXiv
• Data: GitHub Repository
• Slides: Presentation Slides
• Video: ICLR 2023 Presentation

Citation

If you find our work useful, please consider citing:

@inproceedings{chang2023drspider,
  title={Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness},
  author={Chang, Shuaichen and Wang, Jun and Dong, Mingwen and Pan, Lin and Zhu, Henghui and Li, Alexander Hanbo and Lan, Wuwei and Zhang, Sheng and Jiang, Jiarong and Lilien, Joseph and others},
  booktitle={International Conference on Learning Representations},
  year={2023}
}