publications

2024

1. Clinical WS

   ProbGate at EHRSQL 2024: Enhancing SQL Query Generation Accuracy through Probabilistic Threshold Filtering and Error Handling

   Sangryul Kim, Donghee Han, and Sehyun Kim

   In Proceedings of the 6th Clinical Natural Language Processing Workshop , Jun 2024

   Abs Bib HTML Poster Slides

   Recently, deep learning-based language models have significantly enhanced text-to-SQL tasks, with promising applications in retrieving patient records within the medical domain. One notable challenge in such applications is discerning unanswerable queries. Through fine-tuning model, we demonstrate the feasibility of converting medical record inquiries into SQL queries. Additionally, we introduce an entropy-based method to identify and filter out unanswerable results. We further enhance result quality by filtering low-confidence SQL through log probability-based distribution, while grammatical and schema errors are mitigated by executing queries on the actual database.We experimentally verified that our method can filter unanswerable questions, which can be widely utilized even when the parameters of the model are not accessible, and that it can be effectively utilized in practice.

   @inproceedings{kim-etal-2024-probgate,
     title = {{P}rob{G}ate at {EHRSQL} 2024: Enhancing {SQL} Query Generation Accuracy through Probabilistic Threshold Filtering and Error Handling},
     author = {Kim, Sangryul and Han, Donghee and Kim, Sehyun},
     editor = {Naumann, Tristan and Ben Abacha, Asma and Bethard, Steven and Roberts, Kirk and Bitterman, Danielle},
     booktitle = {Proceedings of the 6th Clinical Natural Language Processing Workshop},
     month = jun,
     year = {2024},
     address = {Mexico City, Mexico},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2024.clinicalnlp-1.65},
     pages = {687--696},
   }

2. Arxiv

   eXplainable Bayesian Multi-Perspective Generative Retrieval

   EuiYul Song, Philhoon Oh, Sangryul Kim , and 1 more author

   Jun 2024

   arXiv Bib

   @misc{song2024explainablebayesianmultiperspectivegenerative,
     title = {eXplainable Bayesian Multi-Perspective Generative Retrieval},
     author = {Song, EuiYul and Oh, Philhoon and Kim, Sangryul and Thorne, James},
     year = {2024},
     eprint = {2402.02418},
     archiveprefix = {arXiv},
     primaryclass = {cs.IR},
     url = {https://arxiv.org/abs/2402.02418}
   }

3. EACL

   Re3val: Reinforced and Reranked Generative Retrieval

   EuiYul Song, Sangryul Kim, Haeju Lee , and 2 more authors

   In Findings of the Association for Computational Linguistics: EACL 2024 , Mar 2024

   Abs Bib HTML

   Generative retrieval models encode pointers to information in a corpus as an index within the model’s parameters. These models serve as part of a larger pipeline, where retrieved information conditions generation for knowledge-intensive NLP tasks. However, we identify two limitations: the generative retrieval does not account for contextual information. Secondly, the retrieval can’t be tuned for the downstream readers as decoding the page title is a non-differentiable operation. This paper introduces Re3val, trained with generative reranking and reinforcement learning using limited data. Re3val leverages context acquired via Dense Passage Retrieval to rerank the retrieved page titles and utilizes REINFORCE to maximize rewards generated by constrained decoding. Additionally, we generate questions from our pre-training dataset to mitigate epistemic uncertainty and bridge the domain gap between the pre-training and fine-tuning datasets. Subsequently, we extract and rerank contexts from the KILT database using the rerank page titles. Upon grounding the top five reranked contexts, Re3val demonstrates the Top 1 KILT scores compared to all other generative retrieval models across five KILT datasets.

   @inproceedings{song-etal-2024-re3val,
     title = {Re3val: Reinforced and Reranked Generative Retrieval},
     author = {Song, EuiYul and Kim, Sangryul and Lee, Haeju and Kim, Joonkee and Thorne, James},
     editor = {Graham, Yvette and Purver, Matthew},
     booktitle = {Findings of the Association for Computational Linguistics: EACL 2024},
     month = mar,
     year = {2024},
     address = {St. Julian{'}s, Malta},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2024.findings-eacl.27},
     pages = {393--409},
   }