All Datasets

The Benchmark of Linguistic Minimal Pairs, a challenge set for evaluating the linguistic knowledge of language models (LMs) on major grammatical phenomena in English, finds that state-of-the-art models identify morphological contrasts related to agreement reliably, but they struggle with some subtle semantic and syntactic phenomena.

TAL-SCQ5K are high-quality mathematical competition datasets created by TAL Education Group.

To create these datasets, the authors automatically translated the original CSQA and CODAH datasets, originally available only in English, into 15 other languages.

A native-Korean NLU benchmark covering eight tasks — topic classification, semantic similarity, NLI, NER, relation extraction, dependency parsing, machine reading comprehension, and dialogue state tracking. Over 200,000 labeled examples built from real Korean corpora rather than machine translation.

The DOCCI dataset consists of comprehensive descriptions on 15k images specifically taken with the objective of evaluating T2I and I2T models. These cover a lot of key details in the images, as illustrated below.

The ARC dataset consists of 7,787 science exam questions drawn from a variety of sources, including science questions, provided under license by a research partner affiliated with AI2.

A bilingual (English + Chinese) benchmark for long-context understanding, with 21 tasks across document QA, summarization, few-shot learning, synthetic reasoning, and code completion. 4,750 test samples averaging 5k–15k tokens, scored with a fully automated evaluation.

A multilingual spoken-language dataset for intent detection in e-banking, covering 14 intents across 14 language varieties. 8,168 labeled audio clips at 8 kHz, each paired with a transcription and English translation.

This is the repository for PLOD Dataset subset being used for CW in NLP module 2023-2024 at University of Surrey.

A collaborative benchmark of 204 tasks built to probe large language models beyond single-metric tests, spanning reasoning, knowledge, creativity, and multilingual ability. Contributed by 450 authors across 132 institutions, with canary strings to detect training-data contamination.

A multiple-choice benchmark measuring a model's knowledge across 57 subjects in the humanities, social sciences, STEM, and professional domains. Roughly 16,000 exam-sourced questions in a consistent four-option format, the field's common reference for general-knowledge evaluation.

NIH Chest X-Ray is a large dataset containing chest X-ray images of patients collected by the National Institutes of Health (NIH) of the United States.

MNIST is used to train and evaluate image classification models in complex tasks.