Automotive Cybersecurity QA and RAG Evaluation Dataset

Automotive Cybersecurity QA and RAG Evaluation Dataset This dataset provides a collection of human-curated question–answer pairs in the automotive cybersecurity domain, together with model-generated answers obtained using multiple retrieval-augmented generation (RAG) configurations. It is designed to support research on domain-specific question answering, retrieval-augmented generation, and the evaluation of large language models in automotive cybersecurity. Additional details on the RAG configurations, retrieval strategies, and evaluation methodology are described in the associated publication. The dataset includes: A set of human-written question–answer pairs created by domain experts, serving as ground truth. Model-generated answers produced under four different RAG configurations: ensemble, semantic, syntactic, and two-stage. Dataset Structure human_answers/: Contains human-written reference answers. answers_rag/: Contains model-generated answers, organized by RAG configuration (ensemble, semantic, syntactic, two_stage). Each folder contains CSV files corresponding to different evaluated language models. Data Format All files follow the same schema: q_id: Unique identifier of the question question: The natural language question answer: The corresponding answer (human or model-generated depending on the file) Purpose This dataset can be used to: Benchmark large language models on domain-specific question answering Evaluate and compare different RAG strategies Analyze and compare human-written and model-generated answers Support research on LLM applications in automotive cybersecurity

Перевод пока недоступен