OpenAI has officially launched IndQA, a new multilingual and culture-sensitive benchmark designed to evaluate how effectively AI models can understand and reason through questions grounded in Indian languages and cultural contexts. Released on November 4, 2025, this initiative marks OpenAI’s first major region-specific benchmark, focusing on linguistic diversity, cultural nuances, and contextual intelligence in India — the company’s second-largest user market for ChatGPT.
What Is IndQA?
IndQA stands for Indian Question-Answering benchmark. It currently features 2,278 questions covering 11 Indian languages,
- Hindi, Hinglish, Gujarati, Punjabi, Kannada, Odia, Marathi, Malayalam, Tamil, Bengali, and Telugu
The benchmark spans 10 cultural domains,
- Law and Ethics
- Architecture and Design
- Food and Cuisine
- Everyday Life
- Religion and Spirituality
- Sports and Recreation
- Literature and Linguistics
- Media and Entertainment
- Arts and Culture
- History
It was developed with the input of 261 domain experts, including scholars, journalists, linguists, artists, and subject specialists.
How Does IndQA Work?
- The evaluation process is built around a rubric-based grading system, where each AI-generated response is scored against predefined criteria crafted by experts for each question.
- Each criterion is assigned weighted points based on its relevance and importance.
- A model-based grader checks responses against these criteria, and the final score is calculated accordingly.
- All questions were tested with OpenAI’s most powerful models, including GPT-4o, GPT-4.5, GPT-5, and OpenAI o3 during creation to ensure adversarial robustness.
Benchmark Performance: AI Models Compared
Initial benchmarking results based on IndQA showed significant variance among leading models,
- GPT-5 (Thinking High): 34.9% (Highest overall)
- Gemini 2.5 Pro Thinking: 34.3%
- Gemini 2.5 Flash Thinking: 29.7%
- Grok 4: 28.5%
- OpenAI o3 High: 28.1%
- GPT-4o: 20.3%
- GPT-4 Turbo: 12.1%
Language-wise observations
- Highest performance was seen in Hindi and Hinglish, where GPT-5 scored around 45% and 44% respectively.
- Lowest performance was observed in Bengali and Telugu, revealing gaps in existing AI language models for these scripts.
- OpenAI clarified that IndQA is not a cross-language leaderboard, since the questions differ across languages. Instead, it serves as a within-model benchmark to measure progress over time.
ShivamAs a Content Executive Writer at Adda247, I am dedicated to helping students stay ahead in their competitive exam preparation by providing clear, engaging, and insightful coverage of both major and minor current affairs. With a keen focus on trends and developments that can be crucial for exams, researches and presents daily news in a way that equips aspirants with the knowledge and confidence they need to excel. Through well-crafted content, Its my duty to ensures that learners remain informed, prepared, and ready to tackle any current affairs-related questions in their exams.