Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark

Can MLLMs effectively serve as judges in the multimodal domain, and how closely do their evaluations align with human preferences?

1. Contributions

1. Benchmark: comprehensive benchmark MLLM-AS-A-JUDGE in multimodal domains, with human annotations

2. Two dataset: curate two human preference datasets with high-quality questions MLLM-AS-A-JUDGE-HQ and MLLM-AS-A-JUDGE-HARD dataset with hallucination instances.

   1. MLLM-AS-A-JUDGE-HQ: which showcases responses with a high level of concordance with human judgments

   2. MLLM-AS-A-JUDGE-HARD: which includes responses marked by inconsistency with human preferences and instances of hallucination

3. Implications

   1. alignment with human judgments in pair comparison tasks

   2. discrepancies can be found in scoring evaluation and batch ranking

   3. range of biases and hallucinations, along with inconsistent judgments during the evaluation process

Abstract

Assessing the utility of MLLMs presents considerable challenges, primarily due to the absence of multimodal benchmarks that align with human preferences. Drawing inspiration from the concept of LLM-as-a-Judge within LLMs, this paper introduces a novel benchmark, termed MLLM-as-a-Judge

1. Scoring Evaluation

2. Pair Comparison

3. Batch Ranking.

closer examination reveals persistent challenges in the judgment capacities of LLMs, including diverse biases, hallucinatory responses, and inconsistencies in judgment, even in advanced models such as GPT-4V.

Takeaway

• MLLM (e.g., LLaVA and GPT-4V) demonstrates superior performance in certain datasets and inferior performance in others

• GPT-4V consistently outperforms the baseline across diverse datasets on average

  • However, it remains noteworthy that GPT-4V does not entirely supplant human judges in particular datasets

• multimodal tasks by calculating the similarity between human and MLLMs judgment and easuring human agreement on the analysis and judgment made by those MLLMs.

• across two settings (with, or without vision input), over three distinct tasks (i.e., Scoring Evaluation, Pair Comparison, and Batch Ranking)

Structure

1. Image-instruction pair

   1. containing pairs of images (M ) and their corresponding instructions (I) sourced from 10 diverse domains (e.g., math, chart, diffusion).

   2. MLLMs, generating a set of responses Ri ={r1, r2, . . . , rn} for each pair → The formation of the dataset of image-instruction-responses pairs.

      1. Scoring Evaluation: scale from 1 to 5, with the specific criteria for this rating system

      2. Pair Comparison: direct comparison between two responses, culminating in the identification of the superior one.

      3. Batch Ranking: The responses are systematically arranged in descending order of quality based on a given instruction, without any tie option

• curate a dataset consisting of 4,414 image-text pairs

1. MMLM response collection

   • GPT-4V, Gemini (GeminiTeam, 2023), LLaVA (Liu et al.,2023d), Qwen-VL-Max (Bai et al., 2023a), LLaVA-1.6-34b(Liu et al., 2023d), and CogVLM.

   • obtaining approximately 17,000 responses

     • non-compliant with security regulations excluded

2. comparison with human annotation.

   • conducted by 6 authors of this paper independently. These annotators are proficient in this domain, with different genders, ages, and educational backgrounds to ensure diversity.

Experiment

MLLM Used

• GPT-4V

• Gemini-Pro-Vision-1.0

• LLaVA-1.5-13b

• LLaVA-1.6-7b/13b/34b

• Qwen-VL-Plus/Max

• CogVLM

Evaluation

• Scoring Evaluation: compute the Pearson similarity between the MLLMs’ judgments and human ratings across different sub-datasets

• Pair Comparison: similarity between the MLLM judgments and human decisions using accuracy, F1-score

• Batch Evaluation: employ the Normalized Levenshtein distance to evaluate the similarity between judgments from MLLMs and human annotation

• Human Agreement: considered incorrect due to unique human perspectives

• Analysis Grading: MLLM analysis is assigned a score from 1 to 5, considering relevance, accuracy, creativity, and response granularity.

Empirical Results and Analysis

MLLM Judgment vs Human Annotation

• GPT-4V are closer to human annotations among all settings

• Overall, MLLM judgments perform better on Pair Comparison, while falling short in Scoring Evaluation and Batch Ranking, showing a huge gap between the model and human preferences.

MLLM Judging Consistency

• Conduct six repeated tests with MLLM judgments and calculated the weighted average

• GPT-4V substantially outperforms Gemini across all tasks. Particularly in Pair Comparison, GPT-4V achieves a higher consistency score of 0.675

Human Agreement

• GPT4V achieved around 70% human agreement across all settings, excelling in the Pair Comparison task with 79.3% agreement.

• Chain-of-Thought Do Not Enhance Performance

• Vision Perception Benefits MLLM Judging

  

  • omitting vision information entirely and providing a detailed description of the picture

    • judging tasks significantly improve with picture descriptions, achieving a Pearson similarity of 0.435 in Scoring Evaluation tasks, markedly outperformed judgments made without any vision perception.

Bias and Hallucination Detection

Bias?

• Egocentric Bias (자기 중심적)

  • MLLM relies too heavily on its training data or internal context

  • Instead of prioritizing the input questions or image context, the model tends to trust its internal patterns and data frequency more

    • GPT-4V consistently emphasizes privacy preservation, leading to higher scores for privacy-related questions based on its own metrics.

• Position Bias (위치 편향)

  • gives more weight to information based on its position in text or images.

    • For instance, the model may consider information in the top-left corner of an image to be more important or may select the first option presented in a list more frequently.

    • Batch Ranking prompt includes a sequence like ‘ABCD’, LLaVA replicates this sequence in 88.2% of responses.

      • can be reduced by introducing multiple examples, suggesting that prompts with more examples can better direct these models to follow instructions accurately

• Length Bias (길이 편향)

  • judgment is skewed by the length of a response

    • might mistakenly perceive longer responses as more sophisticated or trustworthy, or it might favor shorter responses as being more direct and accurate.

Hallucination?

• MLLMs generate non-existent information or provide false facts with confidence

• more likely to occur in complex reasoning chains or long-term vision-language contexts

1. Vision Hallucination

   1. MLLM misinterprets an image or recognizes non-existent objects

2. Language Hallucination

   1. MLLM generates incorrect textual information or provides contextually irrelevant answers

Mitigation Techniques

1. Human-in-the-Loop: human reviewers are involved in critical decisions

2. Fact-Checking Modules: external fact-checking mechanisms that cross-reference the model’s outputs

3. Uncertainty Detection: recognize when it is uncertain and flag responses as uncertain