Skip to content
/ Uniaa Public

Unified Multi-modal IAA Baseline and Benchmark

Notifications You must be signed in to change notification settings

KwaiVGI/Uniaa

Folders and files

NameName
Last commit message
Last commit date

Latest commit

13f2294 Â· Sep 27, 2024

History

49 Commits
Sep 25, 2024
Sep 25, 2024
Apr 15, 2024
Sep 27, 2024

Repository files navigation

Uniaa: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark

The Unified Multi-modal Image Aesthetic Assessment Framework, containing a baseline (a) and a benchmark (b). The aesthetic perception performance of UNIAA-LLaVA and other MLLMs is shown in (c).

The IAA Datasets Conversion Paradigm for UNIAA-LLaVA.

The UNIAA-Bench overview. (a) UNIAA-QA contains 5354 Image-Question-Answer samples and (b) UNIAA-Describe contains 501 Image-Description samples. (c) For open-source MLLMs, Logits can be extracted to calculate the score.

Release

  • [9/25] 🔥 Our UNIAA data is released! The corresponding fine-tuning and evaluation code can be found in the GitHub repository folder.
  • [4/15] 🔥 We build the page of UNIAA!

Performance

Aesthetic Perception Performance

Aesthetic Description Performance

Aesthetic Assessment Performance

Zero-shot

Supervised learning on AVA and TAD66K

Training on data of UNIAA

Step 1: Download Images and Json files

Step 2: Training On Specific MLLM

Test on UNIAA-Bench

For Aesthetic Perception

Step 1: Download Images and Json files

Step 2: Run the inference code

Step 3: Calculate the score

For Aesthetic Description

Step 1: Download Images and Json files

Step 2: Run the inference code

Citation

If you find UNIAA useful for your your research and applications, please cite using this BibTeX:

@misc{zhou2024uniaa,
      title={UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark}, 
      author={Zhaokun Zhou and Qiulin Wang and Bin Lin and Yiwei Su and Rui Chen and Xin Tao and Amin Zheng and Li Yuan and Pengfei Wan and Di Zhang},
      year={2024},
      eprint={2404.09619},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Contact

If you have any questions, please feel free to email wangqiulin@kuaishou.com and zhouzhaokun@stu.pku.edu.cn.