Tianlong Chen (陈天龙)

What does not kill you makes you stronger

(CVPR 2021) Troubleshooting Blind Image Quality Models in the Wild

Troubleshooting Blind Image Quality Models in the Wild

[Paper] [Code]


Recently, the group maximum differentiation competition (gMAD) has been used to troubleshoot blind image quality assessment (BIQA) models, with the help of multiple full-reference metrics. When applying this type of approach to troubleshoot “best-performing” BIQA models in the wild, we are faced with a practical challenge: it is highly nontrivial to obtain stronger competing models for efficient failure-spotting in gMAD. Inspired by recent findings that difficult samples of deep models may be exposed through network pruning, we construct a set of “self-competitors”, as random ensembles of pruned versions of the target model. Diverse failures can then be efficiently identified via self-competition in gMAD. Next, we fine-tune both the target and its pruned variants on the human-rated gMAD set. This allows all models to learn from their respective failures, and prepare themselves for the next round of the gMAD competition. Experimental results demonstrate that our method efficiently troubleshoots BIQA models in the wild with significantly improved generalizability.