Fréchet Wavelet Distance: A Domain-Agnostic Metric for Image Generation

Lokesh Veeramacheneni1, Moritz Wolter1, Hilde Kuehne2,3, Juergen Gall1,4,
1University of Bonn 2University of Tuebingen 3MIT-IBM Watson AI Lab 4Lamarr Institute for Machine Learning and Artificial Intelligence
arXiv Code pip

First two images depict the same person, while last image is different. MSE of InceptionV3 predictions is lower for last two images whereas MSE of Wavelet packets show first two images are similar.

Abstract

Modern metrics for generative learning like Fréchet Inception Distance (FID) and DINOv2-Fréchet Distance (FD-DINOv2) demonstrate impressive performance. However, they suffer from various shortcomings, like a bias towards specific generators and datasets. To address this problem, we propose the Fréchet Wavelet Distance (FWD) as a domain-agnostic metric based on the Wavelet Packet Transform (Wp) . FWD provides a sight across a broad spectrum of frequencies in images with a high resolution, preserving both spatial and textural aspects. Specifically, we use Wp to project generated and real images to the packet coefficient space. We then compute the Fréchet distance with the resultant coefficients to evaluate the quality of a generator. This metric is general-purpose and dataset-domain agnostic, as it does not rely on any pre-trained network while being more interpretable due to its ability to compute Fréchet distance per packet, enhancing transparency. We conclude with an extensive evaluation of a wide variety of generators across various datasets that the proposed FWD can generalize and improve robustness to domain shifts and various corruptions compared to other metrics.

Method





We propose the Fréchet Wavelet Distace (FWD) to primarily tackle the dataset-domain bias which inturn leverages Wavelet Packet Transform (Wp). In a nutshell, Wp constructs a tree of images by recursively convolving with predefined filters. Example of Wp in action is provided in below figure.

The metric is computed in three steps. First, we compute the pth packet mean for the set of N generated images μgp=1N∑n=1NW(In)p, where In represents nth image in N images. Second, we compute the convariance matrix as follows Σgp=1N−1∑n=1N(W(In)p−μgp)(W(In)p−μgp)T. Similarly, we compute the mean and covariance for the real set of images and represent as μrp and Σrp respectively. Finally, we compute the Fréchet distance across all P packets as FWD=1P∑p=1P||μrp−μgp||22+Trace(Σrp+Σgp−2ΣrpΣgp).

Results

Interpretability

Since FWD averages Fréchet distances across all frequency bands, this design choice allows one to understand overall score. The below figure depicts per-packet FWD for both StyleGAN2 and DDGAN. We observe frequency characteristics of DDGAN images are near to original dataset compared to StyleGAN2 images.


Domain bias detection

Interpolation end reference image.

FID prefers Proj. FastGAN over DDGAN in all datasets, whereas FWD prefers DDGAN. FD-DINOv2 agree with FWD in all datasets except DNDD.



Computational Complexity

Interpolation end reference image.

Computational efficiency of FID, FD-DINOv2, and FWD.



Use FWD in your evaluation

pip install pytorchfwd

BibTeX

@inproceedings{
veeramacheneni2025fwd,
title={Fr\'echet Wavelet Distance: A Domain-Agnostic Metric for Image Generation},
author={Lokesh Veeramacheneni and Moritz Wolter and Hildegard Kuehne and Juergen Gall},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=QinkNNKZ3b}
}