GAN metrics - Serbipunk/notes GitHub Wiki

cross - reenactment

image

self - reenactment

image

image

computable scores

PSNR (Peak Signal-to-Noise Ratio)

SSIM (Structural Similarity Index Measure)

from skimage.metrics import structural_similarity

theory: https://www.youtube.com/watch?v=-i3NQ-by2b8

LPIPS (Learned Perceptual Image Patch Similarity)

orb_sim

def orb_sim(im1, im2):
    orb = cv2.ORB_create()

    kp_a, desc_a = orb.detectAndCompute(im1, None)
    kp_b, desc_b = orb.detectAndCompute(im2, None)

    # bf matcher
    bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)

    # perform matches
    matches = bf.match(desc_a, desc_b)

    similar_regions = [m for m in matches if m.distance < 50]
    if len(matches) == 0:
        return 0.
    return len(similar_regions) / len(matches)

FID (Frechet Inception Distance)

image

https://www.youtube.com/watch?v=TJeeZFNXi9M

IQA (image quality assessment metrics)

image

CSIM (just for face generation)

cosine similarity of fr embedding

user preference scores

UMTN & UAPP (megaportraits)

image