TraVelGAN - a new approach to the problem of unpaired image to image transformation.

 We have discussed the CycleGAN architecture in a recent post.  CycleGAN is a GAN architecture for learning unpaired image to image translations. Unlike other approaches such as CycleGAN to this difficult problem, a new architecture called TraVelGAN does not rely on pixel to pixel difference between images.  So it doesn't use any cycle consistency constraints (2 GANs helping each other out internal to the architecture).  Instead, it uses a new module called a Siamese Network.

Marco Pasini has a good blog post called 'A New Way to Look at GANs', that provides a good overview explanation of what a Siamese network is all about.

The introduction of a Siamese network into an overall architecture allows for organizing the latent space associated with the algorithm it is working within.  It allows an image to be encoded to a latent vector.

The TraVelGAN architecture is a traditional Generator-Discriminator GAN architecture with the addition of an internal separate Siamese Network.  The Siamese Network encodes images in latent space and aims to reduce distances between transformation vectors of image pairs.

So, the Discriminator guides the Generator into generating images that look realistic.  The Siamese Network guides the Generator so that each original image shares semantics with it's generated version (they sit close together in latent space).

The paper on the TravlGAN architecture 'TraVelGAN: Image-to-image Translation by Transformation Vector Learning' can be found here.


Popular posts from this blog

Simulating the Universe with Machine Learning

CycleGAN: a GAN architecture for learning unpaired image to image transformations

Pix2Pix: a GAN architecture for image to image transformation