Content Image and Style Image Preparation - jcjohnson/neural-style GitHub Wiki

By modifying the style image or content image, you can change the output image produced by Neural-Style.

Content Image:

  • Waifu2x can be used to increase the quality of your content image, resulting in a better output image.

  • Manipulating the image colors.

  • Darkening or brightening the image.

  • Decreasing the image resolution can change the output.

Style Image:

  • Waifu2x can be used to increase the quality of your style image, resulting in a better output image.

  • Manipulating the image colors.

  • Darkening or brightening the image.

  • The quality/resolution should be kept as high as possible so that the output image is not blurry.

Content Image And Style Image Resolution:

  • As you can see in the above comparison, changing the resolution of the content image creates interesting results. The neural net is able to predict what the image would look like if it were a higher resolution. As the resolution of the content image is lowered, it becomes more difficult for the neural net to predict what the content image would look like if it were a higher quality. Keep in mind that the model used in the above example image, was trained on 224x224 images. The comparison image is of the output's at iteration 50. The comparisons at iteration 250 and iteration 500, can be viewed here: https://imgur.com/a/vA95Y

  • Note that while the output won't be blurry with a low quality content image, the content image quality still affects your final output.

  • As you can see in the above comparison, changing the resolution of the style image degrades the output quality.

Output Image Processing:

Making your output image look better with GIMP:

GIMP is a foss alternative to software like Photoshop. This means that GIMP is %100 open source and free as in beer.

Multi-Style Combination:

  • This was a test of an experimental idea using the exact same method as the tiling method in Adobe InDesign described on this page: https://github.com/jcjohnson/neural-style/wiki/Techniques-For-Increasing-Image-Quality-Without-Buying-a-Better-GPU

  • Instead of creating tiles, I made rectangles across specific parts of the image. I then placed multiple content images that had been run in Neural-Style with a different style, in their own separate InDesign layer. I then copied them individually into previously made rectangles and used the basic feather (2p1) and choke 8% to blend them into the image. The image_size value stayed the same for every style image used with the content image. After blending in the rectangles that contain styles from different Neural-Style outputs, you can choose to run the image through Neural-Style with the same parameters again in order to farther "blend" the added parts with the main image, but in order to do so, I suggest adding each style used to the -style_image parameter.

  • Some of the added parts from other style images used on the content image, blend in extremely well while others really stand stand out. I think this concept could be used to subtlety enhance Neural-Style output images.

The full output images from which the rectangles are from, can be found here. The style and content images can be found here.

Iteration Count And Quality:

The more iterations you run, the more "clear/clean" the output image gets. The amount by which the output becomes more "clear/clean", lowers as the iteration count gets higher. So eventually more iterations wont't help. The optimal number of iterations, is dependent on the chosen Neural-Style parameters.

An example of the changing output from iteration 50 to iteration 6050, can be seen below:

(Note the changes to the right eye between iterations)

Hints/Tips:

  • van Gogh's Starry Night painting has been observed to work really well on winter/snowy content images. The blue colors in Starry Night, seem to blend well with white snow.

  • The lighting in your content image will affect the output. Content luminescence matters when choosing a content image.

  • Objects/particles like snow, confetti, etc... will normally be mostly removed by Neural-Style when applying the style.

  • Content images that have motion blur, will normally be fixed during the style transfer process. I am not sure as to what the extent of the Neural-Style's blur fixing capabilities are though.

  • Using Neural-Style's ability to increase the resolution/quality of a content image, opens up interesting ways to use content images created by GANs, because (at least at the momement) GANs cannot create larger images without large amounts of time and GPU resources.