Scripts - jcjohnson/neural-style GitHub Wiki

Multiscale-Resolution-Scripts

  • A set of multires scripts which can easily be used for Neural-Style and any other style transfer similar project.

Neural-Zoom

  • The successor to the original Neural-Zoom.

  • Neural-Zoom let's you create an "infinite zoom" effect with Neural-Style, and other style transfer projects.

neural_style_dir_rng_fix_fc.lua

  • Combines neural_style_dir.lua with neural_style_rng_fix.lua, and makes FC layers usable.

neural_style_dir_rng_fix.lua

  • Combines neural_style_dir.lua with neural_style_rng_fix.lua.

neural-equalchannels.lua

  • This modified version of neural_style.lua attempts to equalize the activation strengths of all the filters/neurons/channels used for the style layers.

  • Currently the channel manipulation technique used does not work well with multiple style images, as per this example.

  • Results from using neural-equalchannels.lua can be found here.

neural_bestchannels.lua

  • This modified version of neural_style.lua attempts to increase the activation strengths of the top filters/neurons/channels used for the style layers.

  • The -nc parameter lets you choose how many of the top strongest activation channels you wish to increase the strength of.

  • You can modify lines 527-541 to match the code in this comment in order to reverse the "strengthening" and instead suppress the top strongest channels.

  • Currently the channel manipulation technique used does not work well with multiple style images, as per this example.

  • Results from using neural_bestchannels.lua can be found here.

convis.lua

  • This modified version of the original convis.lua, creates a single output image showing a heat map of all the activations from a specified layer.

  • Basically it lets you see how the model sees the image.

  • Examples can be found here.

  • Averaging all or some of the layer output images together for a model, can let you see a more complete view of how the model sees an image.

neural_style_adam.lua

  • This modified version of neural_style.lua, lets you manipulate the Adam optimizer's parameters via the command line inputs, just like other Neural-Style parameters.

  • The new parameters which only are only used when -optimizer adam is specified are, -epsilon, -beta1, -beta2, -weight_decay, and -learning_rate_decay. The -learning_rate parameter has also been made to now affect the L-BFGS optimizer.

  • I discovered that using -beta1 0.99, and -epsilon 1e-1 will correct almost all of Adam's flaws in regards to style transfer. More info can be found here. - ProGamerGov

neural-mean2.lua

  • More info can be found here.

VaKonS' Modified Neural-Style

  • Unlike other tiling methods, this version of neural_style.lua implements tiling internally.

  • The -save_iter parameter doesn't work unless you use -save_iter 1. Using -save_iter 1 saves the output after each row of tiles is completed.

  • The -image_size parameter affects the tile size, and not the total final output size. Resize your content image to the desired output.

neural_style_dir.lua

  • This modified version of neural_style.lua adds support for using directories/folders instead of specific images for the -style_image parameter.

  • The -style_image parameter will now automatically detect whether your specified value is a directory, or specific image(s), and then act accordingly. No additional parameters are needed.

neural-gdisp.lua

  • This modified version of neural_style.lua displays the gradient using the display package.

neural_style_rng_fix.lua

  • As per this issue/pull request, the NIN model has a Dropout layer that was added to improve training. This Dropout layer caused randomization that overrides the normal Neural-Style commands that control randomization, like -seed and -init image. This modified neural_style.lua removes the Dropout layer entirely so that it does not affect the style transfer process.

  • While it was designed for the NIN model, it should be able to remove the Dropout layers from any other model that might have them.

neural_style_l2_dd_fc_labels_t7_layer_weights_laplacian_time.lua

  • This version of neural_style.lua has support for simultaneous DeepDream and style transfer, usable FC layers, independent layer weights, Torch model and architecture support (.t7 models), Photorealism enhancements from (from deep-photo-styletransfer), a L2 latent state regularizer, the ability to choose the padding type, classification support using a model's associated category.txt text file, and the ability to keep track of how the setup process and iterations take to be completed (timekeeping).

neural-channels.lua

  • This modified version of neural_style.lua adds the ability to manipulate the channels, used by each layer. It's basically selected channels emphasis, similar to how DeepDream images can be created with specific content using specific channels (Like in the Tensorflow DeepDream examples on the Tensorflow Github page.)

Multi-Region Spatial Control in Neural-Style (Semantically Segmented/Masked Style Transfer)

New Parameters:

-color_codes

  • Ex: blue,green,black,white,red,yellow,grey,lightblue,purple

-content_seg

  • Ex: content_mask.png

-style_seg

  • Ex: style_mask.png or style_mask_1.png,style_mask_2.png

How To Use Segmented Style Transfer:

  • Using the wiki guides I created here: https://github.com/martinbenson/deep-photo-styletransfer/wiki, you can create the required style image masks, and content image masks, with freely available and open source tools. These guides should be extremely simple, and thus those without a lot of experience with GIMP, should still find mask creation easy.

  • The -content_seg and -style_seg parameters must be used together. These parameters are what you use your image masks with.

  • The "-color_codes" parameter helps lower GPU memory usage. To lower the GPU memory usage, only list the colors you used in your content mask. Any unused colors in your style mask(s) are not needed, and thus can be omitted to improve performance.

  • Each region uses the same parameters and the modified code does not support region specific parameters.

  • There are 9 different color regions, "blue,green,black,white,red,yellow,grey,lightblue,purple". This image contains all of them, which you can use to copy the desired mask colors from: https://i.imgur.com/52f4WIo.png

Known Issues:

  • The more content region you use, the more GPU resources are required.

  • The NIN model does not work with the code. Any help with solving this issue would be appreciated.

  • If you are using the G'MIC/GIMP method for mask image creation, than installing G'MIC on OSX/macOS may require some troubleshooting as per an issue reported by a user here: issue #35

  • If you have more than one style image with it's associated mask using the same mask color, then you will have to repeat the color in the -color_codes parameter for it to work. Example: You have a black mask image for your content image, and 2 style images that both have white and black mask images. Using -color_codes black will result in an error, but using -color_codes black,black will not.

neural_style_time.lua

  • This modified version of neural_style.lua adds the ability to keep track of elapsed time with the use of the -time parameter. This new parameter is used in a similar way to the existing -normalize_gradients parameter, as it doesn't not take any values or strings, and instead the time keeping feature is enabled simply by using it.

neural_mirage5.lua

  • This modified version of neural_style.lua is based on htoyryla's blog post here.

  • This modified version of neural_style.lua adds two new features. It allows the user to use a model's FC Layers for using in style transfer. The weight of the FC Layers can be independently controlled with the new -fc_weight command, while the normal content and style transfer layers are controlled with the normal -content_weight, and -style_weight commands.

  • The other feature this modified neural_style.lua adds, is the ability to display what the network is seeing in terms of what classes it was trained on. Though this feature only works with models that have an associated class list text file.

Neural-Tools

  • This project currently implements Scale Control, and Color Control (Luminance-Only style transfer, and color matching with histogram matching) in Neural-Style using external Python scripts. The external scripts allow for the features to be used on pretty much any other style transfer system.

  • The features come from the Controlling Perceptual Factors in Neural Style Transfer research paper.

  • The Neural-Tools wiki (includes extensive usage information and results for comparison with normal style transfer), can be found here.

nt_yuv_hist.lua

Neural-Display

  • Display your styled outputs alongside their content and style image(s), in a single image.

Neural-Graph

  • Features a modified neural_style.lua script that creates a more CSV friendly terminal output.

  • Simple graphing script to graph the loss values.

Neural-Tile

  • A fork of Larger-Neural-Style.

  • Solves all of the issues that the Larger-Neural-Style script had. The math behind the tiling process has also been completely redone.

  • Supports multires and features different variations of the script for different tile combos. For example: 3x3, 2x2, 2x1, and 1x2 tiles.

multiscale-nueral-style

  • A companion shell script for automating and creating a series of neural style images starting with a smaller size and cycling the way up through larger sizes, and defining logic for upsizing operation with a few memory saving tricks along the way. This script was inspired by jcjohnson's example multires script.

multires.sh

  • A simple script made by jcjohnson which slowly increases output image_size while using each output as the init_image value of the next run through with Neural-Style.

  • More information the multires script can be found here.

Neural-Zoom-Legacy

  • For creating endless zooming gifs/videos with Neural-Style, like those seen in Deepdream projects.

Larger-Neural-Style

  • It creates larger Neural-Style images through automated tiling.

  • A shell script which automates the "tiled u-pres" process described here.

  • Neural-Delta

  • Posted by htoyryla here

  • It is used instead of the default neural-style.lua

convis

  • A tool to visualize convolutional layer activations on an input image.

  • Creates images in which the input image is highlighted by each filter in the given conv layer.

  • It basically lets you see what Neural-Style sees on an image.

Neural-Style-Gradient

  • This bash scripts creates a gradient of style transfer images using jcjohnson's neural-style script. This is intended to help anyone who wants to use the NIN model and figure out the best parameters. The .sh file is well commented so you should have an easy time making edits to whatever you like.