FAQ ncnn produce wrong result - nihui/ncnn GitHub Wiki

caffe2ncnn tool assumes the caffemodel is row-major (produced by c++ caffe train command).

The kernel 3x3 weights should be stored as

a b c
d e f
g h i

However, matlab caffe produced col-major caffemodel.

You have to transpose all the kernel weights by yourself or re-training using c++ caffe train command.

Besides, you may interest in https://github.com/conanhujinming/matcaffe2caffe

If your caffemodel is trained using c++ caffe and opencv, then the input image should be BGR order.

If your model is trained using matlab caffe or mxnet or tensorflow, the input image would probably be RGB order.

The channel order can be changed on-the-fly through proper pixel type enum

// construct RGB blob from rgb image
ncnn::Mat in_rgb = ncnn::Mat::from_pixels(rgb_data, ncnn::Mat::PIXEL_RGB, w, h);

// construct BGR blob from bgr image
ncnn::Mat in_bgr = ncnn::Mat::from_pixels(bgr_data, ncnn::Mat::PIXEL_BGR, w, h);

// construct BGR blob from rgb image
ncnn::Mat in_bgr = ncnn::Mat::from_pixels(rgb_data, ncnn::Mat::PIXEL_RGB2BGR, w, h);

// construct RGB blob from bgr image
ncnn::Mat in_rgb = ncnn::Mat::from_pixels(bgr_data, ncnn::Mat::PIXEL_BGR2RGB, w, h);

You shall pass continous pixel buffer to from_pixels family.

If your image is an opencv submat from an image roi, call clone() to get a continous one.

cv::Mat image;// the image
cv::Rect facerect;// the face rectangle

cv::Mat faceimage = image(facerect).clone();// get a continous sub image

ncnn::Mat in = ncnn::Mat::from_pixels(faceimage.data, ncnn::Mat::PIXEL_BGR, faceimage.cols, faceimage.rows);

Apply pre process according to your training configuration

Different model has different pre process config, you may find the following transform config in Data layer section

transform_param {
    mean_value: 103.94
    mean_value: 116.78
    mean_value: 123.68
    scale: 0.017
}

Then the corresponding code for ncnn pre process is

const float mean_vals[3] = { 103.94f, 116.78f, 123.68f };
const float norm_vals[3] = { 0.017f, 0.017f, 0.017f };
in.substract_mean_normalize(mean_vals, norm_vals);

Mean file is not supported currently

So you have to pre process the input data by yourself (use opencv or something)

transform_param {
    mean_file: "imagenet_mean.binaryproto"
}

The blob names for input and extract are differ among models.

For example, squeezenet v1.1 use "data" as input blob and "prob" as output blob while mobilenet-ssd use "data" as input blob and "detection_out" as output blob.

Some models may need multiple input or produce multiple output.

ncnn::Extractor ex = net.create_extractor();

ex.input("data", in);// change "data" to yours
ex.input("mask", mask);// change "mask" to yours

ex.extract("output1", out1);// change "output1" to yours
ex.extract("output2", out2);// change "output2" to yours

Each channel pointer is aligned by 128bit in ncnn Mat structure.

blob may have gaps between channels if (width x height) can not divided exactly by 4

Prefer using ncnn::Mat::from_pixels or ncnn::Mat::from_pixels_resize for constructing input blob from image data

If you do need a continous blob buffer, reshape the output.

// out is the output blob extracted
ncnn::Mat flattened_out = out.reshape(out.w * out.h * out.c);

// plain array, C-H-W
const float* outptr = flattened_out;

The ncnn::Extractor object is stateful, if you reuse for different input, you will always get exact the same result cached inside.

Always create new Extractor to process images in loop unless you do know how the stateful Extractor works.

for (int i=0; i<count; i++)
{
    // always create Extractor
    // it's cheap and almost instantly !
    ncnn::Extractor ex = net.create_extractor();

    // use
    ex.input(your_data[i]);
}