训练alexnet - nonelittlesong/study-caffe GitHub Wiki

一、 生成标签

python代码inputTxt.py:

import argparse
import sys, os
def writeouput(result, path):
    if os.path.isfile(path):
        f = open(path, 'a')
        f.write(result + '\n')
        f.close()
    else:
        f = open(path, 'w')
        f.write(result + '\n')
        f.close()
def main(argv):
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "image_name",
        help = "Input image, directory or npy."
    )
    parser.add_argument(
        "image_label",
        help = "Output npy filename."
    )

    args = parser.parse_args()
    writeouput(args.image_name+" "+args.image_label, '/add/your/path/all.txt')
if __name__ == '__main__':
    main(sys.argv)

shell:

image_path = '/add/your/images/path/*.jpg'
image_label = '0'
for image_name in $image_path; do python inputTxt.py $image_name $image_label; done

二、 生成训练集和测试集

生成train.txt和val.txt
python代码：

from numpy.matlib import random

dataSet = []
fileIn = open('/add/your/path/all.txt')
for line in fileIn.readlines():
    dataSet.append(line.strip())

random.shuffle(dataSet)
pos = int(len(dataSet) * 0.75)
traindata = dataSet[:pos]
testdata = dataSet[pos:]

f = open('/add/your/traintxt/path/train.txt', 'w')
for row in traindata:
    f.write(row + '\n')
f.close()
f = open('/add/your/valtxt/path/val.txt', 'w')
for row in testdata:
    f.write(row + '\n')
f.close

生成lmdb
编辑create_imagenet.sh

#!/usr/bin/env sh
# Create the imagenet lmdb inputs
# N.B. set the path to the imagenet train + val data dirs
set -e
EXAMPLE = /add/your/lmdb/path
# DATA = /add/your/data/path
TOOLS = build/tools

TRAIN_DATA_ROOT = /add/your/path
VAL_DATA_ROOT = /add/your/path

# Set RESIZE=true to resize the images to 255x255. Leave as false if images have
# already been resize using another tool.
RESIZE=false
if $RESIZE; then
  RESIZE_HEIGHT=256
  RESIZE_WIDTH=256
else
  RESIZE_HEIGHT=0
  RESIZE_WIDTH=0
fi

if [ ! -d "$TRAIN_DATA_PATH" ]; then
  echo "Error: TRAIN_DATA_ROOT is not a path to a directory: $TRAIN_DATA_ROOT"
  echo "Set the TRAIN_DATA_ROOT variable in create_imagenet.sh to the path" \
       "where the ImageNet training data is stored."
  exit 1
fi

if [ ! -d "VAL_DATA_ROOT" ]; then
  echo "Error: VAL_DATA_ROOT is not a path to a directory: $VAL_DATA_ROOT"
  echo "Set the VAL_DATA_ROOT variable in create_imagenet.sh to the path" \
       "where the ImageNet validation data is stored."
  exit 1
fi

echo "Creating train lmdb..."
# 调用convert_imageset文件转换文件格式，后面为输入参数

GLOG_logtostderr=1 $TOOL/convert_imageset \
    --resize_height = $RESIZE_HEIGHT \
    --resize_width = $RESIZE_WIDTH \
    --shuffle \
    $TRAIN_DATA_ROOT \
    $EXAMPLE/train.txt \
    $EXAMPLE/train_lmdb

echo "Creating val lmdb..."

GLOG_logtostderr=1 $TOOLS/convert_imageset \
    --resize_height=$RESIZE_HEIGHT \
    --resize_width=$RESIZE_WIDTH \
    --shuffle \
    $VAL_DATA_ROOT \
    $EXAMPLE/pictest.txt \
    $EXAMPLE/test_lmdb

echo "Done."

三、 对现有图片求均值

使用imagenet中的make_imagenet_mean.sh对训练图片数据求均值，将其拷贝到你的文件目录下，修改下路径：

#!/usr/bin/env sh
# Compute the mean image from the imagenet training lmdb
# N.B. this is available in data/ilsvrc12

EXAMPLE = /add/your/path
# DATA = data/ilsvrc12
TOOLS = build/tools

$TOOLS/compute_image_mean $EXAMPLE/train_lmdb \  \

  $EXAMPLE/imagenet_mean.binaryproto

echo "Done."

在caffe根目录下运行`./add/your/path/make_imagenet_mean.sh
(ps. 不在caffe根目录下也成功运行了，不知道有何影响)

四、 设置参数

(1)修改train_val.prototxt

name: "AlexNet"
layer {
  name: "data"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    mirror: true
    crop_size: 227
    mean_file: "/add/your/path/imagenet_mean.binaryproto"
  }
  data_param {
    source: "/add/your/path/train_lmdb"
    batch_size: 50
    backend: LMDB
  }
}
layer {
  name: "data"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TEST
  }
  transform_param {
    mirror: false
    crop_size: 227
    mean_file: "/add/your/path/imagenet_mean.binaryproto" #上同
  }
  data_param {
    source: "/add/your/path/test_lmdb" #上同
    batch_size: 5
    backend: LMDB
  }
}

(2)修改solver.prototxt

net: "/add/your/path/train_val.prototxt" # 网络模型文件路径
test_iter: 100 # test的迭代次数，批处理大小是50，100×50为测试集个数
test_interval: 100 # 训练时每迭代500次测试一次
base_lr: 0.01 # 学习率
lr_policy: "step" # 学习率的改变策略
gamma: 0.1 # 学习率每次改变的值
stepsize: 500 # 步长，每迭代500，base_lr = base_lr*gamma
display: 20 # 每迭代20次显示，前面调参的时候，可以多显示，真正训练可调高该数值
max_iter: 1000 # 最大迭代次数

momentum: 0.9 # 动量

weight_decay: 0.0005 # 权重衰减

snapshot: 200 # 每迭代200次存储一次Caffemodel。即训练好的模型。
snapshot_prefix: "/add/your/path" # 训练好模型的存储位置
slover_mode: CPU

五、 开始训练

#！/usr/bin/env sh
# set -e
./build/tools/caffe train --solver=/add/your/path/solver.prototxt

六、 注意

1. 等号左右两边不要加空格
2. $DATA_ROOT 与 image_name拼接后必须是文件的真实路径