2017 08 02 - PaddlePaddle/Paddle GitHub Wiki

Organize new framework project

• https://github.com/paddlepaddle/paddle/projects/19

Code style

• https://github.com/PaddlePaddle/Paddle/issues/3189
• https://github.com/PaddlePaddle/Paddle/pull/3173
• https://github.com/PaddlePaddle/Paddle/pull/3191#pullrequestreview-53958171

Gradient op

• https://github.com/PaddlePaddle/Paddle/pull/3186#pullrequestreview-53925149
• https://github.com/PaddlePaddle/Paddle/pull/3188#pullrequestreview-53924687

Errands

• https://github.com/PaddlePaddle/Paddle/pull/3190

• Testing demos in Paddle book, running on Paddle cloud.
• Preparing for the Paddle Cloud blog post:
  • https://github.com/PaddlePaddle/Paddle/issues/2930
  • https://github.com/PaddlePaddle/blog/issues/1
  • https://github.com/PaddlePaddle/cloud/issues/262
• Learn caffe2 code.
• Fix develop branch build: https://github.com/PaddlePaddle/Paddle/pull/3106
• Reviews:
  • https://github.com/PaddlePaddle/Paddle/pull/3128#pullrequestreview-53289909
  • https://github.com/PaddlePaddle/Paddle/pull/3086#pullrequestreview-52773091
  • https://github.com/PaddlePaddle/Paddle/pull/3102#issuecomment-319799911

• Completing backward and adding unittests with @yuyang and @dongzhihong
  • https://github.com/PaddlePaddle/Paddle/pull/3068
  • https://github.com/PaddlePaddle/Paddle/pull/3142
• Refactoring grad_op_builder to make it more simple
  • https://github.com/PaddlePaddle/Paddle/pull/3192
• [WIP] Refine document about backward
• Reviews:
  • https://github.com/PaddlePaddle/Paddle/pull/3060
  • https://github.com/PaddlePaddle/Paddle/pull/3188

• Refactor:

  • Complete unittest and Design doc with @yuyang and @fengjiayi https://github.com/PaddlePaddle/Paddle/pull/3068
  • Complete Gaussian Operator https://github.com/PaddlePaddle/Paddle/pull/3060
  • rowwise_add Backward implement https://github.com/PaddlePaddle/Paddle/pull/3183
  • NoGradient macro guard https://github.com/PaddlePaddle/Paddle/pull/3166

• reviews

  • Add unittest for FillZerosLikeOp https://github.com/PaddlePaddle/Paddle/pull/3142
  • cpu/gpu mean op and its unit test https://github.com/PaddlePaddle/Paddle/pull/3135
  • Refactoring grad_op_builder to make it more simple https://github.com/PaddlePaddle/Paddle/pull/3192

Operator --> OpKernel --> Tensor/DeviceContext --> Eigen

• Enable operator gpu unittest

  • https://github.com/PaddlePaddle/Paddle/pull/3050

• Bug fix:

  • fix bug in OpKernel register macro
    • https://github.com/PaddlePaddle/Paddle/pull/3059
  • fix bug in memory GetGPUBuddyAllocator
    • https://github.com/PaddlePaddle/Paddle/pull/3097
  • Fix bug in cuda device context
    • https://github.com/PaddlePaddle/Paddle/pull/3101

• Backward implementation merged.
  • https://github.com/PaddlePaddle/Paddle/pull/3068
• Simplify Scope, remove std::shared_ptr
  • https://github.com/PaddlePaddle/Paddle/pull/3116
• Move pybind into paddle::frmaework
  • https://github.com/PaddlePaddle/Paddle/pull/3173
• Gradient Check Implementation
  • https://github.com/PaddlePaddle/Paddle/pull/3167
• [WIP] Add Gradient Operator MeanOp
  • https://github.com/PaddlePaddle/Paddle/pull/3180
• Some tiny fixes:
  • https://github.com/PaddlePaddle/Paddle/pull/3177
  • https://github.com/PaddlePaddle/Paddle/pull/3178

• FCN:

  • Fix FCN model parameters initialization problem(fine-tuned from vgg16)
  • Debug gradient computation error
  • Change train and test data to be consistent with caffe demo

• DeepSpeech2

  • Refine scripts on paddlecloud
    • https://github.com/PaddlePaddle/models/pull/144

• GNR
  • add GNR model as in the paper: https://github.com/lcy-seso/models/tree/GNR/globally_normalized_reader
  • add a sub nest sequence layer: https://github.com/PaddlePaddle/Paddle/pull/3176
  • modification to sequence softmax layer to enable it activate on each sub-sequence when it gets a nested sequence input:
    • https://github.com/PaddlePaddle/Paddle/compare/develop...lcy-seso:seq_softmax_in_sub_seq_mode

• ISSUE
  • https://github.com/k8sp/sextant/issues/567
• PR:
  • https://github.com/k8sp/sextant/pull/566

• paddle refactor

  • PR

    • merged, design of varient length sequence RNN with @qingqing @luotao @longfei @daoyuan
    • open, LODTensor
    • open, tensor type reduction
    • open, crossentropy gradient OP
    • open, op helper function for generating grad var's name
    • open, move net_op to operators/

  • issues and reviews

    • a discussion on Python v2 API to new operator framework
    • https://github.com/PaddlePaddle/Paddle/pull/3024#pullrequestreview-53368108
    • https://github.com/PaddlePaddle/Paddle/pull/3124#pullrequestreview-53451498
    • https://github.com/PaddlePaddle/Paddle/issues/3119

  • Community construction

    • deploy internal wiki with @yongfeng
      • http://172.19.32.197:4000/
      • wiki content is hosted on icode
      • so that we can fix internal docs by PR

• Discussion Mkldnn related, with @hedaoyuan @liuyibin:
  • code review on mkldnn_fc layer and unit test: https://github.com/PaddlePaddle/Paddle/pull/3096, closed this PR due to separate and reconstruct it for better matching Paddle's style.
  • code review on update MKLDNN cmakes: https://github.com/PaddlePaddle/Paddle/pull/3168
• make setup.sh of deepspeech2 more robust:
  • update installation of libsndfile, add unitest of soundfile: https://github.com/PaddlePaddle/models/pull/177
  • make kenlm install more robust: https://github.com/PaddlePaddle/models/pull/182
  • change the wget method to download large files: https://github.com/PaddlePaddle/models/pull/184
  • add wget in docker production image: https://github.com/PaddlePaddle/Paddle/pull/3172
• code review:
  • Slice Layer: https://github.com/PaddlePaddle/Paddle/pull/3126#pullrequestreview-53402519

• DS2
  • Fix numerical instability for Warp CTC
    https://github.com/gangliao/warp-ctc/pull/5
  • Tune model on THCHS30 dataset (WIP)
  • Debug cudnn_batch_norm with yibing
  • Train character based language model on chinese giga dataset (WIP)
• Code Review
  • https://github.com/PaddlePaddle/models/pull/140#pullrequestreview-53520778
  • https://github.com/PaddlePaddle/models/pull/183#pullrequestreview-53691101
  • https://github.com/PaddlePaddle/models/pull/185#pullrequestreview-53724352

• PaddlePaddle Refactoring：
  • Refine InferShape in RNN op: https://github.com/PaddlePaddle/Paddle/pull/3124
  • Sigmoid backward implementation: https://github.com/PaddlePaddle/Paddle/pull/3181
  • code review:
    • LODTensor Review: https://github.com/PaddlePaddle/Paddle/pull/3109#pullrequestreview-53502252
    • Initialize Gradient Checker: https://github.com/PaddlePaddle/Paddle/pull/3167#pullrequestreview-53745100
• DS2:
  • Bug fix in WarpCTC with @Yaming : https://github.com/PaddlePaddle/Paddle/pull/3105
  • Bug in multi-GPU inference : https://github.com/PaddlePaddle/Paddle/pull/3082
  • Code Review: https://github.com/PaddlePaddle/Paddle/pull/3185
• Code Review:
  • add RowL2NormLayer: https://github.com/PaddlePaddle/Paddle/pull/3083#pullrequestreview-53490369
  • Add ClipLayer: https://github.com/PaddlePaddle/Paddle/pull/3112
• Other:
  • Help haoshuang to debug FCN
  • Discuss 3D-Image requirements with medical image team. organized by Yongfeng.

• PaddleCloud
  • Bug fix: https://github.com/PaddlePaddle/Paddle/pull/3128
  • etcd embed for unit test: https://github.com/PaddlePaddle/Paddle/pull/3086
  • Deploy DLNEL PaddleCloud on the official cluster.
  • [WIP] PaddleCloud PPT
  • Following up new PServer supports sparse updater. https://github.com/PaddlePaddle/Paddle/issues/3144
• MPI
  • Submit paddle cluster job with local training package.

• PaddlePaddle Image Enhancement
  • Faster R-CNN: RCNNDetectionLayer in developing.
• Cooperation with Video Analysis Team in IDL:
  • Add ClipLayer PR(merged): https://github.com/PaddlePaddle/Paddle/pull/3112
  • Add RowL2NormLayer PR(merged): https://github.com/PaddlePaddle/Paddle/pull/3083

DS2:

• Fix the dimension mismatch problem in mfcc training
• Fix the CUDNN_STATUS_NOT_SUPPORTED error (issue#929) (with@yaming@qingqing)
  • https://github.com/PaddlePaddle/models/pull/185
  • https://github.com/PaddlePaddle/Paddle/pull/3185
• Refine CTC decoders in C++
  • https://github.com/PaddlePaddle/models/pull/139/files
• Keep track of the optimization of Paddle in MKL_DNN

Code Review:

• https://github.com/PaddlePaddle/models/pull/183
• https://github.com/PaddlePaddle/models/pull/182
• https://github.com/PaddlePaddle/models/pull/157

• Paddlebook docker testing
  • https://github.com/PaddlePaddle/book/pull/357
  • https://github.com/PaddlePaddle/book/issues/356
• IDL medical image processing
  • 3D-Conv

• dlnel GPU cluster deployment and testing
• Big model training https://github.com/PaddlePaddle/Paddle/issues/3199
• enable pip install paddle: https://github.com/PaddlePaddle/Paddle/pull/3102
• try porting tensor/op to go: https://github.com/PaddlePaddle/cloud/issues/265 https://github.com/typhoonzero/babyduck
• follow up mixed CPU and GPU: https://github.com/PaddlePaddle/Paddle/issues/3089
• PaddleCloud blog and PPT writing

Merged:

• Refine compute code in operators #3179 https://github.com/PaddlePaddle/Paddle/pull/3179
• Fix pre-commit #3161 https://github.com/PaddlePaddle/Paddle/pull/3161
• cpu/gpu mean op and its unit test #3135 https://github.com/PaddlePaddle/Paddle/pull/3135
• ENH: Add EigenScalar #3133 https://github.com/PaddlePaddle/Paddle/pull/3133
• ENH: Add comments for memory and memcpy #3095 https://github.com/PaddlePaddle/Paddle/pull/3195
• Check Insufficient Alloc in tensor #3094 https://github.com/PaddlePaddle/Paddle/pull/3194

Review:

• Install GCC 4.8 in Dockerfile so to work with Eigen GPU code #3190
• add check in OperatorContext Input/Output #3136
• cpu/gpu mean op and its unit test #3135
• Add Backtrace for enforce #3134
• Add CopyFrom interface for base DeviceContext in Tensor. #3103
• Fix bug in cuda device context #3101
• fix bug in memory GetGPUBuddyAllocator

• Mobile
  • Detect model
    • Some performance-related optimization development, and performance tuning on RK3288.
    • https://github.com/PaddlePaddle/Paddle/pull/3126
    • https://github.com/PaddlePaddle/Paddle/pull/3141
  • Recognition Model
    • Performance debugging on P9.
  • Review
    • https://github.com/PaddlePaddle/Mobile/pull/1
    • https://github.com/PaddlePaddle/Paddle/pull/3163
• Review
  • https://github.com/PaddlePaddle/Paddle/pull/3096
  • https://github.com/PaddlePaddle/Paddle/pull/3101
  • https://github.com/PaddlePaddle/Paddle/pull/3105

• pr

  • https://github.com/PaddlePaddle/Paddle/pull/3163 ignore im2col if not necessary in conv 1 * 1
  • https://github.com/PaddlePaddle/Mobile/pull/1 Results of tests on flower102 of mobilenet and pruning.

• learning neon.

• DS2:
  • Realtime ASR demo: https://github.com/PaddlePaddle/models/pull/186
  • Refactor and simplify the inference-related scripts: https://github.com/PaddlePaddle/models/pull/183
  • Overall Progress: Some key bugs fixed. 8/17 Done. (Task List: https://github.com/PaddlePaddle/models/issues/176) (By Yaming, Yibing, Qingqing, Luotao and Xinghai)
  • Code Reviews: https://github.com/PaddlePaddle/Paddle/pull/3082#pullrequestreview-52567129

• use operator context and infer context
  • https://github.com/PaddlePaddle/Paddle/pull/3024
• export Backward to python
  • https://github.com/PaddlePaddle/Paddle/pull/3174
• Softmax grad op
  • https://github.com/PaddlePaddle/Paddle/pull/3164
• add check in OperatorContext Input/Output
  • https://github.com/PaddlePaddle/Paddle/pull/3136

• Backward implementation merged.
  • https://github.com/PaddlePaddle/Paddle/pull/3068
• Simplify Scope, remove std::shared_ptr
  • https://github.com/PaddlePaddle/Paddle/pull/3116
• Refine InferShape in RNN op:
  • https://github.com/PaddlePaddle/Paddle/pull/3124