2017 11 22 - PaddlePaddle/Paddle GitHub Wiki

• Intel code review: https://github.com/PaddlePaddle/Paddle/pull/5808#pullrequestreview-78243382 (shortened the code)
• FP16 https://github.com/PaddlePaddle/Paddle/pull/5716#pullrequestreview-78316455
• Fix CI: https://github.com/PaddlePaddle/Paddle/pull/5848#pullrequestreview-78575256
• ROI Pool operator: https://github.com/PaddlePaddle/Paddle/pull/5831#pullrequestreview-78577531
• Unpool 2D Max operator: https://github.com/PaddlePaddle/Paddle/pull/5826#pullrequestreview-78599143
• nnpack: https://github.com/PaddlePaddle/Paddle/issues/5863
• operator LaTeX math
  • https://github.com/PaddlePaddle/Paddle/issues/5852
  • https://github.com/PaddlePaddle/Paddle/issues/5865
  • https://github.com/PaddlePaddle/Paddle/issues/5866

• Update the design doc for distributed architecture: https://github.com/PaddlePaddle/Paddle/pull/5859
• Implement the FTRL optimizer: https://github.com/PaddlePaddle/Paddle/pull/5785
• Add Log Loss : https://github.com/PaddlePaddle/Paddle/pull/5854
• Re-write the design doc for data reader: https://github.com/PaddlePaddle/Paddle/pull/5849

• remove ProtoData: https://github.com/PaddlePaddle/Paddle/pull/5775
• fix a lot of warnings when compile openBlas: https://github.com/PaddlePaddle/Paddle/pull/5704
• MKL
  • Update the VGG and ResNet benchmark when NUMA=ON: https://github.com/PaddlePaddle/Paddle/pull/5578/commits/44609c2a2e73ecb36a6ef1e3711424d22e348a26
  • fix compiler error in "WITH_MKL": https://github.com/PaddlePaddle/Paddle/pull/5748
  • fix some warning with MKLDNN related codes and etc: https://github.com/PaddlePaddle/Paddle/pull/5797
  • code review on MKLDNN:
    • auto set cpu env when mkldnn or mklml enabled
    • mkldnn only need one trainer
    • add option WITH_MKL
    • enable mkldnn_concat layer
    • enable mkldnn benchmark with googlenet
    • refine MKLDNNLayer
• [code review] Update installation docs: https://github.com/PaddlePaddle/Paddle/pull/5818

• Build testcapi environment:
  • https://github.com/gongweibao/testcapi
• Move Python functions to Go:
  • https://github.com/gongweibao/cloud/commits/addkubeconfig
• Add more efficient LRN GPU implement.
  • https://github.com/gongweibao/Paddle/commit/c30bfc65a9793a07157ccdbc0ee3adb7727bfcfa

• Add dot_prod_layer

  https://github.com/PaddlePaddle/Paddle/pull/5724

• Update the doc of layers.py

  https://github.com/PaddlePaddle/Paddle/pull/5742

• Accelerating the BeamSearch of conv_seq2seq (Doing)

• PR reveiw

  https://github.com/PaddlePaddle/Paddle/pull/5736

• add maxout op merge done https://github.com/PaddlePaddle/Paddle/issues/5569

• add unpool 2dmax op, code review ing https://github.com/PaddlePaddle/Paddle/issues/5787

• Add ROI Pool Operator (Doing) https://github.com/PaddlePaddle/Paddle/pull/5831
• Add Sequence Slice Operator (Done) https://github.com/PaddlePaddle/Paddle/pull/5546

• feature

  • 07/Label semantic roles
  • enforce shape of backward target to be {1}
  • beam_search_decode support multi data type

• bugfix && code refine

  • correct optimizer import
  • revert print in test_layers
  • WIP(fix nccl dependent)

• Init Python server

• PR review:
  • The dot product layer:
    • https://github.com/PaddlePaddle/Paddle/pull/5724#pullrequestreview-77345570
  • Make epsilon configurable for batch norm layer:
    • https://github.com/PaddlePaddle/Paddle/pull/5692#pullrequestreview-77360260
    • https://github.com/PaddlePaddle/Paddle/pull/5692#pullrequestreview-78001803
  • NCE operator:
    • https://github.com/PaddlePaddle/Paddle/pull/5480#pullrequestreview-78010402

• experiment for GAN
  • https://github.com/PaddlePaddle/Paddle/pull/5743
  • Coveraged but not generate a good image. Currently WIP
• Enhancement for Evaluators
  • Reduce lines of code by using layers.py. Not create op directly
  • Add comments
  • https://github.com/PaddlePaddle/Paddle/pull/5824
• Enhancement for operators
  • We need multiple precision for operators
    • Feature/support int64 for sum
      • https://github.com/PaddlePaddle/Paddle/pull/5832
    • Support many data types of several operators
      • https://github.com/PaddlePaddle/Paddle/pull/5731
• Python API for IfElse and its unittests
  • https://github.com/PaddlePaddle/Paddle/pull/5624

• [Speed compiling]: Refine cmake about CUDA to automatically detect GPU arch
  • https://github.com/PaddlePaddle/Paddle/pull/5713
  • local machine testing: centos, cuda 7.5, make -j8, WITH_GPU=ON
    • raw time: 31m24.320s
    • PR#5573: 26m43.523s
    • This PR: 15m0.158s
• Fix operators in debug mode
  • Fix cos_sim_op in debug mode: https://github.com/PaddlePaddle/Paddle/pull/5707
  • Fix bilinear_tensor_product_op in debug mode: https://github.com/PaddlePaddle/Paddle/pull/5703
  • Others need to be fixed: https://github.com/PaddlePaddle/Paddle/issues/5693
• Snapdragon NPE Survey: https://github.com/PaddlePaddle/Mobile/wiki/Snapdragon-NPE-Survey
• TfLite Survey: https://github.com/PaddlePaddle/Mobile/wiki/TfLite-Survey
• Code Review:
  • https://github.com/PaddlePaddle/Paddle/pull/5571#pullrequestreview-77683046
  • https://github.com/PaddlePaddle/Paddle/pull/5832
  • https://github.com/PaddlePaddle/Paddle/pull/5804
  • https://github.com/PaddlePaddle/Paddle/pull/5800

• Update settings for Aishell and Librispeech example
  https://github.com/PaddlePaddle/DeepSpeech/pull/13

• Add script for VoxForge dataset preparation
  https://github.com/PaddlePaddle/DeepSpeech/pull/16

• Fix CI
  https://github.com/PaddlePaddle/DeepSpeech/pull/18

• Adapt server end for demo
  https://github.com/PaddlePaddle/DeepSpeech/pull/28

• Code Review
  https://github.com/PaddlePaddle/DeepSpeech/pull/29
  https://github.com/PaddlePaddle/DeepSpeech/pull/21
  https://github.com/PaddlePaddle/DeepSpeech/pull/19
  https://github.com/PaddlePaddle/DeepSpeech/pull/8

• Paddle
  • PR
    • open, WIP, chapter 8 machine translation, encoder finished
    • open, beamsearch op support multiple data type
  • reviews
    • https://github.com/PaddlePaddle/Paddle/pull/5798#pullrequestreview-78322210
• visualDL
  • https://github.com/VisualDL/VisualDL/pull/31
  • https://github.com/VisualDL/VisualDL/pull/26
  • https://github.com/VisualDL/VisualDL/pull/23
  • https://github.com/VisualDL/VisualDL/pull/22
  • https://github.com/VisualDL/VisualDL/pull/21

Refactoring:

• support testing when training
• handle dropout and batch_norm operator in testing mode
  • https://github.com/PaddlePaddle/Paddle/pull/5734

Fix and Enhance:

• remove unused v2 framework
  • https://github.com/PaddlePaddle/Paddle/pull/5722

• Add Evaluator to book chapters:
  • https://github.com/PaddlePaddle/Paddle/pull/5778
• Fix bug and refine test_understand_sentiment_lstm
  • https://github.com/PaddlePaddle/Paddle/pull/5781
• Code review:
  • https://github.com/PaddlePaddle/Paddle/pull/5734
  • https://github.com/PaddlePaddle/Paddle/pull/5772
• [WIP]:
  • 1. Refine Python API, unify interface name
  • 2. Dynamic RNN API

• send recv op reviewing: https://github.com/PaddlePaddle/Paddle/pull/5520
• installation doc update: https://github.com/PaddlePaddle/Paddle/pull/5818
• fluid dist train design doc update and discussions: https://github.com/PaddlePaddle/Paddle/pull/5776
• cudnn conv 3d and refines https://github.com/PaddlePaddle/Paddle/pull/5783

• PR
  • fix text classification model: https://github.com/PaddlePaddle/models/pull/477
  • fix pnpair doc: https://github.com/PaddlePaddle/Paddle/pull/5800
  • add python function for reduce op: https://github.com/PaddlePaddle/Paddle/pull/5841
• Implement a Policy Gradients demo with refactored API[WIP]

• PR:
  • Fix GRUUnitOp:
    • https://github.com/PaddlePaddle/Paddle/pull/5804
  • Add the tf2paddle to convert model:
    • https://github.com/PaddlePaddle/models/pull/483
  • Add configurations for inception-resnet-v2 in models(WIP):
• PR Review:
  • https://github.com/PaddlePaddle/Paddle/pull/5736:
  • https://github.com/PaddlePaddle/Paddle/pull/5771:
  • https://github.com/PaddlePaddle/Paddle/pull/5480:

Op porting:

• Add cudnn pool3d (Merged)
• Add conv3d_trans_cudnn_op (Merged)
• Add padding for conv_transpose_op (Merged)

Fix and Enhance:

• [Add checking for CAPI] Output argument is empty or not.
• remove vector::erase
• Fix pool max with index.(Mask type should be int, not float) (Merged)
• fix bug (conv2d groups: 'num_channels % groups is not 0:' ) (Merged)
• reorder parameters of layer(Merged)

code review

• Conv cudnn 3d
• Add maxout operator.
• Add sequence slice operator

• TeamCity CI, modify develop Docker image to manlinux standar, https://github.com/PaddlePaddle/Paddle/issues/4050#issuecomment-344489740
• Bug Fix:
  • https://github.com/PaddlePaddle/cloud/pull/503
  • https://github.com/PaddlePaddle/cloud/pull/501
  • https://github.com/PaddlePaddle/Paddle/pull/5794
  • https://github.com/PaddlePaddle/Paddle/pull/5717

• PR:
  • Make epsilon configurable for batch norm layer [merged]:
    • https://github.com/PaddlePaddle/Paddle/pull/5692
  • Bug fix in dense example of CAPI [merged]:
    • https://github.com/PaddlePaddle/Paddle/pull/5817
  • Make a general framework for doc in how_to_use_capi:
    • https://github.com/PaddlePaddle/models/pull/385
• PR Review:
  • Fix bilinear_tensor_product_op in debug mode:
    • https://github.com/PaddlePaddle/Paddle/pull/5703
• Convert model:
  • Capi implement for resnet prediction.

• PR:
  • Implement DeepFM for CTR prediction:
    • https://github.com/PaddlePaddle/models/pull/485

• [Operator]Normalization methods:
  • BatchNorm Enhancement(support more dimension && options):
    • PR: https://github.com/PaddlePaddle/Paddle/pull/5845
  • Normalization refactorization:
    • PR WIP: https://github.com/PaddlePaddle/Paddle/pull/5686

• Refactorization

  • Python API
    • https://github.com/PaddlePaddle/Paddle/pull/5620 -Python API add the math ops in python
  • Benchmark accuracy/performance -https://github.com/PaddlePaddle/Paddle/pull/5680 -https://github.com/PaddlePaddle/Paddle/issues/5862
  • fix Tensor copytovector
    • https://github.com/PaddlePaddle/Paddle/pull/5455

• Reviews

  • FTRL optimizer
    • https://github.com/PaddlePaddle/Paddle/pull/5785
  • use Evaluator in tests
    • https://github.com/PaddlePaddle/Paddle/pull/5778
  • improve the initializer
    • https://github.com/PaddlePaddle/Paddle/pull/5760
  • removed the used models.py
    • https://github.com/PaddlePaddle/Paddle/pull/5645
  • simplify the demos
    • https://github.com/PaddlePaddle/Paddle/pull/5615

• Python API Refactorization

  • Improve the initializer Interface for fc, sequence_conv and conv2d layers https://github.com/PaddlePaddle/Paddle/pull/5760
  • Implementing the MSRA initializer for rectifier units https://github.com/PaddlePaddle/Paddle/pull/5805
  • WIP: Parameter Attribute Design and initializers https://github.com/PaddlePaddle/Paddle/issues/5819

• Paddle Mobile

  • WIP: Investigate Android NN https://github.com/PaddlePaddle/Mobile/issues/30

• Code Review

  • https://github.com/PaddlePaddle/Paddle/pull/5699
  • https://github.com/PaddlePaddle/Paddle/pull/5745