2018 01 10 - PaddlePaddle/Paddle GitHub Wiki

• Bug fix: https://github.com/PaddlePaddle/Paddle/pull/7308#pullrequestreview-87361282
• Inference engine design doc: https://github.com/PaddlePaddle/Paddle/pull/7315#pullrequestreview-87330611
• Merge buildtools repo into paddle: https://github.com/PaddlePaddle/Paddle/pull/7326
• Fluid Inference: https://github.com/PaddlePaddle/Paddle/pull/7339#pullrequestreview-87633025

• Finishing up Android Camera Demo https://github.com/PaddlePaddle/Mobile/pull/61

[WIP] Unittests for MultiDevice

• https://github.com/PaddlePaddle/Paddle/pull/7396
• https://github.com/PaddlePaddle/Paddle/pull/7381
• https://github.com/PaddlePaddle/Paddle/pull/7292

[WIP] Performance tuning for LoD related op

• https://github.com/PaddlePaddle/Paddle/pull/7247
• https://github.com/PaddlePaddle/Paddle/pull/7240
• https://github.com/PaddlePaddle/Paddle/pull/7219

Performance enhancements

• https://github.com/PaddlePaddle/Paddle/pull/7208
• https://github.com/PaddlePaddle/Paddle/pull/7196

Code style enhancement

• https://github.com/PaddlePaddle/Paddle/pull/7160
• https://github.com/PaddlePaddle/Paddle/pull/7131

• Distribution Fluid image classification: https://github.com/PaddlePaddle/Paddle/pull/7435
• Async send recv:https://github.com/PaddlePaddle/Paddle/pull/7133

• Fluid

  • PR/LoD add validation check

• VisualDL

  • https://github.com/PaddlePaddle/VisualDL/pull/73

  • https://github.com/PaddlePaddle/VisualDL/pull/79

  • https://github.com/PaddlePaddle/VisualDL/pull/78

  • https://github.com/PaddlePaddle/VisualDL/pull/68

  • https://github.com/PaddlePaddle/VisualDL/pull/64

  • https://github.com/PaddlePaddle/VisualDL/pull/67

  • https://github.com/PaddlePaddle/VisualDL/pull/58#pullrequestreview-86573129

• Adapt warp-CTC grad op for gradient checking
  • https://github.com/PaddlePaddle/Paddle/pull/7414
• Add python API for warp-CTC op
  • https://github.com/PaddlePaddle/Paddle/pull/7438
• Review:
  • https://github.com/PaddlePaddle/Paddle/pull/7325

• Add learning curve plot script

  https://github.com/PaddlePaddle/Paddle/pull/7311

• profile the model vgg16 and mnist

Memory optimization on Fluid:

• https://github.com/PaddlePaddle/Paddle/projects/38

Memory optimization result(by bytes):

Model	Before	After	Saving
VGG	1729499136	1132920832	34.5%
Resnet	1277345792	666173440	47.8%

Memory optimization transpiler:

• add memory optimization design doc
  • https://github.com/PaddlePaddle/Paddle/pull/7206
• add simple memory optimization transpiler
  • https://github.com/PaddlePaddle/Paddle/pull/7356
• Refine memory optimization transpiler
  • https://github.com/PaddlePaddle/Paddle/pull/7394

Memory optimization Demo:

• Memory optimization for fit a line demo
  • https://github.com/PaddlePaddle/Paddle/pull/7321
• memory optimization for vgg and resnet in image classification demo
  • https://github.com/PaddlePaddle/Paddle/pull/7385

Fix and enhance:

• add general memory usage interface for both CPU/CUDA
  • https://github.com/PaddlePaddle/Paddle/pull/7352
• refine batch norm python layer
  • https://github.com/PaddlePaddle/Paddle/pull/7348
• set stop gradient for mask in dropout layer
  • https://github.com/PaddlePaddle/Paddle/pull/7390

• Add and merge sequence erase operator
  • https://github.com/PaddlePaddle/Paddle/pull/7325
• Update and merge edit distance operator
  • https://github.com/PaddlePaddle/Paddle/pull/5300
• Update and merge the parsing part of profiler for Fluid
  • https://github.com/PaddlePaddle/Paddle/pull/7043
• Code Review:
  • https://github.com/PaddlePaddle/Paddle/pull/7290

• OpenBLAS:
  • update Openblas benchmark: https://github.com/PaddlePaddle/Paddle/pull/7295/
  • auto set openblas env in v1/v2: https://github.com/PaddlePaddle/Paddle/pull/7397
• inference: [WIP] Create an independent project, like capi dense example to verify the link to fluid library.
• fix compile error in profiler.cc: https://github.com/PaddlePaddle/Paddle/pull/7290
• code review:
  • Stop gradient when pool_type=='max': https://github.com/PaddlePaddle/Paddle/pull/7284
  • Add grad_op_maker for sequence_pool: https://github.com/PaddlePaddle/Paddle/pull/7363
  • Add flag use_mkl_packed: https://github.com/PaddlePaddle/Paddle/pull/7183/
  • capi pacakge: https://github.com/PaddlePaddle/Paddle/pull/7237, https://github.com/PaddlePaddle/Paddle/pull/7255
  • fix ds2 of MKLDNNLayer issue: https://github.com/PaddlePaddle/Paddle/pull/7395

• Error clip design doc and implementation:
  • https://github.com/PaddlePaddle/Paddle/pull/7213
• Test error clip on MNIST
  • https://github.com/PaddlePaddle/Paddle/pull/7431
• Review of adversarial sample generating model from Baidu security lab
  • https://github.com/PaddlePaddle/Paddle/pull/7270
• WhileOp code clean
  • https://github.com/PaddlePaddle/Paddle/pull/7361
• Other reviews:
  • https://github.com/PaddlePaddle/Paddle/pull/7371
  • https://github.com/PaddlePaddle/Paddle/pull/7176
  • https://github.com/PaddlePaddle/Paddle/pull/7247
  • https://github.com/PaddlePaddle/Paddle/pull/7269
  • https://github.com/PaddlePaddle/Paddle/pull/7278
  • https://github.com/PaddlePaddle/Paddle/pull/7363

• Fix adagrad op by removing broadcast of Eigen.
  • https://github.com/PaddlePaddle/Paddle/pull/7203
• Fix the docstring of 'filter groups' in img_conv_layer.
  • https://github.com/PaddlePaddle/Paddle/pull/7301
• Enhance learning curve plot script.
  • https://github.com/PaddlePaddle/Paddle/pull/7311
• ResNet-50 performance comparison with TensorFlow.

• Follow comments to merge the C-API documentation.

  • https://github.com/PaddlePaddle/Paddle/pull/6596/commits/4d4df084a70276ecb230fed056aaa1b454eaf203
  • https://github.com/PaddlePaddle/Paddle/pull/6596/commits/476c6173513715d493c8ec216cdeff28c2c8f798

• issues about plans for the NMT task.

  • To implement NMT, some enhancements are required to to current implementation:
    1. Support padding_idx in the lookup_table_op: https://github.com/PaddlePaddle/Paddle/issues/7309
    2. Enhance matmul_op to support 4-D inputs: https://github.com/PaddlePaddle/Paddle/issues/7319
    3. A general purpose normalization operator: https://github.com/PaddlePaddle/Paddle/issues/7350
  • TODO list for RNN search: https://github.com/PaddlePaddle/Paddle/issues/7357
  • TODO list for Transformer: https://github.com/PaddlePaddle/Paddle/issues/7355
  • TODO list for ConvS2S: https://github.com/PaddlePaddle/Paddle/issues/7353

• Enhance reorder_lod_tensor_by_rank_op to support reorderring Tensor(Merged)
  • https://github.com/PaddlePaddle/Paddle/pull/7251
• convS2S related paper and code reading

• PR
  • https://github.com/PaddlePaddle/Paddle/pull/7429 book chapter 6, distributed version
• Issue
  • https://github.com/PaddlePaddle/Paddle/issues/7422
  • Build issue fix in Mac: https://github.com/PaddlePaddle/Paddle/issues/6786#issuecomment-355211203

• Mobile
  • Follow up OCR mobile inference library switch to paddle mobile inference library. At present, paddle mobile inference library is better than ocr's inference library in both memory usage and computational performance.
  • https://github.com/PaddlePaddle/Paddle/pull/7441

• Refine broadcast_scale layer
  • https://github.com/PaddlePaddle/Paddle/pull/7188
• Implement SE-ResNeXt for image classification
  • https://github.com/PaddlePaddle/models/pull/570
• Minor doc fixes
  • https://github.com/PaddlePaddle/Paddle/pull/7241
  • https://github.com/PaddlePaddle/models/pull/571

• Image
  • Work plan on image: https://github.com/PaddlePaddle/Paddle/issues/7253
  • SSD design on Fluid: https://github.com/PaddlePaddle/Paddle/issues/7402
• Code Review:
  • Fix the docstring of 'filter groups' in img_conv_layer. https://github.com/PaddlePaddle/Paddle/pull/7301
  • Add COWPtr and its unittest. https://github.com/PaddlePaddle/Paddle/pull/7240
  • Correctly handle lod information of image operators: https://github.com/PaddlePaddle/Paddle/pull/7219
  • Release the middle layer's output memory: https://github.com/PaddlePaddle/Paddle/pull/7159
  • Add the parsing part for the profiling tool: https://github.com/PaddlePaddle/Paddle/pull/7043
  • Add edit distance operator: https://github.com/PaddlePaddle/Paddle/pull/5300#pullrequestreview-87720377

• Finished end-to-end demo of how to integrate VisualDL into MXNet while training MNIST and visualize the accuracy using VisualDL’s Scalar component https://github.com/PaddlePaddle/VisualDL/tree/demo_mxnet
• Recorded a 5-min video for Baidu USA
• [WIP] Integrate VisualDL’s Graph into MXNet (blocked by onnx-mxnet conversion)
• Translated PaddlePaddle brochure

• update prior box operator
  • https://github.com/PaddlePaddle/Paddle/pull/6150
• ssd discuss
  • https://github.com/PaddlePaddle/Paddle/issues/7402

• Fluid Heterogeneous device support
  • runtime switch for sync GPU operators
    • https://github.com/PaddlePaddle/Paddle/pull/7384
  • fix init device pools
    • https://github.com/PaddlePaddle/Paddle/pull/7345
  • add layout infer
    • https://github.com/PaddlePaddle/Paddle/pull/7233
  • refine CUDNNDeviceContext
    • https://github.com/PaddlePaddle/Paddle/pull/7207
• benchmark - add cifar10 resnet - https://github.com/dzhwinter/benchmark/pull/42 - refine benchmark repository - https://github.com/dzhwinter/benchmark/pull/41
• Reviews
  • benchmark
    • https://github.com/dzhwinter/benchmark/pull/35
    • https://github.com/dzhwinter/benchmark/pull/31
    • https://github.com/dzhwinter/benchmark/pull/30
    • https://github.com/dzhwinter/benchmark/pull/25

• Add static_input [WIP]
  https://github.com/PaddlePaddle/Paddle/pull/7434
• Fix sequence_pool
  https://github.com/PaddlePaddle/Paddle/pull/7363
  https://github.com/PaddlePaddle/Paddle/pull/7284
• Bug fix for backward
  https://github.com/PaddlePaddle/Paddle/pull/7278
• Enhance shrink_memory_op
  https://github.com/PaddlePaddle/Paddle/pull/7176
• Code Review
  https://github.com/dzhwinter/benchmark/pull/41#pullrequestreview-87140258
  https://github.com/dzhwinter/benchmark/pull/45#pullrequestreview-87408295
  https://github.com/dzhwinter/benchmark/pull/42#pullrequestreview-87719102

• parallel_do
  • cpu: https://github.com/PaddlePaddle/Paddle/pull/6730
  • gpu: https://github.com/PaddlePaddle/Paddle/pull/7293

• update distributed design doc: https://github.com/PaddlePaddle/Paddle/pull/5776
• variable split to multi-servers: https://github.com/PaddlePaddle/Paddle/pull/7249
• reviews https://github.com/PaddlePaddle/Paddle/pull/7133, https://github.com/PaddlePaddle/Paddle/pull/7276
• distributed benchmark (DOING)

PR

• Add design doc: concurrent data transfer and kernel execution
  • https://github.com/PaddlePaddle/Paddle/pull/7276
• [WIP]Add Channel
  • https://github.com/PaddlePaddle/Paddle/pull/7442
• make mnist.py enable running in GPU
  • https://github.com/dzhwinter/benchmark/pull/45

Code Review

• Feature/use cudnn
  • https://github.com/PaddlePaddle/Paddle/pull/7141
• fix layout transform
  • https://github.com/PaddlePaddle/Paddle/pull/7149

• Inference
  • Format proposal for saving models in Fluid: https://github.com/PaddlePaddle/Paddle/issues/7329
  • Design of InferenceProgramDesc: https://github.com/PaddlePaddle/Paddle/issues/7328
  • Work on designing Inference to remove Pickle: https://github.com/PaddlePaddle/Paddle/issues/7221
  • Survey on how TensorFlow and Caffe2 prefer storing models: https://github.com/PaddlePaddle/Paddle/issues/7222
• PR review
  • https://github.com/PaddlePaddle/Paddle/pull/7315/
  • https://github.com/PaddlePaddle/Paddle/pull/7339
• NMT
  • Read and discuss the Transformers paper (https://arxiv.org/abs/1706.03762) for NMT implementation in Fluid
  • TODO list for RNN search: https://github.com/PaddlePaddle/Paddle/issues/7357
  • TODO list for Transformer: https://github.com/PaddlePaddle/Paddle/issues/7355
• Fixing the html to not render io file in Fluid: https://github.com/PaddlePaddle/Paddle/issues/7335

• Inference

  • Design and code for InferenceDesc https://github.com/PaddlePaddle/Paddle/pull/7367
  • https://github.com/PaddlePaddle/Paddle/pull/7315
  • Pytorch Survey https://github.com/PaddlePaddle/Paddle/issues/7224
  • Plan Inference ProgramDesc optimization https://github.com/PaddlePaddle/Paddle/issues/7417

• NMT

  • Study Transformers paper (https://arxiv.org/abs/1706.03762) for NMT implementation in Fluid
  • TODO list for RNN search: https://github.com/PaddlePaddle/Paddle/issues/7357
  • TODO list for Transformer: https://github.com/PaddlePaddle/Paddle/issues/7355

• Refine Memory optimization Doc https://github.com/PaddlePaddle/Paddle/pull/7420

• Fix issues https://github.com/PaddlePaddle/Paddle/issues/7418 and https://github.com/PaddlePaddle/Paddle/issues/6957 in Elementwise Ops - PR https://github.com/PaddlePaddle/Paddle/pull/7445

• https://github.com/PaddlePaddle/Paddle/pull/7178/files#r160868932

• Fix Bias attr in Batch norm https://github.com/PaddlePaddle/Paddle/pull/7449

• PR Review - https://github.com/PaddlePaddle/Paddle/pull/7457#pullrequestreview-88257754

• Fluid
  • List the operators that will be used in inference (for Movidius)
  • Add design doc of inference API for Fluid
    • https://github.com/PaddlePaddle/Paddle/pull/7315
  • Port WarpCTC Operator
    • https://github.com/PaddlePaddle/Paddle/pull/5107
  • Review and discussion about storing format
    • https://github.com/PaddlePaddle/Paddle/pull/7339#discussion_r160593182
    • https://github.com/PaddlePaddle/Paddle/issues/7329#issuecomment-356226703
    • https://github.com/PaddlePaddle/Paddle/issues/7328#issuecomment-356232961
    • https://github.com/PaddlePaddle/Paddle/issues/7222#issuecomment-355461202

• Add link from home page to book (https://github.com/PaddlePaddle/PaddlePaddle.org/pull/369)
• Research and planning for Master process dashboard (https://github.com/PaddlePaddle/Paddle/projects/41)

• Distributed Training
  • understanding code
  • reviews:
    • https://github.com/PaddlePaddle/Paddle/pull/7133#pullrequestreview-86479356
    • https://github.com/PaddlePaddle/Paddle/pull/7410#pullrequestreview-88026057
    • https://github.com/PaddlePaddle/Paddle/pull/7429#pullrequestreview-88033132
  • Distributed label semantic role PR: https://github.com/PaddlePaddle/Paddle/pull/7425

• Fluid

  • Mixed Device

    1. mixed device design https://github.com/PaddlePaddle/Paddle/pull/7412
    2. Cpu gpu transform merged https://github.com/PaddlePaddle/Paddle/pull/7085
    3. add back priority https://github.com/PaddlePaddle/Paddle/pull/7294
    4. reorganize data transform related code https://github.com/PaddlePaddle/Paddle/pull/7391
    5. [WIP]init complete data layout transform https://github.com/PaddlePaddle/Paddle/pull/7440

  • beam search

    1. [WIP]Nmt model https://github.com/PaddlePaddle/Paddle/pull/7340
    2. add lod tensor ToAbsOffset test https://github.com/PaddlePaddle/Paddle/pull/7372
    3. Topk share lod https://github.com/PaddlePaddle/Paddle/pull/7373
    4. change data type of beam_search op https://github.com/PaddlePaddle/Paddle/pull/7374
    5. fix GetDims bug https://github.com/PaddlePaddle/Paddle/pull/7340

  • Review

    1. set stop gradient for mask in dropout layer: https://github.com/PaddlePaddle/Paddle/pull/7390
    2. refine batch norm python layer: https://github.com/PaddlePaddle/Paddle/pull/7348
    3. Add Fluid Compiler design doc https://github.com/PaddlePaddle/Paddle/pull/7178

PR:

• Remove pickling tools for inference example: https://github.com/PaddlePaddle/Paddle/pull/7262
• Fluid documentation: https://github.com/PaddlePaddle/Paddle/pull/7151
• Fluid documentation: https://github.com/PaddlePaddle/Paddle/pull/7274

PR review:

• Fluid compiler: https://github.com/PaddlePaddle/Paddle/pull/7178
• Inference design: https://github.com/PaddlePaddle/Paddle/pull/7315

Others:

• NMT project: reading and discussing transformer paper
• TODOlist for NMT: https://github.com/PaddlePaddle/Paddle/issues/7355
• Understanding fluid code for inference (https://github.com/PaddlePaddle/Paddle/issues/7145)

• Inference

  • Discussions on the Fluid inference framework design
  • Remove dependency on Pickle https://github.com/PaddlePaddle/Paddle/pull/7339

• Document and User Experience

  • https://github.com/PaddlePaddle/Paddle/pull/7448
  • https://github.com/PaddlePaddle/Paddle/issues/7376
  • https://github.com/PaddlePaddle/Paddle/issues/7332
  • https://github.com/PaddlePaddle/Paddle/issues/7377

• NMT

  • Read and discuss the transformer paper
  • To-do list for transformer: https://github.com/PaddlePaddle/Paddle/issues/7355

• Review

  • https://github.com/PaddlePaddle/Paddle/pull/7262#pullrequestreview-87106170
  • https://github.com/PaddlePaddle/Paddle/pull/7315#discussion_r160285268
  • https://github.com/PaddlePaddle/Paddle/issues/7329#issuecomment-356181056

• https://github.com/PaddlePaddle/PaddlePaddle.org/pull/371 Initial setup for the VisualDL subdomain homepage
• Register visualdl.ai domain
• Paddleboard project direction discussion and exploration on debugger.

• Paddleboard visualization research and prototyping
• VisualDL design specification building, logo design