2018 01 17 - PaddlePaddle/Paddle GitHub Wiki

• Inference library installation directory structure: https://github.com/PaddlePaddle/Paddle/pull/7572#pullrequestreview-89201869
• Rename WITH_C_API into PADDLE_BUILD_INFERENCE_LIB: https://github.com/PaddlePaddle/Paddle/issues/7578
• Better directory structure: https://github.com/PaddlePaddle/Paddle/issues/7542
• Read Dockerfile from stdin: https://github.com/PaddlePaddle/Paddle/issues/7502
• Restructure go/api package: https://github.com/PaddlePaddle/Paddle/issues/7496
• Clean up EDL controller: https://github.com/PaddlePaddle/Paddle/issues/7494
• Restructure controller (edl) pacakge: https://github.com/PaddlePaddle/Paddle/issues/7493
• Rename dir controller into edl: https://github.com/PaddlePaddle/Paddle/issues/7492
• Better function register resource type: https://github.com/PaddlePaddle/Paddle/issues/7491
• The imperative programming paradigm: https://github.com/PaddlePaddle/Paddle/issues/7464
• Move content in the buildtools repo into paddle repo: https://github.com/PaddlePaddle/Paddle/issues/7326
• Requiring adding issue to describe PRs:
  • https://github.com/PaddlePaddle/Paddle/pull/7605#pullrequestreview-89570801
  • https://github.com/PaddlePaddle/Paddle/pull/7606#pullrequestreview-89572298
  • https://github.com/PaddlePaddle/Paddle/pull/7597#pullrequestreview-89573043
  • https://github.com/PaddlePaddle/Paddle/pull/7573#pullrequestreview-89573885

• PR
  • fix display error of C-API doc:
    • https://github.com/PaddlePaddle/Paddle/pull/7475
  • Update comments for reorder_lod_tansor_by_rank_op and shrink_rnn_memory_op:
    • https://github.com/PaddlePaddle/Paddle/pull/7457
  • wrapper for element-wise math operators:
    • https://github.com/PaddlePaddle/Paddle/pull/7485
  • wrapper for l2 normalize: https://github.com/PaddlePaddle/Paddle/pull/7574
  • small fix: https://github.com/PaddlePaddle/Paddle/pull/7609
  • WIP: prepare dataset for WMT16, and merge it to paddel.dataset

2018-01-18

• fluid distributed performance test and enhancements:
  • https://github.com/PaddlePaddle/Paddle/pull/7608
  • https://github.com/PaddlePaddle/Paddle/pull/7539
  • https://github.com/PaddlePaddle/Paddle/pull/7573
  • need to update transpiler https://github.com/PaddlePaddle/Paddle/issues/7583
• code coverage update:
  • https://github.com/PaddlePaddle/Paddle/issues/7569
  • https://github.com/PaddlePaddle/Paddle/issues/7559
  • https://github.com/PaddlePaddle/Paddle/issues/7551
  • https://github.com/PaddlePaddle/Paddle/issues/7546

• Fluid distributed training
  • Support GPU[Merged], https://github.com/PaddlePaddle/Paddle/pull/7537
  • Benchamrk design doc[Merged], https://github.com/PaddlePaddle/Paddle/pull/7410
  • Split SelectedRows Op[Reviewing], https://github.com/PaddlePaddle/Paddle/pull/7604
• Code Review
  • https://github.com/PaddlePaddle/Paddle/issues/7465#issuecomment-357604855
  • https://github.com/PaddlePaddle/Paddle/pull/7276#pullrequestreview-88994428

• inference:
  • compile and install the shared library of fluid inference: https://github.com/PaddlePaddle/Paddle/pull/7572
  • a simple independent example project: https://github.com/luotao1/fluid_inference_example
• Intel:
  • collect the Intel demand from 2018's 10 aspects: https://github.com/PaddlePaddle/Paddle/issues/7561
  • code review: https://github.com/PaddlePaddle/Paddle/pull/7592
• doc:
  • add how to update whl in build_from_source.md: https://github.com/PaddlePaddle/Paddle/pull/7471
  • add a brief introduction of MKL-DNN work: https://github.com/PaddlePaddle/Paddle/pull/7518
  • better directory structure: https://github.com/PaddlePaddle/Paddle/issues/7542
  • code review:
    • https://github.com/PaddlePaddle/Paddle/pull/7470
    • https://github.com/PaddlePaddle/Paddle/pull/7445
    • https://github.com/PaddlePaddle/Paddle/pull/7475/

• OCR CTC/Attention
  • Adapt Warp-CTC grad op for gradient checking
    • https://github.com/PaddlePaddle/Paddle/pull/7414
  • Add CTC align op
    • https://github.com/PaddlePaddle/Paddle/pull/7527
  • Add python API for Warp-CTC op
    • https://github.com/PaddlePaddle/Paddle/pull/7438
  • Restructure im2sequence op
    • https://github.com/PaddlePaddle/Paddle/pull/4866
  • Add python API for im2sequence op
    • https://github.com/PaddlePaddle/Paddle/pull/7595
  • Add python wrapper for CTC evaluator[WIP]

• PR review:
  • Inference design doc: https://github.com/PaddlePaddle/Paddle/pull/7315
  • Inference design, further iteration: https://github.com/PaddlePaddle/Paddle/pull/7315
• Work on paddlepaddle.org:
  • Render IO in fluid: https://github.com/PaddlePaddle/Paddle/issues/7335
  • Build fails: https://github.com/PaddlePaddle/Paddle/issues/7459
  • PR for fixing the rendering the IO file : https://github.com/PaddlePaddle/Paddle/pull/7470
• Add documentation for other functions of IO: https://github.com/PaddlePaddle/Paddle/issues/7495
• Polish executor: https://github.com/PaddlePaddle/Paddle/pull/7499
• Study Select case in Go: https://github.com/PaddlePaddle/Paddle/issues/7620
• Will start studying more about Distributed training.

• Helped fixing CI issue with Helin

• Helped with CI machine CPU upgrade and new server/rack procurement

• PR

  • https://github.com/PaddlePaddle/Paddle/pull/7469 book chapter 1, distributed version
  • https://github.com/PaddlePaddle/Paddle/pull/7619 Fluid dist train guide

• Issue

  • https://github.com/PaddlePaddle/Paddle/issues/7465 Pserver tracking trainer number issue
  • https://github.com/PaddlePaddle/Paddle/issues/7627 distributed fluid build issue

• Fix async bugs:https://github.com/PaddlePaddle/Paddle/pull/7435
• code review:
  • https://github.com/PaddlePaddle/Paddle/pull/7573
  • https://github.com/PaddlePaddle/Paddle/pull/7529
  • https://github.com/PaddlePaddle/Paddle/pull/7593
• add image classification distribution version: https://github.com/PaddlePaddle/Paddle/pull/7544

• Implement fluid version of SE-ResNeXt-152
• Implement data augmentation and reader for training with ImageNet
  • https://github.com/PaddlePaddle/models/pull/577

• Profile the model vgg16 and mnist

• Update FAQ

  https://github.com/PaddlePaddle/Paddle/pull/7584

• PR Review

  https://github.com/dzhwinter/benchmark/pull/49

  https://github.com/dzhwinter/benchmark/pull/56

Merged PR:

• Add unittests for Parallel.for
  • https://github.com/PaddlePaddle/Paddle/pull/7474

Several enhancements:

• https://github.com/PaddlePaddle/Paddle/pull/7453
• https://github.com/PaddlePaddle/Paddle/pull/7587

Survey:

• [Compiler] Halide
  • http://halide-lang.org/
• [Cluster] Ray project
  • https://github.com/ray-project/ray
  • Use ray to construct a pserver/trainer
• [Compiler] TVM
  • Doing

Work in process:

• [Performance Tuning] add performance tuning for parallel.for

• [BugFix] SparseMatrix.resize()

Profiling:

• ResNet_cifar10 performance comparison with TensorFlow on cifar10 dataset.

• [WIP]Fix adagrad op when using GPU.

• Distributed Fluid Framework
  • Implement Book chapter 02 in distributed framework https://github.com/PaddlePaddle/Paddle/pull/7629
  • Fix send_op https://github.com/PaddlePaddle/Paddle/pull/7625
  • [WIP] Move regularization and error clipping to parameter server side https://github.com/PaddlePaddle/Paddle/issues/7432
• Concurrent Programming in Fluid
  • Survey concurrent programming paradigms in Go https://talks.golang.org/2012/concurrency.slide#1 https://talks.golang.org/2013/advconc.slide#1
• Add Code of conduct https://github.com/PaddlePaddle/Paddle/pull/7579
• Fixing the rendering for Reduce operators https://github.com/PaddlePaddle/Paddle/pull/7622
• Fix Batch Norm Initialization https://github.com/PaddlePaddle/Paddle/pull/7449
• Fix Block design doc https://github.com/PaddlePaddle/Paddle/pull/7460
• Fix concepts in VarDesc design https://github.com/PaddlePaddle/Paddle/pull/7462
• Code Review
  • https://github.com/PaddlePaddle/Paddle/pull/7626#pullrequestreview-89634743
  • https://github.com/PaddlePaddle/Paddle/pull/6920#pullrequestreview-89613407
  • https://github.com/PaddlePaddle/Paddle/pull/7457#pullrequestreview-88257754
  • https://github.com/PaddlePaddle/Paddle/pull/7619#pullrequestreview-89573331
  • https://github.com/PaddlePaddle/Paddle/pull/7645#issuecomment-358558862

• EDL clean up
  • Restructure EDL package
  • Fix unit test, make test fail immediately on error
  • Add develop branch build status badge
• Concurrent Programming learning and discussion
• Distributed Fluid
  • Recv OP: use BlockDesc* instread of ProgramDesc proto as Attribute
• Attend ONNX meeting

• Error clip test on MNIST
  • https://github.com/PaddlePaddle/Paddle/pull/7431
• Gradient clip by global norm
  • elementwise_max/min op: https://github.com/PaddlePaddle/Paddle/pull/7538
  • elementwise op scalar input supporting: https://github.com/PaddlePaddle/Paddle/pull/7593
  • gradient clip by global norm: https://github.com/PaddlePaddle/Paddle/pull/7637
• Fixes and updates:
  • https://github.com/PaddlePaddle/Paddle/pull/7586
  • https://github.com/PaddlePaddle/Paddle/pull/7529
• review a new algorithm from Baidu security lab:
  • https://github.com/PaddlePaddle/Paddle/pull/7454

• paddle

  • open,bugfix/beam_search_op
  • merged,feature/relative/abstract LoD check
  • merged,feature/print_op

• VisualDL

  https://github.com/PaddlePaddle/VisualDL/pull/87 https://github.com/PaddlePaddle/VisualDL/pull/92 https://github.com/PaddlePaddle/VisualDL/pull/99 https://github.com/PaddlePaddle/VisualDL/pull/100 https://github.com/PaddlePaddle/VisualDL/pull/104 https://github.com/PaddlePaddle/VisualDL/pull/105 https://github.com/PaddlePaddle/VisualDL/pull/115 https://github.com/PaddlePaddle/VisualDL/pull/120 https://github.com/PaddlePaddle/VisualDL/pull/132 https://github.com/PaddlePaddle/VisualDL/pull/133 https://github.com/PaddlePaddle/VisualDL/pull/134 https://github.com/PaddlePaddle/VisualDL/pull/135 https://github.com/PaddlePaddle/VisualDL/pull/136 https://github.com/PaddlePaddle/VisualDL/pull/137 https://github.com/PaddlePaddle/VisualDL/pull/138 https://github.com/PaddlePaddle/VisualDL/pull/139 https://github.com/PaddlePaddle/VisualDL/pull/140 https://github.com/PaddlePaddle/VisualDL/pull/144 https://github.com/PaddlePaddle/VisualDL/pull/150 https://github.com/PaddlePaddle/VisualDL/pull/151 https://github.com/PaddlePaddle/VisualDL/pull/152 https://github.com/PaddlePaddle/VisualDL/pull/156 https://github.com/PaddlePaddle/VisualDL/pull/157 https://github.com/PaddlePaddle/VisualDL/pull/160 https://github.com/PaddlePaddle/VisualDL/pull/161 https://github.com/PaddlePaddle/VisualDL/pull/166 https://github.com/PaddlePaddle/VisualDL/pull/169 https://github.com/PaddlePaddle/VisualDL/pull/171 https://github.com/PaddlePaddle/VisualDL/pull/174

• Decouple ext scorer init & inference & decoding for the convenience of tuning
  • https://github.com/PaddlePaddle/DeepSpeech/pull/122
• Enhance GPU kernel of sequence erase op
• https://github.com/PaddlePaddle/Paddle/pull/7603
• Add python doc for dynamic_lstm
• https://github.com/PaddlePaddle/Paddle/pull/7640
• [WIP] Implement LSTM with projection layer

Code Review:

• enbale auto set env of V1 on Mac
  • https://github.com/PaddlePaddle/Paddle/pull/7413
• ctc greedy decoder (align)
  • https://github.com/PaddlePaddle/Paddle/pull/7527
• Refine profiler and expose to Python
  • https://github.com/PaddlePaddle/Paddle/pull/7576
• Refine vgg16
  • https://github.com/dzhwinter/benchmark/pull/53

• add iou similarity operator: https://github.com/PaddlePaddle/Paddle/pull/7566
• Add mine_hard_examples_op(WIP):https://github.com/PaddlePaddle/Paddle/issues/7639
• Discuss with Vision team

• Exposing use_cudnn
  • https://github.com/PaddlePaddle/Paddle/pull/7536
• Refine fluid vgg16
  • https://github.com/dzhwinter/benchmark/pull/53
• Fix bug - minst.py
  • https://github.com/dzhwinter/benchmark/pull/52
• Refine tf mnist
  • https://github.com/dzhwinter/benchmark/pull/47
• [WIP]Enhance matmul_op to support 4-D inputs
  • https://github.com/PaddlePaddle/Paddle/issues/7319
• Refine conv2d_transpose layer doc
  • https://github.com/PaddlePaddle/Paddle/pull/6920
• update:concurrent data transfer and kernel execution
  • https://github.com/PaddlePaddle/Paddle/pull/7276

Code Review

• Make elementwise_op supporting scalar input Y (#7593)
  • https://github.com/PaddlePaddle/Paddle/pull/7593
• Refine profiler and expose to Python
  • https://github.com/PaddlePaddle/Paddle/pull/7576
• "fix gpu init"
  • https://github.com/PaddlePaddle/Paddle/pull/7528

• PR
  • Add python wrapper for gated linear unit and split_op:
    • https://github.com/PaddlePaddle/Paddle/pull/7525
  • Add Python wrapper for dot-product-attention and matmul_op:
    • https://github.com/PaddlePaddle/Paddle/pull/7602
• Review
  • https://github.com/PaddlePaddle/Paddle/pull/7485
  • https://github.com/PaddlePaddle/Paddle/pull/7574
  • https://github.com/PaddlePaddle/Paddle/pull/7457

• Finish training part for fluid's seq2seq
  https://github.com/PaddlePaddle/Paddle/pull/7540
  https://github.com/PaddlePaddle/Paddle/pull/7517
  https://github.com/PaddlePaddle/Paddle/pull/7434
  https://github.com/dzhwinter/benchmark/pull/63
  https://github.com/dzhwinter/benchmark/pull/58
  https://github.com/dzhwinter/benchmark/pull/56

• Code Review
  https://github.com/PaddlePaddle/Paddle/pull/7406#pullrequestreview-88066893
  https://github.com/dzhwinter/benchmark/pull/42#pullrequestreview-88738786
  https://github.com/PaddlePaddle/Paddle/pull/7521#pullrequestreview-88738707
  https://github.com/PaddlePaddle/Paddle/pull/6150#pullrequestreview-88802234
  https://github.com/PaddlePaddle/Paddle/pull/4866#pullrequestreview-89319460

• Image
  • SSD on Fluid:
    • Review the design with other colleagues.
    • Issue: https://github.com/PaddlePaddle/Paddle/issues/7488
    • Develop bipartite graph match operator. https://github.com/PaddlePaddle/Paddle/issues/7615
  • Code Review:
    • Add CTC decode: https://github.com/PaddlePaddle/Paddle/pull/7527
    • Add python API for warp-ctc op: https://github.com/PaddlePaddle/Paddle/pull/7438
    • Adapt warpctc grad op for gradient checking: https://github.com/PaddlePaddle/Paddle/pull/7414
    • prior box operator for ssd: https://github.com/PaddlePaddle/Paddle/pulls/wanghaox
    • other PRs
• Profiling tool:
  • Refine profiler and expose to Python: https://github.com/PaddlePaddle/Paddle/pull/7576
  • Upgrade pybind and refine pybind/protobuf.cc : https://github.com/PaddlePaddle/Paddle/pull/7563

• Fluid

  • Mixed Device

    1. things need to do to complete multi device transform https://github.com/PaddlePaddle/Paddle/issues/7589
    2. complete data layout transform https://github.com/PaddlePaddle/Paddle/pull/7440
    3. update switch kernel documentation https://github.com/PaddlePaddle/Paddle/pull/7597

  • machine translation model

    1. Nmt model https://github.com/PaddlePaddle/Paddle/pull/7340

  • BUG fix & code optimize

    1. fix while_grad_op first step loss lod problem https://github.com/PaddlePaddle/Paddle/pull/7490
    2. change DEVICE_TYPE in op_registry to LIBRARY_TYPE https://github.com/PaddlePaddle/Paddle/pull/7588
    3. Add get set lod for infershape https://github.com/PaddlePaddle/Paddle/pull/7606
    4. clean lod_tensor_test https://github.com/PaddlePaddle/Paddle/pull/7618

  • Review

    1. Exposing use_cudnn https://github.com/PaddlePaddle/Paddle/pull/7536
    2. Bugfix/beamsearch op https://github.com/PaddlePaddle/Paddle/pull/7611
    3. Fix typo in batch norm bias initialization https://github.com/PaddlePaddle/Paddle/pull/7449

• VisualDL

  1. release the first version.

  https://github.com/PaddlePaddle/VisualDL/pull/114 https://github.com/PaddlePaddle/VisualDL/pull/117 https://github.com/PaddlePaddle/VisualDL/pull/119 https://github.com/PaddlePaddle/VisualDL/pull/125 https://github.com/PaddlePaddle/VisualDL/pull/143 https://github.com/PaddlePaddle/VisualDL/pull/165 https://github.com/PaddlePaddle/VisualDL/pull/168 https://github.com/PaddlePaddle/VisualDL/pull/107 https://github.com/PaddlePaddle/VisualDL/pull/91

• Fluid
  • [Update] Add design doc of inference API for fluid
    • https://github.com/PaddlePaddle/Paddle/pull/7315
  • [issue] How about implement another Run() in framework::Executor
    • https://github.com/PaddlePaddle/Paddle/issues/7610
  • [Review]
    • compile and install the shared library of fluid inference: https://github.com/PaddlePaddle/Paddle/pull/7572#pullrequestreview-89383880

• VisualDL Launch
  • added MXNet demo:
    • https://github.com/PaddlePaddle/VisualDL/pull/116
    • https://github.com/PaddlePaddle/VisualDL/pull/142
  • VisualDL website content:
    • https://github.com/PaddlePaddle/VisualDL/pull/158
    • https://github.com/PaddlePaddle/VisualDL/pull/162
    • https://github.com/PaddlePaddle/VisualDL/pull/163
    • https://github.com/PaddlePaddle/VisualDL/pull/167
  • VisualDL launch communication with Baidu USA Product Team
  • VisualDL Docker : https://hub.docker.com/r/daminglu/visualdl_w_mock_data/
  • [WIP] PyTorch Demo

• Inference

  • Discussions on the Fluid inference framework design
  • Adding feed/fetch ops to programDesc https://github.com/PaddlePaddle/Paddle/pull/7636
  • Storing formats https://github.com/PaddlePaddle/Paddle/issues/7580
  • Discussions https://github.com/PaddlePaddle/Paddle/pull/7315#discussion_r161934210
  • Executor https://github.com/PaddlePaddle/Paddle/issues/7610

• Document

  • https://github.com/PaddlePaddle/models/pull/574

• PR review:

  • https://github.com/PaddlePaddle/Paddle/pull/7315

Inference:

• Design for input protobuf: https://github.com/PaddlePaddle/Paddle/issues/7580
• Storing format: https://github.com/PaddlePaddle/Paddle/issues/7329
• Discussion about several aspects of inference framework design: - https://github.com/PaddlePaddle/Paddle/issues/7314 - https://github.com/PaddlePaddle/Paddle/issues/7610

PR:

• Batch norm: design doc correction: https://github.com/PaddlePaddle/Paddle/pull/7543
• (won't be pursued further) layer norm: https://github.com/PaddlePaddle/Paddle/pull/7560

• Designed visualdl.paddlepaddle.org
• Started designing summary operator, studied necessary Paddle code for the same

• Initial setup for the visual DL homepage. (https://github.com/PaddlePaddle/PaddlePaddle.org/pull/371)
• Update logos and image assets on VisualDL (https://github.com/PaddlePaddle/PaddlePaddle.org/pull/372)
• Research on Interactive Debugger