2018 04 18 - PaddlePaddle/Paddle GitHub Wiki

PR: https://github.com/PaddlePaddle/Paddle/pull/10038

• Refactor all the build related shell scripts, centralize the parameter control into one script file and run different build jobs using options.
• Write scripts to control docker container startup option.

1. Unify REGISTER_OP and REGISTER_OPERATOR
   1. https://github.com/PaddlePaddle/Paddle/pull/9911
   2. Update new_op readme: https://github.com/PaddlePaddle/Paddle/pull/10031
   3. https://github.com/PaddlePaddle/Paddle/pull/10034
2. Review
   1. https://github.com/PaddlePaddle/Paddle/pull/10039
   2. https://github.com/PaddlePaddle/Paddle/pull/9911
   3. https://github.com/PaddlePaddle/Paddle/pull/9978
   4. https://github.com/PaddlePaddle/Paddle/pull/9959
   5. https://github.com/PaddlePaddle/Paddle/pull/9937

• inference:
  • fuse batch norm for conv operator with bias: https://github.com/PaddlePaddle/Paddle/pull/9792
  • auto find tensorrt library and install in user root: https://github.com/PaddlePaddle/Paddle/pull/9958
  • [WIP] tensorrt design doc with @yanchunwei
• fix compiler error in paddle:latest-dev image: https://github.com/PaddlePaddle/Paddle/pull/9907
• code review:
  • doc
    • https://github.com/PaddlePaddle/Paddle/pull/9820
    • https://github.com/PaddlePaddle/Paddle/pull/10031
  • mkldnn:
    • The mkldnn tests are skipped when the MKLDNN flag is OFF: https://github.com/PaddlePaddle/Paddle/pull/9985 https://github.com/PaddlePaddle/Paddle/pull/9881/
    • Added Epsilon for softmax MKLDNN op: https://github.com/PaddlePaddle/Paddle/pull/9993
  • add tensorrt library: https://github.com/PaddlePaddle/Paddle/pull/9891

• Inference Framework
  • Check the GPU usage based on inference unittest, the observed GPU usage is over 80%.
  • [Merged] Fix a bug in save_inference_model and prune when the program is initialized by load_inference_model
    • https://github.com/PaddlePaddle/Paddle/pull/10011
  • Review
    • fuse batch norm for conv operator, https://github.com/PaddlePaddle/Paddle/pull/9792
• paddle-mobile
  • Switch to git-flow branching model, https://github.com/PaddlePaddle/paddle-mobile/issues/120
  • Where and how to orgnize the codes of new version, https://github.com/PaddlePaddle/paddle-mobile/issues/122

DeepASR:

• Merge the error rate measuring script
  • https://github.com/PaddlePaddle/models/pull/834
• Add the aishell example & release data
  • https://github.com/PaddlePaddle/models/pull/861

ONNX convertor:

• Support conversion of recognize_digits models(conv&mlp)
  • https://github.com/PaddlePaddle/paddle-onnx/pull/19
• Some important feedforward models to be supported at first stage
  • https://github.com/PaddlePaddle/paddle-onnx/issues/24
• [WIP ]Design the unittest framework to test operator's conversion
• Fix the bug in executor.fetch_var
  • https://github.com/PaddlePaddle/Paddle/pull/10044

• float16 support
  • [WIP] write a float16 transpiler that append cast op and cast weights to float16: https://github.com/PaddlePaddle/Paddle/pull/10008
  • Add float16 example and add float16 support to save op: https://github.com/PaddlePaddle/Paddle/pull/9864
  • add print support to float16: https://github.com/PaddlePaddle/Paddle/pull/9960
  • Fix cuda 7.5 compile error on cublas:
    • https://github.com/PaddlePaddle/Paddle/pull/9885
    • https://github.com/PaddlePaddle/Paddle/pull/9908
  • Review: https://github.com/PaddlePaddle/Paddle/pull/10011#pullrequestreview-113325674

• Fix computing of output dims in the im2sequence op.

  • https://github.com/PaddlePaddle/Paddle/pull/9978

• Remove intermediate output's gradient from inputs of gru_unit_grad op

  • https://github.com/PaddlePaddle/Paddle/pull/10029

• Debug diff of convolution layer in OCR CTC model

  • https://github.com/PaddlePaddle/models/issues/855

• Debug forward of OCR attention model with VIS colleagues

• Review:

  • https://github.com/PaddlePaddle/models/pull/852

• aws tool merged, running benchmark test

  • https://github.com/PaddlePaddle/Paddle/pull/9638
  • https://github.com/PaddlePaddle/Paddle/pull/10037

• Issues

  • https://github.com/PaddlePaddle/Paddle/issues/10043
  • https://github.com/PaddlePaddle/Paddle/issues/10036

• review

  • https://github.com/PaddlePaddle/Paddle/pull/9932
  • https://github.com/PaddlePaddle/edl/pull/20
  • https://github.com/PaddlePaddle/edl/pull/21

• NMT:
  • Add Python wrapper for label smoothing.
    • https://github.com/PaddlePaddle/Paddle/pull/9940
  • Add label smoothing in Transformer.
    • https://github.com/PaddlePaddle/models/pull/858
  • ParallelExecutor related to Transformer.
    • https://github.com/PaddlePaddle/models/pull/845
    • https://github.com/PaddlePaddle/models/pull/863
  • Transformer experiments related.

• Parallel_exe
  • Feature/add reduce op handle
    • https://github.com/PaddlePaddle/Paddle/pull/9946
  • feature/Add Broadcast and Gather op handle[merged]
    • https://github.com/PaddlePaddle/Paddle/pull/9825
  • feature/refine_gather_reduce
    • https://github.com/PaddlePaddle/Paddle/pull/9922
• Review
  • Fix FetchTensor on CPU
    • https://github.com/PaddlePaddle/Paddle/pull/10019
  • Clean MultiDevicesGraphBuilder
    • https://github.com/PaddlePaddle/Paddle/pull/10017
  • Add parallel executor tests
    • https://github.com/PaddlePaddle/Paddle/pull/9950
  • Using constructor for VarHandle
    • https://github.com/PaddlePaddle/Paddle/pull/9975
  • Feature/mix cpu gpu op
    • https://github.com/PaddlePaddle/Paddle/pull/9898
  • Add Top-k Python API.
    • https://github.com/PaddlePaddle/Paddle/pull/9973

• accuracy local/cluster
  • mobilenet
  • se_resnext
  • vgg16
  • parallelExecutor
• MPI-Enabld
  • design doc https://github.com/PaddlePaddle/Paddle/pull/9490
  • coding

• Fluid support distributed lookup table:

  • https://github.com/PaddlePaddle/Paddle/projects/56
  • https://github.com/PaddlePaddle/Paddle/issues/9211
  1. Dist transpiler support prefetch [done] https://github.com/PaddlePaddle/Paddle/pull/9714

• Fluid support async training:

  • https://github.com/PaddlePaddle/Paddle/projects/61
  • https://github.com/PaddlePaddle/Paddle/issues/9941
  1. Add distributed training overview doc. https://github.com/PaddlePaddle/Paddle/pull/9937
  2. [WIP] listen_and_serv_op support async update https://github.com/PaddlePaddle/Paddle/pull/10042

• Code clean&optimize

  1. specify version for pip https://github.com/PaddlePaddle/Paddle/pull/9980

• SSD:
  • mAP with ParallelExe: 72.9% ~ 73.1% (Caffe baseline 72.7%)
  • Add eval.py and fix bug for MobileNet-SSD.
    • https://github.com/PaddlePaddle/models/pull/852
  • Merge Doc: https://github.com/PaddlePaddle/models/pull/801
• SE-ResNeXt:
  • Add unit test to compare the training between ParallelDo and ParallelExecutor.
    • https://github.com/PaddlePaddle/Paddle/pull/10016
  • [WIP] train model and do some tests and add eval.py.
• Others:
  • Refine batch_norm_op: https://github.com/PaddlePaddle/Paddle/pull/9871
  • Add Top-k Python API: https://github.com/PaddlePaddle/Paddle/pull/9973
  • Discussion about Q2 work Plan.
• Code review:
  • https://github.com/PaddlePaddle/Paddle/pull/9308
  • https://github.com/PaddlePaddle/models/pull/798
  • https://github.com/PaddlePaddle/models/pull/839

• PR:

  • Add 4 docs to fluid/dev

    • https://github.com/PaddlePaddle/Paddle/pull/10024

  • Remove dense_vector from v2 api documents

    • https://github.com/PaddlePaddle/Paddle/pull/9896

  • Regenerate initializer.rst

    • https://github.com/PaddlePaddle/Paddle/pull/9870

• Review two models

  • https://github.com/PaddlePaddle/models/pull/773

• Tracing the segmentation fault when using beam search for sequence to sequence model. https://github.com/PaddlePaddle/Paddle/issues/10026
• [WIP] Trying to add kernel to beam search op’s implementation

• Code Clean Up

  • Fix compiler warnings in split_op and chunks_test https://github.com/PaddlePaddle/Paddle/pull/9914
  • Fix cpplint issues with beam_search_op and beam_search_decode_op https://github.com/PaddlePaddle/Paddle/pull/9962
  • Fix CPPLint issues with Chunk_eval_op https://github.com/PaddlePaddle/Paddle/pull/9964
  • Fix cpplint issues in Detection_map_op and conv_op https://github.com/PaddlePaddle/Paddle/pull/9969
  • Fix CPPLint issues in expand_op, gather_op and get_places_op https://github.com/PaddlePaddle/Paddle/pull/10000
  • Fix conv_mkldnn_op.cc which is causing CI failure https://github.com/PaddlePaddle/Paddle/pull/10002
  • Fix CPPLint errors in multiclass_nms, nccl, nce, reduce and save_load_combine https://github.com/PaddlePaddle/Paddle/pull/10032
  • Fix CPPLint errors in some framework files https://github.com/PaddlePaddle/Paddle/pull/10041
  • Disabling Channel Test to fix CI https://github.com/PaddlePaddle/Paddle/pull/10001
  • Fix warnings in sgd_op.h https://github.com/PaddlePaddle/Paddle/pull/9888

• Fluid Imperative Programming

  • Discuss possible fluid imperative programming paradigms https://github.com/PaddlePaddle/Paddle/issues/9466
  • Paddle API v4 Proposal https://github.com/PaddlePaddle/Paddle/issues/9912
  • Worked with Thuan and Helin on Imperative Programming Scaffold https://github.com/PaddlePaddle/Paddle/pull/10039

• PR Reviews

  • https://github.com/PaddlePaddle/Paddle/pull/10034#pullrequestreview-113372679
  • https://github.com/PaddlePaddle/Paddle/pull/9963#pullrequestreview-112639152
  • https://github.com/PaddlePaddle/Paddle/pull/9947#pullrequestreview-112543745
  • https://github.com/PaddlePaddle/Paddle/pull/9924#pullrequestreview-112540852
  • https://github.com/PaddlePaddle/Paddle/pull/9920#pullrequestreview-112525290
  • https://github.com/PaddlePaddle/Paddle/pull/9918#pullrequestreview-112199320
  • https://github.com/PaddlePaddle/Paddle/pull/9908#pullrequestreview-112165365
  • https://github.com/PaddlePaddle/Paddle/pull/9876#pullrequestreview-111759504
  • https://github.com/PaddlePaddle/Paddle/pull/9870#pullrequestreview-111758355

• distributed training accuracy test:
  • https://docs.google.com/spreadsheets/d/1D5Xc_TfGfMV5aKh4ZJS_b4js3Mnn06H1Po0iuECZLr4/edit#gid=0
• split_byref_op to reduce copy when sending tensors: https://github.com/PaddlePaddle/Paddle/pull/9956
• Fixes:
  • https://github.com/PaddlePaddle/Paddle/issues/10040
  • https://github.com/PaddlePaddle/Paddle/pull/10027
  • https://github.com/PaddlePaddle/Paddle/pull/9935
  • https://github.com/PaddlePaddle/Paddle/pull/9933
  • https://github.com/PaddlePaddle/Paddle/pull/9931
  • https://github.com/PaddlePaddle/Paddle/issues/10014
• Reviews:
  • https://github.com/PaddlePaddle/Paddle/pull/9490
  • https://github.com/PaddlePaddle/Paddle/pull/9937
  • https://github.com/PaddlePaddle/Paddle/pull/9901
  • https://github.com/PaddlePaddle/Paddle/pull/9787
  • https://github.com/PaddlePaddle/Paddle/pull/9897

• lookup remote table
  • auto-grown sparse table, https://github.com/PaddlePaddle/Paddle/pull/9897
• async update design doc,https://github.com/PaddlePaddle/Paddle/pull/9932
• bug fix
  • https://github.com/PaddlePaddle/Paddle/pull/9901
  • https://github.com/PaddlePaddle/Paddle/pull/9926
  • https://github.com/PaddlePaddle/Paddle/pull/9971
  • https://github.com/PaddlePaddle/Paddle/pull/10047
• review
  • https://github.com/PaddlePaddle/Paddle/pull/9956#pullrequestreview-113065070

• Fluid Imperative API with Thuan and Abhinav
  • API Design: https://github.com/PaddlePaddle/Paddle/issues/9912
  • API scaffold: https://github.com/PaddlePaddle/Paddle/pull/10039
  • Project: https://github.com/PaddlePaddle/Paddle/projects/60
• Reviews:
  • https://github.com/PaddlePaddle/Paddle/pull/10039#pullrequestreview-113435983
  • https://github.com/PaddlePaddle/Paddle/pull/10039#pullrequestreview-113435983
  • https://github.com/PaddlePaddle/Paddle/pull/10037#pullrequestreview-113421820
  • https://github.com/PaddlePaddle/Paddle/pull/9932/files#r181879029
  • https://github.com/PaddlePaddle/Paddle/pull/9854#pullrequestreview-111416552
  • https://github.com/PaddlePaddle/Paddle/pull/9663#pullrequestreview-111400906
  • https://github.com/PaddlePaddle/Paddle/pull/9848#pullrequestreview-111332997

• [Memory] reuse relu/sigmoid operator input variable, to save memory cost
  • https://github.com/PaddlePaddle/Paddle/pull/9740
• [Speed] accelerate 10x times in per minibatch
  • https://github.com/PaddlePaddle/Paddle/pull/9902
• [Speed] try to accelerate the scope.FindVar
  • https://github.com/PaddlePaddle/Paddle/issues/9232
• 解决部署机器环境问题(高优)
  • 尚未解决

• infer

  • upgrade docker image cudnn from v5 to v7
  • integrate tensorrt compile
  • tensorRT engine
  • discuss with alajin about merging to Paddle

• CE

  • preparing server

• reviews

  • https://github.com/PaddlePaddle/Paddle/pull/9958#pullrequestreview-112712076

• Paddle Mobile moved into PaddlePaddle org
  • https://github.com/PaddlePaddle/paddle-mobile
• Add feed for transformer
  • https://github.com/PaddlePaddle/models/pull/864
• Clean up ParallelExecutor codes
  • https://github.com/PaddlePaddle/Paddle/pull/10022
• Refine memory optimizer
  • https://github.com/PaddlePaddle/Paddle/pull/9905
• Profile new Transformer for ParallelExecutor
  • single device 0.138, 4 devices 0.240

• Released VisualDL v1.0.0

  • Fastened interactive graph component for the release (slider, large graph download, panning and zooming, etc)
  • Updated VisualDL docs and tutorial contents for the new release
  • Recorded demo videos and validated the release on different platforms (Linux/Mac, Python2/Python3)
  • Merged PRs and Closed Issues:
    • https://github.com/PaddlePaddle/VisualDL/pull/395
    • https://github.com/PaddlePaddle/VisualDL/pull/391
    • https://github.com/PaddlePaddle/VisualDL/pull/389
    • https://github.com/PaddlePaddle/VisualDL/pull/388
    • https://github.com/PaddlePaddle/VisualDL/pull/387
    • https://github.com/PaddlePaddle/VisualDL/issues/404
    • https://github.com/PaddlePaddle/VisualDL/issues/403
    • https://github.com/PaddlePaddle/VisualDL/issues/397
    • https://github.com/PaddlePaddle/VisualDL/issues/386
    • https://github.com/PaddlePaddle/VisualDL/issues/372

• Plans and Targets for VisualDL v1.1.0

  • Discussed with Jeff and Nicky for the next major release ( in ~ 1 month)
    • Plan to setup an online VisualDL playground to give users first hand experience
    • https://github.com/PaddlePaddle/VisualDL/issues/242

• Reviewed PRs:

  • https://github.com/PaddlePaddle/VisualDL/pull/413
  • https://github.com/PaddlePaddle/VisualDL/pull/412
  • https://github.com/PaddlePaddle/VisualDL/pull/411
  • https://github.com/PaddlePaddle/VisualDL/pull/400
  • https://github.com/PaddlePaddle/VisualDL/pull/398
  • https://github.com/PaddlePaddle/VisualDL/pull/394
  • https://github.com/PaddlePaddle/VisualDL/pull/392
  • https://github.com/PaddlePaddle/VisualDL/pull/385

• Paddle:
  • Refine Imperative Programming API proposal (https://github.com/PaddlePaddle/Paddle/issues/9912)
  • Imperative Programming Design Documentation and Scaffolding (https://github.com/PaddlePaddle/Paddle/pull/10039)
  • Add sphinx documentation for Mobile (https://github.com/PaddlePaddle/Paddle/pull/9963)
• PaddlePaddle.org
  • Add in document generator for paddle mobile docs (https://github.com/PaddlePaddle/PaddlePaddle.org/pull/465)
  • Travis build is failing for to deploy documentation (https://github.com/PaddlePaddle/Mobile/pull/89)

• Document and fix cpplint errors of fluid/platform: https://github.com/PaddlePaddle/Paddle/pull/9918
• Fix cpplint of transform_test.cu: https://github.com/PaddlePaddle/Paddle/pull/9915
• Document transform.h and fix cpplint errors: https://github.com/PaddlePaddle/Paddle/pull/9913
• Redesign the interface of device_ptr_cast: https://github.com/PaddlePaddle/Paddle/pull/9910

• Manage the VisualDL 1.0.0 release

• Add release the note for book keeping: https://github.com/PaddlePaddle/VisualDL/pull/400

• Update Travis to reduce the build time and fix failure: https://github.com/PaddlePaddle/VisualDL/pull/398

• Fix Scalar X-axis selection not working issue: https://github.com/PaddlePaddle/VisualDL/pull/394

• Fix PPO VisualDL doc tool generation issue: https://github.com/PaddlePaddle/PaddlePaddle.org/pull/470

• Plan for VisuaDL 1.1.0: https://github.com/PaddlePaddle/VisualDL/projects/6

• Prepared Marketing article for TianYe

• WIP: Re-design the VisualDL documentation layout and update documentation: https://github.com/PaddlePaddle/VisualDL/issues/408

• Reviewed PRs and Issues:

  • https://github.com/PaddlePaddle/PaddlePaddle.org/pull/465
  • https://github.com/PaddlePaddle/Paddle/pull/9963
  • https://github.com/PaddlePaddle/VisualDL/pull/391
  • https://github.com/PaddlePaddle/VisualDL/pull/395
  • Issue: https://github.com/PaddlePaddle/PaddlePaddle.org/issues/469

• Paddle Onnx:
  • https://github.com/PaddlePaddle/paddle-onnx/pull/21
  • PR review: https://github.com/PaddlePaddle/paddle-onnx/pull/19
  • PR review: https://github.com/PaddlePaddle/paddle-onnx/pull/26
• Code cleanup:
  • https://github.com/PaddlePaddle/Paddle/pull/9837

PR review:

• https://github.com/PaddlePaddle/Paddle/pull/9962
• https://github.com/PaddlePaddle/Paddle/pull/9828
• https://github.com/PaddlePaddle/Paddle/pull/10032

• Prepare Visual DL 1.0 release
  • Generate build and analyze error with native Python, VirtualEnv, Anaconda
• Update README for installation guide
  • Add Virtualenv and Anaconda install guide: https://github.com/PaddlePaddle/VisualDL/pull/411
  • Add install by sources guide: https://github.com/PaddlePaddle/VisualDL/pull/412
  • Add chinese version: https://github.com/PaddlePaddle/VisualDL/pull/413
• Fix unit tests and make sure all unit tests pass
  • https://github.com/PaddlePaddle/VisualDL/pull/385