2018 02 28 - PaddlePaddle/Paddle GitHub Wiki

Main focus: ParallelDo in Multiple GPUs

Issue:

• The problem of improving the performance of Parallel_Do
  • https://github.com/PaddlePaddle/Paddle/issues/8592

PR:

• Backward on parallel do using nccl:
  • https://github.com/PaddlePaddle/Paddle/pull/8361
• Python framework:
  • https://github.com/PaddlePaddle/Paddle/pull/8471

Review:

• https://github.com/PaddlePaddle/Paddle/pull/8516
• https://github.com/PaddlePaddle/Paddle/pull/8489
• https://github.com/PaddlePaddle/Paddle/pull/8402
• https://github.com/PaddlePaddle/Paddle/pull/8471

Inference:

• Integrate float16 into data_type_transform:
  • https://github.com/PaddlePaddle/Paddle/pull/8619
• Enable is_test for batch norm and dropout op:
  • https://github.com/PaddlePaddle/Paddle/pull/8642
• [WIP] add float16 GEMM GPU function in math_function
• Review:
  • https://github.com/PaddlePaddle/Paddle/pull/8404#pullrequestreview-99532186

Inference: PR/issue:

• Profiling C++ inference api for recognize digits model: https://github.com/PaddlePaddle/Paddle/pull/8497
• Results for this analysis: https://github.com/sidgoyal78/paddle_notes/blob/master/benchmark/recoginze_digits.md
• Survey TensorRT for inference: https://github.com/PaddlePaddle/Paddle/issues/8492

• CSP
  • Exposing Channel to be used as a Variable and integrating with Fluid https://github.com/PaddlePaddle/Paddle/pull/8486
  • Add unit tests for ChannelHolder https://github.com/PaddlePaddle/Paddle/pull/8486
  • Add Go_op, Channel_create, channel_close, channel_send and channel_receive ops https://github.com/PaddlePaddle/Paddle/pull/8593
  • Adding more unit tests for ChannelHolder class https://github.com/PaddlePaddle/Paddle/pull/8668
  • Review:
    • https://github.com/PaddlePaddle/Paddle/pull/8519#pullrequestreview-100326088
    • https://github.com/PaddlePaddle/Paddle/pull/8666#pullrequestreview-100283573

• Exposing Channel to be used as a Variable and integrating with Fluid: https://github.com/PaddlePaddle/Paddle/pull/8486
• Add Go_op, Channel_create, channel_close, channel_send and channel_receive ops: https://github.com/PaddlePaddle/Paddle/pull/8593
• nvcc fatal errors on TeamCity: https://github.com/PaddlePaddle/Paddle/issues/8501
• PR review:
  • https://github.com/PaddlePaddle/Paddle/pull/8519
  • https://github.com/PaddlePaddle/Paddle/pull/8510

• inference:
  • refine inference_lib_dist after code move, and add it to docker/build.sh: https://github.com/PaddlePaddle/Paddle/pull/8379
  • combine batch_size_like.cc into batch_size_like.h: https://github.com/PaddlePaddle/Paddle/pull/8604
• compile:
  • Move Fluid C++ code from /paddle to /paddle/fluid:
    • https://github.com/PaddlePaddle/Paddle/pull/8344/commits/77f04fd97aadad413815b111f6f85052da623dd5
    • https://github.com/PaddlePaddle/Paddle/pull/8363
  • move Fluid API doc/code out of V2 API doc/code:
    • https://github.com/PaddlePaddle/Paddle/pull/8548/
    • https://github.com/PaddlePaddle/models/pull/668
    • https://github.com/PaddlePaddle/Paddle/pull/8560
  • set the default option of WITH_FAST_BUNDLE_TEST be OFF： https://github.com/PaddlePaddle/Paddle/pull/8563
• document
  • reduce doc build time in travis ci (from 30+ min to 4 min):
    • https://github.com/PaddlePaddle/Paddle/pull/8369
    • https://github.com/PaddlePaddle/Paddle/pull/8615
  • Adjust the structure of API, Operators, cluster and quick start (Both Chinese and English):
    • https://github.com/PaddlePaddle/Paddle/pull/8266
    • https://github.com/PaddlePaddle/Paddle/pull/8321
    • https://github.com/PaddlePaddle/Paddle/pull/8324
    • https://github.com/PaddlePaddle/Paddle/pull/8537
    • https://github.com/PaddlePaddle/Paddle/pull/8658
  • update generate_paddle_docs.sh in paddlepaddle.org repo: https://github.com/PaddlePaddle/PaddlePaddle.org/pull/409
• code review:
  • Simplify the cmake of inference: https://github.com/PaddlePaddle/Paddle/pull/8272
  • Refine cmake for cudnn op: https://github.com/PaddlePaddle/Paddle/pull/8591
  • [Intel] MKLDNN conv2d and pool2d OP kernels added: https://github.com/PaddlePaddle/Paddle/pull/8451
  • [doc] add introduction:
    • https://github.com/PaddlePaddle/Paddle/pull/8566
    • https://github.com/PaddlePaddle/Paddle/pull/8644
    • https://github.com/PaddlePaddle/Paddle/pull/8622

• fix dist traning bug, make sure demo code work.
  • https://github.com/PaddlePaddle/Paddle/pull/8586
  • https://github.com/PaddlePaddle/Paddle/pull/8600
  • https://github.com/PaddlePaddle/Paddle/pull/8659
• fix v2 async sgd update, https://github.com/PaddlePaddle/Paddle/pull/8474
• english toturial doc on cloud repo, https://github.com/PaddlePaddle/cloud/pull/621
• review:
  • https://github.com/PaddlePaddle/cloud/pull/617#pullrequestreview-99173766
  • https://github.com/PaddlePaddle/Paddle/pull/8538#pullrequestreview-99175816
  • https://github.com/PaddlePaddle/Paddle/pull/8656#pullrequestreview-100022647
  • https://github.com/PaddlePaddle/Paddle/pull/8634#pullrequestreview-99954888
• turn on cmake flag WITH_DISTRIBUTE on CI so that the wheel package support distributed training.

• Enhance layer_generator
  • https://github.com/PaddlePaddle/Paddle/pull/8543
  • mean(x=layer_out) --> mean(layer_out)
• Moving unique_name to python. We can reset the unique_name generator now
  • https://github.com/PaddlePaddle/Paddle/pull/8524
• Demo about switch optmizers
  • https://github.com/PaddlePaddle/Paddle/pull/8570
• Demo about stack denoising autoencoder
  • https://github.com/PaddlePaddle/Paddle/pull/8531
• Make global_step as a global variable in Fluid
  • https://github.com/PaddlePaddle/Paddle/pull/8564
• Several Enhancements
  • https://github.com/PaddlePaddle/Paddle/pull/8657
  • https://github.com/PaddlePaddle/Paddle/pull/8649

• learning rate decay (https://github.com/PaddlePaddle/Paddle/issues/7769)

  • Fix compare op https://github.com/PaddlePaddle/Paddle/pull/8532
  • create learning rate for multi program https://github.com/PaddlePaddle/Paddle/pull/8545
  • change learning_rate_decay to learning_rate_scheduler https://github.com/PaddlePaddle/Paddle/pull/8583

• multi gpu profile

  • parallel-do should not merge the gradient of parameter that stop_gradient=True https://github.com/PaddlePaddle/Paddle/pull/8652
  • se_resnet_50 multi-gpu profile https://github.com/PaddlePaddle/Paddle/issues/8661

• add c-api quick start https://github.com/PaddlePaddle/Paddle/pull/8566

• discuss

  • The problem of improving the performance of Parallel_Do https://github.com/PaddlePaddle/Paddle/issues/8592

• Review

  • Fine Tune MNIST by Adam and SGD https://github.com/PaddlePaddle/Paddle/pull/8570
  • simplify shape inference code https://github.com/PaddlePaddle/Paddle/pull/8087
  • Enhance layer_function_generator https://github.com/PaddlePaddle/Paddle/pull/8543
  • Moving unique_name to python https://github.com/PaddlePaddle/Paddle/pull/8524

• A new design of model save/load:
  • https://github.com/PaddlePaddle/Paddle/issues/8521
  • a related PR: https://github.com/PaddlePaddle/Paddle/pull/8534
• [WIP] Disassemble evaluator：
  • https://github.com/PaddlePaddle/Paddle/pull/8643
• Reviews:
  • https://github.com/PaddlePaddle/Paddle/pull/8489
  • https://github.com/PaddlePaddle/Paddle/pull/8505
  • https://github.com/PaddlePaddle/Paddle/pull/8511

• SSD on Fluid:
  • [Merged] Enhance bipartite_match_op to support argmax matching after bipartite matching.
    • https://github.com/PaddlePaddle/Paddle/pull/8580
  • [Merged] Register more data type for reshape operator.
    • https://github.com/PaddlePaddle/Paddle/pull/8617
  • [Merged] Enable the SSD loss to support normalization by the total number of output locations.
    • https://github.com/PaddlePaddle/Paddle/pull/8630
  • [Merged] Fix the backward transpiler bug in ssd_loss API.
    • https://github.com/PaddlePaddle/Paddle/pull/8648
  • Verify the correctness of SSD loss:
    • https://github.com/PaddlePaddle/Paddle/issues/8626
  • [WIP] Verify the correctness of detection output
  • Review:
    • Fix box coder op: https://github.com/PaddlePaddle/Paddle/pull/8647
• Other:
  • Help HuaWei to debug the AR demo on Fluid.

DeepASR:

• Convergence verification on single GPU
  • https://github.com/PaddlePaddle/models/issues/676
• Performance profiling
  • https://github.com/PaddlePaddle/models/issues/669
  • https://github.com/PaddlePaddle/models/issues/637
• Some enhancements
  • https://github.com/PaddlePaddle/models/pull/660
  • https://github.com/PaddlePaddle/models/pull/653
  • https://github.com/PaddlePaddle/models/pull/663
• Fix the profiler's bug in multi-gpu mode
  • https://github.com/PaddlePaddle/Paddle/pull/8596

Code Review:

• https://github.com/PaddlePaddle/models/pull/667
• https://github.com/PaddlePaddle/Paddle/pull/8365

Main focus: ParallelDo in Multiple GPUs

• PR:

  • Run optimizer inside parallel do if the forward pass contains parallel do

• Reviews:

  • https://github.com/PaddlePaddle/Paddle/pull/8665#pullrequestreview-100223755
  • https://github.com/PaddlePaddle/Paddle/issues/8504#event-1488164262
  • https://github.com/PaddlePaddle/Paddle/issues/8480#event-1488164544
  • https://github.com/PaddlePaddle/Paddle/pull/8471#pullrequestreview-98784602
  • https://github.com/PaddlePaddle/Paddle/issues/8500
  • https://github.com/PaddlePaddle/Paddle/issues/8592#issuecomment-368693237
  • https://github.com/PaddlePaddle/Paddle/pull/8550#pullrequestreview-100248472

• CI fixes:

  • https://github.com/PaddlePaddle/Paddle/issues/8501
  • https://github.com/PaddlePaddle/Paddle/issues/8487

• fulid

  • PR, conv sequence to sequence
    • finished, tuning with data
  • [PR, v2 API doc overview]https://github.com/PaddlePaddle/Paddle/pull/8547

• visualdl

  • issue, discussion about backend performance tune
  • repo, dependency engine for VDL

• Distribute training:
  • Tensorflow: https://github.com/PaddlePaddle/Paddle/pull/8522
    • https://github.com/PaddlePaddle/Paddle/pull/8513
  • Change script to support tensorflow distribution on k8s: https://github.com/PaddlePaddle/cloud/pull/617
• Document:
  • cluster train: https://github.com/PaddlePaddle/Paddle/pull/8622
  • Fix docment generation bugs: https://github.com/PaddlePaddle/PaddlePaddle.org/pull/418
• Review:
  • https://github.com/PaddlePaddle/cloud/pull/625
  • https://github.com/PaddlePaddle/cloud/pull/623
  • https://github.com/PaddlePaddle/cloud/pull/614
  • https://github.com/PaddlePaddle/cloud/pull/611
  • https://github.com/PaddlePaddle/Paddle/pull/8573

• Inference Framework
  • [Merged] Refine the inference API and unittest
    • https://github.com/PaddlePaddle/Paddle/pull/8404
  • Write a basic userguide of Fluid inference
    • https://github.com/Xreki/Xreki.github.io/blob/master/fluid/inference/inference_support_in_fluid.md
  • Review
    • combine batch_size_like.cc into batch_size_like.h: https://github.com/PaddlePaddle/Paddle/pull/8604
    • Inference example and unittest for NMT model: https://github.com/PaddlePaddle/Paddle/pull/8314
• Get rid of the dependency of Go compiler when WITH_GOLANG is OFF
  • https://github.com/PaddlePaddle/Paddle/pull/8610

• Profile data reader for DeepASR
  https://github.com/PaddlePaddle/models/issues/673
• RNN Beam search
  https://github.com/PaddlePaddle/Paddle/issues/8603
  https://github.com/PaddlePaddle/models/pull/675

• Debug and tune the Transformer model referenced with the Pytorch implementation.
• PR:
  • Remove the losses from paddings in Transformer
    • https://github.com/PaddlePaddle/models/pull/665
  • Add learning rate scheduling in Transformer
    • https://github.com/PaddlePaddle/models/pull/671

• CI speed up
  • https://github.com/PaddlePaddle/Paddle/pull/8362
• Bisect culprit commit:
  • https://github.com/PaddlePaddle/Paddle/pull/8442
• Timeline Profiler
  • https://github.com/PaddlePaddle/Paddle/pull/8542
• Reviews
  • https://github.com/PaddlePaddle/Paddle/pull/8564
  • https://github.com/PaddlePaddle/Paddle/pull/8593
  • https://github.com/PaddlePaddle/Paddle/pull/8563

• PR

  • [WIP]Refine concat_op
    • https://github.com/PaddlePaddle/Paddle/pull/8669
  • Refine Sum in elementwise_op_function
    • https://github.com/PaddlePaddle/Paddle/pull/8539
  • Refine cmake for cudnn op
    • https://github.com/PaddlePaddle/Paddle/pull/8591
  • Add tuple type
    • https://github.com/PaddlePaddle/Paddle/pull/8519
  • Fix conv_op bug
    • https://github.com/PaddlePaddle/Paddle/pull/8613
    • https://github.com/PaddlePaddle/Paddle/pull/8551
  • refine FQA doc
    • https://github.com/PaddlePaddle/Paddle/pull/8644
  • fix get_mid_dims annotation
    • https://github.com/PaddlePaddle/Paddle/pull/8490

• Review

  • Refine Sum in elementwise_op_function
    • https://github.com/PaddlePaddle/Paddle/pull/8539
  • Add Go_op, Channel_create, channel_close, channel_send and channel_receive ops
    • https://github.com/PaddlePaddle/Paddle/pull/8593
  • Add unit tests for ChannelHolder
    • https://github.com/PaddlePaddle/Paddle/pull/8510
  • Extend current profiler for timeline and more features
    • https://github.com/PaddlePaddle/Paddle/pull/8542
  • Enhance bipartite_match_op to support argmax matching after bipartite matching
    • https://github.com/PaddlePaddle/Paddle/pull/8580
  • Add ceil_mode option for pool2d and pool3d
    • https://github.com/PaddlePaddle/Paddle/pull/8651

• CI updates

  • CI docker graph location update, teamcity data relocation to larger storage
  • daily team city backup
  • daily docker prune

• Review

  • https://github.com/PaddlePaddle/cloud/pull/621

• Pr

  • https://github.com/PaddlePaddle/cloud/pull/625
  • https://github.com/PaddlePaddle/cloud/pull/626

• GPU perf
  • https://github.com/PaddlePaddle/Paddle/pull/8550
  • https://github.com/PaddlePaddle/Paddle/issues/8638
  • https://github.com/PaddlePaddle/Paddle/pull/8634
  • https://github.com/PaddlePaddle/Paddle/pull/8573
  • https://github.com/PaddlePaddle/Paddle/pull/8538
  • https://github.com/PaddlePaddle/Paddle/pull/8512
  • review: https://github.com/PaddlePaddle/Paddle/pull/8600
  • review: https://github.com/PaddlePaddle/cloud/pull/617
  • reviews etc.
• EDL:
  • https://github.com/PaddlePaddle/cloud/pull/624

PaddlePaddle.org

• Fix issue of MathJax equations and images not rendering correctly when user clicks on a new link. (https://github.com/PaddlePaddle/PaddlePaddle.org/pull/412)
• Fix issue with permalink not showing entire title (https://github.com/PaddlePaddle/PaddlePaddle.org/pull/415)

Paddle

• Add Go_op, Channel_create, channel_close, channel_send and channel_receive ops (https://github.com/PaddlePaddle/Paddle/pull/8593)
• Unittests concurrency (https://github.com/PaddlePaddle/Paddle/pull/8666)

Other

• Worked on Visual Debugger Tech Talk with Varun

• Fix error message on charts: https://github.com/PaddlePaddle/VisualDL/pull/279
• Add the Histogram related Vue files: https://github.com/PaddlePaddle/VisualDL/pull/278
• Add the Graph.vue and the Config.vue for Graph tab: https://github.com/PaddlePaddle/VisualDL/pull/277
• Fix the incorrect pagination issue: https://github.com/PaddlePaddle/VisualDL/pull/276
• Allow the navigation bar to persist the selected item style. : https://github.com/PaddlePaddle/VisualDL/pull/274
• Include the pymdownx extensions: https://github.com/PaddlePaddle/PaddlePaddle.org/pull/425
• Fix incorrect markdown and RST file: https://github.com/PaddlePaddle/Paddle/pull/8667

• Fix scalar issues: https://github.com/PaddlePaddle/VisualDL/pull/283
• Show Scalar Data and add ExpandPanel https://github.com/PaddlePaddle/VisualDL/pull/272
• Add theme and UI https://github.com/PaddlePaddle/VisualDL/pull/280
• Fix font size and reorganize css and stylus https://github.com/PaddlePaddle/VisualDL/pull/287

• Add Go_op, Channel_create, channel_close, channel_send and channel_receive ops (https://github.com/PaddlePaddle/Paddle/pull/8593)
• Worked on Visual Debugger Tech Talk with Thuan