bagging&boosting原理 - peter-xbs/CommonCodes GitHub Wiki

GBDT

boosting算法精选资料:

• https://zhuanlan.zhihu.com/p/81182006
• https://zpz.github.io/blog/gradient-boosting-tree-for-binary-classification/

补充： Logistic regression的两种形式

● -yln(p) - (1-y)ln(1-p)   # [0, 1]编码
● Sigma(-ln(1/(1+exp(-t*yi)) = Sigma(ln(1+exp(-t*yi)) # [1, -1]编码 t为1/-1

一阶导分别为：

● y - p
● -t/(1+exp(t*yi))

因此在残差计算时，上述两种一阶导数均可使用

参考： https://blog.csdn.net/fjsd155/article/details/88394222

LTR

LTR非常好的资料 源码 + 论文对照阅读

• https://github.com/haowei01/pytorch-examples/blob/master/ranking/LambdaRank.py
• https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/MSR-TR-2010-82.pdf

lambdamart: 这个初步预估会比较难

• https://github.com/lezzago/LambdaMart/blob/master/lambdamart.py

implementation

• https://stackoverflow.com/questions/64294962/how-to-implement-learning-to-rank-using-lightgbm