General Information

This brick provides you with an easy interface for creating your own out-of-box and distributed gradient boosted decision tree for your regression tasks (if you need to solve a classification task, please, check LGBM Binary or LGBM Multiclass brick). Due to its leaf-wise processing nature, the created model can be easily trained on large datasets, while giving formidable results.

The models are built on three important principles:

• Weak learners
• Gradient Optimization
• Boosting Technique

In this case, the weak learners are multiple sequential specialized decision trees, which do the following things:

• the first tree learns how to fit to the target variable
• the second one learns how to fit the difference between the predictions of the first tree and the ground truth (real data)
• The next tree learns how to fit the residuals of the second tree and so on.

All those trees are trained by propagating the gradients of errors throughout the system.

The main drawback of the LGBM Binary is that finding the best split points in each tree node is both a time-consuming and memory-consuming operation.

Description

Brick Locations

• Bricks → Analytics → Data Mining / ML → Regression Models → LGBM Regressor
• Bricks → Use Cases → Demand Forecasting → Modelling → LGBM Regressor

Brick Parameters

• Learning rate

  Boosting learning rate. This parameter controls how quickly or slowly the algorithm will learn a problem. Generally, a bigger learning rate will allow a model to learn faster while a smaller learning rate will lead to a more optimal outcome.

• Number of iterations

  A number of boosting iterations. This parameter is recommended to be set inversely to the learning rate selected (decrease one while increasing second).

• Number of leaves

  The main parameter to control model complexity. Higher values should increase accuracy but might lead to overfitting.