Nikos G.’s Post

Most data scientists don't know this, to my surprise. Bagging trees and random forests do not overfit when you increase the number of trees, but do underfit if the number is low. Adding extra trees just increases training time without any benefit after some point. #datascience #ai #python (image from ISLP, page 348)

  • chart

If your only goal is accuracy then going past 200 trees rarely makes a difference in my opinion. But, secondary metrics, like feature importance (for example) benefit greatly from even going up to 2000 trees. They will stabilize better at that level, from run to run.

But they can and will overfit with too deep trees, small splits, etc. Always use an optimizer if you want a robust solution. I use Optuna. If you have enough data optimize on a cross validation inside the loop and confirm on a completely held out sample (test set).

See more comments

To view or add a comment, sign in

Explore content categories