Categories: PGPDMStudent Speak

Boosting vs Random Forest Classifiers

Classification problem is quite popular in various domains such as finance and telecommunication, for example, to predict the churn in telecommunication.  Selection of a method, out of classical or machine learning algorithms, depends on business priorities.  Each comes with certain advantages and disadvantages.  In this blog, I will examine two machine learning algorithm – boosting and random forest.


In boosting, n number of samples are created using bootstrapping – sampling with replacement.  Boosting works on weak classifiers that have high bias and low variation works iteratively on weak learners, and more weightage is given to misclassified learners in next iteration. The final classification is done after combining prediction of each classifier. Boosting uses base model as decision tree generally. However, linear regression or logistic regression can be used as a base model too. It is best to grow a tree with no pruning and trees with 2-8 leaves work well. The process flow of common boosting method- ADABOOST-is as following:

Random forest

This ensemble method works on bootstrapped samples and uncorrelated classifiers. Alternatively, this model learns from various over grown trees and a final decision is made based on the majority.  In this method, predictors are also sampled for each node. It best works on over fitted models that have low bias and high variation and is a bagged model.

I have personally used fgl dataset of R to compare these two methods. Fgl has predictors as chemical elements- RI, Na, Mg, Al, Si, K, Ca, Ba, and Fe and target variable is a type of forensic glass – WinF, WinNF, Veh, Con, Tabl, and Head.

In short, not only is a random forest more accurate model than boosting model but also it is more explainable that it gives importance of various predictors.  See the importance of predictors as given by random forest. Boosting is used for data sets that has high dimensions.

This is a guest post by Rajiv Pandey. He is currently doing the Executive Program In Business Analytics course (EPBA) from Jigsaw Academy.

Also Read 

Top Data Science Experts on Twitter/LinkedIn You Should Follow

How to Shift a Career From IT to Analytics?


Published by

Recent Posts

Books on Analytics

Analytics is a vast field. At the one end, it overlaps with statistics and higher…

Career in analytics in a KPO

Do you love to explore and investigate information? Do you find spreadsheets to be a…

Indian companies using analytics

India has developed into the global hub for analytics. A large number of MNCs have…

IBM: Betting big on analytics

International Business Machines Corp. Or IBM as it is popularly known recently announced its restructuring…

How to build a successful career in analytics

So you have got a job as an analyst in your dream company? Here are…

What’s the sentiment on “sentiment analysis”?

What's the sentiment on "sentiment analysis"? Is the field ready to take off?