Sections 9 and 10 are on tree-based methods. There are three main methods:
<ul>
<li>Decision Trees (Section 9)</li>
<li><a target="_blank" href="https://nurfadhilah.tech/section-10-random-forests-and-boosted-trees">Random Forests (Section 10)</a> </li>
<li><a target="_blank" href="https://nurfadhilah.tech/section-10-random-forests-and-boosted-trees">Boosted Trees (Section 10)</a> </li>
</ul>
Each of these methods stems from the basic decision tree algorithm. Fundamentally, tree-based methods rely on the ability to split data based on information from features. Require a mathematical definition of information and the ability to measure it.
Classification and Regression Tree (CART) introduces many concepts:
<ul>
<li>Cross validation of Trees</li>
<li>Pruning Trees</li>
<li>Surrogate Splits</li>
<li>Variable Importance Scores</li>
<li>Search for Linear Splits</li>
</ul>
Limitations of a single decision tree:
<ul>
<li>Single feature for root node</li>
<li>Splitting criteria can lead to some features not being used</li>
<li>Potential for overfitting to data</li>
</ul>
References:

<ul>
<li><a target="_blank" href="https://www.statlearning.com/">An Introduction to Statistical Learning</a> (Download free pdf)</li>
<li><a target="_blank" href="https://www.udemy.com/course/python-for-machine-learning-data-science-masterclass/">Jose Portilla's 2021 Python for Machine Learning &amp; Data Science Masterclass</a> </li>
</ul>
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1632758712523/zMprCMvJu.png" alt="15_ml_9_27Sep21.png" />

Sections 9 and 10 are on tree-based methods. There are three main methods:
- Decision Trees (Section 9)
-  [Random Forests (Section 10)](https://nurfadhilah.tech/section-10-random-forests-and-boosted-trees) 
-  [Boosted Trees (Section 10)](https://nurfadhilah.tech/section-10-random-forests-and-boosted-trees) 

Each of these methods stems from the basic decision tree algorithm. Fundamentally, tree-based methods rely on the ability to split data based on information from features. Require a mathematical definition of information and the ability to measure it.

Classification and Regression Tree (CART) introduces many concepts:
- Cross validation of Trees
- Pruning Trees
- Surrogate Splits
- Variable Importance Scores
- Search for Linear Splits

Limitations of a single decision tree:
- Single feature for root node
- Splitting criteria can lead to some features not being used
- Potential for overfitting to data

**References:
**
- [An Introduction to Statistical Learning](https://www.statlearning.com/)  (Download free pdf)
- [Jose Portilla's 2021 Python for Machine Learning & Data Science Masterclass](https://www.udemy.com/course/python-for-machine-learning-data-science-masterclass/) 

![15_ml_9_27Sep21.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1632758712523/zMprCMvJu.png)


Section 9: Decision Trees