GBDT回归的python官方例子详解

妖狐艹你老母 2022-09-27 06:50 259阅读 0赞

简单的GBDT调用见官方例子：

方法地址：[http://scikit-learn.org/stable/auto\_examples/ensemble/plot\_gradient\_boosting\_regression.html\#sphx-glr-auto-examples-ensemble-plot-gradient-boosting-regression-py][http_scikit-learn.org_stable_auto_examples_ensemble_plot_gradient_boosting_regression.html_sphx-glr-auto-examples-ensemble-plot-gradient-boosting-regression-py]

import
       
        numpy as np 
       
        import
       
        matplotlib.pyplot as plt 
       
        from
       
        sklearn 
       
        import
       
        ensemble 
       
        from
       
        sklearn 
       
        import
       
        datasets 
       
        from
       
        sklearn.utils 
       
        import
       
        shuffle 
       
        from
       
        sklearn.metrics 
       
        import
       
        mean_squared_error boston 
       
        =
       
        datasets.load_boston() X, y 
       
        =
       
        shuffle(boston.data, boston.target, random_state
       
        =
       
        13
       
        ) X 
       
        =
       
        X.astype(np.float32) offset 
       
        =
       
        int(X.shape[0] 
       
        *
       
        
       
        0.9
       
        ) X_train, y_train 
       
        =
       
        X[:offset], y[:offset] X_test, y_test 
       
        =
       
        X[offset:], y[offset:] params 
       
        =
       
        { 
       
        '
       
        n_estimators
       
        '
       
        : 
       
        500
       
        , 
       
        '
       
        max_depth
       
        '
       
        : 
       
        4
       
        , 
       
        '
       
        min_samples_split
       
        '
       
        : 
       
        2
       
        , 
       
        '
       
        learning_rate
       
        '
       
        : 
       
        0.01
       
        , 
       
        '
       
        loss
       
        '
       
        : 
       
        '
       
        ls
       
        '
       
        } clf 
       
        =
       
        ensemble.GradientBoostingRegressor(
       
        **
       
        params) clf.fit(X_train, y_train) 
       
        #
       
        训练
       
        
       
        mse 
       
        =
       
        mean_squared_error(y_test, clf.predict(X_test)) 
       
        #
       
        预测并且计算MSE
       
        #
       
        print 2.7和3.0版本有区别
       
        
       
        print
       
        
       
        "
       
        MSE: %.4f
       
        "
       
        
       
        %
       
        mse 
       
        #
       
        compute test set deviance
       
        
       
        test_score 
       
        =
       
        np.zeros((params[
       
        '
       
        n_estimators
       
        '
       
        ],), dtype
       
        =
       
        np.float64) 
       
        #
       
        矩阵转化
       
        
       
        
       
        for
       
        i, y_pred 
       
        in
       
        enumerate(clf.staged_predict(X_test)): test_score[i] 
       
        =
       
        clf.loss_(y_test, y_pred) plt.figure(figsize
       
        =
       
        (
       
        12
       
        , 
       
        6
       
        )) plt.subplot(
       
        1
       
        , 
       
        2
       
        , 
       
        1
       
        ) plt.title(
       
        '
       
        Deviance
       
        '
       
        ) plt.plot(np.arange(params[
       
        '
       
        n_estimators
       
        '
       
        ]) 
       
        +
       
        
       
        1
       
        , clf.train_score_, 
       
        '
       
        b-
       
        '
       
        , label
       
        =
       
        '
       
        Training Set Deviance
       
        '
       
        ) plt.plot(np.arange(params[
       
        '
       
        n_estimators
       
        '
       
        ]) 
       
        +
       
        
       
        1
       
        , test_score, 
       
        '
       
        r-
       
        '
       
        , label
       
        =
       
        '
       
        Test Set Deviance
       
        '
       
        ) plt.legend(loc
       
        =
       
        '
       
        upper right
       
        '
       
        ) plt.xlabel(
       
        '
       
        Boosting Iterations
       
        '
       
        ) plt.ylabel(
       
        '
       
        Deviance
       
        '
       
        ) 
       
        #
       
        Plot feature importance
       
        
       
        feature_importance 
       
        =
       
        clf.feature_importances_ 
       
        #
       
        make importances relative to max importance
       
        
       
        feature_importance 
       
        =
       
        
       
        100.0
       
        
       
        *
       
        (feature_importance 
       
        /
       
        feature_importance.max()) sorted_idx 
       
        =
       
        np.argsort(feature_importance) pos 
       
        =
       
        np.arange(sorted_idx.shape[0]) 
       
        +
       
        .
       
        5
       
        plt.subplot(
       
        1
       
        , 
       
        2
       
        , 
       
        2
       
        ) plt.barh(pos, feature_importance[sorted_idx], align
       
        =
       
        '
       
        center
       
        '
       
        ) plt.yticks(pos, boston.feature_names[sorted_idx]) plt.xlabel(
       
        '
       
        Relative Importance
       
        '
       
        ) plt.title(
       
        '
       
        Variable Importance
       
        '
       
        ) plt.show()

输出：

[![image][]][image 1]

结论：LSTAT>RM>DIS………

房产的数据结构：

\- CRIM     per capita crime rate by town  
\- ZN       proportion of residential land zoned for lots over 25,000 sq.ft.  
\- INDUS    proportion of non-retail business acres per town  
\- CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)  
\- NOX      nitric oxides concentration (parts per 10 million)  
\- RM       average number of rooms per dwelling  
\- AGE      proportion of owner-occupied units built prior to 1940  
\- DIS      weighted distances to five Boston employment centres  
\- RAD      index of accessibility to radial highways  
\- TAX      full-value property-tax rate per $10,000  
\- PTRATIO  pupil-teacher ratio by town  
\- B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town  
\- LSTAT    % lower status of the population  
\- MEDV     Median value of owner-occupied homes in $1000's

源代码读取csv文件方法详解：

def load_boston():
        """Load and return the boston house-prices dataset (regression).
    
        ==============     ==============
        Samples total                 506
        Dimensionality                 13
        Features           real, positive
        Targets             real 5. - 50.
        ==============     ==============
    
        Returns
        -------
        data : Bunch
            Dictionary-like object, the interesting attributes are:
            'data', the data to learn, 'target', the regression targets,
            and 'DESCR', the full description of the dataset.
    
        Examples
        --------
        >>> from sklearn.datasets import load_boston
        >>> boston = load_boston()
        >>> print(boston.data.shape)
        (506, 13)
        """
        module_path = dirname(__file__)
    
        fdescr_name = join(module_path, 'descr', 'boston_house_prices.rst')
        with open(fdescr_name) as f:  #with用法可以自动关闭，读取文件声明
            descr_text = f.read()
    
        data_file_name = join(module_path, 'data', 'boston_house_prices.csv')
        with open(data_file_name) as f:
            data_file = csv.reader(f)  #csv读取文件，获得迭代器
            temp = next(data_file)     #读取文件头，获得总行数和字段名
            n_samples = int(temp[0])
            n_features = int(temp[1])
            data = np.empty((n_samples, n_features)) #建立矩阵
            target = np.empty((n_samples,))
            temp = next(data_file)  # names of features
            feature_names = np.array(temp)
    
            for i, d in enumerate(data_file):  #一行一行通过迭代器读取文件设置到矩阵中
                data[i] = np.asarray(d[:-1], dtype=np.float)
                target[i] = np.asarray(d[-1], dtype=np.float)
    
        return Bunch(data=data,
                     target=target,
                     # last column is target value
                     feature_names=feature_names[:-1],
                     DESCR=descr_text)

**方法说明：**

1、with是python的一种用法，可以自动关闭对应的资源

2、`next`(*iterator*\[, *default*\])  迭代器遍历方法

3、`reader`(*csvfile*, *dialect='excel'*, *\*\*fmtparams*) ：返回一个迭代器

Return a reader object which will iterate over lines in the given *csvfile*.

类的主要方法介绍：

构造函数：  
loss  
learning\_rate  
n\_estimators  
max\_depth  
criterion  
min\_samples\_split  
min\_samples\_leaf  
min\_weight\_fraction\_leaf  
subsample  
max\_features  
max\_leaf\_nodes  
min\_impurity\_split  
alpha  
init  
verbose  
warm\_start  
random\_state  
presort

属性

方法  
feature\_importances\_ ：重要的特性  
oob\_improvement\_：  
train\_score\_ ：训练分数  
loss\_ ：损失函数  
\`init\` : 评估期  
estimators\_ : ndarray of DecisionTreeRegressor, shape = \[n\_estimators, 1\]

方法：

[![image][image 2]][image_image 2]

后续待补充详细参数说明

[http_scikit-learn.org_stable_auto_examples_ensemble_plot_gradient_boosting_regression.html_sphx-glr-auto-examples-ensemble-plot-gradient-boosting-regression-py]: http://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_regression.html#sphx-glr-auto-examples-ensemble-plot-gradient-boosting-regression-py
[image]: /images/20220708/291913bd94b64373b9928558abe0553f.png
[image 1]: https://img-blog.csdn.net/20170308133554473
[image 2]: https://img-blog.csdn.net/20170308142439961
[image_image 2]: /images/20220708/7561ed34dd59476182fa666804300a99.png