NLP-3:机器学习和模型评价初步

ゝ一世哀愁。 2023-05-30 09:15 10阅读 0赞

--------------------

title: ‘NLP-3:机器学习和模型评价初步’  
date: 2019-11-08 23:14:24  
mathjax: true  
categories:

*  nlp-自然语言处理  
    tags:
 *  nlp-自然语言处理

--------------------

### 文章目录 ###

*  NLP-3:机器学习和模型评价初步
 *   *  KNN(K-nearest neighbors,K-近邻算法)
     *  决策树模型(decision tree)
     *   *  决策树如何决定哪一个值或者哪一个特征进行分割
     *  Naive Bayesian Classification (朴素贝叶斯)
     *  K-Means算法

# NLP-3:机器学习和模型评价初步 #

*  由经验生成的数据。
    
    1.  优点：全面
    2.  缺点：抽象性弱，数据多且冗余，不易修改维护

## KNN(K-nearest neighbors,K-近邻算法) ##

*  如果一个样本在特征空间中的 k 个最相似(即特征空间中最邻近)的样本中的大多数属于某一个类别，则该样本也属于这个类别
    
    ![format_png][]

## 决策树模型(decision tree) ##

*  决策树模型 (Decision Tree)。将问题的抽象层次变高。减少了 The First Book 的数据。
    
    关键是找到合适的分割条件。将最显著的特征放在最前面，权重最大
 *  信息熵
    
     E n t r o y = − ∑ i m P r ( x i ) l o g ( P r ( x i ) ) Entroy =- \\sum\_i^m Pr(x\_i) log(Pr(x\_i)) Entroy=−i∑mPr(xi)log(Pr(xi))  
    熵指的是一个群体的混论程度就是熵
    
        def entropy(elements):
            '''群体混乱程度'''
            counter = Counter(elements)
        # print(counter)类似于统计map一样，键是元素，值是次数
        
            probs = [counter[c]/len(elements) for c in set(elements)]#这里只是使用的set函数而已
            #print(probs)
            return -sum(p*np.log(p) for p in probs)
        entropy([1,1,1,1])
        entropy([1,2,3,4])
        #这里只是混乱程度不是大小
        entropy([1,2,2,3])

### 决策树如何决定哪一个值或者哪一个特征进行分割 ###

*  示例
    
        mock_data = { 
            'gender':['F', 'F', 'F', 'F', 'M', 'M', 'M'],
            'income': ['+10', '-10', '+10', '+10', '+10', '+10', '-10'],
            'family_number': [1, 1, 2, 1, 1, 1, 2],
            'bought': [1, 1, 1, 0, 0, 0, 1],
        }
        import pandas as pd
        dataset = pd.DataFrame(mock_data)# 将set组装成下面模样
        dataset
    
    ![format_png 1][]
 *  其实是为了寻找一个分割的线
    
    split\_by gender:  
    \[1,1,1,0\],\[0,0,1\]  
    split\_by\_income:  
    \[1,1,0,0,0\],\[1,1\]  
    split\_by\_family\_number:  
    \[1,1,0,0,0\] \[1,1\]  
    if split\_by\_some\_feature:  
    \[1,1,1,1\],\[0,0,0\]  
    如果存在这样数据，那么对我们信息切分最好:也就是优先级最高  
    entropy(第一部分)和entropy(第二部分)为0
 *      print(entropy([1,1,1,0])+entropy([0,0,1]))
        print(entropy([1,1,0,0,0])+entropy([1,1]))
        print(entropy([1,1,1,1])+entropy([0,0,0]))
 *  决策树在选择决策的过程，决策顺序的其实是按照某种特征，进行分割之后按照熵最小原则进行的

def find_the_min_spilter(training_data:pd.DataFrame,target:str)->str:
        x_fields = set(training_data.columns.tolist())-{ target}
        #print(x_fields)
        spliter = None
        min_entropy = float('inf')
        for f in x_fields:
            ic(f)
            values = set(training_data[f])#这里面的f就是每条记录的结果,类似于列的列头,然后把所有的结果进行统计，放到数组里面
            ic(values)
            for v in values:
                sub_spliter_1 = training_data[training_data[f] == v][target].tolist()#统计某一个值中买了的有多少，转化为list
                ic(sub_split_1)
                entroy_1 = entropy(sub_spliter_1)#信息熵
                ic(entroy_1)
                sub_spliter_2 = training_data[training_data[f] !=v ][target].tolist()
                ic(sub_split_2)
                entroy_2 = entropy(sub_spliter_2)
                ic(entroy_2)
                entroy_v = entroy_1 + entroy_2
                ic(entroy_v)
                if entroy_v <= min_entropy:
                    min_entropy = entroy_v
                    spliter = (f,v)
                    #print(spliter)
    # 这个代码是冗余的，
        print('spliter is: {}'.format(spliter))
        print('the min entropy is: {}'.format(min_entropy))
        return spliter

## Naive Bayesian Classification (朴素贝叶斯) ##

![format_png 2][]

[朴素贝叶斯朴素在哪里][Link 1]

## K-Means算法 ##

*  选择K个点作为初始质心 repeat 将每个点指派到最近的质心，形成K个簇 重新计算每个簇的质心 until 簇不发生变化或达到最大迭代次数
 *  数据示例：
    
        from sklearn.cluster import KMeans
        X1= [random.randint(0, 100) for _ in range(100)]
        X2 = [random.randint(0, 100) for _ in range(100)]
        plt.scatter(X1, X2)
    
    ![image][]
 *  数据示例
    
        tranning_data = [[x1, x2] for x1, x2 in zip(X1, X2)]
        cluster = KMeans(n_clusters=3,max_iter=500)
        cluster.cluster_centers_
        from collections import defaultdict
        centers = defaultdict(list)
        for label, location in zip(cluster.labels_, tranning_data):
            centers[label].append(location)
        color = ['red', 'green', 'grey', 'black', 'yellow', 'orange']
        
        for i, c in enumerate(centers):
            for location in centers[c]:
                plt.scatter(*location, c=color[i])
                
        for center in cluster.cluster_centers_:
            plt.scatter(*center, s=100)
    
    ![image][image 1]
 *  计算复杂度
    
     O ( I ∗ N ∗ K ∗ d ) O(I\*N\*K\*d) O(I∗N∗K∗d)
    
    1.  I:迭代次数
    2.  N：点的数目
    3.  K：聚类的数目
    4.  d：距离

[format_png]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9idWNrZXQtMTI1ODc0MTcxOS5jb3MuYXAtYmVpamluZy5teXFjbG91ZC5jb20vTmF0dXJhbC1MYW5ndWFnZS1Qcm9jZXNzaW5nLUxlY3R1cmUwMy8xNTcyMDc0NTUzNTg0LnBuZw?x-oss-process=image/format,png
[format_png 1]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9idWNrZXQtMTI1ODc0MTcxOS5jb3MuYXAtYmVpamluZy5teXFjbG91ZC5jb20vTmF0dXJhbC1MYW5ndWFnZS1Qcm9jZXNzaW5nLUxlY3R1cmUwMy8xNTcyMDc5NzE5ODM2LnBuZw?x-oss-process=image/format,png
[format_png 2]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly91Yy1yLmdpdGh1Yi5pby9wdWJsaWMvaW1hZ2VzL2FuYWx5dGljcy9uYWl2ZV9iYXllcy9uYWl2ZV9iYXllc19pY29uLnBuZw?x-oss-process=image/format,png
[Link 1]: https://blog.csdn.net/weixin_42479155/article/details/102828724
[image]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9idWNrZXQtMTI1ODc0MTcxOS5jb3MuYXAtYmVpamluZy5teXFjbG91ZC5jb20vTmF0dXJhbC1MYW5ndWFnZS1Qcm9jZXNzaW5nLUxlY3R1cmUwMy8xNTcyMDk0NDY0OTQxLnBuZw?x-oss-process=image/format,png
[image 1]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9waG90b3MtMTI1ODc0MTcxOS5jb3MuYXAtYmVpamluZy5teXFjbG91ZC5jb20vaW1hZ2VzLzIwMTkxMTA5MDEwNi5wbmc?x-oss-process=image/format,png