nlp中文本处理的一些常用方法

太过爱你忘了你带给我的痛 2022-03-22 17:52 252阅读 0赞

##### 从sentence(str)中找到会重复出现的多位的keyword的起始位置与结束位置的索引 #####

def get_key_idxs(sentence, keyword):
       k_len = len(keyword)
       res = []
       for i in range(len(sentence) - k_len + 1):
           if sentence[i:i+k_len] == keyword:
               s = str(i) + '-' +str(i+k_len)
               res.append(s)
       return res
               
    sentence = '只有小米公司才生成小米手机'
    keyword = '小米'
    get_key_idxs(sentence, keyword)    # ['2-4', '9-11']

##### 从sentence(str)中找到会重复出现的一位的keyword的索引 #####

sentence = '只有小米公司才生成小米手机'
    keyword = '小'
    indx = [i for i, k in enumerate(sentence) if k == keyword]  # [2,9]

##### 计算文本间距 #####

def text_distance(str1, str2):
        m, n = len(str1), len(str2)
        dp = [[0 for x in range(n+1)] for x in range(m+1)]
        for i in range(m+1):
            for j in range(n+1):
                if i == 0:
                    dp[i][j] = j
                elif j == 0:
                    dp[i][j] = i
                elif str1[i-1] == str2[j-1]:
                    dp[i][j] = dp[i-1][j-1]
                else:
                    dp[i][j] = 1 + min(dp[i][j-1],      # Insert
                                      dp[i-1][j],       # Remove
                                      dp[i-1][j-1])     # Replace
        return dp[m][n]
    
    text_distance('go', 'going')     # output:3
    text_distance('whose', 'who')     # output:2
    text_distance('who', 'how')     # output:2

##### 二维list转一维 #####

a = [[1,2],[3,4]]
    b = [i for k in l for i in k]   #  [1,2,3,4]

##### 将数组中所有0转为1，所有1转为0 #####

a = [1,0,0,1,0,0,1]
    for i in range(len(a)):
        a[i] ^= 1             #  [0, 1, 1, 0, 1, 1, 0]

滚回去做测试了，此篇停更…