评估 | 自动摘要评估

￡神魔★判官ぃ 2022-12-13 12:56 203阅读 0赞

1.rouge评估，英文中文都可以。

中文，都可以使用rouge评估，代码：

# -*- coding: utf-8 -*-
    from rouge import Rouge
    hypothesis = "我 爱 最美 的 中国"
    reference = "我 爱 中国"
    rouge = Rouge()
    scores = rouge.get_scores(hypothesis, reference)
    print("rouge-1:",scores[0]['rouge-1'])
    print("rouge-2:",scores[0]['rouge-2'])
    print("rouge-l:",scores[0]['rouge-l'])

效果：

rouge-1: \{'f': 0.7499999953125, 'p': 0.6, 'r': 1.0\}  
rouge-2: \{'f': 0.33333332888888895, 'p': 0.25, 'r': 0.5\}  
rouge-l: \{'f': 0.7499999953125, 'p': 0.6, 'r': 1.0\}

英文：

# -*- coding: utf-8 -*-
    from rouge import Rouge
    hypothesis = "there is an apple on the desktop"
    reference = "there is an apple"
    rouge = Rouge()
    scores = rouge.get_scores(hypothesis, reference)
    print("rouge-1:",scores[0]['rouge-1'])
    print("rouge-2:",scores[0]['rouge-2'])
    print("rouge-l:",scores[0]['rouge-l'])

效果：

rouge-1: \{'f': 0.7272727226446282, 'p': 0.5714285714285714, 'r': 1.0\}  
rouge-2: \{'f': 0.6666666622222223, 'p': 0.5, 'r': 1.0\}  
rouge-l: \{'f': 0.7272727226446282, 'p': 0.5714285714285714, 'r': 1.0\}

2.pyrouge评估，只有英文可以

# -*- coding: utf-8 -*-
    from pyrouge import Rouge155
    r = Rouge155()
    
    r.system_dir = 'system_summaries'
    r.model_dir = 'model_summaries'
    r.system_filename_pattern = '(\d+).txt' #参考摘要
    r.model_filename_pattern = '#ID#_reference.txt'#生成摘要
    
    output = r.convert_and_evaluate()
    print(output)

**注意命名格式**

效果：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3Nvbmd4aWFvbGluZ2Jhb2Jhbw_size_16_color_FFFFFF_t_70][]

区别：后者可以显示更多评估，前者是只有rouge-1、rouge-2、rouge-l三种。

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3Nvbmd4aWFvbGluZ2Jhb2Jhbw_size_16_color_FFFFFF_t_70]: /images/20221123/3277a24536c745afa2443728e4252bbf.png