深度学习-自然语言处理(NLP)-第三方库(工具包)：sumy【抽取网页、文本、文件中的“关键句”】

Dear 丶 2023-10-04 10:22 21阅读 0赞

## 一、抽取中文文章的关键句【自动摘要】 ##

打印结果：

urlContentSummary = 
    
    ******************
    ================================================================================
    plainTextSummary = 
    
    ******************
    ================================================================================
    fileDataSummary = 
    
    ******************
    
    Process finished with exit code 0

## 二、抽取英文文章的关键句【自动摘要】 ##

#!usr/bin/env python
    # encoding:utf-8
    from __future__ import division
    
    '''
    __Author__:沂水寒城
    功能：基于开源模块sumy的简单文本摘要
    文本摘要方法参考学习可以借鉴阮一峰下面的文章：
    http://www.ruanyifeng.com/blog/2013/03/automatic_summarization.html
    '''
    import nltk
    nltk.download('punkt')
    
    from sumy.nlp.stemmers import Stemmer
    from sumy.utils import get_stop_words
    from sumy.parsers.html import HtmlParser
    from sumy.nlp.tokenizers import Tokenizer
    from sumy.parsers.plaintext import PlaintextParser
    from sumy.summarizers.lsa import LsaSummarizer as Summarizer
    
    # 设置输出句子总数
    SENTENCES_COUNT = 5
    
    
    def urlContentSummary(url, language):
        '''
        基于URL内容的文本的摘要方法
        '''
        parser = HtmlParser.from_url(url, Tokenizer(language))
        stemmer = Stemmer(language)
        summarizer = Summarizer(stemmer)
        summarizer.stop_words = get_stop_words(language)
        print("urlContentSummary = \n")
        for sentence in summarizer(parser.document, SENTENCES_COUNT):
            print(sentence)
    
    
    def plainTextSummary(data, language):
        '''
        基于明文数据内容的摘要方法
        '''
        parser = PlaintextParser.from_string(data, Tokenizer(language))
        stemmer = Stemmer(language)
        summarizer = Summarizer(stemmer)
        summarizer.stop_words = get_stop_words(language)
        print("plainTextSummary = \n")
        for sentence in summarizer(parser.document, SENTENCES_COUNT):
            print(sentence)
    
    
    def fileDataSummary(files, language):
        '''
        基于文件数据内容的摘要方法
        '''
        parser = PlaintextParser.from_file(files, Tokenizer(language))
        stemmer = Stemmer(language)
        summarizer = Summarizer(stemmer)
        summarizer.stop_words = get_stop_words(language)
        print("fileDataSummary = \n")
        for sentence in summarizer(parser.document, SENTENCES_COUNT):
            print(sentence)
    
    
    if __name__ == "__main__":
        # url = "https://lareviewofbooks.org/article/a-pulse-on-the-eye-the-art-of-helen-frankenthaler/"
        data = """AN ART HISTORIAN at Stanford University, Alexander Nemerov never pretends he’s an objective observer in his masterful new biography, Fierce Poise: Helen Frankenthaler and 1950s New York.
                    He seems to take pleasure in throwing caution to the wind.
                    His narrative combines an intense infatuation for his subject with his own autobiographical confessions, and the result is a dazzling collage of impressions and interpretations that leaves the reader spellbound.
                    
                    Nemerov never actually met Helen Frankenthaler, the abstract expressionist painter who died at the age of 83 in 2011, but his father taught the artist when she attended Bennington in 1963.
                    Despite never having had a direct encounter with her, he feels as if he has always known her, prompting his decision to address her as Helen throughout the book.
                    To do otherwise, he insists, would seem inauthentic to him.
                    He fell in love with her work early on but was not able to tackle this project until a few decades had passed.
                    He concedes that he spent his 20s lost in some sort of undefined malaise — unlike Helen who, at a young age, was already painting some of her most cherished works.
                    But with marriage, fatherhood, and the death of his beloved father, Nemerov finally felt ready to take her on full throttle.
                    
                    Before Nemerov even finishes his unorthodox introduction, he has piqued our interest.
                    He tells us that he was always impressed by the size and stunning colors that infused Helen’s works, and the way she seemed to notice things in ordinary daily life that others missed, such as “the neon sign’s reflection in a wintry puddle, the cream of dirty white at the lip of the wave.” He saw bursts of euphoria springing from her canvasses, eruptions that were somehow not frivolous or silly, and which showed him that “joy itself was serious, that prettiness had its edges, and that guilt, anger, and indignation were not the only games in town.”
                    
                    Helen Frankenthaler was born in 1928, the daughter of wealthy German Jews.
                    Her father was New York Supreme Court Justice Alfred Frankenthaler, and Helen was the clear favorite of his three girls.
                    Her artistic bent was obvious early on.
                    She would draw with chalk on the sidewalk, would sometimes spill droplets of her mother’s blood-red nail polish into a bucket of water, delighting in the magical swirls that instantly appeared.
                    But after her father’s death when she was only 11, she began having migraines and bouts of hypochondria.
                    She was helped by a kind headmistress at The Dalton School who paid her special attention.
                    She also took solace in art class, where her teacher, Rufino Tamayo, encouraged her to make large, brightly colored portraits inspired by Cubism and Surrealism.
                    He taught her the mechanics of painting: how to mix varnish, turpentine, linseed oil, and tube pigments.
                    She adored him, but when he criticized anything about her work, she shrank away in disgust.
                    Nemerov believes this is because she already was imbued with an innate confidence and inner vision that was immune to criticism of any sort.
                    
                    Helen also developed an affinity for her art professor at Bennington, Paul Feeley, who showed her how to paint in the style of Picasso.
                    Feeley was impressed by her devotion and seriousness.
                    Nemerov writes, “Helen felt that art was more than a polite skill, a bit of polish and refinement, more than recipe making or getting married or a way to pass the time,” but rather “a religious enterprise.”
                    
                    Nemerov devotes large swaths of his book to Helen’s tumultuous relationships with men who were usually more powerful than she was in the art world, not to mention considerably older.
                    Her relationship with Clement Greenberg, the esteemed art critic, went on for years, enduring tumultuous highs and lows before it deteriorated into complete chaos.
                    Helen claims she could no longer tolerate his moodiness and controlling nature.
                    Without saying so, Nemerov unconsciously seems to take her side, refusing to come out and directly censure her sometimes questionable behavior.
                    Despite the difficulties the couple endured, Helen was forever thankful to Greenberg for introducing her to Jackson Pollock, who inspired her like no other artist.
                    She adored the way Pollock worked: sloppily rolling out his large canvas on his studio floor, moving it around haphazardly before beginning to pour paint in swirls and dots, varied patterns that seemed both “improvised and controlled.” There was a radiant energy, she felt, that glowed from his paintings, and she knew it was something she could emulate with her own style and vision.
                    
                    Years later, she married the abstract expressionist painter Robert Motherwell, who already was the father of two small girls from an earlier marriage.
                    Their initial years together were promising, but she eventually found herself drowning in her partner’s increasingly erratic bouts of depression and binge drinking.
                    Nemerov once again seems to go mute when it comes to Helen’s role in the demise of their decade-long marriage.
                    
                    Let me be clear.
                    It is not that Nemerov is unaware that Helen had difficulties dealing with people.
                    Intimacy was not her strong suit.
                    She could be insensitive, manipulative, obtuse, competitive, and mercilessly self-absorbed.
                    Nemerov acknowledges these faults several times over.
                    He even goes so far as to point out that Motherwell’s daughters, now grown, recall how hurt and shocked they were when Helen asked them one day out of the blue if she could adopt them, seemingly unaware of their devotion to their real mother.
                    Nemerov doesn’t avoid such harsh revelations; rather, he states them for the record and moves on, as if he is unwilling or unable to fully deal with them.
                    He remains a starry-eyed lover of sorts, lost in his own enchantment.
                    
                    Nemerov lingers over one of Helen’s most famous works, Mountains and Sea (1952), a large landscape filled with blue, salmon, red, and sea-foam green.
                    No definitive forms emerge from the canvas, which the artist knew was something spectacular when she finished it.
                    She felt that she had mastered the perfect fusion of structure and spontaneity that Pollock had shown her, but in her own unique style.
                    Greenberg, her partner at the time, was awestruck by it, insisting that it represented a rawness and unselfconsciousness that was symbolic of an essentially Jewish yearning for freedom — not the sort of freedom their ancestors had sought, from enslavement and foreign domination, but freedom from the constraints and tedium of traditional Jewry itself.
                    
                    Nemerov takes us on a magical mystery tour of his own mind, as he imagines Helen painting another one of her masterworks, Open Wall (1953).
                    He pictures himself in the studio with her, though she is unaware that she is being observed.
                    He thinks to himself that she must have
                    started at left center, with the two sinuous vertical columns of pink.
                    Perhaps then she moved on to the left, where the rectangular slab of blue gradually asserted itself under her eyes, staging the need for a matching form on the right, which became the wild field of bronze.
                    Bigger and more varicolored and reckless than its blue counterpart, the bronze also extends a dark stalk toward the picture’s center.
                    Then perhaps, stepping back, she reexamined the pink columns.
                    With their lively, fretted contours, elastic and bending, the long shapes call to mind an abstract form of “nude,” a bending Daphne or Venus, at any rate something sensual and energetic, a pulse on the eye.
                    
                    
                    When he finishes his fantasy, we realize that he has shown us what it might actually feel like to stand in front of blankness and create beauty and meaning out of our own deepest yearnings.
                    
                    Nemerov is not hampered by a desire to remain impartial, and that is his strength.
                    He has a unique sensibility that allows him to imaginatively show us not only the person Helen might have been but also who he is or was, and what they both might become together.
                    It is their collision, even with its blind spots, that takes center stage here, setting off brilliant sparks of perception and recognition.
         """
    
        # urlContentSummary(url, 'english')
        # print('=' * 80)
    
        plainTextSummary(data, 'english')
        print('=' * 80)
    
        fileDataSummary('test_english.txt', 'english')

打印结果：

[nltk_data] Downloading package punkt to
    [nltk_data]     C:\Users\Administrator\AppData\Roaming\nltk_data...
    [nltk_data]   Package punkt is already up-to-date!
    plainTextSummary = 
    
    AN ART HISTORIAN at Stanford University, Alexander Nemerov never pretends he’s an objective observer in his masterful new biography, Fierce Poise: Helen Frankenthaler and 1950s New York.
    He tells us that he was always impressed by the size and stunning colors that infused Helen’s works, and the way she seemed to notice things in ordinary daily life that others missed, such as “the neon sign’s reflection in a wintry puddle, the cream of dirty white at the lip of the wave.” He saw bursts of euphoria springing from her canvasses, eruptions that were somehow not frivolous or silly, and which showed him that “joy itself was serious, that prettiness had its edges, and that guilt, anger, and indignation were not the only games in town.”
    Nemerov writes, “Helen felt that art was more than a polite skill, a bit of polish and refinement, more than recipe making or getting married or a way to pass the time,” but rather “a religious enterprise.”
    Nemerov devotes large swaths of his book to Helen’s tumultuous relationships with men who were usually more powerful than she was in the art world, not to mention considerably older.
    He even goes so far as to point out that Motherwell’s daughters, now grown, recall how hurt and shocked they were when Helen asked them one day out of the blue if she could adopt them, seemingly unaware of their devotion to their real mother.
    ================================================================================
    fileDataSummary = 
    
    AN ART HISTORIAN at Stanford University, Alexander Nemerov never pretends he’s an objective observer in his masterful new biography, Fierce Poise: Helen Frankenthaler and 1950s New York.
    He tells us that he was always impressed by the size and stunning colors that infused Helen’s works, and the way she seemed to notice things in ordinary daily life that others missed, such as “the neon sign’s reflection in a wintry puddle, the cream of dirty white at the lip of the wave.” He saw bursts of euphoria springing from her canvasses, eruptions that were somehow not frivolous or silly, and which showed him that “joy itself was serious, that prettiness had its edges, and that guilt, anger, and indignation were not the only games in town.”
    Nemerov writes, “Helen felt that art was more than a polite skill, a bit of polish and refinement, more than recipe making or getting married or a way to pass the time,” but rather “a religious enterprise.”
    Nemerov devotes large swaths of his book to Helen’s tumultuous relationships with men who were usually more powerful than she was in the art world, not to mention considerably older.
    He even goes so far as to point out that Motherwell’s daughters, now grown, recall how hurt and shocked they were when Helen asked them one day out of the blue if she could adopt them, seemingly unaware of their devotion to their real mother.
    
    Process finished with exit code 0