grep sed awk命令学习过滤文件内容

ゞ浴缸里的玫瑰 2022-10-01 00:52 175阅读 0赞

# grep #

grep \[options\] pattern \[**files**\]

Search one or more files for lines that match a regular expression pattern. Exit status is 0 if any lines match, 1 if none match, and 2 for errors. See also egrep and fgrep.

Options  
**-c**, –count  
Print only a count of matched lines. With -v or –revert-match option, count nonmatching lines.  
\-i, –ignore-case  
Ignore uppercase and lowercase distinctions.  
\-l, –files-with-matches  
List the names of **files** with matches but not individual matched lines; scanning per file stops on the first match.  
\-n, –line-number  
Print lines and their line numbers.  
\-q, –quiet, –silent  
Suppress normal output in favor of quiet mode; scanning stops on the first match.  
\-v, –invert-match  
Print all lines that don’t match pattern.  
\-E, -extended-regexp  
Act like **egrep**, recognizing extended regular expressions such as (UN|POS)IX to find UNIX and POSIX.  
\-F, –fixed-strings  
Act like **fgrep**, recognizing only fixed strings instead of regular expressions. Useful when searching for characters that grep normally recognizes as metacharacters.

Examples  
\[root@localhost-ziyi home\]\# grep -c /bin/ /etc/passwd  
6  
\[root@localhost-ziyi home\]\# grep ‘^\#include’ /usr/include/zlib.h  
\#include “zconf.h”  
\[root@localhost-ziyi home\]\# grep **-l** ‘^\#include’ /usr/include/zlib.h  
/usr/include/zlib.h  
\[root@localhost-ziyi home\]\# grep **-n** ‘^\#include’ /usr/include/zlib.h  
34:\#include “zconf.h”  
\[root@localhost-ziyi home\]\#grep –E “3306|1521” /etc/services

--------------------

*  sed  
    此文甚好，向高人学习了  
    [http://www.cnblogs.com/emanlee/archive/2013/09/07/3307642.html][http_www.cnblogs.com_emanlee_archive_2013_09_07_3307642.html]
    
    Stream editor(流编辑器，一次处理一行内容) for filtering and transforming text. A stream editor is used to perform basic text transformations on an input stream(a file or input from a pipeline).  
    \-e : script,  
    add the script to the commands to be executed  
    \-d :  
    add the contents of script-file to the commands to be executed
    
     *  删除以空行开头的人  
        \#sed ‘/^$/d’ test.txt
     *  p 打印  
        \[root@localhost-ziyi /\]\# sed /oldboy/**p** /data/test.txt **p:print**  
        test  
        liyao  
        oldboy  
        Oldboy
     *  \-n取消默认输出  
        \[root@localhost-ziyi /\]\# sed **-n** /oldboy/p /data/test.txt n:仅 打印被过滤的, 取消默认输出  
        Oldboy
     *  打印多行  
        sed -n ‘20,30p’ ett.txt n:取消默认输出
     *  **-i** :in place  
        sed **–i** s\#SELINUX=enforcing\#SELINUX=disabled\#g /etc/selinux/config  
        **-r**: 忽略正则, use extended regular expressions in the script.  
        sed -**rn** ‘s\#^.*dr:(.*) Bc.\*$\#\\1\#gp’  
        基本sed编程举例:  
        使用p(rint)显示行: sed -n ‘2p’ temp.txt 只显示第2行,使用选项n  
        打印范围: sed -n ‘1,3p’ temp.txt 打印第1行到第3行  
        打印模式: sed -n ‘/movie/’p temp.txt 打印含movie的行  
        使用模式和行号查询: sed -n ‘3,/movie/’p temp.txt 只在第3行查找movie并打印  
        显示整个文件: sed -n ‘1,$’p temp.txt $为最后一行  
        任意字符: sed -n ‘/.\*ing/’p temp.txt 注意是.\*ing,而不是\*ing  
        打印行号: sed -e ‘/music/=’ temp.txt  
        附加文本:(创建sed脚本文件)chmod u+x script.sed,运行时./script.sed temp.txt  
        \#!/bin/sed -f  
        /name1/ a/ \#a/表示此处换行添加文本  
        HERE ADD NEW LINE. \#添加的文本内容  
        插入文本: /name1/ a/ 改成 4 i/ 4表示行号,i插入  
        修改文本: /name1/ a/ 改成 /name1/ c/ 将修改整行,c修改  
        删除文本: sed ‘1d’ temp.txt 或者 sed ‘1,4d’ temp.txt  
        替换文本: sed ‘s/source/OKSTR/’ temp.txt 将source替换成OKSTR  
        sed ‘s//$//g’ temp.txt 将文本中所有的$符号全部删除  
        sed ‘s/source/OKSTR/w temp2.txt’ temp.txt 将替换后的记录写入文件temp2.txt  
        替换修改字符串: sed ‘s/source/”ADD BEFORE” &/p’ temp.txt  
        结果将在source字符串前面加上”ADD BEFORE”,这里的&表示找到的source字符并保存  
        sed结果写入到文件: sed ‘1,2 w temp2.txt’ temp.txt  
        sed ‘/name/ w temp2.txt’ temp.txt  
        从文件中读文本: sed ‘/name/r temp2.txt’ temp.txt  
        在每列最后加文本: sed ‘s/\[0-9\]\*/& Pass/g’ temp.txt  
        从shell向sed传值: echo $NAME | sed “s/go/$REP/g” 注意需要使用双引号

--------------------

1.  awk  
    
    
    1.  带\[ \]和不带的区别  
        [http://www.cnblogs.com/emanlee/p/3327576.html][http_www.cnblogs.com_emanlee_p_3327576.html]  
        \[root@localhost-ziyi /\]\# awk **/\[^lzc\]/** /data/test.txt  
        test  
        liyao  
        \[root@localhost-ziyi /\]\# awk **/^lzc/** /data/test.txt  
        lzc
    2.  awk 用法：awk ’ pattern \{action\} ’
    3.  变量名的含义

<table> 
 <thead> 
  <tr> 
   <th>变量名</th> 
   <th>含义</th> 
   <th>变量名</th> 
   <th>含义</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>FILENAME</td> 
   <td>当前输入的文件名</td> 
   <td>FS</td> 
   <td>输入域的分隔符，默认为空格</td> 
  </tr> 
  <tr> 
   <td>NF</td> 
   <td>每行域的个数</td> 
   <td>NR</td> 
   <td>每行的行号</td> 
  </tr> 
 </tbody> 
</table>

> 有用的例子  
> \[root@localhost-ziyi home\]\# cat test.txt  
> 10 11 12 13 14 15 16  
> 20 21 22 23 24 25 26  
> 30 31 32 33 34 35 36  
> 40 41 42 43 44 45 46  
> \[root@localhost-ziyi home\]\# awk ‘/11/’ test.txt  
> 10 11 12 13 14 15 16  
> \[root@localhost-ziyi home\]\# awk ‘/11/,/23/’ test.txt  
> 10 11 12 13 14 15 16  
> 20 21 22 23 24 25 26  
> \[root@localhost-ziyi home\]\# awk ‘/^\[10 20\]/’ test.txt  
> 10 11 12 13 14 15 16  
> 20 21 22 23 24 25 26  
> \[root@localhost-ziyi home\]\# awk ‘/^(10|20)/’ test.txt  
> 10 11 12 13 14 15 16  
> 20 21 22 23 24 25 26  
> \[root@localhost-ziyi home\]\# awk ‘$2==11’ test.txt  
> 10 11 12 13 14 15 16  
> \[root@localhost-ziyi home\]\# awk ‘$2==”11”’ test.txt  
> 10 11 12 13 14 15 16  
> \[root@localhost-ziyi home\]\# awk \{‘print NR,NF,$1,$NF’\} test.txt  
> 1 7 10 16  
> 2 7 20 26  
> 3 7 30 36  
> 4 7 40 46  
> \[root@localhost-ziyi home\]\# awk **‘/31/ \{print $1 $2\}’** test.txt  
> 3031  
> \[root@localhost-ziyi home\]\# cat test.txt\*\*|\*\*awk ‘\{print $2\}’ :awk可通过管道获得输入  
> 11  
> 21  
> 31  
> 41  
> \[root@localhost-ziyi home\]\# awk -**F** “5” ‘\{print $1\}’ test.txt ：指定行的分隔符. 也可指定多个：-F ‘\[ :\\t|\]’代表空格，冒号，tab和|  
> 10 11 12 13 14 1  
> 20 21 22 23 24 2  
> 30 31 32 33 34 3  
> 40 41 42 43 44 4  
> \[root@localhost-ziyi home\]\# awk -F “5” ‘\{print $2\}’ test.txt  
> 16  
> 26  
> 36  
> 46  
> \[root@localhost-ziyi home\]\# awk **‘BEGIN \{FS=”5”\}** \{print $1\}’ test.txt ：设定新的分隔符，而且‘’里面可以有多个\{\}。BEGIN 表示在处理任意行之前进行的操作。  
> 10 11 12 13 14 1  
> 20 21 22 23 24 2  
> 30 31 32 33 34 3  
> 40 41 42 43 44 4  
> \[root@localhost-ziyi home\]\# awk ‘/33/ \{count++;\} END \{print “tom was founds “count” times “\}’ test.txt  
> tom was founds 1 times  
> 在awk中调用系统变量必须用单引号，如果是双引号，则表示字符串  
> Flag=abcd  
> awk ‘\{print ‘$Flag’\}’ 结果为abcd  
> awk ‘\{print “$Flag”\}’ 结果为$Flag

[http_www.cnblogs.com_emanlee_archive_2013_09_07_3307642.html]: http://www.cnblogs.com/emanlee/archive/2013/09/07/3307642.html
[http_www.cnblogs.com_emanlee_p_3327576.html]: http://www.cnblogs.com/emanlee/p/3327576.html