大数据五次作业回顾

傷城~ 2023-09-25 22:39 105阅读 0赞

### 文章目录 ###

*   *  1. 大数据作业1
     *   *  1.本地运行模式部分
         *  2. 使用scp安全拷贝部分
     *  2. 大数据作业2
     *   *  1、Rrsync远程同步工具部分
         *  2、xsync集群分发脚本部分
         *  3、集群部署部分
     *  3. 大数据作业3
     *   *  1. 配置历史服务器及日志
         *  2. 日志部分
         *  3. 其他
     *  4. 大数据作业4
     *   *  编写本地wordcount案例
         *  一、源代码
         *  二、信息截图
     *  5. 大数据作业5
     *   *  编写手机号码流量统计案例
         *  一、源代码
         *  二、信息截图

## 1. 大数据作业1 ##

作业内容：  
1.本地运行模式  
1）在hadoop100中创建wcinput文件夹  
2）在wcinput文件下创建一个姓名.txt文件  
3）编辑文件，在文件中输入单词，单词包括自己姓名  
4）执行程序，并查看结果，要求结果打印每个词出现了几次

2.使用scp安全拷贝  
1）分别在hadoop100、hadoop102、hadoop103中新建文件夹及自己姓名1.txt、自己姓名2.txt文件  
1）在hadoop100上将姓名1.txt文件拷贝至102得对应文件夹中  
2）在hadoop102上拷贝hadoop100中姓名2.txt文件  
3）在hadoop102上将hadoop100中姓名1.txt、姓名2.txt文件拷贝到hadoop103对应文件夹中

### 1.本地运行模式部分 ###

![在这里插入图片描述][b9203ffad8e345c8bb62cee00c38aef5.png]  
![在这里插入图片描述][9b6037de02b24e86aa11d54a23777444.png]  
![在这里插入图片描述][07b681a79b8c41988d8b9f3a1393dc95.png]

![在这里插入图片描述][4d3957afb996453182c273eae2b1c68c.png]

### 2. 使用scp安全拷贝部分 ###

![在这里插入图片描述][3c131e34d3084fe993783d15ea1c9a43.png]

![在这里插入图片描述][05947f34a4424072905cffdd71ab8fe7.png]  
![在这里插入图片描述][08b3b60140b74e9eaefc3c59a1f4654c.png]

![在这里插入图片描述][e612fd0ce35d4804910215618863e453.png]  
![在这里插入图片描述][999d1b63bafa4ad4b717e986a3acdf19.png]

## 2. 大数据作业2 ##

作业内容：

1.  Rrsync远程同步工具  
    1）删除hadoop102中/opt/module下hadoop-3.1.3文件夹  
    2）使用rsync将hadoop中/opt/module下hadoop-3.1.3文件夹发送至hadoop102相同目录下
2.  xsync集群分发脚本

1.  理解脚本内容
2.  在hadoop中编写脚本
3.  将环境变量my\_env分发到102和103服务器中，并实现免密功能

1.  集群部署  
    1）修改配置文件并启动集群  
    2）分别在本机和集群创建姓名+学号文件夹  
    3）创建姓名.txt并上传至集群中

### 1、Rrsync远程同步工具部分 ###

![在这里插入图片描述][7f8b01eda4334b998c8b57eb27c2744d.png]  
![在这里插入图片描述][b63f4174e7a24c3cad6640c518272679.png]

![在这里插入图片描述][0ed68bcaa2e44f60977f4c9555a46f17.png]

![在这里插入图片描述][104af2aaf73b4849b03d72f862056c51.png]

### 2、xsync集群分发脚本部分 ###

![在这里插入图片描述][7c5bc9cb302244c187800fa5caaf3ad4.png]

![在这里插入图片描述][92e5f5c331d64116bf37f6e8981b2512.png]  
![在这里插入图片描述][5fc8f81412a44c0dbfa78ebdad13efe3.png]  
![在这里插入图片描述][880b0a51205f4c3caa62c8500daa1586.png]  
![在这里插入图片描述][b2c647be4ab14c649aad8c93615059b0.png]

![在这里插入图片描述][620a4414ed254e518987df98dff2b56e.png]  
![在这里插入图片描述][a3a6be156c4e44689e78e44eeb08391c.png]

![在这里插入图片描述][74880dc9ea734ba3adccf559c85d5525.png]

![在这里插入图片描述][f54042c115d249259b33278fbca32be8.png]

![在这里插入图片描述][962deb532236484a87a19a21dfc7b2e6.png]  
![在这里插入图片描述][e50dfc53f3494d14a46a8b8a9a760377.png]  
![在这里插入图片描述][967936ecd12348a998671efafcdefdfe.png]

![在这里插入图片描述][cea436f13b0442ebbb49b07f27e908e5.png]  
![在这里插入图片描述][90caf1a6965b4ca1a356bb82a4727c61.png]

![在这里插入图片描述][041fab1c095d4056b32f6549b4cab6d7.png]

![在这里插入图片描述][9527bfdf548b4fec811e0b2eeb815ec0.png]

![在这里插入图片描述][a01d2a784d384e0fae73e428068d1d5c.png]

### 3、集群部署部分 ###

3.1 配置core-site.xml  
![在这里插入图片描述][27cc49cafdac45c18166cd15302f54f1.png]

3.2 配置hdfs-site.xml  
![在这里插入图片描述][f1872883ca514d4d8229815e12ecbc33.png]

3.3 配置mapred-site.xml  
![在这里插入图片描述][bb54ba1565264b22959769710058c827.png]  
3.4 配置yarn-site.xml

![在这里插入图片描述][111bfa4190684442a2754b2487bda487.png]

![在这里插入图片描述][f861d872d2f7494ab1923b0c995075d4.png]

![在这里插入图片描述][d2a341a0e041408fb6a4345367394852.png]

![在这里插入图片描述][e4bbfaced68146cab0400dc250657dd3.png]

![在这里插入图片描述][b7de9afd47c64b43913440f9dd61cbab.png]

![在这里插入图片描述][e7f5b9753d9a4f2f859fa9d123912639.png]

![在这里插入图片描述][df989c474dce4456be321af10b34299e.png]  
![在这里插入图片描述][e6aec72ddce247e59df13bada2ce81f1.png]  
![在这里插入图片描述][ddf651a95c66496f8082bdb1b2a2046e.png]

![在这里插入图片描述][2c071f403be749ae865e97a0d136ab5a.png]

![在这里插入图片描述][b30c899b143b43d8babbb4cd0b50fcdd.png]  
![在这里插入图片描述][fe941b6e53ab4d2baea01c863a697795.png]  
![在这里插入图片描述][93775d717d104bb188d78d5680c5742b.png]

![在这里插入图片描述][01db4e78b5b8434298cdd810166494e1.png]  
![在这里插入图片描述][5497ab5bf83d4e219be0e3a78a16dab3.png]

## 3. 大数据作业3 ##

### 1. 配置历史服务器及日志 ###

历史服务器部分：  
![在这里插入图片描述][f996b2539e3c4c1b88a3730248ed429b.png]

![在这里插入图片描述][d374f9c406814225be2454bb55941679.png]

![在这里插入图片描述][9f26895313784642a439ce7dd8afee9c.png]

![在这里插入图片描述][22aa706a1092433181dcbabc77605c98.png]

![在这里插入图片描述][45a4f5399a5e44d585d67dda7a696e8a.png]

![在这里插入图片描述][53250b2bda9f4bd4852f07de5a618c4c.png]

### 2. 日志部分 ###

![在这里插入图片描述][762213c56e9a43eebb543cec5f25babe.png]

![在这里插入图片描述][1cc4772bea2a4af3aa04024396764143.png]

![在这里插入图片描述][cd85765180e4452e99f00a044788041d.png]  
![在这里插入图片描述][0d12a523f5a44d848af3796bfd36588d.png]

### 3. 其他 ###

1.  创建学校文件夹并上传至集群
2.  创建专业.txt（文本内容为姓名）并上传至集群学校文件夹中
3.  创建姓名.txt（文本内容为学号）并拼接至集群专业.txt文件内容中
4.  将集群中拼接后文件下载至本地服务器的当前目录中

![在这里插入图片描述][09edd28f8a184298961cd4b577e570b1.png]

![在这里插入图片描述][0341650c702c4d3f8afca352430e63a4.png]  
![在这里插入图片描述][f6257f316f234bcdbb3da890f1ba1bf0.png]  
![在这里插入图片描述][74c0dff7189841329de5200b3f5719bd.png]

![在这里插入图片描述][89f2d34075634c19a94e3c0c8584450f.png]

![在这里插入图片描述][3be40d60eff443ab9eed7f1a4dd874b9.png]

![在这里插入图片描述][56e33d7bb8c44f72a8334ca520091967.png]

![在这里插入图片描述][39df5d8e4b154b618caeb7136daa451f.png]

![在这里插入图片描述][c8d4fe3ef252453188acdfa062e500a7.png]

## 4. 大数据作业4 ##

### 编写本地wordcount案例 ###

### 一、源代码 ###

1.  com.igeek.mapreduceDemo.wordcount.WordCountDriver

package com.igeek.mapreduceDemo.wordcount;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.io.IntWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Job;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
    
    import java.io.IOException;
    
    public class WordCountDriver {
        
        public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
        
    //        1.获取配置信息，获取job对象实例
            Configuration conf=new Configuration();
            Job job=Job.getInstance(conf);
    //        2.关联本Driver得jar路径
            job.setJarByClass(WordCountDriver.class);
    //        3.关联map和reduce
            job.setMapperClass(WordCountMapper.class);
            job.setReducerClass(WordCountReducer.class);
    //        4.设置map得输出kv类型
            job.setMapOutputKeyClass(Text.class);
            job.setMapOutputValueClass(IntWritable.class);
    //        5.设置最终输出得kv类型
            job.setOutputKeyClass(Text.class);
            job.setOutputValueClass(IntWritable.class);
    //        6.设置输入和输出路径
    //        FileInputFormat.setInputPaths(job,new Path("d:\\Desktop\\Hello.txt"));
            FileInputFormat.setInputPaths(job,new Path("f:\\Documentation\\Hello.txt"));
            FileOutputFormat.setOutputPath(job,new Path("f:\\Documentation\\outHello1"));
    //        FileOutputFormat.setOutputPath(job,new Path("d:\\Desktop\\outHello1"));
    //        7.提交job
            boolean boo=job.waitForCompletion(true);
            System.out.println(boo);
    
        }
    }

1.  com.igeek.mapreduceDemo.wordcount.WordCountMapper

package com.igeek.mapreduceDemo.wordcount;
    
    import org.apache.hadoop.io.IntWritable;
    import org.apache.hadoop.io.LongWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Mapper;
    
    import java.io.IOException;
    
    /**
     * KEYIN   Map阶段得输入key类型  LongWritable
     * VALUEIN Map阶段输入value类型  Text
     * KEYIN   map阶段输出key类型   Text
     * VALUEIN  map阶段输出得value类型  IntWritable
     *
     */
    public class WordCountMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
        
        Text outk=new Text();
    //    目前不进行聚合，只统计次数，所以给个1
        IntWritable outV=new IntWritable(1);
        @Override
        protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
        
    //        1.将数据转为string类型
            String line=value.toString();
    //        2.根据空格进行切割
            String[] words=line.split(" ");
    //        3。输出
            for(String word:words){
        
                outk.set(word);
                context.write(outk,outV);
            }
        }
    }

1.  com.igeek.mapreduceDemo.wordcount.WordCountReducer

package com.igeek.mapreduceDemo.wordcount;
    
    import org.apache.hadoop.io.IntWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Reducer;
    
    import java.io.IOException;
    
    /**
     * KEYIN   Reduce阶段得输入key类型   Text
     * VALUEIN reduce阶段输入value类型  IntWritable
     * KEYIN   reduce阶段输出key类型   Text
     * VALUEIN  reduce阶段输出得value类型  IntWritable
     *
     */
    public class WordCountReducer extends Reducer<Text, IntWritable,Text, IntWritable> {
        
        IntWritable outv=new IntWritable();
        @Override
        protected void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
        
    //        1.累计求和
            int sum=0;
            for(IntWritable count:values){
        
              sum+=count.get();
            }
    //        2.输出
            outv.set(sum);
            context.write(key,outv);
        }
    }

1.  com.igeek.mapreduceDemo.wordcount2.WordCountDriver

package com.igeek.mapreduceDemo.wordcount2;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.io.IntWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Job;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
    
    import java.io.IOException;
    
    public class WordCountDriver {
        
        public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
        
    //        1.获取配置信息，获取job对象实例
            Configuration conf=new Configuration();
            Job job=Job.getInstance(conf);
    //        2.关联本Driver得jar路径
            job.setJarByClass(WordCountDriver.class);
    //        3.关联map和reduce
            job.setMapperClass(WordCountMapper.class);
            job.setReducerClass(WordCountReducer.class);
    //        4.设置map得输出kv类型
            job.setMapOutputKeyClass(Text.class);
            job.setMapOutputValueClass(IntWritable.class);
    //        5.设置最终输出得kv类型
            job.setOutputKeyClass(Text.class);
            job.setOutputValueClass(IntWritable.class);
    //        6.设置输入和输出路径
            FileInputFormat.setInputPaths(job,new Path(args[0]));
            FileOutputFormat.setOutputPath(job,new Path(args[1]));
    //        7.提交job
            boolean boo=job.waitForCompletion(true);
            System.out.println(boo);
    
        }
    }

1.  com.igeek.mapreduceDemo.wordcount2.WordCountMapper

package com.igeek.mapreduceDemo.wordcount2;
    
    import org.apache.hadoop.io.IntWritable;
    import org.apache.hadoop.io.LongWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Mapper;
    
    import java.io.IOException;
    
    /**
     * KEYIN   Map阶段得输入key类型  LongWritable
     * VALUEIN Map阶段输入value类型  Text
     * KEYIN   map阶段输出key类型   Text
     * VALUEIN  map阶段输出得value类型  IntWritable
     *
     */
    public class WordCountMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
        
        Text outk=new Text();
    //    目前不进行聚合，只统计次数，所以给个1
        IntWritable outV=new IntWritable(1);
        String i;
        int age;
        @Override
        protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
        
    //        1.将数据转为string类型
            String line=value.toString();
    //        2.根据空格进行切割
            String[] words=line.split(" ");
    //        3。输出
            for(String word:words){
        
                outk.set(word);
                context.write(outk,outV);
            }
        }
    
        @Override
        public String toString() {
        
            return "WordCountMapper{" +
                    "i='" + i + '\'' +
                    ", age=" + age +
                    '}';
        }
    }

1.  com.igeek.mapreduceDemo.wordcount2.WordCountReducer

package com.igeek.mapreduceDemo.wordcount2;
    
    import org.apache.hadoop.io.IntWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Reducer;
    
    import java.io.IOException;
    
    /**
     * KEYIN   Reduce阶段得输入key类型   Text
     * VALUEIN reduce阶段输入value类型  IntWritable
     * KEYIN   reduce阶段输出key类型   Text
     * VALUEIN  reduce阶段输出得value类型  IntWritable
     *
     */
    public class WordCountReducer extends Reducer<Text, IntWritable,Text, IntWritable> {
        
        IntWritable outv=new IntWritable();
        @Override
        protected void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
        
    //        1.累计求和
            int sum=0;
            for(IntWritable count:values){
        
              sum+=count.get();
            }
    //        2.输出
            outv.set(sum);
            context.write(key,outv);
        }
    }

1.  pom.xml补充

<dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>3.3.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-hdfs</artifactId>
            <version>3.1.3</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.30</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

### 二、信息截图 ###

![在这里插入图片描述][3c5674e1081b419aaabdc374ba30684d.png]

1.  项目打包上传至集群
2.  集群测试自写wordcount

![在这里插入图片描述][184e0b25c7fa4a0ab90f592f7bc70bfb.png]

![在这里插入图片描述][cd4e84fd76574a448d859f40501d9776.png]

![在这里插入图片描述][0dbeb0961a0c42b6af8c2b256a12ff3d.png]

![在这里插入图片描述][07f395f7d5d94a1093ef0c051092357b.png]

![在这里插入图片描述][8b5e5dc7cab646919374c9c6f2e75aad.png]  
![在这里插入图片描述][6c711882ae894e89afe14967d0ec93bc.png]

![在这里插入图片描述][d36e7f73d01248de891cfdc03d5ed157.png]

![在这里插入图片描述][64e59a737b3347818f372a5e90a3b231.png]

## 5. 大数据作业5 ##

### 编写手机号码流量统计案例 ###

1.  编写本地序列化案例实现手机号码流量统计
2.  项目打包上传至集群
3.  集群测试

### 一、源代码 ###

1、 com.igeek.mapreduceDemo.flow.FlowBean

package com.igeek.mapreduceDemo.flow;
    
    import org.apache.hadoop.io.Writable;
    
    import java.io.DataInput;
    import java.io.DataOutput;
    import java.io.IOException;
    
    /**
     * 1.实现writable接口
     * 2.重写序列化和反序列化方法
     * 3.提供空参构造
     * 4.tostring
     */
    
    public class FlowBean implements Writable {
        
        private long upFlow;//上行流量
        private long downFlow;//下行流量
        private long sumFlow;//总流量
    
        public FlowBean() {
        
        }
    
        public long getUpFlow() {
        
            return upFlow;
    
    }
        public long getDownFlow() {
        
            return downFlow;
        }
    
        public void setDownFlow(long downFlow) {
        
            this.downFlow = downFlow;
        }
    
        public long getSumFlow() {
        
            return sumFlow;
        }
    
        public void setSumFlow() {
        
            this.sumFlow = this.downFlow+this.upFlow;
        }
    
        public void setUpFlow(long upFlow) {
        
            this.upFlow = upFlow;
        }
    
    
        @Override
        public void write(DataOutput out) throws IOException {
        
            out.writeLong(upFlow);
            out.writeLong(downFlow);
            out.writeLong(sumFlow);
        }
    
        @Override
        public void readFields(DataInput in) throws IOException {
        
            this.upFlow=in.readLong();
            this.downFlow=in.readLong();
            this.sumFlow=in.readLong();
        }
    
        @Override
        public String toString() {
        
            return  upFlow + "\t"+downFlow+"\t" + sumFlow;
        }
    }

2、 com.igeek.mapreduceDemo.flow.FlowMapper

package com.igeek.mapreduceDemo.flow;
    
    import org.apache.hadoop.io.LongWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Mapper;
    
    import java.io.IOException;
    
    public class FlowMapper extends Mapper<LongWritable, Text,Text,FlowBean> {
        
        private Text outk=new Text();
        private FlowBean outv=new FlowBean();
        @Override
        protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
        
            //获取一行数据，并转储为字符串
            String line=value.toString();
            //切割对象
            String[] splits=line.split("\t");
            String phone=splits[1];
            String up=splits[splits.length-3];
            String down=splits [splits.length-2];
            //封装outk，outv
            outk.set(phone);
            outv.setUpFlow(Long.parseLong(up));
            outv.setDownFlow(Long.parseLong(down));
            outv.getSumFlow();
            //写出outk，outv
            context.write(outk,outv);
    
    
    
        }
    }

3、 com.igeek.mapreduceDemo.flow.FlowReduce

package com.igeek.mapreduceDemo.flow;
    
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Reducer;
    
    import java.io.IOException;
    
    public class FlowReducer extends Reducer<Text,FlowBean,Text,FlowBean> {
        
        private FlowBean outv=new FlowBean();
    
        @Override
        protected void reduce(Text key, Iterable<FlowBean> values, Context context) throws IOException, InterruptedException {
        
            long totalUp=0;
            long totalDown=0;
    
            for (FlowBean flowBean: values){
        
                totalUp+=flowBean.getUpFlow();
                totalDown+=flowBean.getDownFlow();
            }
            //封装outv
            outv.setUpFlow(totalUp);
            outv.setDownFlow(totalDown);
            outv.setSumFlow();
            //写出outk，outv
            context.write(key,outv);
    
        }
    }

4、 com.igeek.mapreduceDemo.flow.FlowDriver

package com.igeek.mapreduceDemo.flow;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Job;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
    
    import java.io.IOException;
    
    
    public class FlowDriver {
        
        public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
        
    
            Configuration conf=new Configuration();
            Job job=Job.getInstance(conf);
    
            job.setJarByClass(FlowDriver.class);
    
            job.setMapperClass(FlowMapper.class);
            job.setReducerClass(FlowReducer.class);
    
            job.setMapOutputKeyClass(Text.class);
            job.setMapOutputValueClass(FlowBean.class);
    
            job.setOutputKeyClass(Text.class);
            job.setMapOutputValueClass(FlowBean.class);
    
            FileInputFormat.setInputPaths(job,new Path("E:\\QQdown\\phone_data.txt"));
            FileOutputFormat.setOutputPath(job,new Path("F:\\Documentation\\flowOutPut"));
    
            boolean b =job.waitForCompletion(true);
            System.out.println(b?0:1);
        }
    
    
    }

5、com.igeek.mapreduceDemo.Phone\_DataDriver.FlowMapper

package com.igeek.mapreduceDemo.Phone_DataDriver;
    
    import com.igeek.mapreduceDemo.flow.FlowBean;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Reducer;
    
    import java.io.IOException;
    
    public class FlowReducer extends Reducer<Text,FlowBean,Text,FlowBean> {
        
        private FlowBean outv=new FlowBean();
        int phone;
        int up;
        int down;
        int sum;
    
        @Override
        public String toString() {
        
            return "FlowReducer{" +
                    "phone=" + phone +
                    ", up=" + up +
                    ", down=" + down +
                    ", sum=" + sum +
                    '}';
        }
    
        @Override
        protected void reduce(Text key, Iterable<FlowBean> values, Context context) throws IOException, InterruptedException {
        
            long totalUp=0;
            long totalDown=0;
    
            for (FlowBean flowBean: values){
        
                totalUp+=flowBean.getUpFlow();
                totalDown+=flowBean.getDownFlow();
            }
            //封装outv
            outv.setUpFlow(totalUp);
            outv.setDownFlow(totalDown);
            outv.setSumFlow();
            //写出outk，outv
            context.write(key,outv);
    
    
    
        }
    }

6、com.igeek.mapreduceDemo.Phone\_DataDriver.FlowReducer

7、com.igeek.mapreduceDemo.Phone\_DataDriver.PhoneDriver

package com.igeek.mapreduceDemo.Phone_DataDriver;
    
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.io.IntWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Job;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
    
    import java.io.IOException;
    
    public class PhoneDriver {
        
        public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
        
    //        1.获取配置信息，获取job对象实例
            Configuration conf=new Configuration();
            Job job=Job.getInstance(conf);
    //        2.关联本Driver得jar路径
            job.setJarByClass(PhoneDriver.class);
    //        3.关联map和reduce
            job.setMapperClass(FlowMapper.class);
            job.setReducerClass(FlowReducer.class);
    //        4.设置map得输出kv类型
            job.setMapOutputKeyClass(Text.class);
            job.setMapOutputValueClass(IntWritable.class);
    //        5.设置最终输出得kv类型
            job.setOutputKeyClass(Text.class);
            job.setOutputValueClass(IntWritable.class);
    //        6.设置输入和输出路径
            FileInputFormat.setInputPaths(job,new Path(args[0]));
            FileOutputFormat.setOutputPath(job,new Path(args[1]));
    //        7.提交job
            boolean boo=job.waitForCompletion(true);
            System.out.println(boo);
    
        }
    }

### 二、信息截图 ###

![在这里插入图片描述][3b422aa3d94648d9aa40100779a9100d.png]  
![在这里插入图片描述][46d22fb49b3e46a693163bda31212ac4.png]

![在这里插入图片描述][461ed777d60f4f9e88ce40e48c039c71.png]

![在这里插入图片描述][b35aa1635dd7415f926d101ed203c9d5.png]

![在这里插入图片描述][f9211abea8ca4dee9827146d9fdd28ac.png]  
![在这里插入图片描述][a69a3f1437b940f7a22e1297ef65b75b.png]  
![在这里插入图片描述][3773bd03dd8f4a7e93c27a4e9da44155.png]  
![在这里插入图片描述][587c6937f2df4763b1bb73a4b781ca8f.png]

![在这里插入图片描述][3dd54921049e473299828bcebdd3eb80.png]  
![在这里插入图片描述][5d7a78e543604671af85e42e9cf6d5a7.png]

[b9203ffad8e345c8bb62cee00c38aef5.png]: https://img-blog.csdnimg.cn/b9203ffad8e345c8bb62cee00c38aef5.png
[9b6037de02b24e86aa11d54a23777444.png]: https://img-blog.csdnimg.cn/9b6037de02b24e86aa11d54a23777444.png
[07b681a79b8c41988d8b9f3a1393dc95.png]: https://img-blog.csdnimg.cn/07b681a79b8c41988d8b9f3a1393dc95.png
[4d3957afb996453182c273eae2b1c68c.png]: https://img-blog.csdnimg.cn/4d3957afb996453182c273eae2b1c68c.png
[3c131e34d3084fe993783d15ea1c9a43.png]: https://img-blog.csdnimg.cn/3c131e34d3084fe993783d15ea1c9a43.png
[05947f34a4424072905cffdd71ab8fe7.png]: https://img-blog.csdnimg.cn/05947f34a4424072905cffdd71ab8fe7.png
[08b3b60140b74e9eaefc3c59a1f4654c.png]: https://img-blog.csdnimg.cn/08b3b60140b74e9eaefc3c59a1f4654c.png
[e612fd0ce35d4804910215618863e453.png]: https://img-blog.csdnimg.cn/e612fd0ce35d4804910215618863e453.png
[999d1b63bafa4ad4b717e986a3acdf19.png]: https://img-blog.csdnimg.cn/999d1b63bafa4ad4b717e986a3acdf19.png
[7f8b01eda4334b998c8b57eb27c2744d.png]: https://img-blog.csdnimg.cn/7f8b01eda4334b998c8b57eb27c2744d.png
[b63f4174e7a24c3cad6640c518272679.png]: https://img-blog.csdnimg.cn/b63f4174e7a24c3cad6640c518272679.png
[0ed68bcaa2e44f60977f4c9555a46f17.png]: https://img-blog.csdnimg.cn/0ed68bcaa2e44f60977f4c9555a46f17.png
[104af2aaf73b4849b03d72f862056c51.png]: https://img-blog.csdnimg.cn/104af2aaf73b4849b03d72f862056c51.png
[7c5bc9cb302244c187800fa5caaf3ad4.png]: https://img-blog.csdnimg.cn/7c5bc9cb302244c187800fa5caaf3ad4.png
[92e5f5c331d64116bf37f6e8981b2512.png]: https://img-blog.csdnimg.cn/92e5f5c331d64116bf37f6e8981b2512.png
[5fc8f81412a44c0dbfa78ebdad13efe3.png]: https://img-blog.csdnimg.cn/5fc8f81412a44c0dbfa78ebdad13efe3.png
[880b0a51205f4c3caa62c8500daa1586.png]: https://img-blog.csdnimg.cn/880b0a51205f4c3caa62c8500daa1586.png
[b2c647be4ab14c649aad8c93615059b0.png]: https://img-blog.csdnimg.cn/b2c647be4ab14c649aad8c93615059b0.png
[620a4414ed254e518987df98dff2b56e.png]: https://img-blog.csdnimg.cn/620a4414ed254e518987df98dff2b56e.png
[a3a6be156c4e44689e78e44eeb08391c.png]: https://img-blog.csdnimg.cn/a3a6be156c4e44689e78e44eeb08391c.png
[74880dc9ea734ba3adccf559c85d5525.png]: https://img-blog.csdnimg.cn/74880dc9ea734ba3adccf559c85d5525.png
[f54042c115d249259b33278fbca32be8.png]: https://img-blog.csdnimg.cn/f54042c115d249259b33278fbca32be8.png
[962deb532236484a87a19a21dfc7b2e6.png]: https://img-blog.csdnimg.cn/962deb532236484a87a19a21dfc7b2e6.png
[e50dfc53f3494d14a46a8b8a9a760377.png]: https://img-blog.csdnimg.cn/e50dfc53f3494d14a46a8b8a9a760377.png
[967936ecd12348a998671efafcdefdfe.png]: https://img-blog.csdnimg.cn/967936ecd12348a998671efafcdefdfe.png
[cea436f13b0442ebbb49b07f27e908e5.png]: https://img-blog.csdnimg.cn/cea436f13b0442ebbb49b07f27e908e5.png
[90caf1a6965b4ca1a356bb82a4727c61.png]: https://img-blog.csdnimg.cn/90caf1a6965b4ca1a356bb82a4727c61.png
[041fab1c095d4056b32f6549b4cab6d7.png]: https://img-blog.csdnimg.cn/041fab1c095d4056b32f6549b4cab6d7.png
[9527bfdf548b4fec811e0b2eeb815ec0.png]: https://img-blog.csdnimg.cn/9527bfdf548b4fec811e0b2eeb815ec0.png
[a01d2a784d384e0fae73e428068d1d5c.png]: https://img-blog.csdnimg.cn/a01d2a784d384e0fae73e428068d1d5c.png
[27cc49cafdac45c18166cd15302f54f1.png]: https://img-blog.csdnimg.cn/27cc49cafdac45c18166cd15302f54f1.png
[f1872883ca514d4d8229815e12ecbc33.png]: https://img-blog.csdnimg.cn/f1872883ca514d4d8229815e12ecbc33.png
[bb54ba1565264b22959769710058c827.png]: https://img-blog.csdnimg.cn/bb54ba1565264b22959769710058c827.png
[111bfa4190684442a2754b2487bda487.png]: https://img-blog.csdnimg.cn/111bfa4190684442a2754b2487bda487.png
[f861d872d2f7494ab1923b0c995075d4.png]: https://img-blog.csdnimg.cn/f861d872d2f7494ab1923b0c995075d4.png
[d2a341a0e041408fb6a4345367394852.png]: https://img-blog.csdnimg.cn/d2a341a0e041408fb6a4345367394852.png
[e4bbfaced68146cab0400dc250657dd3.png]: https://img-blog.csdnimg.cn/e4bbfaced68146cab0400dc250657dd3.png
[b7de9afd47c64b43913440f9dd61cbab.png]: https://img-blog.csdnimg.cn/b7de9afd47c64b43913440f9dd61cbab.png
[e7f5b9753d9a4f2f859fa9d123912639.png]: https://img-blog.csdnimg.cn/e7f5b9753d9a4f2f859fa9d123912639.png
[df989c474dce4456be321af10b34299e.png]: https://img-blog.csdnimg.cn/df989c474dce4456be321af10b34299e.png
[e6aec72ddce247e59df13bada2ce81f1.png]: https://img-blog.csdnimg.cn/e6aec72ddce247e59df13bada2ce81f1.png
[ddf651a95c66496f8082bdb1b2a2046e.png]: https://img-blog.csdnimg.cn/ddf651a95c66496f8082bdb1b2a2046e.png
[2c071f403be749ae865e97a0d136ab5a.png]: https://img-blog.csdnimg.cn/2c071f403be749ae865e97a0d136ab5a.png
[b30c899b143b43d8babbb4cd0b50fcdd.png]: https://img-blog.csdnimg.cn/b30c899b143b43d8babbb4cd0b50fcdd.png
[fe941b6e53ab4d2baea01c863a697795.png]: https://img-blog.csdnimg.cn/fe941b6e53ab4d2baea01c863a697795.png
[93775d717d104bb188d78d5680c5742b.png]: https://img-blog.csdnimg.cn/93775d717d104bb188d78d5680c5742b.png
[01db4e78b5b8434298cdd810166494e1.png]: https://img-blog.csdnimg.cn/01db4e78b5b8434298cdd810166494e1.png
[5497ab5bf83d4e219be0e3a78a16dab3.png]: https://img-blog.csdnimg.cn/5497ab5bf83d4e219be0e3a78a16dab3.png
[f996b2539e3c4c1b88a3730248ed429b.png]: https://img-blog.csdnimg.cn/f996b2539e3c4c1b88a3730248ed429b.png
[d374f9c406814225be2454bb55941679.png]: https://img-blog.csdnimg.cn/d374f9c406814225be2454bb55941679.png
[9f26895313784642a439ce7dd8afee9c.png]: https://img-blog.csdnimg.cn/9f26895313784642a439ce7dd8afee9c.png
[22aa706a1092433181dcbabc77605c98.png]: https://img-blog.csdnimg.cn/22aa706a1092433181dcbabc77605c98.png
[45a4f5399a5e44d585d67dda7a696e8a.png]: https://img-blog.csdnimg.cn/45a4f5399a5e44d585d67dda7a696e8a.png
[53250b2bda9f4bd4852f07de5a618c4c.png]: https://img-blog.csdnimg.cn/53250b2bda9f4bd4852f07de5a618c4c.png
[762213c56e9a43eebb543cec5f25babe.png]: https://img-blog.csdnimg.cn/762213c56e9a43eebb543cec5f25babe.png
[1cc4772bea2a4af3aa04024396764143.png]: https://img-blog.csdnimg.cn/1cc4772bea2a4af3aa04024396764143.png
[cd85765180e4452e99f00a044788041d.png]: https://img-blog.csdnimg.cn/cd85765180e4452e99f00a044788041d.png
[0d12a523f5a44d848af3796bfd36588d.png]: https://img-blog.csdnimg.cn/0d12a523f5a44d848af3796bfd36588d.png
[09edd28f8a184298961cd4b577e570b1.png]: https://img-blog.csdnimg.cn/09edd28f8a184298961cd4b577e570b1.png
[0341650c702c4d3f8afca352430e63a4.png]: https://img-blog.csdnimg.cn/0341650c702c4d3f8afca352430e63a4.png
[f6257f316f234bcdbb3da890f1ba1bf0.png]: https://img-blog.csdnimg.cn/f6257f316f234bcdbb3da890f1ba1bf0.png
[74c0dff7189841329de5200b3f5719bd.png]: https://img-blog.csdnimg.cn/74c0dff7189841329de5200b3f5719bd.png
[89f2d34075634c19a94e3c0c8584450f.png]: https://img-blog.csdnimg.cn/89f2d34075634c19a94e3c0c8584450f.png
[3be40d60eff443ab9eed7f1a4dd874b9.png]: https://img-blog.csdnimg.cn/3be40d60eff443ab9eed7f1a4dd874b9.png
[56e33d7bb8c44f72a8334ca520091967.png]: https://img-blog.csdnimg.cn/56e33d7bb8c44f72a8334ca520091967.png
[39df5d8e4b154b618caeb7136daa451f.png]: https://img-blog.csdnimg.cn/39df5d8e4b154b618caeb7136daa451f.png
[c8d4fe3ef252453188acdfa062e500a7.png]: https://img-blog.csdnimg.cn/c8d4fe3ef252453188acdfa062e500a7.png
[3c5674e1081b419aaabdc374ba30684d.png]: https://img-blog.csdnimg.cn/3c5674e1081b419aaabdc374ba30684d.png
[184e0b25c7fa4a0ab90f592f7bc70bfb.png]: https://img-blog.csdnimg.cn/184e0b25c7fa4a0ab90f592f7bc70bfb.png
[cd4e84fd76574a448d859f40501d9776.png]: https://img-blog.csdnimg.cn/cd4e84fd76574a448d859f40501d9776.png
[0dbeb0961a0c42b6af8c2b256a12ff3d.png]: https://img-blog.csdnimg.cn/0dbeb0961a0c42b6af8c2b256a12ff3d.png
[07f395f7d5d94a1093ef0c051092357b.png]: https://img-blog.csdnimg.cn/07f395f7d5d94a1093ef0c051092357b.png
[8b5e5dc7cab646919374c9c6f2e75aad.png]: https://img-blog.csdnimg.cn/8b5e5dc7cab646919374c9c6f2e75aad.png
[6c711882ae894e89afe14967d0ec93bc.png]: https://img-blog.csdnimg.cn/6c711882ae894e89afe14967d0ec93bc.png
[d36e7f73d01248de891cfdc03d5ed157.png]: https://img-blog.csdnimg.cn/d36e7f73d01248de891cfdc03d5ed157.png
[64e59a737b3347818f372a5e90a3b231.png]: https://img-blog.csdnimg.cn/64e59a737b3347818f372a5e90a3b231.png
[3b422aa3d94648d9aa40100779a9100d.png]: https://img-blog.csdnimg.cn/3b422aa3d94648d9aa40100779a9100d.png
[46d22fb49b3e46a693163bda31212ac4.png]: https://img-blog.csdnimg.cn/46d22fb49b3e46a693163bda31212ac4.png
[461ed777d60f4f9e88ce40e48c039c71.png]: https://img-blog.csdnimg.cn/461ed777d60f4f9e88ce40e48c039c71.png
[b35aa1635dd7415f926d101ed203c9d5.png]: https://img-blog.csdnimg.cn/b35aa1635dd7415f926d101ed203c9d5.png
[f9211abea8ca4dee9827146d9fdd28ac.png]: https://img-blog.csdnimg.cn/f9211abea8ca4dee9827146d9fdd28ac.png
[a69a3f1437b940f7a22e1297ef65b75b.png]: https://img-blog.csdnimg.cn/a69a3f1437b940f7a22e1297ef65b75b.png
[3773bd03dd8f4a7e93c27a4e9da44155.png]: https://img-blog.csdnimg.cn/3773bd03dd8f4a7e93c27a4e9da44155.png
[587c6937f2df4763b1bb73a4b781ca8f.png]: https://img-blog.csdnimg.cn/587c6937f2df4763b1bb73a4b781ca8f.png
[3dd54921049e473299828bcebdd3eb80.png]: https://img-blog.csdnimg.cn/3dd54921049e473299828bcebdd3eb80.png
[5d7a78e543604671af85e42e9cf6d5a7.png]: https://img-blog.csdnimg.cn/5d7a78e543604671af85e42e9cf6d5a7.png