有如下資料

computer,huangxiaoming,85,86,41,75,93,42,85

computer,xuzheng,54,52,86,91,42

computer,huangbo,85,42,96,38

english,zhaobenshan,54,52,86,91,42,85,75

english,liuyifei,85,41,75,21,85,96,14

algorithm,liuyifei,75,85,62,48,54,96,15

computer,huangjiaju,85,75,86,85,85

english,liuyifei,76,95,86,74,68,74,48

english,huangdatou,48,58,67,86,15,33,85

algorithm,huanglei,76,95,86,74,68,74,48

algorithm,huangjiaju,85,75,86,85,85,74,86

computer,huangdatou,48,58,67,86,15,33,85

english,zhouqi,85,86,41,75,93,42,85,75,55,47,22

english,huangbo,85,42,96,38,55,47,22

algorithm,liutao,85,75,85,99,66

computer,huangzitao,85,86,41,75,93,42,85

math,wangbaoqiang,85,86,41,75,93,42,85

computer,liujialing,85,41,75,21,85,96,14,74,86

computer,liuyifei,75,85,62,48,54,96,15

computer,liutao,85,75,85,99,66,88,75,91

computer,huanglei,76,95,86,74,68,74,48

english,liujialing,75,85,62,48,54,96,15

math,huanglei,76,95,86,74,68,74,48

math,huangjiaju,85,75,86,85,85,74,86

math,liutao,48,58,67,86,15,33,85

english,huanglei,85,75,85,99,66,88,75,91

math,xuzheng,54,52,86,91,42,85,75

math,huangxiaoming,85,75,85,99,66,88,75,91

math,liujialing,85,86,41,75,93,42,85,75

english,huangxiaoming,85,86,41,75,93,42,85

algorithm,huangdatou,48,58,67,86,15,33,85

algorithm,huangzitao,85,86,41,75,93,42,85,75
 

一、資料解釋

資料欄位個數不固定：

第一個是課程名稱，總共四個課程，computer，math，english，algorithm，

第二個是學生姓名，後面是每次考試的分數

二、統計需求：

1、統計每門課程的參考人數和課程平均分，

public class pinjunfenP {
static class mymapper extends Mapper<LongWritable, Text, Text, Text>{
        @Override
        protected void map(LongWritable key, Text value,  Context context)  throws IOException, InterruptedException {
            String[] fields = value.toString().split(",");
            for (String string : fields) {
                System.out.print(string + "  ");
            }
            //取出科目
            String kemu = fields[0];            
            //取出成績放到新陣列中去
            StringBuffer cjstr=new StringBuffer();
            for (int i = 2; i < fields.length; i++) {
                cjstr.append(fields[i]);
                 //不是最後一個成績的時候append一個逗號
                 if (i!=fields.length-1) {
                     cjstr.append(",");
                }
            }
            context.write(new Text(kemu), new Text(cjstr.toString()));
        }
    }

    static class myreducer extends Reducer<Text, Text, Text, Text>{
        @Override
        protected void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException {
            //記錄人數
            int count=0;
            double avgofone;
            double avg;
            int sumall=0;
            for (Text text : values) {
                //一個人的平均成績
                String[] str = text.toString().split(",");
                int sum=0;
                int num=0;
                for (String string : str) {
                    sum+=Integer.parseInt(string);
                    num++;
                }
                avgofone=(sum/num);
                count++;
                sumall+=avgofone;
            }
            //一個科目的平均成績
            avg=sumall/count;
            String outString=count+","+avg;
            Text t=new Text(outString);
            context.write(key, t);
        }
    }

    public static void main(String[] args) throws Exception {
        Configuration conf = new Configuration();
        Job job=Job.getInstance(conf);
        //設定主函式的入口
        job.setJarByClass(pinjunfenP.class);
        //設定job的mapper的類
        job.setMapperClass(mymapper.class);
        //設定job的reduce類
        job.setReducerClass(myreducer.class);
        //設定map輸出key的型別
        job.setMapOutputKeyClass(Text.class);
        //設定map輸出value的型別
        job.setMapOutputValueClass(Text.class);     
        //設定reduce輸出key的型別
        job.setOutputKeyClass(Text.class);
        //設定reduce輸出value的型別
        job.setOutputValueClass(Text.class);        
        //檔案輸入類
        Path inpath=new Path("hdfs://master:9000/hadoop/chengji");
        FileInputFormat.addInputPath(job, inpath);
        //檔案輸出類
        Path outpath=new Path("hdfs://master:9000/hadoop/chengjiout");
        FileOutputFormat.setOutputPath(job,outpath);
        //提交job
        job.waitForCompletion(true);
    }
}
 

執行的時候報空指標，發現是stringbuffer沒有new物件
最後統計結果如下
algorithm    6,71.0
computer    10,69.0
english    9,66.0
math    7,72.0

2、在1的結果上，按照平均分降序排列

3、統計每一個學生的每一門課程的平均分，在對結果按照科目 在按照平均分進行降序排序