【问题】

I have a file with a large number of rows. Each row contains 5 columns delimited by tabs. I want to find all rows that have the same value for the first 4 columns but have different values for the 5th column.

name	age	address	phone	city
eric	5	add1	1234	City1
jerry	5	add1	1234	City2
eric	5	add1	1234	City3
eric	5	add1	1234	City4
jax	5	add1	1234	City5
jax	5	add1	1234	City6
niko	5	add1	1234	City7

The result for this table should be

eric      5      add1      1234     City1
eric      5      add1      1234     City3
eric      5      add1      1234     City4
jax       5      add1      1234     City5
jax       5      add1      1234     City6

I tried using uniq -u -f4 after sort but that ignores the first 4 fields which in this case would return all the rows.

别人给出的解答，但楼主已用ruby解答，未跑完下面代码：

use strict;
use warnings;
use Text::CSV_XS qw(csv);

my @csv_files = @ARGV;

# Parse all the CSV files into arrays of arrays.
my $data1 = csv( in => $csv_files[0], sep_char => "\t" );

# Parse the other CSV files into hashes of rows keyed on the columns we're going to search on.
my $data2 = csv( in             => $csv_files[1],
                 sep_char       => "\t",
                 headers        => ["code", "num1", "num2"],
                 key => "code"
            );
my $data3 = csv( in             => $csv_files[2],
                 sep_char       => "\t",
                 headers        => ["CODE"],
                 key            => "CODE"
            );

for my $row1 (@$data1) {
    my $row2 = $data2->{$row1->[0]};
    my $row3 = $data3->{$row1->[1]};

    if( $row2 && $row3 ) {
        print join "\t", $row1->[0], $row1->[1], $row2->{num1}, $row2->{num2};
        print "\n";
    }
}

【回答】

只要按前4个字段分组，再找出成员计数大于1的组，将各组数据合并即可。Awk和ruby缺乏结构化计算函数，代码会很复杂，而且性能不高。如无特殊要求建议用SPL实现，代码简单易懂：

A1：读取文本文件file1.txt中的内容。

A2：按照字段name,age,address,phone分组，并选择成员计数大于1的组，最后合并数据。