List, sort compare folder contents

List files in a directory at a given level which have trailing .nc in filename

cd /g/data1/v27/projects/EOS_delivery/LANDSAT

export contents of subdirectories to file:
ls -1 */* | grep nc > $home/NBAR_netCDF.txt

remove _BX.nc from filenames in text file:
sed -i 's/_BX.nc//g' $home/NBAR_netCDF.txt

collect filelist to compare with:
cd /g/data1/v10/NBAR
ls -1 * | grep NBAR_P54_GANBAR | grep -v _2013 > $home/NBAR_list.txt

count how many files:
ls -1 * | grep NBAR_P54_GANBAR | grep -v _2013 | wc -l

sort files and compare:
eg. file to sort - LS7_ETM_NBAR_P54_GANBAR01-002_115_079_20121219
eg. sort by date component - character 37 to 45 in the example above
sort -k 1.37,1.45 diff.txt > diff.sorted.txt
grep -Fxv -f $home/NBAR_netCDF.sorted.txt $home/NBAR_list.sorted.txt > $home/diff.txt

No comments:

Post a Comment