Use csvtool - allanrogerr/public GitHub Wiki

csvtool

Go to linux instance, copy sample file

scp -P 20521 -o "ServerAliveInterval=5" -o "ServerAliveCountMax=100000" -o "StrictHostKeyChecking=off" /Users/allanreid/Documents/MinIO/comcast/On_demand_report_2024-04-25T19_01_58.641Z_4a98a610-0336-11ef-9e0d-a197b12ff748.csv [email protected]:/home/ubuntu/On_demand_report_2024-04-25T19_01_58.641Z_4a98a610-0336-11ef-9e0d-a197b12ff748.csv

ssh -p 20521 [email protected]

sudo apt install csvtool -y

csvtool format "%6\n" On_demand_report_2024-04-25T19_01_58.641Z_4a98a610-0336-11ef-9e0d-a197b12ff748.csv | cut -d":" -f5- | grep -v "^_source.log$"| tr "\'" "\"" | jq -c 'select((.EventName|contains("s3:ObjectRemoved:"))) | .Records[]|.source.host'  | sort | uniq -c

Command explanation

csvtool format "%6\n" Prints the 6th column(comma separated) then processes with
  cut -d":" -f5- # Prints the 5th field delimited by colon
  grep -v "^_source.log$" exclude this specific column value
  tr "\'" "\"" trim " ", then apply jq, select where event is ObjectRemoved, then count unique entries