Find file with time range fileA and fileB

Clash Royale CLAN TAG#URR8PPP up vote
0
down vote
favorite
I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample
[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters
testcase1:
time:2015-02-15-22:28:02
how to give him the matching snort.log.file for where the attack data contains
testcase2:
time:2015-02-15-18:22:02
how to give him the matching snort.log.file for where the attack data contains
I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.
command-line files find time
add a comment |Â
up vote
0
down vote
favorite
I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample
[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters
testcase1:
time:2015-02-15-22:28:02
how to give him the matching snort.log.file for where the attack data contains
testcase2:
time:2015-02-15-18:22:02
how to give him the matching snort.log.file for where the attack data contains
I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.
command-line files find time
Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't thinkfindon its own can provide that.
â steeldriver
Feb 15 at 15:06
Thankq.. Any Ideas or suggestions I can take from there..
â shekhar
Feb 15 at 15:13
You can usedateto parse timestamps with known formats, say to convert them to Unix time.
â wjandrea
Feb 15 at 16:40
What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
â dessert
Feb 16 at 7:55
@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
â shekhar
Feb 16 at 14:05
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample
[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters
testcase1:
time:2015-02-15-22:28:02
how to give him the matching snort.log.file for where the attack data contains
testcase2:
time:2015-02-15-18:22:02
how to give him the matching snort.log.file for where the attack data contains
I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.
command-line files find time
I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample
[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters
testcase1:
time:2015-02-15-22:28:02
how to give him the matching snort.log.file for where the attack data contains
testcase2:
time:2015-02-15-18:22:02
how to give him the matching snort.log.file for where the attack data contains
I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.
command-line files find time
command-line files find time
edited Feb 17 at 11:34
muru
130k19275470
130k19275470
asked Feb 15 at 14:46
shekhar
322139
322139
Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't thinkfindon its own can provide that.
â steeldriver
Feb 15 at 15:06
Thankq.. Any Ideas or suggestions I can take from there..
â shekhar
Feb 15 at 15:13
You can usedateto parse timestamps with known formats, say to convert them to Unix time.
â wjandrea
Feb 15 at 16:40
What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
â dessert
Feb 16 at 7:55
@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
â shekhar
Feb 16 at 14:05
add a comment |Â
Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't thinkfindon its own can provide that.
â steeldriver
Feb 15 at 15:06
Thankq.. Any Ideas or suggestions I can take from there..
â shekhar
Feb 15 at 15:13
You can usedateto parse timestamps with known formats, say to convert them to Unix time.
â wjandrea
Feb 15 at 16:40
What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
â dessert
Feb 16 at 7:55
@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
â shekhar
Feb 16 at 14:05
Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think
find on its own can provide that.â steeldriver
Feb 15 at 15:06
Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think
find on its own can provide that.â steeldriver
Feb 15 at 15:06
Thankq.. Any Ideas or suggestions I can take from there..
â shekhar
Feb 15 at 15:13
Thankq.. Any Ideas or suggestions I can take from there..
â shekhar
Feb 15 at 15:13
You can use
date to parse timestamps with known formats, say to convert them to Unix time.â wjandrea
Feb 15 at 16:40
You can use
date to parse timestamps with known formats, say to convert them to Unix time.â wjandrea
Feb 15 at 16:40
What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
â dessert
Feb 16 at 7:55
What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
â dessert
Feb 16 at 7:55
@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
â shekhar
Feb 16 at 14:05
@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
â shekhar
Feb 16 at 14:05
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
Assuming you have an input like this:
time="time:2015-02-15-18:22:02"
Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:
time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)
I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.
Depending on what this timestamp represents you can then use find to output the log's filename:
if the filename already contains the timestamp, you can use
snort.log.$timedirectlyif it's the file's modification time:
find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$timeThis will find every log file with a modification time between
time- 1 second andtime.if you need to find the oldest log file from after the given timestamp, you can use
sortandhead:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1if you need to find the log file that (in any way) contains the given timestamp, use
grep:grep -l $time *If you have a lot of long log files maybe combine this with
findand/orparallelto make it faster, e.g.:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
Assuming you have an input like this:
time="time:2015-02-15-18:22:02"
Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:
time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)
I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.
Depending on what this timestamp represents you can then use find to output the log's filename:
if the filename already contains the timestamp, you can use
snort.log.$timedirectlyif it's the file's modification time:
find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$timeThis will find every log file with a modification time between
time- 1 second andtime.if you need to find the oldest log file from after the given timestamp, you can use
sortandhead:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1if you need to find the log file that (in any way) contains the given timestamp, use
grep:grep -l $time *If you have a lot of long log files maybe combine this with
findand/orparallelto make it faster, e.g.:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +
add a comment |Â
up vote
1
down vote
accepted
Assuming you have an input like this:
time="time:2015-02-15-18:22:02"
Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:
time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)
I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.
Depending on what this timestamp represents you can then use find to output the log's filename:
if the filename already contains the timestamp, you can use
snort.log.$timedirectlyif it's the file's modification time:
find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$timeThis will find every log file with a modification time between
time- 1 second andtime.if you need to find the oldest log file from after the given timestamp, you can use
sortandhead:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1if you need to find the log file that (in any way) contains the given timestamp, use
grep:grep -l $time *If you have a lot of long log files maybe combine this with
findand/orparallelto make it faster, e.g.:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +
add a comment |Â
up vote
1
down vote
accepted
up vote
1
down vote
accepted
Assuming you have an input like this:
time="time:2015-02-15-18:22:02"
Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:
time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)
I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.
Depending on what this timestamp represents you can then use find to output the log's filename:
if the filename already contains the timestamp, you can use
snort.log.$timedirectlyif it's the file's modification time:
find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$timeThis will find every log file with a modification time between
time- 1 second andtime.if you need to find the oldest log file from after the given timestamp, you can use
sortandhead:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1if you need to find the log file that (in any way) contains the given timestamp, use
grep:grep -l $time *If you have a lot of long log files maybe combine this with
findand/orparallelto make it faster, e.g.:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +
Assuming you have an input like this:
time="time:2015-02-15-18:22:02"
Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:
time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)
I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.
Depending on what this timestamp represents you can then use find to output the log's filename:
if the filename already contains the timestamp, you can use
snort.log.$timedirectlyif it's the file's modification time:
find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$timeThis will find every log file with a modification time between
time- 1 second andtime.if you need to find the oldest log file from after the given timestamp, you can use
sortandhead:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1if you need to find the log file that (in any way) contains the given timestamp, use
grep:grep -l $time *If you have a lot of long log files maybe combine this with
findand/orparallelto make it faster, e.g.:find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +
edited Feb 17 at 12:38
answered Feb 17 at 10:22
dessert
20k55795
20k55795
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);
;
$window.on('scroll', onScroll);
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1006488%2ffind-file-with-time-range-filea-and-fileb%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);
;
$window.on('scroll', onScroll);
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);
;
$window.on('scroll', onScroll);
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);
;
$window.on('scroll', onScroll);
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think
findon its own can provide that.â steeldriver
Feb 15 at 15:06
Thankq.. Any Ideas or suggestions I can take from there..
â shekhar
Feb 15 at 15:13
You can use
dateto parse timestamps with known formats, say to convert them to Unix time.â wjandrea
Feb 15 at 16:40
What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
â dessert
Feb 16 at 7:55
@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
â shekhar
Feb 16 at 14:05