Find file with time range fileA and fileB

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP








up vote
0
down vote

favorite












I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample


[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP


I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters



testcase1:



time:2015-02-15-22:28:02 


how to give him the matching snort.log.file for where the attack data contains



testcase2:



time:2015-02-15-18:22:02


how to give him the matching snort.log.file for where the attack data contains



I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.










share|improve this question























  • Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
    – steeldriver
    Feb 15 at 15:06










  • Thankq.. Any Ideas or suggestions I can take from there..
    – shekhar
    Feb 15 at 15:13










  • You can use date to parse timestamps with known formats, say to convert them to Unix time.
    – wjandrea
    Feb 15 at 16:40










  • What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
    – dessert
    Feb 16 at 7:55











  • @dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
    – shekhar
    Feb 16 at 14:05














up vote
0
down vote

favorite












I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample


[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP


I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters



testcase1:



time:2015-02-15-22:28:02 


how to give him the matching snort.log.file for where the attack data contains



testcase2:



time:2015-02-15-18:22:02


how to give him the matching snort.log.file for where the attack data contains



I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.










share|improve this question























  • Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
    – steeldriver
    Feb 15 at 15:06










  • Thankq.. Any Ideas or suggestions I can take from there..
    – shekhar
    Feb 15 at 15:13










  • You can use date to parse timestamps with known formats, say to convert them to Unix time.
    – wjandrea
    Feb 15 at 16:40










  • What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
    – dessert
    Feb 16 at 7:55











  • @dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
    – shekhar
    Feb 16 at 14:05












up vote
0
down vote

favorite









up vote
0
down vote

favorite











I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample


[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP


I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters



testcase1:



time:2015-02-15-22:28:02 


how to give him the matching snort.log.file for where the attack data contains



testcase2:



time:2015-02-15-18:22:02


how to give him the matching snort.log.file for where the attack data contains



I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.










share|improve this question















I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample


[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP


I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters



testcase1:



time:2015-02-15-22:28:02 


how to give him the matching snort.log.file for where the attack data contains



testcase2:



time:2015-02-15-18:22:02


how to give him the matching snort.log.file for where the attack data contains



I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.







command-line files find time






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Feb 17 at 11:34









muru

130k19275470




130k19275470










asked Feb 15 at 14:46









shekhar

322139




322139











  • Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
    – steeldriver
    Feb 15 at 15:06










  • Thankq.. Any Ideas or suggestions I can take from there..
    – shekhar
    Feb 15 at 15:13










  • You can use date to parse timestamps with known formats, say to convert them to Unix time.
    – wjandrea
    Feb 15 at 16:40










  • What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
    – dessert
    Feb 16 at 7:55











  • @dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
    – shekhar
    Feb 16 at 14:05
















  • Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
    – steeldriver
    Feb 15 at 15:06










  • Thankq.. Any Ideas or suggestions I can take from there..
    – shekhar
    Feb 15 at 15:13










  • You can use date to parse timestamps with known formats, say to convert them to Unix time.
    – wjandrea
    Feb 15 at 16:40










  • What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
    – dessert
    Feb 16 at 7:55











  • @dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
    – shekhar
    Feb 16 at 14:05















Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
– steeldriver
Feb 15 at 15:06




Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
– steeldriver
Feb 15 at 15:06












Thankq.. Any Ideas or suggestions I can take from there..
– shekhar
Feb 15 at 15:13




Thankq.. Any Ideas or suggestions I can take from there..
– shekhar
Feb 15 at 15:13












You can use date to parse timestamps with known formats, say to convert them to Unix time.
– wjandrea
Feb 15 at 16:40




You can use date to parse timestamps with known formats, say to convert them to Unix time.
– wjandrea
Feb 15 at 16:40












What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
– dessert
Feb 16 at 7:55





What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
– dessert
Feb 16 at 7:55













@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
– shekhar
Feb 16 at 14:05




@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
– shekhar
Feb 16 at 14:05










1 Answer
1






active

oldest

votes

















up vote
1
down vote



accepted












Assuming you have an input like this:



time="time:2015-02-15-18:22:02"


Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



Depending on what this timestamp represents you can then use find to output the log's filename:



  • if the filename already contains the timestamp, you can use snort.log.$time directly



  • if it's the file's modification time:



    find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


    This will find every log file with a modification time between time - 1 second and time.




  • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



    find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



  • if you need to find the log file that (in any way) contains the given timestamp, use grep:



    grep -l $time *


    If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



    find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +






share|improve this answer






















    Your Answer







    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "89"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1006488%2ffind-file-with-time-range-filea-and-fileb%23new-answer', 'question_page');

    );

    Post as a guest






























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    1
    down vote



    accepted












    Assuming you have an input like this:



    time="time:2015-02-15-18:22:02"


    Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



    time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


    I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



    Depending on what this timestamp represents you can then use find to output the log's filename:



    • if the filename already contains the timestamp, you can use snort.log.$time directly



    • if it's the file's modification time:



      find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


      This will find every log file with a modification time between time - 1 second and time.




    • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



      find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



    • if you need to find the log file that (in any way) contains the given timestamp, use grep:



      grep -l $time *


      If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



      find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +






    share|improve this answer


























      up vote
      1
      down vote



      accepted












      Assuming you have an input like this:



      time="time:2015-02-15-18:22:02"


      Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



      time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


      I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



      Depending on what this timestamp represents you can then use find to output the log's filename:



      • if the filename already contains the timestamp, you can use snort.log.$time directly



      • if it's the file's modification time:



        find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


        This will find every log file with a modification time between time - 1 second and time.




      • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



        find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



      • if you need to find the log file that (in any way) contains the given timestamp, use grep:



        grep -l $time *


        If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



        find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +






      share|improve this answer
























        up vote
        1
        down vote



        accepted







        up vote
        1
        down vote



        accepted








        Assuming you have an input like this:



        time="time:2015-02-15-18:22:02"


        Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



        time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


        I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



        Depending on what this timestamp represents you can then use find to output the log's filename:



        • if the filename already contains the timestamp, you can use snort.log.$time directly



        • if it's the file's modification time:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


          This will find every log file with a modification time between time - 1 second and time.




        • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



        • if you need to find the log file that (in any way) contains the given timestamp, use grep:



          grep -l $time *


          If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +






        share|improve this answer
















        Assuming you have an input like this:



        time="time:2015-02-15-18:22:02"


        Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



        time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


        I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



        Depending on what this timestamp represents you can then use find to output the log's filename:



        • if the filename already contains the timestamp, you can use snort.log.$time directly



        • if it's the file's modification time:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


          This will find every log file with a modification time between time - 1 second and time.




        • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



        • if you need to find the log file that (in any way) contains the given timestamp, use grep:



          grep -l $time *


          If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Feb 17 at 12:38

























        answered Feb 17 at 10:22









        dessert

        20k55795




        20k55795



























             

            draft saved


            draft discarded















































             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1006488%2ffind-file-with-time-range-filea-and-fileb%23new-answer', 'question_page');

            );

            Post as a guest













































































            Popular posts from this blog

            Which professions warranted travel in Medieval times?

            How do so many people here on Academia.SE, and in general, afford lavish higher education programs?

            Trouble downloading packages list due to a “Hash sum mismatch” error