Find file with time range fileA and fileB

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP








up vote
0
down vote

favorite












I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample


[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP


I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters



testcase1:



time:2015-02-15-22:28:02 


how to give him the matching snort.log.file for where the attack data contains



testcase2:



time:2015-02-15-18:22:02


how to give him the matching snort.log.file for where the attack data contains



I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.










share|improve this question























  • Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
    – steeldriver
    Feb 15 at 15:06










  • Thankq.. Any Ideas or suggestions I can take from there..
    – shekhar
    Feb 15 at 15:13










  • You can use date to parse timestamps with known formats, say to convert them to Unix time.
    – wjandrea
    Feb 15 at 16:40










  • What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
    – dessert
    Feb 16 at 7:55











  • @dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
    – shekhar
    Feb 16 at 14:05














up vote
0
down vote

favorite












I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample


[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP


I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters



testcase1:



time:2015-02-15-22:28:02 


how to give him the matching snort.log.file for where the attack data contains



testcase2:



time:2015-02-15-18:22:02


how to give him the matching snort.log.file for where the attack data contains



I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.










share|improve this question























  • Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
    – steeldriver
    Feb 15 at 15:06










  • Thankq.. Any Ideas or suggestions I can take from there..
    – shekhar
    Feb 15 at 15:13










  • You can use date to parse timestamps with known formats, say to convert them to Unix time.
    – wjandrea
    Feb 15 at 16:40










  • What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
    – dessert
    Feb 16 at 7:55











  • @dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
    – shekhar
    Feb 16 at 14:05












up vote
0
down vote

favorite









up vote
0
down vote

favorite











I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample


[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP


I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters



testcase1:



time:2015-02-15-22:28:02 


how to give him the matching snort.log.file for where the attack data contains



testcase2:



time:2015-02-15-18:22:02


how to give him the matching snort.log.file for where the attack data contains



I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.










share|improve this question















I am working on script where I will be receiving inputs as time and foldername,I need to find a file (snort.log.*) which is falling in that time range what my user enters (he will be seeing the alert file the bold time stamp and entering that time for actual snort.log.** file which contains the details of attack and alert file contains the metadata only)
here is directory sample


[jump1@widids01 snort-3]# ls -ltr | tail -6
-rwxr-x--- 1 snort snort 301731 Feb 2 18:18 snort.log.1517557015
-rwxr-x--- 1 snort snort 5720853 Feb 5 21:11 snort.log.1517563475
-rwxr-x--- 1 snort snort 3566073 Feb 7 16:00 snort.log.1517832965
-rwxr-x--- 1 snort snort 18999 Feb 7 20:46 snort.log.1518003111
-rwxr-x--- 1 snort snort 20705098 Feb 15 19:00 snort.log.1518004113
-rw------- 1 snort snort 357271 Feb 15 22:22 snort.log.1518688923
-rwxr-x--- 1 snort snort 50782147 Feb 15 22:22 alert
[jump1@widids01 snort-3]# tail -3 alert
02/15-18:22:02.204655 [1:2403380:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 41 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP
02/15-22:22:03.335107 [1:2403408:38412] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 55 [Classification: Misc Attack] [Priority: 2] TCP


I am facing the issues here the snort.log.[epoch] i.e [epoch] is file generated time, so if user enters



testcase1:



time:2015-02-15-22:28:02 


how to give him the matching snort.log.file for where the attack data contains



testcase2:



time:2015-02-15-18:22:02


how to give him the matching snort.log.file for where the attack data contains



I tried with find . -newermt "2018-02-15 18:38:00" ! -newermt "2018-02-15 18:39:00" I am not getting proper output.







command-line files find time






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Feb 17 at 11:34









muru

130k19275470




130k19275470










asked Feb 15 at 14:46









shekhar

322139




322139











  • Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
    – steeldriver
    Feb 15 at 15:06










  • Thankq.. Any Ideas or suggestions I can take from there..
    – shekhar
    Feb 15 at 15:13










  • You can use date to parse timestamps with known formats, say to convert them to Unix time.
    – wjandrea
    Feb 15 at 16:40










  • What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
    – dessert
    Feb 16 at 7:55











  • @dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
    – shekhar
    Feb 16 at 14:05
















  • Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
    – steeldriver
    Feb 15 at 15:06










  • Thankq.. Any Ideas or suggestions I can take from there..
    – shekhar
    Feb 15 at 15:13










  • You can use date to parse timestamps with known formats, say to convert them to Unix time.
    – wjandrea
    Feb 15 at 16:40










  • What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
    – dessert
    Feb 16 at 7:55











  • @dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
    – shekhar
    Feb 16 at 14:05















Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
– steeldriver
Feb 15 at 15:06




Presumably you want the oldest file whose modification time is newer than the given timestamp? I don't think find on its own can provide that.
– steeldriver
Feb 15 at 15:06












Thankq.. Any Ideas or suggestions I can take from there..
– shekhar
Feb 15 at 15:13




Thankq.. Any Ideas or suggestions I can take from there..
– shekhar
Feb 15 at 15:13












You can use date to parse timestamps with known formats, say to convert them to Unix time.
– wjandrea
Feb 15 at 16:40




You can use date to parse timestamps with known formats, say to convert them to Unix time.
– wjandrea
Feb 15 at 16:40












What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
– dessert
Feb 16 at 7:55





What does the timestamp the user enters represent? Is it the log file's modification time, is it the last part of the log file's filename or is it actually a time range (time stamp entered till now?) from which the oldest/newest log should be matched? Please edit and clarify.
– dessert
Feb 16 at 7:55













@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
– shekhar
Feb 16 at 14:05




@dessert the user blindly pass a date from alert file(please check the bold ones) I need to take that input and see where that time pcap data is present.
– shekhar
Feb 16 at 14:05










1 Answer
1






active

oldest

votes

















up vote
1
down vote



accepted












Assuming you have an input like this:



time="time:2015-02-15-18:22:02"


Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



Depending on what this timestamp represents you can then use find to output the log's filename:



  • if the filename already contains the timestamp, you can use snort.log.$time directly



  • if it's the file's modification time:



    find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


    This will find every log file with a modification time between time - 1 second and time.




  • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



    find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



  • if you need to find the log file that (in any way) contains the given timestamp, use grep:



    grep -l $time *


    If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



    find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +






share|improve this answer






















    Your Answer







    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "89"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1006488%2ffind-file-with-time-range-filea-and-fileb%23new-answer', 'question_page');

    );

    Post as a guest






























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    1
    down vote



    accepted












    Assuming you have an input like this:



    time="time:2015-02-15-18:22:02"


    Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



    time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


    I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



    Depending on what this timestamp represents you can then use find to output the log's filename:



    • if the filename already contains the timestamp, you can use snort.log.$time directly



    • if it's the file's modification time:



      find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


      This will find every log file with a modification time between time - 1 second and time.




    • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



      find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



    • if you need to find the log file that (in any way) contains the given timestamp, use grep:



      grep -l $time *


      If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



      find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +






    share|improve this answer


























      up vote
      1
      down vote



      accepted












      Assuming you have an input like this:



      time="time:2015-02-15-18:22:02"


      Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



      time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


      I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



      Depending on what this timestamp represents you can then use find to output the log's filename:



      • if the filename already contains the timestamp, you can use snort.log.$time directly



      • if it's the file's modification time:



        find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


        This will find every log file with a modification time between time - 1 second and time.




      • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



        find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



      • if you need to find the log file that (in any way) contains the given timestamp, use grep:



        grep -l $time *


        If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



        find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +






      share|improve this answer
























        up vote
        1
        down vote



        accepted







        up vote
        1
        down vote



        accepted








        Assuming you have an input like this:



        time="time:2015-02-15-18:22:02"


        Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



        time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


        I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



        Depending on what this timestamp represents you can then use find to output the log's filename:



        • if the filename already contains the timestamp, you can use snort.log.$time directly



        • if it's the file's modification time:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


          This will find every log file with a modification time between time - 1 second and time.




        • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



        • if you need to find the log file that (in any way) contains the given timestamp, use grep:



          grep -l $time *


          If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +






        share|improve this answer
















        Assuming you have an input like this:



        time="time:2015-02-15-18:22:02"


        Then you can convert it to a Unix timestamp (seconds since epoch) and save it in variable timestamp with:



        time=$(date -d$(<<<"$time" sed 's/time:(.*)-/1T/') +%s)


        I use sed here to get rid of the time: prefix and change the colon between date and time to T (ISO 8601), date is then used to convert the timestamp to seconds since epoch.



        Depending on what this timestamp represents you can then use find to output the log's filename:



        • if the filename already contains the timestamp, you can use snort.log.$time directly



        • if it's the file's modification time:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) ! -newermt @$time


          This will find every log file with a modification time between time - 1 second and time.




        • if you need to find the oldest log file from after the given timestamp, you can use sort and head:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -print0 | sort -z | head -zn1



        • if you need to find the log file that (in any way) contains the given timestamp, use grep:



          grep -l $time *


          If you have a lot of long log files maybe combine this with find and/or parallel to make it faster, e.g.:



          find . -type f -name "*snort.log.*" -newermt @$(($time-1)) -exec grep -l $time +







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Feb 17 at 12:38

























        answered Feb 17 at 10:22









        dessert

        20k55795




        20k55795



























             

            draft saved


            draft discarded















































             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1006488%2ffind-file-with-time-range-filea-and-fileb%23new-answer', 'question_page');

            );

            Post as a guest













































































            Popular posts from this blog

            How do so many people here on Academia.SE, and in general, afford lavish higher education programs?

            Trouble downloading packages list due to a “Hash sum mismatch” error

            How do I move numbers in filenames, in a batch renaming operation?