Python - Search for a specific time range in the text file (sed -n equivalent)

advertisements

I'm trying to make a python script that outputs a specific time range from a log file (similar to the sed command listed below):

sed -n '/2017-01-26 18:00/ , /2017-01-26 18:02/p' /logfile.log

2017-01-26 18:00:00
2017-01-26 18:01:01
2017-01-26 18:01:02
2017-01-26 18:01:09
2017-01-26 18:01:09
2017-01-26 18:01:11
2017-01-26 18:02:01

My python script is searching for a fixed string and not like the sed command above (I suspect that I am doing something wrong, but I can't find the error - please check the code below):

Please point me where the code should be changed and also advises for code enhancement. Thanks!

#!/usr/bin/python
import datetime, time, os, sys, re
from datetime import timedelta
counter = 0
avgtime = 0

now = datetime.datetime.utcnow()
pasttime = now - datetime.timedelta(minutes=5)

timestamp = now.strftime("%y%m%d")
fiveago   = now - timedelta(minutes=5,seconds=now.second)
current   = now.strftime("%Y-%m-%d %H:%M")
pasttime  = fiveago.strftime("%Y-%m-%d %H:%M")
pattern   = str(current + "|" + pasttime)

f = open('/logs/' + sys.argv[1] + '/' + 'u_ex' + timestamp + '.log', 'r')
for line in f:
        if "POST" in line:
                if re.search(pattern, line, re.IGNORECASE):
                        date = line.split(' ')[1]
                        time = line.split(' ')[14]
                        avgtime += int(time)
                        counter += 1
                        print(date,time)
f.close()

print(pattern)
print("Total amount of time: ",counter)
print("Total scan time: ",avgtime)
print("Average scan time: ",avgtime / counter)


IIUC, you need the enteries from the log between time stamp that you pass.

import datetime, time, os, sys, re
from datetime import timedelta
counter = 0
avgtime = 0

now = datetime.datetime.utcnow()
pasttime = now - datetime.timedelta(minutes=100000)

timestamp = now.strftime("%y%m%d")
fiveago   = now - timedelta(minutes=5,seconds=now.second)
current   = now.strftime("%Y-%m-%d %H:%M")
pasttime  = fiveago.strftime("%Y-%m-%d %H:%M")
pattern   = str(current + "|" + pasttime)

print "Start time: ", pasttime ,"End time: ",current ,"\n\n"

filename ='/logs/' + sys.argv[1] + '/' + 'u_ex' + timestamp + '.log'
with open(filename, 'r') as f:
    contents = f.readlines()
for line in contents:
    if "POST" in line:
        date = line.split(' ')[1]
        time = line.split(' ')[14]
        logdatetime=date+" "+time

        if logdatetime <= current and logdatetime >= pasttime:
            print "yes, within the interval : " ,logdatetime

output

Start time:  2017-01-26 20:23 End time:  2017-01-26 20:28 

yes, within the interval :  2017-01-26 20:23:20
yes, within the interval :  2017-01-26 20:23:01
yes, within the interval :  2017-01-26 20:23:02

input used for this

POST 2017-01-26 20:23:20 XX
POST 2017-01-26 20:23:01 XC
POST 2017-01-26 20:23:02 CV
POST 2017-01-26 20:20:09 DAF
POST 2017-01-26 20:20:09 fASF
POST 2017-01-26 20:20:11 Sfas
POST 2017-01-26 20:20:01 fsAf
POST 2017-01-26 20:20:02 asf
POST 2017-01-26 20:20:03 asf