Board logo

标题: [文本处理] 【已解决】紧急求助批处理IIS日志多文件处理,万分感谢! [打印本页]

作者: shniutan    时间: 2022-4-21 00:17     标题: 【已解决】紧急求助批处理IIS日志多文件处理,万分感谢!

本帖最后由 shniutan 于 2022-5-5 08:43 编辑

非常感谢idwma,qixiaobin0715,xczxczxcz 三位大神的无私帮助,谢谢你们!

有3个IIS日志文件,如下:

a.txt  //每行包含1个infoID,各不相同

2022-1-2 08:00:01 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80
2022-1-2 08:00:06 192.168.1.10 post \file\detail.aspx\infoid=1q1w3e4r-6y2u8i 80
2022-1-2 08:00:11 192.168.1.10 post \file\detail.aspx\infoid=1q2w4e4r-6y3u8i 80
2022-1-2 08:00:16 192.168.1.10 post \file\detail.aspx\infoid=1q3w3e4r-6y4u8i 80
2022-1-2 08:00:21 192.168.1.10 post \file\detail.aspx\infoid=1q4w3e4r-6y5u8i 80
2022-1-2 08:00:26 192.168.1.10 post \file\detail.aspx\infoid=1q5w3e4r-6y6u8i 80
2022-1-2 08:00:31 192.168.1.10 post \file\detail.aspx\infoid=1q6w3e4r-6y7u8i 80
2022-1-2 08:00:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80
2022-1-2 08:00:41 192.168.1.10 post \file\detail.aspx\infoid=1q8w3e4r-6y9u8i 80
2022-1-2 08:00:46 192.168.1.10 post \file\detail.aspx\infoid=2q2w3e4r-6y1u8i 80



b.txt  //每行包含用户名、IP地址,有用户名、IP地址均重复的行

2022-1-2 08:00:05 192.168.1.10 get \bbs\login.aspx\username=张三&password=6y1u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:00:15 192.168.1.10 get \bbs\login.aspx\username=李四关&password=6y1u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:00:25 192.168.1.10 get \bbs\login.aspx\username=王五&password=6y1u8i 80 - 192.168.1.25 200 0 0
2022-1-2 08:00:35 192.168.1.10 get \bbs\login.aspx\username=孙六&password=6y1u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:00:45 192.168.1.10 get \bbs\login.aspx\username=刘七地&password=6y1u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:00:55 192.168.1.10 get \bbs\login.aspx\username=杨八猫&password=6y1u8i 80 - 192.168.1.31 200 0 0
2022-1-2 08:01:05 192.168.1.10 get \bbs\login.aspx\username=江九&password=6y1u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:03:25 192.168.1.10 get \bbs\login.aspx\username=周十&password=6y1u8i 80 - 192.168.1.35 200 0 0
2022-1-2 08:03:45 192.168.1.10 get \bbs\login.aspx\username=巴一巴&password=6y1u8i 80 - 192.168.1.37 200 0 0
2022-1-2 08:04:25 192.168.1.10 get \bbs\login.aspx\username=赵二&password=6y1u8i 80 - 192.168.1.39 200 0 0
2022-1-2 09:00:25 192.168.1.10 get \bbs\login.aspx\username=王五&password=6y1u8i 80 - 192.168.1.25 200 0 0
2022-1-2 09:00:35 192.168.1.10 get \bbs\login.aspx\username=孙六&password=6y1u8i 80 - 192.168.1.27 200 0 0
2022-1-2 09:00:45 192.168.1.10 get \bbs\login.aspx\username=刘七地&password=6y1u8i 80 - 192.168.1.29 200 0 0
2022-1-2 09:00:55 192.168.1.10 get \bbs\login.aspx\username=杨八猫&password=6y1u8i 80 - 192.168.1.31 200 0 0


c.txt  //每行包含文件ID、IP地址,存在同一个IP访问不同文件ID的情况,也存在不同IP访问同一文件的情况

2022-1-2 08:00:01 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:00:02 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:00:03 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.25 200 0 0
2022-1-2 08:00:04 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:00:06 192.168.1.10 post \file\detail.aspx\infoid=1q1w3e4r-6y2u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:00:11 192.168.1.10 post \file\detail.aspx\infoid=1q2w4e4r-6y3u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:00:16 192.168.1.10 post \file\detail.aspx\infoid=1q3w3e4r-6y4u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:00:21 192.168.1.10 post \file\detail.aspx\infoid=1q4w3e4r-6y5u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:00:26 192.168.1.10 post \file\detail.aspx\infoid=1q5w3e4r-6y6u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:00:31 192.168.1.10 post \file\detail.aspx\infoid=1q6w3e4r-6y7u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:00:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:00:41 192.168.1.10 post \file\detail.aspx\infoid=1q8w3e4r-6y9u8i 80 - 192.168.1.35 200 0 0
2022-1-2 08:00:46 192.168.1.10 post \file\detail.aspx\infoid=2q2w3e4r-6y1u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:00:51 192.168.1.10 post \file\detail.aspx\infoid=3q2w3e4r-6y1u8i 80 - 192.168.1.37 200 0 0
2022-1-2 08:00:56 192.168.1.10 post \file\detail.aspx\infoid=4q2w3e4r-6y7u8i 80 - 192.168.1.35 200 0 0
2022-1-2 08:01:16 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:01:51 192.168.1.10 post \file\detail.aspx\infoid=5q2w3e4r-6y7u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:01:56 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.25 200 0 0
2022-1-2 08:01:26 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:01:36 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:01:46 192.168.1.10 post \file\detail.aspx\infoid=6q2w3e4r-6y7u8i 80 - 192.168.1.25 200 0 0
2022-1-2 08:02:21 192.168.1.10 post \file\detail.aspx\infoid=7q2w3e4r-6y7u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:03:36 192.168.1.10 post \file\detail.aspx\infoid=8q2w3e4r-6y7u8i 80 - 192.168.1.29 200 0 0
2022-1-2 08:10:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80 - 192.168.1.31 200 0 0
2022-1-2 08:20:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80 - 192.168.1.31 200 0 0
2022-1-2 08:30:36 192.168.1.10 post \file\detail.aspx\infoid=1q7w3e4r-6y8u8i 80 - 192.168.1.31 200 0 0
2022-1-2 08:33:46 192.168.1.10 post \file\detail.aspx\infoid=2q2w3e4r-6y1u8i 80 - 192.168.1.33 200 0 0
2022-1-2 08:34:21 192.168.1.10 post \file\detail.aspx\infoid=1q4w3e4r-6y5u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:34:26 192.168.1.10 post \file\detail.aspx\infoid=1q5w3e4r-6y6u8i 80 - 192.168.1.27 200 0 0
2022-1-2 08:40:01 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.21 200 0 0
2022-1-2 08:40:02 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.23 200 0 0
2022-1-2 08:40:03 192.168.1.10 post \file\detail.aspx\infoid=1q2w3e4r-6y1u8i 80 - 192.168.1.25 200 0 0

希望能用批处理实现:
1.删除b.txt中用户名、IP地址均重复的行,输出结果到d.txt。
2.从c.txt中找出a.txt中每个infoID的被访问情况,分别输出到e1.txt、e2.txt、……中,再根据d.txt中用户名和IP地址的对应关系,统计出e1.txt、e2.txt、……中每个IP地址对应用户的访问次数,分别输出infoid、IP地址、用户名、访问次数信息到f1.txt、f2.txt、……中。
3.查找f1.txt、f2.txt、……所有文件中infoid被同一用户访问超过2次的记录,汇总输出到新的文件中,每个用户1个文件,输出信息包括IP地址、用户名、infoid、访问次数。

万分感谢!
作者: idwma    时间: 2022-4-21 20:18

假设def的文件都是多余的设只要结果
  1. #@&cls&powershell "type %~s0|out-string|iex"&pause&exit
  2. $a=@{}
  3. gc b.txt|%{$a[$_ -replace '.*name=(.*)&.*-\s(\S+).*','$1 $2']=$_}
  4. $b=(gc .\c.txt) -match (((gc a.txt) -replace '.*infoid=(\S+).*','$1') -join '|')
  5. $c=@{}
  6. $a.keys|%{$t=$_ -split '\s+'; $c[$t[1]]=$t[0]}
  7. $d=@{}
  8. $b|%{$t=$_ -replace '.*infoid=([\S]+).*-\s([\d\.]+).*','$1 $2';$d[$t]+=1}
  9. $e=$d.keys|%{$t=$_ -split '\s+';"{0} {1}`t{2} {3}" -f $t[1],$c[$t[1]],$t[0],$d[$_]}
  10. $f=@{}
  11. $e|%{$t=$_ -split '\s+';$f[$t[1]]+=[int]$t[3]}
  12. $g=($f.keys|?{$f[$_] -gt 2}) -join '|'
  13. $h=@{}
  14. $e -match $g|%{$t=$_ -split '\s+';$h[$t[1]]+=@($_)}
  15. $h.keys|%{sc "$_.txt" $h[$_]}
复制代码

作者: qixiaobin0715    时间: 2022-4-22 14:28

回复 1# shniutan
描述太乱,写代码时也改来改去很乱,不知这样汇总是不是你要的结果:
  1. @echo off
  2. setlocal enabledelayedexpansion
  3. for /f "tokens=8,12 delims=-&= " %%i in (b.txt) do set #%%j=%%i
  4. (for /f "tokens=1* delims==" %%x in (a.txt) do echo,%%y)>1.log
  5. findstr /g:1.log c.txt>Newfile.txt
  6. (for /f "tokens=8,9,11 delims=-= " %%a in (Newfile.txt) do (
  7.     set /a "_%%a%%b+=1"
  8.     if "!_%%a%%b!" == "2" echo,%%a-%%b
  9. ))>1.log
  10. findstr /g:1.log Newfile.txt>2.log
  11. (for /f "tokens=8,9,11 delims=-= " %%a in (2.log) do (
  12.     if not defined @%%a%%b%%c echo,%%c !#%%c! %%a-%%b !_%%a%%b!
  13.     set @%%a%%b%%c=true
  14. ))>1.log
  15. sort 1.log>Newfile.txt
  16. del *.log
  17. pause
复制代码

作者: shniutan    时间: 2022-4-22 18:54

@idwma,@qixiaobin0715

非常感谢两位大佬帮忙!
抱歉之前没有表述清楚。又重新梳理一下需求:
1.不需要删除b.txt中IP地址和人名重复的行这步操作,b.txt文件作用就是通过IP地址找对对应人名。
2.在c.txt中找出每个IP地址访问的infoid(仅统计a.txt中列出的)次数,并根据b.txt中IP地址和人名的对应关系,输出相关信息到1个文件中。

最终输出的结果应该是:

192.168.1.21 张三        1q2w3e4r-6y1u8i 2
192.168.1.21 张三        1q2w4e4r-6y3u8i 1
192.168.1.23 李四关        1q2w3e4r-6y1u8i 2
192.168.1.23 李四关        1q1w3e4r-6y2u8i 1
192.168.1.23 李四关        1q4w3e4r-6y5u8i 2
192.168.1.25 王五        1q2w3e4r-6y1u8i 2
192.168.1.27 孙六        1q2w3e4r-6y1u8i 1
192.168.1.27 孙六        1q5w3e4r-6y6u8i 2
192.168.1.29 刘七地        1q3w3e4r-6y4u8i 1
192.168.1.29 刘七地        1q6w3e4r-6y7u8i 1
192.168.1.31 杨八猫        1q7w3e4r-6y8u8i 3
192.168.1.33 江九        1q7w3e4r-6y8u8i 1
192.168.1.33 江九        2q2w3e4r-6y1u8i 2
192.168.1.35 周十        1q8w3e4r-6y9u8i 1


//总次数应该是a.txt中infoid在c.txt中出现的总次数,22次。
作者: idwma    时间: 2022-4-22 19:03

本帖最后由 idwma 于 2022-4-22 19:07 编辑
  1. #@&cls&powershell "type %~s0|out-string|iex"&pause&exit
  2. $a=@{}
  3. gc b.txt|%{$a[$_ -replace '.*name=(.*)&.*-\s(\S+).*','$1 $2']=$_}
  4. $b=(gc .\c.txt) -match (((gc a.txt) -replace '.*infoid=(\S+).*','$1') -join '|')
  5. $c=@{}
  6. $a.keys|%{$t=$_ -split '\s+'; $c[$t[1]]=$t[0]}
  7. $d=@{}
  8. $b|%{$t=$_ -replace '.*infoid=([\S]+).*-\s([\d\.]+).*','$1 $2';$d[$t]+=1}
  9. $e=$d.keys|%{$t=$_ -split '\s+';"{0} {1}`t{2} {3}" -f $t[1],$c[$t[1]],$t[0],$d[$_]}
  10. $f=@{}
  11. $e|%{$t=$_ -split '\s+';$f[$t[1]]+=[int]$t[3]}
  12. $g=($f.keys) -join '|'
  13. $h=@{}
  14. $e -match $g|%{$t=$_ -split '\s+';$h[$t[1]]+=@($_)}
  15. $h.values|sc abc.txt
复制代码

作者: shniutan    时间: 2022-4-23 22:21

本帖最后由 shniutan 于 2022-4-24 10:26 编辑

回复 5# idwma
非常感谢百忙之中帮我解决问题!

a.txt
2010/12/23        05:38:35        inforID=AAF5AB44-389D-44AD-84EA-A45C6EE74AFE
2011/01/18        01:06:33        inforID=7717F5D7-A409-4D48-9E87-1A92367C3593
2011/11/15        01:37:00        inforID=E9D8DE66-A11E-4778-8CEC-64C46B657E6D
2012/02/14        02:53:43        inforID=580552F2-4A22-41FB-B5A8-8E4806E90C08
2013/05/07        06:23:04        inforID=99D32186-121E-4FF7-AB54-0F816687A0A5
2014/08/06        06:12:07        inforID=1F698943-4766-4C48-ADB5-256ED89E1AF8
2015/07/21        05:23:39        inforID=C8FC9543-453E-44AB-9188-FC72963689D1
2017/08/02        11:20:26        inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE
2021/10/25        09:28:17        inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299
2022/04/02        10:04:52        inforID=C049F1E8-CA33-436B-9F67-45E5A8E792D5

b.txt  //因系统原因,2013年前的日志包含了浏览器信息
2010-12-16 04:46:00 192.168.10.1 GET /bbs/LoginForum.aspx UserName=工程部_张三a&Password=qaeswdrf 80 - 192.168.37.153 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727) 200 0 0
2010-12-26 04:46:33 192.168.10.1 GET /bbs/LoginForum.aspx UserName=服务部_李四博a&Password=qaeswdrf 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+InfoPath.1;+.NET+CLR+1.1.4322) 200 0 0
2011-12-06 04:47:05 192.168.10.1 GET /bbs/LoginForum.aspx UserName=投诉部_王欣零a&Password=qaeswdrf 80 - 192.168.81.18 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2012-02-16 04:47:43 192.168.10.1 GET /bbs/LoginForum.aspx UserName=人事部_江五a&Password=qaeswdrf 80 - 192.168.29.193 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.2;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.5.21022;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) 200 0 0
2013-10-16 04:48:23 192.168.10.1 GET /bbs/LoginForum.aspx UserName=技术部_郭肥来a&Password=qaeswdrf 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) 200 0 0
2014-11-04 01:28:36 192.168.10.1 GET /bbs/LoginForum.aspx UserName=工程部_赵六华a&Password=qaeswdrf 80 - 192.168.37.136 200 0 0
2015-01-17 01:28:53 192.168.10.1 GET /bbs/LoginForum.aspx UserName=服务部_周小宁a&Password=qaeswdrf 80 - 192.168.102.31 200 0 0
2016-01-08 01:29:04 192.168.10.1 GET /bbs/LoginForum.aspx UserName=人事部_肖逄文a&Password=qaeswdrf 80 - 192.168.29.10 200 0 0
2018-02-04 01:29:26 192.168.10.1 GET /bbs/LoginForum.aspx UserName=投诉部_刘瑾a&Password=qaeswdrf 80 - 192.168.81.31 200 0 0
2021-10-04 01:29:57 192.168.10.1 GET /bbs/LoginForum.aspx UserName=技术部_龙一a&Password=qaeswdrf 80 - 192.168.30.20 200 0 0

c.txt  //因系统原因,2013年前的日志包含了浏览器信息
2011-01-18 03:33:09 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.20 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+Maxthon;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-01-18 03:36:09 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727) 200 0 0
2011-01-18 01:20:04 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1) 200 0 0
2011-01-19 06:59:55 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 07:10:04 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.29.10 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 07:10:11 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2) 200 0 0
2011-11-15 07:10:15 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 08:41:46 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.30.20 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+Maxthon;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2) 200 0 0
2013-05-07 06:23:11 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.29.10 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1;+InfoPath.2) 200 0 0
2013-05-07 06:24:23 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2013-05-07 06:25:12 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1;+InfoPath.2) 200 0 0
2013-05-07 06:28:22 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121B-4FF7-AB54-0F816687A0A5 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2;+TheWorld) 200 0 0
2013-05-07 06:26:05 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.37.153 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30) 200 0 0
2015-07-21 05:31:13 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.37.153 200 0 0
2015-07-21 05:31:30 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.102.11 200 0 0
2015-07-21 05:32:35 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.30.20 200 0 0
2015-07-21 05:33:21 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.37.153 200 0 0
2017-08-02 23:39:06 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.81.18 200 0 0
2017-08-02 23:39:31 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.30.20 200 0 0
2017-08-02 23:49:21 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.30.20 200 0 0
2017-08-02 23:53:54 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.102.11 200 0 0
2021-10-25 09:36:34 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.29.10 200 0 0
2021-10-25 09:36:44 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.30.1 200 0 0
2021-10-25 09:37:04 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.30.20 200 0 0
2021-10-25 09:37:50 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.37.153 200 0 0
2021-10-25 09:41:07 192.168.10.1 POST /File/Detail.aspx inforID=A0C8CDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.29.10 200 0 0

运行后报错,能输出结果文件,但输出的结果中多统计了2条未包含在a.txt中文件的访问记录。报错信息如下:

索引操作失败;数组索引的计算结果为空值。
所在位置 行:6 字符: 38
+     $a.keys|%{$t=$_ -split '\s+'; $c[ <<<< $t[1]]=$t[0]}
    + CategoryInfo          : InvalidOperation: (System.Collections.Hashtable:
   Hashtable) [], RuntimeException
    + FullyQualifiedErrorId : NullArrayIndex
作者: qixiaobin0715    时间: 2022-4-24 09:32

回复 4# shniutan
加上顶楼的限定条件“所有文件中infoid被同一用户访问超过2次的记录”,总次数应当是17次:
  1. @echo off
  2. setlocal enabledelayedexpansion
  3. for /f "tokens=8,12 delims=-&= " %%i in (b.txt) do set #%%j=%%i
  4. (for /f "tokens=1* delims==" %%x in (a.txt) do echo,%%y)>1.log
  5. findstr /g:1.log c.txt>Newfile.txt
  6. (for /f "tokens=8,9,11 delims=-= " %%a in (Newfile.txt) do (
  7.     set /a "_%%a%%b+=1"
  8.     set /a "_%%a%%b%%c+=1"
  9.     if "!_%%a%%b!" == "2" echo,%%a-%%b
  10. ))>1.log
  11. findstr /g:1.log Newfile.txt>2.log
  12. (for /f "tokens=8,9,11 delims=-= " %%a in (2.log) do (
  13.     if not defined @%%a%%b%%c (
  14.         echo,%%c !#%%c! %%a-%%b !_%%a%%b%%c!
  15.         set /a Sum+=!_%%a%%b%%c!
  16.     )
  17.     set @%%a%%b%%c=true
  18. ))>1.log
  19. sort 1.log>Newfile.txt
  20. echo,Sum !Sum!>>Newfile.txt
  21. del *.log
  22. pause
复制代码

作者: shniutan    时间: 2022-4-24 10:38

回复 7# qixiaobin0715

非常感谢!
用您的代码试了一下下面的文件,好像出不来结果。另外,不再限定访问次数为2以上了。
最终想实现的就是:
在c.txt中找出每个IP地址访问的inforID(仅统计a.txt中列出的)次数,并根据b.txt中IP地址和人名的对应关系,输出IP地址、人名、inforID、访问次数信息到1个文件中。

a.txt
2010/12/23        05:38:35        inforID=AAF5AB44-389D-44AD-84EA-A45C6EE74AFE
2011/01/18        01:06:33        inforID=7717F5D7-A409-4D48-9E87-1A92367C3593
2011/11/15        01:37:00        inforID=E9D8DE66-A11E-4778-8CEC-64C46B657E6D
2012/02/14        02:53:43        inforID=580552F2-4A22-41FB-B5A8-8E4806E90C08
2013/05/07        06:23:04        inforID=99D32186-121E-4FF7-AB54-0F816687A0A5
2014/08/06        06:12:07        inforID=1F698943-4766-4C48-ADB5-256ED89E1AF8
2015/07/21        05:23:39        inforID=C8FC9543-453E-44AB-9188-FC72963689D1
2017/08/02        11:20:26        inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE
2021/10/25        09:28:17        inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299
2022/04/02        10:04:52        inforID=C049F1E8-CA33-436B-9F67-45E5A8E792D5

b.txt  //因系统原因,2013年前的日志包含了浏览器信息
2010-12-16 04:46:00 192.168.10.1 GET /bbs/LoginForum.aspx UserName=工程部_张三a&Password=qaeswdrf 80 - 192.168.37.153 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727) 200 0 0
2010-12-26 04:46:33 192.168.10.1 GET /bbs/LoginForum.aspx UserName=服务部_李四博a&Password=qaeswdrf 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+InfoPath.1;+.NET+CLR+1.1.4322) 200 0 0
2011-12-06 04:47:05 192.168.10.1 GET /bbs/LoginForum.aspx UserName=投诉部_王欣零a&Password=qaeswdrf 80 - 192.168.81.18 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2012-02-16 04:47:43 192.168.10.1 GET /bbs/LoginForum.aspx UserName=人事部_江五a&Password=qaeswdrf 80 - 192.168.29.193 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.2;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.648;+.NET+CLR+3.5.21022;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) 200 0 0
2013-10-16 04:48:23 192.168.10.1 GET /bbs/LoginForum.aspx UserName=技术部_郭肥来a&Password=qaeswdrf 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) 200 0 0
2014-11-04 01:28:36 192.168.10.1 GET /bbs/LoginForum.aspx UserName=工程部_赵六华a&Password=qaeswdrf 80 - 192.168.37.136 200 0 0
2015-01-17 01:28:53 192.168.10.1 GET /bbs/LoginForum.aspx UserName=服务部_周小宁a&Password=qaeswdrf 80 - 192.168.102.31 200 0 0
2016-01-08 01:29:04 192.168.10.1 GET /bbs/LoginForum.aspx UserName=人事部_肖逄文a&Password=qaeswdrf 80 - 192.168.29.10 200 0 0
2018-02-04 01:29:26 192.168.10.1 GET /bbs/LoginForum.aspx UserName=投诉部_刘瑾a&Password=qaeswdrf 80 - 192.168.81.31 200 0 0
2021-10-04 01:29:57 192.168.10.1 GET /bbs/LoginForum.aspx UserName=技术部_龙一a&Password=qaeswdrf 80 - 192.168.30.20 200 0 0

c.txt  //因系统原因,2013年前的日志包含了浏览器信息
2011-01-18 03:33:09 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.20 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+Maxthon;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-01-18 03:36:09 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727) 200 0 0
2011-01-18 01:20:04 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1) 200 0 0
2011-01-19 06:59:55 192.168.10.1 POST /File/Detail.aspx inforID=7717F5D7-A409-4D48-9E87-1A92367C3593 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 07:10:04 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.29.10 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 07:10:11 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2) 200 0 0
2011-11-15 07:10:15 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2011-11-15 08:41:46 192.168.10.1 POST /File/Detail.aspx inforID=E20400B4-86B2-475A-AE61-39A727254AF3 80 - 192.168.30.20 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+Maxthon;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2) 200 0 0
2013-05-07 06:23:11 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.29.10 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1;+InfoPath.2) 200 0 0
2013-05-07 06:24:23 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727) 200 0 0
2013-05-07 06:25:12 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.102.11 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.1;+InfoPath.2) 200 0 0
2013-05-07 06:28:22 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121B-4FF7-AB54-0F816687A0A5 80 - 192.168.30.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+InfoPath.2;+TheWorld) 200 0 0
2013-05-07 06:26:05 192.168.10.1 POST /File/Detail.aspx inforID=99D32186-121E-4FF7-AB54-0F816687A0A5 80 - 192.168.37.153 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.04506.30) 200 0 0
2015-07-21 05:31:13 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.37.153 200 0 0
2015-07-21 05:31:30 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.102.11 200 0 0
2015-07-21 05:32:35 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.30.20 200 0 0
2015-07-21 05:33:21 192.168.10.1 POST /File/Detail.aspx inforID=C8FC9543-453E-44AB-9188-FC72963689D1 80 - 192.168.37.153 200 0 0
2017-08-02 23:39:06 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.81.18 200 0 0
2017-08-02 23:39:31 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.30.20 200 0 0
2017-08-02 23:49:21 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.30.20 200 0 0
2017-08-02 23:53:54 192.168.10.1 POST /File/Detail.aspx inforID=AAA31733-09CE-4015-A327-7C4BAC2B25EE 80 - 192.168.102.11 200 0 0
2021-10-25 09:36:34 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.29.10 200 0 0
2021-10-25 09:36:44 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.30.1 200 0 0
2021-10-25 09:37:04 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.30.20 200 0 0
2021-10-25 09:37:50 192.168.10.1 POST /File/Detail.aspx inforID=A0C8BDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.37.153 200 0 0
2021-10-25 09:41:07 192.168.10.1 POST /File/Detail.aspx inforID=A0C8CDE0-6A46-4128-B566-5F40D0B83299 80 - 192.168.29.10 200 0 0
作者: qixiaobin0715    时间: 2022-4-24 11:06

本帖最后由 qixiaobin0715 于 2022-4-24 11:21 编辑

回复 8# shniutan
你把a、b、c文本文件传到网盘上,可帮你测试。
你顶楼的文本好像与你实际文本并不相同。
作者: shniutan    时间: 2022-4-24 11:49

回复 9# qixiaobin0715

谢谢,之前发的是和实际不太一样,实际文本就是8楼的3个文件
作者: qixiaobin0715    时间: 2022-4-24 11:54

回复 10# shniutan
最好还是把文件上传,这样复制有时还是会与源文件有差距的。
作者: Batcher    时间: 2022-4-24 12:04

回复 10# shniutan


    建议把真实文件打包上传到网盘
作者: shniutan    时间: 2022-4-24 13:48

回复 12# Batcher
谢谢,上传到哪个网盘?是通用网盘还是本网站自有网盘?

另外,请教个问题,问什么我回复帖子无法以图片或附件方式上传图片,图片小于50k,png格式的
作者: Batcher    时间: 2022-4-24 14:17

回复 13# shniutan


如果需要上传文件,可以用阿里云盘或百度网盘。

如果需要上传截图,可以找个图床,例如:
http://bbs.bathome.net/thread-60985-1-1.html
作者: shniutan    时间: 2022-4-24 15:29

回复 11# qixiaobin0715

感谢!文件已上传网盘。
链接:https://pan.baidu.com/s/1uAttTDidnZ1xVluD1oTh_w
提取码:nlg7
作者: shniutan    时间: 2022-4-24 15:29

回复 14# Batcher

谢谢指导
作者: qixiaobin0715    时间: 2022-4-24 15:40

回复 15# shniutan
写的有点乱,凑合用吧。bat文件存为ANSI编码:
  1. @echo off
  2. setlocal enabledelayedexpansion
  3. for /f "tokens=7,12 delims=&= " %%i in (b.txt) do set #%%j=%%i
  4. (for /f "tokens=1* delims==" %%i in (a.txt) do echo,%%j)>Newfile.txt
  5. findstr /g:Newfile.txt c.txt>1.log
  6. (for /f "tokens=1* delims==" %%i in (1.log) do echo,%%j)>Newfile.txt
  7. for /f "tokens=1-5,7 delims=- " %%a in (Newfile.txt) do set /a "_%%a%%b%%c%%d%%e%%f+=1"
  8. (for /f "tokens=1-5,7 delims=- " %%a in (Newfile.txt) do (
  9.     if not defined @%%a%%b%%c%%d%%e%%f (
  10.         echo,%%f !#%%f! %%a-%%b-%%c-%%d-%%e !_%%a%%b%%c%%d%%e%%f!
  11.         set /a Sum+=!_%%a%%b%%c%%d%%e%%f!
  12.     )
  13.     set @%%a%%b%%c%%d%%e%%f=true
  14. ))>1.log
  15. sort 1.log>Newfile.txt
  16. echo,Sum !Sum!>>Newfile.txt
  17. del 1.log
  18. pause
复制代码

作者: xczxczxcz    时间: 2022-4-24 19:46

本帖最后由 xczxczxcz 于 2022-4-24 19:48 编辑

给你一个好玩的,保存为 xxx.ps1, 右键执行
  1. $a='.\a.txt';
  2. $b='.\b.txt';
  3. $c='.\c.txt';
  4. $hash=[Collections.Generic.HashSet[string]]::new();
  5. $infoID = @();
  6. (gc $a -ReadCount 0).foreach{
  7. $id = $_ -replace '^.*?=|\s+\d+$';
  8. if($hash.Add($id)){ $infoID+=$id }
  9. }
  10. $hash.Clear();
  11. $user = @();
  12. (gc $b -ReadCount 0).Foreach{
  13. $s = $_ -replace '^.*?=(.*?)&.*?-\s+((\d+\.){3}\d+).*$','$2,$1' -split ',';
  14. if($hash.Add($s[0]+$s[1])){
  15. $user += @{ $s[0].ToString() = $s[1]}
  16. }
  17. }
  18. $hash.Clear();
  19. $ipCount=@();
  20. (gc $c -ReadCount 0).Foreach{
  21. $s = $_ -replace '^.*?=(.*?)\s.*?-\s+((\d+\.){3}\d+).*$','$1,$2' -split ',';
  22. if($infoID.contains($s[0])){
  23. $ipCount+=[PsCustomObject]@{id=$s[0]; ip=$s[1]; name=$user."$($s[1])"; }
  24. }
  25. }
  26. ($ipcount|group name).Foreach{
  27. $ip =$_.Group[0].ip;
  28. $name = $_.Name;
  29. $_.group|group id|%{$ip + "`t" + $name + "`t`t" + $_.Name + "`t" + $_.Count}
  30. }
  31. &cmd /c set/p=全部完成 退出<nul`&pause>nul;exit
复制代码

作者: xczxczxcz    时间: 2022-4-24 19:47

这是结果:
  1. 192.168.30.20   技术部_龙一a            7717F5D7-A409-4D48-9E87-1A92367C3593  1
  2. 192.168.30.20   技术部_龙一a            C8FC9543-453E-44AB-9188-FC72963689D1  1
  3. 192.168.30.20   技术部_龙一a            AAA31733-09CE-4015-A327-7C4BAC2B25EE  2
  4. 192.168.30.20   技术部_龙一a            A0C8BDE0-6A46-4128-B566-5F40D0B83299  1
  5. 192.168.30.1    技术部_郭肥来a          7717F5D7-A409-4D48-9E87-1A92367C3593  2
  6. 192.168.30.1    技术部_郭肥来a          99D32186-121E-4FF7-AB54-0F816687A0A5  1
  7. 192.168.30.1    技术部_郭肥来a          A0C8BDE0-6A46-4128-B566-5F40D0B83299  1
  8. 192.168.102.11  服务部_李四博a          7717F5D7-A409-4D48-9E87-1A92367C3593  1
  9. 192.168.102.11  服务部_李四博a          99D32186-121E-4FF7-AB54-0F816687A0A5  1
  10. 192.168.102.11  服务部_李四博a          C8FC9543-453E-44AB-9188-FC72963689D1  1
  11. 192.168.102.11  服务部_李四博a          AAA31733-09CE-4015-A327-7C4BAC2B25EE  1
  12. 192.168.29.10   人事部_肖逄文a          99D32186-121E-4FF7-AB54-0F816687A0A5  1
  13. 192.168.29.10   人事部_肖逄文a          A0C8BDE0-6A46-4128-B566-5F40D0B83299  1
  14. 192.168.37.153  工程部_张三a            99D32186-121E-4FF7-AB54-0F816687A0A5  1
  15. 192.168.37.153  工程部_张三a            C8FC9543-453E-44AB-9188-FC72963689D1  2
  16. 192.168.37.153  工程部_张三a            A0C8BDE0-6A46-4128-B566-5F40D0B83299  1
  17. 192.168.81.18   投诉部_王欣零a          AAA31733-09CE-4015-A327-7C4BAC2B25EE  1
复制代码

作者: shniutan    时间: 2022-5-5 08:40

回复 19# xczxczxcz

试了试,好像不行,用楼上的代码可以了,非常感谢
作者: shniutan    时间: 2022-5-5 08:40

回复 17# qixiaobin0715

用这个代码没问题,非常感谢




欢迎光临 批处理之家 (http://www.bathome.net/) Powered by Discuz! 7.2