[文本处理] 批处理怎样提取txt文档中的特定字符串？

batbat001

{"[{"Title":"敏儿演剧史:剧本类","Summary":"上海 : 商务印书馆, **二十二年十月[1933.10]印行 : 王云五发行","PageSum":"101页","Id":"4b1e7731b001378fe85e3470b23e19ab","Pid":"f721283beed81168a7f56770c2e37f42"},{""Title":"上海新学会社图书目录","Summary":"上海 : 上海新学会社, [1936]","PageSum":"1册","Id":"18046902990da9f8c0e3b7081c62b7d9","Pid":"e24b0fcfdd69228163ee2a40179b3a10"},{""Title":"京师译学馆生理卫生学讲义","Summary":"[出版地不详] : [出版者不详], [19??]","PageSum":"1册","Id":"56373fe68f3834c486e0edfd3682ed96","Pid":"81d7172b2f060a842cc78a0486c16647"},{"Title":"中国中部奥陶纪头足类化石","Summary":"[北京] : 农矿部地质调查所 : 国立北平研究院地学研究, **十九年七月[1930.7]印行","PageSum":"18,101页","Id":"0b6ea856ab6a52c74415ced060d1028c","Pid":"7c902869e5461cc9bdb309aa3a0fd9a7"}

有若干txt文档，格式均为utf-8，每个txt里面的内容大致如上，为一些图书的目录信息，所有目录信息都混放在了同一行。
需求：提取每本书的信息到一个excel表格中，包括"Title"、"Summary"、"PageSum"、"Id"、"Pid“字段的内容信息，每组信息一行。

或者，能否把每个txt混在一行的所有书的拆开，把每本书的信息变为单独的一行。这样我可以手动导入excel。

希望能得到大神出手。谢谢了！

terse

本想用JS提取的  看了一下 发现这个文档不知道是不是没贴全还是怎么回事， 开始三个字符  {"[  和引号问题 ""Title" 会在获取数据时出错

batbat001

回复 2# terse


   上传附件总是不成功，所以剪切了一部分，应该是没剪全。
我尝试着用grep，可是总是搞不定。:'(

hfxiang

将

BEGIN {
	RS = "},{"
	print "\"Title\",\"Summary\",\"PageSum\",\"Id\tPid\""
}
{
	Title = gensub(/^.*"Title":"([^"]+)",".*$/, "\"\\1\"", "g", $0)
	Summary = gensub(/^.*"Summary":"([^"]+)",".*$/, "\"\\1\"", "g", $0)
	PageSum = gensub(/^.*"PageSum":"([^"]+)",".*$/, "\"\\1\"", "g", $0)
	Id = gensub(/^.*"Id":"([^"]+)",".*$/, "\"\\1\"", "g", $0)
	Pid = gensub(/^.*"Pid":"([^"]+)".*$/, "\"\\1\"", "g", $0)
	print Title "," Summary "," PageSum "," Id "," Pid
}
复制代码

保存为a.awk
下载gawk（ http://bcn.bathome.net/tool/4.1.0/gawk.exe ）,执行

gawk -f.\a.awk 输入文本.txt>输出文本.txt
复制代码

结果

"Title","Summary","PageSum","Id	Pid"
"敏儿演剧史：剧本类","上海：商务印书馆，** 二十二年十月 [1933.10] 印行：王云五发行","101页","4b1e7731b001378fe85e3470b23e19ab","f721283beed81168a7f56770c2e37f42"
"上海新学会社图书目录","上海：上海新学会社，[1936]","1 册","18046902990da9f8c0e3b7081c62b7d9","e24b0fcfdd69228163ee2a40179b3a10"
"京师译学馆生理卫生学讲义","[出版地不详] : [出版者不详], [19??]","1册 ","56373fe68f3834c486e0edfd3682ed96","81d7172b2f060a842cc78a0486c16647"
"中国中部奥陶纪头足类化石","[北京] : 农矿部地质调查所：国立北平研究院地学研究，** 十九年七月 [1930.7] 印行","18,101页","0b6ea856ab6a52c74415ced060d1028c","7c902869e5461cc9bdb309aa3a0fd9a7"
复制代码

batbat001

{"categoryId":null,"totalCount":380,"pageCount":38,"pageSize":10,"documents":[{"Title":"理学的疗法","Title_F":"理学的疗法","TI":["理学的疗法"],"TitleInitial":"L","CLC":["R454"],"Language":"汉语","Summary":"重庆 : 商务印书馆, **三十四年一月[1945.1]出版兼印行 : 王云五发行","Summary_F":"重庆 : 商务印书馆, **三十四年一月[1945.1]出版兼印行 : 王云五发行","LiteratureCategory":10,"WorkSubject":["物理疗法"],"PageSum":"46页","Dimensions":["18×13cm"],"ProvisionActivityDate":["1945"],"ContributionRole":["著"],"Contribution":["刘雄"],"AU":["刘雄"],"PublicationOrganization":["王云五"],"ProvisionActivityPlace":["重庆"],"ProvisionActivityOrganization":["商务印书馆"],"_version_":1738744139389861888,"showMoreEdition":true,"showEntity":false,"entity":[{"OcrTag":true,"Id":"24175e103f83cd8526d3b1c805eeaf98","Pid":"a7769ea13a60e20e94ce1df28f5bb0c1","LiteratureCategory":10,"_version_":1738744221921181696}],"Abstract":null,"LiteratureCategoryPieceTypeId":101,"viewPermission":true,"hasEntites":true},{"Title":"**哲学 三 晚明诸儒之学术及其精神","Title_F":"**哲学 三 晚明诸儒之学术及其精神","TI":["**哲学 三 晚明诸儒之学术及其精神"],"TitleInitial":"G","CLC":["B248"],"Language":"汉语","Summary":"[出版地不详] : 中央训练团党政高级训练班, [19??]印","Summary_F":"[出版地不详] : 中央训练团党政高级训练班, [19??]印","LiteratureCategory":10,"WorkSubject":["哲学","中国","明代"],"PageSum":"12页","Dimensions":["18×13cm"],"ProvisionActivityDate":["19??"],"ContributionRole":["讲"],"Contribution":["钱穆"],"AU":["钱穆"],"ProvisionActivityPlace":["出版地不详"],"ProvisionActivityOrganization":["中央训练团党政高级训练班"],"_version_":1739945908735311872,"showMoreEdition":false,"showEntity":false,"entity":[{"OcrTag":true,"Id":"31ae82462ecddebba36cca9c355da8db","Pid":"ae257d2a67fd6bf7f1bab6ae092d73f3","LiteratureCategory":10,"_version_":1739945929203515397}],"Abstract":null,"LiteratureCategoryPieceTypeId":101,"viewPermission":true,"hasEntites":true}]}

batbat001

回复 4# hfxiang


    收到。我试一下。感谢感谢！ :)

hfxiang

回复 6# batbat001


如果用gawk4.1（utf-8的支持不稳定）不能正确出结果，可尝试用uawk（对utf-8的支持较好）
下载地址：https://www.aliyundrive.com/s/rHoCCwmAww5 （注意要下载全部文件）
提取密码：1234

batbat001

回复 7# hfxiang


    gawk好像不能正确出结果。我再试一下这个。

Batcher

回复 1# batbat001


如果需要上传文件，可以用阿里云盘或百度网盘。

如果需要上传截图，可以找个图床，例如：
http://bbs.bathome.net/thread-60985-1-1.html

batbat001

回复 7# hfxiang


    非常棒！运行顺畅！非常感谢！

batbat001

回复 9# Batcher


    感谢版主提醒！论坛有你们真好！

WHY

修改正则表达式模式，取消正则多选分支结构。

<# :
@echo off
PowerShell -C ". ([ScriptBlock]::Create((gc -Literal '%~f0') -Join \"`r`n\")) '%~dp0'"
pause & exit
#>

param($path);

$reg = '"([TSPI][a-zA-Z]+)":("[^"]*")';
$dic = New-Object 'System.Collections.Generic.Dictionary[string, [Collections.ArrayList]]';
$Hash = @{Title=$true; Summary=$true; PageSum=$true; Id=$true; Pid=$true}

forEach( $file In (dir -Literal $path -Filter *.txt) ){
    $str = [IO.File]::ReadAllText($file.FullName, [Text.Encoding]::UTF8);
    forEach( $m In [regex]::Matches($str, $reg) ){
        $key = $m.Groups[1].Value;
        If( $Hash.ContainsKey($key) ){
            If( !$dic.ContainsKey($key) ){
                $dic[$key] = @();
            }
            [void]$dic[$key].Add($m.Groups[2].Value);
            $count = $dic[$key].Count;
        }
    }
}

$out = [Collections.ArrayList]@();
[void]$out.Add('"' + ($dic.Keys -join '","') + '"');

for( $i=0; $i -lt $count; $i++ ){
    $str = '';
    forEach( $key In $dic.Keys ){
        $str += $dic[$key][$i] + ',';
    }
    [void]$out.Add($str);
}

[IO.File]::WriteAllLines('result.csv', $out, [Text.Encoding]::UTF8);
复制代码

terse

@if (0) == (0) echo off
dir /b/a-d *.txt|cscript -nologo -e:jscript "%~0"
pause & exit
@end
function adoText(path) {
       var stream,content;
       stream = new ActiveXObject("ADODB.Stream");
       stream.type = 2;
       stream.charset = 'utf-8';
       stream.open();
       stream.loadFromFile(path);
       result = stream.readText(-1);
       stream.close();
       return result;
}

var fso=new ActiveXObject("Scripting.FileSystemObject");
new_file = 'new_result.csv'
f = fso.CreateTextFile(new_file, true);
f.WriteLine("Title"+','+"Summary"+','+"PageSum"+','+"Id"+','+"Pid");

while (!WSH.StdIn.AtEndOfStream){
       var  file = WScript.StdIn.Readline();
       var content =  adoText(file);
       var obj = new Function("return" + unescape(content))();
       var arr = obj.documents;
       for(var i = 0, len = arr.length; i < len; i++ ){
            f.WriteLine("\""+arr[i].Title+"\""+','+"\""+arr[i].Summary+"\""+','+"\""+arr[i].PageSum+"\""+','+arr[i].entity[0].Id+','+arr[i].entity[0].Pid);
       }   
}
f.Close()
复制代码

terse

PS版 正则提取

$result = @('"Title","Summary","PageSum","Id","Pid"');
$count = ($result -split(',')).Length
$num = [pscustomobject] @{ Value = 0 }
$reg = '(?<="(Title|Summary|PageSum|(?:I|Pi)d)":)"([^"]*)"';
$group= (select-string *.txt -Pattern $reg -AllMatches -Encoding utf8 ).Matches.Value;
$groups = $($group | Group { [math]::Floor($num.Value++ /$count)})
$result += $groups.ForEach({$_.Group -join(',')})

[IO.File]::WriteAllLines('new_result.csv', $result, [Text.Encoding]::UTF8);
复制代码

WHY

仅处理5楼所列的标准的JSON格式：

@echo off
echo;"Title","Summary","PageSum","Id","Pid">输出.csv
PowerShell "$obj=(gc 输入.txt -Raw -Enc UTF8 | ConvertFrom-Json).documents; for($i=0;$i -lt $obj.Title.Count;$i++){'\"'+(@($obj.Title[$i],$obj.Summary[$i],$obj.PageSum[$i],$obj.entity[$i].Id,$obj.entity[$i].Pid) -join '\",\"')+'\"'}">>输出.csv
pause
复制代码