对电脑日志复杂的键值对提取 每个json数据不一样要求通用脚本模板

_source
"{'@timestamp': '2020-09-21T00:46:21.676Z', 'service': {'type': 'system'}, 'ecs': {'version': '1.5.0'}, 'host': {'name': 'DESKTOP-B8KO4F2'}, 'agent': {'type': 'auditbeat', 'version': '7.8.0', 'hostname': 'DESKTOP-B8KO4F2', 'ephemeral_id': '4e9babb5-fe64-492c-bfee-60bc48d5ecbb', 'id': '6c8b1fa8-661a-4cd7-a982-6a5104deb6e7', 'name': 'DESKTOP-B8KO4F2'}, 'event': {'kind': 'event', 'category': ['process'], 'type': ['start'], 'action': 'process_started', 'module': 'system', 'dataset': 'process'}, 'process': {'args': ['C:\Windows\system32\SearchFilterHost.exe', '0', '736', '740', '748', '8192', '744'], 'pid': 6196, 'start': '2020-09-21T00:46:12.450Z', 'hash': {'sha1': '38998a827eb2e7b4879bc4e8b09de632793e4976'}, 'ppid': 7776, 'working_directory': 'C:\Windows\system32', 'entity_id': 'kOE5rcBfgspO0Juc', 'name': 'SearchFilterHost.exe', 'executable': 'C:\Windows\System32\SearchFilterHost.exe'}, 'message': 'Process SearchFilterHost.exe (PID: 6196) by user NT AUTHORITY\SYSTEM STARTED', 'user': {'group': {'id': 'S-1-5-18'}, 'name': 'NT AUTHORITY\SYSTEM', 'id': 'S-1-5-18'}}"

可以遍历每个日志读取成字符串进行处理，单个可这样：

import pandas as pd
d=eval(s)
df=pd.DataFrame(d)
print(df)

如有帮助，请点采纳。

import pandas as pd

df = pd.DataFrame({'@timestamp': '2020-09-21T00:46:21.676Z', 'service': {'type': 'system'}, 'ecs': {'version': '1.5.0'}, 'host': {'name': 'DESKTOP-B8KO4F2'}, 'agent': {'type': 'auditbeat', 'version': '7.8.0', 'hostname': 'DESKTOP-B8KO4F2', 'ephemeral_id': '4e9babb5-fe64-492c-bfee-60bc48d5ecbb', 'id': '6c8b1fa8-661a-4cd7-a982-6a5104deb6e7', 'name': 'DESKTOP-B8KO4F2'}, 'event': {'kind': 'event', 'category': ['process'], 'type': ['start'], 'action': 'process_started', 'module': 'system', 'dataset': 'process'}, 'process': {'args': ['C:\\Windows\\system32\\SearchFilterHost.exe', '0', '736', '740', '748', '8192', '744'], 'pid': 6196, 'start': '2020-09-21T00:46:12.450Z', 'hash': {'sha1': '38998a827eb2e7b4879bc4e8b09de632793e4976'}, 'ppid': 7776, 'working_directory': 'C:\\Windows\\system32', 'entity_id': 'kOE5rcBfgspO0Juc', 'name': 'SearchFilterHost.exe', 'executable': 'C:\\Windows\\System32\\SearchFilterHost.exe'}, 'message': 'Process SearchFilterHost.exe (PID: 6196) by user NT AUTHORITY\\SYSTEM STARTED', 'user': {'group': {'id': 'S-1-5-18'}, 'name': 'NT AUTHORITY\\SYSTEM', 'id': 'S-1-5-18'}})

df

@timestampserviceecshostagenteventprocessmessageuser
type2020-09-21T00:46:21.676ZsystemNaNNaNauditbeat[start]NaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
version2020-09-21T00:46:21.676ZNaN1.5.0NaN7.8.0NaNNaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
name2020-09-21T00:46:21.676ZNaNNaNDESKTOP-B8KO4F2DESKTOP-B8KO4F2NaNSearchFilterHost.exeProcess SearchFilterHost.exe (PID: 6196) by us...NT AUTHORITY\SYSTEM
hostname2020-09-21T00:46:21.676ZNaNNaNNaNDESKTOP-B8KO4F2NaNNaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
ephemeral_id2020-09-21T00:46:21.676ZNaNNaNNaN4e9babb5-fe64-492c-bfee-60bc48d5ecbbNaNNaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
id2020-09-21T00:46:21.676ZNaNNaNNaN6c8b1fa8-661a-4cd7-a982-6a5104deb6e7NaNNaNProcess SearchFilterHost.exe (PID: 6196) by us...S-1-5-18
kind2020-09-21T00:46:21.676ZNaNNaNNaNNaNeventNaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
category2020-09-21T00:46:21.676ZNaNNaNNaNNaN[process]NaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
action2020-09-21T00:46:21.676ZNaNNaNNaNNaNprocess_startedNaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
module2020-09-21T00:46:21.676ZNaNNaNNaNNaNsystemNaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
dataset2020-09-21T00:46:21.676ZNaNNaNNaNNaNprocessNaNProcess SearchFilterHost.exe (PID: 6196) by us...NaN
args2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaN[C:\Windows\system32\SearchFilterHost.exe, 0, ...Process SearchFilterHost.exe (PID: 6196) by us...NaN
pid2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaN6196Process SearchFilterHost.exe (PID: 6196) by us...NaN
start2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaN2020-09-21T00:46:12.450ZProcess SearchFilterHost.exe (PID: 6196) by us...NaN
hash2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaN{'sha1': '38998a827eb2e7b4879bc4e8b09de632793e...Process SearchFilterHost.exe (PID: 6196) by us...NaN
ppid2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaN7776Process SearchFilterHost.exe (PID: 6196) by us...NaN
working_directory2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaNC:\Windows\system32Process SearchFilterHost.exe (PID: 6196) by us...NaN
entity_id2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaNkOE5rcBfgspO0JucProcess SearchFilterHost.exe (PID: 6196) by us...NaN
executable2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaNC:\Windows\System32\SearchFilterHost.exeProcess SearchFilterHost.exe (PID: 6196) by us...NaN
group2020-09-21T00:46:21.676ZNaNNaNNaNNaNNaNNaNProcess SearchFilterHost.exe (PID: 6196) by us...{'id': 'S-1-5-18'}
df.process

type                                                               NaN
version                                                            NaN
name                                              SearchFilterHost.exe
hostname                                                           NaN
ephemeral_id                                                       NaN
id                                                                 NaN
kind                                                               NaN
category                                                           NaN
action                                                             NaN
module                                                             NaN
dataset                                                            NaN
args                 [C:\Windows\system32\SearchFilterHost.exe, 0, ...
pid                                                               6196
start                                         2020-09-21T00:46:12.450Z
hash                 {'sha1': '38998a827eb2e7b4879bc4e8b09de632793e...
ppid                                                              7776
working_directory                                  C:\Windows\system32
entity_id                                             kOE5rcBfgspO0Juc
executable                    C:\Windows\System32\SearchFilterHost.exe
group                                                              NaN
Name: process, dtype: object

for  item in df.process:
    print(item)

nan
nan
SearchFilterHost.exe
nan
nan
nan
nan
nan
nan
nan
nan
['C:\\Windows\\system32\\SearchFilterHost.exe', '0', '736', '740', '748', '8192', '744']
6196
2020-09-21T00:46:12.450Z
{'sha1': '38998a827eb2e7b4879bc4e8b09de632793e4976'}
7776
C:\Windows\system32
kOE5rcBfgspO0Juc
C:\Windows\System32\SearchFilterHost.exe
nan