maxzhao

不要害怕Exception和Error

0%

Edited on In

大模型

ChatGLM 【大语言模型-可针对特定领域】

应用场景：智能客服、机器人

LangChain 【语言模型的应用程序框架】

通义千问 【大语言模型服务】

开源免费，可以直接作为服务

AllenNLP【大语言模型工具】

自然语言处理研究的开源工具包

知识图谱

Neo4j【图库】

免费

模型

LAMA【图像处理】

数据处理

开源：

  • kettle
  • nifi

本文地址： https://github.com/maxzhao-it/blog/post/b988e32b/

Edited on In ,

前言

交换机（Exchange）为rabbitmq独特的概念，用到的最常见的是4中类型：

  1. Direct: 先匹配, 再投送。即在绑定时设定一个routing_key, 消息的routing_key匹配时, 才会被交换器投送到绑定的队列中去. 交换机跟队列必须是精确的对应关系，这种最为简单。
  2. Topic: 转发消息主要是根据通配符。在这种交换机下，队列和交换机的绑定会定义一种路由模式，那么，通配符就要在这种路由模式和路由键之间匹配后交换机才能转发消息 这种可以认为是Direct 的灵活版
  3. Headers: 也是根据规则匹配, 相较于 direct 和 topic 固定地使用 routingkey , headers则是一个自定义匹配规则的类型， 在队列与交换器绑定时会设定一组键值对规则，消息中也包括一组键值对( headers属性)，当这些键值对有一对或全部匹配时，消息被投送到对应队列
  4. Fanout : 消息广播模式，不管路由键或者是路由模式，会把消息发给绑定给它的全部队列，如果配置了routingkey会被忽略

案例

topic

集群环境样例，同队列

发送端：topic=a.topic
接收端A： queue_a_1 routingKey=a.topic.routing
接收端B： queue_a_1 routingKey=a.topic.routing

发送端发送数据：C
接收端A、B： queue_a_1 任意端收到 C

集群环境样例，不同队列

发送端：topic=a.topic
接收端A： queue_a_1 routingKey=a.topic.routing
接收端B： queue_a_1 routingKey=a.topic.routing
接收端C： queue_a_2 routingKey=a.topic.routing

发送端发送数据：C
接收端A、B： queue_a_1 任意端收到 C
接收端C： queue_a_2 收到 C

集群环境样例，不同队列2

发送端：topic=a.topic
接收端A： queue_a_1 routingKey=a.topic.routing
接收端B： queue_a_1 routingKey=a.topic.routing
接收端C： queue_a_2 routingKey=a.topic.routing
接收端D： queue_a_2 routingKey=a.topic.#
接收端E： queue_a_2 routingKey=a.topic.#

发送端发送数据：C
接收端A、B： queue_a_1 任意端收到 C
接收端C、D、E： queue_a_2 任意端收到 C

本文地址： https://github.com/maxzhao-it/blog/post/934456223/

Edited on In ,

配置中文

setting -> 搜索 Language & Region -> 添加中文 -> TranslationLanguages -> Chinese Download -> 并将简体中文移动到第一个

重启

引导优化

1
diskutil list

macos-disk-0.png

disk0 虚拟硬盘
disk1 Opencore 镜像盘
disk2 是系统盘

现在要把 disk1s1 覆盖 disk0s1

1
sudo dd if=/dev/disk1s1 of=/dev/disk0s1

关机，分离 OpenCore，启动

开机自启

必须要做 引导优化

下载 https://github.com/corpnewt/ProperTree

ProperTree.zip

安装 python3

运行 propertree/buildapp-select.command

选择 /Library/Frameworks/Python.framework/Versions/3.12/bin/python3

加载引导

1
sudo diskutil mount disk0s1

打开访达的位置-> EFI/OC/config.plist

cmd +f 搜索 showpicker ，选项改为 False，cmd +s 保存

重启，就可以自动重启了

使用 U盘

使用USB端口，选择U盘的端口，下次再次插入 U 盘、硬盘，键盘都可以直通。

USB2.0 与 USB2.0 是不同的端口，所以同一个接口需要配置两次才可以兼容 USB2.0 和 USB3.0

开启 VNC 登录

setting -> sharing -> Screen Sharing -> + -> 选择当前账号

开启 SSH 登录

setting -> sharing -> Remote Login -> + -> 选择当前账号

显卡直通

NVRAM-boot-args

改为 keepsyms=1 -v agdpmod=pikera

本文地址： https://github.com/maxzhao-it/blog/post/9be1de010/

Edited on In ,

下载镜像

https://github.com/thenickdude/KVM-Opencore/releases

下载 OpenCore-v20.iso.gz 找到最新的就可以

创建 MacOS镜像

Ubuntu 系统下

打包

好用

1
2
3
4
5
cd ~/
git clone https://github.com/thenickdude/OSX-KVM
cd ~/OSX-KVM/scripts/monterey
apt install -y qemu-utils make
make Monterey-recovery.img

弃用

1
2
3
4
5
cd ~/
git clone https://github.com/thenickdude/OSX-KVM
cd ~/OSX-KVM/scripts/ventura
apt install -y qemu-utils make
make Ventura-recovery.img

iso上传 pve

将

  • Monterey-recovery.img
  • OpenCore-v20.iso

上传到 pve ISO 镜像中

这里有 2023-11-25下载的版本：

OpenCore-v20.iso.gz

OSX-KVM-master.zip

修改 vm.conf

这里VM 的id 为 1000

1
vim /etc/pve/qemu-server/1000.conf

在第二行添加参数（intel 的处理器）

1
args: -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -global nec-usb-xhci.msi=off -cpu host,kvm=on,vendor=GenuineIntel,+kvm_pv_unhalt,+kvm_pv_eoi,+hypervisor,+invtsc

将两个ISO的配置修改：

1
media=cdrom  改为 cache=unsafe

结果如下:

配置 VM

这里 VM ID 设置为 1000





启动系统

选第一个按回车就可以了

配置磁盘


结束后左上角叉掉

  • Agree
  • Agree
  • 选择硬盘 maxmac

等待重启

  • 第一次选择 install
  • 第二次选择 install
  • 第三次选择 maxmac
  • 第四次选择 maxmac

启动后选择 China

Migration Assistant 选择 Not Now

时区选择shanghai

下一步优化 MacOS

PVE8中的MacOS系统优化

修改时间

1
date -v+8H "+%m%d%H%M%S" | xargs date

本文地址： https://github.com/maxzhao-it/blog/post/9be1de00/

Edited on In

版本：

默认安装路径 /opt/logstash 、 /opt/filebeat

Logstash 配置

测试接收 filebeat

新建文件 config/first-pipeline.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
input {
    beats {
        port => "5044"
    }
}
# The filter part of this file is commented out to indicate that it is
# optional.
filter {
    grok {
        match => { "message" => "(?<year>\d{4})(?<month>\d{2})(?<day>\d{2})(?<hour>\d{2})(?<minute>\d{2})(?<second>\d{2})(?<millisecond>\d{3}):%{LOGLEVEL:logLevel}%{SPACE}\[%{INT:thread}\]%{SPACE}\[%{WORD:projectName}\]%{SPACE}\[(?<cId>.*?|\d{0,})\]%{SPACE}\[%{DATA:cpId}\]%{SPACE}\[%{DATA:rTId}\]%{SPACE}%{JAVACLASS:java}\(%{INT:javaNum}\)\:%{SPACE}%{JAVALOGMESSAGE:msg}"}
    }       
	mutate {
        add_field => {
            "timestamp" => "%{year}%{month}%{day}%{hour}%{minute}%{second}%{millisecond}"
        }
		remove_field =>["log","input","ecs","event","host","agent","message"]
		
    }
	date {
		match => [ "timestamp", "yyyyMMddHHmmssSSS" ]
		target => "@timestamp"
	}
}
output {
    stdout { codec => rubydebug }
    if "test" in [projectName] {
        elasticsearch {
            hosts => "192.168.2.8:9200"
            index => "app-log-api-%{+YYYY.MM.dd}"
        }
    }
    if "y" in [projectName] {
        elasticsearch {
            hosts => "192.168.2.8:9200"
            index => "app-log-opt-%{+YYYY.MM.dd}"
        }
    }
    if "x" in [projectName] {
        elasticsearch {
            hosts => "192.168.2.8:9200"
            index => "app-log-sso-%{+YYYY.MM.dd}"
        }
    }
}

测试

1
bin/logstash -f first-pipeline.conf --config.test_and_exit

启动

1
2
# 自动更新 --config.reload.automatic
bin/logstash -f first-pipeline.conf --config.reload.automatic

Filebeat 配置

Filebeat.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
filebeat.inputs:
- type: filestream
  id: my-filestream-id
  enabled: true
  paths:
  # 日志文件路径修改
    - /data/test-log/*.log
  prospector.scanner.exclude_files: ['\.gz$']
  parsers:
  - multiline:
      type: pattern
      # 与模式不匹配的连续行将附加到匹配的上一行
      pattern: '^\d'
      negate: true
      # 与模式匹配的连续行将附加到不匹配的上一行
      #pattern: '^[[:space:]]+(at|\.{3})[[:space:]]+\b|^[a-z]'
      #negate: false
      match: after
      timeout: 30s
      max_lines: 100
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
# 不用es 这里要关闭
setup.template.enabled: false
setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false
#setup.kibana:
output.elasticsearch:
  enabled: false
output.logstash:
  enabled: true
  hosts: ["127.0.0.1:5044"]
  compression_level: 1
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

启动

1
filebeat -c filebeat.yml -e

附

grok 解析日志

新版本的 logstash 自带 grok 插件

环境信息：

- 部署路径：/opt/logstash
- Log4j2 对接方式：Socket
- 部署地址：192.168.2.8
- logstash建议使用 grok插件

grok是一个十分强大的logstash filter插件，他可以通过正则解析任意文本，将非结构化日志数据解析成结构化和方便查询的结构。
在线测试 官方pattern 插件安装
安装插件

1
2
# 查看已安装
/opt/logstash/bin/logstash-plugin list --verbose

离线安装

1
2
3
4
5
6
7
mkdir /opt/logstash/plugins
cd  /opt/logstash/plugins
wget https://github.com/logstash-plugins/logstash-filter-grok/archive/refs/tags/v4.4.3.tar.gz
tar -zxvf v4.4.2.tar.gz
vim /opt/logstash/Gemfile
# 写入
# gem "logstash-filter-grok", :path => "path"

在线安装

1
2
bin/logstash-plugin install logstash-filter-grok
bin/logstash-plugin update logstash-filter-grok

部分 grok 表达式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
USERNAME [a-zA-Z0-9._-]+
USER %{USERNAME}
EMAILLOCALPART [a-zA-Z0-9!#$%&'*+\-/=?^_`{|}~]{1,64}(?:\.[a-zA-Z0-9!#$%&'*+\-/=?^_`{|}~]{1,62}){0,63}
EMAILADDRESS %{EMAILLOCALPART}@%{HOSTNAME}
INT (?:[+-]?(?:[0-9]+))
BASE10NUM (?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))
NUMBER (?:%{BASE10NUM})
BASE16NUM (?<![0-9A-Fa-f])(?:[+-]?(?:0x)?(?:[0-9A-Fa-f]+))
BASE16FLOAT \b(?<![0-9A-Fa-f.])(?:[+-]?(?:0x)?(?:(?:[0-9A-Fa-f]+(?:\.[0-9A-Fa-f]*)?)|(?:\.[0-9A-Fa-f]+)))\b

POSINT \b(?:[1-9][0-9]*)\b
NONNEGINT \b(?:[0-9]+)\b
WORD \b\w+\b
NOTSPACE \S+
SPACE \s*
DATA .*?
GREEDYDATA .*
QUOTEDSTRING (?>(?<!\\)(?>"(?>\\.|[^\\"]+)+"|""|(?>'(?>\\.|[^\\']+)+')|''|(?>`(?>\\.|[^\\`]+)+`)|``))
UUID [A-Fa-f0-9]{8}-(?:[A-Fa-f0-9]{4}-){3}[A-Fa-f0-9]{12}
# URN, allowing use of RFC 2141 section 2.3 reserved characters
URN urn:[0-9A-Za-z][0-9A-Za-z-]{0,31}:(?:%[0-9a-fA-F]{2}|[0-9A-Za-z()+,.:=@;$_!*'/?#-])+

# Networking
MAC (?:%{CISCOMAC}|%{WINDOWSMAC}|%{COMMONMAC})
CISCOMAC (?:(?:[A-Fa-f0-9]{4}\.){2}[A-Fa-f0-9]{4})
WINDOWSMAC (?:(?:[A-Fa-f0-9]{2}-){5}[A-Fa-f0-9]{2})
COMMONMAC (?:(?:[A-Fa-f0-9]{2}:){5}[A-Fa-f0-9]{2})
IPV6 ((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?
IPV4 (?<![0-9])(?:(?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5]))(?![0-9])
IP (?:%{IPV6}|%{IPV4})
HOSTNAME \b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(\.?|\b)
IPORHOST (?:%{IP}|%{HOSTNAME})
HOSTPORT %{IPORHOST}:%{POSINT}

# paths (only absolute paths are matched)
PATH (?:%{UNIXPATH}|%{WINPATH})
UNIXPATH (/[[[:alnum:]]_%!$@:.,+~-]*)+
TTY (?:/dev/(pts|tty([pq])?)(\w+)?/?(?:[0-9]+))
WINPATH (?>[A-Za-z]+:|\\)(?:\\[^\\?*]*)+
URIPROTO [A-Za-z]([A-Za-z0-9+\-.]+)+
URIHOST %{IPORHOST}(?::%{POSINT})?
# uripath comes loosely from RFC1738, but mostly from what Firefox doesn't turn into %XX
URIPATH (?:/[A-Za-z0-9$.+!*'(){},~:;=@#%&_\-]*)+
URIQUERY [A-Za-z0-9$.+!*'|(){},~@#%&/=:;_?\-\[\]<>]*
# deprecated (kept due compatibility):
URIPARAM \?%{URIQUERY}
URIPATHPARAM %{URIPATH}(?:\?%{URIQUERY})?
URI %{URIPROTO}://(?:%{USER}(?::[^@]*)?@)?(?:%{URIHOST})?(?:%{URIPATH}(?:\?%{URIQUERY})?)?

# Months: January, Feb, 3, 03, 12, December
MONTH \b(?:[Jj]an(?:uary|uar)?|[Ff]eb(?:ruary|ruar)?|[Mm](?:a|ä)?r(?:ch|z)?|[Aa]pr(?:il)?|[Mm]a(?:y|i)?|[Jj]un(?:e|i)?|[Jj]ul(?:y|i)?|[Aa]ug(?:ust)?|[Ss]ep(?:tember)?|[Oo](?:c|k)?t(?:ober)?|[Nn]ov(?:ember)?|[Dd]e(?:c|z)(?:ember)?)\b
MONTHNUM (?:0?[1-9]|1[0-2])
MONTHNUM2 (?:0[1-9]|1[0-2])
MONTHDAY (?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])

# Days: Monday, Tue, Thu, etc...
DAY (?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)

# Years?
YEAR (?>\d\d){1,2}
HOUR (?:2[0123]|[01]?[0-9])
MINUTE (?:[0-5][0-9])
# '60' is a leap second in most time standards and thus is valid.
SECOND (?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)
TIME (?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])
# datestamp is YYYY/MM/DD-HH:MM:SS.UUUU (or something like it)
DATE_US %{MONTHNUM}[/-]%{MONTHDAY}[/-]%{YEAR}
DATE_EU %{MONTHDAY}[./-]%{MONTHNUM}[./-]%{YEAR}
ISO8601_TIMEZONE (?:Z|[+-]%{HOUR}(?::?%{MINUTE}))
ISO8601_SECOND %{SECOND}
TIMESTAMP_ISO8601 %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?
DATE %{DATE_US}|%{DATE_EU}
DATESTAMP %{DATE}[- ]%{TIME}
TZ (?:[APMCE][SD]T|UTC)
DATESTAMP_RFC822 %{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}
DATESTAMP_RFC2822 %{DAY}, %{MONTHDAY} %{MONTH} %{YEAR} %{TIME} %{ISO8601_TIMEZONE}
DATESTAMP_OTHER %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{TZ} %{YEAR}
DATESTAMP_EVENTLOG %{YEAR}%{MONTHNUM2}%{MONTHDAY}%{HOUR}%{MINUTE}%{SECOND}

# Syslog Dates: Month Day HH:MM:SS
SYSLOGTIMESTAMP %{MONTH} +%{MONTHDAY} %{TIME}
PROG [\x21-\x5a\x5c\x5e-\x7e]+
SYSLOGPROG %{PROG:[process][name]}(?:\[%{POSINT:[process][pid]:int}\])?
SYSLOGHOST %{IPORHOST}
SYSLOGFACILITY <%{NONNEGINT:[log][syslog][facility][code]:int}.%{NONNEGINT:[log][syslog][priority]:int}>
HTTPDATE %{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT}

# Shortcuts
QS %{QUOTEDSTRING}

# Log formats
SYSLOGBASE %{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:[host][hostname]} %{SYSLOGPROG}:

# Log Levels
LOGLEVEL ([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo?(?:rmation)?|INFO?(?:RMATION)?|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)

部分 grok java 表达式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
JAVACLASS (?:[a-zA-Z$_][a-zA-Z$_0-9]*\.)*[a-zA-Z$_][a-zA-Z$_0-9]*
#Space is an allowed character to match special cases like 'Native Method' or 'Unknown Source'
JAVAFILE (?:[a-zA-Z$_0-9. -]+)
#Allow special <init>, <clinit> methods
JAVAMETHOD (?:(<(?:cl)?init>)|[a-zA-Z$_][a-zA-Z$_0-9]*)
#Line number is optional in special cases 'Native method' or 'Unknown source'
JAVASTACKTRACEPART %{SPACE}at %{JAVACLASS:[java][log][origin][class][name]}\.%{JAVAMETHOD:[log][origin][function]}\(%{JAVAFILE:[log][origin][file][name]}(?::%{INT:[log][origin][file][line]:int})?\)
# Java Logs
JAVATHREAD (?:[A-Z]{2}-Processor[\d]+)
JAVALOGMESSAGE (?:.*)

# MMM dd, yyyy HH:mm:ss eg: Jan 9, 2014 7:13:13 AM
# matches default logging configuration in Tomcat 4.1, 5.0, 5.5, 6.0, 7.0
CATALINA7_DATESTAMP %{MONTH} %{MONTHDAY}, %{YEAR} %{HOUR}:%{MINUTE}:%{SECOND} (?:AM|PM)
CATALINA7_LOG %{CATALINA7_DATESTAMP:timestamp} %{JAVACLASS:[java][log][origin][class][name]}(?: %{JAVAMETHOD:[log][origin][function]})?\s*(?:%{LOGLEVEL:[log][level]}:)? %{JAVALOGMESSAGE:message}

# 31-Jul-2020 16:40:38.578 in Tomcat 8.5/9.0
CATALINA8_DATESTAMP %{MONTHDAY}-%{MONTH}-%{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}
CATALINA8_LOG %{CATALINA8_DATESTAMP:timestamp} %{LOGLEVEL:[log][level]} \[%{DATA:[java][log][origin][thread][name]}\] %{JAVACLASS:[java][log][origin][class][name]}\.(?:%{JAVAMETHOD:[log][origin][function]})? %{JAVALOGMESSAGE:message}

CATALINA_DATESTAMP (?:%{CATALINA8_DATESTAMP})|(?:%{CATALINA7_DATESTAMP})
CATALINALOG (?:%{CATALINA8_LOG})|(?:%{CATALINA7_LOG})

# in Tomcat 5.5, 6.0, 7.0 it is the same as catalina.out logging format
TOMCAT7_LOG %{CATALINA7_LOG}
TOMCAT8_LOG %{CATALINA8_LOG}

# NOTE: a weird log we started with - not sure what TC version this should match out of the box (due the | delimiters)
TOMCATLEGACY_DATESTAMP %{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}(?: %{ISO8601_TIMEZONE})?
TOMCATLEGACY_LOG %{TOMCATLEGACY_DATESTAMP:timestamp} \| %{LOGLEVEL:[log][level]} \| %{JAVACLASS:[java][log][origin][class][name]} - %{JAVALOGMESSAGE:message}

TOMCAT_DATESTAMP (?:%{CATALINA8_DATESTAMP})|(?:%{CATALINA7_DATESTAMP})|(?:%{TOMCATLEGACY_DATESTAMP})

TOMCATLOG (?:%{TOMCAT8_LOG})|(?:%{TOMCAT7_LOG})|(?:%{TOMCATLEGACY_LOG})

本文地址： https://github.com/maxzhao-it/blog/post/af5eafbf/

Edited on In ,

这里先将日志直接通过 filebeat 输出到文件用于测试

log4j2配置日志输出

1
%d{yyyyMMddHHmmssSSS}:%p [%T] [${pn}] [%X{cId}] [%X{cpId}] [%X{rTId}] %c{1.}(%L): %m%xwEx%n

%X{cId} 在java 中可以使用 ThreadContext.put("cId","test") 实现

filebeat.yml 配置

这里输出到日志文件测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
filebeat.inputs:
# 新版的日志文件采集，不要使用 log
- type: filestream
  id: my-filestream-id
  enabled: true
  paths:
    - /data/test-log/*.log
  #exclude_lines: ['^DBG']
  #include_lines: ['^ERR', '^WARN']
  prospector.scanner.exclude_files: ['\.gz$']
  #fields:
  #  level: debug
  #  review: 1
  parsers:
  - multiline:
      type: pattern
      # 与模式不匹配的连续行将附加到匹配的上一行
      pattern: '^\d'
      negate: true
      # 与模式匹配的连续行将附加到不匹配的上一行，解决日志文件输出到一行的问题
      #pattern: '^[[:space:]]+(at|\.{3})[[:space:]]+\b|^[a-z]'
      #negate: false
      match: after
      timeout: 30s
      max_lines: 100
# ============================== Filebeat modules ==============================
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
# 不用es 这里要关闭
setup.template.enabled: false
setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false
setup.kibana:
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  enabled: false
output.logstash:
  enabled: false
  hosts: ["127.0.0.1:5044"]
  compression_level: 1
output.file:
  enabled: true
  path: "/data/filebeat"
  filename: filebeat
  rotate_every_kb: 10240
  # 保留的最多文件个数
  number_of_files: 7
  # 文件权限
  permissions: 0600
  # 写入文件存在则写入新文件
  rotate_on_startup: false
  # 默认输出 json
  #codec
# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

multiline.pattern表达式

本文地址： https://github.com/maxzhao-it/blog/post/374dd8da/

Edited on In

virtio-win-0.1.229.iso
Win11_23H2_Chinese_Simplified_x64.iso

上传镜像

local -> ISO 镜像

创建虚拟机

操作系统选择 Microsoft Windows

机型 q35
SCSI VirtIO SCSI single
BIOS OVMF (UEFI)
添加 TMP
格式 QEMU映像格式 (qcow2)

磁盘:
总线 SCSI
缓存 Write back (不安全)
格式 QEMU映像格式 (qcow2)
SSD 仿真 勾选

CPU：
类别 host

添加ISO

添加 CD

IDE

virtio-win-0.1.229.iso

选项启动顺序

scsi0 ide0 ide2 不能使用 ide1

跳过登录

shift + F10

oobe\bypassnro

win11激活

windows11激活
powershell

irm https://massgrave.dev/get | iex
选 1

离线的可以下载 MAS https://link.zhihu.com/?target=https%3A//github.com/massgravel/Microsoft-Activation-Scripts

安装驱动

virtio-win-0.1.229.iso
中的 virtio-gt-6x

本文地址： https://github.com/maxzhao-it/blog/post/af1c9197/