Merge remote-tracking branch 'upstream/debug' into debug
This commit is contained in:
1
.gitignore
vendored
1
.gitignore
vendored
@@ -3,6 +3,7 @@ data1/
|
|||||||
mongodb/
|
mongodb/
|
||||||
NapCat.Framework.Windows.Once/
|
NapCat.Framework.Windows.Once/
|
||||||
log/
|
log/
|
||||||
|
logs/
|
||||||
/test
|
/test
|
||||||
/src/test
|
/src/test
|
||||||
message_queue_content.txt
|
message_queue_content.txt
|
||||||
|
|||||||
@@ -77,6 +77,10 @@
|
|||||||
- [🎀 新手配置指南](docs/installation_cute.md) - 通俗易懂的配置教程,适合初次使用的猫娘
|
- [🎀 新手配置指南](docs/installation_cute.md) - 通俗易懂的配置教程,适合初次使用的猫娘
|
||||||
- [⚙️ 标准配置指南](docs/installation_standard.md) - 简明专业的配置说明,适合有经验的用户
|
- [⚙️ 标准配置指南](docs/installation_standard.md) - 简明专业的配置说明,适合有经验的用户
|
||||||
|
|
||||||
|
### 常见问题
|
||||||
|
|
||||||
|
- [❓ 快速 Q & A ](docs/fast_q_a.md) - 针对新手的疑难解答,适合完全没接触过编程的新手
|
||||||
|
|
||||||
<div align="left">
|
<div align="left">
|
||||||
<h3>了解麦麦 </h3>
|
<h3>了解麦麦 </h3>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
14
bot.py
14
bot.py
@@ -19,6 +19,19 @@ env_mask = {key: os.getenv(key) for key in os.environ}
|
|||||||
|
|
||||||
uvicorn_server = None
|
uvicorn_server = None
|
||||||
|
|
||||||
|
# 配置日志
|
||||||
|
log_path = os.path.join(os.getcwd(), "logs")
|
||||||
|
if not os.path.exists(log_path):
|
||||||
|
os.makedirs(log_path)
|
||||||
|
|
||||||
|
# 添加文件日志,启用rotation和retention
|
||||||
|
logger.add(
|
||||||
|
os.path.join(log_path, "maimbot_{time:YYYY-MM-DD}.log"),
|
||||||
|
rotation="00:00", # 每天0点创建新文件
|
||||||
|
retention="30 days", # 保留30天的日志
|
||||||
|
level="INFO",
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
|
||||||
def easter_egg():
|
def easter_egg():
|
||||||
# 彩蛋
|
# 彩蛋
|
||||||
@@ -207,7 +220,6 @@ def raw_main():
|
|||||||
time.tzset()
|
time.tzset()
|
||||||
|
|
||||||
easter_egg()
|
easter_egg()
|
||||||
load_logger()
|
|
||||||
init_config()
|
init_config()
|
||||||
init_env()
|
init_env()
|
||||||
load_env()
|
load_env()
|
||||||
|
|||||||
BIN
docs/API_KEY.png
Normal file
BIN
docs/API_KEY.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 47 KiB |
BIN
docs/MONGO_DB_0.png
Normal file
BIN
docs/MONGO_DB_0.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 13 KiB |
BIN
docs/MONGO_DB_1.png
Normal file
BIN
docs/MONGO_DB_1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
BIN
docs/MONGO_DB_2.png
Normal file
BIN
docs/MONGO_DB_2.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
149
docs/fast_q_a.md
Normal file
149
docs/fast_q_a.md
Normal file
@@ -0,0 +1,149 @@
|
|||||||
|
## 快速更新Q&A❓
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
- 这个文件用来记录一些常见的新手问题。
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
### 完整安装教程
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
[MaiMbot简易配置教程](https://www.bilibili.com/video/BV1zsQ5YCEE6)
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
### Api相关问题
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
- 为什么显示:"缺失必要的API KEY" ❓
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
|
||||||
|
<img src="API_KEY.png" width=650>
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>你需要在 [Silicon Flow Api](https://cloud.siliconflow.cn/account/ak)
|
||||||
|
>网站上注册一个账号,然后点击这个链接打开API KEY获取页面。
|
||||||
|
>
|
||||||
|
>点击 "新建API密钥" 按钮新建一个给MaiMBot使用的API KEY。不要忘了点击复制。
|
||||||
|
>
|
||||||
|
>之后打开MaiMBot在你电脑上的文件根目录,使用记事本或者其他文本编辑器打开 [.env.prod](../.env.prod)
|
||||||
|
>这个文件。把你刚才复制的API KEY填入到 "SILICONFLOW_KEY=" 这个等号的右边。
|
||||||
|
>
|
||||||
|
>在默认情况下,MaiMBot使用的默认Api都是硅基流动的。
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
|
||||||
|
- 我想使用硅基流动之外的Api网站,我应该怎么做 ❓
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>你需要使用记事本或者其他文本编辑器打开config目录下的 [bot_config.toml](../config/bot_config.toml)
|
||||||
|
>然后修改其中的 "provider = " 字段。同时不要忘记模仿 [.env.prod](../.env.prod)
|
||||||
|
>文件的写法添加 Api Key 和 Base URL。
|
||||||
|
>
|
||||||
|
>举个例子,如果你写了 " provider = \"ABC\" ",那你需要相应的在 [.env.prod](../.env.prod)
|
||||||
|
>文件里添加形如 " ABC_BASE_URL = https://api.abc.com/v1 " 和 " ABC_KEY = sk-1145141919810 " 的字段。
|
||||||
|
>
|
||||||
|
>**如果你对AI没有较深的了解,修改识图模型和嵌入模型的provider字段可能会产生bug,因为你从Api网站调用了一个并不存在的模型**
|
||||||
|
>
|
||||||
|
>这个时候,你需要把字段的值改回 "provider = \"SILICONFLOW\" " 以此解决bug。
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
### MongoDB相关问题
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
- 我应该怎么清空bot内存储的表情包 ❓
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>打开你的MongoDB Compass软件,你会在左上角看到这样的一个界面:
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
><img src="MONGO_DB_0.png" width=250>
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>点击 "CONNECT" 之后,点击展开 MegBot 标签栏
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
><img src="MONGO_DB_1.png" width=250>
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>点进 "emoji" 再点击 "DELETE" 删掉所有条目,如图所示
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
><img src="MONGO_DB_2.png" width=450>
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>你可以用类似的方式手动清空MaiMBot的所有服务器数据。
|
||||||
|
>
|
||||||
|
>MaiMBot的所有图片均储存在 [data](../data) 文件夹内,按类型分为 [emoji](../data/emoji) 和 [image](../data/image)
|
||||||
|
>
|
||||||
|
>在删除服务器数据时不要忘记清空这些图片。
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
- 为什么我连接不上MongoDB服务器 ❓
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>这个问题比较复杂,但是你可以按照下面的步骤检查,看看具体是什么问题
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
> 1. 检查有没有把 mongod.exe 所在的目录添加到 path。 具体可参照
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>  [CSDN-windows10设置环境变量Path详细步骤](https://blog.csdn.net/flame_007/article/details/106401215)
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
>  **需要往path里填入的是 exe 所在的完整目录!不带 exe 本体**
|
||||||
|
>
|
||||||
|
><br>
|
||||||
|
>
|
||||||
|
> 2. 待完成
|
||||||
|
>
|
||||||
|
><br>
|
||||||
@@ -1,141 +0,0 @@
|
|||||||
cbb569e - Create 如果你更新了版本,点我.txt
|
|
||||||
a91ef7b - 自动升级配置文件脚本
|
|
||||||
ed18f2e - 新增了知识库一键启动漂亮脚本
|
|
||||||
80ed568 - fix: 删除print调试代码
|
|
||||||
c681a82 - 修复小名无效问题
|
|
||||||
e54038f - fix: 从 nixpkgs 增加 numpy 依赖,以避免出现 libc++.so 找不到的问题
|
|
||||||
26782c9 - fix: 修复 ENVIRONMENT 变量在同一终端下不能被覆盖的问题
|
|
||||||
8c34637 - 提高健壮性
|
|
||||||
2688a96 - close SengokuCola/MaiMBot#225 让麦麦可以正确读取分享卡片
|
|
||||||
cd16e68 - 修复表情包发送时的缺失参数
|
|
||||||
b362c35 - feat: 更新 flake.nix ,采用 venv 的方式生成环境,nixos用户也可以本机运行项目了
|
|
||||||
3c8c897 - 屏蔽一个臃肿的debug信息
|
|
||||||
9d0152a - 修复了合并过程中造成的代码重复
|
|
||||||
956135c - 添加一些注释
|
|
||||||
a412741 - 将print变为logger.debug
|
|
||||||
3180426 - 修复了没有改掉的typo字段
|
|
||||||
aea3bff - 添加私聊过滤开关,更新config,增加约束
|
|
||||||
cda6281 - chore: update emoji_manager.py
|
|
||||||
baed856 - 修正了私聊屏蔽词输出
|
|
||||||
66a0f18 - 修复了私聊时产生reply消息的bug
|
|
||||||
3bf5cd6 - feat: 新增运行时重载配置文件;新增根据不同环境(dev;prod)显示不同级别的log
|
|
||||||
33cd83b - 添加私聊功能
|
|
||||||
aa41f0d - fix: 放反了
|
|
||||||
ef8691c - fix: 修改message继承逻辑,修复回复消息无法识别
|
|
||||||
7d017be - fix:模型降级
|
|
||||||
e1019ad - fix: 修复变量拼写错误并优化代码可读性
|
|
||||||
c24bb70 - fix: 流式输出模式增加结束判断与token用量记录
|
|
||||||
60a9376 - 添加logger的debug输出开关,默认为不开启
|
|
||||||
bfa9a3c - fix: 添加群信息获取的错误处理 (#173)
|
|
||||||
4cc5c8e - 修正.env.prod和.env.dev的生成
|
|
||||||
dea14c1 - fix: 模型降级目前只对硅基流动的V3和R1生效
|
|
||||||
b6edbea - fix: 图片保存路径不正确
|
|
||||||
01a6fa8 - fix: 删除神秘test
|
|
||||||
20f009d - 修复systemctl强制停止maimbot的问题
|
|
||||||
af962c2 - 修复了情绪管理器没有正确导入导致发布出消息
|
|
||||||
0586700 - 按照Sourcery提供的建议修改systemctl管理指南
|
|
||||||
e48b32a - 在手动部署教程中增加使用systemctl管理
|
|
||||||
5760412 - fix: 小修
|
|
||||||
1c9b0cc - fix: 修复部分cq码解析错误,merge
|
|
||||||
b6867b9 - fix: 统一使用os.getenv获取数据库连接信息,避免从config对象获取不存在的值时出现KeyError
|
|
||||||
5e069f7 - 修复记忆保存时无时间信息的bug
|
|
||||||
73a3e41 - 修复记忆更新bug
|
|
||||||
52c93ba - refactor: use Base64 for emoji CQ codes
|
|
||||||
67f6d7c - fix: 保证能运行的小修改
|
|
||||||
c32c4fb - refactor: 修改配置文件的版本号
|
|
||||||
a54ca8c - Merge remote-tracking branch 'upstream/debug' into feat_regix
|
|
||||||
8cbf9bb - feat: 史上最好的消息流重构和图片管理
|
|
||||||
9e41c4f - feat: 修改 bot_config 0.0.5 版本的变更日志
|
|
||||||
eede406 - fix: 修复nonebot无法加载项目的问题
|
|
||||||
00e02ed - fix: 0.0.5 版本的增加分层控制项
|
|
||||||
0f99d6a - Update docs/docker_deploy.md
|
|
||||||
c789074 - feat: 增加ruff依赖
|
|
||||||
ff65ab8 - feat: 修改默认的ruff配置文件,同时消除config的所有不符合规范的地方
|
|
||||||
bf97013 - feat: 精简日志,禁用Uvicorn/NoneBot默认日志;启动方式改为显示加载uvicorn,以便优雅shutdown
|
|
||||||
d9a2863 - 优化Docker部署文档更新容器部分
|
|
||||||
efcf00f - Docker部署文档追加更新部分
|
|
||||||
a63ce96 - fix: 更新情感判断模型配置(使配置文件里的 llm_emotion_judge 生效)
|
|
||||||
1294c88 - feat: 增加标准化格式化设置
|
|
||||||
2e8cd47 - fix: 避免可能出现的日程解析错误
|
|
||||||
043a724 - 修一下文档跳转,小美化(
|
|
||||||
e4b8865 - 支持别名,可以用不同名称召唤机器人
|
|
||||||
7b35ddd - ruff 哥又有新点子
|
|
||||||
7899e67 - feat: 重构完成开始测试debug
|
|
||||||
354d6d0 - 记忆系统优化
|
|
||||||
6cef8fd - 修复时区,删去napcat用不到的端口
|
|
||||||
cd96644 - 添加使用说明
|
|
||||||
84495f8 - fix
|
|
||||||
204744c - 修改配置名与修改过滤对象为raw_message
|
|
||||||
a03b490 - Update README.md
|
|
||||||
2b2b342 - feat: 增加 ruff 依赖
|
|
||||||
72a6749 - fix: 修复docker部署时区指定问题
|
|
||||||
ee579bc - Update README.md
|
|
||||||
1b611ec - resolve SengokuCola/MaiMBot#167 根据正则表达式过滤消息
|
|
||||||
6e2ea82 - refractor: 几乎写完了,进入测试阶段
|
|
||||||
2ffdfef - More
|
|
||||||
e680405 - fix: typo 'discription'
|
|
||||||
68b3f57 - Minor Doc Update
|
|
||||||
312f065 - Create linux_deploy_guide_for_beginners.md
|
|
||||||
ed505a4 - fix: 使用动态路径替换硬编码的项目路径
|
|
||||||
8ff7bb6 - docs: 更新文档,修正格式并添加必要的换行符
|
|
||||||
6e36a56 - feat: 增加 MONGODB_URI 的配置项,并将所有env文件的注释单独放在一行(python的dotenv有时无法正确处理行内注释)
|
|
||||||
4baa6c6 - feat: 实现MongoDB URI方式连接,并统一数据库连接代码。
|
|
||||||
8a32d18 - feat: 优化willing_manager逻辑,增加回复保底概率
|
|
||||||
c9f1244 - docs: 改进README.md文档格式和排版
|
|
||||||
e1b484a - docs: 添加CLAUDE.md开发指南文件(用于Claude Code)
|
|
||||||
a43f949 - fix: remove duplicate message(CR comments)
|
|
||||||
fddb641 - fix: 修复错误的空值检测逻辑
|
|
||||||
8b7876c - fix: 修复没有上传tag的问题
|
|
||||||
6b4130e - feat: 增加stable-dev分支的打包
|
|
||||||
052e67b - refactor: 日志打印优化(终于改完了,爽了
|
|
||||||
a7f9d05 - 修复记忆整理传入格式问题
|
|
||||||
536bb1d - fix: 更新情感判断模型配置
|
|
||||||
8d99592 - fix: logger初始化顺序
|
|
||||||
052802c - refactor: logger promotion
|
|
||||||
8661d94 - doc: README.md - telegram version information
|
|
||||||
5746afa - refactor: logger in src\plugins\chat\bot.py
|
|
||||||
288dbb6 - refactor: logger in src\plugins\chat\__init__.py
|
|
||||||
8428a06 - fix: memory logger optimization (CR comment)
|
|
||||||
665c459 - 改进了可视化脚本
|
|
||||||
6c35704 - fix: 调用了错误的函数
|
|
||||||
3223153 - feat: 一键脚本新增记忆可视化
|
|
||||||
3149dd3 - fix: mongodb.zip 无法解压 fix:更换执行命令的方法 fix:当 db 不存在时自动创建 feat: 一键安装完成后启动麦麦
|
|
||||||
089d6a6 - feat: 针对硅基流动的Pro模型添加了自动降级功能
|
|
||||||
c4b0917 - 一个记忆可视化小脚本
|
|
||||||
6a71ea4 - 修复了记忆时间bug,config添加了记忆屏蔽关键词
|
|
||||||
1b5344f - fix: 优化bot初始化的日志&格式
|
|
||||||
41aa974 - fix: 优化chat/config.py的日志&格式
|
|
||||||
980cde7 - fix: 优化scheduler_generator日志&格式
|
|
||||||
31a5514 - fix: 调整全局logger加载顺序
|
|
||||||
8baef07 - feat: 添加全局logger初始化设置
|
|
||||||
5566f17 - refractor: 几乎写完了,进入测试阶段
|
|
||||||
6a66933 - feat: 添加开发环境.env.dev初始化
|
|
||||||
411ff1a - feat: 安装 MongoDB Compass
|
|
||||||
0de9eba - feat: 增加实时更新贡献者列表的功能
|
|
||||||
f327f45 - fix: 优化src/plugins/chat/__init__.py的import
|
|
||||||
826daa5 - fix: 当虚拟环境存在时跳过创建
|
|
||||||
f54de42 - fix: time.tzset 仅在类 Unix 系统可用
|
|
||||||
47c4990 - fix: 修复docker部署场景下时间错误的问题
|
|
||||||
e23a371 - docs: 添加 compose 注释
|
|
||||||
1002822 - docs: 标注 Python 最低版本
|
|
||||||
564350d - feat: 校验 Python 版本
|
|
||||||
4cc4482 - docs: 添加傻瓜式脚本
|
|
||||||
757173a - 带麦麦看了心理医生,让她没那么容易陷入负面情绪
|
|
||||||
39bb99c - 将错别字生成提取到配置,一句一个错别字太烦了!
|
|
||||||
fe36847 - feat: 超大型重构
|
|
||||||
e304dd7 - Update README.md
|
|
||||||
b7cfe6d - feat: 发布第 0.0.2 版本配置模板
|
|
||||||
ca929d5 - 补充Docker部署文档
|
|
||||||
1e97120 - 补充Docker部署文档
|
|
||||||
25f7052 - fix: 修复兼容性选项和目前第一个版本之间的版本间隙 0.0.0 版,并将所有的直接退出修改为抛出异常
|
|
||||||
c5bdc4f - 防ipv6炸,虽然小概率事件
|
|
||||||
d86610d - fix: 修复不能加载环境变量的问题
|
|
||||||
2306ebf - feat: 因为判断临界版本范围比较麻烦,增加 notice 字段,删除原本的判断逻辑(存在故障)
|
|
||||||
dd09576 - fix: 修复 TypeError: BotConfig.convert_to_specifierset() takes 1 positional argument but 2 were given
|
|
||||||
18f839b - fix: 修复 missing 1 required positional argument: 'INNER_VERSION'
|
|
||||||
6adb5ed - 调整一些细节,docker部署时可选数据库账密
|
|
||||||
07f48e9 - fix: 利用filter来过滤环境变量,避免直接删除key造成的 RuntimeError: dictionary changed size during iteration
|
|
||||||
5856074 - fix: 修复无法进行基础设置的问题
|
|
||||||
32aa032 - feat: 发布 0.0.1 版本的配置文件
|
|
||||||
edc07ac - feat: 重构配置加载器,增加配置文件版本控制和程序兼容能力
|
|
||||||
0f492ed - fix: 修复 BASE_URL/KEY 组合检查中被 GPG_KEY 干扰的问题
|
|
||||||
@@ -235,10 +235,10 @@ class ChatBot:
|
|||||||
is_head=not mark_head,
|
is_head=not mark_head,
|
||||||
is_emoji=False,
|
is_emoji=False,
|
||||||
)
|
)
|
||||||
print(f"bot_message: {bot_message}")
|
logger.debug(f"bot_message: {bot_message}")
|
||||||
if not mark_head:
|
if not mark_head:
|
||||||
mark_head = True
|
mark_head = True
|
||||||
print(f"添加消息到message_set: {bot_message}")
|
logger.debug(f"添加消息到message_set: {bot_message}")
|
||||||
message_set.add_message(bot_message)
|
message_set.add_message(bot_message)
|
||||||
|
|
||||||
# message_set 可以直接加入 message_manager
|
# message_set 可以直接加入 message_manager
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ import time
|
|||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from typing import Dict, List, Optional, Union
|
from typing import Dict, List, Optional, Union
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
import requests
|
import requests
|
||||||
|
|
||||||
# 解析各种CQ码
|
# 解析各种CQ码
|
||||||
|
|||||||
@@ -246,7 +246,7 @@ class EmojiManager:
|
|||||||
# 即使表情包已存在,也检查是否需要同步到images集合
|
# 即使表情包已存在,也检查是否需要同步到images集合
|
||||||
description = existing_emoji.get('discription')
|
description = existing_emoji.get('discription')
|
||||||
# 检查是否在images集合中存在
|
# 检查是否在images集合中存在
|
||||||
existing_image = image_manager.db.db.images.find_one({'hash': image_hash})
|
existing_image = image_manager.db.images.find_one({'hash': image_hash})
|
||||||
if not existing_image:
|
if not existing_image:
|
||||||
# 同步到images集合
|
# 同步到images集合
|
||||||
image_doc = {
|
image_doc = {
|
||||||
@@ -256,7 +256,7 @@ class EmojiManager:
|
|||||||
'description': description,
|
'description': description,
|
||||||
'timestamp': int(time.time())
|
'timestamp': int(time.time())
|
||||||
}
|
}
|
||||||
image_manager.db.db.images.update_one(
|
image_manager.db.images.update_one(
|
||||||
{'hash': image_hash},
|
{'hash': image_hash},
|
||||||
{'$set': image_doc},
|
{'$set': image_doc},
|
||||||
upsert=True
|
upsert=True
|
||||||
@@ -318,7 +318,7 @@ class EmojiManager:
|
|||||||
'description': description,
|
'description': description,
|
||||||
'timestamp': int(time.time())
|
'timestamp': int(time.time())
|
||||||
}
|
}
|
||||||
image_manager.db.db.images.update_one(
|
image_manager.db.images.update_one(
|
||||||
{'hash': image_hash},
|
{'hash': image_hash},
|
||||||
{'$set': image_doc},
|
{'$set': image_doc},
|
||||||
upsert=True
|
upsert=True
|
||||||
|
|||||||
@@ -88,13 +88,13 @@ def get_cloest_chat_from_db(db, length: int, timestamp: str):
|
|||||||
list: 消息记录列表,每个记录包含时间和文本信息
|
list: 消息记录列表,每个记录包含时间和文本信息
|
||||||
"""
|
"""
|
||||||
chat_records = []
|
chat_records = []
|
||||||
closest_record = db.db.messages.find_one({"time": {"$lte": timestamp}}, sort=[('time', -1)])
|
closest_record = db.messages.find_one({"time": {"$lte": timestamp}}, sort=[('time', -1)])
|
||||||
|
|
||||||
if closest_record:
|
if closest_record:
|
||||||
closest_time = closest_record['time']
|
closest_time = closest_record['time']
|
||||||
chat_id = closest_record['chat_id'] # 获取chat_id
|
chat_id = closest_record['chat_id'] # 获取chat_id
|
||||||
# 获取该时间戳之后的length条消息,保持相同的chat_id
|
# 获取该时间戳之后的length条消息,保持相同的chat_id
|
||||||
chat_records = list(db.db.messages.find(
|
chat_records = list(db.messages.find(
|
||||||
{
|
{
|
||||||
"time": {"$gt": closest_time},
|
"time": {"$gt": closest_time},
|
||||||
"chat_id": chat_id # 添加chat_id过滤
|
"chat_id": chat_id # 添加chat_id过滤
|
||||||
@@ -128,7 +128,7 @@ async def get_recent_group_messages(db, chat_id:str, limit: int = 12) -> list:
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
# 从数据库获取最近消息
|
# 从数据库获取最近消息
|
||||||
recent_messages = list(db.db.messages.find(
|
recent_messages = list(db.messages.find(
|
||||||
{"chat_id": chat_id},
|
{"chat_id": chat_id},
|
||||||
).sort("time", -1).limit(limit))
|
).sort("time", -1).limit(limit))
|
||||||
|
|
||||||
@@ -162,7 +162,7 @@ async def get_recent_group_messages(db, chat_id:str, limit: int = 12) -> list:
|
|||||||
|
|
||||||
|
|
||||||
def get_recent_group_detailed_plain_text(db, chat_stream_id: int, limit: int = 12, combine=False):
|
def get_recent_group_detailed_plain_text(db, chat_stream_id: int, limit: int = 12, combine=False):
|
||||||
recent_messages = list(db.db.messages.find(
|
recent_messages = list(db.messages.find(
|
||||||
{"chat_id": chat_stream_id},
|
{"chat_id": chat_stream_id},
|
||||||
{
|
{
|
||||||
"time": 1, # 返回时间字段
|
"time": 1, # 返回时间字段
|
||||||
|
|||||||
@@ -289,6 +289,7 @@ class ImageManager:
|
|||||||
async def get_image_description(self, image_base64: str) -> str:
|
async def get_image_description(self, image_base64: str) -> str:
|
||||||
"""获取普通图片描述,带查重和保存功能"""
|
"""获取普通图片描述,带查重和保存功能"""
|
||||||
try:
|
try:
|
||||||
|
print("处理图片中")
|
||||||
# 计算图片哈希
|
# 计算图片哈希
|
||||||
image_bytes = base64.b64decode(image_base64)
|
image_bytes = base64.b64decode(image_base64)
|
||||||
image_hash = hashlib.md5(image_bytes).hexdigest()
|
image_hash = hashlib.md5(image_bytes).hexdigest()
|
||||||
@@ -296,12 +297,15 @@ class ImageManager:
|
|||||||
# 查询缓存的描述
|
# 查询缓存的描述
|
||||||
cached_description = self._get_description_from_db(image_hash, 'image')
|
cached_description = self._get_description_from_db(image_hash, 'image')
|
||||||
if cached_description:
|
if cached_description:
|
||||||
|
print("图片描述缓存中")
|
||||||
return f"[图片:{cached_description}]"
|
return f"[图片:{cached_description}]"
|
||||||
|
|
||||||
# 调用AI获取描述
|
# 调用AI获取描述
|
||||||
prompt = "请用中文描述这张图片的内容。如果有文字,请把文字都描述出来。并尝试猜测这个图片的含义。最多200个字。"
|
prompt = "请用中文描述这张图片的内容。如果有文字,请把文字都描述出来。并尝试猜测这个图片的含义。最多200个字。"
|
||||||
description, _ = await self._llm.generate_response_for_image(prompt, image_base64)
|
description, _ = await self._llm.generate_response_for_image(prompt, image_base64)
|
||||||
|
|
||||||
|
print(f"描述是{description}")
|
||||||
|
|
||||||
if description is None:
|
if description is None:
|
||||||
logger.warning("AI未能生成图片描述")
|
logger.warning("AI未能生成图片描述")
|
||||||
return "[图片]"
|
return "[图片]"
|
||||||
|
|||||||
@@ -5,101 +5,98 @@ from typing import Dict
|
|||||||
from .config import global_config
|
from .config import global_config
|
||||||
from .chat_stream import ChatStream
|
from .chat_stream import ChatStream
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
|
||||||
class WillingManager:
|
class WillingManager:
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.chat_reply_willing: Dict[str, float] = {} # 存储每个聊天流的回复意愿
|
|
||||||
self.chat_reply_willing: Dict[str, float] = {} # 存储每个聊天流的回复意愿
|
self.chat_reply_willing: Dict[str, float] = {} # 存储每个聊天流的回复意愿
|
||||||
self._decay_task = None
|
self._decay_task = None
|
||||||
self._started = False
|
self._started = False
|
||||||
|
|
||||||
async def _decay_reply_willing(self):
|
async def _decay_reply_willing(self):
|
||||||
"""定期衰减回复意愿"""
|
"""定期衰减回复意愿"""
|
||||||
while True:
|
while True:
|
||||||
await asyncio.sleep(5)
|
await asyncio.sleep(5)
|
||||||
for chat_id in self.chat_reply_willing:
|
for chat_id in self.chat_reply_willing:
|
||||||
self.chat_reply_willing[chat_id] = max(0, self.chat_reply_willing[chat_id] * 0.6)
|
self.chat_reply_willing[chat_id] = max(0, self.chat_reply_willing[chat_id] * 0.6)
|
||||||
for chat_id in self.chat_reply_willing:
|
|
||||||
self.chat_reply_willing[chat_id] = max(0, self.chat_reply_willing[chat_id] * 0.6)
|
def get_willing(self, chat_stream: ChatStream) -> float:
|
||||||
|
|
||||||
def get_willing(self,chat_stream:ChatStream) -> float:
|
|
||||||
"""获取指定聊天流的回复意愿"""
|
"""获取指定聊天流的回复意愿"""
|
||||||
stream = chat_stream
|
stream = chat_stream
|
||||||
if stream:
|
if stream:
|
||||||
return self.chat_reply_willing.get(stream.stream_id, 0)
|
return self.chat_reply_willing.get(stream.stream_id, 0)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
def set_willing(self, chat_id: str, willing: float):
|
def set_willing(self, chat_id: str, willing: float):
|
||||||
"""设置指定聊天流的回复意愿"""
|
"""设置指定聊天流的回复意愿"""
|
||||||
self.chat_reply_willing[chat_id] = willing
|
self.chat_reply_willing[chat_id] = willing
|
||||||
def set_willing(self, chat_id: str, willing: float):
|
|
||||||
"""设置指定聊天流的回复意愿"""
|
async def change_reply_willing_received(
|
||||||
self.chat_reply_willing[chat_id] = willing
|
self,
|
||||||
|
chat_stream: ChatStream,
|
||||||
async def change_reply_willing_received(self,
|
topic: str = None,
|
||||||
chat_stream:ChatStream,
|
is_mentioned_bot: bool = False,
|
||||||
topic: str = None,
|
config=None,
|
||||||
is_mentioned_bot: bool = False,
|
is_emoji: bool = False,
|
||||||
config = None,
|
interested_rate: float = 0,
|
||||||
is_emoji: bool = False,
|
) -> float:
|
||||||
interested_rate: float = 0) -> float:
|
|
||||||
"""改变指定聊天流的回复意愿并返回回复概率"""
|
"""改变指定聊天流的回复意愿并返回回复概率"""
|
||||||
# 获取或创建聊天流
|
# 获取或创建聊天流
|
||||||
stream = chat_stream
|
stream = chat_stream
|
||||||
chat_id = stream.stream_id
|
chat_id = stream.stream_id
|
||||||
|
|
||||||
current_willing = self.chat_reply_willing.get(chat_id, 0)
|
current_willing = self.chat_reply_willing.get(chat_id, 0)
|
||||||
|
|
||||||
# print(f"初始意愿: {current_willing}")
|
|
||||||
if is_mentioned_bot and current_willing < 1.0:
|
if is_mentioned_bot and current_willing < 1.0:
|
||||||
current_willing += 0.9
|
current_willing += 0.9
|
||||||
print(f"被提及, 当前意愿: {current_willing}")
|
logger.debug(f"被提及, 当前意愿: {current_willing}")
|
||||||
elif is_mentioned_bot:
|
elif is_mentioned_bot:
|
||||||
current_willing += 0.05
|
current_willing += 0.05
|
||||||
print(f"被重复提及, 当前意愿: {current_willing}")
|
logger.debug(f"被重复提及, 当前意愿: {current_willing}")
|
||||||
|
|
||||||
if is_emoji:
|
if is_emoji:
|
||||||
current_willing *= 0.1
|
current_willing *= 0.1
|
||||||
print(f"表情包, 当前意愿: {current_willing}")
|
logger.debug(f"表情包, 当前意愿: {current_willing}")
|
||||||
|
|
||||||
print(f"放大系数_interested_rate: {global_config.response_interested_rate_amplifier}")
|
logger.debug(f"放大系数_interested_rate: {global_config.response_interested_rate_amplifier}")
|
||||||
interested_rate *= global_config.response_interested_rate_amplifier #放大回复兴趣度
|
interested_rate *= global_config.response_interested_rate_amplifier # 放大回复兴趣度
|
||||||
if interested_rate > 0.4:
|
if interested_rate > 0.4:
|
||||||
# print(f"兴趣度: {interested_rate}, 当前意愿: {current_willing}")
|
# print(f"兴趣度: {interested_rate}, 当前意愿: {current_willing}")
|
||||||
current_willing += interested_rate-0.4
|
current_willing += interested_rate - 0.4
|
||||||
|
|
||||||
current_willing *= global_config.response_willing_amplifier #放大回复意愿
|
current_willing *= global_config.response_willing_amplifier # 放大回复意愿
|
||||||
# print(f"放大系数_willing: {global_config.response_willing_amplifier}, 当前意愿: {current_willing}")
|
# print(f"放大系数_willing: {global_config.response_willing_amplifier}, 当前意愿: {current_willing}")
|
||||||
|
|
||||||
reply_probability = max((current_willing - 0.45) * 2, 0)
|
reply_probability = max((current_willing - 0.45) * 2, 0)
|
||||||
|
|
||||||
# 检查群组权限(如果是群聊)
|
# 检查群组权限(如果是群聊)
|
||||||
if chat_stream.group_info:
|
if chat_stream.group_info:
|
||||||
if chat_stream.group_info.group_id in config.talk_frequency_down_groups:
|
if chat_stream.group_info.group_id in config.talk_frequency_down_groups:
|
||||||
reply_probability = reply_probability / global_config.down_frequency_rate
|
reply_probability = reply_probability / global_config.down_frequency_rate
|
||||||
|
|
||||||
reply_probability = min(reply_probability, 1)
|
reply_probability = min(reply_probability, 1)
|
||||||
if reply_probability < 0:
|
if reply_probability < 0:
|
||||||
reply_probability = 0
|
reply_probability = 0
|
||||||
|
|
||||||
self.chat_reply_willing[chat_id] = min(current_willing, 3.0)
|
self.chat_reply_willing[chat_id] = min(current_willing, 3.0)
|
||||||
return reply_probability
|
return reply_probability
|
||||||
|
|
||||||
def change_reply_willing_sent(self, chat_stream:ChatStream):
|
def change_reply_willing_sent(self, chat_stream: ChatStream):
|
||||||
"""开始思考后降低聊天流的回复意愿"""
|
"""开始思考后降低聊天流的回复意愿"""
|
||||||
stream = chat_stream
|
stream = chat_stream
|
||||||
if stream:
|
if stream:
|
||||||
current_willing = self.chat_reply_willing.get(stream.stream_id, 0)
|
current_willing = self.chat_reply_willing.get(stream.stream_id, 0)
|
||||||
self.chat_reply_willing[stream.stream_id] = max(0, current_willing - 2)
|
self.chat_reply_willing[stream.stream_id] = max(0, current_willing - 2)
|
||||||
|
|
||||||
def change_reply_willing_after_sent(self,chat_stream:ChatStream):
|
def change_reply_willing_after_sent(self, chat_stream: ChatStream):
|
||||||
"""发送消息后提高聊天流的回复意愿"""
|
"""发送消息后提高聊天流的回复意愿"""
|
||||||
stream = chat_stream
|
stream = chat_stream
|
||||||
if stream:
|
if stream:
|
||||||
current_willing = self.chat_reply_willing.get(stream.stream_id, 0)
|
current_willing = self.chat_reply_willing.get(stream.stream_id, 0)
|
||||||
if current_willing < 1:
|
if current_willing < 1:
|
||||||
self.chat_reply_willing[stream.stream_id] = min(1, current_willing + 0.2)
|
self.chat_reply_willing[stream.stream_id] = min(1, current_willing + 0.2)
|
||||||
|
|
||||||
async def ensure_started(self):
|
async def ensure_started(self):
|
||||||
"""确保衰减任务已启动"""
|
"""确保衰减任务已启动"""
|
||||||
if not self._started:
|
if not self._started:
|
||||||
@@ -107,5 +104,6 @@ class WillingManager:
|
|||||||
self._decay_task = asyncio.create_task(self._decay_reply_willing())
|
self._decay_task = asyncio.create_task(self._decay_reply_willing())
|
||||||
self._started = True
|
self._started = True
|
||||||
|
|
||||||
|
|
||||||
# 创建全局实例
|
# 创建全局实例
|
||||||
willing_manager = WillingManager()
|
willing_manager = WillingManager()
|
||||||
|
|||||||
@@ -349,7 +349,7 @@ class Hippocampus:
|
|||||||
def sync_memory_to_db(self):
|
def sync_memory_to_db(self):
|
||||||
"""检查并同步内存中的图结构与数据库"""
|
"""检查并同步内存中的图结构与数据库"""
|
||||||
# 获取数据库中所有节点和内存中所有节点
|
# 获取数据库中所有节点和内存中所有节点
|
||||||
db_nodes = list(self.memory_graph.db.db.graph_data.nodes.find())
|
db_nodes = list(self.memory_graph.db.graph_data.nodes.find())
|
||||||
memory_nodes = list(self.memory_graph.G.nodes(data=True))
|
memory_nodes = list(self.memory_graph.G.nodes(data=True))
|
||||||
|
|
||||||
# 转换数据库节点为字典格式,方便查找
|
# 转换数据库节点为字典格式,方便查找
|
||||||
@@ -377,7 +377,7 @@ class Hippocampus:
|
|||||||
'created_time': created_time,
|
'created_time': created_time,
|
||||||
'last_modified': last_modified
|
'last_modified': last_modified
|
||||||
}
|
}
|
||||||
self.memory_graph.db.db.graph_data.nodes.insert_one(node_data)
|
self.memory_graph.db.graph_data.nodes.insert_one(node_data)
|
||||||
else:
|
else:
|
||||||
# 获取数据库中节点的特征值
|
# 获取数据库中节点的特征值
|
||||||
db_node = db_nodes_dict[concept]
|
db_node = db_nodes_dict[concept]
|
||||||
@@ -385,7 +385,7 @@ class Hippocampus:
|
|||||||
|
|
||||||
# 如果特征值不同,则更新节点
|
# 如果特征值不同,则更新节点
|
||||||
if db_hash != memory_hash:
|
if db_hash != memory_hash:
|
||||||
self.memory_graph.db.db.graph_data.nodes.update_one(
|
self.memory_graph.db.graph_data.nodes.update_one(
|
||||||
{'concept': concept},
|
{'concept': concept},
|
||||||
{'$set': {
|
{'$set': {
|
||||||
'memory_items': memory_items,
|
'memory_items': memory_items,
|
||||||
@@ -396,7 +396,7 @@ class Hippocampus:
|
|||||||
)
|
)
|
||||||
|
|
||||||
# 处理边的信息
|
# 处理边的信息
|
||||||
db_edges = list(self.memory_graph.db.db.graph_data.edges.find())
|
db_edges = list(self.memory_graph.db.graph_data.edges.find())
|
||||||
memory_edges = list(self.memory_graph.G.edges(data=True))
|
memory_edges = list(self.memory_graph.G.edges(data=True))
|
||||||
|
|
||||||
# 创建边的哈希值字典
|
# 创建边的哈希值字典
|
||||||
@@ -428,11 +428,11 @@ class Hippocampus:
|
|||||||
'created_time': created_time,
|
'created_time': created_time,
|
||||||
'last_modified': last_modified
|
'last_modified': last_modified
|
||||||
}
|
}
|
||||||
self.memory_graph.db.db.graph_data.edges.insert_one(edge_data)
|
self.memory_graph.db.graph_data.edges.insert_one(edge_data)
|
||||||
else:
|
else:
|
||||||
# 检查边的特征值是否变化
|
# 检查边的特征值是否变化
|
||||||
if db_edge_dict[edge_key]['hash'] != edge_hash:
|
if db_edge_dict[edge_key]['hash'] != edge_hash:
|
||||||
self.memory_graph.db.db.graph_data.edges.update_one(
|
self.memory_graph.db.graph_data.edges.update_one(
|
||||||
{'source': source, 'target': target},
|
{'source': source, 'target': target},
|
||||||
{'$set': {
|
{'$set': {
|
||||||
'hash': edge_hash,
|
'hash': edge_hash,
|
||||||
@@ -451,7 +451,7 @@ class Hippocampus:
|
|||||||
self.memory_graph.G.clear()
|
self.memory_graph.G.clear()
|
||||||
|
|
||||||
# 从数据库加载所有节点
|
# 从数据库加载所有节点
|
||||||
nodes = list(self.memory_graph.db.db.graph_data.nodes.find())
|
nodes = list(self.memory_graph.db.graph_data.nodes.find())
|
||||||
for node in nodes:
|
for node in nodes:
|
||||||
concept = node['concept']
|
concept = node['concept']
|
||||||
memory_items = node.get('memory_items', [])
|
memory_items = node.get('memory_items', [])
|
||||||
@@ -468,7 +468,7 @@ class Hippocampus:
|
|||||||
if 'last_modified' not in node:
|
if 'last_modified' not in node:
|
||||||
update_data['last_modified'] = current_time
|
update_data['last_modified'] = current_time
|
||||||
|
|
||||||
self.memory_graph.db.db.graph_data.nodes.update_one(
|
self.memory_graph.db.graph_data.nodes.update_one(
|
||||||
{'concept': concept},
|
{'concept': concept},
|
||||||
{'$set': update_data}
|
{'$set': update_data}
|
||||||
)
|
)
|
||||||
@@ -485,7 +485,7 @@ class Hippocampus:
|
|||||||
last_modified=last_modified)
|
last_modified=last_modified)
|
||||||
|
|
||||||
# 从数据库加载所有边
|
# 从数据库加载所有边
|
||||||
edges = list(self.memory_graph.db.db.graph_data.edges.find())
|
edges = list(self.memory_graph.db.graph_data.edges.find())
|
||||||
for edge in edges:
|
for edge in edges:
|
||||||
source = edge['source']
|
source = edge['source']
|
||||||
target = edge['target']
|
target = edge['target']
|
||||||
@@ -501,7 +501,7 @@ class Hippocampus:
|
|||||||
if 'last_modified' not in edge:
|
if 'last_modified' not in edge:
|
||||||
update_data['last_modified'] = current_time
|
update_data['last_modified'] = current_time
|
||||||
|
|
||||||
self.memory_graph.db.db.graph_data.edges.update_one(
|
self.memory_graph.db.graph_data.edges.update_one(
|
||||||
{'source': source, 'target': target},
|
{'source': source, 'target': target},
|
||||||
{'$set': update_data}
|
{'$set': update_data}
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -56,13 +56,13 @@ def get_cloest_chat_from_db(db, length: int, timestamp: str):
|
|||||||
list: 消息记录字典列表,每个字典包含消息内容和时间信息
|
list: 消息记录字典列表,每个字典包含消息内容和时间信息
|
||||||
"""
|
"""
|
||||||
chat_records = []
|
chat_records = []
|
||||||
closest_record = db.db.messages.find_one({"time": {"$lte": timestamp}}, sort=[('time', -1)])
|
closest_record = db.messages.find_one({"time": {"$lte": timestamp}}, sort=[('time', -1)])
|
||||||
|
|
||||||
if closest_record and closest_record.get('memorized', 0) < 4:
|
if closest_record and closest_record.get('memorized', 0) < 4:
|
||||||
closest_time = closest_record['time']
|
closest_time = closest_record['time']
|
||||||
group_id = closest_record['group_id']
|
group_id = closest_record['group_id']
|
||||||
# 获取该时间戳之后的length条消息,且groupid相同
|
# 获取该时间戳之后的length条消息,且groupid相同
|
||||||
records = list(db.db.messages.find(
|
records = list(db.messages.find(
|
||||||
{"time": {"$gt": closest_time}, "group_id": group_id}
|
{"time": {"$gt": closest_time}, "group_id": group_id}
|
||||||
).sort('time', 1).limit(length))
|
).sort('time', 1).limit(length))
|
||||||
|
|
||||||
@@ -74,7 +74,7 @@ def get_cloest_chat_from_db(db, length: int, timestamp: str):
|
|||||||
return ''
|
return ''
|
||||||
|
|
||||||
# 更新memorized值
|
# 更新memorized值
|
||||||
db.db.messages.update_one(
|
db.messages.update_one(
|
||||||
{"_id": record["_id"]},
|
{"_id": record["_id"]},
|
||||||
{"$set": {"memorized": current_memorized + 1}}
|
{"$set": {"memorized": current_memorized + 1}}
|
||||||
)
|
)
|
||||||
@@ -323,7 +323,7 @@ class Hippocampus:
|
|||||||
self.memory_graph.G.clear()
|
self.memory_graph.G.clear()
|
||||||
|
|
||||||
# 从数据库加载所有节点
|
# 从数据库加载所有节点
|
||||||
nodes = self.memory_graph.db.db.graph_data.nodes.find()
|
nodes = self.memory_graph.db.graph_data.nodes.find()
|
||||||
for node in nodes:
|
for node in nodes:
|
||||||
concept = node['concept']
|
concept = node['concept']
|
||||||
memory_items = node.get('memory_items', [])
|
memory_items = node.get('memory_items', [])
|
||||||
@@ -334,7 +334,7 @@ class Hippocampus:
|
|||||||
self.memory_graph.G.add_node(concept, memory_items=memory_items)
|
self.memory_graph.G.add_node(concept, memory_items=memory_items)
|
||||||
|
|
||||||
# 从数据库加载所有边
|
# 从数据库加载所有边
|
||||||
edges = self.memory_graph.db.db.graph_data.edges.find()
|
edges = self.memory_graph.db.graph_data.edges.find()
|
||||||
for edge in edges:
|
for edge in edges:
|
||||||
source = edge['source']
|
source = edge['source']
|
||||||
target = edge['target']
|
target = edge['target']
|
||||||
@@ -371,7 +371,7 @@ class Hippocampus:
|
|||||||
使用特征值(哈希值)快速判断是否需要更新
|
使用特征值(哈希值)快速判断是否需要更新
|
||||||
"""
|
"""
|
||||||
# 获取数据库中所有节点和内存中所有节点
|
# 获取数据库中所有节点和内存中所有节点
|
||||||
db_nodes = list(self.memory_graph.db.db.graph_data.nodes.find())
|
db_nodes = list(self.memory_graph.db.graph_data.nodes.find())
|
||||||
memory_nodes = list(self.memory_graph.G.nodes(data=True))
|
memory_nodes = list(self.memory_graph.G.nodes(data=True))
|
||||||
|
|
||||||
# 转换数据库节点为字典格式,方便查找
|
# 转换数据库节点为字典格式,方便查找
|
||||||
@@ -394,7 +394,7 @@ class Hippocampus:
|
|||||||
'memory_items': memory_items,
|
'memory_items': memory_items,
|
||||||
'hash': memory_hash
|
'hash': memory_hash
|
||||||
}
|
}
|
||||||
self.memory_graph.db.db.graph_data.nodes.insert_one(node_data)
|
self.memory_graph.db.graph_data.nodes.insert_one(node_data)
|
||||||
else:
|
else:
|
||||||
# 获取数据库中节点的特征值
|
# 获取数据库中节点的特征值
|
||||||
db_node = db_nodes_dict[concept]
|
db_node = db_nodes_dict[concept]
|
||||||
@@ -403,7 +403,7 @@ class Hippocampus:
|
|||||||
# 如果特征值不同,则更新节点
|
# 如果特征值不同,则更新节点
|
||||||
if db_hash != memory_hash:
|
if db_hash != memory_hash:
|
||||||
# logger.info(f"更新节点内容: {concept}")
|
# logger.info(f"更新节点内容: {concept}")
|
||||||
self.memory_graph.db.db.graph_data.nodes.update_one(
|
self.memory_graph.db.graph_data.nodes.update_one(
|
||||||
{'concept': concept},
|
{'concept': concept},
|
||||||
{'$set': {
|
{'$set': {
|
||||||
'memory_items': memory_items,
|
'memory_items': memory_items,
|
||||||
@@ -416,10 +416,10 @@ class Hippocampus:
|
|||||||
for db_node in db_nodes:
|
for db_node in db_nodes:
|
||||||
if db_node['concept'] not in memory_concepts:
|
if db_node['concept'] not in memory_concepts:
|
||||||
# logger.info(f"删除多余节点: {db_node['concept']}")
|
# logger.info(f"删除多余节点: {db_node['concept']}")
|
||||||
self.memory_graph.db.db.graph_data.nodes.delete_one({'concept': db_node['concept']})
|
self.memory_graph.db.graph_data.nodes.delete_one({'concept': db_node['concept']})
|
||||||
|
|
||||||
# 处理边的信息
|
# 处理边的信息
|
||||||
db_edges = list(self.memory_graph.db.db.graph_data.edges.find())
|
db_edges = list(self.memory_graph.db.graph_data.edges.find())
|
||||||
memory_edges = list(self.memory_graph.G.edges())
|
memory_edges = list(self.memory_graph.G.edges())
|
||||||
|
|
||||||
# 创建边的哈希值字典
|
# 创建边的哈希值字典
|
||||||
@@ -445,12 +445,12 @@ class Hippocampus:
|
|||||||
'num': 1,
|
'num': 1,
|
||||||
'hash': edge_hash
|
'hash': edge_hash
|
||||||
}
|
}
|
||||||
self.memory_graph.db.db.graph_data.edges.insert_one(edge_data)
|
self.memory_graph.db.graph_data.edges.insert_one(edge_data)
|
||||||
else:
|
else:
|
||||||
# 检查边的特征值是否变化
|
# 检查边的特征值是否变化
|
||||||
if db_edge_dict[edge_key]['hash'] != edge_hash:
|
if db_edge_dict[edge_key]['hash'] != edge_hash:
|
||||||
logger.info(f"更新边: {source} - {target}")
|
logger.info(f"更新边: {source} - {target}")
|
||||||
self.memory_graph.db.db.graph_data.edges.update_one(
|
self.memory_graph.db.graph_data.edges.update_one(
|
||||||
{'source': source, 'target': target},
|
{'source': source, 'target': target},
|
||||||
{'$set': {'hash': edge_hash}}
|
{'$set': {'hash': edge_hash}}
|
||||||
)
|
)
|
||||||
@@ -461,7 +461,7 @@ class Hippocampus:
|
|||||||
if edge_key not in memory_edge_set:
|
if edge_key not in memory_edge_set:
|
||||||
source, target = edge_key
|
source, target = edge_key
|
||||||
logger.info(f"删除多余边: {source} - {target}")
|
logger.info(f"删除多余边: {source} - {target}")
|
||||||
self.memory_graph.db.db.graph_data.edges.delete_one({
|
self.memory_graph.db.graph_data.edges.delete_one({
|
||||||
'source': source,
|
'source': source,
|
||||||
'target': target
|
'target': target
|
||||||
})
|
})
|
||||||
@@ -487,9 +487,9 @@ class Hippocampus:
|
|||||||
topic: 要删除的节点概念
|
topic: 要删除的节点概念
|
||||||
"""
|
"""
|
||||||
# 删除节点
|
# 删除节点
|
||||||
self.memory_graph.db.db.graph_data.nodes.delete_one({'concept': topic})
|
self.memory_graph.db.graph_data.nodes.delete_one({'concept': topic})
|
||||||
# 删除所有涉及该节点的边
|
# 删除所有涉及该节点的边
|
||||||
self.memory_graph.db.db.graph_data.edges.delete_many({
|
self.memory_graph.db.graph_data.edges.delete_many({
|
||||||
'$or': [
|
'$or': [
|
||||||
{'source': topic},
|
{'source': topic},
|
||||||
{'target': topic}
|
{'target': topic}
|
||||||
|
|||||||
@@ -115,13 +115,13 @@ def get_cloest_chat_from_db(db, length: int, timestamp: str):
|
|||||||
list: 消息记录字典列表,每个字典包含消息内容和时间信息
|
list: 消息记录字典列表,每个字典包含消息内容和时间信息
|
||||||
"""
|
"""
|
||||||
chat_records = []
|
chat_records = []
|
||||||
closest_record = db.db.messages.find_one({"time": {"$lte": timestamp}}, sort=[('time', -1)])
|
closest_record = db.messages.find_one({"time": {"$lte": timestamp}}, sort=[('time', -1)])
|
||||||
|
|
||||||
if closest_record and closest_record.get('memorized', 0) < 4:
|
if closest_record and closest_record.get('memorized', 0) < 4:
|
||||||
closest_time = closest_record['time']
|
closest_time = closest_record['time']
|
||||||
group_id = closest_record['group_id']
|
group_id = closest_record['group_id']
|
||||||
# 获取该时间戳之后的length条消息,且groupid相同
|
# 获取该时间戳之后的length条消息,且groupid相同
|
||||||
records = list(db.db.messages.find(
|
records = list(db.messages.find(
|
||||||
{"time": {"$gt": closest_time}, "group_id": group_id}
|
{"time": {"$gt": closest_time}, "group_id": group_id}
|
||||||
).sort('time', 1).limit(length))
|
).sort('time', 1).limit(length))
|
||||||
|
|
||||||
@@ -133,7 +133,7 @@ def get_cloest_chat_from_db(db, length: int, timestamp: str):
|
|||||||
return ''
|
return ''
|
||||||
|
|
||||||
# 更新memorized值
|
# 更新memorized值
|
||||||
db.db.messages.update_one(
|
db.messages.update_one(
|
||||||
{"_id": record["_id"]},
|
{"_id": record["_id"]},
|
||||||
{"$set": {"memorized": current_memorized + 1}}
|
{"$set": {"memorized": current_memorized + 1}}
|
||||||
)
|
)
|
||||||
@@ -163,7 +163,7 @@ class Memory_cortex:
|
|||||||
default_time = datetime.datetime.now().timestamp()
|
default_time = datetime.datetime.now().timestamp()
|
||||||
|
|
||||||
# 从数据库加载所有节点
|
# 从数据库加载所有节点
|
||||||
nodes = self.memory_graph.db.db.graph_data.nodes.find()
|
nodes = self.memory_graph.db.graph_data.nodes.find()
|
||||||
for node in nodes:
|
for node in nodes:
|
||||||
concept = node['concept']
|
concept = node['concept']
|
||||||
memory_items = node.get('memory_items', [])
|
memory_items = node.get('memory_items', [])
|
||||||
@@ -180,7 +180,7 @@ class Memory_cortex:
|
|||||||
created_time = default_time
|
created_time = default_time
|
||||||
last_modified = default_time
|
last_modified = default_time
|
||||||
# 更新数据库中的节点
|
# 更新数据库中的节点
|
||||||
self.memory_graph.db.db.graph_data.nodes.update_one(
|
self.memory_graph.db.graph_data.nodes.update_one(
|
||||||
{'concept': concept},
|
{'concept': concept},
|
||||||
{'$set': {
|
{'$set': {
|
||||||
'created_time': created_time,
|
'created_time': created_time,
|
||||||
@@ -196,7 +196,7 @@ class Memory_cortex:
|
|||||||
last_modified=last_modified)
|
last_modified=last_modified)
|
||||||
|
|
||||||
# 从数据库加载所有边
|
# 从数据库加载所有边
|
||||||
edges = self.memory_graph.db.db.graph_data.edges.find()
|
edges = self.memory_graph.db.graph_data.edges.find()
|
||||||
for edge in edges:
|
for edge in edges:
|
||||||
source = edge['source']
|
source = edge['source']
|
||||||
target = edge['target']
|
target = edge['target']
|
||||||
@@ -212,7 +212,7 @@ class Memory_cortex:
|
|||||||
created_time = default_time
|
created_time = default_time
|
||||||
last_modified = default_time
|
last_modified = default_time
|
||||||
# 更新数据库中的边
|
# 更新数据库中的边
|
||||||
self.memory_graph.db.db.graph_data.edges.update_one(
|
self.memory_graph.db.graph_data.edges.update_one(
|
||||||
{'source': source, 'target': target},
|
{'source': source, 'target': target},
|
||||||
{'$set': {
|
{'$set': {
|
||||||
'created_time': created_time,
|
'created_time': created_time,
|
||||||
@@ -256,7 +256,7 @@ class Memory_cortex:
|
|||||||
current_time = datetime.datetime.now().timestamp()
|
current_time = datetime.datetime.now().timestamp()
|
||||||
|
|
||||||
# 获取数据库中所有节点和内存中所有节点
|
# 获取数据库中所有节点和内存中所有节点
|
||||||
db_nodes = list(self.memory_graph.db.db.graph_data.nodes.find())
|
db_nodes = list(self.memory_graph.db.graph_data.nodes.find())
|
||||||
memory_nodes = list(self.memory_graph.G.nodes(data=True))
|
memory_nodes = list(self.memory_graph.G.nodes(data=True))
|
||||||
|
|
||||||
# 转换数据库节点为字典格式,方便查找
|
# 转换数据库节点为字典格式,方便查找
|
||||||
@@ -280,7 +280,7 @@ class Memory_cortex:
|
|||||||
'created_time': data.get('created_time', current_time),
|
'created_time': data.get('created_time', current_time),
|
||||||
'last_modified': data.get('last_modified', current_time)
|
'last_modified': data.get('last_modified', current_time)
|
||||||
}
|
}
|
||||||
self.memory_graph.db.db.graph_data.nodes.insert_one(node_data)
|
self.memory_graph.db.graph_data.nodes.insert_one(node_data)
|
||||||
else:
|
else:
|
||||||
# 获取数据库中节点的特征值
|
# 获取数据库中节点的特征值
|
||||||
db_node = db_nodes_dict[concept]
|
db_node = db_nodes_dict[concept]
|
||||||
@@ -288,7 +288,7 @@ class Memory_cortex:
|
|||||||
|
|
||||||
# 如果特征值不同,则更新节点
|
# 如果特征值不同,则更新节点
|
||||||
if db_hash != memory_hash:
|
if db_hash != memory_hash:
|
||||||
self.memory_graph.db.db.graph_data.nodes.update_one(
|
self.memory_graph.db.graph_data.nodes.update_one(
|
||||||
{'concept': concept},
|
{'concept': concept},
|
||||||
{'$set': {
|
{'$set': {
|
||||||
'memory_items': memory_items,
|
'memory_items': memory_items,
|
||||||
@@ -301,10 +301,10 @@ class Memory_cortex:
|
|||||||
memory_concepts = set(node[0] for node in memory_nodes)
|
memory_concepts = set(node[0] for node in memory_nodes)
|
||||||
for db_node in db_nodes:
|
for db_node in db_nodes:
|
||||||
if db_node['concept'] not in memory_concepts:
|
if db_node['concept'] not in memory_concepts:
|
||||||
self.memory_graph.db.db.graph_data.nodes.delete_one({'concept': db_node['concept']})
|
self.memory_graph.db.graph_data.nodes.delete_one({'concept': db_node['concept']})
|
||||||
|
|
||||||
# 处理边的信息
|
# 处理边的信息
|
||||||
db_edges = list(self.memory_graph.db.db.graph_data.edges.find())
|
db_edges = list(self.memory_graph.db.graph_data.edges.find())
|
||||||
memory_edges = list(self.memory_graph.G.edges(data=True))
|
memory_edges = list(self.memory_graph.G.edges(data=True))
|
||||||
|
|
||||||
# 创建边的哈希值字典
|
# 创建边的哈希值字典
|
||||||
@@ -332,11 +332,11 @@ class Memory_cortex:
|
|||||||
'created_time': data.get('created_time', current_time),
|
'created_time': data.get('created_time', current_time),
|
||||||
'last_modified': data.get('last_modified', current_time)
|
'last_modified': data.get('last_modified', current_time)
|
||||||
}
|
}
|
||||||
self.memory_graph.db.db.graph_data.edges.insert_one(edge_data)
|
self.memory_graph.db.graph_data.edges.insert_one(edge_data)
|
||||||
else:
|
else:
|
||||||
# 检查边的特征值是否变化
|
# 检查边的特征值是否变化
|
||||||
if db_edge_dict[edge_key]['hash'] != edge_hash:
|
if db_edge_dict[edge_key]['hash'] != edge_hash:
|
||||||
self.memory_graph.db.db.graph_data.edges.update_one(
|
self.memory_graph.db.graph_data.edges.update_one(
|
||||||
{'source': source, 'target': target},
|
{'source': source, 'target': target},
|
||||||
{'$set': {
|
{'$set': {
|
||||||
'hash': edge_hash,
|
'hash': edge_hash,
|
||||||
@@ -350,7 +350,7 @@ class Memory_cortex:
|
|||||||
for edge_key in db_edge_dict:
|
for edge_key in db_edge_dict:
|
||||||
if edge_key not in memory_edge_set:
|
if edge_key not in memory_edge_set:
|
||||||
source, target = edge_key
|
source, target = edge_key
|
||||||
self.memory_graph.db.db.graph_data.edges.delete_one({
|
self.memory_graph.db.graph_data.edges.delete_one({
|
||||||
'source': source,
|
'source': source,
|
||||||
'target': target
|
'target': target
|
||||||
})
|
})
|
||||||
@@ -365,9 +365,9 @@ class Memory_cortex:
|
|||||||
topic: 要删除的节点概念
|
topic: 要删除的节点概念
|
||||||
"""
|
"""
|
||||||
# 删除节点
|
# 删除节点
|
||||||
self.memory_graph.db.db.graph_data.nodes.delete_one({'concept': topic})
|
self.memory_graph.db.graph_data.nodes.delete_one({'concept': topic})
|
||||||
# 删除所有涉及该节点的边
|
# 删除所有涉及该节点的边
|
||||||
self.memory_graph.db.db.graph_data.edges.delete_many({
|
self.memory_graph.db.graph_data.edges.delete_many({
|
||||||
'$or': [
|
'$or': [
|
||||||
{'source': topic},
|
{'source': topic},
|
||||||
{'target': topic}
|
{'target': topic}
|
||||||
|
|||||||
@@ -235,7 +235,7 @@ class LLM_request:
|
|||||||
delta_content = ""
|
delta_content = ""
|
||||||
accumulated_content += delta_content
|
accumulated_content += delta_content
|
||||||
# 检测流式输出文本是否结束
|
# 检测流式输出文本是否结束
|
||||||
finish_reason = chunk["choices"][0]["finish_reason"]
|
finish_reason = chunk["choices"][0].get("finish_reason")
|
||||||
if finish_reason == "stop":
|
if finish_reason == "stop":
|
||||||
usage = chunk.get("usage", None)
|
usage = chunk.get("usage", None)
|
||||||
if usage:
|
if usage:
|
||||||
|
|||||||
@@ -13,6 +13,8 @@ from pathlib import Path
|
|||||||
import jieba
|
import jieba
|
||||||
from pypinyin import Style, pinyin
|
from pypinyin import Style, pinyin
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
|
||||||
class ChineseTypoGenerator:
|
class ChineseTypoGenerator:
|
||||||
def __init__(self,
|
def __init__(self,
|
||||||
@@ -38,7 +40,9 @@ class ChineseTypoGenerator:
|
|||||||
self.max_freq_diff = max_freq_diff
|
self.max_freq_diff = max_freq_diff
|
||||||
|
|
||||||
# 加载数据
|
# 加载数据
|
||||||
print("正在加载汉字数据库,请稍候...")
|
# print("正在加载汉字数据库,请稍候...")
|
||||||
|
logger.info("正在加载汉字数据库,请稍候...")
|
||||||
|
|
||||||
self.pinyin_dict = self._create_pinyin_dict()
|
self.pinyin_dict = self._create_pinyin_dict()
|
||||||
self.char_frequency = self._load_or_create_char_frequency()
|
self.char_frequency = self._load_or_create_char_frequency()
|
||||||
|
|
||||||
|
|||||||
@@ -176,7 +176,7 @@ class KnowledgeLibrary:
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
current_hash = self.calculate_file_hash(file_path)
|
current_hash = self.calculate_file_hash(file_path)
|
||||||
processed_record = self.db.db.processed_files.find_one({"file_path": file_path})
|
processed_record = self.db.processed_files.find_one({"file_path": file_path})
|
||||||
|
|
||||||
if processed_record:
|
if processed_record:
|
||||||
if processed_record.get("hash") == current_hash:
|
if processed_record.get("hash") == current_hash:
|
||||||
@@ -197,14 +197,14 @@ class KnowledgeLibrary:
|
|||||||
"split_length": knowledge_length,
|
"split_length": knowledge_length,
|
||||||
"created_at": datetime.now()
|
"created_at": datetime.now()
|
||||||
}
|
}
|
||||||
self.db.db.knowledges.insert_one(knowledge)
|
self.db.knowledges.insert_one(knowledge)
|
||||||
result["chunks_processed"] += 1
|
result["chunks_processed"] += 1
|
||||||
|
|
||||||
split_by = processed_record.get("split_by", []) if processed_record else []
|
split_by = processed_record.get("split_by", []) if processed_record else []
|
||||||
if knowledge_length not in split_by:
|
if knowledge_length not in split_by:
|
||||||
split_by.append(knowledge_length)
|
split_by.append(knowledge_length)
|
||||||
|
|
||||||
self.db.db.processed_files.update_one(
|
self.db.knowledges.processed_files.update_one(
|
||||||
{"file_path": file_path},
|
{"file_path": file_path},
|
||||||
{
|
{
|
||||||
"$set": {
|
"$set": {
|
||||||
@@ -322,7 +322,7 @@ class KnowledgeLibrary:
|
|||||||
{"$project": {"content": 1, "similarity": 1, "file_path": 1}}
|
{"$project": {"content": 1, "similarity": 1, "file_path": 1}}
|
||||||
]
|
]
|
||||||
|
|
||||||
results = list(self.db.db.knowledges.aggregate(pipeline))
|
results = list(self.db.knowledges.aggregate(pipeline))
|
||||||
return results
|
return results
|
||||||
|
|
||||||
# 创建单例实例
|
# 创建单例实例
|
||||||
@@ -346,7 +346,7 @@ if __name__ == "__main__":
|
|||||||
elif choice == '2':
|
elif choice == '2':
|
||||||
confirm = input("确定要删除所有知识吗?这个操作不可撤销!(y/n): ").strip().lower()
|
confirm = input("确定要删除所有知识吗?这个操作不可撤销!(y/n): ").strip().lower()
|
||||||
if confirm == 'y':
|
if confirm == 'y':
|
||||||
knowledge_library.db.db.knowledges.delete_many({})
|
knowledge_library.db.knowledges.delete_many({})
|
||||||
console.print("[green]已清空所有知识![/green]")
|
console.print("[green]已清空所有知识![/green]")
|
||||||
continue
|
continue
|
||||||
elif choice == '1':
|
elif choice == '1':
|
||||||
|
|||||||
Reference in New Issue
Block a user