feat(database): 完成API层、Utils层和兼容层重构 (Stage 4-6)
Stage 4: API层重构
=================
新增文件:
- api/crud.py (430行): CRUDBase泛型类，提供12个CRUD方法
* get, get_by, get_multi, create, update, delete
* count, exists, get_or_create, bulk_create, bulk_update
* 集成缓存: 自动缓存读操作，写操作清除缓存
* 集成批处理: 可选use_batch参数透明使用AdaptiveBatchScheduler
- api/query.py (461行): 高级查询构建器
* QueryBuilder: 链式调用，MongoDB风格操作符
- 操作符: __gt, __lt, __gte, __lte, __ne, __in, __nin, __like, __isnull
- 方法: filter, filter_or, order_by, limit, offset, no_cache
- 执行: all, first, count, exists, paginate
* AggregateQuery: 聚合查询
- sum, avg, max, min, group_by_count
- api/specialized.py (461行): 业务特定API
* ActionRecords: store_action_info, get_recent_actions
* Messages: get_chat_history, get_message_count, save_message
* PersonInfo: get_or_create_person, update_person_affinity
* ChatStreams: get_or_create_chat_stream, get_active_streams
* LLMUsage: record_llm_usage, get_usage_statistics
* UserRelationships: get_user_relationship, update_relationship_affinity
- 更新api/__init__.py: 导出所有API接口
Stage 5: Utils层实现
===================
新增文件:
- utils/decorators.py (320行): 数据库操作装饰器
* @retry: 自动重试失败操作，指数退避
* @timeout: 超时控制
* @cached: 自动缓存函数结果
* @measure_time: 性能测量，慢查询日志
* @transactional: 事务管理，自动提交/回滚
* @db_operation: 组合装饰器
- utils/monitoring.py (330行): 性能监控系统
* DatabaseMonitor: 单例监控器
* OperationMetrics: 操作指标 (次数、时间、错误)
* DatabaseMetrics: 全局指标
- 连接池统计
- 缓存命中率
- 批处理统计
- 预加载统计
* 便捷函数: get_monitor, record_operation, print_stats
- 更新utils/__init__.py: 导出装饰器和监控函数
Stage 6: 兼容层实现
==================
新增目录: compatibility/
- adapter.py (370行): 向后兼容适配器
* 完全兼容旧API签名: db_query, db_save, db_get, store_action_info
* 支持MongoDB风格操作符 (\, \, \)
* 内部使用新架构 (QueryBuilder + CRUDBase)
* 保持返回dict格式不变
* MODEL_MAPPING: 25个模型映射
- __init__.py: 导出兼容API
更新database/__init__.py:
- 导出核心层 (engine, session, models, migration)
- 导出优化层 (cache, preloader, batch_scheduler)
- 导出API层 (CRUD, Query, 业务API)
- 导出Utils层 (装饰器, 监控)
- 导出兼容层 (db_query, db_save等)
核心特性
========
类型安全: Generic[T]提供完整类型推断
缓存透明: 自动缓存，用户无需关心
批处理透明: 可选批处理，自动优化高频写入
链式查询: 流畅的API设计
业务封装: 常用操作封装成便捷函数
向后兼容: 兼容层保证现有代码无缝迁移
性能监控: 完整的指标收集和报告
统计数据
========
- 新增文件: 7个
- 代码行数: ~2050行
- API函数: 14个业务API + 6个装饰器
- 兼容函数: 5个 (db_query, db_save, db_get等)
下一步
======
- 更新28个文件的import语句 (从sqlalchemy_database_api迁移)
- 移动旧文件到old/目录
- 编写Stage 4-6的测试
- 集成测试验证兼容性
This commit is contained in:
Windpicker-owo
@@ -0,0 +1,126 @@
|
||||
"""数据库模块
|
||||
|
||||
重构后的数据库模块，提供：
|
||||
- 核心层：引擎、会话、模型、迁移
|
||||
- 优化层：缓存、预加载、批处理
|
||||
- API层：CRUD、查询构建器、业务API
|
||||
- Utils层：装饰器、监控
|
||||
- 兼容层：向后兼容的API
|
||||
"""
|
||||
|
||||
# ===== 核心层 =====
|
||||
from src.common.database.core import (
|
||||
Base,
|
||||
check_and_migrate_database,
|
||||
get_db_session,
|
||||
get_engine,
|
||||
get_session_factory,
|
||||
)
|
||||
|
||||
# ===== 优化层 =====
|
||||
from src.common.database.optimization import (
|
||||
AdaptiveBatchScheduler,
|
||||
DataPreloader,
|
||||
MultiLevelCache,
|
||||
get_batch_scheduler,
|
||||
get_cache,
|
||||
get_preloader,
|
||||
)
|
||||
|
||||
# ===== API层 =====
|
||||
from src.common.database.api import (
|
||||
AggregateQuery,
|
||||
CRUDBase,
|
||||
QueryBuilder,
|
||||
# ActionRecords API
|
||||
get_recent_actions,
|
||||
# ChatStreams API
|
||||
get_active_streams,
|
||||
# Messages API
|
||||
get_chat_history,
|
||||
get_message_count,
|
||||
# PersonInfo API
|
||||
get_or_create_person,
|
||||
# LLMUsage API
|
||||
get_usage_statistics,
|
||||
record_llm_usage,
|
||||
# 业务API
|
||||
save_message,
|
||||
store_action_info,
|
||||
update_person_affinity,
|
||||
)
|
||||
|
||||
# ===== Utils层 =====
|
||||
from src.common.database.utils import (
|
||||
cached,
|
||||
db_operation,
|
||||
get_monitor,
|
||||
measure_time,
|
||||
print_stats,
|
||||
record_cache_hit,
|
||||
record_cache_miss,
|
||||
record_operation,
|
||||
reset_stats,
|
||||
retry,
|
||||
timeout,
|
||||
transactional,
|
||||
)
|
||||
|
||||
# ===== 兼容层（向后兼容旧API）=====
|
||||
from src.common.database.compatibility import (
|
||||
MODEL_MAPPING,
|
||||
build_filters,
|
||||
db_get,
|
||||
db_query,
|
||||
db_save,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
# 核心层
|
||||
"Base",
|
||||
"get_engine",
|
||||
"get_session_factory",
|
||||
"get_db_session",
|
||||
"check_and_migrate_database",
|
||||
# 优化层
|
||||
"MultiLevelCache",
|
||||
"DataPreloader",
|
||||
"AdaptiveBatchScheduler",
|
||||
"get_cache",
|
||||
"get_preloader",
|
||||
"get_batch_scheduler",
|
||||
# API层 - 基础类
|
||||
"CRUDBase",
|
||||
"QueryBuilder",
|
||||
"AggregateQuery",
|
||||
# API层 - 业务API
|
||||
"store_action_info",
|
||||
"get_recent_actions",
|
||||
"get_chat_history",
|
||||
"get_message_count",
|
||||
"save_message",
|
||||
"get_or_create_person",
|
||||
"update_person_affinity",
|
||||
"get_active_streams",
|
||||
"record_llm_usage",
|
||||
"get_usage_statistics",
|
||||
# Utils层
|
||||
"retry",
|
||||
"timeout",
|
||||
"cached",
|
||||
"measure_time",
|
||||
"transactional",
|
||||
"db_operation",
|
||||
"get_monitor",
|
||||
"record_operation",
|
||||
"record_cache_hit",
|
||||
"record_cache_miss",
|
||||
"print_stats",
|
||||
"reset_stats",
|
||||
# 兼容层
|
||||
"MODEL_MAPPING",
|
||||
"build_filters",
|
||||
"db_query",
|
||||
"db_save",
|
||||
"db_get",
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user