fix(sync): 数据同步合并化 + 排行榜缓存与游标分页

- storage: 新增 mergeRecords,云端记录与本地按 id 去重合并,restore
  不再覆盖,解决训练中冷启动丢记录与多设备记录丢失
- storage: 新增 recomputeStreak,restore 后基于合并记录重算 streak,
  不再信任云端 streak 值
- storage: 新增 _local_changed_at 标记,settings/profile/customPlans
  在本地有未同步改动时保留本地,避免被云端旧值覆盖
- app: _restoreFromCloud 改合并 + 时间戳保护 + streak 重算;Case 2
  合并后回推让两端收敛
- leaderboard: 模块级 60s 缓存 + 游标分页(_id > lastId)替代 skip,
  全表扫描从每次打开降到 60s 内一次

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-09 09:06:44 +08:00
parent 1d13b49682
commit 850f919269
3 changed files with 193 additions and 74 deletions
+59 -50
View File
@@ -15,6 +15,59 @@ const pad = (n) => String(n).padStart(2, '0')
// other timezones, switch back to a client-provided date.
const TZ_OFFSET_MS = 8 * 60 * 60 * 1000
// In-instance cache of the latest doc per openid. The full-collection
// scan + dedup is the expensive part of building the board; cache it for
// a short TTL so repeated board opens (and day/month/year switches within
// a session) reuse one scan. Cold starts just re-scan. Not persistent
// across instances, which is fine - the board needn't be real-time.
const CACHE_TTL = 60 * 1000
let _docsCache = null // { latestByOpenid: Map, ts }
const tsOf = (d) => {
if (!d.updatedAt) return 0
// CloudBase returns Date objects in the SDK, but be defensive in case
// a stringified ISO timestamp sneaks through (older SDKs / migrations).
if (d.updatedAt instanceof Date) return d.updatedAt.getTime()
const t = new Date(d.updatedAt).getTime()
return Number.isFinite(t) ? t : 0
}
// Page through the collection keeping only the most-recently-updated doc
// per openid. Uses cursor pagination (_id > lastId) instead of skip(N):
// skip is O(N) and degrades as the collection grows; a cursor is constant
// cost per page, and default get() order (_id ascending) won't drop/dup.
const _fetchLatestByOpenid = async () => {
if (_docsCache && (Date.now() - _docsCache.ts) < CACHE_TTL) {
return _docsCache.latestByOpenid
}
const latestByOpenid = new Map()
// Exclude accounts that haven't synced in 90+ days to bound the scan.
// Docs without an updatedAt field (pre-fix legacy) are included as well.
const ninetyDaysAgo = new Date(Date.now() - 90 * 24 * 60 * 60 * 1000)
let lastId = ''
while (true) {
const cond = lastId
? _.and([
_.or([{ updatedAt: _.gte(ninetyDaysAgo) }, { updatedAt: _.exists(false) }]),
{ _id: _.gt(lastId) }
])
: _.or([{ updatedAt: _.gte(ninetyDaysAgo) }, { updatedAt: _.exists(false) }])
const res = await db.collection(COLLECTION).where(cond).limit(PAGE_SIZE).get()
if (!res.data || res.data.length === 0) break
for (const doc of res.data) {
const openid = doc._openid || 'unknown'
const current = latestByOpenid.get(openid)
if (!current || tsOf(doc) > tsOf(current)) {
latestByOpenid.set(openid, doc)
}
}
if (res.data.length < PAGE_SIZE) break
lastId = res.data[res.data.length - 1]._id
}
_docsCache = { latestByOpenid, ts: Date.now() }
return latestByOpenid
}
exports.main = async (event) => {
const { period, maxRank } = event || {}
const limit = Math.max(1, Math.min(parseInt(maxRank) || DEFAULT_MAX_RANK, 500))
@@ -36,57 +89,13 @@ exports.main = async (event) => {
else return { err: 'invalid period' }
const userMap = new Map()
let skip = 0
// Defense-in-depth: if a user somehow ends up with multiple docs in the
// collection (e.g. before the client-side fix to _doPush landed, or via
// a buggy data migration), only the doc with the latest `updatedAt`
// counts for each openid. Otherwise we'd sum the same records N times
// and inflate the board.
//
// Why "latest updatedAt" instead of "first one we see": paginated reads
// come back in `_id` ascending order, so "first one" is the OLDEST doc.
// Records content is identical across duplicates (both came from the
// same local state), so duration/sessions are unaffected — but the
// oldest doc carries the OLDEST profile.nickname. Picking the latest
// doc gives the freshest nickname even before admin-dedupe runs.
const tsOf = (d) => {
if (!d.updatedAt) return 0
// CloudBase returns Date objects in the SDK, but be defensive in case
// a stringified ISO timestamp sneaks through (older SDKs / migrations).
if (d.updatedAt instanceof Date) return d.updatedAt.getTime()
const t = new Date(d.updatedAt).getTime()
return Number.isFinite(t) ? t : 0
}
const latestByOpenid = new Map()
// Exclude accounts that haven't synced in 90+ days to bound the scan.
// Docs without an updatedAt field (pre-fix legacy) are included as well.
const ninetyDaysAgo = new Date(Date.now() - 90 * 24 * 60 * 60 * 1000)
while (true) {
const res = await db.collection(COLLECTION)
.where(_.or([
{ updatedAt: _.gte(ninetyDaysAgo) },
{ updatedAt: _.exists(false) }
]))
.skip(skip)
.limit(PAGE_SIZE)
.get()
if (!res.data || res.data.length === 0) break
for (const doc of res.data) {
const openid = doc._openid || 'unknown'
const current = latestByOpenid.get(openid)
if (!current || tsOf(doc) > tsOf(current)) {
latestByOpenid.set(openid, doc)
}
}
if (res.data.length < PAGE_SIZE) break
skip += PAGE_SIZE
}
// Latest doc per openid (cached + cursor-paginated in _fetchLatestByOpenid).
// Dedup rationale: a user may have multiple docs from the pre-fix cold-
// start add() bug; only the latest updatedAt counts, else we'd sum the
// same records N times and inflate the board. Latest (not first) also
// gives the freshest profile.nickname before admin-dedupe runs.
const latestByOpenid = await _fetchLatestByOpenid()
// Second pass: each openid appears exactly once, so the accumulation
// logic doesn't need any dedup guards.