feat: update xianyu web login and Hermes optimizer

2026-05-06 20:36:19 +08:00
parent 6384156097
commit d5294a3f9b
15 changed files with 800 additions and 62 deletions
@@ -338,19 +338,27 @@ Login UX, persistent-profile lock recovery (`ProcessSingleton` / `SingletonLock`

 Web task editing, manual save-only tasks, scheduling, `maxItems`, AI-optimize fallback, API smoke tests, and stale port restart pitfalls are captured in `references/web-task-editing-scheduling-ai-optimize-2026-05.md`.

+Web AI optimize fallback and automatic Hermes analysis/notification are captured in `references/web-ai-fallback-and-auto-analysis-2026-05.md`.
+
 Web cache cleanup implementation notes are captured in `references/web-cache-cleanup-2026-05.md`.

 Web dashboard homepage/productized empty-state implementation notes are captured in `references/web-home-dashboard-polish-2026-05.md`.

 Home dashboard clickable overview/workflow cards and deep-link behavior are captured in `references/web-home-clickable-dashboard-cards-2026-05.md`.

+Portable skill/project packaging, safe archive exclusions, install script shape, and self-hosted Gitea publishing information requirements are captured in `references/portable-package-and-gitea-publishing-2026-05.md`.
+
+Public Gitea publish verification, token-scrubbed remote handling, and final reporting checklist are captured in `references/gitea-public-publish-verification-2026-05.md`.
+
 Return navigation after deep-linking from the home dashboard is captured in `references/web-home-return-navigation-2026-05.md`.

+Portable export packaging for sharing this skill/project with other Hermes instances is captured in `references/portable-skill-package-2026-05.md`.
+


 Important BOSS UX correction: do not present a direct `login.taobao.com` screenshot as the "Xianyu QR" even though Goofish uses Alibaba/Taobao unified login under the hood. Open/login from `goofish.com` first, label any redirect honestly, and prefer the visible browser login flow when BOSS wants to scan manually.

-Real QR-login screenshot flow for Feishu/mobile convenience is captured in `references/qr-login-screenshot-2026-05.md`. When BOSS says Cookie JSON is inconvenient, prefer opening the real Taobao/Goofish login page in the persistent profile, screenshotting the QR, keeping the browser alive briefly, and replying with `MEDIA:/tmp/xianyu-login-qr.png`.
+Web QR source labeling + Hermes optimize handoff notes are captured in `references/web-qr-source-and-hermes-optimize-2026-05.md`.

 ## Troubleshooting

@@ -0,0 +1,102 @@
+# Public Gitea Publish Verification Notes (2026-05)
+
+## Context
+
+BOSS changed the publishing requirement from private to public for the portable `xianyu-hunter-hermes-skill` package. The repository was created on self-hosted Gitea and pushed as a public repo.
+
+Verified public repository shape:
+
+```text
+https://gitea.chickliu.fun/Hermes/xianyu-hunter-hermes-skill
+```
+
+Package/repo contents:
+
+```text
+README.md
+install.sh
+skill/xianyu-hunter-monitor/
+project/xianyu-hunter/
+releases/xianyu-hunter-hermes-skill.zip
+releases/xianyu-hunter-hermes-skill.tar.gz
+releases/SHA256SUMS.txt
+.gitignore
+```
+
+## Recommended public publish sequence
+
+1. Create or check the Gitea repo via API, explicitly setting `private:false`.
+2. Build a clean staging worktree from the already-vetted portable package, not directly from the runtime project clone.
+3. Add `.gitignore` that excludes runtime/sensitive state even if future work occurs in the published repo:
+
+```gitignore
+node_modules/
+browser-data/
+data/tasks.json
+data/config.json
+data/notify-config.json
+reports/
+*.log
+.runtime/
+.DS_Store
+```
+
+4. Commit and push to `main`.
+5. If using an HTTPS token in the remote URL for push, immediately scrub the token after push:
+
+```bash
+git remote set-url origin 'https://gitea.example.com/OWNER/REPO.git'
+git remote -v
+```
+
+## Verification commands
+
+Use both API and unauthenticated/raw access checks where possible:
+
+```bash
+GITEA_URL='https://gitea.example.com'
+OWNER='Hermes'
+REPO='xianyu-hunter-hermes-skill'
+TOKEN='...'
+
+curl -sS -H "Authorization: token $TOKEN" \
+  -o /tmp/gitea_repo_verify.json \
+  -w 'HTTP %{http_code} size=%{size_download}\n' \
+  "$GITEA_URL/api/v1/repos/$OWNER/$REPO"
+
+python3 - <<'PY'
+import json
+d=json.load(open('/tmp/gitea_repo_verify.json'))
+print('full_name:', d.get('full_name'))
+print('html_url:', d.get('html_url'))
+print('clone_url:', d.get('clone_url'))
+print('private:', d.get('private'))
+print('default_branch:', d.get('default_branch'))
+print('empty:', d.get('empty'))
+PY
+
+curl -sS -L -o /tmp/gitea_readme.md \
+  -w 'HTTP %{http_code} size=%{size_download}\n' \
+  "$GITEA_URL/$OWNER/$REPO/raw/branch/main/README.md"
+head -5 /tmp/gitea_readme.md
+
+git ls-remote "$GITEA_URL/$OWNER/$REPO.git" HEAD refs/heads/main
+```
+
+Expected checks:
+
+- API returns `HTTP 200`.
+- `private: False` for public repos.
+- `default_branch: main`.
+- `empty: False`.
+- Raw README returns `HTTP 200`.
+- `git ls-remote` prints `HEAD` and `refs/heads/main` hashes.
+
+## Reporting to BOSS
+
+Keep the final response concise and include:
+
+- Repository URL.
+- Public/private status confirmation.
+- Main verification results.
+- Note that the local remote URL no longer contains the token if a token was used.
@@ -0,0 +1,163 @@
+# Portable Skill Package + Gitea Publishing Notes (2026-05)
+
+## What was packaged
+
+A portable archive was created for sharing the local `xianyu-hunter-monitor` workflow with other Hermes installations. The useful package shape is:
+
+```text
+xianyu-hunter-hermes-skill/
+  README.md
+  install.sh
+  skill/xianyu-hunter-monitor/
+  project/xianyu-hunter/
+```
+
+Recommended optional repo/release shape when publishing to a Git service:
+
+```text
+README.md
+install.sh
+skill/xianyu-hunter-monitor/
+project/xianyu-hunter/
+releases/
+  xianyu-hunter-hermes-skill.zip
+  xianyu-hunter-hermes-skill.tar.gz
+  SHA256SUMS.txt
+```
+
+## Exclude from the package
+
+Do **not** include local runtime or sensitive state:
+
+- `browser-data/` — Goofish/Taobao login profile and cookies.
+- `node_modules/`.
+- `.git/`.
+- `.runtime/`.
+- `reports/`.
+- `xianyu-server.log` and other logs.
+- `data/tasks.json`.
+- `data/config.json`.
+- real `data/notify-config.json` because it can contain delivery targets or tokens.
+
+It is safe to include an example notification config:
+
+```json
+{
+  "enabled": false,
+  "notifyOnlyGood": true,
+  "minScore": 75,
+  "channels": []
+}
+```
+
+## Minimal README contents
+
+The README should explain:
+
+1. Purpose: Hermes Skill + local Web/CLI 闲鱼采集助手.
+2. Default safety mode: only collect/filter/analyze; no seller contact, no purchase, no payment.
+3. Requirements: Hermes Agent, Node.js >= 18, npm, network access to `https://www.goofish.com/`.
+4. Install:
+   ```bash
+   bash install.sh
+   ```
+5. Default install destinations:
+   ```text
+   ~/.hermes/skills/social-media/xianyu-hunter-monitor/
+   ~/xianyu-hunter/
+   ```
+6. Optional project destination:
+   ```bash
+   XIANHU_INSTALL_DIR="$HOME/apps/xianyu-hunter" bash install.sh
+   ```
+7. Start dashboard:
+   ```bash
+   cd ~/xianyu-hunter
+   node tools/xianyu_ops.mjs start --port 3000
+   ```
+8. Login flow: Web UI → `账号登录 / Cookie` → get QR → scan → refresh status.
+9. Example monitoring command via CLI and natural-language Hermes prompt.
+10. Troubleshooting: login wall, captcha, port conflict, skill not found.
+
+## install.sh pattern
+
+Use a script that installs the skill and project without requiring many decisions:
+
+```bash
+#!/usr/bin/env bash
+set -euo pipefail
+ROOT="$(cd "$(dirname "$0")" && pwd)"
+HERMES_HOME="${HERMES_HOME:-$HOME/.hermes}"
+SKILL_DEST="$HERMES_HOME/skills/social-media/xianyu-hunter-monitor"
+PROJECT_DEST="${XIANHU_INSTALL_DIR:-$HOME/xianyu-hunter}"
+
+mkdir -p "$(dirname "$SKILL_DEST")"
+rm -rf "$SKILL_DEST"
+cp -R "$ROOT/skill/xianyu-hunter-monitor" "$SKILL_DEST"
+
+if [ -e "$PROJECT_DEST" ]; then
+  backup="$PROJECT_DEST.backup.$(date +%Y%m%d-%H%M%S)"
+  mv "$PROJECT_DEST" "$backup"
+fi
+cp -R "$ROOT/project/xianyu-hunter" "$PROJECT_DEST"
+cd "$PROJECT_DEST"
+npm install
+npx playwright install chromium
+```
+
+## Verification before sharing
+
+Run these checks before delivering archives:
+
+```bash
+OUT=/path/to/xianyu-hunter-hermes-skill
+for bad in \
+  'project/xianyu-hunter/browser-data' \
+  'project/xianyu-hunter/node_modules' \
+  'project/xianyu-hunter/.git' \
+  'project/xianyu-hunter/reports' \
+  'project/xianyu-hunter/data/tasks.json' \
+  'project/xianyu-hunter/data/config.json' \
+  'project/xianyu-hunter/data/notify-config.json'; do
+  test ! -e "$OUT/$bad" || { echo "BAD_INCLUDED $bad"; exit 2; }
+done
+```
+
+Also scan for common secret patterns (`sk-...`, `OPENAI_API_KEY=`, `access_token`) while excluding explanatory docs if needed.
+
+Generate both formats and hashes:
+
+```bash
+tar -czf xianyu-hunter-hermes-skill.tar.gz xianyu-hunter-hermes-skill
+zip -qr xianyu-hunter-hermes-skill.zip xianyu-hunter-hermes-skill
+shasum -a 256 xianyu-hunter-hermes-skill.tar.gz xianyu-hunter-hermes-skill.zip > SHA256SUMS.txt
+```
+
+## Publishing to self-hosted Gitea
+
+Ask the user for these details:
+
+```text
+Gitea 地址：
+仓库归属：用户或组织名
+仓库名：
+是否新建仓库：是/否
+公开/私有：public/private
+认证方式：Personal Access Token / SSH / 已配置 HTTPS 凭据
+Token（如需 API 建仓或 HTTPS push）：
+```
+
+If token is provided, prefer a short-lived token with repo create/read/write permission and remind the user they can revoke it after upload.
+
+For an existing repo, a clone URL plus working SSH/HTTPS credentials is enough. For a new repo, create via Gitea API or ask the user to create it, then push:
+
+```bash
+git init
+git add .
+git commit -m "Initial xianyu-hunter Hermes skill package"
+git branch -M main
+git remote add origin <gitea-clone-url>
+git push -u origin main
+```
+
+Avoid committing generated runtime state even if the project directory contains it locally.
@@ -0,0 +1,92 @@
+# Portable Skill Package for Other Hermes Instances (2026-05)
+
+## Trigger
+
+Use this when BOSS asks to "打包这个 skill 发给别人/其他 Hermes 使用" or wants a low-configuration portable delivery of the xianyu-hunter workflow.
+
+## Package Shape
+
+Preferred export directory:
+
+```text
+/Users/chick/.Hermes/workspace/exports/xianyu-hunter-hermes-skill/
+├── skill/xianyu-hunter-monitor/      # copied Hermes skill with references/
+├── project/xianyu-hunter/            # portable web/CLI project
+├── README.md                         # end-user installation/use guide
+└── install.sh                        # one-command installer
+```
+
+Create both archives for compatibility:
+
+```text
+xianyu-hunter-hermes-skill.zip
+xianyu-hunter-hermes-skill.tar.gz
+```
+
+## What to Include
+
+- `~/.hermes/skills/social-media/xianyu-hunter-monitor/` including `SKILL.md` and `references/`.
+- The local `xianyu-hunter` project source including `server.mjs`, `lib/`, `public/`, `tools/`, `package.json`, `package-lock.json`, docs/assets if useful.
+- A safe `data/notify-config.example.json` with disabled/empty channels.
+- Empty `data/.gitkeep` so the directory exists after unpack.
+- `README.md` with: prerequisites, one-command install, manual install, login flow, start command, sample monitoring command, Hermes usage examples, and safety boundaries.
+- `install.sh` that copies the skill into `$HERMES_HOME/skills/social-media/xianyu-hunter-monitor`, copies the project to `${XIANHU_INSTALL_DIR:-$HOME/xianyu-hunter}`, runs `npm install`, and runs `npx playwright install chromium`.
+
+## What to Exclude
+
+Always exclude runtime/sensitive state:
+
+```text
+.git/
+node_modules/
+browser-data/
+data/tasks.json
+data/config.json
+data/notify-config.json
+reports/
+.runtime/
+*.log
+```
+
+Do not include real cookies, historical product data, API keys, notification targets, generated reports, or server logs.
+
+## Verification Checklist
+
+Before sending the archive:
+
+1. Verify forbidden paths are absent from the export tree and archive.
+2. Run a secret scan for common API-key/token patterns outside documentation files.
+3. Build both `tar.gz` and `zip`.
+4. Print SHA-256 hashes and archive sizes.
+5. List an archive sample and confirm it contains `README.md`, `install.sh`, the skill directory, and project source.
+6. Send the archive with `MEDIA:/absolute/path/to/file` so Feishu uploads it as an attachment.
+
+Example verification commands:
+
+```bash
+cd /Users/chick/.Hermes/workspace/exports
+for bad in \
+  'project/xianyu-hunter/browser-data' \
+  'project/xianyu-hunter/node_modules' \
+  'project/xianyu-hunter/.git' \
+  'project/xianyu-hunter/reports' \
+  'project/xianyu-hunter/data/tasks.json' \
+  'project/xianyu-hunter/data/config.json' \
+  'project/xianyu-hunter/data/notify-config.json'; do
+  test ! -e "xianyu-hunter-hermes-skill/$bad" || { echo "BAD_INCLUDED $bad"; exit 2; }
+done
+
+tar -czf xianyu-hunter-hermes-skill.tar.gz xianyu-hunter-hermes-skill
+zip -qr xianyu-hunter-hermes-skill.zip xianyu-hunter-hermes-skill
+shasum -a 256 xianyu-hunter-hermes-skill.tar.gz xianyu-hunter-hermes-skill.zip
+```
+
+## User-Facing Response Pattern
+
+Keep the response practical and concise:
+
+- Say it is packaged.
+- State sensitive/runtime data was excluded.
+- Provide both archive attachments if available.
+- Include the minimal install/start commands.
+- Include one natural-language Hermes example.
@@ -0,0 +1,104 @@
+# Web AI Optimize Fallback and Automatic Hermes Analysis — 2026-05
+
+## Context
+
+BOSS reported two UX problems in the local xianyu-hunter Web UI:
+
+1. 新建任务/任务编辑里的“AI 润色/AI 优化”提示 `AI不可用`，容易误以为功能坏了。
+2. Hermes 分析需要手动点“执行分析”，BOSS 希望采集完成后自动分析、自动通知。
+
+## Root Cause
+
+The local fork intentionally runs in Hermes-main-control mode and does not require project-level AI API configuration. `/api/tasks/optimize` already caught `chatCompletion()` failures and returned a local fallback task, but the frontend text said `AI不可用，已用本地规则优化`, which surfaced the expected no-AI state as an error.
+
+Hermes analysis existed as `POST /api/tasks/:id/analyze`, but `TaskManager.startTask()` ended after local detail collection. No automatic analysis/report/notification was triggered from the normal run pipeline.
+
+## Implemented Shape
+
+- Rename Web copy from “AI 优化” to “任务智能润色”.
+- Keep `/api/tasks/optimize` API compatible, but treat local fallback as a successful no-configuration path.
+- Frontend fallback text: `已用本地规则优化（无需配置 AI）`.
+- Add task config flags:
+  - `autoAnalyze: true` by default.
+  - `autoNotify: true` by default.
+- Add visible task form checkboxes:
+  - `采集完成后自动执行 Hermes 分析`
+  - `分析后按通知设置自动推送`
+- After `_runPipeline()` completes and task is not stopped, call `_runAutoAnalysis()` before setting stage to `completed`.
+- `_runAutoAnalysis()`:
+  1. Skips if `autoAnalyze === false`.
+  2. Skips if no `products.fineFiltered` candidates.
+  3. Calls `analyzeTask()` with `limit=maxItems`.
+  4. Calls `renderAnalysisMarkdown()` and persists `summary.reportPath`.
+  5. Calls `sendNotifications()` unless `autoNotify === false`.
+  6. Logs success/skip/failure into task logs.
+
+## Files Touched
+
+```text
+server.mjs
+lib/task.mjs
+public/app.js
+public/index.html
+```
+
+## Verification Commands
+
+```bash
+cd /Users/chick/.Hermes/workspace/research/xianyu-hunter
+node --check server.mjs
+node --check lib/task.mjs
+node --check public/app.js
+```
+
+Restart stale port manually if needed:
+
+```bash
+lsof -nP -iTCP:3000 -sTCP:LISTEN
+kill -9 <pid>
+PORT=3000 node server.mjs
+```
+
+API fallback smoke test:
+
+```bash
+curl -sS http://127.0.0.1:3000/api/tasks/optimize \
+  -H 'Content-Type: application/json' \
+  -d '{"product":"e3-1245v3 cpu","budget":"150","currentRequirements":"价格150左右，能正常点亮，排除坏件维修配件"}'
+```
+
+Expected response can be `source: local-fallback`; this is OK. It should include `task.autoAnalyze: true` and `task.autoNotify: true`.
+
+Task create smoke test should persist config flags:
+
+```bash
+curl -sS http://127.0.0.1:3000/api/tasks \
+  -H 'Content-Type: application/json' \
+  -d '{"name":"验证自动分析任务","queries":["test"],"autoStart":false,"autoAnalyze":true,"autoNotify":true}'
+```
+
+Unit-style auto-analysis check without starting scheduler:
+
+```bash
+XIANHU_DISABLE_SCHEDULER=1 node --input-type=module <<'JS'
+import { TaskManager } from './lib/task.mjs';
+const tm = new TaskManager(() => {});
+const task = tm.createTask({ name: '自动分析单元验证', queries: ['e3-1245v3 cpu'], maxItems: 2, autoAnalyze: true, autoNotify: false, autoStart: false });
+const full = tm.getTaskFull(task.id);
+full.products.fineFiltered = [
+  { id:'unit-1', title:'Intel Xeon E3 1245 V3 CPU 正常使用', price:'¥150', detailPrice:'¥150', sellerLocation:'广东', description:'自用拆机，能正常点亮，不是维修件', href:'https://example.com/1' },
+  { id:'unit-2', title:'E3 CPU 坏件维修练手', price:'¥50', detailPrice:'¥50', sellerLocation:'广东', description:'坏件', href:'https://example.com/2' },
+];
+const logs=[];
+await tm._runAutoAnalysis(full, msg => logs.push(msg));
+console.log(tm.getTaskFull(task.id).hermesAnalysis);
+tm.deleteTask(task.id);
+JS
+```
+
+## Pitfalls
+
+- If API tests still show missing `autoAnalyze/autoNotify`, the old server process is still listening. `tools/xianyu_ops.mjs stop` may say `no-recorded-server`; use `lsof` and kill the actual PID.
+- `source: local-fallback` from `/api/tasks/optimize` is not a failure in Hermes mode.
+- Do not reintroduce project-level AI API gating for task creation or collection.
+- Do not automatically contact sellers; this automation only performs collect → local prefilter → Hermes analysis → notification.
@@ -0,0 +1,50 @@
+# Web QR login source labeling + Hermes optimize handoff (2026-05)
+
+## Trigger
+
+BOSS corrected two UX/architecture problems in the local xianyu-hunter Web dashboard:
+
+1. The login modal must not present a direct `login.taobao.com` screenshot as if it were a pure “闲鱼二维码”. Goofish/闲鱼 may legitimately redirect to Taobao/Alibaba unified login, but the UI must say this honestly and start from `goofish.com` first.
+2. “任务智能润色” should call Hermes Agent to generate the task configuration, not the project’s old internal `chatCompletion` / AI API config path.
+
+## Implemented local flow
+
+Files in `/Users/chick/.Hermes/workspace/research/xianyu-hunter`:
+
+- `lib/login.mjs`
+  - `getLoginQr()` opens `https://www.goofish.com/` first.
+  - If the official flow redirects to `login.taobao.com` / `havanaone`, the API returns `qrSource: "淘宝/阿里统一登录"` and `qrUrl`.
+  - It prefers screenshotting a QR-like element (`canvas`, QR images, qrcode containers, iframe descendants) over whole-body screenshots.
+  - It logs the true source instead of calling the screenshot “闲鱼二维码” unconditionally.
+- `public/index.html` / `public/app.js`
+  - Login panel now displays QR source and explains that Taobao/Alibaba unified login is an official redirected login chain.
+  - Help copy says it starts from 闲鱼官网 and no longer disguises Taobao screenshots as Xianyu QR.
+- `lib/hermes_optimizer.mjs`
+  - Spawns Hermes CLI (`HERMES_BIN` default `/Users/chick/.local/bin/hermes`) with a JSON-only prompt.
+  - Sanitizes returned task fields and falls back to local deterministic rules on failure.
+- `server.mjs`
+  - `/api/tasks/optimize` imports `optimizeTaskWithHermes` and `buildLocalOptimizedTask` and no longer imports/calls `chatCompletion`.
+
+## Verification commands
+
+```bash
+cd /Users/chick/.Hermes/workspace/research/xianyu-hunter
+node --check server.mjs && node --check lib/login.mjs && node --check lib/hermes_optimizer.mjs && node --check public/app.js
+
+# If old behavior still appears, kill the actual stale PID on port 3000; xianyu_ops stop may say no-recorded-server.
+lsof -tiTCP:3000 -sTCP:LISTEN | xargs kill
+node tools/xianyu_ops.mjs start --port 3000
+
+curl -sS -X POST http://127.0.0.1:3000/api/tasks/optimize \
+  -H 'Content-Type: application/json' \
+  -d '{"product":"e3-1245v3 cpu","budget":"150左右","currentRequirements":"只看CPU本体，排除主板套装/散热器/坏件/维修/求购"}' | python3 -m json.tool
+# Expect: source == "hermes" when Hermes CLI is available.
+
+curl -sS -X POST http://127.0.0.1:3000/api/login/qr \
+  -H 'Content-Type: application/json' -d '{}' | python3 -m json.tool
+# Expect: qrSource/qrUrl present. If already logged in, qrImage may be empty and logs say no scan is needed.
+```
+
+## Pitfall
+
+After code edits, port 3000 may still be served by a stale Node PID that was not recorded in `.runtime`. If `/api/tasks/optimize` still returns an old warning such as “AI 未配置：请先在控制台设置 API 密钥和模型”, kill the real listening PID with `lsof -tiTCP:3000 -sTCP:LISTEN | xargs kill` and restart.