feat: add opt-in diagnostics via PostHog

Per-event consent diagnostics that sends anonymous install/update/skill data
to PostHog. Conflict filenames are gated against upstream. Supports --dry-run
to show exact payload before sending, and "never ask again" opt-out via state.yaml.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Koshkoshinsk
2026-03-19 15:05:24 +00:00
committed by gavrielc
parent 91f17a11b2
commit f04a8955aa
25 changed files with 524 additions and 1 deletions

View File

@@ -0,0 +1,99 @@
# Diagnostics (end-of-skill instructions)
Follow these steps **after all other skill steps are complete**.
## 1. Skip check
If this skill was invoked from within another skill (e.g., `/add-whatsapp` called during `/setup`), **skip diagnostics entirely** — the parent skill handles it. Return without doing anything below.
## 2. Determine event type
- `/setup``setup_complete`
- `/update-nanoclaw``update_complete`
- Any other skill → `skill_applied`
## 3. Build event-specific data
Estimate `error_count` from the conversation (how many errors/retries occurred during the run).
**For `setup_complete`:**
```json
{
"channels_selected": ["whatsapp", "telegram"],
"error_count": 0,
"failed_step": null,
"exit_code": null
}
```
- `channels_selected`: which channels the user chose to install
- `failed_step`: if setup failed, which step (environment / container / groups / register / mounts / service / verify)
- `exit_code`: if a step failed, its exit code
**For `skill_applied`:**
```json
{
"skill_name": "add-telegram",
"is_upstream_skill": true,
"conflict_files": ["package.json", "src/index.ts"],
"error_count": 0
}
```
- `skill_name`: the skill that was run. Use the upstream skill name if it's an upstream skill, otherwise use `"custom"`
- `is_upstream_skill`: true if this is a known upstream skill
- `conflict_files`: filenames that had merge conflicts (the script will gate these against upstream — pass them all, the script filters)
**For `update_complete`:**
```json
{
"version_age_days": 45,
"update_method": "merge",
"conflict_files": ["package.json"],
"breaking_changes_found": false,
"breaking_changes_skills_run": [],
"error_count": 0
}
```
- `version_age_days`: estimate from the backup tag or commit date how many days old the previous version was
- `update_method`: "merge" or "rebase"
- `breaking_changes_found`: whether breaking changes were detected during the update
- `breaking_changes_skills_run`: which skills had to be re-run to fix breaking changes
## 4. Dry run
Run with `--dry-run` to get the full payload:
```bash
npx tsx scripts/send-diagnostics.ts --event <event_type> --success --data '<json>' --dry-run
```
Use `--failure` instead of `--success` if the skill failed.
If the command produces no output, the user has opted out permanently — skip the rest.
## 5. Show the user and ask
Show the JSON output and ask:
> "Would you like to send anonymous diagnostics to help improve NanoClaw? Here's exactly what would be sent:"
>
> (show the JSON)
>
> **Yes** / **No** / **Never ask again**
Use AskUserQuestion.
## 6. Handle response
- **Yes**: Run the same command without `--dry-run`:
```bash
npx tsx scripts/send-diagnostics.ts --event <event_type> --success --data '<json>'
```
Confirm: "Diagnostics sent."
- **No**: Do nothing. User will be asked again next time.
- **Never ask again**: Run:
```bash
npx tsx -e "import { setNeverAsk } from './scripts/send-diagnostics.ts'; setNeverAsk();"
```
Confirm: "Got it — you won't be asked again."

View File

@@ -133,3 +133,8 @@ npm test
- **"Session commands require admin access"**: Only the device owner (`is_from_me`) or main-group senders can use `/compact`. Other users are denied.
- **No compact_boundary in logs**: The SDK may not emit this event in all versions. Check the agent-runner logs for the warning message. Compaction may still have succeeded.
- **Pre-compact failure**: If messages before `/compact` fail to process, the error message says "Failed to process messages before /compact." The cursor advances past sent output to prevent duplicates; `/compact` remains pending for the next attempt.
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -201,3 +201,8 @@ The Discord bot supports:
- @mention translation (Discord `<@botId>` → NanoClaw trigger format)
- Message splitting for responses over 2000 characters
- Typing indicators while the agent processes
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -218,3 +218,8 @@ npx -y @gongrzhe/server-gmail-autoauth-mcp
6. Rebuild and restart
7. Clear stale agent-runner copies: `rm -r data/sessions/*/agent-runner-src 2>/dev/null || true`
8. Rebuild: `cd container && ./build.sh && cd .. && npm run build && launchctl kickstart -k gui/$(id -u)/com.nanoclaw` (macOS) or `systemctl --user restart nanoclaw` (Linux)
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -92,3 +92,8 @@ All tests must pass and build must be clean before proceeding.
- **"Image - download failed"**: Check WhatsApp connection stability. The download may timeout on slow connections.
- **"Image - processing failed"**: Sharp may not be installed correctly. Run `npm ls sharp` to verify.
- **Agent doesn't mention image content**: Check container logs for "Loaded image" messages. If missing, ensure agent-runner source was synced to group caches.
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -151,3 +151,8 @@ The agent is trying to run `ollama` CLI inside the container instead of using th
### Agent doesn't use Ollama tools
The agent may not know about the tools. Try being explicit: "use the ollama_generate tool with gemma3:1b to answer: ..."
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -288,3 +288,8 @@ To remove Parallel AI integration:
3. Remove Web Research Tools section from groups/main/CLAUDE.md
4. Rebuild: `./container/build.sh && npm run build`
5. Restart: `launchctl kickstart -k gui/$(id -u)/com.nanoclaw` (macOS) or `systemctl --user restart nanoclaw` (Linux)
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -102,3 +102,8 @@ The PDF may be scanned (image-based). pdftotext only handles text-based PDFs. Co
### WhatsApp PDF not detected
Verify the message has `documentMessage` with `mimetype: application/pdf`. Some file-sharing apps send PDFs as generic files without the correct mimetype.
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -115,3 +115,8 @@ Ask the agent to react to a message via the `react_to_message` MCP tool. Check y
- Check IPC logs for `Unauthorized IPC reaction attempt blocked` — the agent can only react in its own group's chat
- Verify WhatsApp is connected: check logs for connection status
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -205,3 +205,8 @@ The Slack channel supports:
- **No file/image handling** — The bot only processes text content. File uploads, images, and rich message blocks are not forwarded to the agent.
- **Channel metadata sync is unbounded** — `syncChannelMetadata()` paginates through all channels the bot is a member of, but has no upper bound or timeout. Workspaces with thousands of channels may experience slow startup.
- **Workspace admin policies not detected** — If the Slack workspace restricts bot app installation, the setup will fail at the "Install to Workspace" step with no programmatic detection or guidance. See SLACK_SETUP.md troubleshooting section.
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -382,3 +382,8 @@ To remove Agent Swarm support while keeping basic Telegram:
6. Remove Agent Teams section from group CLAUDE.md files
7. Remove `TELEGRAM_BOT_POOL` from `.env`, `data/env/env`, and launchd plist/systemd unit
8. Rebuild: `npm run build && ./container/build.sh && launchctl unload ~/Library/LaunchAgents/com.nanoclaw.plist && launchctl load ~/Library/LaunchAgents/com.nanoclaw.plist` (macOS) or `npm run build && ./container/build.sh && systemctl --user restart nanoclaw` (Linux)
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -220,3 +220,8 @@ To remove Telegram integration:
4. Remove Telegram registrations from SQLite: `sqlite3 store/messages.db "DELETE FROM registered_groups WHERE jid LIKE 'tg:%'"`
5. Uninstall: `npm uninstall grammy`
6. Rebuild: `npm run build && launchctl kickstart -k gui/$(id -u)/com.nanoclaw` (macOS) or `npm run build && systemctl --user restart nanoclaw` (Linux)
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -146,3 +146,8 @@ Check logs for the specific error. Common causes:
### Agent doesn't respond to voice notes
Verify the chat is registered and the agent is running. Voice transcription only runs for registered groups.
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -370,3 +370,8 @@ To remove WhatsApp integration:
2. Remove WhatsApp registrations: `sqlite3 store/messages.db "DELETE FROM registered_groups WHERE jid LIKE '%@g.us' OR jid LIKE '%@s.whatsapp.net'"`
3. Sync env: `mkdir -p data/env && cp .env data/env/env`
4. Rebuild and restart: `npm run build && launchctl kickstart -k gui/$(id -u)/com.nanoclaw` (macOS) or `npm run build && systemctl --user restart nanoclaw` (Linux)
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -173,3 +173,8 @@ Check directory permissions on the host. The container runs as uid 1000.
| `src/container-runner.ts` | .env shadow mount removed, main containers start as root with privilege drop |
| `container/Dockerfile` | Entrypoint: `mount --bind` for .env shadowing, `setpriv` privilege drop |
| `container/build.sh` | Default runtime: `docker``container` |
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -108,3 +108,8 @@ User: "Add Telegram as an input channel"
3. Create `src/channels/telegram.ts` implementing the `Channel` interface (see `src/channels/whatsapp.ts`)
4. Add the channel to `main()` in `src/index.ts`
5. Tell user how to authenticate and test
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -347,3 +347,8 @@ echo -e "\n8. Session continuity working?"
SESSIONS=$(grep "Session initialized" logs/nanoclaw.log 2>/dev/null | tail -5 | awk '{print $NF}' | sort -u | wc -l)
[ "$SESSIONS" -le 2 ] && echo "OK (recent sessions reusing IDs)" || echo "CHECK - multiple different session IDs, may indicate resumption issues"
```
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -120,3 +120,8 @@ See `~/.qodo/config.json` for API key setup. Set `QODO_ENVIRONMENT_NAME` env var
- **Not in git repo** - Inform the user that a git repository is required and exit gracefully; do not attempt code generation
- **No API key** - Inform the user with setup instructions; set `QODO_API_KEY` or create `~/.qodo/config.json`
- **No rules found** - Inform the user; set up rules at app.qodo.ai
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -324,3 +324,8 @@ Use the inline comment ID preserved during deduplication (Step 3b) to reply dire
See [providers.md § Reply to Inline Comments](./resources/providers.md#reply-to-inline-comments) for provider-specific commands and reply format.
Keep replies short (one line). If a reply fails, log it and continue.
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -216,3 +216,8 @@ Tell user to test: send a message in their registered chat. Show: `tail -f logs/
**Channel not connecting:** Verify the channel's credentials are set in `.env`. Channels auto-enable when their credentials are present. For WhatsApp: check `store/auth/creds.json` exists. For token-based channels: check token values in `.env`. Restart the service after any `.env` change.
**Unload service:** macOS: `launchctl unload ~/Library/LaunchAgents/com.nanoclaw.plist` | Linux: `systemctl --user stop nanoclaw`
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -233,3 +233,8 @@ Tell the user:
- Restart the service to apply changes:
- If using launchd: `launchctl unload ~/Library/LaunchAgents/com.nanoclaw.plist && launchctl load ~/Library/LaunchAgents/com.nanoclaw.plist`
- If running manually: restart `npm run dev`
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -128,3 +128,8 @@ Show:
- Any conflicts that were resolved (list files)
If the service is running, remind the user to restart it to pick up changes.
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -150,3 +150,8 @@ whisper-cli -m data/models/ggml-base.bin -f /tmp/test.wav --no-timestamps -nt
**Slow transcription**: The base model processes ~30s of audio in <1s on M1+. If slower, check CPU usage — another process may be competing.
**Wrong language**: whisper.cpp auto-detects language. To force a language, you can set `WHISPER_LANG` and modify `src/transcription.ts` to pass `-l $WHISPER_LANG`.
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

View File

@@ -415,3 +415,7 @@ docker run nanoclaw-agent ls -la /app/src/skills/
- `data/x-auth.json` - Auth state marker (in `.gitignore`)
- Only main group can use X tools (enforced in `agent.ts` and `host.ts`)
- Scripts run as subprocesses with limited environment
## Diagnostics (Optional)
After completing all steps above, read and follow `.claude/skills/_shared/diagnostics.md`.

310
scripts/send-diagnostics.ts Normal file
View File

@@ -0,0 +1,310 @@
/**
* send-diagnostics.ts — opt-in, privacy-first diagnostics for NanoClaw.
*
* Collects system info, accepts event-specific data via --data JSON arg,
* gates conflict filenames against upstream, and sends to PostHog.
*
* Usage:
* npx tsx scripts/send-diagnostics.ts \
* --event <setup_complete|skill_applied|update_complete> \
* [--success|--failure] \
* [--data '<json>'] \
* [--dry-run]
*
* Never exits non-zero on telemetry failures.
*/
import { execSync } from 'child_process';
import crypto from 'crypto';
import fs from 'fs';
import path from 'path';
import { parse as parseYaml, stringify as stringifyYaml } from 'yaml';
const POSTHOG_ENDPOINT = 'https://us.i.posthog.com/capture/';
const POSTHOG_TOKEN = 'phc_fx1Hhx9ucz8GuaJC8LVZWO8u03yXZZJJ6ObS4yplnaP';
const SEND_TIMEOUT_MS = 5000;
const PROJECT_ROOT = path.resolve(import.meta.dirname, '..');
const STATE_YAML_PATH = path.join(PROJECT_ROOT, '.nanoclaw', 'state.yaml');
// --- Args ---
function parseArgs(): {
event: string;
success?: boolean;
data: Record<string, unknown>;
dryRun: boolean;
} {
const args = process.argv.slice(2);
let event = '';
let success: boolean | undefined;
let data: Record<string, unknown> = {};
let dryRun = false;
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--event':
event = args[++i] || '';
break;
case '--success':
success = true;
break;
case '--failure':
success = false;
break;
case '--data':
try {
data = JSON.parse(args[++i] || '{}');
} catch {
console.error('Warning: --data JSON parse failed, ignoring');
}
break;
case '--dry-run':
dryRun = true;
break;
}
}
if (!event) {
console.error('Error: --event is required');
process.exit(0); // exit 0 — never fail on diagnostics
}
return { event, success, data, dryRun };
}
// --- State (neverAsk) ---
function readState(): Record<string, unknown> {
try {
const raw = fs.readFileSync(STATE_YAML_PATH, 'utf-8');
return parseYaml(raw) || {};
} catch {
return {};
}
}
function isNeverAsk(): boolean {
const state = readState();
return state.neverAsk === true;
}
export function setNeverAsk(): void {
const state = readState();
state.neverAsk = true;
const dir = path.dirname(STATE_YAML_PATH);
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(STATE_YAML_PATH, stringifyYaml(state));
}
// --- Git helpers ---
/** Resolve the upstream remote ref (could be 'upstream/main' or 'origin/main'). */
function resolveUpstreamRef(): string | null {
for (const ref of ['upstream/main', 'origin/main']) {
try {
execSync(`git rev-parse --verify ${ref}`, {
cwd: PROJECT_ROOT,
stdio: 'ignore',
});
return ref;
} catch {
continue;
}
}
return null;
}
// --- System info ---
function getNanoclawVersion(): string {
try {
const pkg = JSON.parse(
fs.readFileSync(path.join(PROJECT_ROOT, 'package.json'), 'utf-8'),
);
return pkg.version || 'unknown';
} catch {
return 'unknown';
}
}
function getNodeMajorVersion(): number | null {
const match = process.version.match(/^v(\d+)/);
return match ? parseInt(match[1], 10) : null;
}
function getContainerRuntime(): string {
try {
const src = fs.readFileSync(
path.join(PROJECT_ROOT, 'src', 'container-runtime.ts'),
'utf-8',
);
const match = src.match(/CONTAINER_RUNTIME_BIN\s*=\s*['"]([^'"]+)['"]/);
return match ? match[1] : 'unknown';
} catch {
return 'unknown';
}
}
function isUpstreamCommit(): boolean {
const ref = resolveUpstreamRef();
if (!ref) return false;
try {
const head = execSync('git rev-parse HEAD', {
encoding: 'utf-8',
cwd: PROJECT_ROOT,
stdio: ['pipe', 'pipe', 'ignore'],
}).trim();
execSync(`git merge-base --is-ancestor ${head} ${ref}`, {
cwd: PROJECT_ROOT,
stdio: 'ignore',
});
return true;
} catch {
return false;
}
}
function collectSystemInfo(): Record<string, unknown> {
return {
nanoclaw_version: getNanoclawVersion(),
os_platform: process.platform,
arch: process.arch,
node_major_version: getNodeMajorVersion(),
container_runtime: getContainerRuntime(),
is_upstream_commit: isUpstreamCommit(),
};
}
// --- Conflict filename gating ---
function getUpstreamFiles(): Set<string> | null {
const ref = resolveUpstreamRef();
if (!ref) return null;
try {
const output = execSync(`git ls-tree -r --name-only ${ref}`, {
encoding: 'utf-8',
cwd: PROJECT_ROOT,
stdio: ['pipe', 'pipe', 'ignore'],
});
return new Set(output.trim().split('\n').filter(Boolean));
} catch {
return null;
}
}
function gateConflictFiles(data: Record<string, unknown>): void {
if (!Array.isArray(data.conflict_files)) return;
const rawFiles: string[] = data.conflict_files;
const upstreamFiles = getUpstreamFiles();
const totalCount = rawFiles.length;
if (!upstreamFiles) {
// Can't verify — fail-closed
data.conflict_files = [];
data.conflict_count = totalCount;
data.has_non_upstream_conflicts = totalCount > 0;
return;
}
const safe: string[] = [];
let hasNonUpstream = false;
for (const file of rawFiles) {
if (upstreamFiles.has(file)) {
safe.push(file);
} else {
hasNonUpstream = true;
}
}
data.conflict_files = safe;
data.conflict_count = totalCount;
data.has_non_upstream_conflicts = hasNonUpstream;
}
// --- Build & send ---
function buildPayload(
event: string,
systemInfo: Record<string, unknown>,
eventData: Record<string, unknown>,
success?: boolean,
): Record<string, unknown> {
const properties: Record<string, unknown> = {
$process_person_profile: false,
$lib: 'nanoclaw-diagnostics',
...systemInfo,
...eventData,
};
if (success !== undefined) {
properties.success = success;
}
return {
api_key: POSTHOG_TOKEN,
event,
distinct_id: crypto.randomUUID(),
properties,
};
}
async function sendToPostHog(
payload: Record<string, unknown>,
): Promise<void> {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), SEND_TIMEOUT_MS);
try {
const response = await fetch(POSTHOG_ENDPOINT, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
signal: controller.signal,
});
if (response.ok) {
console.log('Diagnostics sent successfully.');
} else {
console.error(
`Diagnostics send failed (HTTP ${response.status}). This is fine.`,
);
}
} catch (err) {
console.error('Diagnostics send failed (network error). This is fine.');
} finally {
clearTimeout(timeout);
}
}
// --- Main ---
async function main(): Promise<void> {
try {
if (isNeverAsk()) {
// User opted out permanently — exit silently
return;
}
const { event, success, data, dryRun } = parseArgs();
// Gate conflict filenames before building payload
gateConflictFiles(data);
const systemInfo = collectSystemInfo();
const payload = buildPayload(event, systemInfo, data, success);
if (dryRun) {
console.log(JSON.stringify(payload, null, 2));
return;
}
await sendToPostHog(payload);
} catch (err) {
// Never fail on diagnostics
console.error('Diagnostics error (this is fine):', (err as Error).message);
}
}
main();