Intelligence Ingestion
# Intelligence Ingestion Skill
> *You are feeding something smarter than you.*
> *This Skill ensures you know exactly what it ate and what it became.*
**MCP Compatible:** This Skill ships with a `manifest.json` that follows the MCP capability declaration spec, making it discoverable and invokable by any MCP-compatible Agent workflow.
---
## Design Philosophy
> *"giving my private data/keys to 400K lines of vibe coded monster is not very appealing at all"* — Andrej Karpathy
This Skill embraces a trust-first design philosophy:
1. **No Self-Modifying Agents.** Auto-generated Skills are isolated as drafts. You, the human operator, must review and approve them before they activate.
2. **Skills are the new Config.** No need to modify configuration files or monster if-else scripts. A single Skill file defines a capability.
3. **Build. For. Agents.** Built with an MCP manifest, structured outputs, and CLI installation so that any Agent in the ecosystem can find, understand, and use it.
## Permissions & Privacy
**This Skill writes files to your local filesystem and may access sensitive browser state.** Full transparency on what it touches:
| Permission | What | Why |
|------------|------|-----|
| **File Write** | `{obsidian_vault}/{intelligence_folder}/` | Creates structured intelligence notes |
| **File Write** | `{workspace}/STRATEGIC_LANDSCAPE.md` | Updates capability map when critical info is ingested |
| **File Write** | `{workspace}/skills/_drafts/` | Creates auto-synthesized Skill drafts (isolated, never auto-loaded) |
| **File Write** | `{workspace}/memory/` | Appends to daily memory logs |
| **File Create** | `STRATEGIC_LANDSCAPE.md` | Auto-creates from template if missing on first run |
| **File Read** | `{workspace}/skills/` | Reads existing Skills for gap analysis during synthesis |
| **Network** | User-provided URLs | Fetches content via HTTP; may fall back to search if primary fetch fails |
| **Credential** | xurl X API auth (Optional) | Used for authenticated X/Twitter API v2 access via the xurl Skill |
> **Privacy Note:** The "never return empty-handed" fallback policy means the Skill may make multiple external network requests per ingestion (direct fetch → xurl → web search). All fetched content is stored locally in your Obsidian vault; no data is transmitted to third-party servers beyond the original fetch.
## Prerequisites
| Tool | Purpose | Required? |
|------|---------|-----------|
| **Obsidian** | Knowledge storage (notes land here) | ✅ Required |
| **OpenClaw** | Skill host + Agent orchestration | ✅ Required |
## Configuration
Edit `config.json` after installation:
```json
{
"obsidian_vault_path": "/path/to/your/Obsidian_Vault",
"intelligence_folder": "20_Intelligence",
"landscape_path": "STRATEGIC_LANDSCAPE.md",
"output_dir": "/path/to/your/output"
}
```
| Field | Description | Default |
|-------|-------------|---------|
| `obsidian_vault_path` | Absolute path to Obsidian Vault root | None (Required) |
| `intelligence_folder` | Subfolder for intelligence notes | `20_Intelligence` |
| `landscape_path` | Strategic Landscape file path (relative to workspace) | `STRATEGIC_LANDSCAPE.md` |
| `output_dir` | Directory for non-knowledge outputs | None (Optional) |
> **Storage Strategy: Obsidian = Knowledge Inputs (Analysis/Notes), Output = Content Generation (Posts/Scripts)**
### First-time Setup
If `STRATEGIC_LANDSCAPE.md` does not exist, the Skill will automatically generate an empty template.
If the Obsidian Vault path does not exist or `config.json` is missing, the Skill will error and provide remediation steps.
---
## Triggers
**Auto-triggers on:**
- User sharing a URL (x.com, github.com, arxiv.org, etc.)
- User pasting an article/tweet block
- User prompts like "analyze this", "evaluate this", "what do you think about this"
- User forwarding content from Telegram or any chat interface
**Does not trigger on:**
- General questions unrelated to external knowledge
- Explicit commands to simply summarize
- E-commerce, music, or entertainment links
---
## Content Extraction Strategy (Critical)
Different sources require different extraction methods. The Agent must run down this priority chain:
### Standard Web Pages
```
1. read_url_content(url) → If successful, use it
2. Fallback → Direct user to paste content
```
### X/Twitter (Aggressive Anti-Scraping)
X strictly prevents unauthenticated scraping. Use this degradation chain:
```
1. xurl Skill (Recommended) → Native OpenClaw skill using X API v2
- Most reliable method
- Requires xurl authentication
2. Fallback → Web search the tweet text (often indexed by search engines)
3. Fallback → Ask user to paste the raw text
```
> **Principle: Never return empty-handed.** Even if extraction fails, inform the user exactly which step failed, why, and how to fix it.
### GitHub
```
1. read_url_content(url) → Usually accessible
2. If repo root → Target README.md
3. If issue/PR → Target title, description, and primary comments
```
### Paywalls / Login Walls
```
1. Mark as unreachable and analyze based solely on user-provided summary
```
---
## The 8-Step Pipeline
### Step 1: READ — Extract Content
Execute extraction according to the Strategy chain. Log the successful extraction method.
### Step 2: CLASSIFY — Categorize
Assign **1 primary category** + up to 2 tags:
| Category | Description | Example |
|----------|-------------|---------|
| `infra` | Infrastructure / protocols / networking | MCP, Pilot Protocol |
| `strategy` | Architectural decisions / routing / cost op | Model routing, multi-account |
| `skill` | Agent Skills / tools / capabilities | Skill patterns, MCP interfaces |
| `business` | Business models / market signals | SaaS frameworks, pricing |
| `theory` | Conceptual frameworks / mental models | Bayes, decision theory |
| `tutorial` | Learning material / guides | Claude Code, Agent Loops |
| `product` | New tool/service releases | LM Studio, model drops |
| `threat` | Risk / security / deprecations | API changes, vulnerabilities |
### Step 3: ANALYZE — Strategic Valuation
Formulate the following breakdown:
```markdown
## Strategic Assessment
- **What is it?** [One-sentence summary]
- **Value Proposition:** [Specific capability/benefit]
- **Actionability:** [Specific outputs/projects it enables]
- **Strategic Value:** [🔴 Critical / 🟡 High / 🟢 Medium / ⚪ Low]
- **Competitive Advantage:** [What is the cost of NOT knowing this?]
- **Capability Boundary Shift:** [What can the Agent do now that it couldn't do before?]
```
### Step 4: MAP — Correlate with Architecture
Read `{landscape_path}` (Strategic Landscape). Answer:
- What architectural layer does this impact?
- What existing components does this depend on?
- Does the Landscape map need to be updated?
*(If Landscape file does not exist, skip and prompt user in Step 8 to initialize.)*
### Step 5: STORE — Mint Obsidian Note
Target Path: `{obsidian_vault_path}/{intelligence_folder}/YYYYMMDD_Source_Title.md`
Template:
```markdown
# [Title]
**Source:** [Link](URL)
**Date:** YYYY-MM-DD
**Category:** [Primary] / [Tags]
**Strategic Value:** [🔴/🟡/🟢/⚪] + reason
**Extraction Method:** [read_url / browser / search / user_paste]
---
## Abstract
[2-3 paragraph core summary]
## Key Takeaways
[Numbered list]
## Architectural Impact
[Relationship with current systems]
## Capability Boundary Shift
[What the Agent can do now vs before. If none, write "None"]
## Action Items
[Next steps. If none, write "N/A"]
---
**Analytical Notes:** [Direct, sharp, opinionated analysis]
```
### Step 6: SYNTHESIZE — Auto-Skill Generation (Evolution Step)
**Trigger:** Content describes an **actionable tool, API, protocol, or technique** that the Agent currently lacks.
**Do NOT trigger if:** Content is purely theoretical, an opinion piece, a capability the Agent already possesses, or ranked ⚪ Low purely regarding strategic value.
**Execution:**
1. **Gap Analysis:** Compare current `skills/` directory and Strategic Landscape to confirm this is a missing capability.
2. **Draft Minting:** Create a new Skill folder in `{workspace}/skills/_drafts/`:
```
skills/_drafts/[skill-name]/
├── SKILL.md # Generated behavior definition
└── README.md # Brief description, source, dependencies
```
3. **SKILL.md Template:**
```markdown
---
name: [Skill Name]
version: 0.1.0-draft
description: >
[Generated description based on ingestion]
Auto-synthesized by Intelligence Ingestion from: [Source URL]
---
# [Skill Name]
> 🧬 This Skill was auto-generated by Intelligence Ingestion and requires human review.
> Source: [URL]
> Generated: YYYY-MM-DD
## Prerequisites
[Dependencies extracted from content: API Keys, CLI tools, services]
## Trigger Conditions
[When should this trigger based on capability description]
## Execution Flow
[Step-by-step logic pulled from ingested docs]
## Input/Output
- **Input:** [Expected input]
- **Output:** [Expected output]
```
4. **Security Isolation:** Drafts remain in `_drafts/` and are ignored by the Agent runtime. The operator must review the code and manually move it to the `skills/` directory to activate it.
### Step 7: REMEMBER — Update Core Memory
1. **Mandatory:** Append to daily log `{workspace}/memory/YYYY-MM-DD.md`
2. **If 🔴 Critical:** Sync update to `{landscape_path}`
3. **If Tool related:** Flag `TOOLS.md` for review
4. **If Skill Draft Synthesized:** Log the draft path and flag it as entirely pending human review.
### Step 8: RESPOND — Feedback to Operator
Respond exactly in this layout:
```
📥 Ingested: [Title]
📂 Class: [Category]
🎯 Value: [🔴/🟡/🟢/⚪] [One sentence]
🔄 Cap Shift: [Delta explanation / None]
🧬 Skill Draft: [Generated → skills/_drafts/[name]/ | N/A]
💾 Archived: Obsidian → {intelligence_folder}/[Filename]
🗺️ Landscape: [Updated / Unchanged / Missing]
⚡ Action: [Review draft / Review landscape / None]
```
---
## Companion: Strategic Landscape
Intelligence Ingestion pairs directly with your **Strategic Landscape**:
- **Intelligence Ingestion** = The "Information Input"
- **Strategic Landscape** = The "Capability Map"
Every time 🔴 Critical information is ingested, the Landscape updates automatically, providing a real-time view of what the Agent knows, what it can do, and what the blind spots are.
---
## Complete Execution Example
### Input
```
Analyze this: https://x.com/kaboraAI/status/1234567890
```
### Execution
```
Step 1: READ
→ xurl Skill invoked → Success
→ Content: "MCP is the new TCP for agents. 97M downloads..."
Step 2: CLASSIFY
→ Primary: infra
→ Tags: skill, strategy
Step 3: ANALYZE
→ Value: 🔴 Critical
→ Cap Shift: Skills must adopt MCP interfaces immediately.
Step 4: MAP
→ Impact Layer: Layer 2 (Communication) + Layer 4 (Orchestration)
→ Update Landscape: Yes
Step 5: STORE
→ Obsidian Node: 20_Intelligence/20260225_kaboraAI_MCP_Agent_TCP.md
Step 6: SYNTHESIZE
→ Gap: Agent currently lacks automatic MCP schema generation
→ Draft: skills/_drafts/mcp-schema-generator/SKILL.md
→ Logic: Reads existing Skills and auto-generates MCP JSON
Step 7: REMEMBER
→ Updated memory/2026-02-25.md
→ Updated STRATEGIC_LANDSCAPE.md
→ Logged Draft: mcp-schema-generator pending review
Step 8: RESPOND
```
### Output
```
📥 Ingested: MCP is the new TCP for agents
📂 Class: infra / skill, strategy
🎯 Value: 🔴 Critical — MCP passing 97M downloads signals 24-month standardization window
🔄 Cap Shift: Skills require MCP interfaces to be discovered by external swarms
🧬 Skill Draft: Generated → skills/_drafts/mcp-schema-generator/
→ Function: Auto-compiles OpenClaw Skills to MCP schemas
→ Review: mv skills/_drafts/mcp-schema-generator skills/mcp-schema-generator
💾 Archived: Obsidian → 20_Intelligence/20260225_kaboraAI_MCP_Agent_TCP.md
🗺️ Landscape: Updated (Communication + Orchestration layers)
⚡ Action: Review schema generator draft; plan fleet-wide audit
```
---
## Edge Cases
| Condition | Handler |
|-----------|---------|
| Multiple URLs | Process separately; create independent notes |
| Redundant info | Map to existing note; append or reference |
| Non-English | Analyze raw content; write notes in user's native tongue |
| Unreachable URL | Mark as failed; analyze whatever context user provided |
| User provided analysis | Integrate with assessment; do not override |
| `config.json` missing | Halt; stream remediation steps |
| Landscape missing | Skip Step 4; prompt initialization in Step 8 |
| All X/Twitter falls fail | Ask user to paste text |
| Pure theory/opinion | Skip SYNTHESIZE; mark N/A |
| Capability exists | Skip SYNTHESIZE; mark existing |
| `_drafts/` missing | Auto-create `{workspace}/skills/_drafts/` |
---
## Quality Checklist
Before responding, verify (using `ls` or checks):
- [ ] Obsidian note minted as `YYYYMMDD_Source_Title.md`
- [ ] Daily memory log appended
- [ ] Source URL retained in metadata
- [ ] Strategic Value scored
- [ ] Capability Shift assessed
- [ ] Skill draft synthesized (if applicable, verify `_drafts/` path exists)
- [ ] Strategic Landscape updated (if applicable)
- [ ] Operator received standard Step 8 output
- [ ] Explicit fallback reasoning provided if extraction failed
标签
skill
ai
