;
; agent-tool-learning.scm
;
; How an OpenCog agent could:
; 1. Learn from corpus
; 2. Discover knowledge limits
; 3. Learn tool structure (including computers)
; 4. Eventually use search engines to expand knowledge
;
; The key insight: SAME recursive structure discovery applies to TOOLS
;
(use-modules (srfi srfi-1))
(use-modules (ice-9 format))
(display "═══════════════════════════════════════════════════════════════════\n")
(display " AGENT TOOL LEARNING: From Corpus to Search Engine \n")
(display "═══════════════════════════════════════════════════════════════════\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; THE BOOTSTRAPPING PROBLEM
;; ═══════════════════════════════════════════════════════════════════
(display "THE BOOTSTRAPPING PROBLEM\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " The agent starts knowing NOTHING about tools.\n")
(display " How does it discover that:\n")
(display " - Tools exist?\n")
(display " - Tools have input/output structure?\n")
(display " - Specific tools (search engines) can answer questions?\n")
(display " - Queries must be formulated in specific ways?\n\n")
(display " Answer: The SAME method that works for language:\n")
(display " COUNT → CLUSTER → STRUCTURE → USE\n\n")
(display " But applied to ACTIONS and their EFFECTS.\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; PHASE 1: CORPUS LEARNING (what we've built)
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "PHASE 1: Corpus Learning (already covered)\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " Input: Text corpus\n")
(display " Process: COUNT → CLUSTER → LAYERS\n")
(display " Output:\n")
(display " - Word associations (PMI)\n")
(display " - Word categories (clusters)\n")
(display " - Semantic frames\n")
(display " - Scripts\n")
(display " - VAGUENESS MAP (where knowledge is weak)\n\n")
(display " The vagueness map tells the agent:\n")
(display " 'I don't know enough about X, Y, Z'\n")
(display " 'I need more information about topic T'\n\n")
(display " But the agent doesn't yet know HOW to get more information.\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; PHASE 2: DISCOVERING THAT ACTIONS HAVE EFFECTS
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "PHASE 2: Discovering Action-Effect Structure\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " The agent needs an EMBODIMENT - even minimal:\n")
(display " - Can perform actions (send commands)\n")
(display " - Can observe results (receive responses)\n\n")
(display " Minimal embodiment for computer use:\n")
(display " - ACTION: Send text to a shell/interface\n")
(display " - OBSERVATION: Receive text back\n\n")
(display " The agent explores RANDOMLY at first:\n\n")
(display " ┌─────────────────────────────────────────────────────────────┐\n")
(display " │ Action: 'asdfgh' → Response: 'command not found' │\n")
(display " │ Action: 'help' → Response: [long text] │\n")
(display " │ Action: 'ls' → Response: [file list] │\n")
(display " │ Action: 'cat file.txt' → Response: [file contents] │\n")
(display " │ Action: 'echo hello' → Response: 'hello' │\n")
(display " └─────────────────────────────────────────────────────────────┘\n\n")
(display " From random exploration, the agent learns:\n")
(display " - Some actions produce errors (negative signal)\n")
(display " - Some actions produce information (positive signal)\n")
(display " - Actions have STRUCTURE (command + arguments)\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; PHASE 2b: COUNTING ACTION PATTERNS
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "PHASE 2b: Counting Action Patterns (same method!)\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " Apply the SAME recursive structure discovery to actions:\n\n")
(display " Layer 0: Raw action-response pairs\n")
(display " ('ls', [file list])\n")
(display " ('ls -la', [detailed file list])\n")
(display " ('cat X', [contents of X])\n")
(display " ('echo Y', Y)\n\n")
(display " Layer 1: Count co-occurrences\n")
(display " 'ls' co-occurs with file-listing responses\n")
(display " 'cat' co-occurs with file-content responses\n")
(display " '-la' co-occurs with 'ls' and detailed output\n")
(display " PMI('cat', file-content) = HIGH\n")
(display " PMI('ls', file-listing) = HIGH\n\n")
(display " Layer 2: Cluster actions by effect similarity\n")
(display " CLUSTER-READ: {cat, less, head, tail} → show file contents\n")
(display " CLUSTER-LIST: {ls, dir, find} → show file names\n")
(display " CLUSTER-WRITE: {echo, printf, write} → produce output\n")
(display " CLUSTER-ERROR: {asdf, qwerty, ...} → produce errors\n\n")
(display " Layer 3: Discover tool GRAMMAR\n")
(display " Pattern: COMMAND [OPTIONS] [ARGUMENTS]\n")
(display " Pattern: COMMAND takes FILENAME as argument\n")
(display " Pattern: OPTIONS modify COMMAND behavior\n\n")
(display " The agent discovers TOOL STRUCTURE the same way it\n")
(display " discovered LANGUAGE STRUCTURE: by counting patterns.\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; PHASE 3: DISCOVERING INFORMATION-SEEKING TOOLS
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "PHASE 3: Discovering Information-Seeking Tools\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " Key insight: Some tools ANSWER QUESTIONS.\n\n")
(display " The agent notices:\n")
(display " - 'cat file' → returns TEXT\n")
(display " - 'grep word file' → returns MATCHING text\n")
(display " - 'man command' → returns EXPLANATION text\n\n")
(display " These tools have a special property:\n")
(display " INPUT: A query (word, topic, question)\n")
(display " OUTPUT: Information RELATED to the query\n\n")
(display " The agent can TEST this:\n")
(display " - Run 'grep bank corpus.txt'\n")
(display " - Observe: output contains sentences with 'bank'\n")
(display " - Learn: 'grep' is an INFORMATION-RETRIEVAL tool\n\n")
(display " CRITICAL DISCOVERY:\n")
(display " ┌─────────────────────────────────────────────────────────────┐\n")
(display " │ Some tools REDUCE VAGUENESS about specific topics! │\n")
(display " │ │\n")
(display " │ Agent has high vagueness about 'bank' │\n")
(display " │ Agent runs 'grep bank corpus.txt' │\n")
(display " │ Agent gets MORE EXAMPLES of 'bank' in context │\n")
(display " │ Agent's vagueness about 'bank' DECREASES │\n")
(display " └─────────────────────────────────────────────────────────────┘\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; PHASE 4: DISCOVERING NETWORKED INFORMATION TOOLS
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "PHASE 4: Discovering Networked Information Tools\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " The agent has learned:\n")
(display " - Tools exist that return information\n")
(display " - 'grep' searches LOCAL files\n")
(display " - But local files are LIMITED\n\n")
(display " Exploration continues:\n\n")
(display " ┌─────────────────────────────────────────────────────────────┐\n")
(display " │ Action: 'curl http://example.com' │\n")
(display " │ Response: [HTML text from remote server] │\n")
(display " │ │\n")
(display " │ DISCOVERY: Some tools retrieve REMOTE information! │\n")
(display " └─────────────────────────────────────────────────────────────┘\n\n")
(display " The agent clusters network tools:\n")
(display " CLUSTER-NETWORK: {curl, wget, fetch} → get remote text\n")
(display " Pattern: These tools take URLs as arguments\n")
(display " Pattern: URLs have structure (protocol://host/path)\n\n")
(display " Further exploration reveals:\n")
(display " - Different URLs return different information\n")
(display " - Some URLs are SEARCH ENGINES\n")
(display " - Search engines take QUERIES and return RESULTS\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; PHASE 5: DISCOVERING SEARCH ENGINE STRUCTURE
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "PHASE 5: Discovering Search Engine Structure\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " The agent experiments with URLs:\n\n")
(display " ┌─────────────────────────────────────────────────────────────┐\n")
(display " │ curl 'https://search.example.com/?q=bank' │\n")
(display " │ Response: [pages about 'bank'] │\n")
(display " │ │\n")
(display " │ curl 'https://search.example.com/?q=river+bank' │\n")
(display " │ Response: [pages about river banks] │\n")
(display " │ │\n")
(display " │ curl 'https://search.example.com/?q=financial+bank' │\n")
(display " │ Response: [pages about financial banks] │\n")
(display " └─────────────────────────────────────────────────────────────┘\n\n")
(display " The agent discovers SEARCH ENGINE GRAMMAR:\n\n")
(display " Pattern: base_url + '?q=' + query_terms\n")
(display " Pattern: query_terms joined with '+'\n")
(display " Pattern: more specific queries → more specific results\n")
(display " Pattern: results contain text ABOUT the query topic\n\n")
(display " CRITICAL INSIGHT:\n")
(display " ┌─────────────────────────────────────────────────────────────┐\n")
(display " │ Search engines are VAGUENESS-REDUCTION tools! │\n")
(display " │ │\n")
(display " │ High vagueness about 'bank'? │\n")
(display " │ → Query 'bank' → get many contexts │\n")
(display " │ → Analyze contexts → reduce vagueness │\n")
(display " │ │\n")
(display " │ Can't disambiguate 'bank' in sentence? │\n")
(display " │ → Query 'river bank' AND 'financial bank' │\n")
(display " │ → Get distinguishing information │\n")
(display " │ → Build better discrimination model │\n")
(display " └─────────────────────────────────────────────────────────────┘\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; PHASE 6: CLOSING THE LOOP
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "PHASE 6: Closing the Loop - Autonomous Knowledge Expansion\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " The agent now has:\n")
(display " 1. Knowledge base (from corpus)\n")
(display " 2. Vagueness map (where knowledge is weak)\n")
(display " 3. Tool knowledge (how to use search engines)\n")
(display " 4. Query formulation (how to ask for information)\n\n")
(display " THE AUTONOMOUS LOOP:\n\n")
(display " ┌─────────────────────────────────────────────────────────────┐\n")
(display " │ │\n")
(display " │ 1. IDENTIFY high-vagueness areas in knowledge base │\n")
(display " │ 'I'm uncertain about the senses of word W' │\n")
(display " │ │\n")
(display " │ 2. GENERATE discriminating queries │\n")
(display " │ 'What distinguishes sense A from sense B?' │\n")
(display " │ → query terms: words with high discriminating power │\n")
(display " │ │\n")
(display " │ 3. FORMULATE search query │\n")
(display " │ base_url + '?q=' + discriminating_terms │\n")
(display " │ │\n")
(display " │ 4. EXECUTE query │\n")
(display " │ curl 'https://search.example.com/?q=...' │\n")
(display " │ │\n")
(display " │ 5. PARSE results │\n")
(display " │ Extract text from search results │\n")
(display " │ │\n")
(display " │ 6. INTEGRATE into knowledge base │\n")
(display " │ Apply Layer 0-N processing to new text │\n")
(display " │ Update co-occurrence counts │\n")
(display " │ Update clusters, frames, scripts │\n")
(display " │ │\n")
(display " │ 7. RECALCULATE vagueness │\n")
(display " │ Did the new information help? │\n")
(display " │ If vagueness still high → goto 1 │\n")
(display " │ │\n")
(display " │ REPEAT until vagueness is acceptable │\n")
(display " │ │\n")
(display " └─────────────────────────────────────────────────────────────┘\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; ATOMSPACE REPRESENTATION
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "ATOMSPACE REPRESENTATION OF TOOL KNOWLEDGE\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display "; Tool structure learned from exploration\n")
(display "(Evaluation\n")
(display " (Predicate \"tool-pattern\")\n")
(display " (List\n")
(display " (Concept \"grep\") ; tool name\n")
(display " (List (Variable \"pattern\") ; first argument: search pattern\n")
(display " (Variable \"file\")) ; second argument: file to search\n")
(display " (Concept \"matching-lines\"))) ; output type\n\n")
(display "; Search engine structure\n")
(display "(Evaluation\n")
(display " (Predicate \"tool-pattern\")\n")
(display " (List\n")
(display " (Concept \"web-search\")\n")
(display " (List (Concept \"base-url\")\n")
(display " (Variable \"query-terms\"))\n")
(display " (Concept \"search-results\")))\n\n")
(display "; Vagueness-reduction capability\n")
(display "(Evaluation\n")
(display " (Predicate \"reduces-vagueness-about\")\n")
(display " (List\n")
(display " (Concept \"web-search\") ; this tool\n")
(display " (Variable \"topic\") ; any topic\n")
(display " (Number 0.3))) ; by this much (expected)\n\n")
(display "; Query generation rule\n")
(display "(Bind\n")
(display " (And\n")
(display " ; Find high-vagueness word\n")
(display " (Evaluation (Predicate \"vagueness\")\n")
(display " (List (Variable \"$word\") (Variable \"$v\")))\n")
(display " (Greater (Variable \"$v\") (Number 0.3))\n")
(display " ; Find discriminating terms for that word\n")
(display " (Evaluation (Predicate \"discriminates\")\n")
(display " (List (Variable \"$term\") (Variable \"$word\") ...)))\n")
(display " ; Generate search action\n")
(display " (ExecutionOutput\n")
(display " (Schema \"web-search\")\n")
(display " (List (Variable \"$word\") (Variable \"$term\"))))\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; THE KEY INSIGHT
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "THE KEY INSIGHT: Same Structure All The Way\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " Language learning: COUNT word patterns → STRUCTURE emerges\n")
(display " Tool learning: COUNT action patterns → STRUCTURE emerges\n")
(display " Query formulation: COUNT what reduces vagueness → STRATEGY emerges\n\n")
(display " The agent doesn't need SPECIAL KNOWLEDGE about computers.\n")
(display " It discovers tool structure the same way it discovers word structure:\n")
(display " - Observe patterns\n")
(display " - Count co-occurrences\n")
(display " - Cluster by similarity\n")
(display " - Use clusters for prediction/action\n\n")
(display " ┌─────────────────────────────────────────────────────────────┐\n")
(display " │ │\n")
(display " │ TOOLS are just another domain with learnable structure. │\n")
(display " │ │\n")
(display " │ The same COUNT → CLUSTER → USE method applies. │\n")
(display " │ │\n")
(display " │ Words have grammar. Tools have grammar. │\n")
(display " │ Sentences have meaning. Actions have effects. │\n")
(display " │ Context disambiguates words. Context selects tools. │\n")
(display " │ │\n")
(display " └─────────────────────────────────────────────────────────────┘\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; IMPLEMENTATION SKETCH
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "IMPLEMENTATION SKETCH\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " MINIMAL REQUIREMENTS:\n\n")
(display " 1. Text interface to environment\n")
(display " - Agent can send strings (actions)\n")
(display " - Agent receives strings (observations)\n")
(display " - Could be: shell, REPL, API, chat interface\n\n")
(display " 2. Exploration drive\n")
(display " - Random actions initially\n")
(display " - Bias toward actions that produced information\n")
(display " - Bias away from actions that produced errors\n\n")
(display " 3. Vagueness-driven goal\n")
(display " - Agent WANTS to reduce vagueness\n")
(display " - This drives query generation\n")
(display " - This drives tool selection\n\n")
(display " 4. Integration loop\n")
(display " - New information → update knowledge base\n")
(display " - Updated knowledge → recalculate vagueness\n")
(display " - New vagueness map → new queries\n\n")
(display " PSEUDOCODE:\n\n")
(display " agent_loop():\n")
(display " knowledge_base = learn_from_corpus(initial_corpus)\n")
(display " tool_knowledge = {}\n")
(display " \n")
(display " while True:\n")
(display " # Phase A: Identify knowledge gaps\n")
(display " vagueness_map = compute_vagueness(knowledge_base)\n")
(display " high_vagueness = filter(v > threshold, vagueness_map)\n")
(display " \n")
(display " if empty(high_vagueness):\n")
(display " # Knowledge is sufficient\n")
(display " wait_for_new_input()\n")
(display " continue\n")
(display " \n")
(display " # Phase B: Select target and generate query\n")
(display " target = select_highest_priority(high_vagueness)\n")
(display " discriminators = get_discriminating_terms(target)\n")
(display " \n")
(display " # Phase C: Select tool and execute\n")
(display " if know_search_tool(tool_knowledge):\n")
(display " query = formulate_search_query(target, discriminators)\n")
(display " result = execute_tool('web-search', query)\n")
(display " else:\n")
(display " # Explore to discover tools\n")
(display " action = explore_random_or_guided()\n")
(display " result = execute(action)\n")
(display " tool_knowledge = update_tool_knowledge(action, result)\n")
(display " \n")
(display " # Phase D: Integrate results\n")
(display " if is_text(result):\n")
(display " knowledge_base = integrate_text(knowledge_base, result)\n")
(display " \n")
(display " # Loop continues...\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; CHALLENGES AND SOLUTIONS
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "CHALLENGES AND SOLUTIONS\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " CHALLENGE 1: How to start exploring tools?\n")
(display " ─────────────────────────────────────────────\n")
(display " Solution: Random exploration with reward signal\n")
(display " - Actions that return text = positive\n")
(display " - Actions that return errors = negative\n")
(display " - Actions that return RELEVANT text = very positive\n")
(display " - Learn which action patterns work\n\n")
(display " CHALLENGE 2: How to know when tool is useful for vagueness?\n")
(display " ──────────────────────────────────────────────────────────────\n")
(display " Solution: Measure vagueness before and after\n")
(display " - Run tool, integrate result, recalculate vagueness\n")
(display " - Did vagueness decrease? → tool is useful for this\n")
(display " - Learn: (tool, topic-type) → vagueness-reduction\n\n")
(display " CHALLENGE 3: How to formulate good queries?\n")
(display " ────────────────────────────────────────────\n")
(display " Solution: Use discriminating power\n")
(display " - High-vagueness word = ambiguous\n")
(display " - Find words that DISTINGUISH senses\n")
(display " - Use those as query terms\n")
(display " - Example: 'bank' is vague\n")
(display " → 'river bank' vs 'financial bank' are discriminating\n")
(display " → Query both, compare results\n\n")
(display " CHALLENGE 4: How to parse search results?\n")
(display " ─────────────────────────────────────────\n")
(display " Solution: Treat as more corpus\n")
(display " - Search results are TEXT\n")
(display " - Apply same Layer 0-N processing\n")
(display " - Extract sentences, count patterns\n")
(display " - The structure-discovery system handles it\n\n")
(display " CHALLENGE 5: Avoiding infinite loops / bad queries?\n")
(display " ─────────────────────────────────────────────────────\n")
(display " Solution: Track query effectiveness\n")
(display " - If query didn't reduce vagueness → don't repeat\n")
(display " - If same vagueness persists → try different approach\n")
(display " - Budget: limit queries per topic per time period\n\n")
;; ═══════════════════════════════════════════════════════════════════
;; SUMMARY
;; ═══════════════════════════════════════════════════════════════════
(display "═══════════════════════════════════════════════════════════════════\n")
(display "SUMMARY: The Path from Corpus to Search Engine\n")
(display "─────────────────────────────────────────────────────────────────\n\n")
(display " ┌─────────────────────────────────────────────────────────────┐\n")
(display " │ │\n")
(display " │ 1. CORPUS LEARNING │\n")
(display " │ Text → PMI → Clusters → Frames → Vagueness Map │\n")
(display " │ │\n")
(display " │ 2. TOOL DISCOVERY │\n")
(display " │ Actions → Effects → Patterns → Tool Grammar │\n")
(display " │ (Same count-cluster method!) │\n")
(display " │ │\n")
(display " │ 3. INFORMATION TOOL RECOGNITION │\n")
(display " │ Some tools return relevant text │\n")
(display " │ Some tools reduce vagueness │\n")
(display " │ Search engines = high vagueness-reduction tools │\n")
(display " │ │\n")
(display " │ 4. QUERY FORMULATION │\n")
(display " │ High vagueness → need discriminating information │\n")
(display " │ Discriminating terms → search query │\n")
(display " │ Query → search engine → results │\n")
(display " │ │\n")
(display " │ 5. INTEGRATION │\n")
(display " │ Results = new text │\n")
(display " │ Process with same Layer 0-N pipeline │\n")
(display " │ Update knowledge base │\n")
(display " │ Recalculate vagueness │\n")
(display " │ │\n")
(display " │ 6. LOOP │\n")
(display " │ Still high vagueness? → more queries │\n")
(display " │ Vagueness acceptable? → wait for new questions │\n")
(display " │ │\n")
(display " └─────────────────────────────────────────────────────────────┘\n\n")
(display " The key: ONE METHOD (count-cluster-structure) applied to:\n")
(display " - Words → language structure\n")
(display " - Actions → tool structure\n")
(display " - Queries → information-seeking strategy\n\n")
(display " The agent bootstraps from passive reader to active learner\n")
(display " by applying the SAME structural analysis to its own actions.\n\n")