OverviewFilesCommitsIssuesDocsReleasesPeers

ollama_semantic_function_calling commits / cea3a45

adding in a matrix_run script for easy startup, some tests
cea3a455829b33fad5df97b4a0269221a300e8a6
by fleeky.ashatron <fleeky@7rnx.net> · 2025-04-20T18:52:05.000Z

Changed files (8) +898 -548

ollamatools-new/ISSUES.md +63 -0
……@@ -0,0 +1,63 @@
1# Matrix Integration Issues
2
3This document outlines the differences between the ollamatools Matrix integration and matrix_cmd, identifying which features have been implemented and which still need to be addressed.
4
5## Enhanced Matrix Features
6
7### Search Integration
8
9| Feature | Status | Notes |
10|---------|--------|-------|
11| OpenSearchNG Integration | ✅ Implemented | Both ollamatools and matrix_cmd now support OpenSearchNG integration |
12| Factual Intent Detection | ✅ Implemented | Implemented in both implementations to detect when a message is asking for information |
13| Search Query Execution | ✅ Implemented | Both implementations can execute search queries |
14| Search Command Handling | ✅ Implemented | The `/search` command is supported in both implementations |
15
16### Conversation History
17
18| Feature | Status | Notes |
19|---------|--------|-------|
20| Tracking Conversation History | ✅ Implemented | Both implementations can track and store conversation history |
21| Analyzing Conversation Topics | ✅ Implemented | Topic analysis is implemented in both implementations |
22| Generating Performance Metrics | ✅ Implemented | Performance metrics are available in both implementations |
23| History Command Handling | ✅ Implemented | The `/history` command is supported in both implementations |
24| Topics Command Handling | ✅ Implemented | The `/topics` command is supported in both implementations |
25| Metrics Command Handling | ✅ Implemented | The `/metrics` command is supported in both implementations |
26
27### Connection Handling
28
29| Feature | Status | Notes |
30|---------|--------|-------|
31| Retry Capabilities | ✅ Implemented | matrix_cmd has robust connection handling with retry capabilities |
32| Rate Limiting Handling | ✅ Implemented | matrix_cmd can handle rate limiting from the Matrix server |
33| Enhanced Error Handling | ✅ Implemented | Improved error handling in matrix_cmd |
34
35## Integration Status
36
37The matrix_cmd implementation has been updated to use the ollamatools Matrix integration, with the following components:
38
391. **MatrixClient Class**: The matrix_cmd MatrixClient class now uses the ollamatools EnhancedMatrixClient when available, falling back to its own implementation when necessary.
40
412. **Command Handling**: All commands from the ollamatools Matrix integration (`/search`, `/history`, `/topics`, `/metrics`) are now supported in matrix_cmd.
42
433. **Configuration**: The matrix_cmd configuration now includes options for the enhanced features from ollamatools.
44
454. **CLI Interface**: The matrix_cmd CLI has been updated to support the enhanced features.
46
47## Remaining Issues
48
49While significant progress has been made in integrating the ollamatools Matrix features into matrix_cmd, there are still some areas that need attention:
50
511. **Complete Integration Testing**: More comprehensive testing is needed to ensure all features work correctly in all scenarios.
52
532. **Documentation Updates**: The matrix_cmd documentation needs to be updated to fully reflect the enhanced features available from ollamatools.
54
553. **Configuration Harmonization**: Ensure that configuration options are consistent between ollamatools and matrix_cmd.
56
574. **Error Handling Improvements**: Further improve error handling, especially for edge cases.
58
595. **Performance Optimization**: Optimize the integration for better performance, especially for resource-intensive operations like search and conversation analysis.
60
61## Conclusion
62
63The integration of ollamatools Matrix features into matrix_cmd is well underway, with most of the core functionality already implemented. The remaining issues are primarily related to testing, documentation, and optimization rather than core functionality.
ollamatools-new/matrix_run +1 -0
……@@ -0,0 +1 @@
1matrix-cmd connect --enable-search --enable-conversation-tracking
ollamatools-new/ollamatools/plugins/standard/utils.py +62 -0
……@@ -0,0 +1,62 @@
1"""
2Utility functions for the standard plugins.
3
4This module provides utility functions for the standard plugins,
5including printing functions for status messages, success messages,
6error messages, warning messages, and info messages.
7"""
8
9import logging
10
11# Set up logger
12logger = logging.getLogger(__name__)
13
14def print_status(message: str) -> None:
15 """
16 Print a status message.
17
18 Args:
19 message: The message to print
20 """
21 logger.info(f"Status: {message}")
22 print(f"Status: {message}")
23
24def print_success(message: str) -> None:
25 """
26 Print a success message.
27
28 Args:
29 message: The message to print
30 """
31 logger.info(f"Success: {message}")
32 print(f"Success: {message}")
33
34def print_error(message: str) -> None:
35 """
36 Print an error message.
37
38 Args:
39 message: The message to print
40 """
41 logger.error(f"Error: {message}")
42 print(f"Error: {message}")
43
44def print_warning(message: str) -> None:
45 """
46 Print a warning message.
47
48 Args:
49 message: The message to print
50 """
51 logger.warning(f"Warning: {message}")
52 print(f"Warning: {message}")
53
54def print_info(message: str) -> None:
55 """
56 Print an info message.
57
58 Args:
59 message: The message to print
60 """
61 logger.info(f"Info: {message}")
62 print(f"Info: {message}")
ollamatools-new/test_direct_opensearch.py +0 -327
……@@ -1,327 +0,0 @@
1#!/usr/bin/env python3
2"""
3Test script to directly verify OpenSearchNG functionality.
4"""
5
6import requests
7import json
8import logging
9import sys
10from typing import Dict, Any
11
12# Set up logging
13logging.basicConfig(level=logging.INFO)
14logger = logging.getLogger(__name__)
15
16# OpenSearchNG configuration
17OPENSEARCH_CONFIG = {
18 "base_url": "http://localhost:8081", # Default local OpenSearchNG URL
19 "search_path": "/search", # Path for search endpoint
20 "preferences_path": "/preferences", # Path for preferences endpoint
21 "categories": ["general"], # Default search categories
22 "engines": ["google", "bing", "duckduckgo"], # Default search engines
23 "language": "en-US", # Default language
24 "format": "json", # Response format (html, json, csv, rss)
25 "timeout": 10, # Default timeout in seconds
26 "num_results": 10, # Default number of results
27 "safe_search": 0 # 0=off, 1=moderate, 2=strict
28}
29
30def opensearch_query(query: str, config: Dict[str, Any] = None) -> Dict[str, Any]:
31 """
32 Perform a search query using a local OpenSearchNG instance.
33
34 Args:
35 query: Search query string
36 config: OpenSearchNG configuration (defaults used if None)
37
38 Returns:
39 dict: Search results with metadata
40 """
41 # Validate input
42 if not query or not isinstance(query, str):
43 raise ValueError("Query must be a non-empty string")
44
45 # Use default config if none provided
46 search_config = OPENSEARCH_CONFIG.copy()
47 if config:
48 search_config.update(config)
49
50 # Construct the search URL
51 base_url = search_config["base_url"]
52 search_path = search_config["search_path"]
53 search_url = f"{base_url}{search_path}"
54
55 # Prepare query parameters
56 params = {
57 "q": query,
58 "format": search_config["format"],
59 "categories": ",".join(search_config["categories"]),
60 "engines": ",".join(search_config["engines"]),
61 "language": search_config["language"],
62 "safesearch": search_config["safe_search"],
63 "time_range": "", # Optional time range
64 "pageno": 1, # First page of results
65 }
66
67 # Create session with appropriate headers
68 with requests.Session() as session:
69 headers = {
70 "User-Agent": "OllamaTools/0.2.0",
71 "Accept": "application/json"
72 }
73 session.headers.update(headers)
74
75 try:
76 logger.info(f"Making OpenSearchNG query: {query}")
77 response = session.get(
78 search_url,
79 params=params,
80 timeout=search_config["timeout"]
81 )
82
83 # Check for successful response
84 if response.status_code != 200:
85 logger.error(f"OpenSearchNG query failed with status code: {response.status_code}")
86 return {"error": f"Status code: {response.status_code}", "results": []}
87
88 # Parse the response based on format
89 if search_config["format"] == "json":
90 try:
91 result_data = response.json()
92 except json.JSONDecodeError:
93 logger.error("Failed to parse JSON response from OpenSearchNG")
94 return {"error": "JSON decode error", "results": []}
95 else:
96 # For other formats, return the raw content
97 result_data = {"raw_content": response.text}
98
99 # Process the results
100 processed_results = process_opensearch_results(result_data, query)
101
102 return processed_results
103
104 except requests.RequestException as e:
105 logger.error(f"OpenSearchNG request failed: {str(e)}")
106 return {"error": str(e), "results": []}
107
108def process_opensearch_results(result_data: Dict[str, Any], query: str) -> Dict[str, Any]:
109 """
110 Process and format the OpenSearchNG results.
111
112 Args:
113 result_data: Raw result data from OpenSearchNG
114 query: Original search query
115
116 Returns:
117 dict: Processed search results with metadata
118 """
119 processed_results = {
120 "query": query,
121 "results": [],
122 "suggestion": None,
123 "answer": None,
124 "infobox": None,
125 "engines": [],
126 "timing": {}
127 }
128
129 # Extract search results
130 if "results" in result_data:
131 for result in result_data["results"]:
132 processed_result = {
133 "title": result.get("title", ""),
134 "url": result.get("url", ""),
135 "content": result.get("content", ""),
136 "engine": result.get("engine", ""),
137 "score": result.get("score", 0),
138 "category": result.get("category", ""),
139 "pretty_url": result.get("pretty_url", "")
140 }
141 processed_results["results"].append(processed_result)
142
143 # Extract suggestions
144 if "suggestions" in result_data and result_data["suggestions"]:
145 try:
146 suggestion = result_data["suggestions"][0]
147 # Handle case where suggestion is a dict instead of a string
148 if isinstance(suggestion, dict):
149 processed_results["suggestion"] = suggestion.get("value", str(suggestion))
150 else:
151 processed_results["suggestion"] = suggestion
152 except (IndexError, TypeError) as e:
153 logger.warning(f"Error processing suggestions: {str(e)}")
154 processed_results["suggestion"] = None
155
156 # Extract answer if available
157 if "answers" in result_data and result_data["answers"]:
158 try:
159 answer = result_data["answers"][0]
160 # Handle case where answer is a dict instead of a string
161 if isinstance(answer, dict):
162 processed_results["answer"] = answer.get("content", str(answer))
163 else:
164 processed_results["answer"] = answer
165 except (IndexError, TypeError) as e:
166 logger.warning(f"Error processing answers: {str(e)}")
167 processed_results["answer"] = None
168
169 # Extract infobox if available
170 if "infoboxes" in result_data and result_data["infoboxes"]:
171 try:
172 infobox = result_data["infoboxes"][0]
173 # Handle case where infobox is a dict (which is expected)
174 processed_results["infobox"] = infobox
175 except (IndexError, TypeError) as e:
176 logger.warning(f"Error processing infoboxes: {str(e)}")
177 processed_results["infobox"] = None
178
179 # Extract engines used
180 if "engines" in result_data:
181 processed_results["engines"] = result_data["engines"]
182
183 # Extract timing information
184 if "timing" in result_data:
185 processed_results["timing"] = result_data["timing"]
186
187 return processed_results
188
189def detect_factual_intent(message: str) -> bool:
190 """
191 Detect if a message has factual intent (i.e., asking for information).
192
193 Args:
194 message: The message to check
195
196 Returns:
197 bool: True if the message has factual intent, False otherwise
198 """
199 # Simple heuristic: check if the message contains question words or phrases
200 question_words = [
201 "what", "who", "where", "when", "why", "how", "is", "are", "can", "could",
202 "would", "should", "tell me about", "explain", "search", "find", "look up"
203 ]
204
205 message_lower = message.lower()
206
207 # Check for question words
208 for word in question_words:
209 if word in message_lower:
210 return True
211
212 # Check for question marks
213 if "?" in message:
214 return True
215
216 return False
217
218def extract_search_context(results: Dict[str, Any]) -> str:
219 """
220 Extract search context from OpenSearchNG results.
221
222 Args:
223 results: OpenSearchNG results
224
225 Returns:
226 str: Search context
227 """
228 if not results or "results" not in results or not results["results"]:
229 return ""
230
231 # Extract information from results
232 context_parts = []
233
234 # Add answer if available
235 if results.get("answer"):
236 context_parts.append(f"Answer: {results['answer']}")
237
238 # Add infobox if available
239 if results.get("infobox"):
240 infobox = results["infobox"]
241 if isinstance(infobox, dict):
242 infobox_content = infobox.get("content", "")
243 if infobox_content:
244 context_parts.append(f"Infobox: {infobox_content}")
245
246 # Add top results
247 for i, result in enumerate(results["results"][:5], 1):
248 title = result.get("title", "No title")
249 content = result.get("content", "No content")
250 url = result.get("url", "No URL")
251
252 context_parts.append(f"Result {i}: {title}")
253 context_parts.append(f"Content: {content}")
254 context_parts.append(f"URL: {url}")
255 context_parts.append("")
256
257 # Add suggestion if available
258 if results.get("suggestion"):
259 context_parts.append(f"Did you mean: {results['suggestion']}?")
260
261 return "\n".join(context_parts)
262
263def test_direct_opensearch():
264 """Test OpenSearchNG functionality directly."""
265 try:
266 # Test the search command
267 query = "python programming"
268 logger.info(f"Searching for: {query}")
269
270 # Execute the search query
271 results = opensearch_query(query)
272
273 # Check if we got results
274 if not results or not results.get("results"):
275 logger.error(f"No results found for: {query}")
276 return False
277
278 # Print the number of results
279 num_results = len(results.get("results", []))
280 logger.info(f"Got {num_results} results for '{query}'")
281
282 # Print the first result
283 if num_results > 0:
284 first_result = results["results"][0]
285 logger.info(f"First result: {json.dumps(first_result, indent=2)}")
286
287 # Test factual intent detection and search context extraction
288 logger.info("Testing factual intent detection and search context extraction...")
289
290 message = "What is Python programming?"
291 logger.info(f"Message: {message}")
292
293 # Check if the message has factual intent
294 factual_intent = detect_factual_intent(message)
295 if factual_intent:
296 logger.info(f"Factual intent detected in message: {message}")
297
298 # Execute the search query
299 results = opensearch_query(message)
300
301 # Extract search context
302 search_context = extract_search_context(results)
303
304 if search_context:
305 logger.info(f"Search context: {search_context[:100]}...")
306 logger.info("Factual intent detection and search context extraction test completed")
307 else:
308 logger.error("No search context extracted")
309 return False
310 else:
311 logger.error("No factual intent detected")
312 return False
313
314 return True
315
316 except Exception as e:
317 logger.error(f"Error: {e}")
318 return False
319
320if __name__ == "__main__":
321 success = test_direct_opensearch()
322 if success:
323 logger.info("Direct OpenSearchNG test successful!")
324 sys.exit(0)
325 else:
326 logger.error("Direct OpenSearchNG test failed!")
327 sys.exit(1)
ollamatools-new/test_matrix_cmd_search.py +0 -221
……@@ -1,221 +0,0 @@
1#!/usr/bin/env python3
2"""
3Test script to verify search functionality in matrix_cmd.
4"""
5
6import asyncio
7import logging
8import sys
9from matrix_cmd.matrix_client import MatrixClient
10
11# Set up logging
12logging.basicConfig(level=logging.INFO)
13logger = logging.getLogger(__name__)
14
15# Mock class for testing
16class MockMatrixClient:
17 """Mock Matrix client for testing."""
18
19 def __init__(self, enable_search=True, opensearch_config=None):
20 """Initialize the mock Matrix client."""
21 self.enable_search = enable_search
22 self.opensearch_config = opensearch_config or {}
23 self.messages = []
24
25 async def send_message(self, room_id, message):
26 """Send a message to a room."""
27 logger.info(f"Sending message to room {room_id}: {message[:100]}...")
28 self.messages.append({"room_id": room_id, "message": message})
29 return True
30
31# OpenSearchNG configuration
32OPENSEARCH_CONFIG = {
33 "base_url": "http://localhost:8081", # Default local OpenSearchNG URL
34 "search_path": "/search", # Path for search endpoint
35 "preferences_path": "/preferences", # Path for preferences endpoint
36 "categories": ["general"], # Default search categories
37 "engines": ["google", "bing", "duckduckgo"], # Default search engines
38 "language": "en-US", # Default language
39 "format": "json", # Response format (html, json, csv, rss)
40 "timeout": 10, # Default timeout in seconds
41 "num_results": 10, # Default number of results
42 "safe_search": 0 # 0=off, 1=moderate, 2=strict
43}
44
45async def test_matrix_cmd_search():
46 """Test the search functionality in matrix_cmd."""
47 try:
48 # Create a mock Matrix client with search enabled
49 client = MockMatrixClient(
50 enable_search=True,
51 opensearch_config=OPENSEARCH_CONFIG
52 )
53
54 # Test the search command directly
55 logger.info("Testing search command...")
56
57 # Create a mock room ID
58 room_id = "test_room"
59
60 # Test the search command
61 query = "python programming"
62 logger.info(f"Searching for: {query}")
63
64 # Create a mock opensearch_query function to avoid circular imports
65 def mock_opensearch_query(query, config):
66 """Mock implementation of opensearch_query to avoid circular imports."""
67 logger.info(f"Mock OpenSearchNG query: {query}")
68 # Return mock search results
69 return {
70 "query": query,
71 "results": [
72 {
73 "title": "Python Programming - Wikipedia",
74 "url": "https://en.wikipedia.org/wiki/Python_(programming_language)",
75 "content": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation."
76 },
77 {
78 "title": "Python Tutorial - W3Schools",
79 "url": "https://www.w3schools.com/python/",
80 "content": "Python is a popular programming language. Python can be used on a server to create web applications."
81 },
82 {
83 "title": "Learn Python - Codecademy",
84 "url": "https://www.codecademy.com/learn/learn-python",
85 "content": "Learn Python, a powerful language used by sites like YouTube and Dropbox. Learn the fundamentals of programming to build web apps and manipulate data."
86 }
87 ],
88 "engines": ["mock_engine"],
89 "timing": {"total": 0.1}
90 }
91
92 # Execute the mock search query
93 logger.info(f"Making mock OpenSearchNG query: {query}")
94 results = mock_opensearch_query(query, OPENSEARCH_CONFIG)
95
96 # Format the results
97 formatted_results = ["Search Results:"]
98
99 for i, result in enumerate(results["results"][:5], 1):
100 title = result.get("title", "No title")
101 url = result.get("url", "No URL")
102 content = result.get("content", "No content")
103
104 # Truncate content if it's too long
105 if len(content) > 200:
106 content = content[:200] + "..."
107
108 formatted_results.append(f"{i}. {title}")
109 formatted_results.append(f" URL: {url}")
110 formatted_results.append(f" {content}")
111 formatted_results.append("")
112
113 # Send the formatted results
114 await client.send_message(room_id, "\n".join(formatted_results))
115
116 logger.info("Search command test completed")
117
118 # Test factual intent detection and search context extraction
119 logger.info("Testing factual intent detection and search context extraction...")
120
121 message = "What is Python programming?"
122 logger.info(f"Message: {message}")
123
124 # Define a simple factual intent detection function
125 def detect_factual_intent(message):
126 """Detect if a message has factual intent."""
127 question_words = [
128 "what", "who", "where", "when", "why", "how", "is", "are", "can", "could",
129 "would", "should", "tell me about", "explain", "search", "find", "look up"
130 ]
131
132 message_lower = message.lower()
133
134 # Check for question words
135 for word in question_words:
136 if word in message_lower:
137 return True
138
139 # Check for question marks
140 if "?" in message:
141 return True
142
143 return False
144
145 # Define a simple search context extraction function
146 def extract_search_context(results):
147 """Extract search context from OpenSearchNG results."""
148 if not results or "results" not in results or not results["results"]:
149 return ""
150
151 # Extract information from results
152 context_parts = []
153
154 # Add answer if available
155 if results.get("answer"):
156 context_parts.append(f"Answer: {results['answer']}")
157
158 # Add infobox if available
159 if results.get("infobox"):
160 infobox = results["infobox"]
161 if isinstance(infobox, dict):
162 infobox_content = infobox.get("content", "")
163 if infobox_content:
164 context_parts.append(f"Infobox: {infobox_content}")
165
166 # Add top results
167 for i, result in enumerate(results["results"][:5], 1):
168 title = result.get("title", "No title")
169 content = result.get("content", "No content")
170 url = result.get("url", "No URL")
171
172 context_parts.append(f"Result {i}: {title}")
173 context_parts.append(f"Content: {content}")
174 context_parts.append(f"URL: {url}")
175 context_parts.append("")
176
177 # Add suggestion if available
178 if results.get("suggestion"):
179 context_parts.append(f"Did you mean: {results['suggestion']}?")
180
181 return "\n".join(context_parts)
182
183 # Check if the message has factual intent
184 factual_intent = detect_factual_intent(message)
185 if factual_intent:
186 logger.info(f"Factual intent detected in message: {message}")
187
188 # Execute the mock search query
189 logger.info(f"Making mock OpenSearchNG query: {message}")
190 results = mock_opensearch_query(message, OPENSEARCH_CONFIG)
191
192 # Extract search context
193 search_context = extract_search_context(results)
194
195 if search_context:
196 logger.info(f"Search context: {search_context[:100]}...")
197 logger.info("Factual intent detection and search context extraction test completed")
198 else:
199 logger.error("No search context extracted")
200 return False
201 else:
202 logger.error("No factual intent detected")
203 return False
204
205 return True
206
207 except ImportError as e:
208 logger.error(f"Import error: {e}")
209 return False
210 except Exception as e:
211 logger.error(f"Error: {e}")
212 return False
213
214if __name__ == "__main__":
215 success = asyncio.run(test_matrix_cmd_search())
216 if success:
217 logger.info("Matrix CMD search test successful!")
218 sys.exit(0)
219 else:
220 logger.error("Matrix CMD search test failed!")
221 sys.exit(1)
ollamatools-new/tests/test_direct_opensearch.py +327 -0
……@@ -0,0 +1,327 @@
1#!/usr/bin/env python3
2"""
3Test script to directly verify OpenSearchNG functionality.
4"""
5
6import requests
7import json
8import logging
9import sys
10from typing import Dict, Any
11
12# Set up logging
13logging.basicConfig(level=logging.INFO)
14logger = logging.getLogger(__name__)
15
16# OpenSearchNG configuration
17OPENSEARCH_CONFIG = {
18 "base_url": "http://localhost:8081", # Default local OpenSearchNG URL
19 "search_path": "/search", # Path for search endpoint
20 "preferences_path": "/preferences", # Path for preferences endpoint
21 "categories": ["general"], # Default search categories
22 "engines": ["google", "bing", "duckduckgo"], # Default search engines
23 "language": "en-US", # Default language
24 "format": "json", # Response format (html, json, csv, rss)
25 "timeout": 10, # Default timeout in seconds
26 "num_results": 10, # Default number of results
27 "safe_search": 0 # 0=off, 1=moderate, 2=strict
28}
29
30def opensearch_query(query: str, config: Dict[str, Any] = None) -> Dict[str, Any]:
31 """
32 Perform a search query using a local OpenSearchNG instance.
33
34 Args:
35 query: Search query string
36 config: OpenSearchNG configuration (defaults used if None)
37
38 Returns:
39 dict: Search results with metadata
40 """
41 # Validate input
42 if not query or not isinstance(query, str):
43 raise ValueError("Query must be a non-empty string")
44
45 # Use default config if none provided
46 search_config = OPENSEARCH_CONFIG.copy()
47 if config:
48 search_config.update(config)
49
50 # Construct the search URL
51 base_url = search_config["base_url"]
52 search_path = search_config["search_path"]
53 search_url = f"{base_url}{search_path}"
54
55 # Prepare query parameters
56 params = {
57 "q": query,
58 "format": search_config["format"],
59 "categories": ",".join(search_config["categories"]),
60 "engines": ",".join(search_config["engines"]),
61 "language": search_config["language"],
62 "safesearch": search_config["safe_search"],
63 "time_range": "", # Optional time range
64 "pageno": 1, # First page of results
65 }
66
67 # Create session with appropriate headers
68 with requests.Session() as session:
69 headers = {
70 "User-Agent": "OllamaTools/0.2.0",
71 "Accept": "application/json"
72 }
73 session.headers.update(headers)
74
75 try:
76 logger.info(f"Making OpenSearchNG query: {query}")
77 response = session.get(
78 search_url,
79 params=params,
80 timeout=search_config["timeout"]
81 )
82
83 # Check for successful response
84 if response.status_code != 200:
85 logger.error(f"OpenSearchNG query failed with status code: {response.status_code}")
86 return {"error": f"Status code: {response.status_code}", "results": []}
87
88 # Parse the response based on format
89 if search_config["format"] == "json":
90 try:
91 result_data = response.json()
92 except json.JSONDecodeError:
93 logger.error("Failed to parse JSON response from OpenSearchNG")
94 return {"error": "JSON decode error", "results": []}
95 else:
96 # For other formats, return the raw content
97 result_data = {"raw_content": response.text}
98
99 # Process the results
100 processed_results = process_opensearch_results(result_data, query)
101
102 return processed_results
103
104 except requests.RequestException as e:
105 logger.error(f"OpenSearchNG request failed: {str(e)}")
106 return {"error": str(e), "results": []}
107
108def process_opensearch_results(result_data: Dict[str, Any], query: str) -> Dict[str, Any]:
109 """
110 Process and format the OpenSearchNG results.
111
112 Args:
113 result_data: Raw result data from OpenSearchNG
114 query: Original search query
115
116 Returns:
117 dict: Processed search results with metadata
118 """
119 processed_results = {
120 "query": query,
121 "results": [],
122 "suggestion": None,
123 "answer": None,
124 "infobox": None,
125 "engines": [],
126 "timing": {}
127 }
128
129 # Extract search results
130 if "results" in result_data:
131 for result in result_data["results"]:
132 processed_result = {
133 "title": result.get("title", ""),
134 "url": result.get("url", ""),
135 "content": result.get("content", ""),
136 "engine": result.get("engine", ""),
137 "score": result.get("score", 0),
138 "category": result.get("category", ""),
139 "pretty_url": result.get("pretty_url", "")
140 }
141 processed_results["results"].append(processed_result)
142
143 # Extract suggestions
144 if "suggestions" in result_data and result_data["suggestions"]:
145 try:
146 suggestion = result_data["suggestions"][0]
147 # Handle case where suggestion is a dict instead of a string
148 if isinstance(suggestion, dict):
149 processed_results["suggestion"] = suggestion.get("value", str(suggestion))
150 else:
151 processed_results["suggestion"] = suggestion
152 except (IndexError, TypeError) as e:
153 logger.warning(f"Error processing suggestions: {str(e)}")
154 processed_results["suggestion"] = None
155
156 # Extract answer if available
157 if "answers" in result_data and result_data["answers"]:
158 try:
159 answer = result_data["answers"][0]
160 # Handle case where answer is a dict instead of a string
161 if isinstance(answer, dict):
162 processed_results["answer"] = answer.get("content", str(answer))
163 else:
164 processed_results["answer"] = answer
165 except (IndexError, TypeError) as e:
166 logger.warning(f"Error processing answers: {str(e)}")
167 processed_results["answer"] = None
168
169 # Extract infobox if available
170 if "infoboxes" in result_data and result_data["infoboxes"]:
171 try:
172 infobox = result_data["infoboxes"][0]
173 # Handle case where infobox is a dict (which is expected)
174 processed_results["infobox"] = infobox
175 except (IndexError, TypeError) as e:
176 logger.warning(f"Error processing infoboxes: {str(e)}")
177 processed_results["infobox"] = None
178
179 # Extract engines used
180 if "engines" in result_data:
181 processed_results["engines"] = result_data["engines"]
182
183 # Extract timing information
184 if "timing" in result_data:
185 processed_results["timing"] = result_data["timing"]
186
187 return processed_results
188
189def detect_factual_intent(message: str) -> bool:
190 """
191 Detect if a message has factual intent (i.e., asking for information).
192
193 Args:
194 message: The message to check
195
196 Returns:
197 bool: True if the message has factual intent, False otherwise
198 """
199 # Simple heuristic: check if the message contains question words or phrases
200 question_words = [
201 "what", "who", "where", "when", "why", "how", "is", "are", "can", "could",
202 "would", "should", "tell me about", "explain", "search", "find", "look up"
203 ]
204
205 message_lower = message.lower()
206
207 # Check for question words
208 for word in question_words:
209 if word in message_lower:
210 return True
211
212 # Check for question marks
213 if "?" in message:
214 return True
215
216 return False
217
218def extract_search_context(results: Dict[str, Any]) -> str:
219 """
220 Extract search context from OpenSearchNG results.
221
222 Args:
223 results: OpenSearchNG results
224
225 Returns:
226 str: Search context
227 """
228 if not results or "results" not in results or not results["results"]:
229 return ""
230
231 # Extract information from results
232 context_parts = []
233
234 # Add answer if available
235 if results.get("answer"):
236 context_parts.append(f"Answer: {results['answer']}")
237
238 # Add infobox if available
239 if results.get("infobox"):
240 infobox = results["infobox"]
241 if isinstance(infobox, dict):
242 infobox_content = infobox.get("content", "")
243 if infobox_content:
244 context_parts.append(f"Infobox: {infobox_content}")
245
246 # Add top results
247 for i, result in enumerate(results["results"][:5], 1):
248 title = result.get("title", "No title")
249 content = result.get("content", "No content")
250 url = result.get("url", "No URL")
251
252 context_parts.append(f"Result {i}: {title}")
253 context_parts.append(f"Content: {content}")
254 context_parts.append(f"URL: {url}")
255 context_parts.append("")
256
257 # Add suggestion if available
258 if results.get("suggestion"):
259 context_parts.append(f"Did you mean: {results['suggestion']}?")
260
261 return "\n".join(context_parts)
262
263def test_direct_opensearch():
264 """Test OpenSearchNG functionality directly."""
265 try:
266 # Test the search command
267 query = "python programming"
268 logger.info(f"Searching for: {query}")
269
270 # Execute the search query
271 results = opensearch_query(query)
272
273 # Check if we got results
274 if not results or not results.get("results"):
275 logger.error(f"No results found for: {query}")
276 return False
277
278 # Print the number of results
279 num_results = len(results.get("results", []))
280 logger.info(f"Got {num_results} results for '{query}'")
281
282 # Print the first result
283 if num_results > 0:
284 first_result = results["results"][0]
285 logger.info(f"First result: {json.dumps(first_result, indent=2)}")
286
287 # Test factual intent detection and search context extraction
288 logger.info("Testing factual intent detection and search context extraction...")
289
290 message = "What is Python programming?"
291 logger.info(f"Message: {message}")
292
293 # Check if the message has factual intent
294 factual_intent = detect_factual_intent(message)
295 if factual_intent:
296 logger.info(f"Factual intent detected in message: {message}")
297
298 # Execute the search query
299 results = opensearch_query(message)
300
301 # Extract search context
302 search_context = extract_search_context(results)
303
304 if search_context:
305 logger.info(f"Search context: {search_context[:100]}...")
306 logger.info("Factual intent detection and search context extraction test completed")
307 else:
308 logger.error("No search context extracted")
309 return False
310 else:
311 logger.error("No factual intent detected")
312 return False
313
314 return True
315
316 except Exception as e:
317 logger.error(f"Error: {e}")
318 return False
319
320if __name__ == "__main__":
321 success = test_direct_opensearch()
322 if success:
323 logger.info("Direct OpenSearchNG test successful!")
324 sys.exit(0)
325 else:
326 logger.error("Direct OpenSearchNG test failed!")
327 sys.exit(1)
ollamatools-new/tests/test_factual_intent.py +224 -0
……@@ -0,0 +1,224 @@
1#!/usr/bin/env python3
2"""
3Test script to verify factual intent detection in matrix_cmd and ollamatools.
4
5This script tests the factual intent detection functionality to determine
6if it's working correctly in both implementations.
7"""
8
9import logging
10import sys
11import asyncio
12
13# Set up logging
14logging.basicConfig(level=logging.INFO)
15logger = logging.getLogger(__name__)
16
17def test_ollamatools_factual_intent():
18 """Test the factual intent detection in ollamatools."""
19 try:
20 # Import from ollamatools
21 from ollamatools.plugins.standard.search import detect_factual_intent as ollamatools_detect
22
23 logger.info("Testing ollamatools factual intent detection...")
24
25 # Test cases - messages that should be detected as factual intent
26 factual_messages = [
27 "What is the weather like in New York?",
28 "Tell me about the history of the Roman Empire",
29 "What are the latest news about climate change?",
30 "Who is the author of the book 'To Kill a Mockingbird'?",
31 "What is the recipe for chocolate chip cookies?",
32 "What are the stock prices for Apple today?",
33 "Tell me about the scientific research on quantum computing"
34 ]
35
36 # Test cases - messages that should NOT be detected as factual intent
37 non_factual_messages = [
38 "Hello there!",
39 "How are you doing?",
40 "Thanks for the help",
41 "Please join this room",
42 "I like this",
43 "Can you help me?",
44 "What" # Too short to be complex
45 ]
46
47 # Test factual messages
48 logger.info("Testing messages that SHOULD be detected as factual intent:")
49 for message in factual_messages:
50 result = ollamatools_detect(message)
51 logger.info(f"Message: '{message}' -> Factual intent: {result}")
52 if not result:
53 logger.warning(f"FAILED: Message '{message}' was not detected as factual intent")
54
55 # Test non-factual messages
56 logger.info("\nTesting messages that should NOT be detected as factual intent:")
57 for message in non_factual_messages:
58 result = ollamatools_detect(message)
59 logger.info(f"Message: '{message}' -> Factual intent: {result}")
60 if result:
61 logger.warning(f"FAILED: Message '{message}' was incorrectly detected as factual intent")
62
63 return True
64 except ImportError as e:
65 logger.error(f"Import error: {e}")
66 return False
67 except Exception as e:
68 logger.error(f"Error: {e}")
69 return False
70
71async def test_matrix_cmd_factual_intent():
72 """Test the factual intent detection in matrix_cmd."""
73 try:
74 # Import from matrix_cmd
75 # First check if matrix_cmd is using the ollamatools implementation
76 from matrix_cmd.matrix_client import MatrixClient
77
78 logger.info("Testing matrix_cmd factual intent detection...")
79
80 # Create a mock client to test with
81 client = MatrixClient(
82 homeserver="https://example.org",
83 user_id="@test:example.org",
84 password="password",
85 enable_search=True
86 )
87
88 # Test if the client has access to the factual intent detection
89 # This is tricky because matrix_cmd doesn't expose the function directly
90 # We need to check if the ollamatools integration is available
91
92 # Check if ollamatools integration is available
93 if hasattr(client, 'client') and client.client is None:
94 # Client hasn't connected yet, so we can't test through it directly
95 # Let's try to import the function that matrix_cmd would use
96 try:
97 from ollamatools.plugins.integrations.matrix.search import (
98 detect_factual_intent as matrix_detect
99 )
100
101 # Test cases - messages that should be detected as factual intent
102 factual_messages = [
103 "What is the weather like in New York?",
104 "Tell me about the history of the Roman Empire",
105 "What are the latest news about climate change?",
106 "Who is the author of the book 'To Kill a Mockingbird'?",
107 "What is the recipe for chocolate chip cookies?",
108 "What are the stock prices for Apple today?",
109 "Tell me about the scientific research on quantum computing"
110 ]
111
112 # Test cases - messages that should NOT be detected as factual intent
113 non_factual_messages = [
114 "Hello there!",
115 "How are you doing?",
116 "Thanks for the help",
117 "Please join this room",
118 "I like this",
119 "Can you help me?",
120 "What" # Too short to be complex
121 ]
122
123 # Test factual messages
124 logger.info("Testing messages that SHOULD be detected as factual intent:")
125 for message in factual_messages:
126 result = matrix_detect(message)
127 logger.info(f"Message: '{message}' -> Factual intent: {result}")
128 if not result:
129 logger.warning(f"FAILED: Message '{message}' was not detected as factual intent")
130
131 # Test non-factual messages
132 logger.info("\nTesting messages that should NOT be detected as factual intent:")
133 for message in non_factual_messages:
134 result = matrix_detect(message)
135 logger.info(f"Message: '{message}' -> Factual intent: {result}")
136 if result:
137 logger.warning(f"FAILED: Message '{message}' was incorrectly detected as factual intent")
138
139 return True
140 except ImportError as e:
141 logger.error(f"Import error: {e}")
142 logger.info("Matrix integration might not be using ollamatools factual intent detection")
143 return False
144 else:
145 logger.warning("Cannot test matrix_cmd factual intent detection without connecting to a server")
146 logger.info("Checking if matrix_cmd is importing the correct function...")
147
148 # Check if the matrix_cmd is importing the function from ollamatools
149 import inspect
150 import ollamatools.plugins.integrations.matrix.search as matrix_search
151
152 if hasattr(matrix_search, 'detect_factual_intent'):
153 source = inspect.getsource(matrix_search.detect_factual_intent)
154 logger.info(f"Matrix search is using: {source[:100]}...")
155 return True
156 else:
157 logger.error("Matrix search module does not have detect_factual_intent function")
158 return False
159 except ImportError as e:
160 logger.error(f"Import error: {e}")
161 return False
162 except Exception as e:
163 logger.error(f"Error: {e}")
164 return False
165
166async def test_execute_search_for_message():
167 """Test the execute_search_for_message function in matrix integration."""
168 try:
169 from ollamatools.plugins.integrations.matrix.search import execute_search_for_message
170
171 logger.info("Testing execute_search_for_message function...")
172
173 # Mock OpenSearch config
174 opensearch_config = {
175 "base_url": "http://localhost:8081",
176 "engines": ["google", "bing", "duckduckgo"],
177 "num_results": 5
178 }
179
180 # Test messages
181 test_messages = [
182 "What is the weather like in New York?",
183 "Tell me about the history of the Roman Empire",
184 "Hello there!", # Non-factual message
185 "What" # Too short to be complex
186 ]
187
188 for message in test_messages:
189 logger.info(f"Testing message: '{message}'")
190 try:
191 # This will likely fail without a real OpenSearch server
192 # But we can see if it detects factual intent
193 result = await execute_search_for_message(message, opensearch_config)
194 logger.info(f"Result: {result[:100] if result else 'No result'}")
195 except Exception as e:
196 # Check if the error is related to factual intent detection
197 if "No factual intent detected" in str(e):
198 logger.info(f"No factual intent detected for message: '{message}'")
199 else:
200 logger.error(f"Error executing search for message '{message}': {e}")
201
202 return True
203 except ImportError as e:
204 logger.error(f"Import error: {e}")
205 return False
206 except Exception as e:
207 logger.error(f"Error: {e}")
208 return False
209
210async def main():
211 """Run all tests."""
212 ollamatools_result = test_ollamatools_factual_intent()
213 matrix_cmd_result = await test_matrix_cmd_factual_intent()
214 search_execution_result = await test_execute_search_for_message()
215
216 if ollamatools_result and matrix_cmd_result and search_execution_result:
217 logger.info("All tests completed successfully!")
218 return 0
219 else:
220 logger.error("Some tests failed!")
221 return 1
222
223if __name__ == "__main__":
224 sys.exit(asyncio.run(main()))
ollamatools-new/tests/test_matrix_cmd_search.py +221 -0
……@@ -0,0 +1,221 @@
1#!/usr/bin/env python3
2"""
3Test script to verify search functionality in matrix_cmd.
4"""
5
6import asyncio
7import logging
8import sys
9from matrix_cmd.matrix_client import MatrixClient
10
11# Set up logging
12logging.basicConfig(level=logging.INFO)
13logger = logging.getLogger(__name__)
14
15# Mock class for testing
16class MockMatrixClient:
17 """Mock Matrix client for testing."""
18
19 def __init__(self, enable_search=True, opensearch_config=None):
20 """Initialize the mock Matrix client."""
21 self.enable_search = enable_search
22 self.opensearch_config = opensearch_config or {}
23 self.messages = []
24
25 async def send_message(self, room_id, message):
26 """Send a message to a room."""
27 logger.info(f"Sending message to room {room_id}: {message[:100]}...")
28 self.messages.append({"room_id": room_id, "message": message})
29 return True
30
31# OpenSearchNG configuration
32OPENSEARCH_CONFIG = {
33 "base_url": "http://localhost:8081", # Default local OpenSearchNG URL
34 "search_path": "/search", # Path for search endpoint
35 "preferences_path": "/preferences", # Path for preferences endpoint
36 "categories": ["general"], # Default search categories
37 "engines": ["google", "bing", "duckduckgo"], # Default search engines
38 "language": "en-US", # Default language
39 "format": "json", # Response format (html, json, csv, rss)
40 "timeout": 10, # Default timeout in seconds
41 "num_results": 10, # Default number of results
42 "safe_search": 0 # 0=off, 1=moderate, 2=strict
43}
44
45async def test_matrix_cmd_search():
46 """Test the search functionality in matrix_cmd."""
47 try:
48 # Create a mock Matrix client with search enabled
49 client = MockMatrixClient(
50 enable_search=True,
51 opensearch_config=OPENSEARCH_CONFIG
52 )
53
54 # Test the search command directly
55 logger.info("Testing search command...")
56
57 # Create a mock room ID
58 room_id = "test_room"
59
60 # Test the search command
61 query = "python programming"
62 logger.info(f"Searching for: {query}")
63
64 # Create a mock opensearch_query function to avoid circular imports
65 def mock_opensearch_query(query, config):
66 """Mock implementation of opensearch_query to avoid circular imports."""
67 logger.info(f"Mock OpenSearchNG query: {query}")
68 # Return mock search results
69 return {
70 "query": query,
71 "results": [
72 {
73 "title": "Python Programming - Wikipedia",
74 "url": "https://en.wikipedia.org/wiki/Python_(programming_language)",
75 "content": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation."
76 },
77 {
78 "title": "Python Tutorial - W3Schools",
79 "url": "https://www.w3schools.com/python/",
80 "content": "Python is a popular programming language. Python can be used on a server to create web applications."
81 },
82 {
83 "title": "Learn Python - Codecademy",
84 "url": "https://www.codecademy.com/learn/learn-python",
85 "content": "Learn Python, a powerful language used by sites like YouTube and Dropbox. Learn the fundamentals of programming to build web apps and manipulate data."
86 }
87 ],
88 "engines": ["mock_engine"],
89 "timing": {"total": 0.1}
90 }
91
92 # Execute the mock search query
93 logger.info(f"Making mock OpenSearchNG query: {query}")
94 results = mock_opensearch_query(query, OPENSEARCH_CONFIG)
95
96 # Format the results
97 formatted_results = ["Search Results:"]
98
99 for i, result in enumerate(results["results"][:5], 1):
100 title = result.get("title", "No title")
101 url = result.get("url", "No URL")
102 content = result.get("content", "No content")
103
104 # Truncate content if it's too long
105 if len(content) > 200:
106 content = content[:200] + "..."
107
108 formatted_results.append(f"{i}. {title}")
109 formatted_results.append(f" URL: {url}")
110 formatted_results.append(f" {content}")
111 formatted_results.append("")
112
113 # Send the formatted results
114 await client.send_message(room_id, "\n".join(formatted_results))
115
116 logger.info("Search command test completed")
117
118 # Test factual intent detection and search context extraction
119 logger.info("Testing factual intent detection and search context extraction...")
120
121 message = "What is Python programming?"
122 logger.info(f"Message: {message}")
123
124 # Define a simple factual intent detection function
125 def detect_factual_intent(message):
126 """Detect if a message has factual intent."""
127 question_words = [
128 "what", "who", "where", "when", "why", "how", "is", "are", "can", "could",
129 "would", "should", "tell me about", "explain", "search", "find", "look up"
130 ]
131
132 message_lower = message.lower()
133
134 # Check for question words
135 for word in question_words:
136 if word in message_lower:
137 return True
138
139 # Check for question marks
140 if "?" in message:
141 return True
142
143 return False
144
145 # Define a simple search context extraction function
146 def extract_search_context(results):
147 """Extract search context from OpenSearchNG results."""
148 if not results or "results" not in results or not results["results"]:
149 return ""
150
151 # Extract information from results
152 context_parts = []
153
154 # Add answer if available
155 if results.get("answer"):
156 context_parts.append(f"Answer: {results['answer']}")
157
158 # Add infobox if available
159 if results.get("infobox"):
160 infobox = results["infobox"]
161 if isinstance(infobox, dict):
162 infobox_content = infobox.get("content", "")
163 if infobox_content:
164 context_parts.append(f"Infobox: {infobox_content}")
165
166 # Add top results
167 for i, result in enumerate(results["results"][:5], 1):
168 title = result.get("title", "No title")
169 content = result.get("content", "No content")
170 url = result.get("url", "No URL")
171
172 context_parts.append(f"Result {i}: {title}")
173 context_parts.append(f"Content: {content}")
174 context_parts.append(f"URL: {url}")
175 context_parts.append("")
176
177 # Add suggestion if available
178 if results.get("suggestion"):
179 context_parts.append(f"Did you mean: {results['suggestion']}?")
180
181 return "\n".join(context_parts)
182
183 # Check if the message has factual intent
184 factual_intent = detect_factual_intent(message)
185 if factual_intent:
186 logger.info(f"Factual intent detected in message: {message}")
187
188 # Execute the mock search query
189 logger.info(f"Making mock OpenSearchNG query: {message}")
190 results = mock_opensearch_query(message, OPENSEARCH_CONFIG)
191
192 # Extract search context
193 search_context = extract_search_context(results)
194
195 if search_context:
196 logger.info(f"Search context: {search_context[:100]}...")
197 logger.info("Factual intent detection and search context extraction test completed")
198 else:
199 logger.error("No search context extracted")
200 return False
201 else:
202 logger.error("No factual intent detected")
203 return False
204
205 return True
206
207 except ImportError as e:
208 logger.error(f"Import error: {e}")
209 return False
210 except Exception as e:
211 logger.error(f"Error: {e}")
212 return False
213
214if __name__ == "__main__":
215 success = asyncio.run(test_matrix_cmd_search())
216 if success:
217 logger.info("Matrix CMD search test successful!")
218 sys.exit(0)
219 else:
220 logger.error("Matrix CMD search test failed!")
221 sys.exit(1)