02bf3a1 · ollama_semantic_function_calling

Changed files (51) +235 -8822

ollamatools-new/matrix_cmd_integration_plan.md +234 -0

…	…	@@ -0,0 +1,234 @@
	1	# Matrix Command-Line Integration Plan
	2
	3	## Overview
	4
	5	This document outlines the plan for integrating the matrix_cmd command-line tool with the ollamatools Matrix integration. The goal is to refactor matrix_cmd to use the ollamatools Matrix integration directly, exposing all of its advanced features through the command-line interface.
	6
	7	## Current State
	8
	9	- ollamatools Matrix Integration: Provides a comprehensive Matrix client with advanced features like search, conversation history tracking, topic analysis, and performance metrics.
	10	- matrix_cmd: A standalone command-line tool for Matrix interactions with its own MatrixClient implementation, lacking the advanced features of ollamatools.
	11
	12	## Integration Goals
	13
	14	1. Refactor matrix_cmd to use the ollamatools Matrix integration directly
	15	2. Expose all ollamatools Matrix features through the matrix_cmd CLI
	16	3. Maintain backward compatibility for existing matrix_cmd users
	17	4. Ensure robust error handling and user feedback
	18
	19	## Implementation Plan
	20
	21	### Phase 1: Dependency Integration
	22
	23	1. Update Dependencies
	24	- Modify `setup.py` to include ollamatools as a dependency
	25	- Add any additional dependencies required by ollamatools
	26
	27	2. Replace MatrixClient Implementation
	28	- Replace the current MatrixClient in matrix_cmd with imports from ollamatools
	29	- Create adapter functions as needed to maintain backward compatibility
	30
	31	### Phase 2: CLI Enhancement
	32
	33	1. Update Command-Line Interface
	34	- Add new command-line options for enhanced features
	35	- Add new subcommands for search, history, topics, and metrics
	36	- Update help text and documentation
	37
	38	2. Extend Configuration Handling
	39	- Update configuration to include all ollamatools options
	40	- Provide sensible defaults for new configuration options
	41	- Ensure backward compatibility for existing configurations
	42
	43	### Phase 3: Feature Implementation
	44
	45	1. Search Integration
	46	- Implement search command and related functionality
	47	- Add OpenSearchNG configuration options
	48	- Implement factual intent detection
	49
	50	2. Conversation History
	51	- Implement history tracking and storage
	52	- Add commands for viewing and analyzing conversation history
	53	- Implement topic analysis
	54
	55	3. Performance Metrics
	56	- Implement metrics tracking and reporting
	57	- Add commands for viewing performance metrics
	58
	59	### Phase 4: Testing and Documentation
	60
	61	1. Testing
	62	- Test all new features and commands
	63	- Verify backward compatibility
	64	- Test error handling and edge cases
	65
	66	2. Documentation
	67	- Update README.md with new features and commands
	68	- Add examples for using new features
	69	- Update configuration documentation
	70
	71	## Detailed Implementation Tasks
	72
	73	### 1. Update `matrix_cmd/matrix_client.py`
	74
	75	```python
	76	# Replace current implementation with imports from ollamatools
	77	from ollamatools.plugins.integrations.matrix.client import MatrixClient, MatrixConfig
	78	from ollamatools.plugins.integrations.matrix.commands import connect_enhanced_matrix
	79	from ollamatools.plugins.integrations.matrix.history import (
	80	handle_history_command,
	81	handle_topics_command,
	82	handle_metrics_command
	83	)
	84	from ollamatools.plugins.integrations.matrix.search import handle_search_command
	85
	86	# Re-export these classes and functions
	87	__all__ = [
	88	'MatrixClient',
	89	'MatrixConfig',
	90	'connect_enhanced_matrix',
	91	'handle_history_command',
	92	'handle_topics_command',
	93	'handle_metrics_command',
	94	'handle_search_command'
	95	]
	96
	97	# Add any matrix_cmd specific extensions or adaptations here
	98	```
	99
	100	### 2. Update `matrix_cmd/main.py`
	101
	102	```python
	103	# Add new command-line options for enhanced features
	104	parser.add_argument(
	105	"--enable-search",
	106	action="store_true",
	107	help="Enable search functionality"
	108	)
	109	parser.add_argument(
	110	"--enable-history",
	111	action="store_true",
	112	help="Enable conversation history tracking"
	113	)
	114
	115	# Add new subcommands
	116	search_parser = subparsers.add_parser("search", help="Search the web")
	117	search_parser.add_argument(
	118	"--query",
	119	required=True,
	120	help="Search query"
	121	)
	122	search_parser.add_argument(
	123	"--room",
	124	required=True,
	125	help="Room ID to send results to"
	126	)
	127
	128	history_parser = subparsers.add_parser("history", help="View conversation history")
	129	history_parser.add_argument(
	130	"--count",
	131	type=int,
	132	default=10,
	133	help="Number of conversations to show"
	134	)
	135	history_parser.add_argument(
	136	"--room",
	137	required=True,
	138	help="Room ID to show history for"
	139	)
	140
	141	# Similar additions for topics and metrics commands
	142	```
	143
	144	### 3. Update `setup.py`
	145
	146	```python
	147	setup(
	148	name="matrix-cmd",
	149	version="0.2.0",
	150	# ...
	151	install_requires=[
	152	"matrix-nio>=0.19.0",
	153	"ollama>=0.1.0",
	154	"pyyaml>=5.1",
	155	"ollamatools>=0.1.0", # Add ollamatools as a dependency
	156	],
	157	# ...
	158	)
	159	```
	160
	161	### 4. Update `matrix_cmd/README.md`
	162
	163	Add documentation for new features and commands:
	164
	165	```markdown
	166	## Enhanced Features
	167
	168	### Search Integration
	169
	170	Search the web directly from Matrix:
	171
	172	```bash
	173	matrix-cmd search --query "Python asyncio tutorial" --room "#dev:example.org"
	174	```
	175
	176	Or use the `/search` command in a Matrix room:
	177
	178	```
	179	/search Python asyncio tutorial
	180	```
	181
	182	### Conversation History
	183
	184	View conversation history:
	185
	186	```bash
	187	matrix-cmd history --count 10 --room "#general:example.org"
	188	```
	189
	190	Or use the `/history` command in a Matrix room:
	191
	192	```
	193	/history 10
	194	```
	195
	196	### Topic Analysis
	197
	198	Analyze conversation topics:
	199
	200	```bash
	201	matrix-cmd topics --room "#general:example.org"
	202	```
	203
	204	Or use the `/topics` command in a Matrix room:
	205
	206	```
	207	/topics
	208	```
	209
	210	### Performance Metrics
	211
	212	View performance metrics:
	213
	214	```bash
	215	matrix-cmd metrics --room "#general:example.org"
	216	```
	217
	218	Or use the `/metrics` command in a Matrix room:
	219
	220	```
	221	/metrics
	222	```
	223	```
	224
	225	## Timeline
	226
	227	1. Week 1: Dependency integration and initial refactoring
	228	2. Week 2: CLI enhancement and configuration updates
	229	3. Week 3: Feature implementation
	230	4. Week 4: Testing, documentation, and final adjustments
	231
	232	## Conclusion
	233
	234	This integration will significantly enhance the capabilities of matrix_cmd by leveraging the advanced features of the ollamatools Matrix integration. Users will benefit from a more powerful command-line tool for Matrix interactions, with features like search, conversation history tracking, topic analysis, and performance metrics.

ollamatools-new/ollamatools/__init__.py +0 -27

…	…	@@ -1,27 +0,0 @@
1		"""OllamaTools package."""
2
3		from ollamatools.version import __version__
4
5		# Expose core components
6		from ollamatools.core.agent import Agent
7		from ollamatools.core.config import Config
8
9		# Expose agent implementations
10		from ollamatools.agents.basic import BasicAgent
11		from ollamatools.agents.modular import ModularAgent
12		from ollamatools.agents.extended import ExtendedAgent
13
14		# Expose plugin system
15		from ollamatools.plugins.base import Plugin
16		from ollamatools.plugins.registry import PluginRegistry
17
18		__all__ = [
19		"__version__",
20		"Agent",
21		"Config",
22		"BasicAgent",
23		"ModularAgent",
24		"ExtendedAgent",
25		"Plugin",
26		"PluginRegistry",
27		]

ollamatools-new/ollamatools/agents/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/agents/basic.py +0 -144

…	…	@@ -1,144 +0,0 @@
1		"""
2		Basic OllamaAgent implementation.
3
4		This module provides the BasicOllamaAgent class, which is a simple wrapper
5		around the core OllamaAgent class, providing a clean interface for basic usage
6		without the additional features of the modular agent.
7		"""
8
9		import logging
10		from typing import Dict, Any, List, Optional
11
12		from ollamatools.core.agent import OllamaAgent
13		from ollamatools.core.config import Config
14		from ollamatools.core.utils.logger import setup_logger
15
16		# Set up logging
17		logger = setup_logger("basic_ollama_agent", level="INFO")
18
19		class BasicOllamaAgent:
20		"""
21		A simple wrapper around the core OllamaAgent class.
22
23		This class provides a clean interface for basic usage of the OllamaAgent
24		without the additional features of the modular agent.
25		"""
26
27		def __init__(
28		self,
29		model: str = "llama3.1:latest",
30		host: str = "http://localhost:11434",
31		permission_prompts: bool = True,
32		sudo_always_prompt: bool = True,
33		config: Optional[Config] = None
34		):
35		"""
36		Initialize the BasicOllamaAgent with specified configuration.
37
38		Args:
39		model: Name of the Ollama model to use
40		host: Ollama server address
41		permission_prompts: Whether to prompt for function execution permission
42		sudo_always_prompt: Whether to always prompt for sudo commands
43		config: Configuration object (overrides individual parameters if provided)
44		"""
45		# Initialize the core OllamaAgent
46		self.agent = OllamaAgent(
47		model=model,
48		host=host,
49		permission_prompts=permission_prompts,
50		sudo_always_prompt=sudo_always_prompt,
51		config=config
52		)
53
54		# Initialize conversation context
55		self.context = []
56
57		# Define default system prompt
58		self.default_system_prompt = """
59		You are an AI assistant with access to various tools to help the user.
60		You can execute system commands, fetch data from the web, and generate documentation.
61		When a user asks for something that requires using one of your tools,
62		use the appropriate function to complete the task.
63		Be helpful, concise, and accurate.
64		"""
65
66		logger.info(f"BasicOllamaAgent initialized with model={self.agent.config.model}, host={self.agent.config.host}")
67
68		def chat(
69		self,
70		message: str,
71		system_prompt: Optional[str] = None,
72		context: Optional[List[Dict[str, str]]] = None
73		) -> str:
74		"""
75		Send a message to Ollama, automatically detecting and executing functions.
76
77		Args:
78		message: User message
79		system_prompt: Optional system prompt (defaults to self.default_system_prompt)
80		context: Optional conversation history (defaults to self.context)
81
82		Returns:
83		str: Response from Ollama, potentially after executing functions
84		"""
85		# Use provided context or instance context
86		if context is None:
87		context = self.context
88
89		# Use provided system prompt or default
90		if system_prompt is None:
91		system_prompt = self.default_system_prompt
92
93		logger.info(f"Sending message to Ollama: {message[:50]}...")
94
95		try:
96		# Send the message to the core agent
97		response = self.agent.chat(
98		message=message,
99		system_prompt=system_prompt,
100		context=context
101		)
102
103		# Update instance context if using it
104		if context is self.context:
105		self.context.append({"role": "user", "content": message})
106		self.context.append({"role": "assistant", "content": response})
107
108		# Limit context size to avoid exceeding token limits
109		if len(self.context) > 10:
110		# Keep only the last 10 messages
111		self.context = self.context[-10:]
112
113		logger.info(f"Received response from Ollama")
114		return response
115
116		except Exception as e:
117		logger.error(f"Error in chat: {str(e)}")
118		return f"Error communicating with Ollama: {str(e)}"
119
120		def reset_context(self) -> None:
121		"""
122		Reset the conversation context.
123		"""
124		self.context = []
125		logger.info("Conversation context reset")
126
127		def get_registered_functions(self) -> List[str]:
128		"""
129		Get a list of registered function names.
130
131		Returns:
132		List[str]: List of registered function names
133		"""
134		return list(self.agent.registered_functions.keys())
135
136		def set_system_prompt(self, prompt: str) -> None:
137		"""
138		Set a new default system prompt.
139
140		Args:
141		prompt: New system prompt
142		"""
143		self.default_system_prompt = prompt
144		logger.info("Default system prompt updated")

ollamatools-new/ollamatools/agents/modular.py +0 -702

…	…	@@ -1,702 +0,0 @@
1		"""
2		ModularOllamaAgent implementation.
3
4		This module provides the ModularOllamaAgent class, which extends
5		the base OllamaAgent with modular components, detailed feedback, and
6		additional features like search integration, command management,
7		issue tracking, and semantic function selection feedback.
8		"""
9
10		import os
11		import sys
12		import json
13		import time
14		import logging
15		import re
16		from typing import Dict, Any, List, Optional, Set, Union, Tuple
17
18		from ollamatools import OllamaAgent, Config
19		from ollamatools.core.utils.validators.command import DEFAULT_ALLOWLIST
20		from ollamatools.utils.logger import setup_logger
21		from ollamatools.plugins.integrations.opensearch.tools import (
22		opensearch_query,
23		opensearch_fetch_url,
24		get_opensearch_info,
25		DEFAULT_OPENSEARCH_CONFIG
26		)
27		from ollamatools.core.errors import (
28		OllamaToolsError,
29		CommandError,
30		FileError,
31		WebError,
32		handle_exception
33		)
34		from ollamatools.plugins.tools.system import execute_command
35
36		# Import agent components
37		from .search import detect_factual_intent, extract_search_context, execute_search
38		from .commands import list_allowed_commands, add_allowed_command, remove_allowed_command
39		from .issues import load_issues, save_issues, add_issue, list_issues, get_issue, update_issue
40		from .utils import print_status, print_success, print_error, print_warning, print_info, suggest_fix
41		from .feedback import (
42		FunctionSelectionFeedback,
43		detect_command_intent,
44		extract_command_from_similar_queries,
45		format_command_result
46		)
47
48		# Set up logging
49		logger = setup_logger("modular_ollama_agent", level="INFO")
50
51		# Extend the default command allowlist with useful commands
52		EXTENDED_ALLOWLIST = DEFAULT_ALLOWLIST.union({
53		"python", "python3", "calc", "bc", "find", "grep", "awk", "sed",
54		"head", "tail", "more", "less", "cat", "echo", "cd", "pwd"
55		})
56
57		# Configuration for OpenSearchNG
58		OPENSEARCH_CONFIG = DEFAULT_OPENSEARCH_CONFIG.copy()
59		OPENSEARCH_CONFIG.update({
60		"base_url": "http://localhost:8081", # Updated to your container's port
61		"engines": ["google", "bing", "duckduckgo", "brave"],
62		"num_results": 5
63		})
64
65		class ModularOllamaAgent:
66		"""
67		A modular OllamaAgent that provides transparent execution and better feedback.
68
69		This agent is designed with a modular architecture that separates functionality
70		into logical components for better maintainability and extensibility. It includes
71		features like search integration, command management, issue tracking, and
72		semantic function selection feedback.
73		"""
74
75		def __init__(
76		self,
77		model: str = "llama3.1:latest",
78		permission_prompts: bool = True,
79		opensearch_config: Optional[Dict[str, Any]] = None,
80		command_allowlist: Optional[Set[str]] = None,
81		feedback_file_path: Optional[str] = None
82		):
83		"""
84		Initialize the agent.
85
86		Args:
87		model: Ollama model to use
88		permission_prompts: Whether to display permission prompts
89		opensearch_config: OpenSearchNG configuration
90		command_allowlist: Custom command allowlist
91		feedback_file_path: Path to the feedback file (default: ~/.ollamatools/function_selection_feedback.jsonl)
92		"""
93		# Initialize OllamaAgent
94		self.agent = OllamaAgent(
95		model=model,
96		permission_prompts=permission_prompts
97		)
98
99		# Store configuration
100		self.opensearch_config = opensearch_config or OPENSEARCH_CONFIG
101		self.command_allowlist = command_allowlist or EXTENDED_ALLOWLIST
102
103		# Initialize state
104		self.context = []
105		self.execution_history = []
106
107		# Initialize issue tracking
108		self.issues = []
109		self.issue_id_counter = 1
110
111		# Path for saving issues
112		self.issues_file = os.path.join(
113		os.path.expanduser('~'),
114		'.ollamatools',
115		'issues.json'
116		)
117
118		# Create directory if it doesn't exist
119		os.makedirs(os.path.dirname(self.issues_file), exist_ok=True)
120
121		# Load existing issues if any
122		self.load_issues()
123
124		# Initialize function selection feedback system
125		self.function_feedback = FunctionSelectionFeedback(feedback_file_path)
126		logger.info("Initialized function selection feedback system")
127
128		# Define system prompt
129		self.system_prompt = f"""
130		You are an AI assistant with modular access to various tools.
131		You always explain what you're doing step by step and show your work.
132
133		When a user asks you a question, you should:
134		1. Identify the most appropriate tool for the task
135		2. Describe what you're going to do before you do it
136		3. Show the results of any commands or functions you execute
137		4. Explain how you interpret these results
138
139		MOST IMPORTANTLY: When calculating math problems, ALWAYS use the 'calc' command rather than
140		doing the calculation yourself or using python -c. For example: calc 2+2
141
142		For searching the web:
143		- Use the 'opensearch_query' function to perform web searches
144		- Use the 'opensearch_fetch_url' function to fetch web page content safely
145		- Specify search engines if needed (available: {', '.join(self.opensearch_config['engines'])})
146
147		For file and system operations:
148		- Use 'find_files' to locate files
149		- Use 'read_file' to view file content
150		- Use 'execute_command' for running shell commands like 'ls', 'pwd', etc.
151
152		For system information queries:
153		- When a user asks about the current directory, use 'pwd' command
154		- When a user asks about file listings, use 'ls' command
155		- When a user asks about the current user, use 'whoami' command
156		- When a user asks about system information, prefer shell commands over other methods
157
158		Always check command permissions carefully and explain what each command does.
159		For directory structure, use 'find .' or 'ls -la' commands.
160
161		When a user asks for information that might require searching the web or retrieving external data,
162		automatically determine if you should use the 'opensearch_query' function instead of requiring explicit search commands.
163
164		For example, if a user asks "can you give me a recipe for chilli sauce?", you should:
165		1. Recognize this as a request that requires external information
166		2. Use the opensearch_query function with an appropriate search query
167		3. Present the results in a helpful format
168
169		Only use tools when necessary - for factual questions you can answer directly, don't use search.
170		"""
171
172		# Check if OpenSearchNG is available
173		try:
174		self.print_status("Checking OpenSearchNG availability...")
175		info = get_opensearch_info(config=self.opensearch_config)
176		self.print_success(f"OpenSearchNG running at {self.opensearch_config['base_url']}")
177
178		# Log available engines
179		if 'engines' in info and info['engines']:
180		self.print_info(f"Available search engines: {', '.join(info['engines'])}")
181		except Exception as e:
182		self.print_error(f"OpenSearchNG not available: {str(e)}")
183		self.print_warning("Falling back to regular search methods")
184		self.suggest_fix("Make sure your OpenSearchNG instance is running at " +
185		f"{self.opensearch_config['base_url']}")
186
187		# Utility methods redirected to utils module
188		def print_status(self, message: str) -> None:
189		print_status(message)
190
191		def print_success(self, message: str) -> None:
192		print_success(message)
193
194		def print_error(self, message: str) -> None:
195		print_error(message)
196
197		def print_warning(self, message: str) -> None:
198		print_warning(message)
199
200		def print_info(self, message: str) -> None:
201		print_info(message)
202
203		def suggest_fix(self, suggestion: str) -> None:
204		suggest_fix(suggestion)
205
206		# Search methods redirected to search module
207		def detect_factual_intent(self, message: str) -> bool:
208		return detect_factual_intent(message)
209
210		def extract_search_context(self, search_results: Dict[str, Any]) -> str:
211		return extract_search_context(search_results)
212
213		def execute_search(self, query: str) -> Dict[str, Any]:
214		return execute_search(query, self.opensearch_config, self.agent.permission_manager, self.execution_history)
215
216		# Command management methods redirected to commands module
217		def list_allowed_commands(self) -> List[str]:
218		return list_allowed_commands(self.command_allowlist)
219
220		def add_allowed_command(self, command: str) -> bool:
221		return add_allowed_command(command, self.command_allowlist)
222
223		def remove_allowed_command(self, command: str) -> bool:
224		return remove_allowed_command(command, self.command_allowlist)
225
226		# Issue tracking methods redirected to issues module
227		def load_issues(self) -> None:
228		load_issues(self)
229
230		def save_issues(self) -> None:
231		save_issues(self)
232
233		def add_issue(self, title: str, description: str, severity: str = 'medium') -> int:
234		return add_issue(self, title, description, severity)
235
236		def list_issues(self, status: Optional[str] = None) -> List[Dict[str, Any]]:
237		return list_issues(self, status)
238
239		def get_issue(self, issue_id: int) -> Optional[Dict[str, Any]]:
240		return get_issue(self, issue_id)
241
242		def update_issue(self, issue_id: int, status: Optional[str] = None,
243		severity: Optional[str] = None, description: Optional[str] = None) -> bool:
244		return update_issue(self, issue_id, status, severity, description)
245
246		def show_execution_history(self, max_entries: int = 10) -> None:
247		"""
248		Display the execution history.
249
250		Args:
251		max_entries: Maximum number of history entries to show
252		"""
253		if not self.execution_history:
254		print("\n📜 Execution history is empty.")
255		return
256
257		# Get the most recent entries
258		recent_entries = self.execution_history[-max_entries:]
259
260		print("\n📜 Execution History:")
261		print("=" * 60)
262
263		for i, entry in enumerate(recent_entries, 1):
264		# Format timestamp
265		timestamp = time.strftime("%H:%M:%S", time.localtime(entry["timestamp"]))
266
267		# Format function and args
268		function_name = entry["function"]
269		args_str = ", ".join(f"{k}={v!r}" for k, v in entry["args"].items())
270
271		# Format status
272		status = "✅ Success" if entry.get("success", False) else "❌ Failed"
273
274		# Print the entry
275		print(f"{i}. [{timestamp}] {function_name}({args_str}) -> {status}")
276
277		# Show additional details for errors
278		if not entry.get("success", False) and "error" in entry:
279		print(f" Error: {entry['error']}")
280
281		# Show additional details for commands
282		if function_name == "execute_command" and "returncode" in entry:
283		print(f" Exit code: {entry['returncode']}")
284
285		# Show additional details for search
286		if function_name == "opensearch_query" and "num_results" in entry:
287		print(f" Results: {entry['num_results']}")
288
289		print("=" * 60)
290
291		def get_directory_structure(self, path: str = '.', max_depth: int = 2) -> Dict[str, Any]:
292		"""
293		Get a formatted directory structure.
294
295		Args:
296		path: Starting directory path
297		max_depth: Maximum directory depth to show
298
299		Returns:
300		dict: Directory structure information
301		"""
302		try:
303		self.print_status(f"Getting directory structure for: {path} (depth: {max_depth})")
304
305		# Execute find command to get the structure
306		from ollamatools.plugins.tools.system import execute_command
307		result = execute_command(
308		command="find",
309		args=[path, "-maxdepth", str(max_depth), "-type", "f", "-o", "-type", "d"],
310		permission_manager=self.agent.permission_manager
311		)
312
313		if "error" in result:
314		return result
315
316		# Process the output into a more structured format
317		lines = result.get("stdout", "").strip().split('\n')
318
319		structure = {
320		"path": path,
321		"depth": max_depth,
322		"entries": lines,
323		"files": [line for line in lines if os.path.isfile(line)],
324		"directories": [line for line in lines if os.path.isdir(line)],
325		"count": {
326		"total": len(lines),
327		"files": len([line for line in lines if os.path.isfile(line)]),
328		"directories": len([line for line in lines if os.path.isdir(line)])
329		}
330		}
331
332		self.print_success(f"Found {structure['count']['total']} entries "
333		f"({structure['count']['files']} files, "
334		f"{structure['count']['directories']} directories)")
335
336		return structure
337
338		except Exception as e:
339		self.print_error(f"Failed to get directory structure: {str(e)}")
340		self.suggest_fix(self._generate_error_fix_suggestion(e))
341
342		return {
343		"error": str(e),
344		"path": path,
345		"depth": max_depth
346		}
347
348		def _generate_error_fix_suggestion(self, error: Exception) -> str:
349		"""
350		Generate a fix suggestion based on the error type.
351
352		Args:
353		error: The exception
354
355		Returns:
356		str: Suggested fix
357		"""
358		if isinstance(error, CommandError):
359		return self._generate_command_fix_suggestion(error)
360		elif isinstance(error, FileError):
361		return self._generate_file_fix_suggestion(error)
362		elif isinstance(error, WebError):
363		return self._generate_web_fix_suggestion(error)
364		else:
365		# Generic suggestion
366		return f"Try a different approach or check if the required resources are available."
367
368		def _generate_command_fix_suggestion(self, error: Union[CommandError, Dict[str, Any]]) -> str:
369		"""
370		Generate a fix suggestion for command errors.
371
372		Args:
373		error: Error object or dictionary
374
375		Returns:
376		str: Suggested fix
377		"""
378		# Extract error details
379		if isinstance(error, CommandError):
380		error_msg = str(error)
381		details = error.details or {}
382		command = details.get('command', '')
383		stderr = details.get('stderr', '')
384		else:
385		error_msg = error.get('error', '')
386		command = error.get('command', '')
387		stderr = error.get('stderr', '')
388
389		# Check for common error patterns
390		if "not in allowed command list" in error_msg:
391		return (f"The command '{command}' is not in the allowed command list. "
392		f"Try using one of these commands instead: {', '.join(sorted(list(self.command_allowlist)[:10]))}... "
393		f"or add this command to the allowlist when initializing the agent.")
394
395		elif "dangerous pattern" in error_msg:
396		pattern = error_msg.split("pattern matching '")[1].split("'")[0] if "pattern matching '" in error_msg else ""
397		return (f"The command contains potentially dangerous pattern: {pattern}. "
398		f"Try reformulating the command to avoid shell metacharacters.")
399
400		elif "not found" in error_msg or "command not found" in stderr:
401		return f"The command '{command}' does not exist on this system. Check the spelling or install the command."
402
403		elif "permission denied" in stderr.lower():
404		return f"Permission denied when executing '{command}'. Try running with appropriate permissions."
405
406		else:
407		return f"Try a different command syntax or check command documentation."
408
409		def _generate_file_fix_suggestion(self, error: FileError) -> str:
410		"""
411		Generate a fix suggestion for file errors.
412
413		Args:
414		error: File error
415
416		Returns:
417		str: Suggested fix
418		"""
419		error_msg = str(error)
420		details = error.details or {}
421		path = details.get('path', '')
422
423		if "not found" in error_msg:
424		return f"The file '{path}' does not exist. Check the path or create the file."
425
426		elif "is a directory" in error_msg:
427		return f"'{path}' is a directory, not a file. Try specifying a file path instead."
428
429		elif "permission denied" in error_msg.lower():
430		return f"Permission denied when accessing '{path}'. Check file permissions."
431
432		elif "exceeds maximum size" in error_msg:
433		return f"The file '{path}' is too large. Try reading a smaller file or increasing max_size."
434
435		elif "outside the allowed directories" in error_msg:
436		return f"The path '{path}' is outside the allowed directories. Try a path within the sandbox."
437
438		else:
439		return f"Check if the file exists and you have appropriate permissions."
440
441		def _generate_web_fix_suggestion(self, error: WebError) -> str:
442		"""
443		Generate a fix suggestion for web errors.
444
445		Args:
446		error: Web error
447
448		Returns:
449		str: Suggested fix
450		"""
451		error_msg = str(error)
452		details = error.details or {}
453		url = details.get('url', '')
454
455		if "domain" in error_msg and "blocked" in error_msg:
456		return f"The domain in '{url}' is blocked for security reasons. Try a different domain."
457
458		elif "scheme" in error_msg and "not allowed" in error_msg:
459		return f"The URL scheme is not allowed. Use http:// or https:// instead."
460
461		elif "Connection refused" in error_msg:
462		return f"Could not connect to '{url}'. Check if the service is running and accessible."
463
464		elif "timeout" in error_msg.lower():
465		return f"The request to '{url}' timed out. Try again later or check your internet connection."
466
467		elif "Status code:" in error_msg and "404" in error_msg:
468		return f"The resource at '{url}' was not found (404). Check the URL for typos."
469
470		elif "Status code:" in error_msg and "403" in error_msg:
471		return f"Access to '{url}' is forbidden (403). The site may require authentication."
472
473		else:
474		return f"Try a different URL or check your internet connection."
475
476		def detect_command_intent(self, message: str) -> Tuple[bool, Optional[str]]:
477		"""
478		Detect if a message is asking for system information that requires a command.
479
480		Args:
481		message: User message
482
483		Returns:
484		Tuple[bool, Optional[str]]: (has_command_intent, suggested_command)
485		"""
486		return detect_command_intent(message, self.function_feedback)
487
488		def provide_function_feedback(self, query: str, correct_function: str, actual_function: str = None) -> bool:
489		"""
490		Allow users to provide feedback on function selection.
491
492		Args:
493		query: The original query
494		correct_function: The function that should have been used
495		actual_function: The function that was actually used (if known)
496
497		Returns:
498		bool: True if feedback was recorded successfully
499		"""
500		if actual_function:
501		# Record explicit feedback
502		self.function_feedback.record_explicit_feedback(
503		query, correct_function, actual_function, user_correction=True
504		)
505		self.print_success(f"Thank you for your feedback! I'll remember to use {correct_function} for similar queries in the future.")
506		return True
507		else:
508		# Try to find the most recent function used for this query
509		for entry in reversed(self.execution_history):
510		if entry.get("query", "") == query:
511		actual_function = entry.get("function")
512		if actual_function and actual_function != correct_function:
513		self.function_feedback.record_explicit_feedback(
514		query, correct_function, actual_function, user_correction=True
515		)
516		self.print_success(f"Thank you for your feedback! I'll remember to use {correct_function} for similar queries in the future.")
517		return True
518
519		# If we couldn't find a matching entry, just record the correct function
520		self.function_feedback.record_explicit_feedback(
521		query, correct_function, "unknown", user_correction=True
522		)
523		self.print_success(f"Thank you for your feedback! I'll remember to use {correct_function} for similar queries in the future.")
524		return True
525
526		def chat(self, message: str) -> str:
527		"""
528		Process a chat message, showing the execution steps transparently.
529
530		Args:
531		message: User message
532
533		Returns:
534		str: Response message
535		"""
536		try:
537		# First check if the message is asking for system information
538		has_command_intent, suggested_command = self.detect_command_intent(message)
539		if has_command_intent and suggested_command:
540		self.print_info(f"This appears to be a question about system information. Using '{suggested_command}' command...")
541		try:
542		# Execute the command
543		result = execute_command(
544		command=suggested_command,
545		permission_manager=self.agent.permission_manager
546		)
547
548		# Record this in execution history
549		self.execution_history.append({
550		"function": "execute_command",
551		"args": {"command": suggested_command},
552		"timestamp": time.time(),
553		"success": "error" not in result,
554		"returncode": result.get("returncode"),
555		"query": message
556		})
557
558		# Record implicit feedback
559		self.function_feedback.record_implicit_feedback(
560		message, "execute_command", 1.0, result
561		)
562
563		if "error" in result:
564		self.print_error(f"Command execution failed: {result['error']}")
565		return f"I tried to run the '{suggested_command}' command to answer your question, but encountered an error: {result['error']}"
566		else:
567		# Format the response based on the command
568		formatted_result = format_command_result(suggested_command, result)
569		return formatted_result
570		except Exception as e:
571		self.print_error(f"Command execution failed: {str(e)}")
572		return f"I tried to run the '{suggested_command}' command to answer your question, but encountered an error: {str(e)}"
573
574		# If not a command, check if the message likely requires a search but doesn't explicitly ask for one
575		if not message.lower().startswith('search '):
576		search_indicators = [
577		"what is", "how to", "who is", "where can", "when did", "why does",
578		"recipe for", "information about", "tell me about", "find", "look up",
579		"latest", "news on", "current", "recent"
580		]
581
582		# Check if the message contains search indicators
583		needs_search = any(indicator in message.lower() for indicator in search_indicators)
584
585		# Check for question structure
586		question_structure = message.lower().startswith(("what", "how", "who", "where", "when", "why", "can", "could"))
587
588		if needs_search or question_structure:
589		# Check if message is asking for external information we likely don't have
590		factual_intent = self.detect_factual_intent(message)
591		if factual_intent:
592		self.print_info("This appears to be a question that might benefit from search. Searching automatically...")
593		try:
594		results = self.execute_search(message)
595		# Process search results before continuing with normal chat flow
596		context = self.extract_search_context(results)
597		# Add context to the user message
598		message = f"{message}\n\nSearch context: {context}"
599
600		# Record implicit feedback
601		self.function_feedback.record_implicit_feedback(
602		message, "opensearch_query", 0.9, results
603		)
604		except Exception as e:
605		self.print_error(f"Automatic search failed: {str(e)}")
606		self.print_info("Continuing without search results")
607
608		# Print what we're doing
609		self.print_status(f"Processing message: {message}")
610
611		# Start timer for perf measurement
612		start_time = time.time()
613
614		# Process LLM response
615		try:
616		response = self.agent.chat(
617		message=message,
618		system_prompt=self.system_prompt,
619		context=self.context
620		)
621
622		# Check if the response looks like a JSON function call
623		if response.strip().startswith('{') and ('function' in response or 'name' in response):
624		self.print_info("Detected potential function call in response")
625		try:
626		# Parse the JSON response
627		func_data = json.loads(response)
628
629		# Extract function name and args
630		function_name = None
631		args = {}
632
633		# Handle different JSON formats
634		if 'function' in func_data:
635		function_name = func_data['function'].get('name')
636		args_str = func_data['function'].get('arguments', '{}')
637		if isinstance(args_str, str):
638		args = json.loads(args_str)
639		else:
640		args = args_str
641		elif 'name' in func_data:
642		function_name = func_data.get('name')
643		args = func_data.get('arguments', {})
644
645		if function_name:
646		self.print_status(f"Executing function: {function_name} with args: {args}")
647
648		# Execute the function
649		if function_name == 'opensearch_query' or function_name == 'search_web':
650		results = self.execute_search(args.get('query', message))
651		context = self.extract_search_context(results)
652		response = f"Here's what I found about '{args.get('query', message)}':\n\n{context}"
653		elif hasattr(self, function_name) and callable(getattr(self, function_name)):
654		# Execute a method of this class
655		method = getattr(self, function_name)
656		result = method(**args)
657		response = f"Here are the results for '{function_name}':\n\n{result}"
658		elif function_name in self.agent.registered_functions:
659		# Execute a registered function
660		result = self.agent.registered_functions[function_name](**args)
661		response = f"Here are the results for '{function_name}':\n\n{result}"
662		else:
663		self.print_error(f"Function '{function_name}' not found")
664		response = f"I tried to call a function '{function_name}' but it's not available. I'll try to help you directly instead."
665		except json.JSONDecodeError:
666		self.print_info("Response looked like JSON but couldn't be parsed as a function call")
667		except Exception as e:
668		self.print_error(f"Error executing function from response: {str(e)}")
669		response = f"I tried to execute a function to help with your request, but encountered an error: {str(e)}"
670		except Exception as e:
671		self.print_error(f"Error communicating with Ollama: {str(e)}")
672		response = f"I encountered an error communicating with the language model: {str(e)}"
673
674		# Update context
675		self.context.append({"role": "user", "content": message})
676		self.context.append({"role": "assistant", "content": response})
677
678		# Limit context size
679		if len(self.context) > 10:
680		self.context = self.context[-10:]
681
682		# Calculate execution time
683		execution_time = time.time() - start_time
684		self.print_success(f"Response generated in {execution_time:.2f} seconds")
685
686		return response
687
688		except OllamaToolsError as e:
689		error_details = handle_exception(e, log_error=True)
690
691		self.print_error(f"Error: {error_details['error_message']}")
692		self.suggest_fix(self._generate_error_fix_suggestion(e))
693
694		return f"I encountered an error: {error_details['error_message']}\n\n" + \
695		f"Suggested fix: {self._generate_error_fix_suggestion(e)}"
696
697		except Exception as e:
698		error_details = handle_exception(e, log_error=True)
699
700		self.print_error(f"Unexpected error: {str(e)}")
701
702		return f"I encountered an unexpected error: {str(e)}"

ollamatools-new/ollamatools/cli/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/cli/main.py +0 -54

…	…	@@ -1,54 +0,0 @@
1		"""Command-line interface for OllamaTools."""
2
3		import argparse
4		import sys
5		from typing import List, Optional
6
7		def parse_args(args: Optional[List[str]] = None) -> argparse.Namespace:
8		"""Parse command-line arguments."""
9		parser = argparse.ArgumentParser(description="OllamaTools - A toolkit for working with Ollama")
10
11		# Add subparsers for different commands
12		subparsers = parser.add_subparsers(dest="command", help="Command to execute")
13
14		# Chat command
15		chat_parser = subparsers.add_parser("chat", help="Start a chat session")
16		chat_parser.add_argument("--model", "-m", default="llama2", help="Model to use for chat")
17		chat_parser.add_argument("--config", "-c", help="Path to configuration file")
18
19		# Run command
20		run_parser = subparsers.add_parser("run", help="Run a script or command")
21		run_parser.add_argument("script", help="Script or command to run")
22		run_parser.add_argument("--model", "-m", default="llama2", help="Model to use for execution")
23
24		# Version command
25		subparsers.add_parser("version", help="Show version information")
26
27		return parser.parse_args(args)
28
29		def main(args: Optional[List[str]] = None) -> int:
30		"""Main entry point for the CLI."""
31		parsed_args = parse_args(args)
32
33		if parsed_args.command == "chat":
34		# Start chat session
35		print(f"Starting chat session with model {parsed_args.model}")
36		# Implementation
37		return 0
38		elif parsed_args.command == "run":
39		# Run script or command
40		print(f"Running {parsed_args.script} with model {parsed_args.model}")
41		# Implementation
42		return 0
43		elif parsed_args.command == "version":
44		# Show version information
45		from ollamatools.version import __version__
46		print(f"OllamaTools version {__version__}")
47		return 0
48		else:
49		# No command specified, show help
50		parse_args(["--help"])
51		return 1
52
53		if __name__ == "__main__":
54		sys.exit(main())

ollamatools-new/ollamatools/config/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/config/defaults.py +0 -344

…	…	@@ -1,344 +0,0 @@
1		"""
2		Configuration system for the OllamaTools plugin architecture.
3
4		This module provides classes and functions for managing plugin configuration.
5		"""
6
7		import os
8		import sys
9		import yaml
10		import json
11		import logging
12		from typing import Dict, Any, List, Optional, Set, Union, Callable
13
14		from ollamatools.plugins.errors import PluginConfigurationError
15
16		# Configure logging
17		logger = logging.getLogger(__name__)
18
19		class PluginConfig:
20		"""Configuration manager for plugins."""
21
22		def __init__(self, config=None, schema=None):
23		"""
24		Initialize the configuration manager.
25
26		Args:
27		config: Initial configuration dictionary or path to config file
28		schema: Optional schema for validation
29		"""
30		self.schema = schema or {}
31		self.config = {}
32
33		if config:
34		self.load_config(config)
35
36		def load_config(self, config):
37		"""
38		Load configuration from a dictionary, JSON, or YAML file.
39
40		Args:
41		config: Configuration source (dict, path to JSON/YAML file)
42
43		Returns:
44		PluginConfig: Self for method chaining
45
46		Raises:
47		PluginConfigurationError: If configuration loading fails
48		"""
49		try:
50		if isinstance(config, dict):
51		# Load from dictionary
52		self.config = config
53		elif isinstance(config, str) and os.path.exists(config):
54		# Load from file
55		with open(config, 'r') as f:
56		if config.endswith('.json'):
57		self.config = json.load(f)
58		elif config.endswith(('.yaml', '.yml')):
59		self.config = yaml.safe_load(f)
60		else:
61		raise PluginConfigurationError(
62		f"Unsupported configuration file format: {config}",
63		details={"file": config}
64		)
65		else:
66		raise PluginConfigurationError(
67		f"Invalid configuration source: {config}",
68		details={"source": str(config)}
69		)
70
71		# Validate the configuration
72		if self.schema:
73		self.validate()
74
75		return self
76
77		except PluginConfigurationError:
78		# Re-raise plugin configuration errors
79		raise
80		except Exception as e:
81		# Wrap other exceptions
82		raise PluginConfigurationError(
83		f"Failed to load configuration: {str(e)}",
84		details={"source": str(config)}
85		) from e
86
87		def validate(self):
88		"""
89		Validate the configuration against the schema.
90
91		Returns:
92		bool: True if valid
93
94		Raises:
95		PluginConfigurationError: If validation fails
96		"""
97		# Simple schema validation
98		for key, schema_item in self.schema.items():
99		# Check required fields
100		if schema_item.get('required', False) and key not in self.config:
101		raise PluginConfigurationError(
102		f"Missing required configuration key: {key}",
103		details={"key": key, "schema": schema_item}
104		)
105
106		# Check type if the key exists
107		if key in self.config:
108		value = self.config[key]
109		expected_type = schema_item.get('type')
110
111		if expected_type and not isinstance(value, expected_type):
112		raise PluginConfigurationError(
113		f"Invalid type for configuration key {key}. Expected {expected_type.__name__}, got {type(value).__name__}",
114		details={"key": key, "expected_type": expected_type.__name__, "actual_type": type(value).__name__}
115		)
116
117		# Check enum values
118		enum_values = schema_item.get('enum')
119		if enum_values and value not in enum_values:
120		raise PluginConfigurationError(
121		f"Invalid value for configuration key {key}. Expected one of {enum_values}, got {value}",
122		details={"key": key, "expected_values": enum_values, "actual_value": value}
123		)
124
125		# Check custom validation
126		validator = schema_item.get('validator')
127		if validator and callable(validator):
128		if not validator(value):
129		raise PluginConfigurationError(
130		f"Validation failed for configuration key {key}",
131		details={"key": key, "value": value}
132		)
133
134		return True
135
136		def get(self, key, default=None):
137		"""
138		Get a configuration value.
139
140		Args:
141		key: Configuration key
142		default: Default value if key is not found
143
144		Returns:
145		Any: Configuration value or default
146		"""
147		return self.config.get(key, default)
148
149		def set(self, key, value):
150		"""
151		Set a configuration value.
152
153		Args:
154		key: Configuration key
155		value: Configuration value
156
157		Returns:
158		PluginConfig: Self for method chaining
159
160		Raises:
161		PluginConfigurationError: If validation fails
162		"""
163		self.config[key] = value
164
165		# Validate if schema exists for this key
166		if self.schema and key in self.schema:
167		schema_item = self.schema[key]
168
169		# Check type
170		expected_type = schema_item.get('type')
171		if expected_type and not isinstance(value, expected_type):
172		raise PluginConfigurationError(
173		f"Invalid type for configuration key {key}. Expected {expected_type.__name__}, got {type(value).__name__}",
174		details={"key": key, "expected_type": expected_type.__name__, "actual_type": type(value).__name__}
175		)
176
177		# Check enum values
178		enum_values = schema_item.get('enum')
179		if enum_values and value not in enum_values:
180		raise PluginConfigurationError(
181		f"Invalid value for configuration key {key}. Expected one of {enum_values}, got {value}",
182		details={"key": key, "expected_values": enum_values, "actual_value": value}
183		)
184
185		# Check custom validation
186		validator = schema_item.get('validator')
187		if validator and callable(validator):
188		if not validator(value):
189		raise PluginConfigurationError(
190		f"Validation failed for configuration key {key}",
191		details={"key": key, "value": value}
192		)
193
194		return self
195
196		def merge(self, other_config):
197		"""
198		Merge another configuration into this one.
199
200		Args:
201		other_config: Another PluginConfig or dictionary
202
203		Returns:
204		PluginConfig: Self for method chaining
205		"""
206		if isinstance(other_config, PluginConfig):
207		other_dict = other_config.to_dict()
208		else:
209		other_dict = other_config
210
211		# Deep merge
212		self._deep_merge(self.config, other_dict)
213
214		# Validate the merged configuration
215		if self.schema:
216		self.validate()
217
218		return self
219
220		def _deep_merge(self, target, source):
221		"""
222		Deep merge two dictionaries.
223
224		Args:
225		target: Target dictionary
226		source: Source dictionary
227		"""
228		for key, value in source.items():
229		if key in target and isinstance(target[key], dict) and isinstance(value, dict):
230		self._deep_merge(target[key], value)
231		else:
232		target[key] = value
233
234		def to_dict(self):
235		"""
236		Convert the configuration to a dictionary.
237
238		Returns:
239		Dict[str, Any]: Configuration dictionary
240		"""
241		return self.config.copy()
242
243		def save(self, file_path, format='yaml'):
244		"""
245		Save the configuration to a file.
246
247		Args:
248		file_path: Path to save the configuration to
249		format: File format ('json' or 'yaml')
250
251		Returns:
252		PluginConfig: Self for method chaining
253
254		Raises:
255		PluginConfigurationError: If saving fails
256		"""
257		try:
258		# Create directory if it doesn't exist
259		os.makedirs(os.path.dirname(file_path), exist_ok=True)
260
261		with open(file_path, 'w') as f:
262		if format.lower() == 'json':
263		json.dump(self.config, f, indent=2)
264		elif format.lower() in ('yaml', 'yml'):
265		yaml.dump(self.config, f, default_flow_style=False)
266		else:
267		raise PluginConfigurationError(
268		f"Unsupported configuration format: {format}",
269		details={"format": format}
270		)
271
272		return self
273
274		except PluginConfigurationError:
275		# Re-raise plugin configuration errors
276		raise
277		except Exception as e:
278		# Wrap other exceptions
279		raise PluginConfigurationError(
280		f"Failed to save configuration: {str(e)}",
281		details={"file_path": file_path, "format": format}
282		) from e
283
284
285		def create_default_config(plugin_name, config_type='shared'):
286		"""
287		Create a default configuration for a plugin.
288
289		Args:
290		plugin_name: Name of the plugin
291		config_type: Type of configuration ('shared', 'matrix', 'agent')
292
293		Returns:
294		Dict[str, Any]: Default configuration
295		"""
296		# Base configuration
297		config = {
298		"enabled": True,
299		"name": plugin_name
300		}
301
302		# Add type-specific defaults
303		if config_type == 'shared':
304		pass # No additional defaults
305		elif config_type == 'matrix':
306		config.update({
307		"commands_enabled": True,
308		"command_prefix": "/"
309		})
310		elif config_type == 'agent':
311		config.update({
312		"functions_enabled": True
313		})
314
315		return config
316
317
318		def load_plugin_config(plugin_name, config_dir=None):
319		"""
320		Load configuration for a plugin from the default location.
321
322		Args:
323		plugin_name: Name of the plugin
324		config_dir: Optional custom configuration directory
325
326		Returns:
327		PluginConfig: Plugin configuration
328		"""
329		if not config_dir:
330		# Use default location
331		config_dir = os.path.join(
332		os.path.expanduser('~'),
333		'.ollamatools',
334		'plugins'
335		)
336
337		# Check for plugin-specific config file
338		config_file = os.path.join(config_dir, f"{plugin_name.lower()}.yaml")
339
340		if os.path.exists(config_file):
341		return PluginConfig(config_file)
342		else:
343		# Return default configuration
344		return PluginConfig(create_default_config(plugin_name))

ollamatools-new/ollamatools/config/loader.py +0 -107

…	…	@@ -1,107 +0,0 @@
1		"""
2		Core configuration management for OllamaTools.
3
4		This module contains the Config class, which handles configuration settings for
5		the OllamaTools library.
6		"""
7
8		import json
9		import os
10		from typing import Optional, Dict, Any
11
12		from pydantic import BaseModel, validator, Field
13
14
15		class Config(BaseModel):
16		"""
17		Configuration class for OllamaTools.
18
19		This class manages configuration settings for the OllamaTools library using
20		Pydantic for validation. It provides methods for loading and saving configuration
21		to a file.
22		"""
23
24		model: str = "llama3.1"
25		host: str = "http://localhost:11434"
26		permission_prompts: bool = True
27		sudo_always_prompt: bool = True
28		timeout_seconds: int = 60
29		max_file_size: Optional[int] = None
30
31		@validator('timeout_seconds')
32		def validate_timeout(cls, v):
33		"""Validate that timeout is positive."""
34		if v <= 0:
35		raise ValueError("Timeout must be positive")
36		return v
37
38		@validator('max_file_size')
39		def validate_max_file_size(cls, v):
40		"""Validate that max_file_size is positive if provided."""
41		if v is not None and v <= 0:
42		raise ValueError("Max file size must be positive")
43		return v
44
45		@classmethod
46		def load_from_file(cls, file_path: str) -> "Config":
47		"""
48		Load configuration from a JSON file.
49
50		Args:
51		file_path: Path to the configuration file
52
53		Returns:
54		Config: Configuration object
55
56		Raises:
57		FileNotFoundError: If the file does not exist
58		json.JSONDecodeError: If the file contains invalid JSON
59		ValueError: If the configuration is invalid
60		"""
61		if not os.path.exists(file_path):
62		raise FileNotFoundError(f"Configuration file not found: {file_path}")
63
64		with open(file_path, 'r') as f:
65		try:
66		config_data = json.load(f)
67		return cls(**config_data)
68		except json.JSONDecodeError:
69		raise json.JSONDecodeError(f"Invalid JSON in configuration file: {file_path}", "", 0)
70
71		def save_to_file(self, file_path: str) -> None:
72		"""
73		Save configuration to a JSON file.
74
75		Args:
76		file_path: Path to the configuration file
77
78		Raises:
79		PermissionError: If the file cannot be written
80		"""
81		try:
82		# Create directory if it doesn't exist
83		os.makedirs(os.path.dirname(os.path.abspath(file_path)), exist_ok=True)
84
85		with open(file_path, 'w') as f:
86		json.dump(self.dict(), f, indent=2)
87		except PermissionError:
88		raise PermissionError(f"Cannot write to configuration file: {file_path}")
89
90		def to_dict(self) -> Dict[str, Any]:
91		"""
92		Convert configuration to a dictionary.
93
94		Returns:
95		Dict[str, Any]: Configuration as a dictionary
96		"""
97		return self.dict(exclude_none=True)
98
99		@classmethod
100		def default_config_path(cls) -> str:
101		"""
102		Get the default configuration file path.
103
104		Returns:
105		str: Default configuration file path
106		"""
107		return os.path.expanduser("~/.config/ollamatools/config.json")

ollamatools-new/ollamatools/config/schemas.py +0 -163

…	…	@@ -1,163 +0,0 @@
1		"""
2		Configuration schemas for OllamaTools plugins.
3
4		This module provides schema definitions for validating plugin configurations.
5		"""
6
7		from typing import Dict, Any, List, Optional, Set, Union, Callable
8
9		# Schema for the ConversationHistoryPlugin
10		CONVERSATION_HISTORY_SCHEMA = {
11		"history_file": {
12		"type": str,
13		"required": False,
14		"description": "Path to the conversation history file"
15		},
16		"max_history": {
17		"type": int,
18		"required": False,
19		"default": 100,
20		"description": "Maximum number of conversations to keep in history"
21		},
22		"enable_topic_analysis": {
23		"type": bool,
24		"required": False,
25		"default": True,
26		"description": "Whether to enable topic analysis"
27		},
28		"enable_metrics": {
29		"type": bool,
30		"required": False,
31		"default": True,
32		"description": "Whether to enable performance metrics"
33		}
34		}
35
36		# Schema for the SearchPlugin
37		SEARCH_PLUGIN_SCHEMA = {
38		"base_url": {
39		"type": str,
40		"required": False,
41		"default": "http://localhost:8081",
42		"description": "Base URL for the OpenSearchNG service"
43		},
44		"engines": {
45		"type": list,
46		"required": False,
47		"default": ["google", "bing", "duckduckgo", "brave"],
48		"description": "List of search engines to use"
49		},
50		"num_results": {
51		"type": int,
52		"required": False,
53		"default": 5,
54		"description": "Number of search results to return"
55		},
56		"enable_auto_search": {
57		"type": bool,
58		"required": False,
59		"default": True,
60		"description": "Whether to enable automatic search for factual queries"
61		}
62		}
63
64		# Schema for the MatrixAgentAdapterPlugin
65		MATRIX_AGENT_ADAPTER_SCHEMA = {
66		"agent_class": {
67		"type": (str, object), # Can be a string path or a class object
68		"required": True,
69		"description": "Agent class to use (string path or class object)"
70		},
71		"agent_config": {
72		"type": dict,
73		"required": False,
74		"default": {},
75		"description": "Configuration for the agent"
76		}
77		}
78
79		# Schema for the MatrixCorePlugin
80		MATRIX_CORE_PLUGIN_SCHEMA = {
81		"homeserver": {
82		"type": str,
83		"required": True,
84		"description": "URL of the Matrix homeserver"
85		},
86		"user_id": {
87		"type": str,
88		"required": True,
89		"description": "Matrix user ID"
90		},
91		"password": {
92		"type": str,
93		"required": True,
94		"description": "Matrix password"
95		},
96		"device_id": {
97		"type": str,
98		"required": False,
99		"description": "Device ID for E2E encryption"
100		},
101		"ssl": {
102		"type": bool,
103		"required": False,
104		"default": True,
105		"description": "Whether to use SSL for the Matrix connection"
106		},
107		"respond_to_all_messages": {
108		"type": bool,
109		"required": False,
110		"default": False,
111		"description": "Whether to respond to all messages (not just mentions)"
112		}
113		}
114
115		# Schema for the ConversationContextPlugin
116		CONVERSATION_CONTEXT_SCHEMA = {
117		"max_context_length": {
118		"type": int,
119		"required": False,
120		"default": 10,
121		"description": "Maximum number of messages to keep in context"
122		},
123		"context_strategy": {
124		"type": str,
125		"required": False,
126		"default": "sliding_window",
127		"enum": ["sliding_window", "token_based", "time_based", "relevance_based"],
128		"description": "Strategy for managing conversation context"
129		},
130		"token_limit": {
131		"type": int,
132		"required": False,
133		"default": 4096,
134		"description": "Maximum number of tokens to keep in context (for token_based strategy)"
135		},
136		"time_window": {
137		"type": int,
138		"required": False,
139		"default": 3600,
140		"description": "Time window in seconds for context (for time_based strategy)"
141		}
142		}
143
144		# Schema registry
145		SCHEMA_REGISTRY = {
146		"ConversationHistoryPlugin": CONVERSATION_HISTORY_SCHEMA,
147		"SearchPlugin": SEARCH_PLUGIN_SCHEMA,
148		"MatrixAgentAdapterPlugin": MATRIX_AGENT_ADAPTER_SCHEMA,
149		"MatrixCorePlugin": MATRIX_CORE_PLUGIN_SCHEMA,
150		"ConversationContextPlugin": CONVERSATION_CONTEXT_SCHEMA
151		}
152
153		def get_schema_for_plugin(plugin_name):
154		"""
155		Get the configuration schema for a plugin.
156
157		Args:
158		plugin_name: Name of the plugin
159
160		Returns:
161		Dict[str, Any]: Schema dictionary or empty dict if not found
162		"""
163		return SCHEMA_REGISTRY.get(plugin_name, {})

ollamatools-new/ollamatools/core/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/core/agent.py +0 -455

…	…	@@ -1,455 +0,0 @@
1		"""
2		Refactored OllamaAgent implementation for OllamaTools.
3
4		This module provides the OllamaAgent class which serves as the
5		primary interface for interacting with Ollama and executing functions.
6		It has been updated to correctly work with Ollama's function calling API.
7		"""
8
9		import os
10		import re
11		import json
12		import inspect
13		import logging
14		from typing import Dict, Any, List, Optional, Callable, Union, Tuple
15
16		try:
17		import ollama
18		except ImportError:
19		print("Warning: ollama-python not installed. OllamaAgent will not function properly.")
20
21		from ollamatools.core.config import Config
22		from ollamatools.core.security import PermissionManager
23		from ollamatools.plugins.tools.system import execute_command, read_file, write_file, find_files
24		from ollamatools.plugins.tools.web import fetch_url, get_weather, search_web
25		from ollamatools.plugins.tools.doc import generate_markdown, extract_context, summarize_text
26
27		# Set up logging
28		logger = logging.getLogger(__name__)
29
30		class OllamaAgent:
31		"""
32		Agent for interacting with Ollama and executing functions.
33
34		This class serves as the main interface for OllamaTools, providing methods
35		to chat with Ollama and automatically execute functions based on the
36		conversation context.
37		"""
38
39		def __init__(
40		self,
41		model: str = "llama3:latest",
42		host: str = "http://localhost:11434",
43		permission_prompts: bool = True,
44		sudo_always_prompt: bool = True,
45		config: Optional[Config] = None
46		):
47		"""
48		Initialize the OllamaAgent with specified configuration.
49
50		Args:
51		model: Name of the Ollama model to use
52		host: Ollama server address
53		permission_prompts: Whether to prompt for function execution permission
54		sudo_always_prompt: Whether to always prompt for sudo commands
55		config: Configuration object (overrides individual parameters if provided)
56		"""
57		if config:
58		self.config = config
59		else:
60		self.config = Config(
61		model=model,
62		host=host,
63		permission_prompts=permission_prompts,
64		sudo_always_prompt=sudo_always_prompt
65		)
66
67		# Initialize the permission manager
68		self.permission_manager = PermissionManager(
69		prompt_enabled=self.config.permission_prompts,
70		sudo_always_prompt=self.config.sudo_always_prompt
71		)
72
73		# Set up the Ollama client
74		self.client = ollama.Client(host=self.config.host)
75		logger.info(f"Initialized Ollama client with host: {self.config.host}")
76
77		# Initialize registered functions dictionary and collect available tools
78		self.registered_functions = {}
79		self.collect_tools()
80
81		logger.info(f"OllamaAgent initialized with model={self.config.model}, host={self.config.host}")
82
83		def chat(
84		self,
85		message: str,
86		system_prompt: Optional[str] = None,
87		context: Optional[List[Dict[str, str]]] = None
88		) -> str:
89		"""
90		Send a message to Ollama, automatically detecting and executing functions.
91
92		Args:
93		message: User message
94		system_prompt: Optional system prompt
95		context: Optional conversation history
96
97		Returns:
98		str: Response from Ollama, potentially after executing functions
99		"""
100		if context is None:
101		context = []
102
103		logger.info(f"Sending message to Ollama: {message[:50]}...")
104
105		try:
106		# Prepare the chat request
107		messages = context + [{"role": "user", "content": message}]
108		options = {}
109
110		# Add system prompt if provided
111		if system_prompt:
112		options["system"] = system_prompt
113
114		# Generate tool definitions
115		tools = self.generate_tool_definitions()
116
117		# Send the request to Ollama with tools
118		response = self.client.chat(
119		model=self.config.model,
120		messages=messages,
121		options=options,
122		tools=tools, # Pass tools with each request
123		stream=False
124		)
125
126		# Extract the assistant's response
127		assistant_content = ""
128		tool_calls = []
129
130		# Handle different response formats based on Ollama version/client
131		if hasattr(response, 'message'):
132		# Direct object access (newer Ollama client)
133		assistant_content = response.message.content
134		tool_calls = getattr(response.message, 'tool_calls', []) or []
135		else:
136		# Dictionary access (older Ollama client)
137		assistant_message = response.get("message", {})
138		assistant_content = assistant_message.get("content", "")
139		tool_calls = assistant_message.get("tool_calls", [])
140
141		# Check if we have tool calls to process
142		if tool_calls and len(tool_calls) > 0:
143		logger.info(f"Detected {len(tool_calls)} tool calls")
144
145		# Process the function calls
146		tool_results = []
147		for tool_call in tool_calls:
148		# Extract function name and arguments based on response structure
149		if hasattr(tool_call, 'function'):
150		# Direct object access
151		function_name = tool_call.function.name
152		function_args = tool_call.function.arguments
153		if isinstance(function_args, str):
154		function_args = json.loads(function_args)
155		else:
156		# Dictionary access
157		function_data = tool_call.get('function', {})
158		function_name = function_data.get('name', '')
159		function_args = function_data.get('arguments', {})
160		if isinstance(function_args, str):
161		function_args = json.loads(function_args)
162
163		# Check if function exists
164		if function_name not in self.registered_functions:
165		logger.error(f"Function '{function_name}' not found in registered functions")
166		tool_results.append({
167		"error": f"Function '{function_name}' not found"
168		})
169		continue
170
171		# Get the function
172		func = self.registered_functions[function_name]
173
174		# Request permission
175		is_sudo = False
176		if function_name == "execute_command" and "command" in function_args:
177		is_sudo = self.permission_manager.detect_sudo(function_args["command"])
178
179		permission_granted = self.permission_manager.request_permission(
180		function_name=function_name,
181		args=function_args,
182		is_sudo=is_sudo
183		)
184
185		# Execute function if permission granted
186		if permission_granted:
187		try:
188		result = func(**function_args)
189		tool_results.append(result)
190		logger.info(f"Successfully executed {function_name}")
191		except Exception as e:
192		logger.error(f"Error executing {function_name}: {str(e)}")
193		tool_results.append({
194		"error": f"Error: {str(e)}"
195		})
196		else:
197		logger.warning(f"Permission denied for {function_name}")
198		tool_results.append({
199		"error": "Permission denied"
200		})
201
202		# Construct tool messages for the follow-up request
203		tool_messages = []
204		for i, result in enumerate(tool_results):
205		# Get function name from original tool call
206		if hasattr(tool_calls[i], 'function'):
207		function_name = tool_calls[i].function.name
208		else:
209		function_name = tool_calls[i].get('function', {}).get('name', '')
210
211		# Add tool message
212		tool_messages.append({
213		"role": "tool",
214		"name": function_name,
215		"content": json.dumps(result)
216		})
217
218		# Update conversation with function results
219		updated_messages = messages + [
220		{"role": "assistant", "content": assistant_content, "tool_calls": tool_calls}
221		] + tool_messages
222
223		# Send follow-up request with tool results
224		try:
225		follow_up_response = self.client.chat(
226		model=self.config.model,
227		messages=updated_messages,
228		options=options,
229		stream=False
230		)
231
232		# Extract final response
233		if hasattr(follow_up_response, 'message'):
234		# Direct object access
235		final_content = follow_up_response.message.content
236		else:
237		# Dictionary access
238		final_content = follow_up_response.get("message", {}).get("content", "")
239
240		logger.info(f"Received final response after tool execution")
241		return final_content
242		except Exception as e:
243		logger.error(f"Error in follow-up request: {str(e)}")
244		return f"Error processing function results: {str(e)}"
245
246		# Return the assistant's response if no tool calls
247		logger.info(f"Received response from Ollama")
248		return assistant_content
249
250		except Exception as e:
251		logger.error(f"Error in chat: {str(e)}")
252		return f"Error communicating with Ollama: {str(e)}"
253
254		def collect_tools(self):
255		"""
256		Collect all available tools from the tool modules.
257
258		This method scans the tool modules and collects function references
259		to be used when executing tools.
260		"""
261		logger.info("Collecting available tools...")
262
263		# Collect tool functions
264		tool_modules = [
265		("system_tools", [execute_command, read_file, write_file, find_files]),
266		("web_tools", [fetch_url, get_weather, search_web]),
267		("doc_tools", [generate_markdown, extract_context, summarize_text])
268		]
269
270		# Register functions
271		for module_name, functions in tool_modules:
272		for func in functions:
273		if func.__name__.startswith('_'):
274		continue
275
276		# Store the function reference
277		self.registered_functions[func.__name__] = func
278
279		logger.info(f"Collected {len(self.registered_functions)} tools")
280
281		def generate_tool_definitions(self) -> List[Dict[str, Any]]:
282		"""
283		Generate tool definitions for all registered functions.
284
285		Returns:
286		List[Dict[str, Any]]: List of tool definitions in Ollama's format
287		"""
288		tools = []
289
290		for func_name, func in self.registered_functions.items():
291		# Get function docstring
292		func_doc = inspect.getdoc(func) or "No description available"
293
294		# Generate tool definition
295		tool_def = self._generate_tool_definition(func_name, func, func_doc)
296		tools.append(tool_def)
297
298		return tools
299
300		def _generate_tool_definition(self, name: str, func: Callable, docstring: str) -> Dict[str, Any]:
301		"""
302		Generate a tool definition from a function according to Ollama's format.
303
304		Args:
305		name: Function name
306		func: Function object
307		docstring: Function docstring
308
309		Returns:
310		dict: Tool definition compatible with Ollama
311		"""
312		# Extract function signature
313		sig = inspect.signature(func)
314
315		# Generate description
316		description = self._extract_description_from_docstring(docstring)
317
318		# Generate parameters
319		parameters = {
320		"type": "object",
321		"properties": {},
322		"required": []
323		}
324
325		for param_name, param in sig.parameters.items():
326		# Skip permission_manager parameter
327		if param_name == "permission_manager":
328		continue
329
330		# Get parameter type
331		param_type = self._get_param_type(param)
332
333		# Get parameter description
334		param_desc = self._extract_param_description(docstring, param_name)
335
336		# Add parameter to properties
337		parameters["properties"][param_name] = {
338		"type": param_type,
339		"description": param_desc
340		}
341
342		# Add to required list if no default value
343		if param.default is inspect.Parameter.empty:
344		parameters["required"].append(param_name)
345
346		# Create tool definition in Ollama's format
347		tool_def = {
348		"type": "function",
349		"function": {
350		"name": name,
351		"description": description,
352		"parameters": parameters
353		}
354		}
355
356		return tool_def
357
358		def _extract_description_from_docstring(self, docstring: str) -> str:
359		"""
360		Extract the main description from a docstring.
361
362		Args:
363		docstring: Function docstring
364
365		Returns:
366		str: Function description
367		"""
368		if not docstring:
369		return "No description available"
370
371		# Split the docstring into lines
372		lines = docstring.strip().split('\n')
373
374		# Extract the first paragraph
375		description = []
376		for line in lines:
377		line = line.strip()
378		if not line or line.startswith('Args:') or line.startswith('Returns:'):
379		break
380		description.append(line)
381
382		return ' '.join(description) if description else "No description available"
383
384		def _extract_param_description(self, docstring: str, param_name: str) -> str:
385		"""
386		Extract parameter description from docstring.
387
388		Args:
389		docstring: Function docstring
390		param_name: Parameter name
391
392		Returns:
393		str: Parameter description
394		"""
395		if not docstring:
396		return "No description available"
397
398		# Look for parameters section
399		pattern = rf"\s+{re.escape(param_name)}:\s(.?)(?:\n\s+\w+:\|$)"
400		match = re.search(pattern, docstring, re.DOTALL)
401
402		if match:
403		description = match.group(1).strip()
404		return description
405
406		return "No description available"
407
408		def _get_param_type(self, param: inspect.Parameter) -> str:
409		"""
410		Get parameter type for Ollama tool definition.
411
412		Args:
413		param: Parameter object
414
415		Returns:
416		str: Parameter type for Ollama
417		"""
418		annotation = param.annotation
419
420		if annotation is inspect.Parameter.empty:
421		return "string"
422
423		# Convert type annotation to string
424		annotation_str = str(annotation)
425
426		# Determine parameter type
427		if "str" in annotation_str:
428		return "string"
429		elif "int" in annotation_str:
430		return "integer"
431		elif "float" in annotation_str:
432		return "number"
433		elif "bool" in annotation_str:
434		return "boolean"
435		elif "List" in annotation_str or "list" in annotation_str:
436		return "array"
437		elif "Dict" in annotation_str or "dict" in annotation_str:
438		return "object"
439		elif "Optional" in annotation_str:
440		# Handle Optional types
441		if "str" in annotation_str:
442		return "string"
443		elif "int" in annotation_str:
444		return "integer"
445		elif "float" in annotation_str:
446		return "number"
447		elif "bool" in annotation_str:
448		return "boolean"
449		elif "List" in annotation_str or "list" in annotation_str:
450		return "array"
451		elif "Dict" in annotation_str or "dict" in annotation_str:
452		return "object"
453
454		# Default to string for unknown types
455		return "string"

ollamatools-new/ollamatools/core/config.py +0 -107

…	…	@@ -1,107 +0,0 @@
1		"""
2		Configuration management for OllamaTools.
3
4		This module contains the Config class, which handles configuration settings for
5		the OllamaTools library.
6		"""
7
8		import json
9		import os
10		from typing import Optional, Dict, Any
11
12		from pydantic import BaseModel, validator, Field
13
14
15		class Config(BaseModel):
16		"""
17		Configuration class for OllamaTools.
18
19		This class manages configuration settings for the OllamaTools library using
20		Pydantic for validation. It provides methods for loading and saving configuration
21		to a file.
22		"""
23
24		model: str = "llama3.1"
25		host: str = "http://localhost:11434"
26		permission_prompts: bool = True
27		sudo_always_prompt: bool = True
28		timeout_seconds: int = 60
29		max_file_size: Optional[int] = None
30
31		@validator('timeout_seconds')
32		def validate_timeout(cls, v):
33		"""Validate that timeout is positive."""
34		if v <= 0:
35		raise ValueError("Timeout must be positive")
36		return v
37
38		@validator('max_file_size')
39		def validate_max_file_size(cls, v):
40		"""Validate that max_file_size is positive if provided."""
41		if v is not None and v <= 0:
42		raise ValueError("Max file size must be positive")
43		return v
44
45		@classmethod
46		def load_from_file(cls, file_path: str) -> "Config":
47		"""
48		Load configuration from a JSON file.
49
50		Args:
51		file_path: Path to the configuration file
52
53		Returns:
54		Config: Configuration object
55
56		Raises:
57		FileNotFoundError: If the file does not exist
58		json.JSONDecodeError: If the file contains invalid JSON
59		ValueError: If the configuration is invalid
60		"""
61		if not os.path.exists(file_path):
62		raise FileNotFoundError(f"Configuration file not found: {file_path}")
63
64		with open(file_path, 'r') as f:
65		try:
66		config_data = json.load(f)
67		return cls(**config_data)
68		except json.JSONDecodeError:
69		raise json.JSONDecodeError(f"Invalid JSON in configuration file: {file_path}", "", 0)
70
71		def save_to_file(self, file_path: str) -> None:
72		"""
73		Save configuration to a JSON file.
74
75		Args:
76		file_path: Path to the configuration file
77
78		Raises:
79		PermissionError: If the file cannot be written
80		"""
81		try:
82		# Create directory if it doesn't exist
83		os.makedirs(os.path.dirname(os.path.abspath(file_path)), exist_ok=True)
84
85		with open(file_path, 'w') as f:
86		json.dump(self.dict(), f, indent=2)
87		except PermissionError:
88		raise PermissionError(f"Cannot write to configuration file: {file_path}")
89
90		def to_dict(self) -> Dict[str, Any]:
91		"""
92		Convert configuration to a dictionary.
93
94		Returns:
95		Dict[str, Any]: Configuration as a dictionary
96		"""
97		return self.dict(exclude_none=True)
98
99		@classmethod
100		def default_config_path(cls) -> str:
101		"""
102		Get the default configuration file path.
103
104		Returns:
105		str: Default configuration file path
106		"""
107		return os.path.expanduser("~/.config/ollamatools/config.json")

ollamatools-new/ollamatools/core/errors.py +0 -384

…	…	@@ -1,384 +0,0 @@
1		"""
2		Error definitions for OllamaTools.
3
4		This module defines custom exceptions and error utilities for standardized
5		error handling across the OllamaTools library.
6		"""
7
8		from typing import Dict, Any, Optional, List, Tuple, Union
9		import logging
10		import sys
11		import traceback
12
13		# Set up logging
14		logger = logging.getLogger(__name__)
15
16		# Define error codes
17		ERROR_CODES = {
18		# General errors
19		'GENERAL_ERROR': 100,
20		'NOT_IMPLEMENTED': 101,
21		'CONFIGURATION_ERROR': 102,
22		'TIMEOUT_ERROR': 103,
23
24		# Permission errors
25		'PERMISSION_DENIED': 200,
26		'PERMISSION_PROMPT_CANCELLED': 201,
27
28		# Command execution errors
29		'COMMAND_VALIDATION_ERROR': 300,
30		'COMMAND_EXECUTION_ERROR': 301,
31		'COMMAND_NOT_FOUND': 302,
32		'COMMAND_TIMEOUT': 303,
33		'COMMAND_NOT_ALLOWED': 304,
34
35		# File operation errors
36		'FILE_NOT_FOUND': 400,
37		'FILE_READ_ERROR': 401,
38		'FILE_WRITE_ERROR': 402,
39		'FILE_PERMISSION_ERROR': 403,
40		'FILE_SIZE_EXCEEDED': 404,
41		'PATH_TRAVERSAL_ERROR': 405,
42		'PATH_VALIDATION_ERROR': 406,
43		'PATH_OUTSIDE_SANDBOX': 407,
44
45		# Web request errors
46		'URL_VALIDATION_ERROR': 500,
47		'URL_FETCH_ERROR': 501,
48		'URL_CONNECTION_ERROR': 502,
49		'URL_RESPONSE_ERROR': 503,
50		'URL_BLOCKED': 504,
51		'URL_TIMEOUT': 505,
52
53		# API errors
54		'API_ERROR': 600,
55		'API_KEY_MISSING': 601,
56		'API_KEY_INVALID': 602,
57		'API_LIMIT_EXCEEDED': 603,
58		'API_SERVICE_UNAVAILABLE': 604,
59
60		# Ollama client errors
61		'OLLAMA_CONNECTION_ERROR': 700,
62		'OLLAMA_MODEL_ERROR': 701,
63		'OLLAMA_API_ERROR': 702,
64		'OLLAMA_PARSING_ERROR': 703,
65		}
66
67
68		class OllamaToolsError(Exception):
69		"""Base exception class for all OllamaTools errors."""
70
71		def __init__(
72		self,
73		message: str,
74		code: Optional[int] = None,
75		details: Optional[Dict[str, Any]] = None
76		):
77		"""
78		Initialize the exception.
79
80		Args:
81		message: Error message
82		code: Error code
83		details: Additional error details
84		"""
85		self.message = message
86		self.code = code or ERROR_CODES.get('GENERAL_ERROR', 100)
87		self.details = details or {}
88
89		# Construct the full error message
90		full_message = f"[Error {self.code}] {message}"
91		if details:
92		full_message += f" - Details: {details}"
93
94		super().__init__(full_message)
95
96
97		class ConfigurationError(OllamaToolsError):
98		"""Error raised for configuration issues."""
99
100		def __init__(
101		self,
102		message: str,
103		details: Optional[Dict[str, Any]] = None
104		):
105		super().__init__(
106		message,
107		code=ERROR_CODES.get('CONFIGURATION_ERROR'),
108		details=details
109		)
110
111
112		class TimeoutError(OllamaToolsError):
113		"""Error raised when an operation times out."""
114
115		def __init__(
116		self,
117		message: str,
118		timeout: Optional[int] = None,
119		details: Optional[Dict[str, Any]] = None
120		):
121		if timeout is not None:
122		details = details or {}
123		details['timeout'] = timeout
124
125		super().__init__(
126		message,
127		code=ERROR_CODES.get('TIMEOUT_ERROR'),
128		details=details
129		)
130
131
132		class PermissionError(OllamaToolsError):
133		"""Error raised when permission is denied for an operation."""
134
135		def __init__(
136		self,
137		message: str,
138		function_name: Optional[str] = None,
139		args: Optional[Dict[str, Any]] = None,
140		details: Optional[Dict[str, Any]] = None
141		):
142		combined_details = details or {}
143		if function_name:
144		combined_details['function'] = function_name
145		if args:
146		combined_details['args'] = args
147
148		super().__init__(
149		message,
150		code=ERROR_CODES.get('PERMISSION_DENIED'),
151		details=combined_details
152		)
153
154
155		class CommandError(OllamaToolsError):
156		"""Error raised for command execution issues."""
157
158		def __init__(
159		self,
160		message: str,
161		command: Optional[str] = None,
162		args: Optional[List[str]] = None,
163		exit_code: Optional[int] = None,
164		stderr: Optional[str] = None,
165		code: Optional[int] = None,
166		details: Optional[Dict[str, Any]] = None
167		):
168		error_code = code or ERROR_CODES.get('COMMAND_EXECUTION_ERROR')
169
170		combined_details = details or {}
171		if command:
172		combined_details['command'] = command
173		if args:
174		combined_details['args'] = args
175		if exit_code is not None:
176		combined_details['exit_code'] = exit_code
177		if stderr:
178		combined_details['stderr'] = stderr
179
180		super().__init__(
181		message,
182		code=error_code,
183		details=combined_details
184		)
185
186
187		class FileError(OllamaToolsError):
188		"""Error raised for file operation issues."""
189
190		def __init__(
191		self,
192		message: str,
193		path: Optional[str] = None,
194		code: Optional[int] = None,
195		details: Optional[Dict[str, Any]] = None
196		):
197		error_code = code or ERROR_CODES.get('FILE_READ_ERROR')
198
199		combined_details = details or {}
200		if path:
201		combined_details['path'] = path
202
203		super().__init__(
204		message,
205		code=error_code,
206		details=combined_details
207		)
208
209
210		class WebError(OllamaToolsError):
211		"""Error raised for web request issues."""
212
213		def __init__(
214		self,
215		message: str,
216		url: Optional[str] = None,
217		status_code: Optional[int] = None,
218		code: Optional[int] = None,
219		details: Optional[Dict[str, Any]] = None
220		):
221		error_code = code or ERROR_CODES.get('URL_FETCH_ERROR')
222
223		combined_details = details or {}
224		if url:
225		combined_details['url'] = url
226		if status_code is not None:
227		combined_details['status_code'] = status_code
228
229		super().__init__(
230		message,
231		code=error_code,
232		details=combined_details
233		)
234
235
236		class ApiError(OllamaToolsError):
237		"""Error raised for API-related issues."""
238
239		def __init__(
240		self,
241		message: str,
242		api_name: Optional[str] = None,
243		status_code: Optional[int] = None,
244		code: Optional[int] = None,
245		details: Optional[Dict[str, Any]] = None
246		):
247		error_code = code or ERROR_CODES.get('API_ERROR')
248
249		combined_details = details or {}
250		if api_name:
251		combined_details['api'] = api_name
252		if status_code is not None:
253		combined_details['status_code'] = status_code
254
255		super().__init__(
256		message,
257		code=error_code,
258		details=combined_details
259		)
260
261
262		class OllamaClientError(OllamaToolsError):
263		"""Error raised for Ollama client issues."""
264
265		def __init__(
266		self,
267		message: str,
268		model: Optional[str] = None,
269		code: Optional[int] = None,
270		details: Optional[Dict[str, Any]] = None
271		):
272		error_code = code or ERROR_CODES.get('OLLAMA_API_ERROR')
273
274		combined_details = details or {}
275		if model:
276		combined_details['model'] = model
277
278		super().__init__(
279		message,
280		code=error_code,
281		details=combined_details
282		)
283
284
285		def get_error_details(exception: Exception) -> Dict[str, Any]:
286		"""
287		Extract detailed information from an exception.
288
289		Args:
290		exception: The exception to extract details from
291
292		Returns:
293		dict: Error details
294		"""
295		error_type = type(exception).__name__
296		error_message = str(exception)
297
298		# Get the traceback
299		tb = traceback.extract_tb(sys.exc_info()[2])
300		tb_formatted = []
301
302		for frame in tb:
303		tb_formatted.append({
304		'file': frame.filename,
305		'line': frame.lineno,
306		'function': frame.name,
307		'code': frame.line
308		})
309
310		# Create the error details
311		details = {
312		'type': error_type,
313		'message': error_message,
314		'traceback': tb_formatted
315		}
316
317		# Add additional details if it's an OllamaToolsError
318		if isinstance(exception, OllamaToolsError):
319		details['code'] = exception.code
320		if exception.details:
321		details['additional_details'] = exception.details
322
323		return details
324
325
326		def format_error_response(exception: Exception) -> Dict[str, Any]:
327		"""
328		Format an exception as a standardized error response.
329
330		Args:
331		exception: The exception to format
332
333		Returns:
334		dict: Formatted error response
335		"""
336		details = get_error_details(exception)
337
338		# Create a user-friendly error response
339		error_response = {
340		'error': True,
341		'error_type': details['type'],
342		'error_message': details['message']
343		}
344
345		# Add code if available
346		if 'code' in details:
347		error_response['error_code'] = details['code']
348
349		return error_response
350
351
352		def handle_exception(
353		exception: Exception,
354		log_error: bool = True,
355		include_traceback: bool = False
356		) -> Dict[str, Any]:
357		"""
358		Handle an exception and return a standardized error response.
359
360		Args:
361		exception: The exception to handle
362		log_error: Whether to log the error
363		include_traceback: Whether to include traceback in the response
364
365		Returns:
366		dict: Error response
367		"""
368		# Log the error if requested
369		if log_error:
370		error_details = get_error_details(exception)
371		logger.error(
372		f"Error: {error_details['type']}: {error_details['message']}",
373		exc_info=True
374		)
375
376		# Format the error response
377		error_response = format_error_response(exception)
378
379		# Include traceback if requested
380		if include_traceback:
381		error_details = get_error_details(exception)
382		error_response['traceback'] = error_details['traceback']
383
384		return error_response

ollamatools-new/ollamatools/core/security.py +0 -419

…	…	@@ -1,419 +0,0 @@
1		"""
2		Permission management system for OllamaTools.
3
4		This module provides security controls for executing functions,
5		with a focus on a prompt-based permission system.
6		"""
7		import os
8		from typing import Dict, Any, Optional, Tuple, Set
9		import logging
10		import re
11
12		from ollamatools.core.errors import PermissionError, ERROR_CODES
13
14		# Set up logging
15		logger = logging.getLogger(__name__)
16
17		class PermissionManager:
18		"""
19		Manages security permissions for function execution.
20
21		The PermissionManager controls access to potentially sensitive
22		operations such as executing system commands or making web requests.
23		It provides a prompt-based approach to security, allowing users to
24		approve or deny operations at runtime.
25
26		Attributes:
27		prompt_enabled (bool): Whether to prompt for permission before operations
28		sudo_always_prompt (bool): Whether to always prompt for sudo commands
29		session_permissions (Set[str]): Set of operations allowed for the session
30		"""
31
32		def __init__(
33		self,
34		prompt_enabled: bool = True,
35		sudo_always_prompt: bool = True
36		):
37		"""
38		Initialize the permission manager.
39
40		Args:
41		prompt_enabled: Whether to prompt for permissions
42		sudo_always_prompt: Whether to always prompt for sudo commands
43		"""
44		self.prompt_enabled = prompt_enabled
45		self.sudo_always_prompt = sudo_always_prompt
46		self.session_permissions = set() # Store allowed operations for the session
47		self.session_permission_patterns = {
48		"read_file": {}, # Directory-based permissions
49		"write_file": {}, # Directory-based permissions
50		"execute_command": {} # Command-based permissions
51		}
52
53		logger.debug("PermissionManager initialized with prompt_enabled=%s, sudo_always_prompt=%s",
54		prompt_enabled, sudo_always_prompt)
55
56		def _generate_permission_key(self, function_name: str, args: Dict[str, Any]) -> str:
57		"""
58		Generate a more flexible permission key that recognizes similar operations.
59
60		Args:
61		function_name: Name of the function
62		args: Function arguments
63
64		Returns:
65		str: Permission key
66		"""
67		# For commands, focus on the command name rather than all arguments
68		if function_name == "execute_command" and "command" in args:
69		command = args["command"]
70		# Just use the base command without all arguments for matching
71		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
72		return f"{function_name}:command={base_command}"
73
74		# For file operations, focus on the operation type and parent directory
75		if function_name in ("read_file", "write_file") and "path" in args:
76		path = args["path"]
77		# Use parent directory for path-based operations to match similar files
78		parent_dir = os.path.dirname(os.path.abspath(path))
79		return f"{function_name}:dir={parent_dir}"
80
81		# For other operations, use the full args as before
82		args_str = ', '.join(f"{k}={v!r}" for k, v in args.items())
83		return f"{function_name}:{args_str}"
84
85		def request_permission(
86		self,
87		function_name: str,
88		args: Dict[str, Any],
89		is_sudo: bool = False
90		) -> bool:
91		"""
92		Request permission to execute a function.
93
94		This method handles the permission request workflow, which may involve
95		prompting the user for approval depending on configuration settings.
96
97		Args:
98		function_name: Name of the function
99		args: Function arguments
100		is_sudo: Whether the command uses sudo
101
102		Returns:
103		bool: True if permission granted, False otherwise
104
105		Raises:
106		PermissionError: If there's an issue with the permission request
107		"""
108		# Format the arguments as a readable string
109		args_str = ', '.join(f"{k}={v!r}" for k, v in args.items())
110
111		# Generate a permission key for the operation
112		permission_key = self._generate_permission_key(function_name, args)
113
114		# Check if we need to prompt for permission
115		should_prompt = self.prompt_enabled
116
117		# Always prompt for sudo if sudo_always_prompt is True
118		if is_sudo and self.sudo_always_prompt:
119		should_prompt = True
120		logger.info("Forcing permission prompt for sudo command")
121
122		# Check if this operation is already allowed for the session
123		if permission_key in self.session_permissions:
124		logger.debug("Permission already granted for session: %s", permission_key)
125		return True
126
127		# Check specific session permission patterns
128		if function_name == "execute_command" and "command" in args:
129		command = args["command"]
130		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
131		if base_command in self.session_permission_patterns["execute_command"]:
132		logger.debug("Permission already granted for command type: %s", base_command)
133		return True
134
135		elif function_name in ("read_file", "write_file") and "path" in args:
136		path = args["path"]
137		parent_dir = os.path.dirname(os.path.abspath(path))
138		if parent_dir in self.session_permission_patterns[function_name]:
139		logger.debug("Permission already granted for directory: %s", parent_dir)
140		return True
141
142		# If prompting is disabled and not overridden by sudo, grant permission
143		if not should_prompt:
144		logger.debug("Permission prompts disabled, automatically granting permission for: %s", permission_key)
145		return True
146
147		# Display the permission prompt
148		print("\n" + "="*60)
149		print("PERMISSION REQUEST")
150		print("="*60)
151		print(f"Function: {function_name}")
152		print(f"Arguments: {args_str}")
153
154		if is_sudo:
155		print("\nWARNING: This operation requires elevated permissions (sudo)")
156		print(" Be cautious when granting permission for sudo operations")
157
158		print("\nOptions:")
159		print(" [A] Allow: Grant permission for this operation")
160		print(" [S] Allow for Session: Grant permission for similar operations in this session")
161		print(" [M] Modify: Modify the operation before execution")
162		print(" [D] Deny: Reject the operation")
163
164		while True:
165		try:
166		response = input("\nYour choice [A/S/M/D]: ").strip().upper()
167
168		if response == 'A': # Allow once
169		logger.info("Permission granted for: %s", permission_key)
170		return True
171
172		elif response == 'S': # Allow for session
173		self.session_permissions.add(permission_key)
174
175		# Store more specific pattern matches for better session handling
176		if function_name == "execute_command" and "command" in args:
177		command = args["command"]
178		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
179		self.session_permission_patterns["execute_command"][base_command] = True
180		logger.info(f"Command '{base_command}' allowed for session")
181
182		elif function_name in ("read_file", "write_file") and "path" in args:
183		path = args["path"]
184		parent_dir = os.path.dirname(os.path.abspath(path))
185		self.session_permission_patterns[function_name][parent_dir] = True
186		logger.info(f"{function_name} allowed for directory: {parent_dir}")
187
188		logger.info("Permission granted for session: %s", permission_key)
189		return True
190
191		elif response == 'M': # Modify
192		modified_args = self.modify_command(function_name, args)
193		if modified_args is not None:
194		# Generate a new permission key for the modified args
195		modified_args_str = ', '.join(f"{k}={v!r}" for k, v in modified_args.items())
196		modified_key = f"{function_name}:{modified_args_str}"
197
198		# Add to session permissions if it's a modification of an existing permission
199		if permission_key in self.session_permissions:
200		self.session_permissions.add(modified_key)
201
202		# Check if sudo status has changed
203		new_is_sudo = False
204		if function_name == "execute_command" and "command" in modified_args:
205		new_is_sudo = self.detect_sudo(modified_args["command"])
206
207		# Call request_permission again with modified arguments
208		logger.info("Requesting permission for modified operation: %s", modified_key)
209		return self.request_permission(
210		function_name,
211		modified_args,
212		is_sudo=new_is_sudo
213		)
214		else:
215		logger.info("Modification cancelled for: %s", permission_key)
216		print("Modification cancelled.")
217		return False
218
219		elif response == 'D': # Deny
220		logger.info("Permission denied for: %s", permission_key)
221		print("Permission denied.")
222		return False
223
224		else:
225		print("Invalid choice. Please enter A, S, M, or D.")
226		except KeyboardInterrupt:
227		logger.warning("Permission request interrupted")
228		print("\nOperation cancelled.")
229		raise PermissionError(
230		"Permission request interrupted by user",
231		function_name=function_name,
232		args=args,
233		code=ERROR_CODES.get('PERMISSION_PROMPT_CANCELLED')
234		)
235
236		def modify_command(
237		self,
238		function_name: str,
239		args: Dict[str, Any]
240		) -> Optional[Dict[str, Any]]:
241		"""
242		Allow the user to modify command arguments.
243
244		Args:
245		function_name: Name of the function
246		args: Current function arguments
247
248		Returns:
249		Optional[Dict[str, Any]]: Modified arguments or None if cancelled
250
251		Raises:
252		PermissionError: If there's an issue with the modification process
253		"""
254		print("\n" + "="*60)
255		print("MODIFY OPERATION")
256		print("="*60)
257		print(f"Function: {function_name}")
258		print("Current arguments:")
259
260		# Create a copy of the arguments that will be modified
261		modified_args = args.copy()
262
263		try:
264		# Present each argument for modification
265		for key, value in args.items():
266		while True:
267		new_value = input(f" {key} [{value}] (Press Enter to keep, 'C' to cancel): ")
268
269		if not new_value: # Keep current value
270		break
271		elif new_value.upper() == 'C': # Cancel the modification
272		return None
273		else:
274		# Try to convert to the same type as the original value
275		try:
276		# Handle different types appropriately
277		if isinstance(value, bool):
278		# Convert string to boolean
279		if new_value.lower() in ('true', 'yes', '1', 'y'):
280		modified_args[key] = True
281		elif new_value.lower() in ('false', 'no', '0', 'n'):
282		modified_args[key] = False
283		else:
284		print(f"Invalid boolean value. Use 'true', 'false', 'yes', 'no', '1', '0', 'y', or 'n'.")
285		continue
286		elif isinstance(value, int):
287		modified_args[key] = int(new_value)
288		elif isinstance(value, float):
289		modified_args[key] = float(new_value)
290		else:
291		# For strings and other types, use the new value as is
292		modified_args[key] = new_value
293		break
294		except ValueError:
295		print(f"Invalid value for {key}. Expected type: {type(value).__name__}")
296
297		# Show the modified arguments for confirmation
298		print("\nModified arguments:")
299		for key, value in modified_args.items():
300		print(f" {key}: {value}")
301
302		# Confirm the modifications
303		confirm = input("\nConfirm these modifications? [Y/n]: ").strip().lower()
304		if confirm in ('', 'y', 'yes'):
305		logger.info("Modified args confirmed for function: %s", function_name)
306		return modified_args
307		else:
308		logger.info("Modified args rejected for function: %s", function_name)
309		return None
310
311		except KeyboardInterrupt:
312		logger.warning("Command modification interrupted")
313		print("\nModification cancelled.")
314		return None
315		except Exception as e:
316		logger.error("Error modifying command: %s", str(e))
317		raise PermissionError(
318		f"Error modifying command: {str(e)}",
319		function_name=function_name,
320		args=args
321		)
322
323		def detect_sudo(self, command: str) -> bool:
324		"""
325		Detect if a command uses sudo.
326
327		Args:
328		command: Command string
329
330		Returns:
331		bool: True if command uses sudo
332		"""
333		# Basic detection for sudo command
334		if not command:
335		return False
336
337		# Split the command to handle commands with arguments
338		cmd_parts = command.strip().split()
339		if not cmd_parts:
340		return False
341
342		# Check if the command starts with sudo
343		if cmd_parts[0].lower() == "sudo":
344		logger.info("Sudo command detected: %s", command)
345		return True
346
347		# More advanced detection - check for sudo within a command string
348		sudo_pattern = r'(^\|\s+)sudo(\s+\|$)'
349		if re.search(sudo_pattern, command):
350		logger.info("Sudo pattern detected in command: %s", command)
351		return True
352
353		return False
354
355		def clear_session_permissions(self):
356		"""
357		Clear all session permissions.
358		"""
359		num_permissions = len(self.session_permissions)
360		self.session_permissions.clear()
361		# Clear pattern-based permissions too
362		for key in self.session_permission_patterns:
363		self.session_permission_patterns[key] = {}
364		logger.info("Cleared %d session permissions", num_permissions)
365
366		def add_session_permission(self, function_name: str, args: Dict[str, Any]):
367		"""
368		Add a specific operation to the session permissions.
369
370		Args:
371		function_name: Name of the function
372		args: Function arguments
373		"""
374		permission_key = self._generate_permission_key(function_name, args)
375		self.session_permissions.add(permission_key)
376
377		# Add to pattern-based permissions
378		if function_name == "execute_command" and "command" in args:
379		command = args["command"]
380		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
381		self.session_permission_patterns["execute_command"][base_command] = True
382
383		elif function_name in ("read_file", "write_file") and "path" in args:
384		path = args["path"]
385		parent_dir = os.path.dirname(os.path.abspath(path))
386		self.session_permission_patterns[function_name][parent_dir] = True
387
388		logger.info("Added session permission: %s", permission_key)
389
390		def has_session_permission(self, function_name: str, args: Dict[str, Any]) -> bool:
391		"""
392		Check if an operation has session permission.
393
394		Args:
395		function_name: Name of the function
396		args: Function arguments
397
398		Returns:
399		bool: True if the operation has session permission
400		"""
401		# Check specific permission key
402		permission_key = self._generate_permission_key(function_name, args)
403		if permission_key in self.session_permissions:
404		return True
405
406		# Check pattern-based permissions
407		if function_name == "execute_command" and "command" in args:
408		command = args["command"]
409		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
410		if base_command in self.session_permission_patterns["execute_command"]:
411		return True
412
413		elif function_name in ("read_file", "write_file") and "path" in args:
414		path = args["path"]
415		parent_dir = os.path.dirname(os.path.abspath(path))
416		if parent_dir in self.session_permission_patterns[function_name]:
417		return True
418
419		return False

ollamatools-new/ollamatools/core/utils/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/core/utils/formatting.py +0 -28

…	…	@@ -1,28 +0,0 @@
1		"""Text formatting utilities for OllamaTools."""
2
3		def format_markdown(text):
4		"""Format text as markdown."""
5		return text
6
7		def format_code(code, language="python"):
8		"""Format code with syntax highlighting."""
9		return f"```{language}\n{code}\n```"
10
11		def format_table(headers, rows):
12		"""Format data as a markdown table."""
13		if not headers or not rows:
14		return ""
15
16		result = []
17
18		# Add headers
19		result.append("\| " + " \| ".join(headers) + " \|")
20
21		# Add separator
22		result.append("\| " + " \| ".join(["---"] * len(headers)) + " \|")
23
24		# Add rows
25		for row in rows:
26		result.append("\| " + " \| ".join(str(cell) for cell in row) + " \|")
27
28		return "\n".join(result)

ollamatools-new/ollamatools/core/utils/logger.py +0 -144

…	…	@@ -1,144 +0,0 @@
1		"""
2		Logging utilities for OllamaTools.
3
4		This module contains logging utilities used by the OllamaTools library.
5		"""
6
7		import logging
8		import os
9		import sys
10		from typing import Optional, Union, Dict, Any, List
11
12
13		def get_logger(
14		name: str,
15		level: Union[int, str] = logging.INFO,
16		log_file: Optional[str] = None,
17		log_format: Optional[str] = None,
18		propagate: bool = False,
19		additional_handlers: Optional[List[logging.Handler]] = None,
20		extra: Optional[Dict[str, Any]] = None
21		) -> logging.Logger:
22		"""
23		Get a configured logger instance.
24
25		Args:
26		name: Logger name
27		level: Log level (can be integer or string like 'INFO', 'DEBUG')
28		log_file: Optional log file path
29		log_format: Optional custom log format
30		propagate: Whether to propagate logs to parent loggers
31		additional_handlers: Optional list of additional handlers to add
32		extra: Optional extra fields to add to all log records
33
34		Returns:
35		logging.Logger: Configured logger instance
36		"""
37		# Convert string level to integer if needed
38		if isinstance(level, str):
39		level = getattr(logging, level.upper())
40
41		# Create or get the logger
42		logger = logging.getLogger(name)
43		logger.setLevel(level)
44		logger.propagate = propagate
45
46		# Remove existing handlers to avoid duplicates if the logger already exists
47		for handler in logger.handlers[:]:
48		logger.removeHandler(handler)
49
50		# Default format if none provided
51		if log_format is None:
52		log_format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
53
54		formatter = logging.Formatter(log_format)
55
56		# Add console handler
57		console_handler = logging.StreamHandler(stream=sys.stdout)
58		console_handler.setLevel(level)
59		console_handler.setFormatter(formatter)
60		logger.addHandler(console_handler)
61
62		# Add file handler if specified
63		if log_file:
64		# Create directory if it doesn't exist
65		log_dir = os.path.dirname(os.path.abspath(log_file))
66		if log_dir:
67		os.makedirs(log_dir, exist_ok=True)
68
69		file_handler = logging.FileHandler(log_file)
70		file_handler.setLevel(level)
71		file_handler.setFormatter(formatter)
72		logger.addHandler(file_handler)
73
74		# Add any additional handlers
75		if additional_handlers:
76		for handler in additional_handlers:
77		logger.addHandler(handler)
78
79		# Add a filter for extra fields if provided
80		if extra:
81		class ExtraFilter(logging.Filter):
82		def filter(self, record):
83		for key, value in extra.items():
84		setattr(record, key, value)
85		return True
86
87		extra_filter = ExtraFilter()
88		logger.addFilter(extra_filter)
89
90		return logger
91
92
93		def enable_debug_logging(logger_names: Optional[List[str]] = None) -> None:
94		"""
95		Enable debug logging for specified loggers or all loggers.
96
97		Args:
98		logger_names: Optional list of logger names to enable debug logging for,
99		if None, enables for root logger
100		"""
101		if logger_names is None:
102		# Enable debug logging for the root logger
103		logging.getLogger().setLevel(logging.DEBUG)
104		else:
105		# Enable debug logging for specific loggers
106		for name in logger_names:
107		logging.getLogger(name).setLevel(logging.DEBUG)
108
109
110		def disable_logging(logger_names: Optional[List[str]] = None) -> None:
111		"""
112		Disable logging for specified loggers or all loggers.
113
114		Args:
115		logger_names: Optional list of logger names to disable logging for,
116		if None, disables for root logger
117		"""
118		if logger_names is None:
119		# Disable logging for the root logger
120		logging.getLogger().setLevel(logging.CRITICAL + 1)
121		else:
122		# Disable logging for specific loggers
123		for name in logger_names:
124		logging.getLogger(name).setLevel(logging.CRITICAL + 1)
125
126
127		def get_ollamatools_logger(
128		component: str,
129		level: Union[int, str] = logging.INFO,
130		log_file: Optional[str] = None
131		) -> logging.Logger:
132		"""
133		Get a logger for a specific OllamaTools component.
134
135		Args:
136		component: Component name (e.g., 'agent', 'system_tools')
137		level: Log level
138		log_file: Optional log file path
139
140		Returns:
141		logging.Logger: Configured logger instance
142		"""
143		logger_name = f"ollamatools.{component}"
144		return get_logger(logger_name, level, log_file)

ollamatools-new/ollamatools/core/utils/validators/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/core/utils/validators/command.py +0 -326

…	…	@@ -1,326 +0,0 @@
1		"""
2		Command validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing shell commands
5		to prevent command injection and other security vulnerabilities.
6		"""
7
8		import os
9		import re
10		import shlex
11		import logging
12		import pathlib
13		from typing import List, Optional, Dict, Set, Tuple, Union
14
15		# Set up logging
16		logger = logging.getLogger(__name__)
17
18		# Default allowlist of commands
19		DEFAULT_ALLOWLIST = {
20		# File operations
21		'ls', 'dir', 'find', 'grep', 'cat', 'less', 'head', 'tail',
22		# Directory operations
23		'pwd', 'cd', 'mkdir', 'rmdir',
24		# System information
25		'uname', 'hostname', 'whoami', 'id', 'ps', 'top', 'df', 'du', 'free',
26		# Network utilities
27		'ping', 'traceroute', 'curl', 'wget', 'netstat', 'dig', 'nslookup',
28		# Text processing
29		'echo', 'printf', 'sed', 'awk', 'cut', 'sort', 'uniq', 'wc',
30		# File manipulation
31		'cp', 'mv', 'rm', 'touch', 'chmod', 'chown', 'chgrp', 'ln',
32		# Compression
33		'tar', 'gzip', 'gunzip', 'zip', 'unzip',
34		# Package management
35		'apt-get', 'apt', 'yum', 'dnf', 'brew',
36		# Process control
37		'kill', 'pkill', 'pgrep',
38		# Admin
39		'sudo',
40		}
41
42		# Commands that require special validation
43		HIGH_RISK_COMMANDS = {
44		'rm', 'mv', 'cp', # File operations that can cause data loss
45		'chmod', 'chown', 'chgrp', # Permission changes
46		'sudo', # Privilege escalation
47		'wget', 'curl', # Network operations
48		}
49
50		# Paths that should be protected (not allowed to be modified)
51		PROTECTED_PATHS = {
52		'/etc', '/bin', '/sbin', '/usr/bin', '/usr/sbin',
53		'/boot', '/lib', '/lib64', '/usr/lib', '/usr/lib64',
54		'/sys', '/proc', '/dev'
55		}
56
57		# Regular expression patterns for dangerous argument patterns
58		DANGEROUS_ARG_PATTERNS = [
59		r'[\|;&`$()]', # Shell metacharacters
60		r'>(>)?[^>]*', # Redirections
61		r'<(<)?[^<]*', # Input redirections
62		r'\$$[^)]*$', # Command substitution
63		r'`[^`]*`', # Backtick command substitution
64		r'\$\{[^}]*\}', # Variable expansion
65		r'\.\.', # Path traversal
66		r'~[^/]*', # Home directory reference
67		r'\\\\', # Double backslash
68		]
69
70		class CommandValidationError(Exception):
71		"""Exception raised for command validation errors."""
72		pass
73
74
75		def is_command_in_allowlist(command: str, custom_allowlist: Optional[Set[str]] = None) -> bool:
76		"""
77		Check if a command is in the allowlist.
78
79		Args:
80		command: Command to check
81		custom_allowlist: Optional custom allowlist to use instead of the default
82
83		Returns:
84		bool: True if command is in allowlist, False otherwise
85		"""
86		allowlist = custom_allowlist if custom_allowlist is not None else DEFAULT_ALLOWLIST
87
88		# Strip path information for the check
89		cmd_name = os.path.basename(command)
90
91		return cmd_name in allowlist
92
93
94		def validate_command_path(command: str) -> str:
95		"""
96		Validate that a command's path is safe.
97
98		Args:
99		command: Command to validate
100
101		Returns:
102		str: Validated command path
103
104		Raises:
105		CommandValidationError: If the command path is not valid
106		"""
107		# Check for absolute path
108		if os.path.isabs(command):
109		cmd_path = pathlib.Path(command).resolve()
110		cmd_name = os.path.basename(command)
111
112		# Ensure the command exists
113		if not cmd_path.exists():
114		raise CommandValidationError(f"Command '{command}' not found")
115
116		# Ensure the path is to a file not a directory
117		if not cmd_path.is_file():
118		raise CommandValidationError(f"Command path '{command}' is not a file")
119
120		# Check if command in a standard location
121		std_paths = ['/bin', '/usr/bin', '/usr/local/bin', '/sbin', '/usr/sbin']
122		parent_path = str(cmd_path.parent)
123		if not any(parent_path.startswith(p) for p in std_paths):
124		logger.warning(f"Command '{command}' is outside standard paths: {parent_path}")
125
126		# Additional permission check
127		if not os.access(cmd_path, os.X_OK):
128		raise CommandValidationError(f"Command '{command}' is not executable")
129
130		return str(cmd_path)
131		else:
132		# For non-absolute paths, we'll rely on the allowlist check
133		return command
134
135
136		def validate_command_args(args: List[str], command: str) -> List[str]:
137		"""
138		Validate command arguments for security issues.
139
140		Args:
141		args: Command arguments to validate
142		command: The command these args apply to (for context-specific checks)
143
144		Returns:
145		List[str]: Validated arguments
146
147		Raises:
148		CommandValidationError: If any arguments are not valid
149		"""
150		validated_args = []
151
152		# Check if command is high risk
153		is_high_risk = os.path.basename(command) in HIGH_RISK_COMMANDS
154
155		for arg in args:
156		# Skip empty args
157		if not arg:
158		continue
159
160		# Check for dangerous patterns
161		for pattern in DANGEROUS_ARG_PATTERNS:
162		if re.search(pattern, arg):
163		raise CommandValidationError(
164		f"Argument '{arg}' contains dangerous pattern matching '{pattern}'"
165		)
166
167		# Special checks for specific commands
168		cmd_name = os.path.basename(command)
169
170		# Special checks for rm command
171		if cmd_name == 'rm' and (arg == '-rf' or arg == '-fr' or arg == '-r' or arg == '-f'):
172		# These arguments need special attention with protected paths
173		if '-r' in arg or '-f' in arg:
174		logger.warning(f"High-risk argument '{arg}' detected for '{command}'")
175
176		# Check file paths in arguments for protected paths
177		if is_high_risk and arg.startswith('/'):
178		arg_path = pathlib.Path(arg).resolve()
179		arg_path_str = str(arg_path)
180
181		# Check against protected paths
182		for protected in PROTECTED_PATHS:
183		if arg_path_str.startswith(protected):
184		raise CommandValidationError(
185		f"Argument '{arg}' references protected path '{protected}'"
186		)
187
188		validated_args.append(arg)
189
190		return validated_args
191
192
193		def sanitize_command(command: str) -> str:
194		"""
195		Sanitize a command string to prevent command injection.
196
197		Args:
198		command: Command string to sanitize
199
200		Returns:
201		str: Sanitized command string
202		"""
203		# Convert to string and trim
204		if not isinstance(command, str):
205		command = str(command)
206
207		sanitized = command.strip()
208
209		# Remove any null bytes or control characters
210		sanitized = re.sub(r'[\x00-\x1F\x7F]', '', sanitized)
211
212		# Remove shell metacharacters
213		dangerous_chars = r'[;&\|`$()"\'\\]'
214		sanitized = re.sub(dangerous_chars, '', sanitized)
215
216		return sanitized
217
218
219		def validate_and_sanitize_command(
220		command: str,
221		args: Optional[List[str]] = None,
222		custom_allowlist: Optional[Set[str]] = None
223		) -> Tuple[str, List[str]]:
224		"""
225		Validate and sanitize a command and its arguments.
226
227		Args:
228		command: Command to validate and sanitize
229		args: Optional command arguments
230		custom_allowlist: Optional custom allowlist
231
232		Returns:
233		tuple: (sanitized_command, sanitized_args)
234
235		Raises:
236		CommandValidationError: If the command or arguments are not valid
237		"""
238		# Handle None args
239		if args is None:
240		args = []
241
242		# Handle case where command contains both command and arguments as a string
243		if isinstance(command, str) and ' ' in command and args is None:
244		try:
245		# Split using shlex to handle quotes properly
246		command_parts = shlex.split(command)
247		command = command_parts[0]
248		args = command_parts[1:] if len(command_parts) > 1 else []
249		logger.debug(f"Split command into cmd='{command}', args={args}")
250		except ValueError as e:
251		raise CommandValidationError(f"Invalid command syntax: {str(e)}")
252
253		# Check if command is in allowlist
254		if not is_command_in_allowlist(command, custom_allowlist):
255		raise CommandValidationError(
256		f"Command '{command}' not in allowed command list"
257		)
258
259		# Validate command path
260		validated_command = validate_command_path(command)
261
262		# Validate and sanitize args
263		validated_args = validate_command_args(args, validated_command)
264
265		return validated_command, validated_args
266
267
268		def is_sudo_command(command: str, args: Optional[List[str]] = None) -> bool:
269		"""
270		Check if a command uses sudo.
271
272		Args:
273		command: Command to check
274		args: Optional command arguments
275
276		Returns:
277		bool: True if the command uses sudo, False otherwise
278		"""
279		if args is None:
280		args = []
281
282		# Check if command is sudo directly
283		if os.path.basename(command) == 'sudo':
284		return True
285
286		# Check if command path contains sudo
287		if 'sudo' in command.split(os.path.sep):
288		return True
289
290		# Check if sudo appears in the full command string
291		full_command = f"{command} {' '.join(args)}"
292		if full_command.lstrip().startswith('sudo '):
293		return True
294
295		return False
296
297
298		def get_secure_env() -> Dict[str, str]:
299		"""
300		Get a sanitized environment dictionary for command execution.
301
302		Returns:
303		dict: Sanitized environment variables
304		"""
305		# Start with a minimal environment
306		secure_env = {
307		'PATH': '/usr/local/bin:/usr/bin:/bin',
308		'LANG': os.environ.get('LANG', 'C.UTF-8'),
309		'LC_ALL': os.environ.get('LC_ALL', 'C.UTF-8'),
310		'TERM': os.environ.get('TERM', 'xterm'),
311		'HOME': os.environ.get('HOME', '/tmp'),
312		'USER': os.environ.get('USER', 'nobody'),
313		}
314
315		# Copy allowed environment variables
316		allowed_vars = {
317		'DISPLAY', 'EDITOR', 'PAGER', 'TZ',
318		'HTTP_PROXY', 'HTTPS_PROXY', 'NO_PROXY',
319		'http_proxy', 'https_proxy', 'no_proxy'
320		}
321
322		for var in allowed_vars:
323		if var in os.environ:
324		secure_env[var] = os.environ[var]
325
326		return secure_env

ollamatools-new/ollamatools/core/utils/validators/path.py +0 -419

…	…	@@ -1,419 +0,0 @@
1		"""
2		Path validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing file paths
5		to prevent path traversal attacks and other file path-related security issues.
6		"""
7
8		import os
9		import re
10		import pathlib
11		from typing import Optional, List, Set, Dict, Any, Union, Tuple
12		import logging
13
14		# Set up logging
15		logger = logging.getLogger(__name__)
16
17		# Default protected paths - should not be modified
18		DEFAULT_PROTECTED_PATHS = {
19		'/etc',
20		'/bin',
21		'/sbin',
22		'/usr/bin',
23		'/usr/sbin',
24		'/usr/lib',
25		'/usr/lib64',
26		'/boot',
27		'/dev',
28		'/proc',
29		'/sys',
30		'/var/lib',
31		'/var/log',
32		'/var/run',
33		'/var/lock',
34		}
35
36		# Default paths to allow even if they're in protected paths
37		DEFAULT_ALLOWED_PATHS = {
38		'/etc/hosts', # commonly read for network configuration
39		'/etc/hostname', # commonly read for system identification
40		'/etc/passwd', # commonly read for user information (non-sensitive part)
41		}
42
43		# Default sandbox path - where operations are restricted to
44		DEFAULT_SANDBOX_PATH = os.path.expanduser('~')
45
46		class PathValidationError(Exception):
47		"""Exception raised for path validation errors."""
48		pass
49
50
51		def normalize_path(path: str) -> str:
52		"""
53		Normalize a file path to resolve symlinks, relative paths, etc.
54
55		Args:
56		path: File path to normalize
57
58		Returns:
59		str: Normalized absolute path
60		"""
61		# Convert to string if it's not already
62		if not isinstance(path, str):
63		path = str(path)
64
65		# Remove any null bytes
66		path = path.replace('\0', '')
67
68		# Trim whitespace
69		path = path.strip()
70
71		# Create an absolute path
72		abs_path = os.path.abspath(path)
73
74		# Normalize the path (resolves ., .., symlinks)
75		try:
76		norm_path = os.path.normpath(abs_path)
77		return norm_path
78		except Exception as e:
79		raise PathValidationError(f"Path normalization error: {str(e)}")
80
81
82		def resolve_symlinks(path: str) -> str:
83		"""
84		Resolve symlinks in a path.
85
86		Args:
87		path: File path that might contain symlinks
88
89		Returns:
90		str: Path with all symlinks resolved
91		"""
92		try:
93		# Use pathlib to resolve symlinks
94		resolved_path = str(pathlib.Path(path).resolve())
95		return resolved_path
96		except Exception as e:
97		raise PathValidationError(f"Failed to resolve symlinks in path '{path}': {str(e)}")
98
99
100		def is_path_traversal_attempt(path: str) -> bool:
101		"""
102		Check if a path contains traversal patterns that might be used in attacks.
103
104		Args:
105		path: File path to check
106
107		Returns:
108		bool: True if the path contains suspicious patterns, False otherwise
109		"""
110		# Normalize for easy pattern matching
111		normalized = os.path.normpath(path)
112
113		# Check for common traversal patterns
114		suspicious_patterns = [
115		r'\.\./', # Parent directory references
116		r'/\.\./', # Parent directory references
117		r'/\.\.$', # Path ending with parent reference
118		r'^\.\./.*', # Path starting with parent reference
119		r'\\\\', # Double backslash (Windows)
120		r'\.\\', # Current directory backslash (Windows)
121		r'//', # Double forward slash
122		r'\0', # Null byte
123		r'%00', # URL-encoded null byte
124		r'%2e%2e%2f', # URL-encoded ../
125		r'%2e%2e/', # URL-encoded ../
126		r'\$\{.*?\}', # Shell variable expansion
127		r'`.*?`', # Command substitution
128		]
129
130		for pattern in suspicious_patterns:
131		if re.search(pattern, path):
132		return True
133
134		return False
135
136
137		def is_path_in_sandbox(
138		path: str,
139		sandbox_path: Optional[str] = None,
140		additional_allowed_paths: Optional[Set[str]] = None
141		) -> bool:
142		"""
143		Check if a path is within the sandbox path or explicitly allowed.
144
145		Args:
146		path: File path to check
147		sandbox_path: Base path that defines the sandbox (default is user's home)
148		additional_allowed_paths: Additional paths to allow
149
150		Returns:
151		bool: True if the path is in the sandbox or allowed, False otherwise
152		"""
153		# Resolve the sandbox path
154		if sandbox_path is None:
155		sandbox_path = DEFAULT_SANDBOX_PATH
156
157		# Normalize all paths for comparison
158		normalized_path = normalize_path(path)
159		normalized_sandbox = normalize_path(sandbox_path)
160
161		# Check if the path is explicitly allowed
162		allowed_paths = DEFAULT_ALLOWED_PATHS.copy()
163		if additional_allowed_paths:
164		allowed_paths.update(additional_allowed_paths)
165
166		if normalized_path in allowed_paths:
167		return True
168
169		# Check if the path is a subdirectory of the sandbox
170		return normalized_path.startswith(normalized_sandbox)
171
172
173		def is_path_protected(
174		path: str,
175		additional_protected_paths: Optional[Set[str]] = None,
176		allowed_paths: Optional[Set[str]] = None
177		) -> bool:
178		"""
179		Check if a path is in a protected area that should not be modified.
180
181		Args:
182		path: File path to check
183		additional_protected_paths: Additional paths to protect
184		allowed_paths: Paths explicitly allowed even if in protected areas
185
186		Returns:
187		bool: True if the path is protected, False otherwise
188		"""
189		# Normalize the path
190		normalized_path = normalize_path(path)
191
192		# Combine default and additional protected paths
193		protected_paths = DEFAULT_PROTECTED_PATHS.copy()
194		if additional_protected_paths:
195		protected_paths.update(additional_protected_paths)
196
197		# Combine default and additional allowed paths
198		all_allowed_paths = DEFAULT_ALLOWED_PATHS.copy()
199		if allowed_paths:
200		all_allowed_paths.update(allowed_paths)
201
202		# First check if the path is explicitly allowed
203		if normalized_path in all_allowed_paths:
204		return False
205
206		# Then check if the path is in a protected area
207		for protected in protected_paths:
208		if normalized_path == protected or normalized_path.startswith(protected + os.path.sep):
209		return True
210
211		return False
212
213
214		def validate_path_for_reading(
215		path: str,
216		sandbox_path: Optional[str] = None,
217		additional_allowed_paths: Optional[Set[str]] = None,
218		resolve_links: bool = True
219		) -> str:
220		"""
221		Validate a path for reading operations (e.g., read_file).
222
223		Args:
224		path: File path to validate
225		sandbox_path: Base path that defines the sandbox
226		additional_allowed_paths: Additional paths to allow
227		resolve_links: Whether to resolve symlinks
228
229		Returns:
230		str: Validated path
231
232		Raises:
233		PathValidationError: If the path does not pass validation
234		"""
235		# Check for empty path
236		if not path:
237		raise PathValidationError("Path cannot be empty")
238
239		# Normalize path
240		normalized_path = normalize_path(path)
241
242		# Check for path traversal attempts
243		if is_path_traversal_attempt(path):
244		raise PathValidationError(f"Path '{path}' contains suspicious traversal patterns")
245
246		# Resolve symlinks if requested
247		if resolve_links:
248		normalized_path = resolve_symlinks(normalized_path)
249
250		# Check if path exists (for reading, the file should exist)
251		if not os.path.exists(normalized_path):
252		raise PathValidationError(f"Path '{path}' does not exist")
253
254		# Check if path is a file (for reading, it should be a file not a directory)
255		if os.path.isdir(normalized_path):
256		raise PathValidationError(f"Path '{path}' is a directory, not a file")
257
258		# Check if path is in sandbox or explicitly allowed
259		if not is_path_in_sandbox(normalized_path, sandbox_path, additional_allowed_paths):
260		raise PathValidationError(f"Path '{path}' is outside the allowed directories")
261
262		return normalized_path
263
264
265		def validate_path_for_writing(
266		path: str,
267		sandbox_path: Optional[str] = None,
268		additional_allowed_paths: Optional[Set[str]] = None,
269		additional_protected_paths: Optional[Set[str]] = None,
270		resolve_links: bool = True
271		) -> str:
272		"""
273		Validate a path for writing operations (e.g., write_file).
274
275		Args:
276		path: File path to validate
277		sandbox_path: Base path that defines the sandbox
278		additional_allowed_paths: Additional paths to allow
279		additional_protected_paths: Additional paths to protect
280		resolve_links: Whether to resolve symlinks
281
282		Returns:
283		str: Validated path
284
285		Raises:
286		PathValidationError: If the path does not pass validation
287		"""
288		# Check for empty path
289		if not path:
290		raise PathValidationError("Path cannot be empty")
291
292		# Normalize path
293		normalized_path = normalize_path(path)
294
295		# Check for path traversal attempts
296		if is_path_traversal_attempt(path):
297		raise PathValidationError(f"Path '{path}' contains suspicious traversal patterns")
298
299		# Resolve symlinks if requested
300		if resolve_links:
301		normalized_path = resolve_symlinks(normalized_path)
302
303		# Check if path is protected (for writing, protected paths are off-limits)
304		if is_path_protected(normalized_path, additional_protected_paths, additional_allowed_paths):
305		raise PathValidationError(f"Path '{path}' is in a protected area")
306
307		# Check if path is in sandbox or explicitly allowed
308		if not is_path_in_sandbox(normalized_path, sandbox_path, additional_allowed_paths):
309		raise PathValidationError(f"Path '{path}' is outside the allowed directories")
310
311		# Ensure the directory exists
312		directory = os.path.dirname(normalized_path)
313		if not os.path.exists(directory):
314		try:
315		# Create the directory if it doesn't exist
316		os.makedirs(directory, exist_ok=True)
317		except Exception as e:
318		raise PathValidationError(f"Cannot create directory for path '{path}': {str(e)}")
319
320		return normalized_path
321
322
323		def validate_path_for_searching(
324		directory: str,
325		pattern: str,
326		sandbox_path: Optional[str] = None,
327		additional_allowed_paths: Optional[Set[str]] = None,
328		resolve_links: bool = True
329		) -> Tuple[str, str]:
330		"""
331		Validate a directory and pattern for file searching operations (e.g., find_files).
332
333		Args:
334		directory: Directory to search in
335		pattern: File pattern to search for
336		sandbox_path: Base path that defines the sandbox
337		additional_allowed_paths: Additional paths to allow
338		resolve_links: Whether to resolve symlinks
339
340		Returns:
341		tuple: (validated_directory, validated_pattern)
342
343		Raises:
344		PathValidationError: If the directory or pattern does not pass validation
345		"""
346		# Check for empty directory or pattern
347		if not directory:
348		raise PathValidationError("Directory cannot be empty")
349
350		if not pattern:
351		raise PathValidationError("Pattern cannot be empty")
352
353		# Normalize directory path
354		normalized_dir = normalize_path(directory)
355
356		# Check for path traversal attempts in directory
357		if is_path_traversal_attempt(directory):
358		raise PathValidationError(f"Directory '{directory}' contains suspicious traversal patterns")
359
360		# Resolve symlinks if requested
361		if resolve_links:
362		normalized_dir = resolve_symlinks(normalized_dir)
363
364		# Check if directory exists
365		if not os.path.exists(normalized_dir):
366		raise PathValidationError(f"Directory '{directory}' does not exist")
367
368		# Check if path is a directory
369		if not os.path.isdir(normalized_dir):
370		raise PathValidationError(f"Path '{directory}' is not a directory")
371
372		# Check if directory is in sandbox or explicitly allowed
373		if not is_path_in_sandbox(normalized_dir, sandbox_path, additional_allowed_paths):
374		raise PathValidationError(f"Directory '{directory}' is outside the allowed directories")
375
376		# Validate and sanitize the pattern
377		# Remove any path components from the pattern
378		sanitized_pattern = os.path.basename(pattern)
379
380		# Check for suspicious patterns in the pattern
381		suspicious_pattern_chars = r'[&\|;`$()<>]'
382		if re.search(suspicious_pattern_chars, sanitized_pattern):
383		raise PathValidationError(f"Pattern '{pattern}' contains suspicious characters")
384
385		return normalized_dir, sanitized_pattern
386
387
388		def configure_path_sandbox(
389		sandbox_path: Optional[str] = None,
390		additional_allowed_paths: Optional[Set[str]] = None,
391		additional_protected_paths: Optional[Set[str]] = None
392		) -> Dict[str, Union[str, Set[str]]]:
393		"""
394		Configure the path sandbox with custom settings.
395
396		Args:
397		sandbox_path: Base path that defines the sandbox
398		additional_allowed_paths: Additional paths to allow
399		additional_protected_paths: Additional paths to protect
400
401		Returns:
402		dict: Configuration dictionary with sandbox settings
403		"""
404		# Start with default values
405		config = {
406		'sandbox_path': sandbox_path or DEFAULT_SANDBOX_PATH,
407		'allowed_paths': DEFAULT_ALLOWED_PATHS.copy(),
408		'protected_paths': DEFAULT_PROTECTED_PATHS.copy()
409		}
410
411		# Add additional allowed paths
412		if additional_allowed_paths:
413		config['allowed_paths'].update(additional_allowed_paths)
414
415		# Add additional protected paths
416		if additional_protected_paths:
417		config['protected_paths'].update(additional_protected_paths)
418
419		return config

ollamatools-new/ollamatools/core/utils/validators/url.py +0 -321

…	…	@@ -1,321 +0,0 @@
1		"""
2		URL validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing URLs
5		to prevent security vulnerabilities like SSRF attacks and URL injection.
6		"""
7
8		import re
9		import ipaddress
10		from urllib.parse import urlparse, parse_qs, urlencode, urlunparse, unquote
11		import logging
12		from typing import Dict, Any, List, Set, Optional, Tuple, Union
13
14		# Set up logging
15		logger = logging.getLogger(__name__)
16
17		# Default allowed schemes
18		DEFAULT_ALLOWED_SCHEMES = {'http', 'https'}
19
20		# Default blocklist of domains
21		DEFAULT_DOMAIN_BLOCKLIST = {
22		'localhost', '127.0.0.1', '0.0.0.0', '::1', # Localhost variations
23		'169.254.169.254', # AWS metadata endpoint
24		'169.254.170.2', # AWS ECS metadata endpoint
25		'169.254.170.23', # AWS Lambda metadata endpoint
26		'10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16', # Private IP ranges
27		'fc00::/7', # Unique local addresses (IPv6)
28		'fd00::/8', # Unique local addresses (IPv6)
29		'fe80::/10', # Link-local addresses (IPv6)
30		}
31
32		# Default allowlist of domains (overrides blocklist)
33		DEFAULT_DOMAIN_ALLOWLIST = set()
34
35		# Suspicious URL patterns
36		SUSPICIOUS_URL_PATTERNS = [
37		r'(file\|ftp\|gopher\|data\|telnet\|ldap\|ssh)://', # Non-HTTP protocols
38		r'@', # URL with credentials
39		r'\.\.', # Directory traversal
40		r'<\|>\|\{\|\}\|\[\|\]\|\\|', # Special characters
41		]
42
43		class URLValidationError(Exception):
44		"""Exception raised for URL validation errors."""
45		pass
46
47
48		def normalize_url(url: str) -> str:
49		"""
50		Normalize a URL to prevent bypass techniques.
51
52		Args:
53		url: URL to normalize
54
55		Returns:
56		str: Normalized URL
57		"""
58		# Handle empty URLs
59		if not url:
60		raise URLValidationError("URL cannot be empty")
61
62		# Ensure URL is a string
63		url = str(url).strip()
64
65		# Remove control characters and whitespace
66		url = re.sub(r'[\x00-\x20\x7F]', '', url)
67
68		try:
69		# Parse the URL
70		parsed = urlparse(url)
71
72		# Check if scheme and netloc exist
73		if not parsed.scheme or not parsed.netloc:
74		raise URLValidationError(f"Invalid URL: {url} - missing scheme or host")
75
76		# Decode any URL-encoded components that might be hiding malicious content
77		path = unquote(parsed.path)
78
79		# Normalize path to remove traversal sequences
80		path = normalize_path(path)
81
82		# Parse and rebuild query parameters to normalize
83		query_params = parse_qs(parsed.query, keep_blank_values=True)
84		normalized_query = urlencode(query_params, doseq=True)
85
86		# Rebuild the normalized URL
87		normalized_url = urlunparse((
88		parsed.scheme.lower(), # Normalize scheme to lowercase
89		parsed.netloc.lower(), # Normalize domain to lowercase
90		path,
91		parsed.params,
92		normalized_query,
93		'' # Remove fragment as it's client-side only
94		))
95
96		return normalized_url
97
98		except Exception as e:
99		raise URLValidationError(f"URL normalization error: {str(e)}")
100
101
102		def normalize_path(path: str) -> str:
103		"""
104		Normalize a URL path to remove traversal sequences and other issues.
105
106		Args:
107		path: URL path to normalize
108
109		Returns:
110		str: Normalized path
111		"""
112		# Remove consecutive slashes
113		path = re.sub(r'/+', '/', path)
114
115		# Remove trailing slash for consistency unless the path is just "/"
116		if path != '/' and path.endswith('/'):
117		path = path[:-1]
118
119		# Process path components to resolve traversal sequences
120		# This is complex, so we'll do it manually
121		components = path.split('/')
122		normalized_components = []
123
124		for component in components:
125		if component == '' or component == '.':
126		# Skip empty components and current directory references
127		continue
128		elif component == '..':
129		# Go up one directory level
130		if normalized_components:
131		normalized_components.pop()
132		else:
133		normalized_components.append(component)
134
135		# Rebuild the path
136		normalized_path = '/' + '/'.join(normalized_components)
137
138		# Ensure path starts with "/"
139		if not normalized_path.startswith('/'):
140		normalized_path = '/' + normalized_path
141
142		return normalized_path
143
144
145		def is_ip_address_in_blocklist(ip_str: str, blocklist: Set[str]) -> bool:
146		"""
147		Check if an IP address is in the blocklist.
148
149		Args:
150		ip_str: IP address string
151		blocklist: Set of blocked IP addresses or CIDR ranges
152
153		Returns:
154		bool: True if the IP is blocked, False otherwise
155		"""
156		try:
157		# Parse the IP address
158		ip = ipaddress.ip_address(ip_str)
159
160		# Check if the IP is directly in the blocklist
161		if ip_str in blocklist:
162		return True
163
164		# Check if the IP is in any of the blocklisted CIDR ranges
165		for block in blocklist:
166		if '/' in block: # It's a CIDR block
167		try:
168		network = ipaddress.ip_network(block, strict=False)
169		if ip in network:
170		return True
171		except ValueError:
172		continue
173
174		return False
175		except ValueError:
176		# Not a valid IP address
177		return False
178
179
180		def validate_url(
181		url: str,
182		allowed_schemes: Optional[Set[str]] = None,
183		domain_allowlist: Optional[Set[str]] = None,
184		domain_blocklist: Optional[Set[str]] = None,
185		) -> str:
186		"""
187		Validate a URL against security rules.
188
189		Args:
190		url: URL to validate
191		allowed_schemes: Optional set of allowed URL schemes (defaults to http, https)
192		domain_allowlist: Optional set of allowed domains
193		domain_blocklist: Optional set of blocked domains
194
195		Returns:
196		str: Validated and normalized URL
197
198		Raises:
199		URLValidationError: If the URL does not pass validation
200		"""
201		# Use default values if not provided
202		if allowed_schemes is None:
203		allowed_schemes = DEFAULT_ALLOWED_SCHEMES
204
205		if domain_blocklist is None:
206		domain_blocklist = DEFAULT_DOMAIN_BLOCKLIST
207
208		if domain_allowlist is None:
209		domain_allowlist = DEFAULT_DOMAIN_ALLOWLIST
210
211		# Normalize the URL
212		try:
213		normalized_url = normalize_url(url)
214		except URLValidationError as e:
215		raise URLValidationError(f"URL normalization failed: {str(e)}")
216
217		# Parse the normalized URL
218		parsed = urlparse(normalized_url)
219
220		# Check scheme
221		if parsed.scheme not in allowed_schemes:
222		raise URLValidationError(
223		f"URL scheme '{parsed.scheme}' not allowed. Allowed schemes: {allowed_schemes}"
224		)
225
226		# Extract hostname
227		hostname = parsed.netloc.split(':')[0].lower()
228
229		# Check allowlist first (override blocklist)
230		if domain_allowlist and hostname in domain_allowlist:
231		return normalized_url
232
233		# Check hostname against blocklist
234		if hostname in domain_blocklist:
235		raise URLValidationError(f"Domain '{hostname}' is blocked")
236
237		# Check for IP address
238		try:
239		ip = ipaddress.ip_address(hostname)
240
241		# Check if IP is in blocklist
242		if is_ip_address_in_blocklist(str(ip), domain_blocklist):
243		raise URLValidationError(f"IP address '{ip}' is blocked")
244
245		# Check for localhost/internal IP
246		if ip.is_loopback or ip.is_private or ip.is_reserved or ip.is_link_local:
247		raise URLValidationError(f"IP address '{ip}' is internal/reserved")
248		except ValueError:
249		# Not an IP address, continue with domain checks
250		pass
251
252		# Check for suspicious patterns
253		for pattern in SUSPICIOUS_URL_PATTERNS:
254		if re.search(pattern, normalized_url):
255		raise URLValidationError(f"URL contains suspicious pattern: {pattern}")
256
257		# Check port if specified
258		if ':' in parsed.netloc:
259		try:
260		port = int(parsed.netloc.split(':')[1])
261		if port < 1 or port > 65535:
262		raise URLValidationError(f"Invalid port number: {port}")
263		except (IndexError, ValueError):
264		raise URLValidationError(f"Invalid port specification in URL: {parsed.netloc}")
265
266		return normalized_url
267
268
269		def configure_domain_lists(
270		additional_allowlist: Optional[Set[str]] = None,
271		additional_blocklist: Optional[Set[str]] = None
272		) -> Tuple[Set[str], Set[str]]:
273		"""
274		Configure domain allowlist and blocklist by adding custom entries.
275
276		Args:
277		additional_allowlist: Additional domains to allow
278		additional_blocklist: Additional domains to block
279
280		Returns:
281		tuple: (configured_allowlist, configured_blocklist)
282		"""
283		# Start with default lists
284		allowlist = DEFAULT_DOMAIN_ALLOWLIST.copy()
285		blocklist = DEFAULT_DOMAIN_BLOCKLIST.copy()
286
287		# Add additional entries
288		if additional_allowlist:
289		allowlist.update(additional_allowlist)
290
291		if additional_blocklist:
292		blocklist.update(additional_blocklist)
293
294		return allowlist, blocklist
295
296
297		def create_url_validator(
298		allowed_schemes: Optional[Set[str]] = None,
299		domain_allowlist: Optional[Set[str]] = None,
300		domain_blocklist: Optional[Set[str]] = None,
301		) -> callable:
302		"""
303		Create a URL validator function with the specified configuration.
304
305		Args:
306		allowed_schemes: Set of allowed URL schemes
307		domain_allowlist: Set of allowed domains
308		domain_blocklist: Set of blocked domains
309
310		Returns:
311		callable: URL validator function
312		"""
313		def validate(url: str) -> str:
314		return validate_url(
315		url,
316		allowed_schemes=allowed_schemes,
317		domain_allowlist=domain_allowlist,
318		domain_blocklist=domain_blocklist
319		)
320
321		return validate

ollamatools-new/ollamatools/plugins/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/plugins/base.py +0 -58

…	…	@@ -1,58 +0,0 @@
1		"""Base plugin interfaces for the OllamaTools plugin system."""
2
3		from abc import ABC, abstractmethod
4		from typing import Any, Dict, List, Optional
5
6		class Plugin(ABC):
7		"""Base class for all plugins."""
8
9		@property
10		@abstractmethod
11		def name(self) -> str:
12		"""Return the name of the plugin."""
13		pass
14
15		@property
16		@abstractmethod
17		def version(self) -> str:
18		"""Return the version of the plugin."""
19		pass
20
21		@abstractmethod
22		def initialize(self, config: Dict[str, Any]) -> None:
23		"""Initialize the plugin with the given configuration."""
24		pass
25
26		@abstractmethod
27		def shutdown(self) -> None:
28		"""Shutdown the plugin and release any resources."""
29		pass
30
31		class ToolPlugin(Plugin):
32		"""Base class for tool plugins."""
33
34		@abstractmethod
35		def get_tools(self) -> List[Dict[str, Any]]:
36		"""Return the list of tools provided by this plugin."""
37		pass
38
39		class StandardPlugin(Plugin):
40		"""Base class for standard plugins."""
41
42		@abstractmethod
43		def process_message(self, message: Dict[str, Any]) -> Dict[str, Any]:
44		"""Process a message and return the result."""
45		pass
46
47		class IntegrationPlugin(Plugin):
48		"""Base class for integration plugins."""
49
50		@abstractmethod
51		def connect(self) -> bool:
52		"""Connect to the external service."""
53		pass
54
55		@abstractmethod
56		def disconnect(self) -> None:
57		"""Disconnect from the external service."""
58		pass

ollamatools-new/ollamatools/plugins/errors.py +0 -142

…	…	@@ -1,142 +0,0 @@
1		"""
2		Error handling for the OllamaTools plugin system.
3
4		This module provides exception classes and utility functions for handling
5		errors in the OllamaTools plugin system.
6		"""
7
8		import sys
9		import logging
10		import traceback
11		from typing import Dict, Any, Optional
12
13		# Configure logging
14		logger = logging.getLogger(__name__)
15
16		class PluginError(Exception):
17		"""Base class for all plugin-related errors."""
18
19		def __init__(self, message, details=None):
20		"""
21		Initialize the error.
22
23		Args:
24		message: Error message
25		details: Optional dictionary with additional error details
26		"""
27		super().__init__(message)
28		self.details = details or {}
29
30
31		class PluginNotFoundError(PluginError):
32		"""Error raised when a plugin is not found."""
33		pass
34
35
36		class PluginInitializationError(PluginError):
37		"""Error raised when a plugin fails to initialize."""
38		pass
39
40
41		class PluginDependencyError(PluginError):
42		"""Error raised when a plugin dependency cannot be satisfied."""
43		pass
44
45
46		class PluginExecutionError(PluginError):
47		"""Error raised when a plugin operation fails during execution."""
48		pass
49
50
51		class PluginConfigurationError(PluginError):
52		"""Error raised when a plugin has invalid configuration."""
53		pass
54
55
56		def format_plugin_error(error: PluginError) -> str:
57		"""
58		Format a plugin error into a user-friendly message.
59
60		Args:
61		error: The plugin error
62
63		Returns:
64		str: Formatted error message
65		"""
66		message = str(error)
67
68		if isinstance(error, PluginNotFoundError):
69		return f"Plugin not found: {message}"
70
71		if isinstance(error, PluginInitializationError):
72		return f"Failed to initialize plugin: {message}"
73
74		if isinstance(error, PluginDependencyError):
75		dependency = error.details.get("dependency", "unknown")
76		return f"Plugin dependency error: {message} (missing: {dependency})"
77
78		if isinstance(error, PluginExecutionError):
79		plugin = error.details.get("plugin", "unknown")
80		operation = error.details.get("operation", "unknown operation")
81		return f"Plugin execution error in {plugin}.{operation}: {message}"
82
83		if isinstance(error, PluginConfigurationError):
84		plugin = error.details.get("plugin", "unknown")
85		return f"Invalid configuration for plugin {plugin}: {message}"
86
87		# Generic plugin error
88		return f"Plugin error: {message}"
89
90
91		def log_plugin_error(error: PluginError, level=logging.ERROR) -> None:
92		"""
93		Log a plugin error with appropriate context.
94
95		Args:
96		error: The plugin error
97		level: Logging level
98		"""
99		# Format the error message
100		message = format_plugin_error(error)
101
102		# Add details if available
103		if error.details:
104		details_str = ", ".join(f"{k}={v}" for k, v in error.details.items())
105		message = f"{message} [Details: {details_str}]"
106
107		# Log the error
108		logger.log(level, message, exc_info=True)
109
110
111		def handle_plugin_exception(error: Exception) -> Dict[str, Any]:
112		"""
113		Handle a plugin exception and return error details.
114
115		Args:
116		error: The exception to handle
117
118		Returns:
119		Dict[str, Any]: Error details
120		"""
121		error_type = type(error).__name__
122		error_message = str(error)
123		error_traceback = traceback.format_exc()
124
125		# Log the error
126		if isinstance(error, PluginError):
127		log_plugin_error(error)
128		else:
129		logger.error(f"Unexpected error: {error_type}: {error_message}", exc_info=True)
130
131		# Create error details
132		error_details = {
133		"error_type": error_type,
134		"error_message": error_message,
135		"traceback": error_traceback
136		}
137
138		# Add plugin-specific details
139		if isinstance(error, PluginError) and error.details:
140		error_details["details"] = error.details
141
142		return error_details

ollamatools-new/ollamatools/plugins/integrations/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/plugins/integrations/matrix/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/plugins/integrations/matrix/client.py +0 -800

…	…	@@ -1,800 +0,0 @@
1		"""
2		Matrix integration tools for OllamaTools.
3
4		This module provides functionality to connect to Matrix homeservers,
5		join rooms, send messages, and listen for incoming messages.
6		It integrates with Ollama to process messages and generate responses.
7
8		Dependencies:
9		- matrix-nio: For Matrix protocol support
10		- matrix-nio[e2e]: For end-to-end encryption support (optional)
11		- pyyaml: For configuration handling
12		"""
13
14		import os
15		import sys
16		import yaml
17		import logging
18		import asyncio
19		import re
20		from typing import Dict, List, Optional, Union, Callable, Any, Set
21		from collections import defaultdict
22
23		# Configure logging
24		logger = logging.getLogger(__name__)
25
26		try:
27		from nio import (
28		AsyncClient,
29		MatrixRoom,
30		RoomMessageText,
31		LoginResponse,
32		JoinError
33		)
34		except ImportError:
35		raise ImportError(
36		"Matrix support requires the 'matrix-nio' package. "
37		"Please install it with 'pip install matrix-nio' or "
38		"'pip install matrix-nio[e2e]' for end-to-end encryption support."
39		)
40
41		# Make ollama import optional
42		try:
43		from ollama import AsyncClient as OllamaAsync
44		OLLAMA_AVAILABLE = True
45		except ImportError:
46		OLLAMA_AVAILABLE = False
47		logger.warning(
48		"The 'ollama' package is not installed. "
49		"Message processing with Ollama will not be available. "
50		"Install it with 'pip install ollama'."
51		)
52
53		class MatrixClient:
54		"""
55		A client for connecting to Matrix homeservers and interacting with rooms.
56
57		This client can connect to a Matrix homeserver, join rooms, send messages,
58		and listen for incoming messages. It integrates with Ollama to process
59		messages and generate responses.
60		"""
61
62		# Command patterns
63		MODEL_COMMAND_PATTERN = re.compile(r'^/model\s+([a-zA-Z0-9:_\-\.]+)$')
64		HELP_COMMAND_PATTERN = re.compile(r'^/help$')
65		LIST_MODELS_COMMAND_PATTERN = re.compile(r'^/models$')
66
67		def __init__(
68		self,
69		homeserver: str,
70		user_id: str,
71		password: str,
72		device_id: str = None,
73		ollama_host: str = "localhost:11434",
74		ollama_model: str = "llama3:latest",
75		ollama_prompt: str = None,
76		available_models: List[str] = None,
77		admin_users: List[str] = None,
78		ssl: bool = True
79		):
80		"""
81		Initialize a new Matrix client.
82
83		Args:
84		homeserver: The URL of the Matrix homeserver
85		user_id: The Matrix user ID (e.g., @user:example.com)
86		password: The password for the Matrix account
87		device_id: A unique identifier for this device (for E2E encryption)
88		ollama_host: The host address for the Ollama API
89		ollama_model: The Ollama model to use for generating responses
90		ollama_prompt: An optional system prompt for the Ollama model
91		available_models: List of available Ollama models
92		admin_users: List of Matrix user IDs that can change models for all users
93		ssl: Whether to use SSL for the Matrix connection
94		"""
95		self.homeserver = homeserver
96		self.user_id = user_id
97		self.password = password
98		self.device_id = device_id
99		self.ollama_host = ollama_host
100		self.ollama_model = ollama_model
101		self.ollama_prompt = ollama_prompt
102		self.ssl = ssl
103
104		# Initialize the Matrix client
105		self.client = AsyncClient(
106		homeserver=self.homeserver,
107		device_id=self.device_id,
108		ssl=self.ssl
109		)
110
111		# Track when the client connected to avoid processing old messages
112		self.connection_time = None
113
114		# Track joined rooms
115		self.rooms = []
116
117		# Message callback
118		self.message_callback = None
119
120		# User context tracking
121		self.user_contexts = defaultdict(list)
122		self.max_context_length = 10 # Maximum number of messages to keep in context
123
124		# User model preferences
125		self.user_models = {}
126
127		# Available models
128		self.available_models = available_models or [ollama_model]
129
130		# Admin users who can change models
131		self.admin_users = set(admin_users or [])
132
133		async def connect(self) -> bool:
134		"""
135		Connect to the Matrix homeserver and log in.
136
137		Returns:
138		bool: True if the connection was successful, False otherwise
139		"""
140		try:
141		# Extract username from user_id (e.g., "@user:example.com" -> "user")
142		username = self.user_id.split(':')[0].lstrip('@')
143
144		# Set the user ID for the client
145		self.client.user = username
146
147		response = await self.client.login(
148		password=self.password
149		)
150
151		if isinstance(response, LoginResponse):
152		logger.info(f"Successfully connected to {self.homeserver} as {self.user_id}")
153		self.connection_time = asyncio.get_event_loop().time()
154		return True
155		else:
156		logger.error(f"Failed to connect to {self.homeserver}: {response}")
157		return False
158
159		except Exception as e:
160		logger.error(f"Error connecting to Matrix server: {e}")
161		return False
162
163		async def join_room(self, room_id: str) -> bool:
164		"""
165		Join a Matrix room.
166
167		Args:
168		room_id: The ID of the room to join (e.g., #room:example.com)
169
170		Returns:
171		bool: True if joining was successful, False otherwise
172		"""
173		try:
174		response = await self.client.join(room_id)
175
176		if isinstance(response, JoinError):
177		logger.error(f"Failed to join room {room_id}: {response}")
178		return False
179
180		logger.info(f"Successfully joined room {room_id}")
181		self.rooms.append(room_id)
182		return True
183
184		except Exception as e:
185		logger.error(f"Error joining room {room_id}: {e}")
186		return False
187
188		async def send_message(self, room_id: str, message: str) -> bool:
189		"""
190		Send a message to a Matrix room.
191
192		Args:
193		room_id: The ID of the room to send the message to
194		message: The message to send
195
196		Returns:
197		bool: True if sending was successful, False otherwise
198		"""
199		try:
200		response = await self.client.room_send(
201		room_id=room_id,
202		message_type="m.room.message",
203		content={
204		"msgtype": "m.text",
205		"body": message
206		}
207		)
208
209		logger.debug(f"Sent message to room {room_id}")
210		return True
211
212		except Exception as e:
213		logger.error(f"Error sending message to room {room_id}: {e}")
214		return False
215
216		async def process_with_ollama(self, message: str, sender: str = None, room_id: str = None) -> str:
217		"""
218		Process a message with Ollama and return the response.
219
220		Args:
221		message: The message to process
222		sender: The Matrix user ID of the sender
223		room_id: The ID of the room where the message was sent
224
225		Returns:
226		str: The response from Ollama
227		"""
228		if not OLLAMA_AVAILABLE:
229		return "Ollama is not available. Please install the 'ollama' package with 'pip install ollama'."
230
231		try:
232		ollama_client = OllamaAsync(host=self.ollama_host)
233
234		# Determine which model to use
235		model = self.ollama_model
236		if sender and sender in self.user_models:
237		model = self.user_models[sender]
238
239		messages = []
240
241		# Add system prompt if provided
242		if self.ollama_prompt:
243		messages.append({
244		"role": "system",
245		"content": self.ollama_prompt
246		})
247
248		# Add conversation context if available
249		if sender and room_id:
250		context_key = f"{sender}:{room_id}"
251		for ctx_msg in self.user_contexts[context_key]:
252		messages.append(ctx_msg)
253
254		# Add user message
255		user_message = {
256		"role": "user",
257		"content": message
258		}
259		messages.append(user_message)
260
261		# Store user message in context
262		if sender and room_id:
263		context_key = f"{sender}:{room_id}"
264		self.user_contexts[context_key].append(user_message)
265		# Trim context if it gets too long
266		if len(self.user_contexts[context_key]) > self.max_context_length:
267		self.user_contexts[context_key] = self.user_contexts[context_key][-self.max_context_length:]
268
269		response = await ollama_client.chat(
270		model=model,
271		messages=messages
272		)
273
274		response_message = response["message"]
275		message_content = response_message["content"]
276
277		# Store assistant response in context
278		if sender and room_id:
279		context_key = f"{sender}:{room_id}"
280		self.user_contexts[context_key].append({
281		"role": "assistant",
282		"content": message_content
283		})
284		# Trim context if it gets too long
285		if len(self.user_contexts[context_key]) > self.max_context_length:
286		self.user_contexts[context_key] = self.user_contexts[context_key][-self.max_context_length:]
287
288		logger.debug(f"Received response from Ollama (model: {model}): {message_content}")
289		return message_content
290
291		except Exception as e:
292		logger.error(f"Error processing message with Ollama: {e}")
293		return f"Error processing message: {e}"
294
295		async def _message_callback(self, room: MatrixRoom, event: RoomMessageText) -> None:
296		"""
297		Internal callback for handling incoming messages.
298
299		Args:
300		room: The room the message was sent in
301		event: The message event
302		"""
303		if not isinstance(event, RoomMessageText):
304		return
305
306		# Get message timestamp
307		message_timestamp = event.server_timestamp / 1000
308
309		# Skip messages from before the client connected
310		if self.connection_time and message_timestamp < self.connection_time:
311		logger.debug("Skipping old message")
312		return
313
314		# Skip messages from the bot itself
315		if event.sender == self.user_id:
316		logger.debug("Skipping message from self")
317		return
318
319		message = event.body
320		room_id = room.room_id
321		sender = event.sender
322
323		logger.info(f"Received message in room {room_id} from {sender}: {message}")
324
325		# If a custom callback is set, use it
326		if self.message_callback:
327		await self.message_callback(room, event, self)
328		return
329
330		# Check if this is a command
331		if message.startswith('/'):
332		await self._handle_command(message, room_id, sender)
333		return
334
335		# Extract username from user_id (e.g., "@user:example.com" -> "user")
336		username = self.user_id.split(':')[0].lstrip('@')
337
338		# Check if the message contains the bot's username (case insensitive)
339		if username.lower() in message.lower():
340		logger.info(f"Username mention detected in message: {message}")
341		# Process with Ollama and send response
342		response = await self.process_with_ollama(message, sender, room_id)
343		await self.send_message(room_id, response)
344		return
345
346		# For other messages without username mention, don't automatically respond
347
348		async def _handle_command(self, message: str, room_id: str, sender: str) -> None:
349		"""
350		Handle commands sent to the bot.
351
352		Args:
353		message: The command message
354		room_id: The ID of the room where the command was sent
355		sender: The Matrix user ID of the sender
356		"""
357		# Check if this is a model change command
358		model_match = self.MODEL_COMMAND_PATTERN.match(message)
359		if model_match:
360		model_name = model_match.group(1)
361		await self._change_model(model_name, room_id, sender)
362		return
363
364		# Check if this is a help command
365		if self.HELP_COMMAND_PATTERN.match(message):
366		await self._send_help(room_id)
367		return
368
369		# Check if this is a list models command
370		if self.LIST_MODELS_COMMAND_PATTERN.match(message):
371		await self._list_models(room_id)
372		return
373
374		# Unknown command
375		await self.send_message(room_id, f"Unknown command: {message}\nType /help for available commands.")
376
377		async def _change_model(self, model_name: str, room_id: str, sender: str) -> None:
378		"""
379		Change the Ollama model for a user.
380
381		Args:
382		model_name: The name of the model to change to
383		room_id: The ID of the room where the command was sent
384		sender: The Matrix user ID of the sender
385		"""
386		# Check if the model is available
387		if model_name not in self.available_models:
388		await self.send_message(
389		room_id,
390		f"Model '{model_name}' is not available. Available models: {', '.join(self.available_models)}"
391		)
392		return
393
394		# Change the model for the user
395		self.user_models[sender] = model_name
396
397		# Clear the user's context
398		for key in list(self.user_contexts.keys()):
399		if key.startswith(f"{sender}:"):
400		self.user_contexts[key] = []
401
402		await self.send_message(room_id, f"Changed model to '{model_name}' for {sender}")
403		logger.info(f"Changed model to '{model_name}' for user {sender}")
404
405		async def _send_help(self, room_id: str) -> None:
406		"""
407		Send help information to a room.
408
409		Args:
410		room_id: The ID of the room to send the help to
411		"""
412		help_text = """
413		Available commands:
414		- /help - Show this help message
415		- /models - List available models
416		- /model <model_name> - Change to a different model
417
418		You can also mention me by my username in your message and I'll respond.
419		For example: "Hey {username}, what's the weather like today?"
420		""".format(username=self.user_id.split(':')[0].lstrip('@'))
421		await self.send_message(room_id, help_text.strip())
422
423		async def _list_models(self, room_id: str) -> None:
424		"""
425		List available models to a room.
426
427		Args:
428		room_id: The ID of the room to send the list to
429		"""
430		models_text = f"Available models: {', '.join(self.available_models)}"
431		await self.send_message(room_id, models_text)
432
433		async def get_available_models(self) -> List[str]:
434		"""
435		Get a list of available Ollama models.
436
437		Returns:
438		List[str]: A list of available model names
439		"""
440		if not OLLAMA_AVAILABLE:
441		return [self.ollama_model]
442
443		try:
444		ollama_client = OllamaAsync(host=self.ollama_host)
445		response = await ollama_client.list()
446		models = [model["name"] for model in response["models"]]
447		return models
448		except Exception as e:
449		logger.error(f"Error getting available models: {e}")
450		return [self.ollama_model]
451
452		async def update_available_models(self) -> None:
453		"""
454		Update the list of available models from the Ollama server.
455		"""
456		models = await self.get_available_models()
457		self.available_models = models
458		logger.info(f"Updated available models: {', '.join(models)}")
459
460		def add_admin_user(self, user_id: str) -> None:
461		"""
462		Add a user to the list of admin users.
463
464		Args:
465		user_id: The Matrix user ID to add as an admin
466		"""
467		self.admin_users.add(user_id)
468		logger.info(f"Added admin user: {user_id}")
469
470		def remove_admin_user(self, user_id: str) -> None:
471		"""
472		Remove a user from the list of admin users.
473
474		Args:
475		user_id: The Matrix user ID to remove as an admin
476		"""
477		if user_id in self.admin_users:
478		self.admin_users.remove(user_id)
479		logger.info(f"Removed admin user: {user_id}")
480
481		def set_message_callback(self, callback: Callable) -> None:
482		"""
483		Set a custom callback for handling incoming messages.
484
485		The callback should be an async function that takes three arguments:
486		- room: The MatrixRoom object
487		- event: The RoomMessageText event
488		- client: This MatrixClient instance
489
490		Args:
491		callback: The callback function
492		"""
493		self.message_callback = callback
494
495		async def listen(self) -> None:
496		"""
497		Start listening for messages in joined rooms.
498
499		This method will block until the client is stopped.
500		"""
501		# Set up the message callback
502		self.client.add_event_callback(self._message_callback, RoomMessageText)
503
504		# Start syncing with the server
505		await self.client.sync_forever(timeout=30000, full_state=True)
506
507		async def disconnect(self) -> None:
508		"""
509		Disconnect from the Matrix homeserver.
510		"""
511		await self.client.logout()
512		await self.client.close()
513		logger.info(f"Disconnected from {self.homeserver}")
514
515
516		class MatrixConfig:
517		"""
518		Configuration handler for Matrix integration.
519		"""
520
521		@staticmethod
522		def load_config(config_path: str) -> Dict:
523		"""
524		Load configuration from a YAML file.
525
526		Args:
527		config_path: Path to the configuration file
528
529		Returns:
530		Dict: The configuration as a dictionary
531		"""
532		try:
533		with open(config_path, 'r') as f:
534		config = yaml.safe_load(f)
535
536		logger.info(f"Loaded configuration from {config_path}")
537		return config
538
539		except Exception as e:
540		logger.error(f"Error loading configuration from {config_path}: {e}")
541		raise
542
543		@staticmethod
544		def save_config(config: Dict, config_path: str) -> None:
545		"""
546		Save configuration to a YAML file.
547
548		Args:
549		config: The configuration as a dictionary
550		config_path: Path to save the configuration to
551		"""
552		try:
553		# Create directory if it doesn't exist
554		os.makedirs(os.path.dirname(config_path), exist_ok=True)
555
556		with open(config_path, 'w') as f:
557		yaml.dump(config, f, default_flow_style=False)
558
559		logger.info(f"Saved configuration to {config_path}")
560
561		except Exception as e:
562		logger.error(f"Error saving configuration to {config_path}: {e}")
563		raise
564
565		@staticmethod
566		def create_default_config() -> Dict:
567		"""
568		Create a default configuration.
569
570		Returns:
571		Dict: The default configuration
572		"""
573		return {
574		"MATRIX": {
575		"HOMESERVER": "https://matrix.example.org",
576		"USER": "@user:example.org",
577		"PASSWORD": "your-password",
578		"DEVICE_ID": "ollamatools-matrix",
579		"ROOMS": [
580		"#room:example.org"
581		],
582		"ADMIN_USERS": [
583		"@admin:example.org"
584		]
585		},
586		"OLLAMA": {
587		"HOST": "localhost:11434",
588		"MODEL": "llama3:latest",
589		"PROMPT": "You are a helpful assistant.",
590		"AVAILABLE_MODELS": [
591		"llama3:latest",
592		"mistral:latest",
593		"gemma:latest"
594		]
595		}
596		}
597
598
599		# Async helper functions for the module API
600
601		async def _connect_to_matrix(
602		homeserver: str,
603		user_id: str,
604		password: str,
605		device_id: str = None,
606		ollama_host: str = "localhost:11434",
607		ollama_model: str = "llama3:latest",
608		ollama_prompt: str = None,
609		available_models: List[str] = None,
610		admin_users: List[str] = None,
611		ssl: bool = True
612		) -> Optional[MatrixClient]:
613		"""
614		Connect to a Matrix homeserver.
615
616		Args:
617		homeserver: The URL of the Matrix homeserver
618		user_id: The Matrix user ID (e.g., @user:example.com)
619		password: The password for the Matrix account
620		device_id: A unique identifier for this device (for E2E encryption)
621		ollama_host: The host address for the Ollama API
622		ollama_model: The Ollama model to use for generating responses
623		ollama_prompt: An optional system prompt for the Ollama model
624		available_models: List of available Ollama models
625		admin_users: List of Matrix user IDs that can change models for all users
626		ssl: Whether to use SSL for the Matrix connection
627
628		Returns:
629		Optional[MatrixClient]: A connected Matrix client, or None if connection failed
630		"""
631		client = MatrixClient(
632		homeserver=homeserver,
633		user_id=user_id,
634		password=password,
635		device_id=device_id,
636		ollama_host=ollama_host,
637		ollama_model=ollama_model,
638		ollama_prompt=ollama_prompt,
639		available_models=available_models,
640		admin_users=admin_users,
641		ssl=ssl
642		)
643
644		success = await client.connect()
645
646		if success:
647		return client
648		else:
649		return None
650
651		async def _join_matrix_room(client: MatrixClient, room_id: str) -> bool:
652		"""
653		Join a Matrix room.
654
655		Args:
656		client: A connected Matrix client
657		room_id: The ID of the room to join (e.g., #room:example.com)
658
659		Returns:
660		bool: True if joining was successful, False otherwise
661		"""
662		return await client.join_room(room_id)
663
664		async def _send_matrix_message(client: MatrixClient, room_id: str, message: str) -> bool:
665		"""
666		Send a message to a Matrix room.
667
668		Args:
669		client: A connected Matrix client
670		room_id: The ID of the room to send the message to
671		message: The message to send
672
673		Returns:
674		bool: True if sending was successful, False otherwise
675		"""
676		return await client.send_message(room_id, message)
677
678		async def _listen_for_matrix_messages(client: MatrixClient) -> None:
679		"""
680		Start listening for messages in joined rooms.
681
682		Args:
683		client: A connected Matrix client
684		"""
685		await client.listen()
686
687
688		# Synchronous wrapper functions for the module API
689
690		def connect_to_matrix(
691		homeserver: str,
692		user_id: str,
693		password: str,
694		device_id: str = None,
695		ollama_host: str = "localhost:11434",
696		ollama_model: str = "llama3:latest",
697		ollama_prompt: str = None,
698		available_models: List[str] = None,
699		admin_users: List[str] = None,
700		ssl: bool = True
701		) -> Optional[MatrixClient]:
702		"""
703		Connect to a Matrix homeserver.
704
705		Args:
706		homeserver: The URL of the Matrix homeserver
707		user_id: The Matrix user ID (e.g., @user:example.com)
708		password: The password for the Matrix account
709		device_id: A unique identifier for this device (for E2E encryption)
710		ollama_host: The host address for the Ollama API
711		ollama_model: The Ollama model to use for generating responses
712		ollama_prompt: An optional system prompt for the Ollama model
713		available_models: List of available Ollama models
714		admin_users: List of Matrix user IDs that can change models for all users
715		ssl: Whether to use SSL for the Matrix connection
716
717		Returns:
718		Optional[MatrixClient]: A connected Matrix client, or None if connection failed
719		"""
720		return asyncio.run(_connect_to_matrix(
721		homeserver=homeserver,
722		user_id=user_id,
723		password=password,
724		device_id=device_id,
725		ollama_host=ollama_host,
726		ollama_model=ollama_model,
727		ollama_prompt=ollama_prompt,
728		available_models=available_models,
729		admin_users=admin_users,
730		ssl=ssl
731		))
732
733		def join_matrix_room(client: MatrixClient, room_id: str) -> bool:
734		"""
735		Join a Matrix room.
736
737		Args:
738		client: A connected Matrix client
739		room_id: The ID of the room to join (e.g., #room:example.com)
740
741		Returns:
742		bool: True if joining was successful, False otherwise
743		"""
744		return asyncio.run(_join_matrix_room(client, room_id))
745
746		def send_matrix_message(client: MatrixClient, room_id: str, message: str) -> bool:
747		"""
748		Send a message to a Matrix room.
749
750		Args:
751		client: A connected Matrix client
752		room_id: The ID of the room to send the message to
753		message: The message to send
754
755		Returns:
756		bool: True if sending was successful, False otherwise
757		"""
758		return asyncio.run(_send_matrix_message(client, room_id, message))
759
760		def listen_for_matrix_messages(client: MatrixClient) -> None:
761		"""
762		Start listening for messages in joined rooms.
763
764		This function will block until the client is stopped.
765
766		Args:
767		client: A connected Matrix client
768		"""
769		asyncio.run(_listen_for_matrix_messages(client))
770
771		def load_matrix_config(config_path: str) -> Dict:
772		"""
773		Load Matrix configuration from a YAML file.
774
775		Args:
776		config_path: Path to the configuration file
777
778		Returns:
779		Dict: The configuration as a dictionary
780		"""
781		return MatrixConfig.load_config(config_path)
782
783		def save_matrix_config(config: Dict, config_path: str) -> None:
784		"""
785		Save Matrix configuration to a YAML file.
786
787		Args:
788		config: The configuration as a dictionary
789		config_path: Path to save the configuration to
790		"""
791		MatrixConfig.save_config(config, config_path)
792
793		def create_default_matrix_config() -> Dict:
794		"""
795		Create a default Matrix configuration.
796
797		Returns:
798		Dict: The default configuration
799		"""
800		return MatrixConfig.create_default_config()

ollamatools-new/ollamatools/plugins/integrations/matrix/commands.py +0 -226

…	…	@@ -1,226 +0,0 @@
1		"""
2		Enhanced Matrix integration tools for OllamaTools.
3
4		This module extends the basic Matrix integration with advanced features from
5		the modular and extended agents, including:
6		- Improved message response logic
7		- Natural language understanding
8		- Search integration
9		- Conversation history tracking and summarization
10		- Topic analysis
11		- Performance metrics
12
13		Dependencies:
14		- matrix-nio: For Matrix protocol support
15		- matrix-nio[e2e]: For end-to-end encryption support (optional)
16		- pyyaml: For configuration handling
17		- ollama: For Ollama API access
18		"""
19
20		import os
21		import sys
22		import yaml
23		import json
24		import asyncio
25		import logging
26		from typing import Dict, List, Optional, Union, Callable, Any, Set, Tuple
27
28		# Import from core module
29		from ollamatools.tools.enhanced_matrix_core import (
30		EnhancedMatrixClient,
31		_connect_enhanced_matrix
32		)
33
34		# Import from search module
35		from ollamatools.plugins.integrations.matrix.search import (
36		MODULAR_AGENT_AVAILABLE,
37		OPENSEARCH_AVAILABLE,
38		get_default_opensearch_config,
39		execute_search_for_message,
40		execute_search_query,
41		handle_search_command
42		)
43
44		# Import from history module
45		from ollamatools.plugins.integrations.matrix.history import (
46		load_conversation_history,
47		save_conversation_history,
48		record_conversation,
49		analyze_conversation_topics,
50		get_performance_metrics,
51		handle_history_command,
52		handle_topics_command,
53		handle_metrics_command
54		)
55
56		# Import from base matrix tools
57		from ollamatools.plugins.integrations.matrix.client import (
58		MatrixConfig,
59		load_matrix_config,
60		save_matrix_config,
61		create_default_matrix_config,
62		RoomMessageText
63		)
64
65		# Configure logging
66		logger = logging.getLogger(__name__)
67
68		# Synchronous wrapper functions for the module API
69
70		def connect_enhanced_matrix(
71		homeserver: str,
72		user_id: str,
73		password: str,
74		device_id: str = None,
75		ollama_host: str = "localhost:11434",
76		ollama_model: str = "llama3:latest",
77		ollama_prompt: str = None,
78		available_models: List[str] = None,
79		admin_users: List[str] = None,
80		opensearch_config: Dict[str, Any] = None,
81		ssl: bool = True,
82		enable_search: bool = True,
83		enable_conversation_tracking: bool = True,
84		max_context_length: int = 10,
85		max_conversation_history: int = 100,
86		respond_to_all_messages: bool = False
87		) -> Optional[EnhancedMatrixClient]:
88		"""
89		Connect to a Matrix homeserver with enhanced features.
90
91		Args:
92		homeserver: The URL of the Matrix homeserver
93		user_id: The Matrix user ID (e.g., @user:example.com)
94		password: The password for the Matrix account
95		device_id: A unique identifier for this device (for E2E encryption)
96		ollama_host: The host address for the Ollama API
97		ollama_model: The Ollama model to use for generating responses
98		ollama_prompt: An optional system prompt for the Ollama model
99		available_models: List of available Ollama models
100		admin_users: List of Matrix user IDs that can change models for all users
101		opensearch_config: Configuration for OpenSearchNG
102		ssl: Whether to use SSL for the Matrix connection
103		enable_search: Whether to enable search functionality
104		enable_conversation_tracking: Whether to enable conversation tracking
105		max_context_length: Maximum number of messages to keep in context
106		max_conversation_history: Maximum number of conversations to keep in history
107		respond_to_all_messages: Whether to respond to all messages (not just mentions)
108
109		Returns:
110		Optional[EnhancedMatrixClient]: A connected Enhanced Matrix client, or None if connection failed
111		"""
112		return asyncio.run(_connect_enhanced_matrix(
113		homeserver=homeserver,
114		user_id=user_id,
115		password=password,
116		device_id=device_id,
117		ollama_host=ollama_host,
118		ollama_model=ollama_model,
119		ollama_prompt=ollama_prompt,
120		available_models=available_models,
121		admin_users=admin_users,
122		opensearch_config=opensearch_config,
123		ssl=ssl,
124		enable_search=enable_search,
125		enable_conversation_tracking=enable_conversation_tracking,
126		max_context_length=max_context_length,
127		max_conversation_history=max_conversation_history,
128		respond_to_all_messages=respond_to_all_messages
129		))
130
131		def join_enhanced_matrix_room(client: EnhancedMatrixClient, room_id: str) -> bool:
132		"""
133		Join a Matrix room with an Enhanced Matrix client.
134
135		Args:
136		client: A connected Enhanced Matrix client
137		room_id: The ID of the room to join (e.g., #room:example.com)
138
139		Returns:
140		bool: True if joining was successful, False otherwise
141		"""
142		return asyncio.run(client.join_room(room_id))
143
144		def send_enhanced_matrix_message(client: EnhancedMatrixClient, room_id: str, message: str) -> bool:
145		"""
146		Send a message to a Matrix room with an Enhanced Matrix client.
147
148		Args:
149		client: A connected Enhanced Matrix client
150		room_id: The ID of the room to send the message to
151		message: The message to send
152
153		Returns:
154		bool: True if sending was successful, False otherwise
155		"""
156		return asyncio.run(client.send_message(room_id, message))
157
158		def listen_for_enhanced_matrix_messages(client: EnhancedMatrixClient) -> None:
159		"""
160		Start listening for messages in joined rooms with an Enhanced Matrix client.
161
162		This function will block until the client is stopped.
163
164		Args:
165		client: A connected Enhanced Matrix client
166		"""
167		asyncio.run(client.listen())
168
169		def create_enhanced_matrix_config() -> Dict:
170		"""
171		Create a default Enhanced Matrix configuration.
172
173		Returns:
174		Dict: The default configuration
175		"""
176		config = create_default_matrix_config()
177
178		# Add enhanced configuration
179		config["ENHANCED"] = {
180		"ENABLE_SEARCH": True,
181		"ENABLE_CONVERSATION_TRACKING": True,
182		"MAX_CONTEXT_LENGTH": 10,
183		"MAX_CONVERSATION_HISTORY": 100,
184		"RESPOND_TO_ALL_MESSAGES": False,
185		"OPENSEARCH": {
186		"BASE_URL": "http://localhost:8081",
187		"ENGINES": [
188		"google",
189		"bing",
190		"duckduckgo",
191		"brave"
192		],
193		"NUM_RESULTS": 5
194		}
195		}
196
197		return config
198
199		def disconnect_enhanced_matrix(client: EnhancedMatrixClient) -> None:
200		"""
201		Disconnect from the Matrix homeserver and clean up resources.
202
203		Args:
204		client: A connected Enhanced Matrix client
205		"""
206		asyncio.run(client.disconnect())
207
208		def process_with_enhanced_ollama(
209		client: EnhancedMatrixClient,
210		message: str,
211		sender: str = None,
212		room_id: str = None
213		) -> str:
214		"""
215		Process a message with Ollama using an Enhanced Matrix client.
216
217		Args:
218		client: A connected Enhanced Matrix client
219		message: The message to process
220		sender: The Matrix user ID of the sender
221		room_id: The ID of the room where the message was sent
222
223		Returns:
224		str: The response from Ollama
225		"""
226		return asyncio.run(client.process_with_ollama(message, sender, room_id))

ollamatools-new/ollamatools/plugins/integrations/matrix/config.py +0 -29

…	…	@@ -1,29 +0,0 @@
1		"""Configuration for Matrix integration."""
2
3		from typing import Any, Dict, Optional
4
5		def load_matrix_config(config_path: Optional[str] = None) -> Dict[str, Any]:
6		"""Load Matrix configuration from the given path."""
7		# Implementation
8		return {}
9
10		def validate_matrix_config(config: Dict[str, Any]) -> bool:
11		"""Validate Matrix configuration."""
12		# Implementation
13		return True
14
15		def get_default_matrix_config() -> Dict[str, Any]:
16		"""Return the default Matrix configuration."""
17		return {
18		"homeserver": "https://matrix.org",
19		"user_id": "@ollamatools:matrix.org",
20		"access_token": "",
21		"device_id": "ollamatools",
22		"room_ids": [],
23		"rate_limit": {
24		"max_requests": 5,
25		"time_window": 1,
26		"backoff_factor": 2,
27		"max_retries": 5,
28		},
29		}

ollamatools-new/ollamatools/plugins/integrations/matrix/history.py +0 -329

…	…	@@ -1,329 +0,0 @@
1		"""
2		Conversation history and analytics for the Enhanced Matrix integration in OllamaTools.
3
4		This module provides functionality for tracking conversation history,
5		analyzing topics, and generating performance metrics for the Enhanced Matrix client.
6
7		Dependencies:
8		- matrix-nio: For Matrix protocol support
9		- ollama: For Ollama API access
10		"""
11
12		import os
13		import json
14		import time
15		import logging
16		from typing import Dict, List, Optional, Any, Set, Tuple
17		from collections import defaultdict
18
19		# Configure logging
20		logger = logging.getLogger(__name__)
21
22		def load_conversation_history(history_file: str) -> List[Dict[str, Any]]:
23		"""
24		Load conversation history from file.
25
26		Args:
27		history_file: Path to the history file
28
29		Returns:
30		List[Dict[str, Any]]: The conversation history
31		"""
32		if os.path.exists(history_file):
33		try:
34		with open(history_file, 'r') as f:
35		history = json.load(f)
36		logger.info(f"Loaded {len(history)} conversations from history")
37		return history
38		except Exception as e:
39		logger.error(f"Failed to load conversation history: {str(e)}")
40		return []
41		else:
42		logger.info("No conversation history file found")
43		return []
44
45		def save_conversation_history(history: List[Dict[str, Any]], history_file: str) -> None:
46		"""
47		Save conversation history to file.
48
49		Args:
50		history: The conversation history
51		history_file: Path to save the history to
52		"""
53		try:
54		# Create directory if it doesn't exist
55		os.makedirs(os.path.dirname(history_file), exist_ok=True)
56
57		with open(history_file, 'w') as f:
58		json.dump(history, f, indent=2)
59		logger.debug(f"Saved {len(history)} conversations to history")
60		except Exception as e:
61		logger.error(f"Failed to save conversation history: {str(e)}")
62
63		def record_conversation(
64		history: List[Dict[str, Any]],
65		user_message: str,
66		assistant_response: str,
67		sender: str,
68		room_id: str,
69		response_time: float,
70		history_file: str,
71		max_history: int = 100
72		) -> None:
73		"""
74		Record a conversation in the history.
75
76		Args:
77		history: The conversation history list to update
78		user_message: The user's message
79		assistant_response: The assistant's response
80		sender: The Matrix user ID of the sender
81		room_id: The ID of the room where the conversation took place
82		response_time: The time taken to generate the response
83		history_file: Path to save the history to
84		max_history: Maximum number of conversations to keep in history
85		"""
86		# Create a conversation record
87		conversation = {
88		"timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
89		"user_message": user_message,
90		"assistant_response": assistant_response,
91		"sender": sender,
92		"room_id": room_id,
93		"response_time": response_time,
94		"tokens_estimated": len(user_message.split()) + len(assistant_response.split())
95		}
96
97		# Add to conversation history
98		history.append(conversation)
99
100		# Trim conversation history if it gets too long
101		if len(history) > max_history:
102		history = history[-max_history:]
103
104		# Save conversation history
105		save_conversation_history(history, history_file)
106
107		def analyze_conversation_topics(conversations: List[Dict[str, Any]]) -> Dict[str, int]:
108		"""
109		Analyze common topics in conversation history.
110
111		Args:
112		conversations: List of conversation records
113
114		Returns:
115		Dict[str, int]: Dictionary of topics and their frequency
116		"""
117		if not conversations:
118		return {}
119
120		# Simple keyword-based topic analysis
121		topics = {
122		"help": 0,
123		"search": 0,
124		"file": 0,
125		"command": 0,
126		"code": 0,
127		"error": 0,
128		"issue": 0,
129		"question": 0,
130		"other": 0
131		}
132
133		for conv in conversations:
134		message = conv.get("user_message", "").lower()
135
136		# Check for keywords
137		if "help" in message or "how" in message:
138		topics["help"] += 1
139		if "search" in message or "find" in message or "look" in message:
140		topics["search"] += 1
141		if "file" in message or "read" in message or "write" in message:
142		topics["file"] += 1
143		if "command" in message or "run" in message or "execute" in message:
144		topics["command"] += 1
145		if "code" in message or "program" in message or "script" in message:
146		topics["code"] += 1
147		if "error" in message or "bug" in message or "fix" in message:
148		topics["error"] += 1
149		if "issue" in message or "problem" in message:
150		topics["issue"] += 1
151		if "?" in message:
152		topics["question"] += 1
153
154		# If no keywords matched, count as other
155		if not any(count > 0 for count in topics.values()):
156		topics["other"] += 1
157
158		# Remove topics with zero count
159		return {k: v for k, v in topics.items() if v > 0}
160
161		def get_performance_metrics(conversations: List[Dict[str, Any]]) -> Dict[str, Any]:
162		"""
163		Get performance metrics from conversation history.
164
165		Args:
166		conversations: List of conversation records
167
168		Returns:
169		Dict[str, Any]: Performance metrics
170		"""
171		if not conversations:
172		return {"error": "No conversation history available"}
173
174		# Calculate metrics
175		total_conversations = len(conversations)
176		total_response_time = sum(conv.get("response_time", 0) for conv in conversations)
177		avg_response_time = total_response_time / total_conversations if total_conversations > 0 else 0
178
179		# Calculate token usage
180		total_tokens = sum(conv.get("tokens_estimated", 0) for conv in conversations)
181		avg_tokens_per_conversation = total_tokens / total_conversations if total_conversations > 0 else 0
182
183		# Find slowest and fastest responses
184		slowest = max(conversations, key=lambda x: x.get("response_time", 0)) if conversations else None
185		fastest = min(conversations, key=lambda x: x.get("response_time", 0)) if conversations else None
186
187		return {
188		"total_conversations": total_conversations,
189		"total_response_time": total_response_time,
190		"avg_response_time": avg_response_time,
191		"total_tokens_estimated": total_tokens,
192		"avg_tokens_per_conversation": avg_tokens_per_conversation,
193		"slowest_response": {
194		"time": slowest.get("response_time", 0) if slowest else 0,
195		"message": slowest.get("user_message", "")[:50] + "..." if slowest and len(slowest.get("user_message", "")) > 50 else slowest.get("user_message", "") if slowest else ""
196		},
197		"fastest_response": {
198		"time": fastest.get("response_time", 0) if fastest else 0,
199		"message": fastest.get("user_message", "")[:50] + "..." if fastest and len(fastest.get("user_message", "")) > 50 else fastest.get("user_message", "") if fastest else ""
200		}
201		}
202
203		async def handle_history_command(client: Any, count: int, room_id: str) -> None:
204		"""
205		Handle the /history command.
206
207		Args:
208		client: The EnhancedMatrixClient instance
209		count: Number of recent conversations to show
210		room_id: The ID of the room where the command was sent
211		"""
212		if not client.enable_conversation_tracking:
213		await client.send_message(room_id, "Conversation tracking is not enabled.")
214		return
215
216		if not client.conversation_history:
217		await client.send_message(room_id, "No conversation history available.")
218		return
219
220		# Get the most recent conversations for this room
221		room_history = [conv for conv in client.conversation_history if conv.get("room_id") == room_id]
222
223		if not room_history:
224		await client.send_message(room_id, "No conversation history available for this room.")
225		return
226
227		# Get the most recent conversations
228		recent = room_history[-count:] if len(room_history) >= count else room_history
229
230		# Format the history
231		formatted_history = ["Recent Conversations:"]
232
233		for i, conv in enumerate(recent, 1):
234		timestamp = conv.get("timestamp", "Unknown time")
235		user_message = conv.get("user_message", "No message")
236		sender = conv.get("sender", "Unknown sender")
237
238		# Truncate message if it's too long
239		if len(user_message) > 100:
240		user_message = user_message[:100] + "..."
241
242		formatted_history.append(f"{i}. {timestamp}")
243		formatted_history.append(f" From: {sender}")
244		formatted_history.append(f" Message: {user_message}")
245		formatted_history.append("")
246
247		# Send the formatted history
248		await client.send_message(room_id, "\n".join(formatted_history))
249
250		async def handle_topics_command(client: Any, room_id: str) -> None:
251		"""
252		Handle the /topics command.
253
254		Args:
255		client: The EnhancedMatrixClient instance
256		room_id: The ID of the room where the command was sent
257		"""
258		if not client.enable_conversation_tracking:
259		await client.send_message(room_id, "Conversation tracking is not enabled.")
260		return
261
262		if not client.conversation_history:
263		await client.send_message(room_id, "No conversation history available.")
264		return
265
266		# Get the conversations for this room
267		room_history = [conv for conv in client.conversation_history if conv.get("room_id") == room_id]
268
269		if not room_history:
270		await client.send_message(room_id, "No conversation history available for this room.")
271		return
272
273		# Analyze topics
274		topics = analyze_conversation_topics(room_history)
275
276		if not topics:
277		await client.send_message(room_id, "No topics found in conversation history.")
278		return
279
280		# Format the topics
281		formatted_topics = ["Conversation Topics:"]
282
283		for topic, count in sorted(topics.items(), key=lambda x: x[1], reverse=True):
284		formatted_topics.append(f"- {topic}: {count} mentions")
285
286		# Send the formatted topics
287		await client.send_message(room_id, "\n".join(formatted_topics))
288
289		async def handle_metrics_command(client: Any, room_id: str) -> None:
290		"""
291		Handle the /metrics command.
292
293		Args:
294		client: The EnhancedMatrixClient instance
295		room_id: The ID of the room where the command was sent
296		"""
297		if not client.enable_conversation_tracking:
298		await client.send_message(room_id, "Conversation tracking is not enabled.")
299		return
300
301		if not client.conversation_history:
302		await client.send_message(room_id, "No conversation history available.")
303		return
304
305		# Get the conversations for this room
306		room_history = [conv for conv in client.conversation_history if conv.get("room_id") == room_id]
307
308		if not room_history:
309		await client.send_message(room_id, "No conversation history available for this room.")
310		return
311
312		# Calculate metrics
313		metrics = get_performance_metrics(room_history)
314
315		# Format the metrics
316		formatted_metrics = ["Performance Metrics:"]
317		formatted_metrics.append(f"- Total conversations: {metrics['total_conversations']}")
318		formatted_metrics.append(f"- Average response time: {metrics['avg_response_time']:.2f}s")
319		formatted_metrics.append(f"- Estimated total tokens: {metrics['total_tokens_estimated']}")
320		formatted_metrics.append(f"- Average tokens per conversation: {metrics['avg_tokens_per_conversation']:.1f}")
321
322		if metrics.get("slowest_response"):
323		formatted_metrics.append(f"- Slowest response: {metrics['slowest_response']['time']:.2f}s for '{metrics['slowest_response']['message']}'")
324
325		if metrics.get("fastest_response"):
326		formatted_metrics.append(f"- Fastest response: {metrics['fastest_response']['time']:.2f}s for '{metrics['fastest_response']['message']}'")
327
328		# Send the formatted metrics
329		await client.send_message(room_id, "\n".join(formatted_metrics))

ollamatools-new/ollamatools/plugins/integrations/matrix/search.py +0 -170

…	…	@@ -1,170 +0,0 @@
1		"""
2		Search functionality for the Enhanced Matrix integration in OllamaTools.
3
4		This module provides search-related functionality for the Enhanced Matrix client,
5		including integration with OpenSearchNG and the modular agent's search capabilities.
6
7		Dependencies:
8		- matrix-nio: For Matrix protocol support
9		- ollama: For Ollama API access
10		- ollamatools.agent.modular: For search integration and natural language understanding
11		- ollamatools.opensearch_web_tools: For web search functionality
12		"""
13
14		import logging
15		from typing import Dict, List, Optional, Any, Union
16
17		# Configure logging
18		logger = logging.getLogger(__name__)
19
20		# Import from modular agent (optional)
21		try:
22		from ollamatools.plugins.standard.search import (
23		detect_factual_intent,
24		extract_search_context,
25		execute_search
26		)
27		MODULAR_AGENT_AVAILABLE = True
28		except ImportError:
29		MODULAR_AGENT_AVAILABLE = False
30		# Only log this as debug since search might be disabled in config
31		logging.debug(
32		"The modular agent components are not available. "
33		"Search functionality will be disabled."
34		)
35
36		# Import from opensearch web tools (optional)
37		try:
38		from ollamatools.plugins.integrations.opensearch.tools import (
39		opensearch_query,
40		opensearch_fetch_url,
41		get_opensearch_info,
42		DEFAULT_OPENSEARCH_CONFIG
43		)
44		OPENSEARCH_AVAILABLE = True
45		except ImportError:
46		OPENSEARCH_AVAILABLE = False
47		# Only log this as debug since search might be disabled in config
48		logging.debug(
49		"The opensearch_web_tools module is not available. "
50		"Search functionality will be disabled."
51		)
52
53		def get_default_opensearch_config() -> Dict[str, Any]:
54		"""
55		Get the default OpenSearchNG configuration.
56
57		Returns:
58		Dict[str, Any]: The default configuration
59		"""
60		if OPENSEARCH_AVAILABLE:
61		return DEFAULT_OPENSEARCH_CONFIG.copy()
62		else:
63		return {}
64
65		async def execute_search_for_message(message: str, opensearch_config: Dict[str, Any]) -> str:
66		"""
67		Execute a search for a message and return the search context.
68
69		Args:
70		message: The message to search for
71		opensearch_config: The OpenSearchNG configuration
72
73		Returns:
74		str: The search context, or an empty string if search failed
75		"""
76		if not MODULAR_AGENT_AVAILABLE or not OPENSEARCH_AVAILABLE:
77		logger.warning("Search functionality is not available.")
78		return ""
79
80		try:
81		factual_intent = detect_factual_intent(message)
82		if factual_intent:
83		logger.info(f"Factual intent detected in message: {message}")
84		try:
85		results = await execute_search_query(message, opensearch_config)
86		search_context = extract_search_context(results)
87		logger.info(f"Search context: {search_context[:100]}...")
88		return search_context
89		except Exception as e:
90		logger.error(f"Error executing search: {e}")
91		return ""
92		else:
93		logger.debug(f"No factual intent detected in message: {message}")
94		return ""
95		except Exception as e:
96		logger.error(f"Error detecting factual intent: {e}")
97		return ""
98
99		async def execute_search_query(query: str, opensearch_config: Dict[str, Any]) -> Dict[str, Any]:
100		"""
101		Execute a search query using OpenSearchNG.
102
103		Args:
104		query: The search query
105		opensearch_config: The OpenSearchNG configuration
106
107		Returns:
108		Dict[str, Any]: Search results
109		"""
110		if not OPENSEARCH_AVAILABLE:
111		logger.warning("OpenSearchNG is not available.")
112		return {"error": "Search functionality is not available.", "results": []}
113
114		try:
115		# Execute the search
116		results = opensearch_query(
117		query=query,
118		config=opensearch_config
119		)
120
121		return results
122		except Exception as e:
123		logger.error(f"Error executing search: {e}")
124		return {"error": str(e), "results": []}
125
126		async def handle_search_command(client: Any, query: str, room_id: str) -> None:
127		"""
128		Handle the /search command.
129
130		Args:
131		client: The EnhancedMatrixClient instance
132		query: The search query
133		room_id: The ID of the room where the command was sent
134		"""
135		if not client.enable_search:
136		await client.send_message(room_id, "Search functionality is not enabled.")
137		return
138
139		await client.send_message(room_id, f"Searching for: {query}...")
140
141		try:
142		results = await execute_search_query(query, client.opensearch_config)
143
144		if not results or not results.get("results"):
145		await client.send_message(room_id, f"No results found for: {query}")
146		return
147
148		# Format the results
149		formatted_results = ["Search Results:"]
150
151		for i, result in enumerate(results["results"][:5], 1):
152		title = result.get("title", "No title")
153		url = result.get("url", "No URL")
154		content = result.get("content", "No content")
155
156		# Truncate content if it's too long
157		if len(content) > 200:
158		content = content[:200] + "..."
159
160		formatted_results.append(f"{i}. {title}")
161		formatted_results.append(f" URL: {url}")
162		formatted_results.append(f" {content}")
163		formatted_results.append("")
164
165		# Send the formatted results
166		await client.send_message(room_id, "\n".join(formatted_results))
167
168		except Exception as e:
169		logger.error(f"Error executing search: {e}")
170		await client.send_message(room_id, f"Error executing search: {str(e)}")

ollamatools-new/ollamatools/plugins/integrations/opensearch/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/plugins/integrations/opensearch/client.py +0 -40

…	…	@@ -1,40 +0,0 @@
1		"""OpenSearch client for OllamaTools."""
2
3		from typing import Any, Dict, List, Optional
4
5		class OpenSearchClient:
6		"""Client for interacting with OpenSearch."""
7
8		def __init__(self, host: str, port: int, username: Optional[str] = None, password: Optional[str] = None):
9		"""Initialize the OpenSearch client."""
10		self.host = host
11		self.port = port
12		self.username = username
13		self.password = password
14		self.connected = False
15
16		def connect(self) -> bool:
17		"""Connect to the OpenSearch server."""
18		# Implementation
19		self.connected = True
20		return True
21
22		def disconnect(self) -> None:
23		"""Disconnect from the OpenSearch server."""
24		# Implementation
25		self.connected = False
26
27		def search(self, index: str, query: Dict[str, Any]) -> List[Dict[str, Any]]:
28		"""Search for documents in the given index."""
29		# Implementation
30		return []
31
32		def index_document(self, index: str, document: Dict[str, Any], id: Optional[str] = None) -> bool:
33		"""Index a document in the given index."""
34		# Implementation
35		return True
36
37		def delete_document(self, index: str, id: str) -> bool:
38		"""Delete a document from the given index."""
39		# Implementation
40		return True

ollamatools-new/ollamatools/plugins/integrations/opensearch/tools.py +0 -425

…	…	@@ -1,425 +0,0 @@
1		"""
2		OpenSearchNG integration for OllamaTools.
3
4		This module extends the web_tools capabilities by adding integration with
5		a locally running OpenSearchNG instance for improved search functionality.
6		"""
7
8		import requests
9		import json
10		import logging
11		from typing import Dict, Any, Optional, List, Union, Set
12		from urllib.parse import quote_plus
13
14		# Import URL validation utilities
15		from ollamatools.core.utils.validators.url import (
16		validate_url,
17		URLValidationError,
18		configure_domain_lists,
19		)
20
21		# Import error handling utilities
22		from ollamatools.core.errors import (
23		WebError,
24		ERROR_CODES,
25		handle_exception
26		)
27
28		# Set up logging
29		logger = logging.getLogger(__name__)
30
31		# Default OpenSearchNG configuration
32		DEFAULT_OPENSEARCH_CONFIG = {
33		"base_url": "http://localhost:8081", # Default local OpenSearchNG URL
34		"search_path": "/search", # Path for search endpoint
35		"preferences_path": "/preferences", # Path for preferences endpoint
36		"categories": ["general"], # Default search categories
37		"engines": ["google", "bing", "duckduckgo"], # Default search engines
38		"language": "en-US", # Default language
39		"format": "json", # Response format (html, json, csv, rss)
40		"timeout": 10, # Default timeout in seconds
41		"num_results": 10, # Default number of results
42		"safe_search": 0 # 0=off, 1=moderate, 2=strict
43		}
44
45		def opensearch_query(
46		query: str,
47		config: Optional[Dict[str, Any]] = None,
48		permission_manager = None,
49		domain_allowlist: Optional[Set[str]] = None,
50		domain_blocklist: Optional[Set[str]] = None
51		) -> Dict[str, Any]:
52		"""
53		Perform a search query using a local OpenSearchNG instance.
54
55		Args:
56		query: Search query string
57		config: OpenSearchNG configuration (defaults used if None)
58		permission_manager: Optional permission manager to request permission
59		domain_allowlist: Optional set of allowed domains for results
60		domain_blocklist: Optional set of blocked domains for results
61
62		Returns:
63		dict: Search results with metadata
64
65		Raises:
66		ValueError: If parameters are invalid
67		WebError: If the search fails
68		PermissionError: If permission is denied
69		"""
70		# Validate input
71		if not query or not isinstance(query, str):
72		raise ValueError("Query must be a non-empty string")
73
74		# Use default config if none provided
75		search_config = DEFAULT_OPENSEARCH_CONFIG.copy()
76		if config:
77		search_config.update(config)
78
79		# Request permission if permission manager is provided
80		if permission_manager:
81		args = {
82		"query": query,
83		"config": search_config,
84		}
85
86		if not permission_manager.request_permission("opensearch_query", args):
87		logger.warning(f"Permission denied for OpenSearchNG query: {query}")
88		raise PermissionError(f"Permission denied for OpenSearchNG query: {query}")
89
90		# Construct the search URL
91		base_url = search_config["base_url"]
92		search_path = search_config["search_path"]
93		search_url = f"{base_url}{search_path}"
94
95		# Validate the OpenSearchNG instance URL
96		try:
97		# Allow localhost for OpenSearchNG access
98		local_allowlist = {"localhost", "127.0.0.1"}
99		if domain_allowlist:
100		local_allowlist.update(domain_allowlist)
101
102		# Extract the host from the base URL for validation
103		from urllib.parse import urlparse
104		parsed_url = urlparse(base_url)
105		hostname = parsed_url.netloc.split(':')[0]
106
107		if hostname not in local_allowlist:
108		validate_url(
109		base_url,
110		domain_allowlist=local_allowlist,
111		domain_blocklist=domain_blocklist
112		)
113		except URLValidationError as e:
114		# Make a special exception for localhost
115		if "localhost" not in base_url and "127.0.0.1" not in base_url:
116		raise ValueError(f"Invalid OpenSearchNG URL: {str(e)}")
117
118		# Prepare query parameters
119		params = {
120		"q": query,
121		"format": search_config["format"],
122		"categories": ",".join(search_config["categories"]),
123		"engines": ",".join(search_config["engines"]),
124		"language": search_config["language"],
125		"safesearch": search_config["safe_search"],
126		"time_range": "", # Optional time range
127		"pageno": 1, # First page of results
128		}
129
130		# Create session with appropriate headers
131		with requests.Session() as session:
132		headers = {
133		"User-Agent": "OllamaTools/0.2.0",
134		"Accept": "application/json"
135		}
136		session.headers.update(headers)
137
138		try:
139		logger.info(f"Making OpenSearchNG query: {query}")
140		response = session.get(
141		search_url,
142		params=params,
143		timeout=search_config["timeout"]
144		)
145
146		# Check for successful response
147		if response.status_code != 200:
148		raise WebError(
149		f"OpenSearchNG query failed with status code: {response.status_code}",
150		url=search_url,
151		status_code=response.status_code,
152		code=ERROR_CODES["URL_RESPONSE_ERROR"]
153		)
154
155		# Parse the response based on format
156		if search_config["format"] == "json":
157		try:
158		result_data = response.json()
159		except json.JSONDecodeError:
160		raise WebError(
161		"Failed to parse JSON response from OpenSearchNG",
162		url=search_url,
163		code=ERROR_CODES["URL_RESPONSE_ERROR"]
164		)
165		else:
166		# For other formats, return the raw content
167		result_data = {"raw_content": response.text}
168
169		# Validate and process the results
170		processed_results = process_opensearch_results(result_data, query)
171
172		return processed_results
173
174		except requests.RequestException as e:
175		raise WebError(
176		f"OpenSearchNG request failed: {str(e)}",
177		url=search_url,
178		code=ERROR_CODES["URL_CONNECTION_ERROR"]
179		)
180
181		def process_opensearch_results(
182		result_data: Dict[str, Any],
183		query: str
184		) -> Dict[str, Any]:
185		"""
186		Process and format the OpenSearchNG results.
187
188		Args:
189		result_data: Raw result data from OpenSearchNG
190		query: Original search query
191
192		Returns:
193		dict: Processed search results with metadata
194		"""
195		processed_results = {
196		"query": query,
197		"results": [],
198		"suggestion": None,
199		"answer": None,
200		"infobox": None,
201		"engines": [],
202		"timing": {}
203		}
204
205		# Extract search results
206		if "results" in result_data:
207		for result in result_data["results"]:
208		processed_result = {
209		"title": result.get("title", ""),
210		"url": result.get("url", ""),
211		"content": result.get("content", ""),
212		"engine": result.get("engine", ""),
213		"score": result.get("score", 0),
214		"category": result.get("category", ""),
215		"pretty_url": result.get("pretty_url", "")
216		}
217		processed_results["results"].append(processed_result)
218
219		# Extract suggestions
220		if "suggestions" in result_data and result_data["suggestions"]:
221		try:
222		suggestion = result_data["suggestions"][0]
223		# Handle case where suggestion is a dict instead of a string
224		if isinstance(suggestion, dict):
225		processed_results["suggestion"] = suggestion.get("value", str(suggestion))
226		else:
227		processed_results["suggestion"] = suggestion
228		except (IndexError, TypeError) as e:
229		logger.warning(f"Error processing suggestions: {str(e)}")
230		processed_results["suggestion"] = None
231
232		# Extract answer if available
233		if "answers" in result_data and result_data["answers"]:
234		try:
235		answer = result_data["answers"][0]
236		# Handle case where answer is a dict instead of a string
237		if isinstance(answer, dict):
238		processed_results["answer"] = answer.get("content", str(answer))
239		else:
240		processed_results["answer"] = answer
241		except (IndexError, TypeError) as e:
242		logger.warning(f"Error processing answers: {str(e)}")
243		processed_results["answer"] = None
244
245		# Extract infobox if available
246		if "infoboxes" in result_data and result_data["infoboxes"]:
247		try:
248		infobox = result_data["infoboxes"][0]
249		# Handle case where infobox is a dict (which is expected)
250		processed_results["infobox"] = infobox
251		except (IndexError, TypeError) as e:
252		logger.warning(f"Error processing infoboxes: {str(e)}")
253		processed_results["infobox"] = None
254
255		# Extract engines used
256		if "engines" in result_data:
257		processed_results["engines"] = result_data["engines"]
258
259		# Extract timing information
260		if "timing" in result_data:
261		processed_results["timing"] = result_data["timing"]
262
263		return processed_results
264
265		def get_opensearch_info(
266		config: Optional[Dict[str, Any]] = None,
267		permission_manager = None
268		) -> Dict[str, Any]:
269		"""
270		Get information about the OpenSearchNG instance.
271
272		Args:
273		config: OpenSearchNG configuration (defaults used if None)
274		permission_manager: Optional permission manager to request permission
275
276		Returns:
277		dict: Information about the OpenSearchNG instance
278
279		Raises:
280		WebError: If the request fails
281		PermissionError: If permission is denied
282		"""
283		# Use default config if none provided
284		search_config = DEFAULT_OPENSEARCH_CONFIG.copy()
285		if config:
286		search_config.update(config)
287
288		# Request permission if permission manager is provided
289		if permission_manager:
290		if not permission_manager.request_permission("get_opensearch_info", {"config": search_config}):
291		logger.warning("Permission denied for getting OpenSearchNG info")
292		raise PermissionError("Permission denied for getting OpenSearchNG info")
293
294		# Construct the URL
295		base_url = search_config["base_url"]
296		config_url = f"{base_url}/config"
297
298		try:
299		logger.info("Getting OpenSearchNG instance information")
300		response = requests.get(
301		config_url,
302		timeout=search_config["timeout"]
303		)
304
305		# Check for successful response
306		if response.status_code != 200:
307		raise WebError(
308		f"OpenSearchNG info request failed with status code: {response.status_code}",
309		url=config_url,
310		status_code=response.status_code,
311		code=ERROR_CODES["URL_RESPONSE_ERROR"]
312		)
313
314		# Parse the response
315		try:
316		config_data = response.json()
317		return config_data
318		except json.JSONDecodeError:
319		raise WebError(
320		"Failed to parse JSON response from OpenSearchNG",
321		url=config_url,
322		code=ERROR_CODES["URL_RESPONSE_ERROR"]
323		)
324
325		except requests.RequestException as e:
326		raise WebError(
327		f"OpenSearchNG info request failed: {str(e)}",
328		url=config_url,
329		code=ERROR_CODES["URL_CONNECTION_ERROR"]
330		)
331
332		def opensearch_fetch_url(
333		url: str,
334		config: Optional[Dict[str, Any]] = None,
335		permission_manager = None,
336		domain_allowlist: Optional[Set[str]] = None,
337		domain_blocklist: Optional[Set[str]] = None
338		) -> Dict[str, Any]:
339		"""
340		Fetch a URL through the OpenSearchNG proxy to avoid direct connections.
341		This provides an additional security layer for web requests.
342
343		Args:
344		url: URL to fetch
345		config: OpenSearchNG configuration (defaults used if None)
346		permission_manager: Optional permission manager to request permission
347		domain_allowlist: Optional set of allowed domains
348		domain_blocklist: Optional set of blocked domains
349
350		Returns:
351		dict: Response content and metadata
352
353		Raises:
354		ValueError: If URL is invalid
355		WebError: If the fetch fails
356		PermissionError: If permission is denied
357		"""
358		# Validate URL
359		try:
360		validated_url = validate_url(
361		url,
362		domain_allowlist=domain_allowlist,
363		domain_blocklist=domain_blocklist
364		)
365		except URLValidationError as e:
366		raise ValueError(f"URL validation failed: {str(e)}")
367
368		# Use default config if none provided
369		search_config = DEFAULT_OPENSEARCH_CONFIG.copy()
370		if config:
371		search_config.update(config)
372
373		# Request permission if permission manager is provided
374		if permission_manager:
375		args = {
376		"url": url,
377		"config": search_config
378		}
379
380		if not permission_manager.request_permission("opensearch_fetch_url", args):
381		logger.warning(f"Permission denied for fetching URL via OpenSearchNG: {url}")
382		raise PermissionError(f"Permission denied for fetching URL via OpenSearchNG: {url}")
383
384		# Construct the URL for the OpenSearchNG proxy
385		base_url = search_config["base_url"]
386		proxy_url = f"{base_url}/url_proxy"
387
388		# Prepare query parameters
389		params = {
390		"url": validated_url
391		}
392
393		try:
394		logger.info(f"Fetching URL via OpenSearchNG proxy: {validated_url}")
395		response = requests.get(
396		proxy_url,
397		params=params,
398		timeout=search_config["timeout"]
399		)
400
401		# Check for successful response
402		if response.status_code != 200:
403		raise WebError(
404		f"URL fetch via OpenSearchNG failed with status code: {response.status_code}",
405		url=proxy_url,
406		status_code=response.status_code,
407		code=ERROR_CODES["URL_RESPONSE_ERROR"]
408		)
409
410		# Create result with metadata
411		result = {
412		"status": response.status_code,
413		"headers": dict(response.headers),
414		"content": response.text,
415		"url": validated_url
416		}
417
418		return result
419
420		except requests.RequestException as e:
421		raise WebError(
422		f"URL fetch via OpenSearchNG failed: {str(e)}",
423		url=proxy_url,
424		code=ERROR_CODES["URL_CONNECTION_ERROR"]
425		)

ollamatools-new/ollamatools/plugins/registry.py +0 -198

…	…	@@ -1,198 +0,0 @@
1		"""
2		Plugin Registry for OllamaTools.
3
4		This module provides the PluginRegistry class for managing OllamaTools plugins.
5		"""
6
7		import os
8		import sys
9		import importlib
10		import logging
11		import pkg_resources
12		from typing import Dict, Any, List, Optional, Set, Type, Callable, Union
13
14		from ollamatools.plugins import OllamaToolsPlugin
15		from ollamatools.plugins.errors import (
16		PluginError,
17		PluginInitializationError,
18		PluginDependencyError,
19		PluginNotFoundError,
20		handle_plugin_exception
21		)
22
23		# Configure logging
24		logger = logging.getLogger(__name__)
25
26		class PluginRegistry:
27		"""Registry for OllamaTools plugins."""
28
29		def __init__(self):
30		"""Initialize the plugin registry."""
31		self.plugins = {}
32		self.initialized_plugins = {}
33
34		def register_plugin(self, plugin_class, config=None):
35		"""
36		Register a plugin class.
37
38		Args:
39		plugin_class: The plugin class to register
40		config: Optional configuration for the plugin
41
42		Returns:
43		PluginRegistry: Self for method chaining
44		"""
45		plugin_name = plugin_class.__name__
46		self.plugins[plugin_name] = {
47		"class": plugin_class,
48		"config": config
49		}
50		logger.debug(f"Registered plugin: {plugin_name}")
51		return self
52
53		def discover_plugins(self):
54		"""
55		Discover plugins using entry points.
56
57		Returns:
58		PluginRegistry: Self for method chaining
59		"""
60		try:
61		for entry_point in pkg_resources.iter_entry_points('ollamatools.plugins'):
62		try:
63		plugin_class = entry_point.load()
64		self.register_plugin(plugin_class)
65		logger.info(f"Discovered plugin: {plugin_class.__name__} from entry point {entry_point.name}")
66		except Exception as e:
67		logger.error(f"Error loading plugin from entry point {entry_point.name}: {e}")
68		except Exception as e:
69		logger.error(f"Error discovering plugins: {e}")
70
71		return self
72
73		def initialize_plugin(self, plugin_name, context):
74		"""
75		Initialize a plugin and its dependencies.
76
77		Args:
78		plugin_name: The name of the plugin to initialize
79		context: Runtime context for the plugin
80
81		Returns:
82		OllamaToolsPlugin: The initialized plugin instance
83
84		Raises:
85		PluginNotFoundError: If the plugin is not found
86		PluginInitializationError: If the plugin fails to initialize
87		PluginDependencyError: If a dependency cannot be initialized
88		"""
89		# Return already initialized plugin
90		if plugin_name in self.initialized_plugins:
91		return self.initialized_plugins[plugin_name]
92
93		# Check if plugin exists
94		if plugin_name not in self.plugins:
95		raise PluginNotFoundError(f"Plugin {plugin_name} not found")
96
97		logger.debug(f"Initializing plugin: {plugin_name}")
98
99		plugin_info = self.plugins[plugin_name]
100		plugin_class = plugin_info["class"]
101		plugin_config = plugin_info["config"]
102
103		try:
104		# Create plugin instance
105		plugin_instance = plugin_class(plugin_config)
106
107		# Initialize dependencies first
108		for dependency in plugin_instance.dependencies:
109		try:
110		self.initialize_plugin(dependency, context)
111		except PluginError as e:
112		raise PluginDependencyError(
113		f"Failed to initialize dependency {dependency} for plugin {plugin_name}",
114		details={"plugin": plugin_name, "dependency": dependency, "original_error": str(e)}
115		) from e
116
117		# Initialize the plugin
118		plugin_instance.initialize(context)
119		self.initialized_plugins[plugin_name] = plugin_instance
120
121		logger.info(f"Initialized plugin: {plugin_name}")
122		return plugin_instance
123
124		except PluginError:
125		# Re-raise plugin errors
126		raise
127		except Exception as e:
128		# Wrap other exceptions
129		raise PluginInitializationError(
130		f"Failed to initialize plugin {plugin_name}: {str(e)}",
131		details={"plugin": plugin_name}
132		) from e
133
134		def initialize_all(self, context):
135		"""
136		Initialize all registered plugins.
137
138		Args:
139		context: Runtime context for the plugins
140
141		Returns:
142		Dict[str, OllamaToolsPlugin]: Dictionary of initialized plugins
143		"""
144		for plugin_name in list(self.plugins.keys()):
145		try:
146		self.initialize_plugin(plugin_name, context)
147		except PluginError as e:
148		logger.error(f"Error initializing plugin {plugin_name}: {e}")
149		handle_plugin_exception(e)
150		except Exception as e:
151		logger.error(f"Unexpected error initializing plugin {plugin_name}: {e}")
152
153		return self.initialized_plugins
154
155		def get_plugin(self, plugin_name):
156		"""
157		Get an initialized plugin.
158
159		Args:
160		plugin_name: The name of the plugin to get
161
162		Returns:
163		Optional[OllamaToolsPlugin]: The plugin instance or None if not found
164		"""
165		return self.initialized_plugins.get(plugin_name)
166
167		def get_plugins_by_type(self, plugin_type):
168		"""
169		Get all initialized plugins of a specific type.
170
171		Args:
172		plugin_type: The plugin type class
173
174		Returns:
175		Dict[str, OllamaToolsPlugin]: Dictionary of plugin names to instances
176		"""
177		return {
178		name: plugin for name, plugin in self.initialized_plugins.items()
179		if isinstance(plugin, plugin_type)
180		}
181
182		def shutdown(self):
183		"""
184		Shutdown all initialized plugins.
185
186		Returns:
187		PluginRegistry: Self for method chaining
188		"""
189		for plugin_name, plugin in list(self.initialized_plugins.items()):
190		try:
191		logger.debug(f"Shutting down plugin: {plugin_name}")
192		plugin.shutdown()
193		logger.info(f"Shut down plugin: {plugin_name}")
194		except Exception as e:
195		logger.error(f"Error shutting down plugin {plugin_name}: {e}")
196
197		self.initialized_plugins = {}
198		return self

ollamatools-new/ollamatools/plugins/standard/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/plugins/standard/commands.py +0 -122

…	…	@@ -1,122 +0,0 @@
1		"""
2		Command management functionality for the ModularOllamaAgent.
3
4		This module provides functions for managing the command allowlist,
5		including listing, adding, and removing commands.
6		"""
7
8		from typing import List, Set
9		from .utils import print_info, print_error, print_success, print_warning
10
11		def list_allowed_commands(command_allowlist: Set[str]) -> List[str]:
12		"""
13		List all currently allowed commands.
14
15		Args:
16		command_allowlist: Set of currently allowed commands
17
18		Returns:
19		List[str]: List of allowed commands
20		"""
21		# Get the current allowlist
22		all_commands = sorted(list(command_allowlist))
23
24		# Print the commands
25		print_info(f"Currently allowed commands ({len(all_commands)}):")
26
27		# Format into categories for better readability
28		categories = {
29		"File Operations": ["ls", "cat", "find", "grep", "head", "tail"],
30		"Directory Operations": ["pwd", "cd", "mkdir", "rmdir"],
31		"System Information": ["uname", "hostname", "whoami", "ps", "top", "df", "du", "free"],
32		"Network Utilities": ["ping", "curl", "wget", "netstat", "dig", "nslookup"],
33		"Text Processing": ["echo", "printf", "sed", "awk", "cut", "sort", "uniq", "wc"],
34		"File Manipulation": ["cp", "mv", "rm", "touch", "chmod", "chown", "chgrp", "ln"],
35		"Other": []
36		}
37
38		# Categorize commands
39		for cmd in all_commands:
40		categorized = False
41		for category, commands in categories.items():
42		if cmd in commands:
43		categorized = True
44		break
45		if not categorized:
46		categories["Other"].append(cmd)
47
48		# Print by category
49		for category, commands in categories.items():
50		if commands:
51		print(f"\n{category}:")
52		for i, cmd in enumerate(sorted(commands)):
53		print(f" {cmd}", end="\t")
54		# New line every 5 commands for readability
55		if (i + 1) % 5 == 0:
56		print()
57		print() # Ensure we end with a newline
58
59		return all_commands
60
61		def add_allowed_command(command: str, command_allowlist: Set[str]) -> bool:
62		"""
63		Add a command to the allowed list.
64
65		Args:
66		command: Command to add
67		command_allowlist: Set of currently allowed commands
68
69		Returns:
70		bool: True if added successfully
71		"""
72		if not command or not isinstance(command, str):
73		print_error("Invalid command name")
74		return False
75
76		# Clean up the command name
77		command = command.strip().lower()
78
79		# Check if already in allowlist
80		if command in command_allowlist:
81		print_info(f"Command '{command}' is already allowed")
82		return True
83
84		# Add to custom allowlist
85		command_allowlist.add(command)
86		print_success(f"Command '{command}' added to allowed list")
87
88		# Show warning for potentially dangerous commands
89		dangerous_commands = {"rm", "mv", "chmod", "chown", "dd", "mkfs", "iptables"}
90		if command in dangerous_commands:
91		print_warning(f"'{command}' is a potentially dangerous command. Use with caution!")
92
93		return True
94
95		def remove_allowed_command(command: str, command_allowlist: Set[str]) -> bool:
96		"""
97		Remove a command from the allowed list.
98
99		Args:
100		command: Command to remove
101		command_allowlist: Set of currently allowed commands
102
103		Returns:
104		bool: True if removed successfully
105		"""
106		if not command or not isinstance(command, str):
107		print_error("Invalid command name")
108		return False
109
110		# Clean up the command name
111		command = command.strip().lower()
112
113		# Check if in allowlist
114		if command not in command_allowlist:
115		print_error(f"Command '{command}' is not in the allowed list")
116		return False
117
118		# Remove from custom allowlist
119		command_allowlist.remove(command)
120		print_success(f"Command '{command}' removed from allowed list")
121
122		return True

ollamatools-new/ollamatools/plugins/standard/conversation/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/plugins/standard/conversation/analysis.py +0 -18

…	…	@@ -1,18 +0,0 @@
1		"""Conversation topic analysis and metrics."""
2
3		from typing import Any, Dict, List, Optional
4
5		def analyze_topics(messages: List[Dict[str, Any]]) -> List[str]:
6		"""Analyze the topics in a conversation."""
7		# Implementation
8		return ["topic1", "topic2"]
9
10		def calculate_metrics(messages: List[Dict[str, Any]]) -> Dict[str, Any]:
11		"""Calculate metrics for a conversation."""
12		# Implementation
13		return {
14		"message_count": len(messages),
15		"average_length": sum(len(m.get("content", "")) for m in messages) / max(1, len(messages)),
16		"user_messages": sum(1 for m in messages if m.get("role") == "user"),
17		"assistant_messages": sum(1 for m in messages if m.get("role") == "assistant"),
18		}

ollamatools-new/ollamatools/plugins/standard/feedback.py +0 -586

…	…	@@ -1,586 +0,0 @@
1		"""
2		Feedback system for the ModularOllamaAgent.
3
4		This module provides a feedback loop system that improves the agent's
5		semantic understanding of user queries by learning from past interactions.
6		It helps map natural language queries to appropriate functions and commands.
7		"""
8
9		import os
10		import re
11		import json
12		import time
13		import logging
14		from typing import Dict, Any, List, Optional, Tuple, Set, Union
15
16		# Set up logging
17		logger = logging.getLogger(__name__)
18
19		class FunctionSelectionFeedback:
20		"""Manages feedback on function selection for continuous improvement."""
21
22		def __init__(self, feedback_file_path=None):
23		"""
24		Initialize the feedback system.
25
26		Args:
27		feedback_file_path: Path to the feedback file (default: ~/.ollamatools/function_selection_feedback.jsonl)
28		"""
29		if feedback_file_path is None:
30		self.feedback_file_path = os.path.join(
31		os.path.expanduser('~'),
32		'.ollamatools',
33		'function_selection_feedback.jsonl'
34		)
35		else:
36		self.feedback_file_path = feedback_file_path
37
38		# Create directory if it doesn't exist
39		os.makedirs(os.path.dirname(self.feedback_file_path), exist_ok=True)
40
41		# Cache of recent feedback for quick lookup
42		self.recent_feedback = []
43		self.max_cache_size = 1000
44
45		# Load existing feedback into cache
46		self._load_recent_feedback()
47
48		def _load_recent_feedback(self):
49		"""Load recent feedback entries into memory cache."""
50		try:
51		if os.path.exists(self.feedback_file_path):
52		with open(self.feedback_file_path, 'r') as f:
53		# Load the most recent entries (up to max_cache_size)
54		lines = f.readlines()
55		for line in lines[-self.max_cache_size:]:
56		try:
57		entry = json.loads(line.strip())
58		self.recent_feedback.append(entry)
59		except json.JSONDecodeError:
60		continue
61		logger.info(f"Loaded {len(self.recent_feedback)} feedback entries from {self.feedback_file_path}")
62		else:
63		logger.info(f"No feedback file found at {self.feedback_file_path}, starting with empty feedback")
64		except Exception as e:
65		logger.error(f"Error loading feedback data: {str(e)}")
66
67		def record_explicit_feedback(self, query, intended_function, actual_function, user_correction=True):
68		"""
69		Record explicit feedback when a function selection was incorrect.
70
71		Args:
72		query: The original user query
73		intended_function: The function that should have been used
74		actual_function: The function that was actually used
75		user_correction: Whether this feedback came directly from a user
76
77		Returns:
78		dict: The recorded feedback entry
79		"""
80		feedback_entry = {
81		"query": query,
82		"intended_function": intended_function,
83		"actual_function": actual_function,
84		"timestamp": time.time(),
85		"source": "user_correction" if user_correction else "system_detection",
86		"similarity_score": self._calculate_query_similarity(query)
87		}
88
89		# Add to cache
90		self.recent_feedback.append(feedback_entry)
91		if len(self.recent_feedback) > self.max_cache_size:
92		self.recent_feedback.pop(0)
93
94		# Write to file
95		try:
96		with open(self.feedback_file_path, 'a') as f:
97		f.write(json.dumps(feedback_entry) + '\n')
98		logger.info(f"Recorded explicit feedback for query: {query}")
99		except Exception as e:
100		logger.error(f"Error recording feedback: {str(e)}")
101
102		return feedback_entry
103
104		def record_implicit_feedback(self, query, function, confidence_score, execution_result):
105		"""
106		Record implicit feedback based on execution results.
107
108		Args:
109		query: The original user query
110		function: The function that was used
111		confidence_score: Confidence score for the function selection (0-1)
112		execution_result: Result of executing the function
113
114		Returns:
115		dict: The recorded feedback entry
116		"""
117		# Determine if the execution was likely successful
118		success_indicators = {
119		"execute_command": lambda r: r.get("returncode", 1) == 0 and not r.get("stderr"),
120		"read_file": lambda r: not (isinstance(r, str) and (r.startswith("Error") or r.startswith("Path validation error"))),
121		"search_web": lambda r: len(r) > 0 if isinstance(r, list) else True
122		}
123
124		likely_success = False
125		if function in success_indicators:
126		likely_success = success_indicators[function](execution_result)
127
128		feedback_entry = {
129		"query": query,
130		"function": function,
131		"timestamp": time.time(),
132		"source": "implicit_feedback",
133		"confidence_score": confidence_score,
134		"likely_success": likely_success,
135		"similarity_score": self._calculate_query_similarity(query)
136		}
137
138		# Only record if confidence was low or execution likely failed
139		if confidence_score < 0.7 or not likely_success:
140		# Add to cache
141		self.recent_feedback.append(feedback_entry)
142		if len(self.recent_feedback) > self.max_cache_size:
143		self.recent_feedback.pop(0)
144
145		# Write to file
146		try:
147		with open(self.feedback_file_path, 'a') as f:
148		f.write(json.dumps(feedback_entry) + '\n')
149		logger.info(f"Recorded implicit feedback for query: {query}")
150		except Exception as e:
151		logger.error(f"Error recording feedback: {str(e)}")
152
153		return feedback_entry
154
155		def _calculate_query_similarity(self, query):
156		"""
157		Calculate similarity between this query and previous queries.
158
159		Args:
160		query: The query to compare
161
162		Returns:
163		list: List of similar queries with their similarity scores
164		"""
165		# Simple word-based similarity for now
166		query_words = set(query.lower().split())
167
168		similarities = []
169		for entry in self.recent_feedback:
170		past_query = entry.get("query", "")
171		past_words = set(past_query.lower().split())
172
173		# Calculate Jaccard similarity
174		if not query_words or not past_words:
175		similarity = 0
176		else:
177		intersection = len(query_words.intersection(past_words))
178		union = len(query_words.union(past_words))
179		similarity = intersection / union if union > 0 else 0
180
181		if similarity > 0.3: # Only include reasonably similar queries
182		similarities.append({
183		"query": past_query,
184		"similarity": similarity,
185		"intended_function": entry.get("intended_function") or entry.get("function"),
186		"timestamp": entry.get("timestamp")
187		})
188
189		# Sort by similarity score
190		similarities.sort(key=lambda x: x["similarity"], reverse=True)
191		return similarities[:5] # Return top 5 similar queries
192
193		def get_function_suggestion(self, query):
194		"""
195		Get function suggestion based on feedback history.
196
197		Args:
198		query: The user query
199
200		Returns:
201		tuple: (suggested_function, confidence_score, similar_queries)
202		"""
203		# Get similar queries
204		similar_queries = self._calculate_query_similarity(query)
205
206		if not similar_queries:
207		return None, 0, []
208
209		# Count function occurrences weighted by similarity
210		function_scores = {}
211		for item in similar_queries:
212		function = item["intended_function"]
213		similarity = item["similarity"]
214
215		if function not in function_scores:
216		function_scores[function] = 0
217
218		function_scores[function] += similarity
219
220		# Find the function with the highest score
221		best_function = None
222		highest_score = 0
223
224		for function, score in function_scores.items():
225		if score > highest_score:
226		highest_score = score
227		best_function = function
228
229		# Calculate confidence based on the highest score and number of similar queries
230		confidence = highest_score / len(similar_queries) if similar_queries else 0
231
232		return best_function, confidence, similar_queries
233
234		def generate_improved_patterns(self):
235		"""
236		Generate improved regex patterns based on feedback data.
237
238		Returns:
239		dict: Dictionary mapping regex patterns to functions
240		"""
241		# Analyze the feedback data
242		analysis = self.analyze_function_selection_patterns()
243
244		if "error" in analysis:
245		return {}
246
247		improved_patterns = {}
248
249		# Process mismatches to generate patterns
250		for mismatch_info in analysis.get("top_mismatches", []):
251		mismatch, data = mismatch_info
252		actual, intended = mismatch.split(" → ")
253
254		# Skip if we don't have enough examples
255		if len(data["examples"]) < 2:
256		continue
257
258		# Generate a pattern from the examples
259		pattern = self._generate_regex_from_examples(data["examples"], intended)
260		if pattern:
261		improved_patterns[pattern] = intended
262
263		# Process query patterns
264		for pattern_info in analysis.get("top_query_patterns", []):
265		pattern_text, data = pattern_info
266		functions = data["functions"]
267
268		if not functions:
269		continue
270
271		# Find the most common function
272		most_common = max(functions.items(), key=lambda x: x[1])
273		function_name, count = most_common
274
275		# Only include if it's a strong pattern (used for the same function >70% of the time)
276		confidence = count / data["count"]
277		if confidence > 0.7:
278		# Create a regex pattern from the text pattern
279		regex_pattern = r"\b" + re.escape(pattern_text) + r".*"
280		improved_patterns[regex_pattern] = function_name
281
282		return improved_patterns
283
284		def _generate_regex_from_examples(self, examples, function):
285		"""
286		Generate a regex pattern from example queries.
287
288		Args:
289		examples: List of example queries
290		function: The target function
291
292		Returns:
293		str: Regex pattern
294		"""
295		if not examples:
296		return None
297
298		# Extract common words across examples
299		common_words = set(examples[0].lower().split())
300		for example in examples[1:]:
301		common_words &= set(example.lower().split())
302
303		# Remove very common words
304		stop_words = {"what", "is", "the", "a", "an", "in", "on", "at", "to", "for", "of", "with", "by"}
305		common_words -= stop_words
306
307		if not common_words:
308		# If no common words, try to find common starting words
309		starts_with = []
310		for example in examples:
311		words = example.lower().split()
312		if words:
313		starts_with.append(words[0])
314
315		# If we have consistent starting words
316		if len(set(starts_with)) == 1 and starts_with[0] not in stop_words:
317		return r"\b" + re.escape(starts_with[0]) + r".*"
318
319		return None
320
321		# Create a pattern with the common words
322		pattern_parts = []
323		for word in common_words:
324		pattern_parts.append(r"\b" + re.escape(word) + r"\b")
325
326		# Combine with lookaheads for flexibility in word order
327		pattern = "".join(f"(?=.{part})" for part in pattern_parts) + "."
328		return pattern
329
330		def analyze_function_selection_patterns(self):
331		"""
332		Analyze patterns in function selection feedback.
333
334		Returns:
335		dict: Analysis results
336		"""
337		try:
338		# Load all feedback data
339		all_feedback = []
340		if os.path.exists(self.feedback_file_path):
341		with open(self.feedback_file_path, 'r') as f:
342		for line in f:
343		try:
344		entry = json.loads(line.strip())
345		all_feedback.append(entry)
346		except json.JSONDecodeError:
347		continue
348
349		if not all_feedback:
350		return {"error": "No feedback data available"}
351
352		# Analyze common mismatches
353		mismatches = {}
354		for entry in all_feedback:
355		if "intended_function" in entry and "actual_function" in entry:
356		if entry["intended_function"] != entry["actual_function"]:
357		key = f"{entry['actual_function']} → {entry['intended_function']}"
358		if key not in mismatches:
359		mismatches[key] = {
360		"count": 0,
361		"examples": []
362		}
363
364		mismatches[key]["count"] += 1
365		if len(mismatches[key]["examples"]) < 5: # Limit examples
366		mismatches[key]["examples"].append(entry["query"])
367
368		# Sort mismatches by frequency
369		sorted_mismatches = sorted(
370		[(k, v) for k, v in mismatches.items()],
371		key=lambda x: x[1]["count"],
372		reverse=True
373		)
374
375		# Analyze query patterns
376		query_patterns = {}
377		for entry in all_feedback:
378		query = entry.get("query", "")
379		words = query.lower().split()
380
381		# Extract potential patterns (first 2-3 words)
382		if len(words) >= 2:
383		pattern = " ".join(words[:2])
384		if pattern not in query_patterns:
385		query_patterns[pattern] = {
386		"count": 0,
387		"functions": {}
388		}
389
390		query_patterns[pattern]["count"] += 1
391
392		function = entry.get("intended_function") or entry.get("function")
393		if function:
394		if function not in query_patterns[pattern]["functions"]:
395		query_patterns[pattern]["functions"][function] = 0
396		query_patterns[pattern]["functions"][function] += 1
397
398		# Sort patterns by frequency
399		sorted_patterns = sorted(
400		[(k, v) for k, v in query_patterns.items()],
401		key=lambda x: x[1]["count"],
402		reverse=True
403		)
404
405		return {
406		"total_feedback_entries": len(all_feedback),
407		"top_mismatches": sorted_mismatches[:10],
408		"top_query_patterns": sorted_patterns[:10],
409		"improvement_suggestions": self._generate_improvement_suggestions(sorted_mismatches, sorted_patterns)
410		}
411		except Exception as e:
412		logger.error(f"Error analyzing feedback: {str(e)}")
413		return {"error": f"Error analyzing feedback: {str(e)}"}
414
415		def _generate_improvement_suggestions(self, mismatches, patterns):
416		"""
417		Generate suggestions for improving function selection.
418
419		Args:
420		mismatches: List of function mismatches
421		patterns: List of query patterns
422
423		Returns:
424		list: Improvement suggestions
425		"""
426		suggestions = []
427
428		# Suggest improvements based on common mismatches
429		if mismatches:
430		for mismatch, data in mismatches[:5]:
431		actual, intended = mismatch.split(" → ")
432		examples = data["examples"]
433
434		if examples:
435		pattern_words = set()
436		for example in examples:
437		pattern_words.update(example.lower().split())
438
439		# Filter out common words
440		common_words = {"what", "is", "the", "a", "an", "in", "on", "at", "to", "for", "of", "with", "by"}
441		pattern_words = pattern_words - common_words
442
443		if pattern_words:
444		suggestion = {
445		"type": "function_mapping",
446		"description": f"Improve mapping from '{actual}' to '{intended}'",
447		"examples": examples[:3],
448		"suggested_keywords": list(pattern_words)[:5],
449		"implementation_hint": f"Add pattern matching for these keywords to select '{intended}'"
450		}
451		suggestions.append(suggestion)
452
453		# Suggest improvements based on query patterns
454		if patterns:
455		for pattern, data in patterns[:5]:
456		functions = data["functions"]
457		if functions:
458		# Find the most common function for this pattern
459		most_common = max(functions.items(), key=lambda x: x[1])
460		function_name, count = most_common
461
462		suggestion = {
463		"type": "query_pattern",
464		"description": f"Add explicit handling for queries starting with '{pattern}'",
465		"suggested_function": function_name,
466		"confidence": count / data["count"],
467		"implementation_hint": f"Add pattern matching for queries starting with '{pattern}' to select '{function_name}'"
468		}
469		suggestions.append(suggestion)
470
471		return suggestions
472
473		def detect_command_intent(message, feedback_system=None):
474		"""
475		Detect if a message is asking for system information that requires a command.
476
477		Args:
478		message: User message
479		feedback_system: Optional FunctionSelectionFeedback instance
480
481		Returns:
482		Tuple[bool, Optional[str]]: (has_command_intent, suggested_command)
483		"""
484		# First check feedback-based patterns if available
485		if feedback_system:
486		suggested_function, confidence, similar_queries = feedback_system.get_function_suggestion(message)
487
488		if suggested_function == "execute_command" and confidence > 0.7:
489		# Extract command from similar queries
490		command = extract_command_from_similar_queries(message, similar_queries)
491		if command:
492		return True, command
493
494		# Map common questions to shell commands
495		command_patterns = {
496		r"(?:what\|which)(?:\s+is)?(?:\s+the)?(?:\s+current)?\s+directory": "pwd",
497		r"where\s+am\s+i": "pwd",
498		r"list\s+(?:the\s+)?(?:files\|contents)": "ls",
499		r"show\s+(?:the\s+)?(?:files\|contents)": "ls",
500		r"(?:what\|which)(?:\s+is)?(?:\s+the)?(?:\s+current)?\s+user": "whoami",
501		r"who\s+am\s+i": "whoami",
502		r"(?:what\|which)(?:\s+is)?(?:\s+the)?(?:\s+current)?\s+date": "date",
503		r"(?:what\|which)(?:\s+is)?(?:\s+the)?(?:\s+current)?\s+time": "date",
504		# Add more patterns as needed
505		}
506
507		# Check if message matches any pattern
508		for pattern, command in command_patterns.items():
509		if re.search(pattern, message.lower()):
510		return True, command
511
512		return False, None
513
514		def extract_command_from_similar_queries(message, similar_queries):
515		"""
516		Extract the most appropriate command from similar queries.
517
518		Args:
519		message: Current user message
520		similar_queries: List of similar queries with their metadata
521
522		Returns:
523		str: Extracted command or None
524		"""
525		# Count command occurrences in similar queries
526		command_counts = {}
527
528		for query_info in similar_queries:
529		# Skip if not execute_command
530		if query_info.get("intended_function") != "execute_command":
531		continue
532
533		# Try to extract the command from the query
534		query = query_info.get("query", "")
535
536		# Check for common command patterns
537		command_mapping = {
538		r"directory\|where\s+am\s+i": "pwd",
539		r"list\s+files\|show\s+files\|contents": "ls",
540		r"user\|who\s+am\s+i": "whoami",
541		r"date\|time": "date"
542		}
543
544		for pattern, cmd in command_mapping.items():
545		if re.search(pattern, query.lower()):
546		if cmd not in command_counts:
547		command_counts[cmd] = 0
548		command_counts[cmd] += query_info.get("similarity", 0.5)
549
550		# Return the most common command
551		if command_counts:
552		return max(command_counts.items(), key=lambda x: x[1])[0]
553
554		return None
555
556		def format_command_result(command, result):
557		"""
558		Format the result of a command execution in a user-friendly way.
559
560		Args:
561		command: The command that was executed
562		result: The command execution result
563
564		Returns:
565		str: Formatted result
566		"""
567		if "error" in result:
568		return f"Error executing command: {result['error']}"
569
570		stdout = result.get("stdout", "").strip()
571		stderr = result.get("stderr", "").strip()
572
573		if command == "pwd":
574		return f"You are currently in: {stdout}"
575		elif command == "ls":
576		return f"Files in the current directory:\n{stdout}"
577		elif command == "whoami":
578		return f"Current user: {stdout}"
579		elif command == "date":
580		return f"Current date and time: {stdout}"
581		else:
582		# Generic formatting
583		response = f"Command result:\n{stdout}"
584		if stderr:
585		response += f"\nErrors/Warnings:\n{stderr}"
586		return response

ollamatools-new/ollamatools/plugins/standard/function_calling.py +0 -38

…	…	@@ -1,38 +0,0 @@
1		"""Plugin for semantic function calling."""
2
3		from typing import Any, Dict, List, Optional
4
5		from ollamatools.plugins.base import StandardPlugin
6
7		class FunctionCallingPlugin(StandardPlugin):
8		"""Plugin for semantic function calling."""
9
10		@property
11		def name(self) -> str:
12		return "function_calling"
13
14		@property
15		def version(self) -> str:
16		return "0.1.0"
17
18		def initialize(self, config: Dict[str, Any]) -> None:
19		"""Initialize the plugin with the given configuration."""
20		self.config = config
21		self.functions = {}
22
23		def shutdown(self) -> None:
24		"""Shutdown the plugin and release any resources."""
25		pass
26
27		def register_function(self, name: str, function: callable, description: str, parameters: Dict[str, Any]) -> None:
28		"""Register a function that can be called semantically."""
29		self.functions[name] = {
30		"function": function,
31		"description": description,
32		"parameters": parameters,
33		}
34
35		def process_message(self, message: Dict[str, Any]) -> Dict[str, Any]:
36		"""Process a message and extract function calls."""
37		# Implementation
38		return message

ollamatools-new/ollamatools/plugins/standard/issues.py +0 -189

…	…	@@ -1,189 +0,0 @@
1		"""
2		Issue tracking functionality for the ModularOllamaAgent.
3
4		This module provides functions for managing issues, including
5		loading, saving, adding, listing, and updating issues.
6		"""
7
8		import os
9		import json
10		import time
11		from typing import Dict, Any, List, Optional
12
13		from .utils import print_info, print_error, print_success
14
15		def load_issues(agent) -> None:
16		"""
17		Load saved issues from file.
18
19		Args:
20		agent: The agent instance
21		"""
22		if os.path.exists(agent.issues_file):
23		try:
24		with open(agent.issues_file, 'r') as f:
25		data = json.load(f)
26		agent.issues = data.get('issues', [])
27		agent.issue_id_counter = data.get('next_id', 1)
28		agent.print_info(f"Loaded {len(agent.issues)} issues from {agent.issues_file}")
29		except Exception as e:
30		agent.print_error(f"Failed to load issues: {str(e)}")
31		agent.issues = []
32		agent.issue_id_counter = 1
33
34		def save_issues(agent) -> None:
35		"""
36		Save issues to file.
37
38		Args:
39		agent: The agent instance
40		"""
41		try:
42		with open(agent.issues_file, 'w') as f:
43		json.dump({
44		'issues': agent.issues,
45		'next_id': agent.issue_id_counter
46		}, f, indent=2)
47		agent.print_info(f"Saved {len(agent.issues)} issues to {agent.issues_file}")
48		except Exception as e:
49		agent.print_error(f"Failed to save issues: {str(e)}")
50
51		def add_issue(agent, title: str, description: str, severity: str = 'medium') -> int:
52		"""
53		Add a new issue to the tracker.
54
55		Args:
56		agent: The agent instance
57		title: Issue title
58		description: Issue description
59		severity: Issue severity (low, medium, high)
60
61		Returns:
62		int: Issue ID
63		"""
64		# Validate severity
65		if severity not in ['low', 'medium', 'high']:
66		severity = 'medium'
67
68		# Create issue
69		issue = {
70		'id': agent.issue_id_counter,
71		'title': title,
72		'description': description,
73		'severity': severity,
74		'status': 'open',
75		'created_at': time.strftime('%Y-%m-%d %H:%M:%S'),
76		'updated_at': time.strftime('%Y-%m-%d %H:%M:%S')
77		}
78
79		# Add to list and increment counter
80		agent.issues.append(issue)
81		agent.issue_id_counter += 1
82
83		# Save issues
84		save_issues(agent)
85
86		agent.print_success(f"Issue #{issue['id']} added: {title}")
87		return issue['id']
88
89		def list_issues(agent, status: Optional[str] = None) -> List[Dict[str, Any]]:
90		"""
91		List all issues, optionally filtered by status.
92
93		Args:
94		agent: The agent instance
95		status: Optional status filter (open, resolved)
96
97		Returns:
98		List[Dict[str, Any]]: List of issues
99		"""
100		# Filter issues if status provided
101		filtered_issues = agent.issues
102		if status:
103		filtered_issues = [i for i in agent.issues if i['status'] == status]
104
105		# Print issues
106		if not filtered_issues:
107		agent.print_info("No issues found")
108		return []
109
110		agent.print_info(f"Issues ({len(filtered_issues)}):")
111		print("\nID \| Severity \| Status \| Title")
112		print("----\|----------\|----------\|-------------------")
113
114		for issue in filtered_issues:
115		print(f"{issue['id']:<4}\| {issue['severity']:<9}\| {issue['status']:<9}\| {issue['title']}")
116
117		return filtered_issues
118
119		def get_issue(agent, issue_id: int) -> Optional[Dict[str, Any]]:
120		"""
121		Get details of a specific issue.
122
123		Args:
124		agent: The agent instance
125		issue_id: Issue ID
126
127		Returns:
128		Optional[Dict[str, Any]]: Issue details or None if not found
129		"""
130		# Find issue
131		for issue in agent.issues:
132		if issue['id'] == issue_id:
133		# Print issue details
134		agent.print_info(f"Issue #{issue['id']}: {issue['title']}")
135		print(f"Status: {issue['status']}")
136		print(f"Severity: {issue['severity']}")
137		print(f"Created: {issue['created_at']}")
138		print(f"Updated: {issue['updated_at']}")
139		print("\nDescription:")
140		print(issue['description'])
141		return issue
142
143		agent.print_error(f"Issue #{issue_id} not found")
144		return None
145
146		def update_issue(
147		agent,
148		issue_id: int,
149		status: Optional[str] = None,
150		severity: Optional[str] = None,
151		description: Optional[str] = None
152		) -> bool:
153		"""
154		Update an existing issue.
155
156		Args:
157		agent: The agent instance
158		issue_id: Issue ID
159		status: New status (open, resolved)
160		severity: New severity (low, medium, high)
161		description: New description
162
163		Returns:
164		bool: True if updated successfully
165		"""
166		# Find issue
167		for i, issue in enumerate(agent.issues):
168		if issue['id'] == issue_id:
169		# Update fields
170		if status and status in ['open', 'resolved']:
171		agent.issues[i]['status'] = status
172
173		if severity and severity in ['low', 'medium', 'high']:
174		agent.issues[i]['severity'] = severity
175
176		if description:
177		agent.issues[i]['description'] = description
178
179		# Update timestamp
180		agent.issues[i]['updated_at'] = time.strftime('%Y-%m-%d %H:%M:%S')
181
182		# Save issues
183		save_issues(agent)
184
185		agent.print_success(f"Issue #{issue_id} updated")
186		return True
187
188		agent.print_error(f"Issue #{issue_id} not found")
189		return False

ollamatools-new/ollamatools/plugins/standard/search.py +0 -208

…	…	@@ -1,208 +0,0 @@
1		"""
2		Search functionality for the ModularOllamaAgent.
3
4		This module provides search-related functionality for the ModularOllamaAgent,
5		including natural language detection of search intent, search execution, and
6		search results processing.
7		"""
8
9		import time
10		import logging
11		from typing import Dict, Any, List, Optional, Set
12
13		from ollamatools.plugins.tools.web import search_web
14		from ollamatools.core.errors import WebError
15		from ollamatools.plugins.integrations.opensearch.tools import opensearch_query
16		from .utils import print_status, print_success, print_error, print_warning, print_info
17
18		# Set up logger
19		logger = logging.getLogger(__name__)
20
21		def detect_factual_intent(message: str) -> bool:
22		"""
23		Detect if a message is likely asking for factual information
24		that would benefit from search.
25
26		Args:
27		message: User message
28
29		Returns:
30		bool: True if message likely needs external information
31		"""
32		# Basic topics that often require external information
33		external_topics = [
34		"recipe", "weather", "news", "sports", "movie", "book", "author",
35		"product", "review", "price", "stock", "company", "history",
36		"person", "celebrity", "politician", "event", "scientific",
37		"research", "study", "statistic", "data", "definition"
38		]
39
40		# Check if message contains any of these topics
41		contains_topic = any(topic in message.lower() for topic in external_topics)
42
43		# Check if it's a complex question (longer questions tend to be more specific)
44		is_complex = len(message.split()) > 6
45
46		# Simple heuristic: if it contains a topic of interest and is reasonably complex
47		return contains_topic and is_complex
48
49		def extract_search_context(search_results: Dict[str, Any]) -> str:
50		"""
51		Extract useful context from search results.
52
53		Args:
54		search_results: Results from opensearch_query
55
56		Returns:
57		str: Formatted context for the agent
58		"""
59		context_parts = []
60
61		# Add direct answer if available
62		if search_results.get("answer"):
63		try:
64		# Handle both string and dict answers
65		answer = search_results["answer"]
66		if isinstance(answer, dict):
67		answer_text = answer.get("content", str(answer))
68		else:
69		answer_text = str(answer)
70		context_parts.append(f"Direct answer: {answer_text}")
71		except Exception as e:
72		logger.warning(f"Error extracting answer for context: {str(e)}")
73
74		# Add snippets from results
75		if search_results.get("results"):
76		context_parts.append("Information from search results:")
77		for i, result in enumerate(search_results["results"][:3]): # Limit to 3 results
78		title = result.get("title", "Untitled")
79		content = result.get("content", "No content")
80		url = result.get("url", "")
81		context_parts.append(f"- {title}: {content} (Source: {url})")
82
83		# Join everything with newlines
84		return "\n".join(context_parts)
85
86		def execute_search(
87		query: str,
88		opensearch_config: Dict[str, Any],
89		permission_manager: Any,
90		execution_history: List[Dict[str, Any]]
91		) -> Dict[str, Any]:
92		"""
93		Execute a search using OpenSearchNG or fall back to standard search.
94
95		Args:
96		query: Search query
97		opensearch_config: OpenSearchNG configuration
98		permission_manager: Permission manager instance
99		execution_history: History of executed functions
100
101		Returns:
102		dict: Search results
103		"""
104		try:
105		print_status(f"Searching for: {query}")
106
107		# Try to use OpenSearchNG first
108		try:
109		# Execute the search with OpenSearchNG
110		results = opensearch_query(
111		query=query,
112		config=opensearch_config,
113		permission_manager=permission_manager
114		)
115
116		# Show the number of results
117		num_results = len(results.get("results", []))
118		print_success(f"Found {num_results} results using OpenSearchNG")
119
120		# Show suggested answer if available
121		if results.get("answer"):
122		try:
123		# Handle both string and dict answers
124		answer = results["answer"]
125		if isinstance(answer, dict):
126		answer_text = answer.get("content", str(answer))
127		else:
128		answer_text = str(answer)
129		print_info(f"Direct answer: {answer_text}")
130		except Exception as e:
131		logger.warning(f"Error displaying answer: {str(e)}")
132
133		# Register in execution history
134		execution_history.append({
135		"function": "opensearch_query",
136		"args": {"query": query},
137		"timestamp": time.time(),
138		"success": True,
139		"num_results": num_results
140		})
141
142		return results
143
144		except (WebError, ImportError, TypeError) as e:
145		# If OpenSearchNG fails, fall back to standard web search
146		if "Could not connect to OpenSearchNG server" in str(e) or isinstance(e, ImportError):
147		print_warning("OpenSearchNG is not available. Using fallback search method.")
148
149		# Use the fallback search method from web_tools
150		search_results = search_web(
151		query,
152		num_results=5,
153		permission_manager=permission_manager
154		)
155
156		# Convert results to the expected format
157		fallback_results = {
158		"query": query,
159		"results": [],
160		"suggestion": None,
161		"answer": None,
162		"engines": ["fallback"],
163		"timing": {"total": 0}
164		}
165
166		for result in search_results:
167		fallback_results["results"].append({
168		"title": result.get("title", ""),
169		"url": result.get("url", ""),
170		"content": result.get("snippet", ""),
171		"engine": "fallback"
172		})
173
174		num_results = len(fallback_results["results"])
175		print_success(f"Found {num_results} results using fallback search")
176
177		# Register in execution history
178		execution_history.append({
179		"function": "search_web_fallback",
180		"args": {"query": query},
181		"timestamp": time.time(),
182		"success": True,
183		"num_results": num_results
184		})
185
186		return fallback_results
187		else:
188		# Re-raise other WebError exceptions
189		raise
190
191		except Exception as e:
192		print_error(f"Search failed: {str(e)}")
193
194		# Register in execution history
195		execution_history.append({
196		"function": "execute_search",
197		"args": {"query": query},
198		"timestamp": time.time(),
199		"success": False,
200		"error": str(e)
201		})
202
203		# Return empty results
204		return {
205		"query": query,
206		"results": [],
207		"error": str(e)
208		}

ollamatools-new/ollamatools/plugins/tools/__init__.py +0 -0

(no content changes — binary or rename)

ollamatools-new/ollamatools/plugins/tools/doc.py +0 -278

…	…	@@ -1,278 +0,0 @@
1		"""
2		Documentation Tools for OllamaTools.
3
4		This module provides utilities for generating and processing documentation.
5		It includes tools for creating formatted documentation, extracting relevant
6		context from documents, and summarizing text.
7
8		All functions support integration with the permission system for security.
9		"""
10
11		import markdown
12		import re
13		from collections import Counter
14		from typing import Optional, Dict, Any, List
15
16
17		def generate_markdown(
18		content: str,
19		title: str,
20		format_type: str = 'markdown',
21		permission_manager = None
22		) -> str:
23		"""
24		Generate formatted documentation.
25
26		Args:
27		content: Document content to format
28		title: Document title
29		format_type: Output format ('markdown' or 'html')
30		permission_manager: Optional permission manager for security checks
31
32		Returns:
33		str: Formatted documentation
34
35		Raises:
36		ValueError: If format_type is not supported
37		PermissionError: If permission is denied
38		"""
39		# Permission check
40		if permission_manager is not None:
41		args = {
42		'content_preview': content[:50] + ('...' if len(content) > 50 else ''),
43		'title': title,
44		'format_type': format_type
45		}
46		if not permission_manager.request_permission('generate_markdown', args):
47		raise PermissionError("Permission denied to generate markdown document")
48
49		# Input validation
50		if not content or not isinstance(content, str):
51		raise ValueError("Content must be a non-empty string")
52
53		if not title or not isinstance(title, str):
54		raise ValueError("Title must be a non-empty string")
55
56		if format_type not in ['markdown', 'html']:
57		raise ValueError("Format type must be 'markdown' or 'html'")
58
59		# Create the formatted document
60		if format_type == 'markdown':
61		# For markdown, we'll create a simple markdown document with a title
62		formatted_doc = f"# {title}\n\n{content}"
63		return formatted_doc
64		else: # html
65		# For HTML, we'll convert the markdown to HTML
66		html_content = markdown.markdown(content)
67		formatted_doc = f"""<!DOCTYPE html>
68		<html>
69		<head>
70		<title>{title}</title>
71		<meta charset="UTF-8">
72		<meta name="viewport" content="width=device-width, initial-scale=1.0">
73		</head>
74		<body>
75		<h1>{title}</h1>
76		{html_content}
77		</body>
78		</html>"""
79		return formatted_doc
80
81
82		def extract_context(
83		query: str,
84		documents: List[str],
85		max_length: int = 1000,
86		permission_manager = None
87		) -> str:
88		"""
89		Extract relevant context from documents based on a query.
90
91		This function extracts passages from documents that are most relevant to the query
92		based on keyword matching. It prioritizes sentences containing query keywords.
93
94		Args:
95		query: Search query to find relevant contexts
96		documents: List of document contents
97		max_length: Maximum total length of extracted context
98		permission_manager: Optional permission manager for security checks
99
100		Returns:
101		str: Extracted context
102
103		Raises:
104		ValueError: If inputs are invalid
105		PermissionError: If permission is denied
106		"""
107		# Permission check
108		if permission_manager is not None:
109		args = {
110		'query': query,
111		'documents_count': len(documents) if documents else 0,
112		'max_length': max_length
113		}
114		if not permission_manager.request_permission('extract_context', args):
115		raise PermissionError("Permission denied to extract context from documents")
116
117		# Input validation
118		if not query or not isinstance(query, str):
119		raise ValueError("Query must be a non-empty string")
120
121		if not documents or not isinstance(documents, list):
122		raise ValueError("Documents must be a non-empty list of strings")
123
124		if not all(isinstance(doc, str) for doc in documents):
125		raise ValueError("All documents must be strings")
126
127		if not max_length or not isinstance(max_length, int) or max_length <= 0:
128		raise ValueError("Max length must be a positive integer")
129
130		# Extract keywords from the query
131		# Normalize by removing common words and punctuation
132		query = query.lower()
133		query_words = re.findall(r'\b\w+\b', query)
134
135		# Remove common stop words
136		stop_words = {'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are', 'of', 'to', 'in',
137		'for', 'with', 'by', 'at', 'on', 'from'}
138		keywords = [word for word in query_words if word not in stop_words]
139
140		if not keywords:
141		# Use the original words if all were stop words
142		keywords = query_words
143
144		# Find sentences in documents containing keywords
145		relevant_sentences = []
146		for doc in documents:
147		# Split documents into sentences (simple heuristic)
148		sentences = re.split(r'(?<=[.!?])\s+', doc)
149
150		for sentence in sentences:
151		# Check for keyword presence
152		sentence_lower = sentence.lower()
153		sentence_words = set(re.findall(r'\b\w+\b', sentence_lower))
154
155		# Calculate relevance score as the number of keywords found
156		keywords_found = sum(1 for keyword in keywords if keyword in sentence_words)
157
158		if keywords_found > 0:
159		relevant_sentences.append((sentence, keywords_found))
160
161		# Sort sentences by relevance (number of keywords found)
162		relevant_sentences.sort(key=lambda x: x[1], reverse=True)
163
164		# Build context up to max_length
165		context = []
166		current_length = 0
167
168		for sentence, _ in relevant_sentences:
169		if current_length + len(sentence) + 1 <= max_length: # +1 for space
170		context.append(sentence)
171		current_length += len(sentence) + 1
172		else:
173		# If we can't fit the next sentence, stop
174		break
175
176		# If no relevant sentences found, return empty string
177		if not context:
178		return ""
179
180		return " ".join(context)
181
182
183		def summarize_text(
184		text: str,
185		max_length: int = 500,
186		permission_manager = None
187		) -> str:
188		"""
189		Generate a summary of text using extractive summarization.
190
191		This function extracts the most important sentences from the text
192		based on word frequency analysis.
193
194		Args:
195		text: Text to summarize
196		max_length: Maximum summary length
197		permission_manager: Optional permission manager for security checks
198
199		Returns:
200		str: Text summary
201
202		Raises:
203		ValueError: If inputs are invalid
204		PermissionError: If permission is denied
205		"""
206		# Permission check
207		if permission_manager is not None:
208		args = {
209		'text_preview': text[:50] + ('...' if len(text) > 50 else ''),
210		'text_length': len(text),
211		'max_length': max_length
212		}
213		if not permission_manager.request_permission('summarize_text', args):
214		raise PermissionError("Permission denied to summarize text")
215
216		# Input validation
217		if not text or not isinstance(text, str):
218		raise ValueError("Text must be a non-empty string")
219
220		if not max_length or not isinstance(max_length, int) or max_length <= 0:
221		raise ValueError("Max length must be a positive integer")
222
223		# If text is already shorter than max_length, return it as is
224		if len(text) <= max_length:
225		return text
226
227		# Split text into sentences
228		sentences = re.split(r'(?<=[.!?])\s+', text)
229
230		# Handle case with only one sentence
231		if len(sentences) <= 1:
232		return text[:max_length] + "..." if len(text) > max_length else text
233
234		# Extract all words, lowercase them, and remove stop words
235		stop_words = {'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are', 'of', 'to', 'in',
236		'for', 'with', 'by', 'at', 'on', 'from'}
237
238		words = []
239		for sentence in sentences:
240		sentence_words = re.findall(r'\b\w+\b', sentence.lower())
241		words.extend([word for word in sentence_words if word not in stop_words])
242
243		# Count word frequencies
244		word_freq = Counter(words)
245
246		# Score sentences based on word frequencies
247		sentence_scores = []
248		for i, sentence in enumerate(sentences):
249		sentence_words = re.findall(r'\b\w+\b', sentence.lower())
250		score = sum(word_freq[word] for word in sentence_words if word not in stop_words)
251		# Normalize by sentence length to avoid bias towards longer sentences
252		if len(sentence_words) > 0:
253		score = score / len(sentence_words)
254		sentence_scores.append((i, sentence, score))
255
256		# Sort sentences by score
257		sentence_scores.sort(key=lambda x: x[2], reverse=True)
258
259		# Select top sentences and restore original order
260		top_sentences = sentence_scores[:len(sentences)//3 + 1] # Take ~1/3 of sentences
261		top_sentences.sort(key=lambda x: x[0]) # Sort by original position
262
263		# Build summary up to max_length
264		summary = []
265		current_length = 0
266
267		for _, sentence, _ in top_sentences:
268		if current_length + len(sentence) + 1 <= max_length: # +1 for space
269		summary.append(sentence)
270		current_length += len(sentence) + 1
271		else:
272		break
273
274		# If no sentences fit, take the first sentence and truncate
275		if not summary:
276		return sentences[0][:max_length] + "..."
277
278		return " ".join(summary)

ollamatools-new/ollamatools/plugins/tools/system.py +0 -335

…	…	@@ -1,335 +0,0 @@
1		"""
2		System tools for OllamaTools.
3		This module provides functions for executing system commands and working with files.
4		"""
5
6		import os
7		import glob
8		import subprocess
9		import logging
10		import shlex
11		from typing import List, Optional, Dict, Any, Union, Set
12
13		# Import the command validation module
14		from ollamatools.core.utils.validators.command import (
15		validate_and_sanitize_command,
16		CommandValidationError,
17		get_secure_env,
18		is_sudo_command
19		)
20
21		# Import the path validation module
22		from ollamatools.core.utils.validators.path import (
23		validate_path_for_reading,
24		validate_path_for_writing,
25		validate_path_for_searching,
26		PathValidationError,
27		configure_path_sandbox
28		)
29
30		# Set up logging
31		logger = logging.getLogger(__name__)
32
33		def execute_command(
34		command: str,
35		args: Optional[List[str]] = None,
36		timeout: int = 60,
37		permission_manager = None,
38		custom_allowlist: Optional[set] = None
39		) -> Dict[str, Any]:
40		"""
41		Execute a shell command with arguments.
42
43		Args:
44		command: Command to execute. Can be a simple command or a full command line.
45		args: Command arguments (used only if command is a simple command)
46		timeout: Command timeout in seconds
47		permission_manager: Permission manager instance
48		custom_allowlist: Optional custom allowlist for commands
49
50		Returns:
51		dict: Result with stdout, stderr, and returncode
52		"""
53		# Handle case where command contains both command and arguments as a string
54		if isinstance(command, str) and ' ' in command and args is None:
55		# Split the command into parts using shlex to handle quotes properly
56		try:
57		command_parts = shlex.split(command)
58		command = command_parts[0]
59		args = command_parts[1:] if len(command_parts) > 1 else []
60		logger.info(f"Split command '{command}' with args {args}")
61		except ValueError as e:
62		logger.error(f"Invalid command syntax: {str(e)}")
63		return {"error": f"Invalid command syntax: {str(e)}"}
64
65		# Default args to empty list if None
66		if args is None:
67		args = []
68
69		# Convert args to strings if they aren't already
70		args = [str(arg) for arg in args]
71
72		# Detect sudo before validation to ensure we prompt correctly
73		is_sudo = is_sudo_command(command, args)
74		if is_sudo:
75		logger.warning(f"Sudo command detected: {command} {' '.join(args)}")
76
77		# Check permission if permission manager is provided
78		if permission_manager:
79		function_args = {"command": command, "args": args, "timeout": timeout}
80		permission_granted = permission_manager.request_permission(
81		function_name="execute_command",
82		args=function_args,
83		is_sudo=is_sudo
84		)
85
86		if not permission_granted:
87		logger.warning(f"Permission denied for command: {command} {' '.join(args)}")
88		return {"error": "Permission denied"}
89
90		try:
91		# Validate and sanitize the command and arguments
92		validated_command, validated_args = validate_and_sanitize_command(
93		command, args, custom_allowlist
94		)
95
96		# Construct the full command
97		full_command = [validated_command] + validated_args
98		logger.info(f"Executing validated command: {' '.join(full_command)}")
99
100		# Get secure environment variables
101		secure_env = get_secure_env()
102
103		# Execute the command
104		result = subprocess.run(
105		full_command,
106		capture_output=True,
107		check=False,
108		text=True,
109		timeout=timeout,
110		env=secure_env
111		)
112
113		# Return the result
114		return {
115		"stdout": result.stdout,
116		"stderr": result.stderr,
117		"returncode": result.returncode,
118		"command": ' '.join(full_command) # Include the executed command for reference
119		}
120		except CommandValidationError as e:
121		logger.error(f"Command validation error: {str(e)}")
122		return {
123		"error": f"Command validation error: {str(e)}",
124		"command": f"{command} {' '.join(args) if args else ''}"
125		}
126		except subprocess.TimeoutExpired:
127		logger.error(f"Command timed out after {timeout} seconds: {command} {' '.join(args)}")
128		return {
129		"error": f"Command timed out after {timeout} seconds",
130		"command": f"{command} {' '.join(args) if args else ''}"
131		}
132		except Exception as e:
133		logger.error(f"Error executing command: {str(e)}")
134		return {
135		"error": str(e),
136		"command": f"{command} {' '.join(args) if args else ''}"
137		}
138
139		def read_file(
140		path: str,
141		max_size: Optional[int] = None,
142		permission_manager = None,
143		sandbox_path: Optional[str] = None,
144		additional_allowed_paths: Optional[Set[str]] = None
145		) -> str:
146		"""
147		Read file contents.
148
149		Args:
150		path: File path
151		max_size: Maximum file size to read in bytes
152		permission_manager: Permission manager instance
153		sandbox_path: Base path that defines the sandbox
154		additional_allowed_paths: Additional paths to allow
155
156		Returns:
157		str: File contents
158		"""
159		# Check permission if permission manager is provided
160		if permission_manager:
161		function_args = {"path": path, "max_size": max_size}
162		permission_granted = permission_manager.request_permission(
163		function_name="read_file",
164		args=function_args
165		)
166
167		if not permission_granted:
168		logger.warning(f"Permission denied for reading file: {path}")
169		return "Permission denied"
170
171		try:
172		# Validate the path for reading
173		validated_path = validate_path_for_reading(
174		path,
175		sandbox_path=sandbox_path,
176		additional_allowed_paths=additional_allowed_paths
177		)
178
179		# Check the file size if max_size is specified
180		if max_size is not None:
181		file_size = os.path.getsize(validated_path)
182		if file_size > max_size:
183		logger.error(f"File size ({file_size} bytes) exceeds maximum size ({max_size} bytes): {validated_path}")
184		return f"File size ({file_size} bytes) exceeds maximum size ({max_size} bytes)"
185
186		# Read the file
187		with open(validated_path, "r") as f:
188		content = f.read()
189
190		return content
191		except PathValidationError as e:
192		logger.error(f"Path validation error: {str(e)}")
193		return f"Path validation error: {str(e)}"
194		except Exception as e:
195		logger.error(f"Error reading file: {str(e)}")
196		return f"Error reading file: {str(e)}"
197
198		def write_file(
199		path: str,
200		content: str,
201		mode: str = 'w',
202		permission_manager = None,
203		sandbox_path: Optional[str] = None,
204		additional_allowed_paths: Optional[Set[str]] = None,
205		additional_protected_paths: Optional[Set[str]] = None
206		) -> bool:
207		"""
208		Write content to a file.
209
210		Args:
211		path: File path
212		content: Content to write
213		mode: Write mode ('w' for write, 'a' for append)
214		permission_manager: Permission manager instance
215		sandbox_path: Base path that defines the sandbox
216		additional_allowed_paths: Additional paths to allow
217		additional_protected_paths: Additional paths to protect
218
219		Returns:
220		bool: True if successful
221		"""
222		# Validate the mode
223		if mode not in ['w', 'a']:
224		logger.error(f"Invalid mode: {mode}")
225		return False
226
227		# Check permission if permission manager is provided
228		if permission_manager:
229		function_args = {"path": path, "content": content, "mode": mode}
230		permission_granted = permission_manager.request_permission(
231		function_name="write_file",
232		args=function_args
233		)
234
235		if not permission_granted:
236		logger.warning(f"Permission denied for writing to file: {path}")
237		return False
238
239		try:
240		# Validate the path for writing
241		validated_path = validate_path_for_writing(
242		path,
243		sandbox_path=sandbox_path,
244		additional_allowed_paths=additional_allowed_paths,
245		additional_protected_paths=additional_protected_paths
246		)
247
248		# Create the directory if it doesn't exist
249		os.makedirs(os.path.dirname(validated_path), exist_ok=True)
250
251		# Write to the file
252		with open(validated_path, mode) as f:
253		f.write(content)
254
255		return True
256		except PathValidationError as e:
257		logger.error(f"Path validation error: {str(e)}")
258		return False
259		except Exception as e:
260		logger.error(f"Error writing to file: {str(e)}")
261		return False
262
263		def find_files(
264		pattern: str,
265		directory: str = '.',
266		recursive: bool = True,
267		permission_manager = None,
268		sandbox_path: Optional[str] = None,
269		additional_allowed_paths: Optional[Set[str]] = None
270		) -> List[str]:
271		"""
272		Find files matching a pattern.
273
274		Args:
275		pattern: File pattern
276		directory: Search directory
277		recursive: Whether to search recursively
278		permission_manager: Permission manager instance
279		sandbox_path: Base path that defines the sandbox
280		additional_allowed_paths: Additional paths to allow
281
282		Returns:
283		List[str]: Matching file paths
284		"""
285		# Check permission if permission manager is provided
286		if permission_manager:
287		function_args = {"pattern": pattern, "directory": directory, "recursive": recursive}
288		permission_granted = permission_manager.request_permission(
289		function_name="find_files",
290		args=function_args
291		)
292
293		if not permission_granted:
294		logger.warning(f"Permission denied for finding files in: {directory}")
295		return []
296
297		try:
298		# Validate the directory and pattern for searching
299		validated_dir, validated_pattern = validate_path_for_searching(
300		directory,
301		pattern,
302		sandbox_path=sandbox_path,
303		additional_allowed_paths=additional_allowed_paths
304		)
305
306		# Construct the search pattern
307		search_pattern = os.path.join(validated_dir, '**', validated_pattern) if recursive else os.path.join(validated_dir, validated_pattern)
308
309		# Find files
310		files = glob.glob(search_pattern, recursive=recursive)
311
312		# Additional validation of found files (check if they're still in the sandbox)
313		validated_files = []
314		for file_path in files:
315		try:
316		# Only include files that are within the sandbox
317		abs_path = os.path.abspath(file_path)
318		if os.path.exists(abs_path) and (
319		is_path_in_sandbox(
320		abs_path,
321		sandbox_path=sandbox_path,
322		additional_allowed_paths=additional_allowed_paths
323		)
324		):
325		validated_files.append(abs_path)
326		except Exception as e:
327		logger.warning(f"Skipping file {file_path}: {str(e)}")
328
329		return validated_files
330		except PathValidationError as e:
331		logger.error(f"Path validation error: {str(e)}")
332		return []
333		except Exception as e:
334		logger.error(f"Error finding files: {str(e)}")
335		return []

ollamatools-new/ollamatools/plugins/tools/web.py +0 -484

…	…	@@ -1,484 +0,0 @@
1		"""
2		Web tools module for OllamaTools.
3
4		This module provides functions for making web requests and processing responses,
5		including weather information fetching and web search functionality.
6		"""
7
8		import requests
9		from typing import Dict, Any, Optional, Union, List, Set
10		import logging
11		import json
12		from urllib.parse import urlparse, quote
13		from bs4 import BeautifulSoup
14
15		# Import the URL validation module
16		from ollamatools.core.utils.validators.url import (
17		validate_url,
18		URLValidationError,
19		configure_domain_lists,
20		create_url_validator
21		)
22
23		# Get logger
24		logger = logging.getLogger(__name__)
25
26		def fetch_url(
27		url: str,
28		method: str = 'GET',
29		params: Optional[Dict[str, str]] = None,
30		headers: Optional[Dict[str, str]] = None,
31		data: Optional[Dict[str, Any]] = None,
32		timeout: int = 30,
33		permission_manager = None,
34		allowed_schemes: Optional[Set[str]] = None,
35		domain_allowlist: Optional[Set[str]] = None,
36		domain_blocklist: Optional[Set[str]] = None
37		) -> Dict[str, Any]:
38		"""
39		Fetch a URL with specified parameters.
40
41		Args:
42		url: URL to fetch
43		method: HTTP method (GET, POST, PUT, DELETE, etc.)
44		params: URL parameters
45		headers: HTTP headers
46		data: Request data
47		timeout: Request timeout in seconds
48		permission_manager: Optional permission manager to request permission
49		allowed_schemes: Optional set of allowed URL schemes
50		domain_allowlist: Optional set of allowed domains
51		domain_blocklist: Optional set of blocked domains
52
53		Returns:
54		dict: Response with status, headers, and content
55
56		Raises:
57		ValueError: If URL or method is invalid
58		requests.RequestException: If the request fails
59		PermissionError: If permission is denied
60		"""
61		# Request permission if permission manager is provided
62		if permission_manager:
63		args = {
64		"url": url,
65		"method": method,
66		"params": params,
67		"headers": headers,
68		"data": data
69		}
70
71		if not permission_manager.request_permission("fetch_url", args):
72		logger.warning(f"Permission denied for web request to {url}")
73		raise PermissionError(f"Permission denied for web request to {url}")
74
75		# Validate and normalize the URL
76		try:
77		validated_url = validate_url(
78		url,
79		allowed_schemes=allowed_schemes,
80		domain_allowlist=domain_allowlist,
81		domain_blocklist=domain_blocklist
82		)
83		logger.debug(f"URL validated and normalized: {validated_url}")
84		except URLValidationError as e:
85		logger.error(f"URL validation error: {str(e)}")
86		raise ValueError(f"URL validation failed: {str(e)}")
87
88		# Validate method
89		valid_methods = ['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'HEAD', 'OPTIONS']
90		method = method.upper()
91		if method not in valid_methods:
92		raise ValueError(f"Invalid HTTP method: {method}. Must be one of {valid_methods}")
93
94		# Set default headers if none provided
95		if headers is None:
96		headers = {}
97
98		# Add default User-Agent if not present
99		if 'User-Agent' not in headers:
100		headers['User-Agent'] = 'OllamaTools/0.1.0 (https://github.com/yourusername/ollamatools)'
101
102		logger.info(f"Making {method} request to {validated_url}")
103
104		try:
105		# Create a session to manage cookies and connection pooling
106		with requests.Session() as session:
107		# Configure session with reasonable security defaults
108		session.verify = True # Verify SSL certificates
109
110		# Make the request
111		response = session.request(
112		method=method,
113		url=validated_url,
114		params=params,
115		headers=headers,
116		json=data if method in ['POST', 'PUT', 'PATCH'] else None,
117		timeout=timeout
118		)
119
120		# Try to parse response as JSON, fall back to text
121		content_type = response.headers.get('Content-Type', '')
122		if 'application/json' in content_type:
123		try:
124		content = response.json()
125		except ValueError:
126		content = response.text
127		else:
128		content = response.text
129
130		result = {
131		"status": response.status_code,
132		"headers": dict(response.headers),
133		"content": content,
134		"url": response.url
135		}
136
137		logger.debug(f"Request to {validated_url} completed with status {response.status_code}")
138		return result
139
140		except requests.RequestException as e:
141		logger.error(f"Request to {validated_url} failed: {str(e)}")
142		raise
143
144
145		def get_weather(
146		location: str,
147		days: int = 3,
148		units: str = 'metric',
149		api_key: Optional[str] = None,
150		permission_manager = None,
151		domain_allowlist: Optional[Set[str]] = None,
152		domain_blocklist: Optional[Set[str]] = None
153		) -> Dict[str, Any]:
154		"""
155		Get weather information for a location.
156
157		Args:
158		location: Location name or coordinates
159		days: Number of forecast days (1-7)
160		units: Temperature units ('metric' or 'imperial')
161		api_key: OpenWeatherMap API key (if None, uses config)
162		permission_manager: Optional permission manager to request permission
163		domain_allowlist: Optional set of allowed domains
164		domain_blocklist: Optional set of blocked domains
165
166		Returns:
167		dict: Weather information including current conditions and forecast
168
169		Raises:
170		ValueError: If parameters are invalid
171		requests.RequestException: If the API request fails
172		PermissionError: If permission is denied
173		"""
174		# Validate parameters
175		if not location:
176		raise ValueError("Location cannot be empty")
177
178		if days < 1 or days > 7:
179		raise ValueError("Days must be between 1 and 7")
180
181		valid_units = ['metric', 'imperial']
182		if units not in valid_units:
183		raise ValueError(f"Units must be one of {valid_units}")
184
185		# Request permission if permission manager is provided
186		if permission_manager:
187		args = {
188		"location": location,
189		"days": days,
190		"units": units
191		}
192
193		if not permission_manager.request_permission("get_weather", args):
194		logger.warning(f"Permission denied for weather request for {location}")
195		raise PermissionError(f"Permission denied for weather request for {location}")
196
197		# Check if API key is provided
198		if api_key is None:
199		# In a real implementation, this would get the API key from the config
200		# For now, we'll use a placeholder message
201		logger.error("No API key provided for weather API")
202		raise ValueError("Weather API key is required. Please provide an API key or set it in the configuration.")
203
204		logger.info(f"Fetching weather for {location}")
205
206		try:
207		# Ensure allowlist includes the weather API domain
208		weather_domain_allowlist = {"api.openweathermap.org"}
209		if domain_allowlist:
210		weather_domain_allowlist.update(domain_allowlist)
211
212		# Base URL for OpenWeatherMap API
213		base_url = "https://api.openweathermap.org/data/2.5"
214
215		# Get current weather
216		current_url = f"{base_url}/weather"
217		params = {
218		"q": location,
219		"appid": api_key,
220		"units": units
221		}
222
223		current_response = fetch_url(
224		current_url,
225		params=params,
226		timeout=10,
227		permission_manager=permission_manager,
228		domain_allowlist=weather_domain_allowlist,
229		domain_blocklist=domain_blocklist
230		)
231		current_data = current_response["content"]
232
233		# Get forecast
234		forecast_url = f"{base_url}/forecast"
235		params = {
236		"q": location,
237		"appid": api_key,
238		"units": units,
239		"cnt": days * 8 # 8 forecasts per day (every 3 hours)
240		}
241
242		forecast_response = fetch_url(
243		forecast_url,
244		params=params,
245		timeout=10,
246		permission_manager=permission_manager,
247		domain_allowlist=weather_domain_allowlist,
248		domain_blocklist=domain_blocklist
249		)
250		forecast_data = forecast_response["content"]
251
252		# Format the response
253		weather_info = {
254		"location": {
255		"name": current_data.get("name", ""),
256		"country": current_data.get("sys", {}).get("country", ""),
257		"coordinates": current_data.get("coord", {})
258		},
259		"current": {
260		"temperature": current_data.get("main", {}).get("temp"),
261		"feels_like": current_data.get("main", {}).get("feels_like"),
262		"humidity": current_data.get("main", {}).get("humidity"),
263		"pressure": current_data.get("main", {}).get("pressure"),
264		"wind_speed": current_data.get("wind", {}).get("speed"),
265		"wind_direction": current_data.get("wind", {}).get("deg"),
266		"description": current_data.get("weather", [{}])[0].get("description", ""),
267		"icon": current_data.get("weather", [{}])[0].get("icon", "")
268		},
269		"forecast": []
270		}
271
272		# Process forecast data
273		for item in forecast_data.get("list", [])[:days * 8]:
274		forecast_item = {
275		"timestamp": item.get("dt"),
276		"temperature": item.get("main", {}).get("temp"),
277		"feels_like": item.get("main", {}).get("feels_like"),
278		"humidity": item.get("main", {}).get("humidity"),
279		"pressure": item.get("main", {}).get("pressure"),
280		"wind_speed": item.get("wind", {}).get("speed"),
281		"wind_direction": item.get("wind", {}).get("deg"),
282		"description": item.get("weather", [{}])[0].get("description", ""),
283		"icon": item.get("weather", [{}])[0].get("icon", "")
284		}
285		weather_info["forecast"].append(forecast_item)
286
287		return weather_info
288
289		except Exception as e:
290		logger.error(f"Weather API request failed: {str(e)}")
291		raise
292
293
294		def search_web(
295		query: str,
296		num_results: int = 5,
297		api_key: Optional[str] = None,
298		permission_manager = None,
299		domain_allowlist: Optional[Set[str]] = None,
300		domain_blocklist: Optional[Set[str]] = None
301		) -> List[Dict[str, str]]:
302		"""
303		Search the web for a query.
304
305		Args:
306		query: Search query
307		num_results: Number of results to return (1-10)
308		api_key: Search API key (if None, uses config)
309		permission_manager: Optional permission manager to request permission
310		domain_allowlist: Optional set of allowed domains
311		domain_blocklist: Optional set of blocked domains
312
313		Returns:
314		List[dict]: Search results with title, URL, and snippet
315
316		Raises:
317		ValueError: If parameters are invalid
318		requests.RequestException: If the API request fails
319		PermissionError: If permission is denied
320		"""
321		# Validate parameters
322		if not query:
323		raise ValueError("Query cannot be empty")
324
325		if num_results < 1 or num_results > 10:
326		raise ValueError("Number of results must be between 1 and 10")
327
328		# Request permission if permission manager is provided
329		if permission_manager:
330		args = {
331		"query": query,
332		"num_results": num_results
333		}
334
335		if not permission_manager.request_permission("search_web", args):
336		logger.warning(f"Permission denied for web search: {query}")
337		raise PermissionError(f"Permission denied for web search: {query}")
338
339		# Check if API key is provided
340		if api_key is None:
341		# In a real implementation, this would get the API key from the config
342		# For now, we'll use a fallback to a simple search method that doesn't require API key
343		logger.warning("No API key provided for search API, using fallback method")
344		return _fallback_search(
345		query,
346		num_results,
347		permission_manager=permission_manager,
348		domain_allowlist=domain_allowlist,
349		domain_blocklist=domain_blocklist
350		)
351
352		logger.info(f"Searching the web for: {query}")
353
354		try:
355		# Ensure allowlist includes the search API domain
356		search_domain_allowlist = {"serpapi.com"}
357		if domain_allowlist:
358		search_domain_allowlist.update(domain_allowlist)
359
360		# Example using the Serpapi (Google Search API)
361		# In a real implementation, you would use a proper search API
362		url = "https://serpapi.com/search"
363		params = {
364		"q": query,
365		"api_key": api_key,
366		"num": num_results
367		}
368
369		response = fetch_url(
370		url,
371		params=params,
372		timeout=15,
373		permission_manager=permission_manager,
374		domain_allowlist=search_domain_allowlist,
375		domain_blocklist=domain_blocklist
376		)
377
378		data = response["content"]
379		results = []
380
381		# Process search results
382		for item in data.get("organic_results", [])[:num_results]:
383		result = {
384		"title": item.get("title", ""),
385		"url": item.get("link", ""),
386		"snippet": item.get("snippet", "")
387		}
388		results.append(result)
389
390		return results
391
392		except Exception as e:
393		logger.error(f"Search API request failed: {str(e)}")
394
395		# Fall back to the simple search method if API request fails
396		logger.info("Falling back to simple search method")
397		return _fallback_search(
398		query,
399		num_results,
400		permission_manager=permission_manager,
401		domain_allowlist=domain_allowlist,
402		domain_blocklist=domain_blocklist
403		)
404
405
406		def _fallback_search(
407		query: str,
408		num_results: int = 5,
409		permission_manager = None,
410		domain_allowlist: Optional[Set[str]] = None,
411		domain_blocklist: Optional[Set[str]] = None
412		) -> List[Dict[str, str]]:
413		"""
414		Fallback search method using DuckDuckGo HTML scraping when API is not available.
415
416		Note: This is a fallback method and not recommended for production use.
417		It may break if the search engine changes its HTML structure.
418
419		Args:
420		query: Search query
421		num_results: Number of results to return
422		permission_manager: Optional permission manager to request permission
423		domain_allowlist: Optional set of allowed domains
424		domain_blocklist: Optional set of blocked domains
425
426		Returns:
427		List[dict]: Search results with title, URL, and snippet
428		"""
429		results = []
430		encoded_query = quote(query)
431		url = f"https://html.duckduckgo.com/html/?q={encoded_query}"
432
433		try:
434		# Ensure allowlist includes the search domain
435		search_domain_allowlist = {"html.duckduckgo.com"}
436		if domain_allowlist:
437		search_domain_allowlist.update(domain_allowlist)
438
439		headers = {
440		"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
441		}
442
443		response = fetch_url(
444		url,
445		headers=headers,
446		timeout=15,
447		permission_manager=permission_manager,
448		domain_allowlist=search_domain_allowlist,
449		domain_blocklist=domain_blocklist
450		)
451
452		soup = BeautifulSoup(response["content"], 'html.parser')
453
454		# Find search results
455		result_elements = soup.select('.result')
456
457		for element in result_elements[:num_results]:
458		title_element = element.select_one('.result__title')
459		link_element = element.select_one('.result__url')
460		snippet_element = element.select_one('.result__snippet')
461
462		if title_element and link_element:
463		title = title_element.get_text(strip=True)
464		# The actual URL is in the href attribute of the <a> tag
465		link = title_element.find('a').get('href', '')
466		snippet = snippet_element.get_text(strip=True) if snippet_element else ""
467
468		# Validate URL
469		try:
470		validate_url(link)
471		results.append({
472		"title": title,
473		"url": link,
474		"snippet": snippet
475		})
476		except URLValidationError:
477		logger.warning(f"Skipping result with invalid URL: {link}")
478
479		return results
480
481		except Exception as e:
482		logger.error(f"Fallback search failed: {str(e)}")
483		# Return empty results if even the fallback fails
484		return []

ollamatools-new/ollamatools/version.py +0 -3

…	…	@@ -1,3 +0,0 @@
1		"""Version information for OllamaTools."""
2
3		__version__ = "0.1.0"

ollamatools-new/setup.py +1 -0

…	…	@@ -54,6 +54,7 @@ setup(
54	54	"matrix-nio>=0.19.0",
55	55	"pyyaml>=5.1",
56	56	"ollama>=0.1.0",
	57	"ollamatools>=0.1.0", # Add ollamatools as a dependency
57	58	],
58	59	entry_points={
59	60	"console_scripts": [

ollama_semantic_function_calling commits / 02bf3a1

Changed files (51) +235 -8822