ollama_semantic_function_calling commits / 6d99bd3

Changed files (162) +0 -33053

ollamatools-project/.github/workflows/python_tests.yml +0 -44

…	…	@@ -1,44 +0,0 @@
1		name: Python Tests
2
3		on:
4		push:
5		branches: [ main ]
6		pull_request:
7		branches: [ main ]
8
9		jobs:
10		test:
11		runs-on: ubuntu-latest
12		strategy:
13		matrix:
14		python-version: [3.8, 3.9, '3.10', '3.11']
15
16		steps:
17		- uses: actions/checkout@v3
18
19		- name: Set up Python ${{ matrix.python-version }}
20		uses: actions/setup-python@v4
21		with:
22		python-version: ${{ matrix.python-version }}
23
24		- name: Install dependencies
25		run: |
26		python -m pip install --upgrade pip
27		pip install pytest pytest-cov flake8
28		pip install -e ".[test]"
29
30		- name: Lint with flake8
31		run: |
32		# stop the build if there are Python syntax errors or undefined names
33		flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
34		# exit-zero treats all errors as warnings
35		flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
36
37		- name: Test with pytest
38		run: |
39		pytest --cov=ollamatools tests/
40
41		- name: Upload coverage report
42		uses: codecov/codecov-action@v3
43		with:
44		fail_ci_if_error: false

ollamatools-project/.gitignore +0 -4

…	…	@@ -1,4 +0,0 @@
1		# Ignore the contents of the submodule directory
2		/ollama-matrix-client/*
3		!/ollama-matrix-client/.git
4

ollamatools-project/AI_SIMPLIFICATION_GUIDE.md +0 -1073

…	…	@@ -1,1073 +0,0 @@
1		# AI-Only OllamaTools Simplification Guide
2
3		This document provides detailed, step-by-step instructions for AI assistants to implement the OllamaTools simplification plan. Follow these instructions precisely to transform the codebase according to the architecture outlined in `SIMPLIFICATION_PLAN.md`.
4
5		## Initial Setup and Analysis
6
7		1. **Clone the repository and checkout the simplify branch**
8		```bash
9		git clone [repository-url]
10		cd ollamatools
11		git checkout simplify
12		```
13
14		2. **Analyze the current codebase structure**
15		- Read through the key files to understand the current architecture:
16		- `ollamatools/__init__.py`
17		- `ollamatools/agent.py`
18		- `ollamatools/agent/modular/agent.py`
19		- `ollamatools/agent/extended/agent.py`
20		- `ollamatools/tools/matrix_tools.py`
21		- `ollamatools/plugins/registry.py`
22		- Identify dependencies between components
23		- Note any circular dependencies that need to be resolved
24
25		3. **Review the simplification plan**
26		- Read `SIMPLIFICATION_PLAN.md` to understand the target architecture
27		- Review `SIMPLIFICATION_PROMPT_TEMPLATE.md` for guidance on implementation
28		- Refer to `SIMPLIFY_README.md` for an overview of the project
29
30		## Phase 1: Core Infrastructure Implementation
31
32		### Step 1: Create Directory Structure
33
34		Create the following directory structure:
35
36		```
37		ollamatools/
38		├── core/
39		│ ├── __init__.py
40		│ ├── agent.py
41		│ ├── config.py
42		│ ├── errors.py
43		│ ├── security.py
44		│ └── utils/
45		│ ├── __init__.py
46		│ ├── logger.py
47		│ ├── validators.py
48		│ └── formatting.py
49		```
50
51		### Step 2: Implement Core Module Initialization
52
53		Create `ollamatools/core/__init__.py`:
54
55		```python
56		"""
57		Core module for OllamaTools.
58
59		This module provides the essential functionality that all other components depend on.
60		It's designed to be minimal, stable, and rarely change.
61		"""
62
63		__version__ = "1.0.0"
64
65		from .agent import Agent
66		from .config import load_config, save_config
67		from .errors import OllamaToolsError, ConfigError, SecurityError
68
69		__all__ = [
70		"Agent",
71		"load_config",
72		"save_config",
73		"OllamaToolsError",
74		"ConfigError",
75		"SecurityError"
76		]
77		```
78
79		### Step 3: Implement Core Error Handling
80
81		Create `ollamatools/core/errors.py`:
82
83		```python
84		"""
85		Error definitions and handling for OllamaTools.
86
87		This module defines the error types used throughout the OllamaTools project
88		and provides utilities for error handling and formatting.
89		"""
90
91		class OllamaToolsError(Exception):
92		"""Base exception for all OllamaTools errors."""
93
94		def __init__(self, message, details=None):
95		"""Initialize the error with a message and optional details."""
96		self.message = message
97		self.details = details or {}
98		super().__init__(message)
99
100
101		class ConfigError(OllamaToolsError):
102		"""Error raised when there's a problem with configuration."""
103		pass
104
105
106		class SecurityError(OllamaToolsError):
107		"""Error raised when there's a security violation."""
108		pass
109
110
111		class ValidationError(OllamaToolsError):
112		"""Error raised when validation fails."""
113		pass
114
115
116		def format_error(error):
117		"""Format an error for display to the user."""
118		if isinstance(error, OllamaToolsError):
119		result = f"Error: {error.message}"
120		if error.details:
121		result += "\nDetails:"
122		for key, value in error.details.items():
123		result += f"\n {key}: {value}"
124		return result
125		return f"Error: {str(error)}"
126
127
128		def handle_error(error, logger=None):
129		"""Handle an error, logging it if a logger is provided."""
130		formatted = format_error(error)
131		if logger:
132		logger.error(formatted)
133		return formatted
134		```
135
136		### Step 4: Implement Core Security
137
138		Create `ollamatools/core/security.py`:
139
140		```python
141		"""
142		Security utilities and permission system for OllamaTools.
143
144		This module provides security-related functionality, including permission
145		checking, validation, and secure defaults.
146		"""
147
148		import os
149		import re
150		from .errors import SecurityError, ValidationError
151
152		# Safe command patterns
153		SAFE_COMMAND_PATTERN = re.compile(r'^[a-zA-Z0-9_\-./\s]+$')
154
155		# Safe path patterns
156		SAFE_PATH_PATTERN = re.compile(r'^[a-zA-Z0-9_\-./\s]+$')
157
158		# Safe URL patterns
159		SAFE_URL_PATTERN = re.compile(r'^https?://[a-zA-Z0-9_\-./]+')
160
161
162		def validate_command(command):
163		"""Validate that a command is safe to execute."""
164		if not command or not isinstance(command, str):
165		raise ValidationError("Command must be a non-empty string")
166
167		if not SAFE_COMMAND_PATTERN.match(command):
168		raise SecurityError(
169		"Command contains potentially unsafe characters",
170		{"command": command}
171		)
172
173		# Check for dangerous commands
174		dangerous_commands = ["rm -rf", "sudo", "chmod", "chown"]
175		for dangerous in dangerous_commands:
176		if dangerous in command:
177		raise SecurityError(
178		f"Command contains potentially dangerous operation: {dangerous}",
179		{"command": command}
180		)
181
182		return True
183
184
185		def validate_path(path):
186		"""Validate that a path is safe to access."""
187		if not path or not isinstance(path, str):
188		raise ValidationError("Path must be a non-empty string")
189
190		if not SAFE_PATH_PATTERN.match(path):
191		raise SecurityError(
192		"Path contains potentially unsafe characters",
193		{"path": path}
194		)
195
196		# Resolve to absolute path
197		abs_path = os.path.abspath(path)
198
199		# Check for path traversal
200		if ".." in path:
201		raise SecurityError(
202		"Path contains directory traversal",
203		{"path": path, "resolved": abs_path}
204		)
205
206		return True
207
208
209		def validate_url(url):
210		"""Validate that a URL is safe to access."""
211		if not url or not isinstance(url, str):
212		raise ValidationError("URL must be a non-empty string")
213
214		if not SAFE_URL_PATTERN.match(url):
215		raise SecurityError(
216		"URL contains potentially unsafe characters or protocol",
217		{"url": url}
218		)
219
220		return True
221
222
223		class PermissionManager:
224		"""Manage permissions for function execution."""
225
226		def __init__(self, config=None):
227		"""Initialize the permission manager with optional configuration."""
228		self.config = config or {}
229		self.permissions = self.config.get("permissions", {})
230		self.default_permission = self.config.get("default_permission", False)
231
232		def check_permission(self, function_name, user_id=None):
233		"""Check if a function is permitted to be executed."""
234		# Check function-specific permission
235		if function_name in self.permissions:
236		return self.permissions[function_name]
237
238		# Check user-specific permission
239		if user_id and user_id in self.permissions:
240		user_perms = self.permissions[user_id]
241		if function_name in user_perms:
242		return user_perms[function_name]
243
244		# Fall back to default permission
245		return self.default_permission
246
247		def grant_permission(self, function_name, user_id=None):
248		"""Grant permission for a function."""
249		if user_id:
250		if user_id not in self.permissions:
251		self.permissions[user_id] = {}
252		self.permissions[user_id][function_name] = True
253		else:
254		self.permissions[function_name] = True
255
256		def revoke_permission(self, function_name, user_id=None):
257		"""Revoke permission for a function."""
258		if user_id:
259		if user_id in self.permissions and function_name in self.permissions[user_id]:
260		self.permissions[user_id][function_name] = False
261		else:
262		self.permissions[function_name] = False
263		```
264
265		### Step 5: Implement Core Utils
266
267		Create `ollamatools/core/utils/__init__.py`:
268
269		```python
270		"""
271		Utility functions for OllamaTools.
272
273		This module provides common utility functions used throughout the OllamaTools project.
274		"""
275
276		from .logger import setup_logger, get_logger
277		from .validators import validate_command, validate_path, validate_url
278		from .formatting import format_text, format_markdown
279
280		__all__ = [
281		"setup_logger",
282		"get_logger",
283		"validate_command",
284		"validate_path",
285		"validate_url",
286		"format_text",
287		"format_markdown"
288		]
289		```
290
291		Create `ollamatools/core/utils/logger.py`:
292
293		```python
294		"""
295		Logging utilities for OllamaTools.
296
297		This module provides logging configuration and utilities.
298		"""
299
300		import logging
301		import os
302		import sys
303		from datetime import datetime
304
305		# Default log format
306		DEFAULT_LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
307
308		# Default log level
309		DEFAULT_LOG_LEVEL = logging.INFO
310
311		# Global logger dictionary
312		_loggers = {}
313
314
315		def setup_logger(name, level=None, log_format=None, log_file=None):
316		"""Set up a logger with the specified configuration."""
317		if name in _loggers:
318		return _loggers[name]
319
320		# Create logger
321		logger = logging.getLogger(name)
322
323		# Set level
324		logger.setLevel(level or DEFAULT_LOG_LEVEL)
325
326		# Create formatter
327		formatter = logging.Formatter(log_format or DEFAULT_LOG_FORMAT)
328
329		# Create console handler
330		console_handler = logging.StreamHandler(sys.stdout)
331		console_handler.setFormatter(formatter)
332		logger.addHandler(console_handler)
333
334		# Create file handler if log_file is specified
335		if log_file:
336		os.makedirs(os.path.dirname(log_file), exist_ok=True)
337		file_handler = logging.FileHandler(log_file)
338		file_handler.setFormatter(formatter)
339		logger.addHandler(file_handler)
340
341		# Store logger
342		_loggers[name] = logger
343
344		return logger
345
346
347		def get_logger(name):
348		"""Get a logger by name, creating it if it doesn't exist."""
349		if name not in _loggers:
350		return setup_logger(name)
351		return _loggers[name]
352		```
353
354		Create `ollamatools/core/utils/validators.py`:
355
356		```python
357		"""
358		Input validation utilities for OllamaTools.
359
360		This module provides validation functions for commands, paths, and URLs.
361		"""
362
363		import os
364		import re
365		from ..errors import ValidationError, SecurityError
366
367		# Safe command patterns
368		SAFE_COMMAND_PATTERN = re.compile(r'^[a-zA-Z0-9_\-./\s]+$')
369
370		# Safe path patterns
371		SAFE_PATH_PATTERN = re.compile(r'^[a-zA-Z0-9_\-./\s]+$')
372
373		# Safe URL patterns
374		SAFE_URL_PATTERN = re.compile(r'^https?://[a-zA-Z0-9_\-./]+')
375
376
377		def validate_command(command):
378		"""Validate that a command is safe to execute."""
379		if not command or not isinstance(command, str):
380		raise ValidationError("Command must be a non-empty string")
381
382		if not SAFE_COMMAND_PATTERN.match(command):
383		raise SecurityError(
384		"Command contains potentially unsafe characters",
385		{"command": command}
386		)
387
388		# Check for dangerous commands
389		dangerous_commands = ["rm -rf", "sudo", "chmod", "chown"]
390		for dangerous in dangerous_commands:
391		if dangerous in command:
392		raise SecurityError(
393		f"Command contains potentially dangerous operation: {dangerous}",
394		{"command": command}
395		)
396
397		return True
398
399
400		def validate_path(path):
401		"""Validate that a path is safe to access."""
402		if not path or not isinstance(path, str):
403		raise ValidationError("Path must be a non-empty string")
404
405		if not SAFE_PATH_PATTERN.match(path):
406		raise SecurityError(
407		"Path contains potentially unsafe characters",
408		{"path": path}
409		)
410
411		# Resolve to absolute path
412		abs_path = os.path.abspath(path)
413
414		# Check for path traversal
415		if ".." in path:
416		raise SecurityError(
417		"Path contains directory traversal",
418		{"path": path, "resolved": abs_path}
419		)
420
421		return True
422
423
424		def validate_url(url):
425		"""Validate that a URL is safe to access."""
426		if not url or not isinstance(url, str):
427		raise ValidationError("URL must be a non-empty string")
428
429		if not SAFE_URL_PATTERN.match(url):
430		raise SecurityError(
431		"URL contains potentially unsafe characters or protocol",
432		{"url": url}
433		)
434
435		return True
436		```
437
438		Create `ollamatools/core/utils/formatting.py`:
439
440		```python
441		"""
442		Text formatting utilities for OllamaTools.
443
444		This module provides functions for formatting text and generating markdown.
445		"""
446
447		def format_text(text, width=80, indent=0):
448		"""Format text to the specified width with optional indentation."""
449		if not text:
450		return ""
451
452		# Split text into words
453		words = text.split()
454
455		# Initialize result
456		result = []
457		current_line = " " * indent
458
459		# Process words
460		for word in words:
461		# Check if adding the word would exceed the width
462		if len(current_line) + len(word) + 1 > width and len(current_line.strip()) > 0:
463		# Add current line to result and start a new line
464		result.append(current_line)
465		current_line = " " * indent + word
466		else:
467		# Add word to current line
468		if len(current_line.strip()) > 0:
469		current_line += " " + word
470		else:
471		current_line += word
472
473		# Add the last line
474		if current_line.strip():
475		result.append(current_line)
476
477		return "\n".join(result)
478
479
480		def format_markdown(text, heading_level=1, code_block=False, language=None):
481		"""Format text as markdown with optional heading and code block."""
482		if not text:
483		return ""
484
485		result = []
486
487		# Add heading if heading_level is specified
488		if heading_level > 0:
489		result.append("#" * heading_level + " " + text.split("\n")[0])
490		text = "\n".join(text.split("\n")[1:])
491
492		# Add code block if specified
493		if code_block:
494		if language:
495		result.append(f"```{language}")
496		else:
497		result.append("```")
498		result.append(text)
499		result.append("```")
500		else:
501		result.append(text)
502
503		return "\n".join(result)
504		```
505
506		### Step 6: Implement Core Configuration
507
508		Create `ollamatools/core/config.py`:
509
510		```python
511		"""
512		Configuration management for OllamaTools.
513
514		This module provides functions for loading, validating, and saving configuration.
515		"""
516
517		import os
518		import yaml
519		from .errors import ConfigError
520
521		# Default configuration
522		DEFAULT_CONFIG = {
523		"agent": {
524		"model": "llama3:latest",
525		"prompt": "You are a helpful assistant.",
526		"temperature": 0.7,
527		"max_tokens": 1000
528		},
529		"security": {
530		"permission_prompts": True,
531		"default_permission": False
532		},
533		"logging": {
534		"level": "INFO",
535		"file": None
536		},
537		"plugins": {
538		"enabled": []
539		}
540		}
541
542
543		def load_config(config_path=None):
544		"""Load configuration from a file."""
545		# Start with default configuration
546		config = DEFAULT_CONFIG.copy()
547
548		# If config_path is specified, load configuration from file
549		if config_path:
550		try:
551		with open(config_path, "r") as f:
552		file_config = yaml.safe_load(f)
553
554		# Merge file configuration with default configuration
555		if file_config:
556		_merge_config(config, file_config)
557		except Exception as e:
558		raise ConfigError(f"Failed to load configuration from {config_path}", {"error": str(e)})
559
560		return config
561
562
563		def save_config(config, config_path):
564		"""Save configuration to a file."""
565		try:
566		# Create directory if it doesn't exist
567		os.makedirs(os.path.dirname(config_path), exist_ok=True)
568
569		# Save configuration to file
570		with open(config_path, "w") as f:
571		yaml.dump(config, f, default_flow_style=False)
572		except Exception as e:
573		raise ConfigError(f"Failed to save configuration to {config_path}", {"error": str(e)})
574
575
576		def validate_config(config, schema=None):
577		"""Validate configuration against a schema."""
578		# If schema is not specified, use a basic schema
579		if not schema:
580		schema = {
581		"agent": {
582		"required": True,
583		"type": dict
584		},
585		"security": {
586		"required": True,
587		"type": dict
588		},
589		"logging": {
590		"required": True,
591		"type": dict
592		},
593		"plugins": {
594		"required": True,
595		"type": dict
596		}
597		}
598
599		# Validate configuration against schema
600		for key, value in schema.items():
601		if value.get("required", False) and key not in config:
602		raise ConfigError(f"Missing required configuration key: {key}")
603
604		if key in config and value.get("type") and not isinstance(config[key], value["type"]):
605		raise ConfigError(f"Invalid type for configuration key: {key}")
606
607		return True
608
609
610		def _merge_config(base_config, override_config):
611		"""Merge override_config into base_config."""
612		for key, value in override_config.items():
613		if key in base_config and isinstance(base_config[key], dict) and isinstance(value, dict):
614		_merge_config(base_config[key], value)
615		else:
616		base_config[key] = value
617		```
618
619		### Step 7: Implement Base Agent
620
621		Create `ollamatools/core/agent.py`:
622
623		```python
624		"""
625		Base agent implementation for OllamaTools.
626
627		This module provides the base Agent class that serves as the foundation
628		for all agent implementations.
629		"""
630
631		import os
632		import json
633		from .errors import OllamaToolsError
634		from .config import load_config
635		from .security import PermissionManager
636
637		class Agent:
638		"""Base agent class for OllamaTools."""
639
640		def __init__(self, model=None, prompt=None, config=None, config_path=None):
641		"""Initialize the agent with the specified configuration."""
642		# Load configuration
643		self.config = config or (load_config(config_path) if config_path else {})
644
645		# Set model and prompt
646		self.model = model or self.config.get("agent", {}).get("model", "llama3:latest")
647		self.prompt = prompt or self.config.get("agent", {}).get("prompt", "You are a helpful assistant.")
648
649		# Initialize permission manager
650		self.permission_manager = PermissionManager(self.config.get("security", {}))
651
652		# Initialize plugin registry
653		self.plugin_registry = {}
654		self.enabled_plugins = set()
655
656		def register_plugin(self, plugin_name, plugin_instance):
657		"""Register a plugin with the agent."""
658		self.plugin_registry[plugin_name] = plugin_instance
659
660		def enable_plugin(self, plugin_name):
661		"""Enable a plugin."""
662		if plugin_name not in self.plugin_registry:
663		raise OllamaToolsError(f"Plugin not found: {plugin_name}")
664
665		self.enabled_plugins.add(plugin_name)
666
667		def disable_plugin(self, plugin_name):
668		"""Disable a plugin."""
669		if plugin_name in self.enabled_plugins:
670		self.enabled_plugins.remove(plugin_name)
671
672		def get_plugin(self, plugin_name):
673		"""Get a plugin by name."""
674		if plugin_name not in self.plugin_registry:
675		raise OllamaToolsError(f"Plugin not found: {plugin_name}")
676
677		return self.plugin_registry[plugin_name]
678
679		def chat(self, message):
680		"""Process a chat message and return a response."""
681		# This is a placeholder implementation that should be overridden by subclasses
682		return f"Base agent received: {message}"
683
684		def check_permission(self, function_name, user_id=None):
685		"""Check if a function is permitted to be executed."""
686		return self.permission_manager.check_permission(function_name, user_id)
687
688		def grant_permission(self, function_name, user_id=None):
689		"""Grant permission for a function."""
690		self.permission_manager.grant_permission(function_name, user_id)
691
692		def revoke_permission(self, function_name, user_id=None):
693		"""Revoke permission for a function."""
694		self.permission_manager.revoke_permission(function_name, user_id)
695
696		def startup(self):
697		"""Perform startup operations."""
698		# This is a placeholder implementation that should be overridden by subclasses
699		pass
700
701		def shutdown(self):
702		"""Perform shutdown operations."""
703		# This is a placeholder implementation that should be overridden by subclasses
704		pass
705		```
706
707		## Phase 2: Plugin System Implementation
708
709		### Step 1: Create Plugin System Directory Structure
710
711		Create the following directory structure:
712
713		```
714		ollamatools/plugins/
715		├── __init__.py
716		├── base.py
717		├── registry.py
718		├── errors.py
719		```
720
721		### Step 2: Implement Plugin System Initialization
722
723		Create `ollamatools/plugins/__init__.py`:
724
725		```python
726		"""
727		Plugin system for OllamaTools.
728
729		This module provides the infrastructure for extending OllamaTools functionality
730		through plugins.
731		"""
732
733		from .base import Plugin, ToolPlugin, IntegrationPlugin, AgentPlugin
734		from .registry import PluginRegistry
735		from .errors import PluginError, PluginNotFoundError, PluginInitializationError
736
737		__all__ = [
738		"Plugin",
739		"ToolPlugin",
740		"IntegrationPlugin",
741		"AgentPlugin",
742		"PluginRegistry",
743		"PluginError",
744		"PluginNotFoundError",
745		"PluginInitializationError"
746		]
747		```
748
749		### Step 3: Implement Plugin Base Classes
750
751		Create `ollamatools/plugins/base.py`:
752
753		```python
754		"""
755		Base plugin interfaces for OllamaTools.
756
757		This module defines the base plugin interfaces that all plugins must implement.
758		"""
759
760		class Plugin:
761		"""Base interface for all OllamaTools plugins."""
762
763		def __init__(self, config=None):
764		"""Initialize the plugin with optional configuration."""
765		self.config = config or {}
766
767		def initialize(self, context):
768		"""Initialize the plugin with the runtime context."""
769		pass
770
771		def shutdown(self):
772		"""Clean up resources when shutting down."""
773		pass
774
775		@property
776		def name(self):
777		"""Get the plugin name."""
778		return self.__class__.__name__
779
780		@property
781		def version(self):
782		"""Get the plugin version."""
783		return "1.0.0"
784
785		@property
786		def dependencies(self):
787		"""Get plugin dependencies."""
788		return []
789
790		def get_capabilities(self):
791		"""Get capabilities provided by this plugin."""
792		return []
793
794
795		class ToolPlugin(Plugin):
796		"""Base class for tool plugins."""
797
798		def execute(self, tool_name, args, context):
799		"""Execute a tool with the specified arguments."""
800		raise NotImplementedError("Tool plugins must implement execute")
801
802		def get_tools(self):
803		"""Get tools provided by this plugin."""
804		return {}
805
806		def get_tool_schema(self, tool_name):
807		"""Get the schema for a tool."""
808		tools = self.get_tools()
809		if tool_name not in tools:
810		return None
811		return tools[tool_name].get("schema", {})
812
813
814		class IntegrationPlugin(Plugin):
815		"""Base class for integration plugins."""
816
817		def connect(self):
818		"""Connect to the external system."""
819		raise NotImplementedError("Integration plugins must implement connect")
820
821		def disconnect(self):
822		"""Disconnect from the external system."""
823		raise NotImplementedError("Integration plugins must implement disconnect")
824
825		def is_connected(self):
826		"""Check if the plugin is connected to the external system."""
827		raise NotImplementedError("Integration plugins must implement is_connected")
828
829		def get_status(self):
830		"""Get the status of the integration."""
831		return {
832		"connected": self.is_connected()
833		}
834
835
836		class AgentPlugin(Plugin):
837		"""Base class for agent plugins."""
838
839		def process_message(self, message, context):
840		"""Process a message and return a response."""
841		return None
842
843		def get_commands(self):
844		"""Get commands provided by this plugin."""
845		return {}
846
847		def on_function_call(self, function_name, args, context):
848		"""Handle a function call."""
849		return None
850		```
851
852		### Step 4: Implement Plugin Registry
853
854		Create `ollamatools/plugins/registry.py`:
855
856		```python
857		"""
858		Plugin registry for OllamaTools.
859
860		This module provides the PluginRegistry class for managing plugins.
861		"""
862
863		import importlib
864		import pkg_resources
865		from .errors import PluginNotFoundError, PluginInitializationError
866
867		class PluginRegistry:
868		"""Registry for OllamaTools plugins."""
869
870		def __init__(self):
871		"""Initialize the plugin registry."""
872		self.plugins = {}
873		self.initialized_plugins = {}
874
875		def register_plugin(self, plugin_class, config=None):
876		"""Register a plugin class."""
877		plugin_name = plugin_class.__name__
878		self.plugins[plugin_name] = {
879		"class": plugin_class,
880		"config": config
881		}
882		return plugin_name
883
884		def discover_plugins(self):
885		"""Discover plugins using entry points."""
886		try:
887		for entry_point in pkg_resources.iter_entry_points('ollamatools.plugins'):
888		plugin_class = entry_point.load()
889		self.register_plugin(plugin_class)
890		except Exception as e:
891		# Log the error but continue
892		print(f"Error discovering plugins: {e}")
893
894		def initialize_plugin(self, plugin_name, context):
895		"""Initialize a plugin and its dependencies."""
896		if plugin_name in self.initialized_plugins:
897		return self.initialized_plugins[plugin_name]
898
899		if plugin_name not in self.plugins:
900		raise PluginNotFoundError(f"Plugin {plugin_name} not found")
901
902		plugin_info = self.plugins[plugin_name]
903		plugin_class = plugin_info["class"]
904		plugin_config = plugin_info["config"]
905
906		try:
907		# Initialize dependencies first
908		plugin_instance = plugin_class(plugin_config)
909		for dependency in plugin_instance.dependencies:
910		self.initialize_plugin(dependency, context)
911
912		# Initialize the plugin
913		plugin_instance.initialize(context)
914		self.initialized_plugins[plugin_name] = plugin_instance
915
916		return plugin_instance
917		except Exception as e:
918		raise PluginInitializationError(
919		f"Failed to initialize plugin {plugin_name}",
920		{"error": str(e)}
921		)
922
923		def get_plugin(self, plugin_name):
924		"""Get an initialized plugin."""
925		if plugin_name not in self.initialized_plugins:
926		raise PluginNotFoundError(f"Plugin {plugin_name} not initialized")
927		return self.initialized_plugins[plugin_name]
928
929		def get_plugins_by_type(self, plugin_type):
930		"""Get all initialized plugins of a specific type."""
931		return {
932		name: plugin for name, plugin in self.initialized_plugins.items()
933		if isinstance(plugin, plugin_type)
934		}
935
936		def shutdown(self):
937		"""Shutdown all initialized plugins."""
938		for plugin_name, plugin in self.initialized_plugins.items():
939		try:
940		plugin.shutdown()
941		except Exception as e:
942		# Log the error but continue
943		print(f"Error shutting down plugin {plugin_name}: {e}")
944
945		self.initialized_plugins = {}
946		```
947
948		### Step 5: Implement Plugin Errors
949
950		Create `ollamatools/plugins/errors.py`:
951
952		```python
953		"""
954		Error definitions for the OllamaTools plugin system.
955
956		This module defines the error types used by the plugin system.
957		"""
958
959		from ..core.errors import OllamaToolsError
960
961		class PluginError(OllamaToolsError):
962		"""Base exception for all plugin-related errors."""
963		pass
964
965
966		class PluginNotFoundError(PluginError):
967		"""Error raised when a plugin is not found."""
968		pass
969
970
971		class PluginInitializationError(PluginError):
972		"""Error raised when a plugin fails to initialize."""
973		pass
974
975
976		class PluginDependencyError(PluginError):
977		"""Error raised when a plugin dependency cannot be satisfied."""
978		pass
979
980
981		def handle_plugin_exception(exception):
982		"""Handle a plugin exception and return error details."""
983		if isinstance(exception, PluginError):
984		return {
985		"error_type": exception.__class__.__name__,
986		"error_message": exception.message,
987		"error_details": exception.details
988		}
989		return {
990		"error_type": "UnknownError",
991		"error_message": str(exception),
992		"error_details": {}
993		}
994		```
995
996		## Phase 3: Standard Plugins Implementation
997
998		### Step 1: Create Standard Plugins Directory Structure
999
1000		Create the following directory structure:
1001
1002		```
1003		ollamatools/plugins/standard/
1004		├── __init__.py
1005		├── search.py
1006		├── commands.py
1007		├── conversation/
1008		│ ├── __init__.py
1009		│ ├── history.py
1010		│ ├── context.py
1011		│ └── analysis.py
1012		├── feedback.py
1013		├── issues.py
1014		└── function_calling.py
1015		```
1016
1017		### Step 2: Implement Standard Plugins Initialization
1018
1019		Create `ollamatools/plugins/standard/__init__.py`:
1020
1021		```python
1022		"""
1023		Standard plugins for OllamaTools.
1024
1025		This module provides standard plugins that implement common functionality
1026		used by different agents.
1027		"""
1028
1029		from .search import SearchPlugin
1030		from .commands import CommandsPlugin
1031		from .conversation.history import ConversationHistoryPlugin
1032		from .conversation.context import ConversationContextPlugin
1033		from .conversation.analysis import ConversationAnalysisPlugin
1034		from .feedback import FeedbackPlugin
1035		from .issues import IssuesPlugin
1036		from .function_calling import FunctionCallingPlugin
1037
1038		__all__ = [
1039		"SearchPlugin",
1040		"CommandsPlugin",
1041		"ConversationHistoryPlugin",
1042		"ConversationContextPlugin",
1043		"ConversationAnalysisPlugin",
1044		"FeedbackPlugin",
1045		"IssuesPlugin",
1046		"FunctionCallingPlugin"
1047		]
1048		```
1049
1050		### Step 3: Implement Search Plugin
1051
1052		Create `ollamatools/plugins/standard/search.py`:
1053
1054		```python
1055		"""
1056		Search plugin for OllamaTools.
1057
1058		This plugin provides search functionality for agents.
1059		"""
1060
1061		import json
1062		import os
1063		from ..base import Plugin
1064
1065		class SearchPlugin(Plugin):
1066		"""Plugin for search functionality."""
1067
1068		def __init__(self, config=None):
1069		super().__init__(config)
1070		self.opensearch_config = config.get("opensearch", {})
1071		self.available = False
1072
1073		def initialize(self, context):

ollamatools-project/AI_SIMPLIFICATION_README.md +0 -54

…	…	@@ -1,54 +0,0 @@
1		# AI-Only OllamaTools Simplification Guide
2
3		This document provides detailed, step-by-step instructions specifically formatted for AI assistants to implement the OllamaTools simplification plan.
4
5		## Purpose
6
7		The `AI_SIMPLIFICATION_GUIDE.md` file is designed to be a comprehensive, code-focused guide that an AI can follow to implement the architectural changes outlined in the `SIMPLIFICATION_PLAN.md` document. It contains:
8
9		1. Detailed implementation steps
10		2. Complete code examples for each component
11		3. Step-by-step instructions for creating the new directory structure
12		4. Specific guidance on how to extract and refactor functionality from the existing codebase
13
14		## Current Status
15
16		**⚠️ WORK IN PROGRESS ⚠️**
17
18		The AI simplification guide is currently incomplete. It covers:
19
20		- Initial setup and analysis
21		- Phase 1: Core Infrastructure Implementation (complete)
22		- Phase 2: Plugin System Implementation (complete)
23		- Phase 3: Standard Plugins Implementation (partial)
24
25		The guide will be expanded to include:
26		- Completion of Phase 3: Standard Plugins Implementation
27		- Phase 4: Tool Plugins Implementation
28		- Phase 5: Integration Plugins Implementation
29		- Phase 6: Agent Implementations
30		- Phase 7: Configuration System Implementation
31		- Phase 8: Command-Line Interface Implementation
32		- Migration guidance for existing code
33
34		## How to Use This Guide
35
36		This guide is specifically formatted for AI assistants to follow. It provides explicit instructions and complete code examples that can be directly implemented. Human developers may find it overly detailed, but it ensures that AI assistants have all the information they need to implement the simplification plan correctly.
37
38		When working with an AI assistant on this project:
39
40		1. Point the AI to the `AI_SIMPLIFICATION_GUIDE.md` file for implementation details
41		2. Provide the AI with access to the existing codebase
42		3. Ask the AI to implement specific phases or components from the guide
43		4. Review the AI's implementation and provide feedback
44
45		## Relationship to Other Documents
46
47		- `SIMPLIFICATION_PLAN.md`: Provides the high-level architecture and strategy
48		- `SIMPLIFICATION_PROMPT_TEMPLATE.md`: Provides templates for creating prompts for specific tasks
49		- `SIMPLIFY_README.md`: Provides an overview of the simplification project
50		- `AI_SIMPLIFICATION_GUIDE.md`: Provides detailed implementation instructions for AI assistants
51
52		## Contributing
53
54		To contribute to this guide, please add detailed implementation instructions for the remaining phases and components. Follow the same format as the existing sections, providing complete code examples and step-by-step instructions.

ollamatools-project/CHANGELOG.md +0 -162

…	…	@@ -1,162 +0,0 @@
1		# Changelog
2
3		All notable changes to the OllamaTools project will be documented in this file.
4
5		The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6		and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
8		## [1.4.0] - 2025-05-15
9
10		### Added
11		- Modular structure for enhanced Matrix client:
12		- Split functionality into multiple modules for better maintainability:
13		- `enhanced_matrix_core.py`: Core client functionality
14		- `enhanced_matrix_search.py`: Search-related functionality
15		- `enhanced_matrix_history.py`: Conversation history and analytics
16		- `enhanced_matrix_tools.py`: Main module that integrates everything
17		- Added new synchronous wrapper functions for enhanced client:
18		- `join_enhanced_matrix_room`
19		- `send_enhanced_matrix_message`
20		- `listen_for_enhanced_matrix_messages`
21		- `disconnect_enhanced_matrix`
22		- `process_with_enhanced_ollama`
23		- Improved error handling and resource cleanup
24		- Better handling of optional dependencies with graceful degradation
25		- Enhanced documentation with modular structure details
26		- Updated examples to use the new modular structure
27
28		## [1.3.0] - 2025-05-10
29
30		### Added
31		- Enhanced Matrix client with modular agent functionality:
32		- New `enhanced_matrix_tools.py` module with `EnhancedMatrixClient` class
33		- Improved message response logic with better mention detection
34		- Natural language understanding from modular agent
35		- Search integration with OpenSearchNG
36		- Conversation history tracking and summarization
37		- Topic analysis for conversation insights
38		- Performance metrics tracking
39		- Additional commands: `/search`, `/history`, `/topics`, `/metrics`
40		- Option to respond to all messages (not just mentions)
41		- Enhanced configuration with `create_enhanced_matrix_config`
42		- Updated example in `examples/enhanced_matrix_client_example.py`
43		- Updated configuration template in `examples/config/enhanced_matrix_config.yaml`
44		- Expanded documentation in `docs/matrix_integration.md`
45
46		## [1.2.0] - 2025-05-01
47
48		### Added
49		- Enhanced Matrix chat integration:
50		- Model selection support with `/model` command
51		- User context tracking for coherent conversations
52		- Admin user management for privileged operations
53		- Command system with `/help` and `/models` commands
54		- Random greeting messages when joining rooms
55		- Automatic model discovery from Ollama server
56		- Enhanced example in `examples/enhanced_matrix_client_example.py`
57		- Updated configuration template in `examples/config/enhanced_matrix_config.yaml`
58		- Expanded documentation in `docs/matrix_integration.md`
59
60		## [1.1.0] - 2025-04-25
61
62		### Added
63		- Matrix chat integration:
64		- New `matrix_tools.py` module for Matrix protocol support
65		- `MatrixClient` class for connecting to Matrix homeservers
66		- Functions for joining rooms, sending messages, and listening for events
67		- Integration with Ollama for processing Matrix messages
68		- Configuration management for Matrix settings
69		- End-to-end encryption support via matrix-nio[e2e]
70		- Example script in `examples/matrix_client_example.py`
71		- Default configuration template in `examples/config/matrix_config.yaml`
72		- Comprehensive documentation in `docs/matrix_integration.md`
73		- Dependencies:
74		- Added matrix-nio for Matrix protocol support
75		- Added pyyaml for configuration handling
76		- Made ollama a direct dependency instead of optional
77
78		### Changed
79		- Updated `tools/__init__.py` to include Matrix tools
80		- Updated requirements.txt with new dependencies
81		- Added Matrix integration as an optional extra in setup.py
82
83		## [1.0.0] - 2025-04-20
84
85		### Breaking Changes
86		- Removed the TransparentOllamaAgent implementation completely
87		- Removed the entire ollamatools/agent/transparent directory
88		- Updated examples to use ModularOllamaAgent instead
89		- Updated tests to use ModularOllamaAgent instead
90		- Updated documentation to reflect the removal of TransparentOllamaAgent
91
92		### Changed
93		- ModularOllamaAgent is now the primary agent implementation
94		- Simplified codebase by focusing on a single agent implementation
95		- Updated migration guide to help users transition to ModularOllamaAgent
96
97		## [0.2.0] - 2025-04-15
98
99		### Added
100		- Comprehensive security enhancements:
101		- Enhanced command validation with command allowlists
102		- Path validation with sandbox restrictions
103		- URL validation with domain allowlists/blocklists
104		- Standardized error handling system
105		- Improved permission management with session permissions
106		- New modules:
107		- `command_validator.py` for robust command sanitization
108		- `path_validator.py` for secure file path handling
109		- `url_validator.py` for safe web request validation
110		- `errors.py` for standardized error definitions
111		- New documentation:
112		- `SECURITY.md` with detailed security guidelines
113		- Security examples in `examples/security_example.py`
114		- Features:
115		- Path sandboxing for file operations
116		- URL normalization to prevent validation bypass
117		- Secure environment variables for command execution
118		- Domain blocklists to prevent SSRF attacks
119		- Symbolic link resolution for file paths
120		- Protection for system directories
121		- Session-based permission system
122
123		### Changed
124		- Updated `execute_command` to use new command validation
125		- Enhanced `read_file`, `write_file`, and `find_files` with path validation
126		- Improved `fetch_url` with URL validation
127		- Updated `PermissionManager` with better sudo detection
128		- Modified permission system to support session permissions
129		- Upgraded error reporting with standardized error classes
130		- Improved logging for security events
131
132		### Security
133		- Fixed command injection vulnerabilities in `execute_command`
134		- Fixed path traversal vulnerabilities in file operations
135		- Fixed SSRF vulnerabilities in `fetch_url`
136		- Added protected path enforcement for system directories
137		- Improved sudo detection for privilege escalation protection
138		- Implemented comprehensive allowlist system for commands
139		- Enhanced permission prompt UI with more options
140
141		## [0.1.0] - 2025-04-10
142
143		### Added
144		- Initial release of OllamaTools
145		- Core OllamaAgent implementation
146		- Security system with permission management
147		- Configuration management
148		- System tools for command execution and file operations
149		- Web tools for HTTP requests, weather data, and web search
150		- Documentation tools for markdown generation and text processing
151		- Logging utilities
152		- Comprehensive test suite
153		- Example scripts for all components
154		- Performance optimization features
155		- Complete documentation
156
157		### Security
158		- Implemented prompt-based permission system
159		- Special handling for sudo commands
160		- Command sanitization to prevent injection attacks
161		- Resource usage limitations
162		- Timeouts for long-running operations

ollamatools-project/COMPLETION_REPORT.md +0 -154

…	…	@@ -1,154 +0,0 @@
1		# Transparent Ollama Agent - Issue Resolution Report
2
3		## Summary
4
5		This report documents the resolution of several issues in the Transparent Ollama Agent. We have successfully addressed three key issues:
6
7		1. **Natural Language Understanding for Search** - Implemented proper function call detection and execution
8		2. **OpenSearchNG Integration Error Handling** - Fixed type errors and improved error handling
9		3. **Inconsistent Function Execution** - Enhanced JSON parsing and function execution logic
10
11		## Issues Resolved
12
13		### 1. Natural Language Understanding for Search
14
15		**Problem**: The agent required explicit search commands rather than understanding natural language requests. Users had to type "search [query]" instead of casual questions like "can you give me a recipe for chilli sauce?".
16
17		**Resolution**:
18		- Enhanced the `chat` method in `TransparentOllamaAgent` to properly detect function calls in LLM responses
19		- Added JSON parsing for responses that appear to be function calls
20		- Implemented direct execution of search functions when detected
21		- Improved result formatting to create user-friendly responses
22
23		**Testing**: Created and executed test scripts that verify the agent properly handles natural language search requests and executes function calls correctly.
24
25		### 2. OpenSearchNG Integration Error Handling
26
27		**Problem**: The agent failed with a type error when OpenSearchNG was not available. When OpenSearchNG was not running, the error "sequence item 0: expected str instance, dict found" occurred, indicating improper error handling.
28
29		**Resolution**:
30		- Enhanced error handling in the OpenSearchNG integration code to properly handle dictionary values in search results
31		- Added type checking and proper conversion for answers, suggestions, and infoboxes
32		- Improved fallback mechanism when OpenSearchNG is not available
33		- Added better error reporting and logging
34
35		**Testing**: Created and executed test scripts that verify the error is properly handled and the agent gracefully falls back to alternative search methods when OpenSearchNG is not available.
36
37		### 3. Inconsistent Function Execution
38
39		**Problem**: The agent sometimes returned function calls as raw JSON instead of executing them. When the LLM returned a function call in JSON format, the agent was not properly parsing and executing it.
40
41		**Resolution**:
42		- Added code to parse JSON function calls and execute them appropriately
43		- Implemented proper error handling for malformed JSON
44		- Enhanced the function execution logic to handle different JSON formats
45
46		**Testing**: Created and executed test scripts that verify the agent correctly detects, parses, and executes function calls, and handles malformed JSON gracefully.
47
48		## Implementation Details
49
50		### Changes to `ollamatools/agent/transparent/agent.py`
51
52		The `chat` method was enhanced to properly detect and execute function calls:
53
54		```python
55		# Check if the response looks like a JSON function call
56		if response.strip().startswith('{') and ('function' in response or 'name' in response):
57		self.print_info("Detected potential function call in response")
58		try:
59		# Parse the JSON response
60		func_data = json.loads(response)
61
62		# Extract function name and args
63		function_name = None
64		args = {}
65
66		# Handle different JSON formats
67		if 'function' in func_data:
68		function_name = func_data['function'].get('name')
69		args_str = func_data['function'].get('arguments', '{}')
70		if isinstance(args_str, str):
71		args = json.loads(args_str)
72		else:
73		args = args_str
74		elif 'name' in func_data:
75		function_name = func_data.get('name')
76		args = func_data.get('arguments', {})
77
78		if function_name:
79		self.print_status(f"Executing function: {function_name} with args: {args}")
80
81		# Execute the function
82		if function_name == 'opensearch_query' or function_name == 'search_web':
83		results = self.execute_search(args.get('query', message))
84		context = self.extract_search_context(results)
85		response = f"Here's what I found about '{args.get('query', message)}':\n\n{context}"
86		# ... other function handling ...
87		```
88
89		### Changes to `ollamatools/opensearch_web_tools.py`
90
91		The `process_opensearch_results` function was updated to handle dictionary values in search results:
92
93		```python
94		# Extract suggestions
95		if "suggestions" in result_data and result_data["suggestions"]:
96		try:
97		suggestion = result_data["suggestions"][0]
98		# Handle case where suggestion is a dict instead of a string
99		if isinstance(suggestion, dict):
100		processed_results["suggestion"] = suggestion.get("value", str(suggestion))
101		else:
102		processed_results["suggestion"] = suggestion
103		except (IndexError, TypeError) as e:
104		logger.warning(f"Error processing suggestions: {str(e)}")
105		processed_results["suggestion"] = None
106
107		# Extract answer if available
108		if "answers" in result_data and result_data["answers"]:
109		try:
110		answer = result_data["answers"][0]
111		# Handle case where answer is a dict instead of a string
112		if isinstance(answer, dict):
113		processed_results["answer"] = answer.get("content", str(answer))
114		else:
115		processed_results["answer"] = answer
116		except (IndexError, TypeError) as e:
117		logger.warning(f"Error processing answers: {str(e)}")
118		processed_results["answer"] = None
119		```
120
121		### Changes to `ollamatools/agent/transparent/search.py`
122
123		The `extract_search_context` function was updated to handle dictionary answers:
124
125		```python
126		# Add direct answer if available
127		if search_results.get("answer"):
128		try:
129		# Handle both string and dict answers
130		answer = search_results["answer"]
131		if isinstance(answer, dict):
132		answer_text = answer.get("content", str(answer))
133		else:
134		answer_text = str(answer)
135		context_parts.append(f"Direct answer: {answer_text}")
136		except Exception as e:
137		logger.warning(f"Error extracting answer for context: {str(e)}")
138		```
139
140		## Test Scripts Created
141
142		1. **test_function_execution.py** - Tests the function execution logic, verifying that the agent correctly detects, parses, and executes function calls.
143
144		2. **test_dict_handling.py** - Tests the handling of dictionary values in search results, verifying that the agent correctly extracts information from dictionary answers and suggestions.
145
146		## Remaining Issues
147
148		One issue remains to be addressed:
149
150		**Code Organization and Modularity** - The transparent agent code is in a single large file with multiple responsibilities. The proposed solution is to refactor the code into a modular structure with separate files for different components.
151
152		## Conclusion
153
154		The Transparent Ollama Agent has been significantly improved with better natural language understanding, more robust error handling, and consistent function execution. These changes make the agent more user-friendly and reliable, especially when dealing with search queries and external services like OpenSearchNG.

ollamatools-project/COMPLETION_REPORT_ENHANCED_MATRIX.md +0 -193

…	…	@@ -1,193 +0,0 @@
1		# Enhanced Matrix Client with Modular Agent Functionality - Completion Report
2
3		## Overview
4
5		This report documents the implementation of the Enhanced Matrix Client with Modular Agent functionality in OllamaTools. The enhanced client extends the basic Matrix client with advanced features from the modular and extended agents, addressing issues with message response logic and adding powerful new capabilities.
6
7		## Latest Update: Modular Structure Implementation
8
9		The Enhanced Matrix Client has been refactored into a modular structure to improve maintainability, extensibility, and to address file size and context limitations. The functionality has been split into multiple modules:
10
11		1. **enhanced_matrix_core.py**: Contains the core `EnhancedMatrixClient` class and basic functionality
12		2. **enhanced_matrix_search.py**: Contains search-related functionality
13		3. **enhanced_matrix_history.py**: Contains conversation history tracking and analytics
14		4. **enhanced_matrix_tools.py**: Main module that imports and re-exports everything
15
16		This modular structure provides several benefits:
17		- Better code organization and maintainability
18		- Easier to extend specific functionality without affecting other parts
19		- Reduced file sizes to avoid context limitations
20		- Clearer separation of concerns
21		- Improved error handling and resource cleanup
22		- Better handling of optional dependencies with graceful degradation
23
24		Additional synchronous wrapper functions have been added to provide a complete API:
25		- `join_enhanced_matrix_room`
26		- `send_enhanced_matrix_message`
27		- `listen_for_enhanced_matrix_messages`
28		- `disconnect_enhanced_matrix`
29		- `process_with_enhanced_ollama`
30
31		The documentation has been updated to reflect the new modular structure, and the example code has been updated to use the new API.
32
33		## Completed Enhancements
34
35		### 1. Improved Message Response Logic
36
37		- Enhanced username detection with case-insensitive matching
38		- Added direct mention detection for full Matrix IDs (e.g., @username:server.com)
39		- Implemented option to respond to all messages (not just mentions)
40		- Fixed issues with message response inconsistency
41
42		### 2. Natural Language Understanding
43
44		- Integrated factual intent detection from modular agent
45		- Added command intent detection for better understanding of user requests
46		- Implemented semantic function selection feedback system
47
48		### 3. Search Integration
49
50		- Added web search capability using OpenSearchNG
51		- Implemented `/search` command for direct search queries
52		- Added automatic search for messages with factual intent
53		- Integrated search context into message processing
54
55		### 4. Conversation History Tracking
56
57		- Implemented conversation history storage and retrieval
58		- Added `/history` command to view recent conversations
59		- Created conversation summarization functionality
60		- Implemented persistent storage of conversation history
61
62		### 5. Topic Analysis
63
64		- Added conversation topic analysis based on keywords
65		- Implemented `/topics` command to view conversation topics
66		- Created topic frequency tracking for insights
67
68		### 6. Performance Metrics
69
70		- Added response time tracking for performance monitoring
71		- Implemented token usage estimation
72		- Added `/metrics` command to view performance metrics
73		- Created tracking for slowest and fastest responses
74
75		### 7. Enhanced Configuration
76
77		- Created `create_enhanced_matrix_config` function for default configuration
78		- Added enhanced configuration options in YAML format
79		- Implemented configuration for all new features
80		- Updated example configuration file
81
82		### 8. Documentation and Examples
83
84		- Updated `docs/matrix_integration.md` with comprehensive documentation
85		- Enhanced `examples/enhanced_matrix_client_example.py` to use new features
86		- Updated `examples/config/enhanced_matrix_config.yaml` with new options
87		- Added entries to `CHANGELOG.md` for the new features
88
89		## Implementation Details
90
91		### EnhancedMatrixClient Class
92
93		The core of the implementation is the `EnhancedMatrixClient` class in `ollamatools/tools/enhanced_matrix_tools.py`. This class extends the base `MatrixClient` class with the following enhancements:
94
95		1. **Improved Message Callback**:
96		- Enhanced message detection logic
97		- Integration with search functionality
98		- Performance tracking
99		- Conversation history recording
100
101		2. **Enhanced Command Handling**:
102		- Added new commands: `/search`, `/history`, `/topics`, `/metrics`
103		- Improved help system with conditional command listing
104		- Better command parsing with regex patterns
105
106		3. **Search Integration**:
107		- Implemented `_execute_search` method for web search
108		- Added search context augmentation for messages
109		- Created search result formatting
110
111		4. **Conversation Tracking**:
112		- Implemented `_record_conversation` method for history tracking
113		- Added conversation loading and saving to persistent storage
114		- Created conversation analysis methods
115
116		### Module API
117
118		The module now provides a comprehensive API with the following functions:
119
120		1. **Connection Functions**:
121		- `connect_enhanced_matrix`: Connect to a Matrix homeserver with an enhanced client
122		- `disconnect_enhanced_matrix`: Properly disconnect and clean up resources
123
124		2. **Room Interaction Functions**:
125		- `join_enhanced_matrix_room`: Join a Matrix room with an enhanced client
126		- `send_enhanced_matrix_message`: Send a message to a Matrix room
127		- `listen_for_enhanced_matrix_messages`: Start listening for messages in joined rooms
128
129		3. **Message Processing Functions**:
130		- `process_with_enhanced_ollama`: Process a message with Ollama using an enhanced client
131
132		4. **Configuration Functions**:
133		- `create_enhanced_matrix_config`: Create a default configuration for the enhanced client
134
135		5. **Status Flags**:
136		- `MODULAR_AGENT_AVAILABLE`: Flag indicating if modular agent components are available
137		- `OPENSEARCH_AVAILABLE`: Flag indicating if OpenSearchNG is available
138
139		Each module also provides specialized functions:
140
141		- **enhanced_matrix_search.py**:
142		- `execute_search_query`: Execute a search query using OpenSearchNG
143		- `execute_search_for_message`: Execute a search for a message and return the search context
144		- `handle_search_command`: Handle the /search command
145
146		- **enhanced_matrix_history.py**:
147		- `load_conversation_history`: Load conversation history from file
148		- `save_conversation_history`: Save conversation history to file
149		- `record_conversation`: Record a conversation in the history
150		- `analyze_conversation_topics`: Analyze common topics in conversation history
151		- `get_performance_metrics`: Get performance metrics from conversation history
152
153		## Future Work
154
155		While the current implementation provides a solid foundation for the enhanced Matrix client, there are several areas for future improvement:
156
157		1. **Streaming Responses**:
158		- Implement streaming for longer responses from Ollama
159		- Add progress indicators during model processing
160
161		2. **Enhanced Admin Controls**:
162		- Add admin-only commands for system management
163		- Implement room-specific model restrictions
164		- Add user management commands
165
166		3. **Performance Optimizations**:
167		- Optimize context storage for large deployments
168		- Implement caching for frequently used models
169		- Add batch processing for high-traffic rooms
170
171		4. **Security Enhancements**:
172		- Add rate limiting to prevent abuse
173		- Implement more granular permission controls
174		- Add encryption for stored contexts
175
176		5. **Integration with Other Agents**:
177		- Add integration with other agent types
178		- Implement agent selection based on message content
179		- Create hybrid agent capabilities
180
181		## Conclusion
182
183		The Enhanced Matrix Client with Modular Agent functionality significantly improves the user experience by addressing issues with message response logic and adding powerful new capabilities from the modular and extended agents. The implementation follows best practices for code organization, error handling, and documentation, providing a solid foundation for future improvements.
184
185		The recent refactoring into a modular structure has further enhanced the codebase by:
186		1. Improving maintainability through clear separation of concerns
187		2. Enhancing extensibility by making it easier to modify specific components
188		3. Solving file size and context limitations by splitting functionality into multiple files
189		4. Providing better error handling and resource cleanup
190		5. Offering a more comprehensive API with additional synchronous wrapper functions
191		6. Handling optional dependencies more gracefully with clear fallback mechanisms
192
193		The enhanced client is now ready for use in production environments and provides a more robust and feature-rich experience for Matrix users. The modular structure ensures that the codebase can continue to evolve and grow without running into the limitations that were previously encountered.

ollamatools-project/COMPLETION_REPORT_MATRIX.md +0 -133

…	…	@@ -1,133 +0,0 @@
1		# Matrix Integration Enhancement Completion Report
2
3		## Overview
4
5		This report documents the enhancements made to the Matrix integration in OllamaTools. The enhancements focus on adding model selection capabilities, user context tracking, and admin user management to the Matrix client.
6
7		## Completed Enhancements
8
9		### 1. Model Selection Support
10
11		- Added command-based model switching with `/model <model_name>` command
12		- Implemented model listing with `/models` command
13		- Added help system with `/help` command
14		- Created model validation to ensure requested models are available
15		- Added automatic model discovery from Ollama server
16		- Implemented configuration-based model specification
17
18		### 2. User Context Tracking
19
20		- Added conversation history tracking for each user in each room
21		- Implemented context management with configurable context length
22		- Created efficient context storage using user and room identifiers
23		- Added automatic context pruning to maintain performance
24		- Ensured context is properly maintained across model switches
25
26		### 3. Admin User Management
27
28		- Added admin user designation in configuration
29		- Implemented admin user management methods
30		- Created foundation for future admin-only commands
31
32		### 4. Random Greeting Feature
33
34		- Added random greeting functionality when joining rooms
35		- Implemented 10 different greeting messages
36		- Ensured greetings are sent only on successful room joins
37
38		### 5. Documentation and Examples
39
40		- Updated `docs/matrix_integration.md` with comprehensive documentation of new features
41		- Created enhanced example in `examples/enhanced_matrix_client_example.py`
42		- Added sample configuration in `examples/config/enhanced_matrix_config.yaml`
43		- Updated CHANGELOG.md to document the enhancements
44
45		## Implementation Details
46
47		### Client Example Enhancements
48
49		The Matrix client examples were enhanced with:
50
51		1. **Random Greeting System**:
52		- Added `get_random_greeting()` function to generate varied greetings
53		- Modified room joining process to send greetings after successful joins
54		- Implemented logging of sent greetings for debugging
55
56		### MatrixClient Class Enhancements
57
58		The `MatrixClient` class in `ollamatools/tools/matrix_tools.py` was enhanced with:
59
60		1. **Command System**:
61		- Added regex patterns for command matching
62		- Implemented command handling in `_handle_command` method
63		- Created specific handlers for each command type
64
65		2. **Model Management**:
66		- Added `available_models` parameter to store available models
67		- Implemented `get_available_models` method to discover models from Ollama
68		- Created `update_available_models` method to refresh model list
69		- Added `_change_model` method to handle model switching
70
71		3. **Context Tracking**:
72		- Added `user_contexts` dictionary to store conversation history
73		- Enhanced `process_with_ollama` to maintain context
74		- Implemented context pruning to limit memory usage
75		- Added context clearing when switching models
76
77		4. **Admin Management**:
78		- Added `admin_users` parameter to designate admin users
79		- Implemented `add_admin_user` and `remove_admin_user` methods
80		- Created foundation for admin-only commands
81
82		### Configuration Enhancements
83
84		The configuration system was enhanced to support:
85
86		1. **Model Configuration**:
87		- Added `AVAILABLE_MODELS` list in the Ollama configuration section
88		- Updated default configuration to include sample models
89
90		2. **Admin Configuration**:
91		- Added `ADMIN_USERS` list in the Matrix configuration section
92		- Updated default configuration to include sample admin users
93
94		### API Enhancements
95
96		The module API was enhanced to support:
97
98		1. **New Parameters**:
99		- Added `available_models` parameter to `connect_to_matrix` function
100		- Added `admin_users` parameter to `connect_to_matrix` function
101
102		2. **Enhanced Process Method**:
103		- Updated `process_with_ollama` to accept sender and room_id parameters
104		- Modified message handling to use enhanced processing
105
106		## Future Work
107
108		While the current enhancements provide a solid foundation for model selection and user context tracking, there are several areas for future improvement:
109
110		1. **Streaming Responses**:
111		- Implement streaming for longer responses from Ollama
112		- Add progress indicators during model processing
113
114		2. **Enhanced Admin Controls**:
115		- Add admin-only commands for system management
116		- Implement room-specific model restrictions
117		- Add user management commands
118
119		3. **Performance Optimizations**:
120		- Optimize context storage for large deployments
121		- Implement caching for frequently used models
122		- Add batch processing for high-traffic rooms
123
124		4. **Security Enhancements**:
125		- Add rate limiting to prevent abuse
126		- Implement more granular permission controls
127		- Add encryption for stored contexts
128
129		## Conclusion
130
131		The enhancements to the Matrix integration significantly improve the user experience by allowing model selection and maintaining conversation context. These features make the Matrix client more versatile and useful for real-world applications.
132
133		The implementation follows best practices for code organization, error handling, and documentation. The enhanced Matrix client is now ready for use in production environments and provides a solid foundation for future improvements.

ollamatools-project/COMPLETION_REPORT_MODULAR.md +0 -134

…	…	@@ -1,134 +0,0 @@
1		# ModularOllamaAgent Implementation - Completion Report
2
3		## Summary
4
5		This report documents the successful implementation of the ModularOllamaAgent, a modular and extensible replacement for the TransparentOllamaAgent. The implementation includes:
6
7		1. **Core Modular Architecture**: Created a modular architecture that separates functionality into logical components
8		2. **Extended Functionality**: Integrated conversation history tracking, topic analysis, and performance metrics
9		3. **Comprehensive Documentation**: Provided detailed documentation for using and extending the agent
10		4. **Backward Compatibility**: Maintained backward compatibility with the TransparentOllamaAgent
11
12		## Implementation Details
13
14		### 1. Core Modular Architecture
15
16		We created a modular architecture with the following components:
17
18		- **agent.py**: Main agent implementation
19		- **search.py**: Search functionality
20		- **commands.py**: Command management
21		- **issues.py**: Issue tracking
22		- **feedback.py**: Semantic function selection feedback
23		- **utils.py**: Utility functions
24
25		This architecture separates concerns, making the code more maintainable and extensible.
26
27		### 2. Extended Functionality
28
29		We extended the ModularOllamaAgent with additional features:
30
31		- **Conversation History Tracking**: Records all conversations, including user messages, agent responses, response times, and estimated token usage
32		- **Conversation Summarization**: Provides summaries of recent conversations
33		- **Topic Analysis**: Analyzes conversation topics based on keywords
34		- **Performance Metrics**: Tracks response times and estimated token usage
35
36		These features are implemented in the ExtendedModularAgent class, which demonstrates how to extend the ModularOllamaAgent with custom functionality.
37
38		### 3. Comprehensive Documentation
39
40		We created detailed documentation for the ModularOllamaAgent:
41
42		- **ModularOllamaAgent Guide**: Comprehensive guide to using the ModularOllamaAgent
43		- **Migration Guide**: Guide for migrating from TransparentOllamaAgent to ModularOllamaAgent
44		- **Module-Level READMEs**: Documentation for each module explaining its purpose and usage
45		- **Example Scripts**: Example scripts demonstrating the ModularOllamaAgent and ExtendedModularAgent
46
47		### 4. Backward Compatibility
48
49		We maintained backward compatibility with the TransparentOllamaAgent:
50
51		- **Alias Class**: Created an alias for ModularOllamaAgent as TransparentOllamaAgent
52		- **Deprecation Warning**: Added a deprecation warning to encourage migration to ModularOllamaAgent
53		- **Same API**: Ensured the ModularOllamaAgent has the same API as the TransparentOllamaAgent
54
55		## Files Created/Modified
56
57		### New Directories
58		- `ollamatools/agent/modular/`
59		- `ollamatools/agent/extended/`
60
61		### New Files
62		- `ollamatools/agent/modular/agent.py`
63		- `ollamatools/agent/modular/search.py`
64		- `ollamatools/agent/modular/commands.py`
65		- `ollamatools/agent/modular/issues.py`
66		- `ollamatools/agent/modular/feedback.py`
67		- `ollamatools/agent/modular/utils.py`
68		- `ollamatools/agent/modular/__init__.py`
69		- `ollamatools/agent/modular/README.md`
70		- `ollamatools/agent/extended/__init__.py`
71		- `ollamatools/agent/extended/README.md`
72		- `examples/modular_agent_example.py`
73		- `examples/extended_modular_agent.py`
74		- `docs/modular_agent_guide.md`
75		- `docs/migration_guide.md`
76
77		### Modified Files
78		- `ollamatools/agent/__init__.py`
79		- `ollamatools/agent/transparent/__init__.py`
80
81		## Code Quality Improvements
82
83		1. **Better Modularity**: Separated functionality into logical components
84		2. **Enhanced Documentation**: Added comprehensive docstrings and type hints
85		3. **Consistent Naming**: Used consistent naming conventions throughout the codebase
86		4. **Error Handling**: Improved error handling and error messages
87		5. **Code Reuse**: Eliminated code duplication through proper modularization
88
89		## Logical Audit
90
91		We conducted a logical audit of each functionality to ensure it achieves the project's goals:
92
93		### Search Functionality
94		- **Goal**: Provide seamless search integration with natural language understanding
95		- **Implementation**: The search module includes intent detection, search execution, and result processing
96		- **Assessment**: The implementation achieves the goal of providing seamless search integration
97
98		### Command Management
99		- **Goal**: Provide a secure and flexible way to manage allowed commands
100		- **Implementation**: The commands module includes listing, adding, and removing commands
101		- **Assessment**: The implementation achieves the goal of providing secure command management
102
103		### Issue Tracking
104		- **Goal**: Provide a way to track and manage issues
105		- **Implementation**: The issues module includes creating, listing, and updating issues
106		- **Assessment**: The implementation achieves the goal of providing issue tracking
107
108		### Feedback System
109		- **Goal**: Improve the agent's ability to understand natural language queries
110		- **Implementation**: The feedback module includes feedback collection, pattern analysis, and command intent detection
111		- **Assessment**: The implementation achieves the goal of improving natural language understanding
112
113		### Extended Functionality
114		- **Goal**: Demonstrate how to extend the agent with custom functionality
115		- **Implementation**: The ExtendedModularAgent includes conversation history tracking, topic analysis, and performance metrics
116		- **Assessment**: The implementation achieves the goal of demonstrating extensibility
117
118		## Future Improvements
119
120		While the current implementation is robust and feature-rich, there are several areas for future improvement:
121
122		1. **Advanced Natural Language Understanding**: Enhance the feedback system with more sophisticated language understanding techniques
123		2. **Plugin Architecture**: Develop a formal plugin architecture for easier extension
124		3. **Configuration Management**: Add proper configuration file support for customizing agent behavior
125		4. **Testing Framework**: Develop a comprehensive testing framework for validating agent functionality
126		5. **Performance Optimization**: Optimize performance for large-scale usage
127
128		## Conclusion
129
130		The ModularOllamaAgent implementation successfully consolidates all functionality from the TransparentOllamaAgent into a modular, extensible architecture. The implementation is well-documented, maintains backward compatibility, and demonstrates how to extend the agent with custom functionality.
131
132		The modular architecture makes the code more maintainable and extensible, while the comprehensive documentation makes it easier for users to understand and use the agent. The logical audit confirms that each functionality achieves its intended goals.
133
134		Overall, the ModularOllamaAgent is a significant improvement over the TransparentOllamaAgent and provides a solid foundation for future development.

ollamatools-project/CONTRIBUTING.md +0 -122

…	…	@@ -1,122 +0,0 @@
1		# Contributing to OllamaTools
2
3		Thank you for considering contributing to OllamaTools! This document provides guidelines and instructions for contributing to the project.
4
5		## Code of Conduct
6
7		Please be respectful and considerate of others when participating in this project. We aim to foster an inclusive and welcoming community.
8
9		## Getting Started
10
11		### Development Environment Setup
12
13		1. Fork the repository
14		2. Clone your fork to your local machine:
15		```bash
16		git clone https://github.com/yourusername/ollamatools.git
17		cd ollamatools
18		```
19		3. Create a virtual environment:
20		```bash
21		python -m venv venv
22		source venv/bin/activate # On Windows, use: venv\Scripts\activate
23		```
24		4. Install the package in development mode:
25		```bash
26		pip install -e ".[dev,test]"
27		```
28
29		### Running Tests
30
31		We use pytest for testing. To run the tests:
32
33		```bash
34		# Run all tests
35		pytest
36
37		# Run specific tests
38		pytest tests/test_config.py
39		pytest tests/test_security.py
40
41		# Run with coverage report
42		pytest --cov=ollamatools
43		```
44
45		## Development Workflow
46
47		1. Create a new branch for your feature or bugfix:
48		```bash
49		git checkout -b feature/your-feature-name
50		```
51		or
52		```bash
53		git checkout -b fix/your-bugfix-name
54		```
55
56		2. Make your changes, following the coding standards.
57
58		3. Add tests for your changes.
59
60		4. Run the tests to ensure they pass:
61		```bash
62		pytest
63		```
64
65		5. Update documentation if necessary.
66
67		6. Commit your changes with a descriptive commit message:
68		```bash
69		git commit -m "Add feature: your feature description"
70		```
71
72		7. Push your branch to your fork:
73		```bash
74		git push origin feature/your-feature-name
75		```
76
77		8. Create a pull request to the main repository.
78
79		## Coding Standards
80
81		- Follow PEP 8 style guidelines
82		- Use type hints for all functions and methods
83		- Write comprehensive docstrings for all public APIs
84		- Include unit tests for all new functionality
85		- Maintain backwards compatibility unless there's a compelling reason not to
86
87		## Pull Request Process
88
89		1. Ensure all tests pass and code meets the coding standards.
90		2. Update documentation to reflect any changes.
91		3. Increment the version number in `setup.py` according to semantic versioning.
92		4. Update the CHANGELOG.md file with your changes.
93		5. The pull request will be reviewed by maintainers, who may request changes.
94		6. Once the pull request is approved, it will be merged into the main branch.
95
96		## Reporting Bugs
97
98		When reporting bugs, please include:
99		- A clear and descriptive title
100		- Steps to reproduce the issue
101		- Expected behavior
102		- Actual behavior
103		- Screenshots if applicable
104		- Your environment (OS, Python version, package versions)
105
106		## Feature Requests
107
108		When requesting features, please include:
109		- A clear and descriptive title
110		- A detailed description of the feature
111		- Why the feature would be useful to most users
112		- Any additional context or screenshots
113
114		## Security Vulnerabilities
115
116		If you discover a security vulnerability, please do NOT open an issue. Instead, email [security@example.com] with details. We take security issues very seriously and will address them promptly.
117
118		## Contact
119
120		If you have any questions, feel free to reach out to the maintainers.
121
122		Thank you for contributing to OllamaTools!

ollamatools-project/IMPLEMENTATION_ISSUES.md +0 -250

…	…	@@ -1,250 +0,0 @@
1		# OllamaTools Implementation Issues and Solutions
2
3		After a thorough code review of the OllamaTools project, focusing on the integration between matrix, modular, and extended agent libraries, I've identified several specific implementation issues that need to be addressed to achieve the unified plugin architecture proposed in the documentation.
4
5		## 1. Improper Module Organization for ExtendedModularAgent
6
7		**Issue:** The `ExtendedModularAgent` is defined in an example file (`examples/extended_modular_agent.py`) rather than as a proper module in the package structure, violating package organization principles.
8
9		**Prompt Plan:**
10		```
11		Create a proper module structure for the ExtendedModularAgent by:
12
13		1. Create a new file at ollamatools/agent/extended/agent.py
14		2. Move the ExtendedModularAgent class from examples/extended_modular_agent.py to this new file
15		3. Add proper imports and exports in ollamatools/agent/extended/__init__.py to expose the ExtendedModularAgent
16		4. Update the examples/extended_modular_agent.py file to import from the new module location
17		5. Ensure all functionality is preserved with the same API
18		6. Add proper docstrings and type hints to the new module
19		```
20
21		## 2. Duplicated Conversation History Functionality
22
23		**Issue:** There is significant duplication between `enhanced_matrix_history.py` and the `ExtendedModularAgent` class, particularly in conversation history tracking, topic analysis, and performance metrics.
24
25		**Prompt Plan:**
26		```
27		Create a shared ConversationHistoryPlugin that can be used by both the Matrix client and agent components:
28
29		1. Create a new file at ollamatools/plugins/conversation_history.py
30		2. Implement a ConversationHistoryPlugin class that follows the SharedPlugin interface from the unified architecture
31		3. Extract the common functionality from enhanced_matrix_history.py and ExtendedModularAgent:
32		- Conversation loading/saving
33		- Recording conversations
34		- Topic analysis
35		- Performance metrics
36		4. Add proper configuration options for file paths, history limits, etc.
37		5. Implement the initialize(), shutdown(), and get_capabilities() methods
38		6. Add unit tests for the new plugin
39		7. Update the ExtendedModularAgent to use this plugin instead of its own implementation
40		8. Update the EnhancedMatrixClient to use this plugin instead of the functions in enhanced_matrix_history.py
41		```
42
43		## 3. Inconsistent Plugin Architecture in Matrix Integration
44
45		**Issue:** The modular agent has a well-defined plugin architecture, but this isn't consistently applied across the matrix integration, leading to ad-hoc integration points and conditional imports.
46
47		**Prompt Plan:**
48		```
49		Refactor the Matrix integration to use the plugin architecture:
50
51		1. Create a new file at ollamatools/plugins/matrix_core.py
52		2. Implement a MatrixCorePlugin class that follows the MatrixPlugin interface
53		3. Extract the core Matrix functionality from enhanced_matrix_core.py into this plugin
54		4. Create a PluginEnabledMatrixClient class that uses the plugin registry
55		5. Update the Matrix client initialization to register and initialize plugins
56		6. Modify the message handling to use plugins for processing messages
57		7. Add proper error handling and logging
58		8. Update the Matrix command handling to use plugins for command registration and execution
59		9. Add unit tests for the new plugin-enabled client
60		10. Update examples to use the new plugin-enabled client
61		```
62
63		## 4. Weak Integration Between Matrix Client and Agent
64
65		**Issue:** The matrix integration and agent architecture are separate components that borrow from each other rather than being integrated in a clean way.
66
67		**Prompt Plan:**
68		```
69		Create a MatrixAgentAdapterPlugin to properly connect Matrix clients with agents:
70
71		1. Create a new file at ollamatools/plugins/matrix_agent_adapter.py
72		2. Implement a MatrixAgentAdapterPlugin class that follows the MatrixPlugin interface
73		3. The plugin should:
74		- Accept an agent class and configuration in its constructor
75		- Initialize the agent in its initialize() method
76		- Handle Matrix messages by passing them to the agent in on_message()
77		- Forward agent commands to Matrix in get_commands()
78		- Clean up the agent in shutdown()
79		4. Add proper error handling and logging
80		5. Add unit tests for the adapter plugin
81		6. Create an example that demonstrates using the adapter to connect a Matrix client with different agent types
82		7. Update the documentation to explain how to use the adapter
83		```
84
85		## 5. Lack of Unified Configuration
86
87		**Issue:** Each component has its own configuration approach, leading to inconsistent configuration handling.
88
89		**Prompt Plan:**
90		```
91		Create a unified configuration system for the OllamaTools project:
92
93		1. Create a new file at ollamatools/config/plugin_config.py
94		2. Implement a PluginConfig class that can:
95		- Load configuration from YAML, JSON, or Python dictionaries
96		- Validate configuration against schemas
97		- Provide default values for missing configuration
98		- Merge configurations from different sources
99		3. Update the plugin registry to use this configuration system
100		4. Add configuration schemas for each plugin type
101		5. Update the plugin initialization to use the new configuration system
102		6. Add utility functions for creating default configurations
103		7. Add unit tests for the configuration system
104		8. Update examples to use the new configuration system
105		9. Add documentation for the configuration system
106		```
107
108		## 6. Inconsistent Error Handling
109
110		**Issue:** Error handling varies across the codebase, with different approaches to exception handling.
111
112		**Prompt Plan:**
113		```
114		Create a unified error handling system for plugins:
115
116		1. Create a new file at ollamatools/plugins/errors.py
117		2. Define a hierarchy of exception classes for plugin-related errors:
118		- PluginError as the base class
119		- PluginInitializationError for initialization failures
120		- PluginDependencyError for dependency issues
121		- PluginExecutionError for runtime errors
122		3. Implement utility functions for handling plugin errors:
123		- format_plugin_error() to create user-friendly error messages
124		- log_plugin_error() to log errors with appropriate context
125		- handle_plugin_exception() to catch and process exceptions
126		4. Update the plugin registry to use the new error handling system
127		5. Update plugins to use the new exception classes
128		6. Add unit tests for the error handling system
129		7. Add documentation for the error handling system
130		```
131
132		## 7. Limited Extensibility for Matrix Commands
133
134		**Issue:** The matrix client's command handling is hardcoded, making it difficult to extend with new commands.
135
136		**Prompt Plan:**
137		```
138		Create a command registry system for Matrix plugins:
139
140		1. Create a new file at ollamatools/plugins/matrix_commands.py
141		2. Implement a MatrixCommandRegistry class that can:
142		- Register command handlers from plugins
143		- Parse and route commands to the appropriate handler
144		- Provide help text for available commands
145		- Handle command errors
146		3. Update the MatrixPlugin interface to include a get_commands() method
147		4. Modify the PluginEnabledMatrixClient to use the command registry
148		5. Create example command plugins:
149		- HelpCommandPlugin for providing help
150		- SearchCommandPlugin for search commands
151		- HistoryCommandPlugin for history commands
152		6. Add unit tests for the command registry
153		7. Update examples to demonstrate adding custom commands
154		8. Add documentation for the command system
155		```
156
157		## 8. No Standardized Plugin Discovery
158
159		**Issue:** There's no standardized way to discover and load plugins across the project.
160
161		**Prompt Plan:**
162		```
163		Implement a plugin discovery system using entry points:
164
165		1. Update the PluginRegistry class to discover plugins using entry points
166		2. Add a setup.py entry_points configuration for built-in plugins
167		3. Create a plugin discovery function that can:
168		- Find plugins in specified directories
169		- Load plugins from entry points
170		- Load plugins from configuration
171		4. Add a plugin validation function to ensure plugins implement the required interfaces
172		5. Update the plugin initialization to use the discovery system
173		6. Add unit tests for the discovery system
174		7. Create an example third-party plugin package
175		8. Add documentation for creating and distributing plugins
176		```
177
178		## 9. Inefficient Conversation Context Management
179
180		**Issue:** The current context management in the matrix client is simplistic and could lead to memory issues.
181
182		**Prompt Plan:**
183		```
184		Create a ConversationContextPlugin for efficient context management:
185
186		1. Create a new file at ollamatools/plugins/conversation_context.py
187		2. Implement a ConversationContextPlugin class that:
188		- Maintains a sliding window of conversation history
189		- Implements intelligent context pruning based on relevance
190		- Supports different context strategies (time-based, token-based, etc.)
191		- Provides methods for adding to and retrieving context
192		3. Add configuration options for context size, pruning strategy, etc.
193		4. Update the Matrix client and agent to use this plugin for context management
194		5. Add unit tests for the context management plugin
195		6. Add documentation for the context management system
196		```
197
198		## 10. Limited Integration with Extended Agent Features
199
200		**Issue:** The matrix client doesn't leverage the more sophisticated history features available in the ExtendedModularAgent.
201
202		**Prompt Plan:**
203		```
204		Create adapter plugins to expose extended agent features to Matrix:
205
206		1. Create a new file at ollamatools/plugins/extended_agent_features.py
207		2. Implement adapter plugins for each extended feature:
208		- TopicAnalysisPlugin for topic analysis
209		- PerformanceMetricsPlugin for performance tracking
210		- ConversationSummaryPlugin for conversation summarization
211		3. Each plugin should:
212		- Follow the MatrixPlugin interface
213		- Add Matrix commands for accessing the feature
214		- Connect to the appropriate agent functionality
215		4. Update the MatrixAgentAdapterPlugin to register these plugins when using an extended agent
216		5. Add unit tests for the adapter plugins
217		6. Create examples demonstrating the extended features in Matrix
218		7. Add documentation for the extended features
219		```
220
221		## Implementation Strategy
222
223		To implement these changes effectively, I recommend a phased approach:
224
225		1. **Phase 1: Core Infrastructure**
226		- Implement the base plugin interfaces and registry
227		- Create the unified configuration system
228		- Implement the error handling system
229
230		2. **Phase 2: Shared Plugins**
231		- Implement the ConversationHistoryPlugin
232		- Implement the ConversationContextPlugin
233		- Create tests for these plugins
234
235		3. **Phase 3: Matrix Integration**
236		- Refactor the Matrix client to use plugins
237		- Implement the command registry
238		- Create the MatrixCorePlugin
239
240		4. **Phase 4: Agent Integration**
241		- Move the ExtendedModularAgent to a proper module
242		- Update agents to use shared plugins
243		- Create the MatrixAgentAdapterPlugin
244
245		5. **Phase 5: Extended Features**
246		- Implement the extended feature adapter plugins
247		- Create the plugin discovery system
248		- Update examples and documentation
249
250		This phased approach ensures that each component builds on the previous ones, creating a solid foundation for the unified plugin architecture.

ollamatools-project/ISSUES.md +0 -399

…	…	@@ -1,399 +0,0 @@
1		# OllamaTools Project Issues
2
3		This document outlines key issues identified in the OllamaTools project, focusing on integration between matrix, modular, and extended agent libraries.
4
5		## 1. Inconsistent Plugin Architecture Across Components
6
7		**Description:**
8		The modular agent has a well-defined plugin architecture, but this architecture isn't consistently applied across the matrix integration. This leads to ad-hoc integration points and conditional imports rather than a clean, extensible design.
9
10		**Impact:**
11		- Difficult to extend the matrix integration with new capabilities
12		- Fragile dependencies between components
13		- Inconsistent code organization
14
15		**Suggested Solution:**
16		Implement a consistent plugin architecture across all components:
17		- Define a clear plugin interface for matrix integrations
18		- Allow matrix clients to load plugins from the modular and extended agent libraries
19		- Create a registry of available plugins that can be dynamically loaded
20
21		**Example Implementation:**
22		```python
23		# Define a plugin interface
24		class MatrixPlugin:
25		def initialize(self, client):
26		"""Initialize the plugin with the client instance."""
27		pass
28
29		def on_message(self, room, event, client):
30		"""Handle incoming messages."""
31		return False # Return True if message was handled
32
33		def get_commands(self):
34		"""Return commands provided by this plugin."""
35		return {}
36
37		# Example search plugin
38		class SearchPlugin(MatrixPlugin):
39		def initialize(self, client):
40		self.opensearch_config = client.opensearch_config
41
42		def on_message(self, room, event, client):
43		# Handle search-related functionality
44		pass
45
46		def get_commands(self):
47		return {
48		"/search": self.handle_search_command
49		}
50		```
51
52		## 2. Duplicated Functionality Between Enhanced Matrix and Extended Agent
53
54		**Description:**
55		There's significant duplication of functionality between `enhanced_matrix_history.py` and the `ExtendedModularAgent` class, particularly in conversation history tracking, topic analysis, and performance metrics.
56
57		**Impact:**
58		- Code duplication leads to maintenance challenges
59		- Inconsistent implementations of the same features
60		- Difficult to ensure feature parity across components
61
62		**Suggested Solution:**
63		Extract common functionality into shared utility modules that can be used by both the matrix integration and the agent libraries:
64		- Create a `conversation_tracking` module with shared functionality
65		- Implement a common interface for history storage and retrieval
66		- Ensure consistent metrics calculation across all components
67
68		**Example Implementation:**
69		```python
70		# In ollamatools/utils/conversation_tracking.py
71		class ConversationTracker:
72		def __init__(self, history_file=None):
73		# Initialize conversation tracking
74		pass
75
76		def record_conversation(self, user_message, assistant_response, metadata=None):
77		# Record a conversation
78		pass
79
80		def analyze_topics(self):
81		# Analyze conversation topics
82		pass
83
84		def get_metrics(self):
85		# Get performance metrics
86		pass
87		```
88
89		## 3. Improper Module Organization for Extended Agent
90
91		**Description:**
92		The `ExtendedModularAgent` is defined in an example file (`examples/extended_modular_agent.py`) rather than as a proper module, and `ollamatools/agent/extended/__init__.py` simply imports from this example. This violates proper package organization principles.
93
94		**Impact:**
95		- Confusing import paths
96		- Difficult to maintain and extend
97		- Potential for circular imports
98		- Inconsistent with the rest of the project structure
99
100		**Suggested Solution:**
101		Reorganize the extended agent as a proper module:
102		- Move the `ExtendedModularAgent` class to `ollamatools/agent/extended/agent.py`
103		- Update `__init__.py` to import from this module
104		- Ensure proper separation of example code and library code
105
106		**Example Implementation:**
107		```python
108		# In ollamatools/agent/extended/agent.py
109		from ollamatools.agent import ModularOllamaAgent
110
111		class ExtendedModularAgent(ModularOllamaAgent):
112		# Implementation here
113		pass
114
115		# In ollamatools/agent/extended/__init__.py
116		from .agent import ExtendedModularAgent
117
118		__all__ = ['ExtendedModularAgent']
119		```
120
121		## 4. Lack of Unified Configuration System
122
123		**Description:**
124		Each component (matrix, modular agent, extended agent) has its own configuration approach, leading to inconsistent configuration handling across the project.
125
126		**Impact:**
127		- Difficult to configure multiple components consistently
128		- Redundant configuration code
129		- Potential for configuration conflicts
130
131		**Suggested Solution:**
132		Implement a unified configuration system:
133		- Create a central configuration registry
134		- Allow components to register their configuration schemas
135		- Provide validation and merging of configurations
136		- Support hierarchical configuration with inheritance
137
138		**Example Implementation:**
139		```python
140		# In ollamatools/config.py
141		class ConfigRegistry:
142		def __init__(self):
143		self.schemas = {}
144		self.defaults = {}
145
146		def register_schema(self, component, schema, defaults=None):
147		# Register configuration schema for a component
148		pass
149
150		def validate_config(self, component, config):
151		# Validate configuration against schema
152		pass
153
154		def get_config(self, component, user_config=None):
155		# Get configuration with defaults applied
156		pass
157		```
158
159		## 5. Weak Integration Between Matrix Client and Agent Architecture
160
161		**Description:**
162		The matrix integration and agent architecture are separate components that borrow from each other rather than being integrated in a clean way. The matrix client uses features from the modular agent conditionally, but doesn't fully leverage the agent's capabilities.
163
164		**Impact:**
165		- Limited ability to use agent features in matrix clients
166		- Inconsistent user experience between direct agent usage and matrix-based interaction
167		- Difficult to extend with new agent capabilities
168
169		**Suggested Solution:**
170		Create a proper integration layer between matrix clients and agents:
171		- Implement an adapter pattern to connect matrix clients to agents
172		- Allow matrix clients to delegate message processing to agents
173		- Ensure consistent behavior between direct agent usage and matrix-based interaction
174
175		**Example Implementation:**
176		```python
177		# In ollamatools/tools/matrix_agent_adapter.py
178		class MatrixAgentAdapter:
179		def __init__(self, agent_class, agent_config=None):
180		# Initialize with agent class and configuration
181		self.agent = agent_class(**(agent_config or {}))
182
183		async def process_message(self, message, sender, room_id):
184		# Process message using the agent
185		return self.agent.chat(message)
186
187		def get_agent_commands(self):
188		# Get commands supported by the agent
189		return self.agent.get_commands()
190		```
191
192		## 6. Inconsistent Error Handling Across Components
193
194		**Description:**
195		Error handling is inconsistent across the codebase, with some components using try/except blocks with specific error types, others using generic exception handling, and others relying on the caller to handle errors.
196
197		**Impact:**
198		- Unpredictable error behavior
199		- Difficult to debug issues
200		- Potential for unhandled exceptions
201
202		**Suggested Solution:**
203		Implement a consistent error handling strategy:
204		- Define a hierarchy of custom exception types
205		- Document which functions can raise which exceptions
206		- Ensure consistent error reporting across components
207		- Implement proper error recovery mechanisms
208
209		**Example Implementation:**
210		```python
211		# In ollamatools/errors.py
212		class OllamaToolsError(Exception):
213		"""Base class for all OllamaTools errors."""
214		pass
215
216		class MatrixError(OllamaToolsError):
217		"""Base class for Matrix-related errors."""
218		pass
219
220		class AgentError(OllamaToolsError):
221		"""Base class for Agent-related errors."""
222		pass
223
224		class ConfigError(OllamaToolsError):
225		"""Configuration error."""
226		pass
227		```
228
229		## 7. Limited Extensibility for Matrix Command Handling
230
231		**Description:**
232		The matrix client's command handling is hardcoded with specific command patterns and handlers, making it difficult to extend with new commands or modify existing ones.
233
234		**Impact:**
235		- Difficult to add new commands
236		- Cannot customize command behavior
237		- Tight coupling between command detection and execution
238
239		**Suggested Solution:**
240		Implement a flexible command registry:
241		- Allow registration of commands with patterns and handlers
242		- Support command categories and help text
243		- Enable plugins to register their own commands
244		- Implement command permissions and validation
245
246		**Example Implementation:**
247		```python
248		# In ollamatools/tools/matrix_commands.py
249		class CommandRegistry:
250		def __init__(self):
251		self.commands = {}
252
253		def register_command(self, pattern, handler, help_text=None, category=None):
254		# Register a command
255		self.commands[pattern] = {
256		'handler': handler,
257		'help_text': help_text,
258		'category': category
259		}
260
261		async def handle_command(self, message, room_id, sender, client):
262		# Find and execute matching command
263		for pattern, command in self.commands.items():
264		match = re.match(pattern, message)
265		if match:
266		return await command['handler'](match, room_id, sender, client)
267		return False
268		```
269
270		## 8. Lack of Standardized Plugin Discovery Mechanism
271
272		**Description:**
273		There's no standardized way to discover and load plugins across the project, leading to manual imports and conditional feature flags.
274
275		**Impact:**
276		- Difficult to extend the system with new plugins
277		- Manual management of dependencies
278		- Inconsistent plugin loading across components
279
280		**Suggested Solution:**
281		Implement a plugin discovery mechanism:
282		- Use entry points or a similar mechanism for plugin registration
283		- Provide a plugin registry that can discover and load plugins
284		- Support plugin dependencies and conflicts
285		- Enable dynamic loading and unloading of plugins
286
287		**Example Implementation:**
288		```python
289		# In ollamatools/plugin.py
290		class PluginRegistry:
291		def __init__(self):
292		self.plugins = {}
293
294		def discover_plugins(self):
295		# Discover plugins using entry points
296		import pkg_resources
297		for entry_point in pkg_resources.iter_entry_points('ollamatools.plugins'):
298		plugin_class = entry_point.load()
299		self.register_plugin(entry_point.name, plugin_class)
300
301		def register_plugin(self, name, plugin_class):
302		# Register a plugin
303		self.plugins[name] = plugin_class
304
305		def get_plugin(self, name):
306		# Get a plugin by name
307		return self.plugins.get(name)
308		```
309
310		## 9. Inefficient Conversation Context Management
311
312		**Description:**
313		The current conversation context management in the matrix client is simplistic and could lead to memory issues with long-running bots or busy rooms.
314
315		**Impact:**
316		- Potential memory leaks
317		- Degraded performance over time
318		- Limited context window management
319
320		**Suggested Solution:**
321		Implement a more sophisticated context management system:
322		- Use a sliding window approach for context
323		- Implement context summarization for long conversations
324		- Add context pruning based on relevance
325		- Support context persistence across restarts
326
327		**Example Implementation:**
328		```python
329		# In ollamatools/utils/context_manager.py
330		class ConversationContext:
331		def __init__(self, max_messages=10, max_tokens=1000):
332		self.messages = []
333		self.max_messages = max_messages
334		self.max_tokens = max_tokens
335
336		def add_message(self, role, content):
337		# Add a message to the context
338		self.messages.append({"role": role, "content": content})
339		self._prune_context()
340
341		def _prune_context(self):
342		# Prune context if it exceeds limits
343		while len(self.messages) > self.max_messages:
344		self.messages.pop(0)
345
346		# Token-based pruning
347		total_tokens = sum(len(m["content"].split()) for m in self.messages)
348		while total_tokens > self.max_tokens and len(self.messages) > 1:
349		removed = self.messages.pop(0)
350		total_tokens -= len(removed["content"].split())
351
352		def get_context(self):
353		# Get the current context
354		return self.messages
355		```
356
357		## 10. Lack of Integration with Extended Agent's History Features
358
359		**Description:**
360		The enhanced matrix client implements its own conversation history tracking, but doesn't leverage the more sophisticated history features available in the ExtendedModularAgent.
361
362		**Impact:**
363		- Duplicated functionality
364		- Inconsistent history tracking between matrix and agent
365		- Limited history analysis capabilities in matrix client
366
367		**Suggested Solution:**
368		Integrate the matrix client with the ExtendedModularAgent's history features:
369		- Create a shared history storage mechanism
370		- Allow the matrix client to use the agent's history analysis
371		- Ensure consistent history tracking across all components
372		- Implement a unified API for history access
373
374		**Example Implementation:**
375		```python
376		# In ollamatools/tools/enhanced_matrix_core.py
377		from ollamatools.agent.extended import ExtendedModularAgent
378
379		class EnhancedMatrixClient(MatrixClient):
380		def __init__(self, *args, **kwargs):
381		super().__init__(*args, **kwargs)
382
383		# Create an agent instance for history tracking
384		self.agent = ExtendedModularAgent(
385		model=self.ollama_model,
386		# Other configuration
387		)
388
389		async def _message_callback(self, room, event):
390		# Process message
391		# ...
392
393		# Use agent for history tracking
394		self.agent.record_conversation(message, response)
395
396		async def handle_history_command(self, count, room_id):
397		# Use agent's history features
398		summary = self.agent.summarize_conversations(count)
399		await self.send_message(room_id, summary)

ollamatools-project/LICENSE +0 -21

…	…	@@ -1,21 +0,0 @@
1		MIT License
2
3		Copyright (c) 2025 OllamaTools Contributors
4
5		Permission is hereby granted, free of charge, to any person obtaining a copy
6		of this software and associated documentation files (the "Software"), to deal
7		in the Software without restriction, including without limitation the rights
8		to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9		copies of the Software, and to permit persons to whom the Software is
10		furnished to do so, subject to the following conditions:
11
12		The above copyright notice and this permission notice shall be included in all
13		copies or substantial portions of the Software.
14
15		THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16		IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17		FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18		AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19		LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20		OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21		SOFTWARE.

ollamatools-project/PHASE1_IMPLEMENTATION_PLAN.md +0 -1105

…	…	@@ -1,1105 +0,0 @@
1		# Phase 1: Core Infrastructure Implementation Plan
2
3		This document outlines the detailed implementation plan for Phase 1 of the unified plugin architecture for OllamaTools. Phase 1 focuses on creating the core infrastructure needed for the plugin system.
4
5		## 1. Base Plugin Interface and Registry
6
7		### 1.1 Create Plugin Interface Module
8
9		**File:** `ollamatools/plugins/__init__.py`
10
11		```python
12		"""
13		OllamaTools Plugin System.
14
15		This module provides the base interfaces and registry for the OllamaTools plugin system.
16		"""
17
18		import os
19		import sys
20		import importlib
21		import logging
22		from typing import Dict, Any, List, Optional, Set, Type, Callable, Union
23
24		# Configure logging
25		logger = logging.getLogger(__name__)
26
27		class OllamaToolsPlugin:
28		"""Base interface for all OllamaTools plugins."""
29
30		def __init__(self, config=None):
31		"""
32		Initialize the plugin with optional configuration.
33
34		Args:
35		config: Optional configuration dictionary for the plugin
36		"""
37		self.config = config or {}
38
39		def initialize(self, context):
40		"""
41		Initialize the plugin with the runtime context.
42
43		Args:
44		context: Runtime context dictionary containing references to other components
45		"""
46		pass
47
48		def shutdown(self):
49		"""Clean up resources when shutting down."""
50		pass
51
52		@property
53		def name(self):
54		"""
55		Get the plugin name.
56
57		Returns:
58		str: The plugin name
59		"""
60		return self.__class__.__name__
61
62		@property
63		def version(self):
64		"""
65		Get the plugin version.
66
67		Returns:
68		str: The plugin version
69		"""
70		return "1.0.0"
71
72		@property
73		def dependencies(self):
74		"""
75		Get plugin dependencies.
76
77		Returns:
78		List[str]: List of plugin names that this plugin depends on
79		"""
80		return []
81
82
83		class MatrixPlugin(OllamaToolsPlugin):
84		"""Base class for Matrix plugins."""
85
86		async def on_message(self, room, event, client):
87		"""
88		Handle an incoming Matrix message.
89
90		Args:
91		room: The room the message was sent in
92		event: The message event
93		client: The Matrix client
94
95		Returns:
96		bool: True if the message was handled, False otherwise
97		"""
98		return False
99
100		def get_commands(self):
101		"""
102		Get Matrix commands provided by this plugin.
103
104		Returns:
105		Dict[str, Callable]: Dictionary of command names to handler functions
106		"""
107		return {}
108
109		async def on_room_join(self, room, client):
110		"""
111		Handle room join event.
112
113		Args:
114		room: The room that was joined
115		client: The Matrix client
116		"""
117		pass
118
119		async def on_client_start(self, client):
120		"""
121		Handle client start event.
122
123		Args:
124		client: The Matrix client
125		"""
126		pass
127
128		async def on_client_stop(self, client):
129		"""
130		Handle client stop event.
131
132		Args:
133		client: The Matrix client
134		"""
135		pass
136
137
138		class AgentPlugin(OllamaToolsPlugin):
139		"""Base class for agent plugins."""
140
141		def process_message(self, message, context):
142		"""
143		Process a message and return a response.
144
145		Args:
146		message: The user message
147		context: Processing context
148
149		Returns:
150		Optional[str]: Response message if handled, None otherwise
151		"""
152		return None
153
154		def get_commands(self):
155		"""
156		Get commands provided by this plugin.
157
158		Returns:
159		Dict[str, Callable]: Dictionary of command names to handler functions
160		"""
161		return {}
162
163		def on_function_call(self, function_name, args, context):
164		"""
165		Handle a function call.
166
167		Args:
168		function_name: The name of the function being called
169		args: The arguments to the function
170		context: Processing context
171
172		Returns:
173		Any: The result of the function call
174		"""
175		return None
176
177
178		class SharedPlugin(OllamaToolsPlugin):
179		"""Base class for plugins that can be used by multiple components."""
180
181		def get_capabilities(self):
182		"""
183		Get capabilities provided by this plugin.
184
185		Returns:
186		List[str]: List of capability names
187		"""
188		return []
189		```
190
191		### 1.2 Create Plugin Registry
192
193		**File:** `ollamatools/plugins/registry.py`
194
195		```python
196		"""
197		Plugin Registry for OllamaTools.
198
199		This module provides the PluginRegistry class for managing OllamaTools plugins.
200		"""
201
202		import os
203		import sys
204		import importlib
205		import logging
206		import pkg_resources
207		from typing import Dict, Any, List, Optional, Set, Type, Callable, Union
208
209		from ollamatools.plugins import OllamaToolsPlugin
210		from ollamatools.plugins.errors import (
211		PluginError,
212		PluginInitializationError,
213		PluginDependencyError,
214		PluginNotFoundError,
215		handle_plugin_exception
216		)
217
218		# Configure logging
219		logger = logging.getLogger(__name__)
220
221		class PluginRegistry:
222		"""Registry for OllamaTools plugins."""
223
224		def __init__(self):
225		"""Initialize the plugin registry."""
226		self.plugins = {}
227		self.initialized_plugins = {}
228
229		def register_plugin(self, plugin_class, config=None):
230		"""
231		Register a plugin class.
232
233		Args:
234		plugin_class: The plugin class to register
235		config: Optional configuration for the plugin
236
237		Returns:
238		PluginRegistry: Self for method chaining
239		"""
240		plugin_name = plugin_class.__name__
241		self.plugins[plugin_name] = {
242		"class": plugin_class,
243		"config": config
244		}
245		logger.debug(f"Registered plugin: {plugin_name}")
246		return self
247
248		def discover_plugins(self):
249		"""
250		Discover plugins using entry points.
251
252		Returns:
253		PluginRegistry: Self for method chaining
254		"""
255		try:
256		for entry_point in pkg_resources.iter_entry_points('ollamatools.plugins'):
257		try:
258		plugin_class = entry_point.load()
259		self.register_plugin(plugin_class)
260		logger.info(f"Discovered plugin: {plugin_class.__name__} from entry point {entry_point.name}")
261		except Exception as e:
262		logger.error(f"Error loading plugin from entry point {entry_point.name}: {e}")
263		except Exception as e:
264		logger.error(f"Error discovering plugins: {e}")
265
266		return self
267
268		def initialize_plugin(self, plugin_name, context):
269		"""
270		Initialize a plugin and its dependencies.
271
272		Args:
273		plugin_name: The name of the plugin to initialize
274		context: Runtime context for the plugin
275
276		Returns:
277		OllamaToolsPlugin: The initialized plugin instance
278
279		Raises:
280		PluginNotFoundError: If the plugin is not found
281		PluginInitializationError: If the plugin fails to initialize
282		PluginDependencyError: If a dependency cannot be initialized
283		"""
284		# Return already initialized plugin
285		if plugin_name in self.initialized_plugins:
286		return self.initialized_plugins[plugin_name]
287
288		# Check if plugin exists
289		if plugin_name not in self.plugins:
290		raise PluginNotFoundError(f"Plugin {plugin_name} not found")
291
292		logger.debug(f"Initializing plugin: {plugin_name}")
293
294		plugin_info = self.plugins[plugin_name]
295		plugin_class = plugin_info["class"]
296		plugin_config = plugin_info["config"]
297
298		try:
299		# Create plugin instance
300		plugin_instance = plugin_class(plugin_config)
301
302		# Initialize dependencies first
303		for dependency in plugin_instance.dependencies:
304		try:
305		self.initialize_plugin(dependency, context)
306		except PluginError as e:
307		raise PluginDependencyError(
308		f"Failed to initialize dependency {dependency} for plugin {plugin_name}",
309		details={"plugin": plugin_name, "dependency": dependency, "original_error": str(e)}
310		) from e
311
312		# Initialize the plugin
313		plugin_instance.initialize(context)
314		self.initialized_plugins[plugin_name] = plugin_instance
315
316		logger.info(f"Initialized plugin: {plugin_name}")
317		return plugin_instance
318
319		except PluginError:
320		# Re-raise plugin errors
321		raise
322		except Exception as e:
323		# Wrap other exceptions
324		raise PluginInitializationError(
325		f"Failed to initialize plugin {plugin_name}: {str(e)}",
326		details={"plugin": plugin_name}
327		) from e
328
329		def initialize_all(self, context):
330		"""
331		Initialize all registered plugins.
332
333		Args:
334		context: Runtime context for the plugins
335
336		Returns:
337		Dict[str, OllamaToolsPlugin]: Dictionary of initialized plugins
338		"""
339		for plugin_name in list(self.plugins.keys()):
340		try:
341		self.initialize_plugin(plugin_name, context)
342		except PluginError as e:
343		logger.error(f"Error initializing plugin {plugin_name}: {e}")
344		handle_plugin_exception(e)
345		except Exception as e:
346		logger.error(f"Unexpected error initializing plugin {plugin_name}: {e}")
347
348		return self.initialized_plugins
349
350		def get_plugin(self, plugin_name):
351		"""
352		Get an initialized plugin.
353
354		Args:
355		plugin_name: The name of the plugin to get
356
357		Returns:
358		Optional[OllamaToolsPlugin]: The plugin instance or None if not found
359		"""
360		return self.initialized_plugins.get(plugin_name)
361
362		def get_plugins_by_type(self, plugin_type):
363		"""
364		Get all initialized plugins of a specific type.
365
366		Args:
367		plugin_type: The plugin type class
368
369		Returns:
370		Dict[str, OllamaToolsPlugin]: Dictionary of plugin names to instances
371		"""
372		return {
373		name: plugin for name, plugin in self.initialized_plugins.items()
374		if isinstance(plugin, plugin_type)
375		}
376
377		def shutdown(self):
378		"""
379		Shutdown all initialized plugins.
380
381		Returns:
382		PluginRegistry: Self for method chaining
383		"""
384		for plugin_name, plugin in list(self.initialized_plugins.items()):
385		try:
386		logger.debug(f"Shutting down plugin: {plugin_name}")
387		plugin.shutdown()
388		logger.info(f"Shut down plugin: {plugin_name}")
389		except Exception as e:
390		logger.error(f"Error shutting down plugin {plugin_name}: {e}")
391
392		self.initialized_plugins = {}
393		return self
394		```
395
396		## 2. Error Handling System
397
398		### 2.1 Create Plugin Error Module
399
400		**File:** `ollamatools/plugins/errors.py`
401
402		```python
403		"""
404		Error handling for the OllamaTools plugin system.
405
406		This module provides exception classes and utility functions for handling
407		errors in the OllamaTools plugin system.
408		"""
409
410		import sys
411		import logging
412		import traceback
413		from typing import Dict, Any, Optional
414
415		# Configure logging
416		logger = logging.getLogger(__name__)
417
418		class PluginError(Exception):
419		"""Base class for all plugin-related errors."""
420
421		def __init__(self, message, details=None):
422		"""
423		Initialize the error.
424
425		Args:
426		message: Error message
427		details: Optional dictionary with additional error details
428		"""
429		super().__init__(message)
430		self.details = details or {}
431
432
433		class PluginNotFoundError(PluginError):
434		"""Error raised when a plugin is not found."""
435		pass
436
437
438		class PluginInitializationError(PluginError):
439		"""Error raised when a plugin fails to initialize."""
440		pass
441
442
443		class PluginDependencyError(PluginError):
444		"""Error raised when a plugin dependency cannot be satisfied."""
445		pass
446
447
448		class PluginExecutionError(PluginError):
449		"""Error raised when a plugin operation fails during execution."""
450		pass
451
452
453		class PluginConfigurationError(PluginError):
454		"""Error raised when a plugin has invalid configuration."""
455		pass
456
457
458		def format_plugin_error(error: PluginError) -> str:
459		"""
460		Format a plugin error into a user-friendly message.
461
462		Args:
463		error: The plugin error
464
465		Returns:
466		str: Formatted error message
467		"""
468		message = str(error)
469
470		if isinstance(error, PluginNotFoundError):
471		return f"Plugin not found: {message}"
472
473		if isinstance(error, PluginInitializationError):
474		return f"Failed to initialize plugin: {message}"
475
476		if isinstance(error, PluginDependencyError):
477		dependency = error.details.get("dependency", "unknown")
478		return f"Plugin dependency error: {message} (missing: {dependency})"
479
480		if isinstance(error, PluginExecutionError):
481		plugin = error.details.get("plugin", "unknown")
482		operation = error.details.get("operation", "unknown operation")
483		return f"Plugin execution error in {plugin}.{operation}: {message}"
484
485		if isinstance(error, PluginConfigurationError):
486		plugin = error.details.get("plugin", "unknown")
487		return f"Invalid configuration for plugin {plugin}: {message}"
488
489		# Generic plugin error
490		return f"Plugin error: {message}"
491
492
493		def log_plugin_error(error: PluginError, level=logging.ERROR) -> None:
494		"""
495		Log a plugin error with appropriate context.
496
497		Args:
498		error: The plugin error
499		level: Logging level
500		"""
501		# Format the error message
502		message = format_plugin_error(error)
503
504		# Add details if available
505		if error.details:
506		details_str = ", ".join(f"{k}={v}" for k, v in error.details.items())
507		message = f"{message} [Details: {details_str}]"
508
509		# Log the error
510		logger.log(level, message, exc_info=True)
511
512
513		def handle_plugin_exception(error: Exception) -> Dict[str, Any]:
514		"""
515		Handle a plugin exception and return error details.
516
517		Args:
518		error: The exception to handle
519
520		Returns:
521		Dict[str, Any]: Error details
522		"""
523		error_type = type(error).__name__
524		error_message = str(error)
525		error_traceback = traceback.format_exc()
526
527		# Log the error
528		if isinstance(error, PluginError):
529		log_plugin_error(error)
530		else:
531		logger.error(f"Unexpected error: {error_type}: {error_message}", exc_info=True)
532
533		# Create error details
534		error_details = {
535		"error_type": error_type,
536		"error_message": error_message,
537		"traceback": error_traceback
538		}
539
540		# Add plugin-specific details
541		if isinstance(error, PluginError) and error.details:
542		error_details["details"] = error.details
543
544		return error_details
545		```
546
547		## 3. Unified Configuration System
548
549		### 3.1 Create Plugin Configuration Module
550
551		**File:** `ollamatools/config/plugin_config.py`
552
553		```python
554		"""
555		Configuration system for the OllamaTools plugin architecture.
556
557		This module provides classes and functions for managing plugin configuration.
558		"""
559
560		import os
561		import sys
562		import yaml
563		import json
564		import logging
565		from typing import Dict, Any, List, Optional, Set, Union, Callable
566
567		from ollamatools.plugins.errors import PluginConfigurationError
568
569		# Configure logging
570		logger = logging.getLogger(__name__)
571
572		class PluginConfig:
573		"""Configuration manager for plugins."""
574
575		def __init__(self, config=None, schema=None):
576		"""
577		Initialize the configuration manager.
578
579		Args:
580		config: Initial configuration dictionary or path to config file
581		schema: Optional schema for validation
582		"""
583		self.schema = schema or {}
584		self.config = {}
585
586		if config:
587		self.load_config(config)
588
589		def load_config(self, config):
590		"""
591		Load configuration from a dictionary, JSON, or YAML file.
592
593		Args:
594		config: Configuration source (dict, path to JSON/YAML file)
595
596		Returns:
597		PluginConfig: Self for method chaining
598
599		Raises:
600		PluginConfigurationError: If configuration loading fails
601		"""
602		try:
603		if isinstance(config, dict):
604		# Load from dictionary
605		self.config = config
606		elif isinstance(config, str) and os.path.exists(config):
607		# Load from file
608		with open(config, 'r') as f:
609		if config.endswith('.json'):
610		self.config = json.load(f)
611		elif config.endswith(('.yaml', '.yml')):
612		self.config = yaml.safe_load(f)
613		else:
614		raise PluginConfigurationError(
615		f"Unsupported configuration file format: {config}",
616		details={"file": config}
617		)
618		else:
619		raise PluginConfigurationError(
620		f"Invalid configuration source: {config}",
621		details={"source": str(config)}
622		)
623
624		# Validate the configuration
625		if self.schema:
626		self.validate()
627
628		return self
629
630		except PluginConfigurationError:
631		# Re-raise plugin configuration errors
632		raise
633		except Exception as e:
634		# Wrap other exceptions
635		raise PluginConfigurationError(
636		f"Failed to load configuration: {str(e)}",
637		details={"source": str(config)}
638		) from e
639
640		def validate(self):
641		"""
642		Validate the configuration against the schema.
643
644		Returns:
645		bool: True if valid
646
647		Raises:
648		PluginConfigurationError: If validation fails
649		"""
650		# Simple schema validation
651		for key, schema_item in self.schema.items():
652		# Check required fields
653		if schema_item.get('required', False) and key not in self.config:
654		raise PluginConfigurationError(
655		f"Missing required configuration key: {key}",
656		details={"key": key, "schema": schema_item}
657		)
658
659		# Check type if the key exists
660		if key in self.config:
661		value = self.config[key]
662		expected_type = schema_item.get('type')
663
664		if expected_type and not isinstance(value, expected_type):
665		raise PluginConfigurationError(
666		f"Invalid type for configuration key {key}. Expected {expected_type.__name__}, got {type(value).__name__}",
667		details={"key": key, "expected_type": expected_type.__name__, "actual_type": type(value).__name__}
668		)
669
670		# Check enum values
671		enum_values = schema_item.get('enum')
672		if enum_values and value not in enum_values:
673		raise PluginConfigurationError(
674		f"Invalid value for configuration key {key}. Expected one of {enum_values}, got {value}",
675		details={"key": key, "expected_values": enum_values, "actual_value": value}
676		)
677
678		# Check custom validation
679		validator = schema_item.get('validator')
680		if validator and callable(validator):
681		if not validator(value):
682		raise PluginConfigurationError(
683		f"Validation failed for configuration key {key}",
684		details={"key": key, "value": value}
685		)
686
687		return True
688
689		def get(self, key, default=None):
690		"""
691		Get a configuration value.
692
693		Args:
694		key: Configuration key
695		default: Default value if key is not found
696
697		Returns:
698		Any: Configuration value or default
699		"""
700		return self.config.get(key, default)
701
702		def set(self, key, value):
703		"""
704		Set a configuration value.
705
706		Args:
707		key: Configuration key
708		value: Configuration value
709
710		Returns:
711		PluginConfig: Self for method chaining
712
713		Raises:
714		PluginConfigurationError: If validation fails
715		"""
716		self.config[key] = value
717
718		# Validate if schema exists for this key
719		if self.schema and key in self.schema:
720		schema_item = self.schema[key]
721
722		# Check type
723		expected_type = schema_item.get('type')
724		if expected_type and not isinstance(value, expected_type):
725		raise PluginConfigurationError(
726		f"Invalid type for configuration key {key}. Expected {expected_type.__name__}, got {type(value).__name__}",
727		details={"key": key, "expected_type": expected_type.__name__, "actual_type": type(value).__name__}
728		)
729
730		# Check enum values
731		enum_values = schema_item.get('enum')
732		if enum_values and value not in enum_values:
733		raise PluginConfigurationError(
734		f"Invalid value for configuration key {key}. Expected one of {enum_values}, got {value}",
735		details={"key": key, "expected_values": enum_values, "actual_value": value}
736		)
737
738		# Check custom validation
739		validator = schema_item.get('validator')
740		if validator and callable(validator):
741		if not validator(value):
742		raise PluginConfigurationError(
743		f"Validation failed for configuration key {key}",
744		details={"key": key, "value": value}
745		)
746
747		return self
748
749		def merge(self, other_config):
750		"""
751		Merge another configuration into this one.
752
753		Args:
754		other_config: Another PluginConfig or dictionary
755
756		Returns:
757		PluginConfig: Self for method chaining
758		"""
759		if isinstance(other_config, PluginConfig):
760		other_dict = other_config.to_dict()
761		else:
762		other_dict = other_config
763
764		# Deep merge
765		self._deep_merge(self.config, other_dict)
766
767		# Validate the merged configuration
768		if self.schema:
769		self.validate()
770
771		return self
772
773		def _deep_merge(self, target, source):
774		"""
775		Deep merge two dictionaries.
776
777		Args:
778		target: Target dictionary
779		source: Source dictionary
780		"""
781		for key, value in source.items():
782		if key in target and isinstance(target[key], dict) and isinstance(value, dict):
783		self._deep_merge(target[key], value)
784		else:
785		target[key] = value
786
787		def to_dict(self):
788		"""
789		Convert the configuration to a dictionary.
790
791		Returns:
792		Dict[str, Any]: Configuration dictionary
793		"""
794		return self.config.copy()
795
796		def save(self, file_path, format='yaml'):
797		"""
798		Save the configuration to a file.
799
800		Args:
801		file_path: Path to save the configuration to
802		format: File format ('json' or 'yaml')
803
804		Returns:
805		PluginConfig: Self for method chaining
806
807		Raises:
808		PluginConfigurationError: If saving fails
809		"""
810		try:
811		# Create directory if it doesn't exist
812		os.makedirs(os.path.dirname(file_path), exist_ok=True)
813
814		with open(file_path, 'w') as f:
815		if format.lower() == 'json':
816		json.dump(self.config, f, indent=2)
817		elif format.lower() in ('yaml', 'yml'):
818		yaml.dump(self.config, f, default_flow_style=False)
819		else:
820		raise PluginConfigurationError(
821		f"Unsupported configuration format: {format}",
822		details={"format": format}
823		)
824
825		return self
826
827		except PluginConfigurationError:
828		# Re-raise plugin configuration errors
829		raise
830		except Exception as e:
831		# Wrap other exceptions
832		raise PluginConfigurationError(
833		f"Failed to save configuration: {str(e)}",
834		details={"file_path": file_path, "format": format}
835		) from e
836
837
838		def create_default_config(plugin_name, config_type='shared'):
839		"""
840		Create a default configuration for a plugin.
841
842		Args:
843		plugin_name: Name of the plugin
844		config_type: Type of configuration ('shared', 'matrix', 'agent')
845
846		Returns:
847		Dict[str, Any]: Default configuration
848		"""
849		# Base configuration
850		config = {
851		"enabled": True,
852		"name": plugin_name
853		}
854
855		# Add type-specific defaults
856		if config_type == 'shared':
857		pass # No additional defaults
858		elif config_type == 'matrix':
859		config.update({
860		"commands_enabled": True,
861		"command_prefix": "/"
862		})
863		elif config_type == 'agent':
864		config.update({
865		"functions_enabled": True
866		})
867
868		return config
869
870
871		def load_plugin_config(plugin_name, config_dir=None):
872		"""
873		Load configuration for a plugin from the default location.
874
875		Args:
876		plugin_name: Name of the plugin
877		config_dir: Optional custom configuration directory
878
879		Returns:
880		PluginConfig: Plugin configuration
881		"""
882		if not config_dir:
883		# Use default location
884		config_dir = os.path.join(
885		os.path.expanduser('~'),
886		'.ollamatools',
887		'plugins'
888		)
889
890		# Check for plugin-specific config file
891		config_file = os.path.join(config_dir, f"{plugin_name.lower()}.yaml")
892
893		if os.path.exists(config_file):
894		return PluginConfig(config_file)
895		else:
896		# Return default configuration
897		return PluginConfig(create_default_config(plugin_name))
898		```
899
900		### 3.2 Create Configuration Schema Module
901
902		**File:** `ollamatools/config/schemas.py`
903
904		```python
905		"""
906		Configuration schemas for OllamaTools plugins.
907
908		This module provides schema definitions for validating plugin configurations.
909		"""
910
911		from typing import Dict, Any, List, Optional, Set, Union, Callable
912
913		# Schema for the ConversationHistoryPlugin
914		CONVERSATION_HISTORY_SCHEMA = {
915		"history_file": {
916		"type": str,
917		"required": False,
918		"description": "Path to the conversation history file"
919		},
920		"max_history": {
921		"type": int,
922		"required": False,
923		"default": 100,
924		"description": "Maximum number of conversations to keep in history"
925		},
926		"enable_topic_analysis": {
927		"type": bool,
928		"required": False,
929		"default": True,
930		"description": "Whether to enable topic analysis"
931		},
932		"enable_metrics": {
933		"type": bool,
934		"required": False,
935		"default": True,
936		"description": "Whether to enable performance metrics"
937		}
938		}
939
940		# Schema for the SearchPlugin
941		SEARCH_PLUGIN_SCHEMA = {
942		"base_url": {
943		"type": str,
944		"required": False,
945		"default": "http://localhost:8081",
946		"description": "Base URL for the OpenSearchNG service"
947		},
948		"engines": {
949		"type": list,
950		"required": False,
951		"default": ["google", "bing", "duckduckgo", "brave"],
952		"description": "List of search engines to use"
953		},
954		"num_results": {
955		"type": int,
956		"required": False,
957		"default": 5,
958		"description": "Number of search results to return"
959		},
960		"enable_auto_search": {
961		"type": bool,
962		"required": False,
963		"default": True,
964		"description": "Whether to enable automatic search for factual queries"
965		}
966		}
967
968		# Schema for the MatrixAgentAdapterPlugin
969		MATRIX_AGENT_ADAPTER_SCHEMA = {
970		"agent_class": {
971		"type": (str, object), # Can be a string path or a class object
972		"required": True,
973		"description": "Agent class to use (string path or class object)"
974		},
975		"agent_config": {
976		"type": dict,
977		"required": False,
978		"default": {},
979		"description": "Configuration for the agent"
980		}
981		}
982
983		# Schema for the MatrixCorePlugin
984		MATRIX_CORE_PLUGIN_SCHEMA = {
985		"homeserver": {
986		"type": str,
987		"required": True,
988		"description": "URL of the Matrix homeserver"
989		},
990		"user_id": {
991		"type": str,
992		"required": True,
993		"description": "Matrix user ID"
994		},
995		"password": {
996		"type": str,
997		"required": True,
998		"description": "Matrix password"
999		},
1000		"device_id": {
1001		"type": str,
1002		"required": False,
1003		"description": "Device ID for E2E encryption"
1004		},
1005		"ssl": {
1006		"type": bool,
1007		"required": False,
1008		"default": True,
1009		"description": "Whether to use SSL for the Matrix connection"
1010		},
1011		"respond_to_all_messages": {
1012		"type": bool,
1013		"required": False,
1014		"default": False,
1015		"description": "Whether to respond to all messages (not just mentions)"
1016		}
1017		}
1018
1019		# Schema for the ConversationContextPlugin
1020		CONVERSATION_CONTEXT_SCHEMA = {
1021		"max_context_length": {
1022		"type": int,
1023		"required": False,
1024		"default": 10,
1025		"description": "Maximum number of messages to keep in context"
1026		},
1027		"context_strategy": {
1028		"type": str,
1029		"required": False,
1030		"default": "sliding_window",
1031		"enum": ["sliding_window", "token_based", "time_based", "relevance_based"],
1032		"description": "Strategy for managing conversation context"
1033		},
1034		"token_limit": {
1035		"type": int,
1036		"required": False,
1037		"default": 4096,
1038		"description": "Maximum number of tokens to keep in context (for token_based strategy)"
1039		},
1040		"time_window": {
1041		"type": int,
1042		"required": False,
1043		"default": 3600,
1044		"description": "Time window in seconds for context (for time_based strategy)"
1045		}
1046		}
1047
1048		# Schema registry
1049		SCHEMA_REGISTRY = {
1050		"ConversationHistoryPlugin": CONVERSATION_HISTORY_SCHEMA,
1051		"SearchPlugin": SEARCH_PLUGIN_SCHEMA,
1052		"MatrixAgentAdapterPlugin": MATRIX_AGENT_ADAPTER_SCHEMA,
1053		"MatrixCorePlugin": MATRIX_CORE_PLUGIN_SCHEMA,
1054		"ConversationContextPlugin": CONVERSATION_CONTEXT_SCHEMA
1055		}
1056
1057		def get_schema_for_plugin(plugin_name):
1058		"""
1059		Get the configuration schema for a plugin.
1060
1061		Args:
1062		plugin_name: Name of the plugin
1063
1064		Returns:
1065		Dict[str, Any]: Schema dictionary or empty dict if not found
1066		"""
1067		return SCHEMA_REGISTRY.get(plugin_name, {})
1068		```
1069
1070		## 4. Directory Structure Setup
1071
1072		### 4.1 Create Plugin Package Structure
1073
1074		```
1075		ollamatools/
1076		├── plugins/
1077		│ ├── __init__.py # Base plugin interfaces
1078		│ ├── registry.py # Plugin registry
1079		│ ├── errors.py # Error handling
1080		│ └── README.md # Documentation
1081		├── config/
1082		│ ├── __init__.py # Package initialization
1083		│ ├── plugin_config.py # Configuration system
1084		│ └── schemas.py # Configuration schemas
1085		```
1086
1087		### 4.2 Create Plugin README
1088
1089		**File:** `ollamatools/plugins/README.md`
1090
1091		```markdown
1092		# OllamaTools Plugin System
1093
1094		This directory contains the core components of the OllamaTools plugin system.
1095
1096		## Overview
1097
1098		The plugin system provides a flexible way to extend OllamaTools functionality through plugins. Plugins can be used to add new features, integrate with external services, or customize behavior.
1099
1100		## Plugin Types
1101
1102		The plugin system supports several types of plugins:
1103
1104		- **SharedPlugin**: Plugins that can be used by multiple components
1105		- **MatrixPlugin

ollamatools-project/README.md +0 -50

…	…	@@ -1,50 +0,0 @@
1		# OllamaTools: Function Library for Ollama LLMs
2
3		[](https://github.com/yourusername/ollamatools/actions/workflows/python-tests.yml)
4		[](https://opensource.org/licenses/MIT)
5		[](https://badge.fury.io/py/ollamatools)
6		[](https://www.python.org/downloads/)
7
8		OllamaTools is a Python library that extends Ollama with semantic function calling capabilities, enabling automatic invocation of system commands, web requests, and documentation utilities based on conversational context.
9
10		## Overview
11
12		OllamaTools allows Ollama LLMs to execute external functions based on semantic understanding of user queries without requiring explicit enabling/disabling of tools. The library focuses on security by implementing a permission system and provides a comprehensive set of tools for system operations, web interactions, and documentation generation.
13
14		## Features
15
16		- **Semantic Function Detection**: Automatically detect and execute functions based on conversation context
17		- **Security-First Design**: Prompt-based permission system that ensures user control over function execution
18		- **System Tools**: Execute commands, read/write files, and find files with proper permission management
19		- **Web Tools**: Make HTTP requests, fetch weather data, and perform web searches
20		- **Documentation Tools**: Generate markdown, extract relevant context, and summarize text
21		- **Configurable**: Extensive configuration options for all components
22		- **Well-Tested**: Comprehensive test suite ensuring reliability and security
23
24		## Installation
25
26		```bash
27		# Install from PyPI
28		pip install ollamatools
29
30		# Install from source
31		git clone https://github.com/yourusername/ollamatools.git
32		cd ollamatools
33		pip install -e .
34		```
35
36		## Quick Start
37
38		### Basic Usage
39
40		```python
41		from ollamatools import OllamaAgent
42
43		# Create agent with default settings
44		agent = OllamaAgent(model="llama3.1")
45
46		# Chat with automatic function detection
47		response = agent.chat("What's the weather like in San Francisco?")
48		print(response)
49
50		#

ollamatools-project/README_REFACTORING.md +0 -79

…	…	@@ -1,79 +0,0 @@
1		# OllamaTools Refactoring Guide
2
3		This guide explains the refactoring approach for the OllamaTools project, which involves creating a parallel directory structure for the new implementation while keeping the original codebase as a reference.
4
5		## Refactoring Approach
6
7		The refactoring approach follows these steps:
8
9		1. Create a new directory (`ollamatools-new`) at the same level as the current project
10		2. Set up the new directory structure according to the simplification plan
11		3. Implement components incrementally following the phased approach in the plan
12		4. Use the original codebase as reference while building the new structure
13		5. Run tests against both codebases to ensure functional equivalence
14		6. Once verified, switch references to the new structure
15		7. Rename directories (old to `ollamatools-legacy`, new to `ollamatools`)
16
17		This approach has several advantages:
18		- Allows for a clean implementation of the new architecture
19		- Minimizes confusion between old and new code
20		- Enables incremental testing and verification
21		- Provides a clear separation between legacy and new code
22		- Makes it easier to follow the phased implementation plan
23		- Allows for easier rollback if issues arise
24
25		## Setup Script
26
27		The `setup_refactored_structure.sh` script automates the creation of the new directory structure. It:
28
29		1. Creates a new directory (`ollamatools-new`) at the parent level
30		2. Copies essential files from the current project (README, setup.py, etc.)
31		3. Creates the main package directory and all subdirectories according to the simplification plan
32		4. Creates empty files for all modules defined in the plan
33		5. Creates a REFACTORING.md file with information about the refactoring process
34
35		### Usage
36
37		To use the script:
38
39		```bash
40		# Make the script executable
41		chmod +x setup_refactored_structure.sh
42
43		# Run the script
44		./setup_refactored_structure.sh
45		```
46
47		## Implementation Process
48
49		After setting up the directory structure, the implementation should follow the phased approach outlined in the simplification plan:
50
51		1. **Core Infrastructure** (core module)
52		2. **Plugin System** (plugins base)
53		3. **Standard Plugins** (plugins/standard)
54		4. **Tool Plugins** (plugins/tools)
55		5. **Integration Plugins** (plugins/integrations)
56		6. **Agent Implementations** (agents)
57		7. **Configuration System** (config)
58		8. **Command-Line Interface** (cli)
59
60		For each component:
61		1. Refer to the original codebase to understand the current functionality
62		2. Implement the component according to the new architecture
63		3. Write tests to verify the implementation
64		4. Document the component
65
66		## Final Migration
67
68		Once the refactored implementation is complete and tested:
69
70		1. Create a backup of the original codebase
71		2. Rename the original directory to `ollamatools-legacy`
72		3. Rename the new directory to `ollamatools`
73		4. Update any references to the project path
74
75		## Additional Resources
76
77		- See `SIMPLIFICATION_PLAN.md` for the overall refactoring plan
78		- See `ai_simplification_guides/` for detailed implementation guides
79		- See `REFACTORING.md` in the new project directory for more information

ollamatools-project/REFACTORING_SCRIPTS.md +0 -153

…	…	@@ -1,153 +0,0 @@
1		# OllamaTools Refactoring Scripts
2
3		This document provides a summary of all the scripts created to help with the refactoring process for the OllamaTools project.
4
5		## Setup Scripts
6
7		### `setup_refactored_structure.sh`
8
9		Creates the new directory structure for the refactored project according to the simplification plan.
10
11		**Usage:**
12		```bash
13		./setup_refactored_structure.sh
14		```
15
16		**What it does:**
17		- Creates a new directory (`ollamatools-new`) at the parent level
18		- Sets up the directory structure according to the simplification plan
19		- Creates empty files for all modules defined in the plan
20		- Copies essential files from the current project (README, setup.py, etc.)
21		- Creates a REFACTORING.md file with information about the refactoring process
22
23		## File Mapping Scripts
24
25		### `generate_file_mapping.sh`
26
27		Generates a mapping between the old and new file structures to guide the refactoring process.
28
29		**Usage:**
30		```bash
31		./generate_file_mapping.sh
32		```
33
34		**What it does:**
35		- Creates a file_mapping.md file with a mapping between the original and new file structures
36		- Each entry includes the original file path, new file path, status, and notes
37		- The mapping serves as a guide for the refactoring process
38
39		### `update_mapping_status.sh`
40
41		Updates the status of a file in the file mapping as it is refactored.
42
43		**Usage:**
44		```bash
45		./update_mapping_status.sh <original_file> <status> [notes]
46		```
47
48		**Example:**
49		```bash
50		./update_mapping_status.sh ollamatools/agent.py Completed "Refactored with updated imports"
51		```
52
53		**What it does:**
54		- Updates the status of a file in the file mapping
55		- Optionally updates the notes for the file
56		- Helps track progress of the refactoring process
57
58		## File Refactoring Scripts
59
60		### `copy_refactor_file.sh`
61
62		Copies a file from the original codebase to the new structure and opens it in the editor for refactoring.
63
64		**Usage:**
65		```bash
66		./copy_refactor_file.sh <source_file> <destination_file>
67		```
68
69		**Example:**
70		```bash
71		./copy_refactor_file.sh ollamatools/agent.py ollamatools/core/agent.py
72		```
73
74		**What it does:**
75		- Copies a file from the original codebase to the new structure
76		- Opens the file in the editor for refactoring
77		- Creates any necessary directories for the destination file
78
79		## Testing Scripts
80
81		### `test_refactored_code.sh`
82
83		Runs tests for both the original and refactored codebases to ensure they have the same functionality.
84
85		**Usage:**
86		```bash
87		./test_refactored_code.sh [options] <test_file>
88		```
89
90		**Options:**
91		- `-o, --original`: Run tests only on the original codebase
92		- `-n, --new`: Run tests only on the new codebase
93		- `-h, --help`: Display help message
94
95		**Example:**
96		```bash
97		./test_refactored_code.sh test_agent.py # Run test_agent.py on both codebases
98		./test_refactored_code.sh --original test_agent.py # Run test_agent.py only on the original codebase
99		./test_refactored_code.sh --new test_agent.py # Run test_agent.py only on the new codebase
100		```
101
102		**What it does:**
103		- Runs tests for both the original and refactored codebases
104		- Compares the results to ensure they have the same functionality
105		- Helps verify that the refactored code maintains the same behavior
106
107		## Finalization Scripts
108
109		### `finalize_refactoring.sh`
110
111		Finalizes the refactoring process by renaming directories and creating backups.
112
113		**Usage:**
114		```bash
115		./finalize_refactoring.sh
116		```
117
118		**What it does:**
119		- Creates a backup of the original codebase
120		- Renames the original directory to `ollamatools-legacy`
121		- Renames the new directory to `ollamatools-project`
122		- Provides a confirmation prompt before proceeding
123
124		## Guide Scripts
125
126		### `refactoring_guide.sh`
127
128		Provides a step-by-step guide for the refactoring process.
129
130		**Usage:**
131		```bash
132		./refactoring_guide.sh
133		```
134
135		**What it does:**
136		- Outlines the steps to follow for the refactoring process
137		- Provides commands to run at each step
138		- Includes notes and warnings for each step
139		- Guides you through the entire refactoring process from setup to finalization
140
141		## Workflow
142
143		The typical workflow for using these scripts is:
144
145		1. Run `./setup_refactored_structure.sh` to create the new directory structure
146		2. Run `./generate_file_mapping.sh` to create the file mapping
147		3. Follow the steps in `./refactoring_guide.sh` to refactor the codebase
148		4. Use `./copy_refactor_file.sh` to copy and refactor files
149		5. Use `./update_mapping_status.sh` to track progress
150		6. Use `./test_refactored_code.sh` to test the refactored code
151		7. Run `./finalize_refactoring.sh` to finalize the refactoring process
152
153		This workflow ensures a systematic and organized approach to the refactoring process, with proper tracking and testing at each step.

ollamatools-project/SECURITY.md +0 -228

…	…	@@ -1,228 +0,0 @@
1		# OllamaTools Security Documentation
2
3		This document provides an overview of the security features and best practices for the OllamaTools library.
4
5		## Security Architecture
6
7		OllamaTools implements multiple layers of security protections:
8
9		1. **Permission System**: A comprehensive permission system that prompts for user approval before executing potentially sensitive operations
10		2. **Command Validation**: Robust validation and sanitization of system commands to prevent command injection attacks
11		3. **Path Validation**: Secure file path handling to prevent path traversal attacks and unauthorized file access
12		4. **URL Validation**: URL validation and sanitization to prevent SSRF and other web-related vulnerabilities
13		5. **Standardized Error Handling**: Consistent error handling with detailed diagnostics for security issues
14
15		## Permission System
16
17		The permission system is the first line of defense:
18
19		- **Interactive Prompts**: Users are prompted to approve sensitive operations before execution
20		- **Session Permissions**: Allows permissions to be granted for the duration of a session
21		- **Operation Modification**: Ability to modify operations before execution
22		- **Sudo Detection**: Special handling for sudo commands that elevate privileges
23
24		### Permission Options
25
26		When prompted for permission, users can:
27
28		- **Allow**: Grant permission for this operation only
29		- **Allow for Session**: Grant permission for similar operations in this session
30		- **Modify**: Modify the operation before execution
31		- **Deny**: Reject the operation
32
33		Example:
34		```
35		================
36		PERMISSION REQUEST
37		================
38		Function: execute_command
39		Arguments: command='ls', args=['-la']
40
41		Options:
42		[A] Allow: Grant permission for this operation
43		[S] Allow for Session: Grant permission for similar operations in this session
44		[M] Modify: Modify the operation before execution
45		[D] Deny: Reject the operation
46
47		Your choice [A/S/M/D]:
48		```
49
50		## Command Validation
51
52		The command validation module provides the following security features:
53
54		- **Command Allowlist**: Only explicitly allowed commands can be executed
55		- **Argument Validation**: Arguments are checked for dangerous patterns
56		- **Shell Injection Prevention**: Shell metacharacters are properly handled
57		- **Secure Environment**: Commands are executed in a sanitized environment
58		- **Timeout Controls**: All commands have configurable timeouts
59
60		### Default Command Allowlist
61
62		The default allowlist includes:
63
64		- Common system utilities (`ls`, `cat`, `pwd`, etc.)
65		- File operations (`cp`, `mv`, `rm`, etc.)
66		- Text processing tools (`grep`, `sed`, `awk`, etc.)
67		- Network utilities (`ping`, `curl`, etc.)
68
69		Custom allowlists can be defined if needed.
70
71		## Path Validation
72
73		File path validation provides:
74
75		- **Path Sandbox**: File operations are restricted to specific directories
76		- **Path Traversal Prevention**: Prevents directory traversal attacks
77		- **Symlink Handling**: Safe handling of symbolic links
78		- **Protected Path Enforcement**: System directories are protected from modification
79		- **Write Protection**: Additional validation for write operations to prevent data loss
80
81		### Path Sandbox Configuration
82
83		By default, file operations are restricted to the user's home directory. This can be customized:
84
85		```python
86		from ollamatools.path_validator import configure_path_sandbox
87
88		# Configure custom sandbox
89		sandbox_config = configure_path_sandbox(
90		sandbox_path="/custom/sandbox/path",
91		additional_allowed_paths={"/path/to/allow"},
92		additional_protected_paths={"/path/to/protect"}
93		)
94		```
95
96		## URL Validation
97
98		URL validation provides:
99
100		- **Scheme Validation**: Only http and https schemes are allowed by default
101		- **Domain Blocklist**: Blocks localhost, internal IPs, and other dangerous domains
102		- **Domain Allowlist**: Allows explicitly approved domains
103		- **URL Normalization**: Normalizes URLs to prevent validation bypass
104		- **Parameter Validation**: Validates URL parameters for security issues
105
106		### URL Validation Configuration
107
108		```python
109		from ollamatools.url_validator import configure_domain_lists
110
111		# Configure custom domain lists
112		allowlist, blocklist = configure_domain_lists(
113		additional_allowlist={"trusted-domain.com"},
114		additional_blocklist={"dangerous-domain.com"}
115		)
116		```
117
118		## Error Handling
119
120		The error handling system provides:
121
122		- **Standardized Error Classes**: Consistent hierarchy of error classes
123		- **Error Codes**: Numeric error codes for each type of error
124		- **Detailed Error Information**: Comprehensive error details for debugging
125		- **Security-Focused Logging**: Appropriate logging of security issues
126
127		### Error Types
128
129		- `OllamaToolsError`: Base exception for all errors
130		- `CommandError`: For command execution issues
131		- `FileError`: For file operation issues
132		- `WebError`: For web request issues
133		- `PermissionError`: For permission issues
134		- `ApiError`: For API-related issues
135		- `ConfigurationError`: For configuration issues
136		- `TimeoutError`: For timeout issues
137		- `OllamaClientError`: For Ollama client issues
138
139		## Best Practices
140
141		### Command Execution
142
143		1. **Minimize Command Usage**: Only use execute_command when absolutely necessary
144		2. **Use Specific Commands**: Be as specific as possible when executing commands
145		3. **Validate User Input**: Never pass unvalidated user input to commands
146		4. **Handle Errors**: Always check command results for errors
147		5. **Set Appropriate Timeouts**: Use timeouts appropriate to the expected runtime
148
149		### File Operations
150
151		1. **Read-Only When Possible**: Use read operations instead of write when possible
152		2. **Be Specific**: Use specific file paths rather than patterns when possible
153		3. **Validate File Contents**: Validate file contents before writing
154		4. **Use Path Validation**: Always use the path validation functions
155		5. **Check For Errors**: Handle file operation errors gracefully
156
157		### Web Requests
158
159		1. **Use Allowlists**: Define domain allowlists for web requests
160		2. **Set Timeouts**: Always set appropriate timeouts for web requests
161		3. **Validate Responses**: Check response status and content
162		4. **Limit Request Rate**: Avoid making too many requests in a short time
163		5. **Handle Errors**: Properly handle web request errors
164
165		## Configuration
166
167		Security features can be configured at different levels:
168
169		### Global Configuration
170
171		```python
172		from ollamatools import Config
173
174		config = Config(
175		permission_prompts=True, # Enable permission prompts
176		sudo_always_prompt=True, # Always prompt for sudo commands
177		timeout_seconds=30, # Default timeout for operations
178		max_file_size=1024 * 1024 # 1 MB max file size
179		)
180		```
181
182		### Function-Level Configuration
183
184		```python
185		from ollamatools.tools.system_tools import execute_command
186
187		# Custom allowlist for this operation
188		custom_allowlist = {"ls", "cat", "grep"}
189
190		result = execute_command(
191		command="ls",
192		args=["-la"],
193		timeout=10,
194		custom_allowlist=custom_allowlist
195		)
196		```
197
198		## Logging and Monitoring
199
200		OllamaTools includes comprehensive logging for security events:
201
202		- Permission requests and decisions
203		- Command execution attempts
204		- File operation access
205		- Web request details
206		- Security validation failures
207
208		Enable debug logging for more detailed security information:
209
210		```python
211		from ollamatools import enable_debug_logging
212
213		enable_debug_logging("ollamatools.security")
214		```
215
216		## Security Updates
217
218		The security enhancements in this version include:
219
220		1. Enhanced command validation and sanitization
221		2. Improved URL validation to prevent SSRF attacks
222		3. New path validation module for file operations
223		4. Standardized error handling
224		5. Better sudo command detection
225		6. Path sandboxing for file operations
226		7. Improved permission request workflow
227
228		Future updates will continue to improve security features based on evolving best practices.

ollamatools-project/SIMPLIFICATION_PLAN.md +0 -394

…	…	@@ -1,394 +0,0 @@
1		# OllamaTools Simplification Plan
2
3		This document outlines a comprehensive plan for simplifying and modularizing the OllamaTools project. It serves as a guide for both human developers and AI assistants working on the codebase.
4
5		## Current State Analysis
6
7		The OllamaTools project has evolved through several phases, resulting in a complex architecture with multiple overlapping components:
8
9		1. **Core Agent Functionality**: Basic Ollama agent with semantic function calling
10		2. **Modular Agent**: A more structured approach with separate modules for search, commands, issues, etc.
11		3. **Extended Agent**: Builds on the modular agent with conversation history and analytics
12		4. **Matrix Integration**: Connects to Matrix chat rooms with basic and enhanced clients
13		5. **Plugin System**: A unified architecture for extending functionality
14
15		This evolution has led to redundancy, complexity, and maintenance challenges.
16
17		## Simplification Goals
18
19		1. **Reduce Redundancy**: Eliminate duplicate code and consolidate similar functionality
20		2. **Improve Modularity**: Create a clear, plugin-based architecture
21		3. **Simplify Interfaces**: Provide consistent, well-documented interfaces
22		4. **Enhance Maintainability**: Make the codebase easier to understand and modify
23		5. **Preserve Functionality**: Maintain all current features and capabilities
24		6. **Streamline Testing**: Improve testability through better separation of concerns
25
26		## Target Architecture
27
28		```
29		ollamatools/
30		├── core/ # Core functionality
31		├── plugins/ # Plugin system and implementations
32		│ ├── standard/ # Standard plugins
33		│ ├── tools/ # Tool plugins
34		│ └── integrations/ # Integration plugins
35		├── agents/ # Agent implementations
36		├── config/ # Configuration system
37		└── cli/ # Command-line interface
38		```
39
40		## Detailed Component Breakdown
41
42		### 1. Core Layer
43
44		The Core Layer provides essential functionality that all other components depend on.
45
46		**Files:**
47		```
48		ollamatools/core/
49		├── __init__.py # Core module initialization
50		├── agent.py # Base agent implementation
51		├── config.py # Configuration management
52		├── errors.py # Error definitions and handling
53		├── security.py # Security utilities and permission system
54		└── utils/
55		├── __init__.py
56		├── logger.py # Logging utilities
57		├── validators.py # Input validation (combines command/path/url validators)
58		└── formatting.py # Text formatting utilities
59		```
60
61		**Key Responsibilities:**
62		- Provide base agent functionality
63		- Handle configuration management
64		- Define error types and handling
65		- Implement security features
66		- Provide utility functions
67
68		### 2. Plugin System
69
70		The Plugin System enables extending functionality through plugins.
71
72		**Files:**
73		```
74		ollamatools/plugins/
75		├── __init__.py # Plugin system initialization
76		├── base.py # Base plugin interfaces
77		├── registry.py # Plugin discovery and management
78		└── errors.py # Plugin-specific error handling
79		```
80
81		**Key Responsibilities:**
82		- Define plugin interfaces
83		- Manage plugin lifecycle
84		- Handle plugin discovery and registration
85		- Resolve plugin dependencies
86
87		### 3. Standard Plugins
88
89		Standard Plugins implement common functionality used by different agents.
90
91		**Files:**
92		```
93		ollamatools/plugins/standard/
94		├── __init__.py
95		├── search.py # Search functionality
96		├── commands.py # Command management
97		├── conversation/
98		│ ├── __init__.py
99		│ ├── history.py # Conversation history tracking
100		│ ├── context.py # Conversation context management
101		│ └── analysis.py # Topic analysis and metrics
102		├── feedback.py # Feedback collection
103		├── issues.py # Issue tracking
104		└── function_calling.py # Semantic function calling
105		```
106
107		**Key Responsibilities:**
108		- Implement search functionality
109		- Manage commands
110		- Track conversation history and context
111		- Collect and analyze feedback
112		- Track issues
113		- Handle semantic function calling
114
115		### 4. Tool Plugins
116
117		Tool Plugins provide specific tools for agents to use.
118
119		**Files:**
120		```
121		ollamatools/plugins/tools/
122		├── __init__.py
123		├── system.py # System tools (file operations, commands)
124		├── web.py # Web tools (HTTP requests, web searches)
125		└── doc.py # Documentation tools
126		```
127
128		**Key Responsibilities:**
129		- Execute system commands
130		- Perform file operations
131		- Make web requests
132		- Generate documentation
133
134		### 5. Integration Plugins
135
136		Integration Plugins connect OllamaTools to external systems.
137
138		**Files:**
139		```
140		ollamatools/plugins/integrations/
141		├── __init__.py
142		├── matrix/
143		│ ├── __init__.py
144		│ ├── client.py # Matrix client core
145		│ ├── commands.py # Matrix-specific commands
146		│ ├── config.py # Matrix configuration
147		│ ├── search.py # Matrix search integration
148		│ ├── history.py # Matrix history tracking
149		│ └── rate_limit.py # Rate limiting for Matrix
150		└── opensearch/
151		├── __init__.py
152		├── client.py # OpenSearch client
153		└── tools.py # OpenSearch web tools
154		```
155
156		**Key Responsibilities:**
157		- Connect to Matrix chat rooms
158		- Process Matrix messages
159		- Integrate with OpenSearch
160		- Handle rate limiting
161
162		### 6. Agent Implementations
163
164		Agent Implementations provide ready-to-use agents with different feature sets.
165
166		**Files:**
167		```
168		ollamatools/agents/
169		├── __init__.py # Agent module initialization
170		├── basic.py # Basic agent implementation
171		├── modular.py # Modular agent implementation
172		└── extended.py # Extended agent implementation
173		```
174
175		**Key Responsibilities:**
176		- Implement different agent types
177		- Compose functionality from plugins
178		- Provide user-friendly interfaces
179
180		### 7. Configuration System
181
182		The Configuration System handles loading, validating, and providing access to configuration.
183
184		**Files:**
185		```
186		ollamatools/config/
187		├── __init__.py # Configuration module initialization
188		├── schemas.py # Configuration schemas
189		├── loader.py # Configuration loading/validation
190		└── defaults.py # Default configurations
191		```
192
193		**Key Responsibilities:**
194		- Define configuration schemas
195		- Load and validate configuration
196		- Provide default configurations
197		- Handle configuration merging
198
199		### 8. Command-Line Interface
200
201		The Command-Line Interface provides a way to interact with OllamaTools from the command line.
202
203		**Files:**
204		```
205		ollamatools/cli/
206		├── __init__.py # CLI module initialization
207		└── main.py # Command-line interface
208		```
209
210		**Key Responsibilities:**
211		- Parse command-line arguments
212		- Execute commands
213		- Provide interactive mode
214		- Handle configuration
215
216		## Implementation Phases
217
218		### Phase 1: Core Infrastructure (Weeks 1-2)
219
220		**Goals:**
221		- Implement the core layer
222		- Create the plugin system
223		- Set up the configuration system
224
225		**Tasks:**
226		1. Create the directory structure
227		2. Implement the base agent class
228		3. Implement the plugin system
229		4. Implement the configuration system
230		5. Write tests for core components
231
232		### Phase 2: Standard Plugins (Weeks 3-4)
233
234		**Goals:**
235		- Implement standard plugins
236		- Extract common functionality from existing code
237		- Ensure backward compatibility
238
239		**Tasks:**
240		1. Implement search plugin
241		2. Implement commands plugin
242		3. Implement conversation plugins
243		4. Implement feedback plugin
244		5. Implement issues plugin
245		6. Implement function calling plugin
246		7. Write tests for standard plugins
247
248		### Phase 3: Tool Plugins (Weeks 5-6)
249
250		**Goals:**
251		- Implement tool plugins
252		- Ensure security and permission handling
253		- Provide comprehensive documentation
254
255		**Tasks:**
256		1. Implement system tools plugin
257		2. Implement web tools plugin
258		3. Implement documentation tools plugin
259		4. Write tests for tool plugins
260		5. Document tool usage and security considerations
261
262		### Phase 4: Integration Plugins (Weeks 7-8)
263
264		**Goals:**
265		- Implement integration plugins
266		- Ensure proper error handling
267		- Provide configuration examples
268
269		**Tasks:**
270		1. Implement Matrix integration plugins
271		2. Implement OpenSearch integration plugins
272		3. Write tests for integration plugins
273		4. Create example configurations
274		5. Document integration setup and usage
275
276		### Phase 5: Agent Implementations (Weeks 9-10)
277
278		**Goals:**
279		- Implement agent classes
280		- Ensure feature parity with existing agents
281		- Provide migration guides
282
283		**Tasks:**
284		1. Implement basic agent
285		2. Implement modular agent
286		3. Implement extended agent
287		4. Write tests for agent implementations
288		5. Create migration guides for existing users
289
290		### Phase 6: Documentation and Examples (Weeks 11-12)
291
292		**Goals:**
293		- Provide comprehensive documentation
294		- Create example applications
295		- Ensure smooth migration path
296
297		**Tasks:**
298		1. Update README and documentation
299		2. Create example applications
300		3. Write migration guides
301		4. Create tutorials and guides
302		5. Update API documentation
303
304		## Migration Strategy
305
306		### For Users
307
308		1. **Basic Agent Users:**
309		- Update imports to use new package structure
310		- Minimal configuration changes required
311
312		2. **Modular Agent Users:**
313		- Update imports to use new package structure
314		- Adapt to new plugin system
315		- Follow migration guide for configuration changes
316
317		3. **Extended Agent Users:**
318		- Update imports to use new package structure
319		- Adapt to new plugin system
320		- Follow migration guide for configuration changes
321
322		4. **Matrix Integration Users:**
323		- Update imports to use new package structure
324		- Adapt to new integration plugins
325		- Follow migration guide for configuration changes
326
327		### For Developers
328
329		1. **Plugin Developers:**
330		- Learn new plugin system
331		- Update plugins to use new interfaces
332		- Follow plugin development guide
333
334		2. **Agent Developers:**
335		- Learn new agent architecture
336		- Update agents to use plugin system
337		- Follow agent development guide
338
339		3. **Integration Developers:**
340		- Learn new integration architecture
341		- Update integrations to use plugin system
342		- Follow integration development guide
343
344		## Testing Strategy
345
346		1. **Unit Tests:**
347		- Test each component in isolation
348		- Use mock objects for dependencies
349		- Ensure high code coverage
350
351		2. **Integration Tests:**
352		- Test interactions between components
353		- Verify plugin loading and initialization
354		- Test configuration loading and validation
355
356		3. **End-to-End Tests:**
357		- Test complete workflows
358		- Verify backward compatibility
359		- Test with real external systems (Matrix, OpenSearch)
360
361		4. **Performance Tests:**
362		- Measure performance impact
363		- Identify bottlenecks
364		- Optimize critical paths
365
366		## Documentation Strategy
367
368		1. **API Documentation:**
369		- Document all public interfaces
370		- Provide examples for common use cases
371		- Include type hints and docstrings
372
373		2. **User Guides:**
374		- Create guides for different user types
375		- Provide step-by-step instructions
376		- Include configuration examples
377
378		3. **Developer Guides:**
379		- Create guides for plugin development
380		- Document architecture and design decisions
381		- Provide best practices
382
383		4. **Migration Guides:**
384		- Create guides for migrating from old to new architecture
385		- Provide code examples for common migration scenarios
386		- Include troubleshooting information
387
388		## Conclusion
389
390		This simplification plan provides a comprehensive roadmap for transforming the OllamaTools project into a more modular, maintainable, and extensible system. By following this plan, we can reduce redundancy, improve code quality, and enhance the developer and user experience while preserving all existing functionality.
391
392		The plan is designed to be implemented incrementally, allowing for continuous testing and validation throughout the process. Each phase builds on the previous one, gradually transforming the codebase into the target architecture.
393
394		By the end of this process, OllamaTools will be a more cohesive, well-structured project that is easier to understand, maintain, and extend.

ollamatools-project/SIMPLIFICATION_PROMPT_TEMPLATE.md +0 -335

…	…	@@ -1,335 +0,0 @@
1		# OllamaTools Simplification Prompt Template
2
3		This document provides a template for prompts that can be used by both humans and AI assistants when working on the OllamaTools simplification project. It's designed to be used alongside the more detailed `SIMPLIFICATION_PLAN.md`.
4
5		## General Prompt Structure
6
7		When working on the OllamaTools simplification project, use the following prompt structure to ensure consistent and effective implementation:
8
9		```
10		# Task: [Brief description of the specific task]
11
12		## Context
13		- Current component: [Specify which component you're working on]
14		- Related files: [List relevant files]
15		- Dependencies: [List dependencies]
16		- Reference: [Link to relevant section in SIMPLIFICATION_PLAN.md]
17
18		## Objective
19		[Clear statement of what needs to be accomplished]
20
21		## Requirements
22		- [Requirement 1]
23		- [Requirement 2]
24		- ...
25
26		## Constraints
27		- [Constraint 1]
28		- [Constraint 2]
29		- ...
30
31		## Expected Output
32		[Description of the expected result]
33
34		## Additional Notes
35		[Any additional information that might be helpful]
36		```
37
38		## Phase-Specific Prompt Templates
39
40		### Phase 1: Core Infrastructure
41
42		```
43		# Task: Implement Core Component
44
45		## Context
46		- Current component: Core Layer - [Specific component]
47		- Related files: [List relevant files]
48		- Dependencies: [List dependencies]
49		- Reference: SIMPLIFICATION_PLAN.md - Core Layer
50
51		## Objective
52		Implement the [specific component] of the Core Layer according to the simplification plan.
53
54		## Requirements
55		- Follow the simplified architecture design
56		- Ensure backward compatibility where possible
57		- Implement clear interfaces
58		- Include comprehensive docstrings
59		- Write unit tests
60
61		## Constraints
62		- Minimize external dependencies
63		- Keep the core layer focused and minimal
64		- Ensure security best practices
65
66		## Expected Output
67		- Implemented [specific component] with tests
68		- Documentation for public interfaces
69		- Example usage
70
71		## Additional Notes
72		This component will be used by multiple other components, so clarity and stability are critical.
73		```
74
75		### Phase 2: Standard Plugins
76
77		```
78		# Task: Implement Standard Plugin
79
80		## Context
81		- Current component: Standard Plugins - [Specific plugin]
82		- Related files: [List relevant files]
83		- Dependencies: [List dependencies]
84		- Reference: SIMPLIFICATION_PLAN.md - Standard Plugins
85
86		## Objective
87		Implement the [specific plugin] according to the simplification plan.
88
89		## Requirements
90		- Implement the plugin interface
91		- Extract functionality from existing code
92		- Ensure backward compatibility
93		- Include comprehensive docstrings
94		- Write unit tests
95
96		## Constraints
97		- Keep plugins focused on a single responsibility
98		- Minimize dependencies between plugins
99		- Follow security best practices
100
101		## Expected Output
102		- Implemented plugin with tests
103		- Documentation for public interfaces
104		- Example usage
105
106		## Additional Notes
107		This plugin will be used by multiple agent implementations, so clarity and stability are important.
108		```
109
110		### Phase 3: Tool Plugins
111
112		```
113		# Task: Implement Tool Plugin
114
115		## Context
116		- Current component: Tool Plugins - [Specific tool]
117		- Related files: [List relevant files]
118		- Dependencies: [List dependencies]
119		- Reference: SIMPLIFICATION_PLAN.md - Tool Plugins
120
121		## Objective
122		Implement the [specific tool] plugin according to the simplification plan.
123
124		## Requirements
125		- Implement the tool plugin interface
126		- Extract functionality from existing code
127		- Ensure proper security and permission handling
128		- Include comprehensive docstrings
129		- Write unit tests
130
131		## Constraints
132		- Keep tools focused on a specific operation
133		- Implement proper validation and error handling
134		- Follow security best practices
135
136		## Expected Output
137		- Implemented tool plugin with tests
138		- Documentation for public interfaces
139		- Example usage
140		- Security considerations
141
142		## Additional Notes
143		Tools interact with external systems, so security and error handling are critical.
144		```
145
146		### Phase 4: Integration Plugins
147
148		```
149		# Task: Implement Integration Plugin
150
151		## Context
152		- Current component: Integration Plugins - [Specific integration]
153		- Related files: [List relevant files]
154		- Dependencies: [List dependencies]
155		- Reference: SIMPLIFICATION_PLAN.md - Integration Plugins
156
157		## Objective
158		Implement the [specific integration] plugin according to the simplification plan.
159
160		## Requirements
161		- Implement the integration plugin interface
162		- Extract functionality from existing code
163		- Ensure proper error handling
164		- Include comprehensive docstrings
165		- Write unit tests
166
167		## Constraints
168		- Keep integrations modular and self-contained
169		- Implement proper validation and error handling
170		- Follow security best practices
171
172		## Expected Output
173		- Implemented integration plugin with tests
174		- Documentation for public interfaces
175		- Example usage
176		- Configuration examples
177
178		## Additional Notes
179		Integrations connect to external systems, so error handling and configuration are critical.
180		```
181
182		### Phase 5: Agent Implementations
183
184		```
185		# Task: Implement Agent
186
187		## Context
188		- Current component: Agent Implementations - [Specific agent]
189		- Related files: [List relevant files]
190		- Dependencies: [List dependencies]
191		- Reference: SIMPLIFICATION_PLAN.md - Agent Implementations
192
193		## Objective
194		Implement the [specific agent] according to the simplification plan.
195
196		## Requirements
197		- Implement the agent using the plugin system
198		- Ensure feature parity with existing implementation
199		- Provide a clean, user-friendly interface
200		- Include comprehensive docstrings
201		- Write unit tests
202
203		## Constraints
204		- Use plugins for functionality rather than implementing directly
205		- Maintain backward compatibility where possible
206		- Follow security best practices
207
208		## Expected Output
209		- Implemented agent with tests
210		- Documentation for public interfaces
211		- Example usage
212		- Migration guide for existing users
213
214		## Additional Notes
215		Agents are the main entry point for users, so usability and documentation are critical.
216		```
217
218		### Phase 6: Documentation and Examples
219
220		```
221		# Task: Create Documentation/Examples
222
223		## Context
224		- Current component: Documentation - [Specific documentation]
225		- Related files: [List relevant files]
226		- Dependencies: [List dependencies]
227		- Reference: SIMPLIFICATION_PLAN.md - Documentation Strategy
228
229		## Objective
230		Create [specific documentation/example] according to the simplification plan.
231
232		## Requirements
233		- Clear, concise explanations
234		- Code examples for common use cases
235		- Step-by-step instructions where appropriate
236		- Consistent formatting and style
237
238		## Constraints
239		- Focus on user needs
240		- Keep examples simple and focused
241		- Follow documentation best practices
242
243		## Expected Output
244		- Completed documentation/example
245		- Any necessary supporting files
246
247		## Additional Notes
248		Documentation is critical for user adoption and developer onboarding.
249		```
250
251		## Example Prompts
252
253		### Example 1: Implementing the Base Agent Class
254
255		```
256		# Task: Implement Base Agent Class
257
258		## Context
259		- Current component: Core Layer - Agent
260		- Related files: ollamatools/core/agent.py
261		- Dependencies: core/config.py, core/errors.py, plugins/base.py
262		- Reference: SIMPLIFICATION_PLAN.md - Core Layer
263
264		## Objective
265		Implement the base Agent class that will serve as the foundation for all agent implementations.
266
267		## Requirements
268		- Implement plugin management (enabling/disabling plugins)
269		- Implement basic chat processing
270		- Implement error handling
271		- Implement lifecycle management (startup/shutdown)
272		- Include comprehensive docstrings
273		- Write unit tests
274
275		## Constraints
276		- Keep the base agent minimal and focused
277		- Avoid dependencies on specific plugins
278		- Ensure security best practices
279
280		## Expected Output
281		- Implemented base Agent class with tests
282		- Documentation for public interfaces
283		- Example usage
284
285		## Additional Notes
286		The base Agent class is the foundation of the entire system, so clarity, stability, and extensibility are critical.
287		```
288
289		### Example 2: Implementing the Search Plugin
290
291		```
292		# Task: Implement Search Plugin
293
294		## Context
295		- Current component: Standard Plugins - Search
296		- Related files: ollamatools/plugins/standard/search.py
297		- Dependencies: plugins/base.py, core/errors.py, integrations/opensearch/
298		- Reference: SIMPLIFICATION_PLAN.md - Standard Plugins
299
300		## Objective
301		Implement the Search plugin that provides search functionality for agents.
302
303		## Requirements
304		- Implement the plugin interface
305		- Extract functionality from existing search code
306		- Implement intent detection for search queries
307		- Implement search result processing
308		- Include comprehensive docstrings
309		- Write unit tests
310
311		## Constraints
312		- Keep the plugin focused on search functionality
313		- Minimize dependencies on other plugins
314		- Follow security best practices
315
316		## Expected Output
317		- Implemented Search plugin with tests
318		- Documentation for public interfaces
319		- Example usage
320
321		## Additional Notes
322		The Search plugin is used by multiple agent implementations and integrations, so clarity and stability are important.
323		```
324
325		## Using This Template
326
327		When working on the OllamaTools simplification project:
328
329		1. **For Humans**: Use this template to create specific task descriptions for yourself or other developers. Fill in the details based on the component you're working on.
330
331		2. **For AI Assistants**: When asking an AI assistant to help with implementation, use this template to provide clear context and requirements. The more specific you are, the better the assistance will be.
332
333		3. **For Both**: Always refer to the detailed `SIMPLIFICATION_PLAN.md` for comprehensive information about the architecture and implementation strategy.
334
335		By using consistent prompts, we can ensure that all contributors (human and AI) have a clear understanding of the tasks and requirements, leading to more efficient and effective implementation of the simplification plan.

ollamatools-project/SIMPLIFY_README.md +0 -85

…	…	@@ -1,85 +0,0 @@
1		# OllamaTools Simplification Project
2
3		This branch (`simplify`) contains the work-in-progress for simplifying and modularizing the OllamaTools project. The goal is to create a more maintainable, extensible, and user-friendly codebase while preserving all existing functionality.
4
5		## Overview
6
7		The OllamaTools project has evolved through several phases, resulting in a complex architecture with multiple overlapping components. This simplification project aims to:
8
9		1. **Reduce Redundancy**: Eliminate duplicate code and consolidate similar functionality
10		2. **Improve Modularity**: Create a clear, plugin-based architecture
11		3. **Simplify Interfaces**: Provide consistent, well-documented interfaces
12		4. **Enhance Maintainability**: Make the codebase easier to understand and modify
13		5. **Preserve Functionality**: Maintain all current features and capabilities
14		6. **Streamline Testing**: Improve testability through better separation of concerns
15
16		## Key Documents
17
18		This branch includes the following key documents:
19
20		1. **[SIMPLIFICATION_PLAN.md](./SIMPLIFICATION_PLAN.md)**: A comprehensive plan detailing the target architecture, component breakdown, implementation phases, migration strategy, testing strategy, and documentation strategy.
21
22		2. **[SIMPLIFICATION_PROMPT_TEMPLATE.md](./SIMPLIFICATION_PROMPT_TEMPLATE.md)**: A template for creating prompts that can be used by both humans and AI assistants when working on specific tasks within the simplification project.
23
24		## Target Architecture
25
26		The simplified architecture organizes the codebase into clear, focused components:
27
28		```
29		ollamatools/
30		├── core/ # Core functionality
31		├── plugins/ # Plugin system and implementations
32		│ ├── standard/ # Standard plugins
33		│ ├── tools/ # Tool plugins
34		│ └── integrations/ # Integration plugins
35		├── agents/ # Agent implementations
36		├── config/ # Configuration system
37		└── cli/ # Command-line interface
38		```
39
40		## Implementation Phases
41
42		The simplification project is divided into six phases:
43
44		1. **Core Infrastructure** (Weeks 1-2): Implement the core layer, plugin system, and configuration system
45		2. **Standard Plugins** (Weeks 3-4): Implement standard plugins and extract common functionality
46		3. **Tool Plugins** (Weeks 5-6): Implement tool plugins with proper security and permission handling
47		4. **Integration Plugins** (Weeks 7-8): Implement integration plugins with proper error handling
48		5. **Agent Implementations** (Weeks 9-10): Implement agent classes with feature parity
49		6. **Documentation and Examples** (Weeks 11-12): Provide comprehensive documentation and examples
50
51		## How to Contribute
52
53		If you're contributing to the simplification project:
54
55		1. **Read the Plan**: Start by reading the [SIMPLIFICATION_PLAN.md](./SIMPLIFICATION_PLAN.md) document to understand the overall architecture and strategy.
56
57		2. **Use the Prompt Template**: When working on a specific task, use the template in [SIMPLIFICATION_PROMPT_TEMPLATE.md](./SIMPLIFICATION_PROMPT_TEMPLATE.md) to create a clear task description.
58
59		3. **Follow the Architecture**: Ensure your implementation follows the target architecture and adheres to the design principles.
60
61		4. **Write Tests**: Include comprehensive tests for all new code.
62
63		5. **Document Your Work**: Add docstrings and update documentation as needed.
64
65		6. **Submit Focused PRs**: Keep pull requests focused on specific components or tasks.
66
67		## Working with AI Assistants
68
69		When working with AI assistants on this project:
70
71		1. **Provide Clear Context**: Use the prompt template to give the AI assistant clear context about the task.
72
73		2. **Reference the Plan**: Point the AI to specific sections of the simplification plan for detailed guidance.
74
75		3. **Review Carefully**: Always review AI-generated code for adherence to the architecture and best practices.
76
77		4. **Iterate**: Be prepared to iterate with the AI to refine the implementation.
78
79		## Current Status
80
81		This branch is currently in the initial planning phase. Implementation will begin with the Core Infrastructure phase.
82
83		## Questions and Feedback
84
85		If you have questions or feedback about the simplification project, please open an issue with the tag `simplification` or contact the project maintainers.

ollamatools-project/UNIFIED_PLUGIN_ARCHITECTURE.md +0 -553

…	…	@@ -1,553 +0,0 @@
1		# Unified Plugin Architecture for OllamaTools
2
3		This document proposes a unified plugin architecture for the OllamaTools project, focusing on better integration between matrix, modular, and extended agent libraries.
4
5		## Overview
6
7		The current OllamaTools project has several components that operate somewhat independently:
8
9		1. **Matrix Integration**: Provides Matrix chat client functionality
10		2. **Modular Agent**: Provides a modular agent architecture with plugins
11		3. **Extended Agent**: Extends the modular agent with additional features
12
13		These components share some functionality but are not well integrated. This proposal outlines a unified plugin architecture that would allow these components to work together more seamlessly.
14
15		## Core Design Principles
16
17		1. **Separation of Concerns**: Each plugin should have a clear, focused responsibility
18		2. **Extensibility**: The system should be easy to extend with new plugins
19		3. **Consistency**: Similar functionality should be implemented in similar ways across components
20		4. **Reusability**: Common functionality should be extracted into shared modules
21		5. **Discoverability**: Plugins should be easy to discover and use
22
23		## Proposed Architecture
24
25		### 1. Plugin Interface
26
27		At the core of the architecture is a common plugin interface:
28
29		```python
30		class OllamaToolsPlugin:
31		"""Base interface for all OllamaTools plugins."""
32
33		def __init__(self, config=None):
34		"""Initialize the plugin with optional configuration."""
35		self.config = config or {}
36
37		def initialize(self, context):
38		"""Initialize the plugin with the runtime context."""
39		pass
40
41		def shutdown(self):
42		"""Clean up resources when shutting down."""
43		pass
44
45		@property
46		def name(self):
47		"""Get the plugin name."""
48		return self.__class__.__name__
49
50		@property
51		def version(self):
52		"""Get the plugin version."""
53		return "1.0.0"
54
55		@property
56		def dependencies(self):
57		"""Get plugin dependencies."""
58		return []
59		```
60
61		### 2. Plugin Registry
62
63		A central registry manages plugin discovery, loading, and dependencies:
64
65		```python
66		class PluginRegistry:
67		"""Registry for OllamaTools plugins."""
68
69		def __init__(self):
70		"""Initialize the plugin registry."""
71		self.plugins = {}
72		self.initialized_plugins = {}
73
74		def register_plugin(self, plugin_class, config=None):
75		"""Register a plugin class."""
76		plugin_name = plugin_class.__name__
77		self.plugins[plugin_name] = {
78		"class": plugin_class,
79		"config": config
80		}
81
82		def discover_plugins(self):
83		"""Discover plugins using entry points."""
84		import pkg_resources
85		for entry_point in pkg_resources.iter_entry_points('ollamatools.plugins'):
86		plugin_class = entry_point.load()
87		self.register_plugin(plugin_class)
88
89		def initialize_plugin(self, plugin_name, context):
90		"""Initialize a plugin and its dependencies."""
91		if plugin_name in self.initialized_plugins:
92		return self.initialized_plugins[plugin_name]
93
94		if plugin_name not in self.plugins:
95		raise ValueError(f"Plugin {plugin_name} not found")
96
97		plugin_info = self.plugins[plugin_name]
98		plugin_class = plugin_info["class"]
99		plugin_config = plugin_info["config"]
100
101		# Initialize dependencies first
102		plugin_instance = plugin_class(plugin_config)
103		for dependency in plugin_instance.dependencies:
104		self.initialize_plugin(dependency, context)
105
106		# Initialize the plugin
107		plugin_instance.initialize(context)
108		self.initialized_plugins[plugin_name] = plugin_instance
109
110		return plugin_instance
111
112		def get_plugin(self, plugin_name):
113		"""Get an initialized plugin."""
114		return self.initialized_plugins.get(plugin_name)
115
116		def shutdown(self):
117		"""Shutdown all initialized plugins."""
118		for plugin_name, plugin in self.initialized_plugins.items():
119		try:
120		plugin.shutdown()
121		except Exception as e:
122		print(f"Error shutting down plugin {plugin_name}: {e}")
123
124		self.initialized_plugins = {}
125		```
126
127		### 3. Specialized Plugin Types
128
129		From the base plugin interface, we can derive specialized plugin types for different components:
130
131		#### 3.1 Agent Plugins
132
133		```python
134		class AgentPlugin(OllamaToolsPlugin):
135		"""Base class for agent plugins."""
136
137		def process_message(self, message, context):
138		"""Process a message and return a response."""
139		return None
140
141		def get_commands(self):
142		"""Get commands provided by this plugin."""
143		return {}
144
145		def on_function_call(self, function_name, args, context):
146		"""Handle a function call."""
147		return None
148		```
149
150		#### 3.2 Matrix Plugins
151
152		```python
153		class MatrixPlugin(OllamaToolsPlugin):
154		"""Base class for Matrix plugins."""
155
156		def on_message(self, room, event, client):
157		"""Handle an incoming Matrix message."""
158		return False
159
160		def get_commands(self):
161		"""Get Matrix commands provided by this plugin."""
162		return {}
163
164		def on_room_join(self, room, client):
165		"""Handle room join event."""
166		pass
167
168		def on_client_start(self, client):
169		"""Handle client start event."""
170		pass
171
172		def on_client_stop(self, client):
173		"""Handle client stop event."""
174		pass
175		```
176
177		#### 3.3 Shared Plugins
178
179		```python
180		class SharedPlugin(OllamaToolsPlugin):
181		"""Base class for plugins that can be used by multiple components."""
182
183		def get_capabilities(self):
184		"""Get capabilities provided by this plugin."""
185		return []
186		```
187
188		### 4. Plugin Implementation Examples
189
190		#### 4.1 Search Plugin
191
192		```python
193		class SearchPlugin(SharedPlugin):
194		"""Plugin for search functionality."""
195
196		def __init__(self, config=None):
197		super().__init__(config)
198		self.opensearch_config = config.get("opensearch", {})
199
200		def initialize(self, context):
201		"""Initialize the search plugin."""
202		try:
203		from ollamatools.opensearch_web_tools import get_opensearch_info
204		self.opensearch_info = get_opensearch_info(self.opensearch_config)
205		self.available = True
206		except ImportError:
207		self.available = False
208
209		def detect_factual_intent(self, message):
210		"""Detect if a message has factual intent that would benefit from search."""
211		if not self.available:
212		return False
213
214		from ollamatools.agent.modular.search import detect_factual_intent
215		return detect_factual_intent(message)
216
217		def execute_search(self, query):
218		"""Execute a search query."""
219		if not self.available:
220		return {"error": "Search functionality not available"}
221
222		from ollamatools.opensearch_web_tools import opensearch_query
223		return opensearch_query(query, self.opensearch_config)
224
225		def get_capabilities(self):
226		"""Get capabilities provided by this plugin."""
227		return ["search"] if self.available else []
228		```
229
230		#### 4.2 Conversation History Plugin
231
232		```python
233		class ConversationHistoryPlugin(SharedPlugin):
234		"""Plugin for conversation history tracking."""
235
236		def __init__(self, config=None):
237		super().__init__(config)
238		self.history_file = config.get("history_file")
239		self.max_history = config.get("max_history", 100)
240		self.conversation_history = []
241
242		def initialize(self, context):
243		"""Initialize the conversation history plugin."""
244		self._load_conversation_history()
245
246		def _load_conversation_history(self):
247		"""Load conversation history from file."""
248		import os
249		import json
250
251		if not self.history_file or not os.path.exists(self.history_file):
252		return
253
254		try:
255		with open(self.history_file, 'r') as f:
256		self.conversation_history = json.load(f)
257		except Exception as e:
258		print(f"Error loading conversation history: {e}")
259
260		def _save_conversation_history(self):
261		"""Save conversation history to file."""
262		import os
263		import json
264
265		if not self.history_file:
266		return
267
268		try:
269		os.makedirs(os.path.dirname(self.history_file), exist_ok=True)
270		with open(self.history_file, 'w') as f:
271		json.dump(self.conversation_history, f, indent=2)
272		except Exception as e:
273		print(f"Error saving conversation history: {e}")
274
275		def record_conversation(self, user_message, assistant_response, metadata=None):
276		"""Record a conversation."""
277		import time
278
279		conversation = {
280		"timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
281		"user_message": user_message,
282		"assistant_response": assistant_response,
283		"metadata": metadata or {}
284		}
285
286		self.conversation_history.append(conversation)
287
288		# Trim conversation history if it gets too long
289		if len(self.conversation_history) > self.max_history:
290		self.conversation_history = self.conversation_history[-self.max_history:]
291
292		self._save_conversation_history()
293
294		return conversation
295
296		def get_recent_conversations(self, n=5):
297		"""Get recent conversations."""
298		return self.conversation_history[-n:] if len(self.conversation_history) >= n else self.conversation_history
299
300		def analyze_topics(self):
301		"""Analyze conversation topics."""
302		# Implementation similar to existing code
303		pass
304
305		def get_metrics(self):
306		"""Get performance metrics."""
307		# Implementation similar to existing code
308		pass
309
310		def get_capabilities(self):
311		"""Get capabilities provided by this plugin."""
312		return ["conversation_history", "topic_analysis", "performance_metrics"]
313
314		def shutdown(self):
315		"""Save conversation history when shutting down."""
316		self._save_conversation_history()
317		```
318
319		#### 4.3 Matrix Agent Adapter Plugin
320
321		```python
322		class MatrixAgentAdapterPlugin(MatrixPlugin):
323		"""Plugin that adapts an agent for use with Matrix."""
324
325		def __init__(self, config=None):
326		super().__init__(config)
327		self.agent_class = config.get("agent_class")
328		self.agent_config = config.get("agent_config", {})
329		self.agent = None
330
331		def initialize(self, context):
332		"""Initialize the plugin with an agent instance."""
333		if not self.agent_class:
334		raise ValueError("Agent class not specified")
335
336		self.agent = self.agent_class(**self.agent_config)
337		self.registry = context.get("plugin_registry")
338
339		def on_message(self, room, event, client):
340		"""Handle an incoming Matrix message using the agent."""
341		message = event.body
342		sender = event.sender
343		room_id = room.room_id
344
345		# Process the message with the agent
346		response = self.agent.chat(message)
347
348		# Record the conversation if history plugin is available
349		history_plugin = self.registry.get_plugin("ConversationHistoryPlugin")
350		if history_plugin:
351		metadata = {
352		"sender": sender,
353		"room_id": room_id,
354		"timestamp": event.server_timestamp
355		}
356		history_plugin.record_conversation(message, response, metadata)
357
358		# Send the response
359		client.send_message(room_id, response)
360
361		return True
362
363		def get_commands(self):
364		"""Get commands from the agent."""
365		if hasattr(self.agent, "get_commands"):
366		return self.agent.get_commands()
367		return {}
368
369		def shutdown(self):
370		"""Clean up the agent."""
371		if hasattr(self.agent, "shutdown"):
372		self.agent.shutdown()
373		```
374
375		### 5. Integration with Existing Components
376
377		#### 5.1 Matrix Client Integration
378
379		```python
380		class EnhancedMatrixClient:
381		"""Enhanced Matrix client with plugin support."""
382
383		def __init__(self, homeserver, user_id, password, **kwargs):
384		"""Initialize the Matrix client with plugins."""
385		self.homeserver = homeserver
386		self.user_id = user_id
387		self.password = password
388		self.kwargs = kwargs
389
390		# Initialize plugin registry
391		self.plugin_registry = PluginRegistry()
392
393		# Register built-in plugins
394		self.register_builtin_plugins()
395
396		# Initialize client
397		self.client = None
398
399		def register_builtin_plugins(self):
400		"""Register built-in plugins."""
401		from ollamatools.plugins.search import SearchPlugin
402		from ollamatools.plugins.conversation_history import ConversationHistoryPlugin
403
404		self.plugin_registry.register_plugin(SearchPlugin, self.kwargs.get("search_config"))
405		self.plugin_registry.register_plugin(ConversationHistoryPlugin, self.kwargs.get("history_config"))
406
407		def register_plugin(self, plugin_class, config=None):
408		"""Register a custom plugin."""
409		self.plugin_registry.register_plugin(plugin_class, config)
410
411		async def connect(self):
412		"""Connect to the Matrix homeserver and initialize plugins."""
413		# Connect to Matrix
414		# ...
415
416		# Initialize plugins
417		context = {
418		"client": self,
419		"plugin_registry": self.plugin_registry
420		}
421
422		for plugin_name in self.plugin_registry.plugins:
423		self.plugin_registry.initialize_plugin(plugin_name, context)
424
425		return True
426
427		async def _message_callback(self, room, event):
428		"""Handle incoming messages using plugins."""
429		# Skip messages from self, etc.
430		# ...
431
432		# Try each plugin until one handles the message
433		for plugin_name, plugin in self.plugin_registry.initialized_plugins.items():
434		if isinstance(plugin, MatrixPlugin):
435		if await plugin.on_message(room, event, self):
436		return
437
438		# Default message handling if no plugin handled it
439		# ...
440
441		async def disconnect(self):
442		"""Disconnect from Matrix and shutdown plugins."""
443		# Disconnect from Matrix
444		# ...
445
446		# Shutdown plugins
447		self.plugin_registry.shutdown()
448		```
449
450		#### 5.2 Modular Agent Integration
451
452		```python
453		class PluginEnabledModularAgent(ModularOllamaAgent):
454		"""ModularOllamaAgent with plugin support."""
455
456		def __init__(self, *args, **kwargs):
457		"""Initialize the agent with plugins."""
458		super().__init__(*args, **kwargs)
459
460		# Initialize plugin registry
461		self.plugin_registry = PluginRegistry()
462
463		# Register built-in plugins
464		self.register_builtin_plugins()
465
466		# Initialize plugins
467		self.initialize_plugins()
468
469		def register_builtin_plugins(self):
470		"""Register built-in plugins."""
471		from ollamatools.plugins.search import SearchPlugin
472		from ollamatools.plugins.conversation_history import ConversationHistoryPlugin
473
474		self.plugin_registry.register_plugin(SearchPlugin, self.config.get("search_config"))
475		self.plugin_registry.register_plugin(ConversationHistoryPlugin, self.config.get("history_config"))
476
477		def register_plugin(self, plugin_class, config=None):
478		"""Register a custom plugin."""
479		self.plugin_registry.register_plugin(plugin_class, config)
480
481		def initialize_plugins(self):
482		"""Initialize all registered plugins."""
483		context = {
484		"agent": self,
485		"plugin_registry": self.plugin_registry
486		}
487
488		for plugin_name in self.plugin_registry.plugins:
489		self.plugin_registry.initialize_plugin(plugin_name, context)
490
491		def chat(self, message):
492		"""Process a chat message using plugins."""
493		# Try each plugin for message processing
494		for plugin_name, plugin in self.plugin_registry.initialized_plugins.items():
495		if isinstance(plugin, AgentPlugin):
496		response = plugin.process_message(message, {"agent": self})
497		if response:
498		return response
499
500		# Fall back to default processing
501		return super().chat(message)
502
503		def shutdown(self):
504		"""Shutdown the agent and all plugins."""
505		self.plugin_registry.shutdown()
506		super().shutdown()
507		```
508
509		## Implementation Plan
510
511		### Phase 1: Core Infrastructure
512
513		1. Create the base plugin interface and registry
514		2. Implement specialized plugin types
515		3. Create utility functions for plugin discovery and loading
516
517		### Phase 2: Shared Plugins
518
519		1. Implement the SearchPlugin
520		2. Implement the ConversationHistoryPlugin
521		3. Create tests for these plugins
522
523		### Phase 3: Matrix Integration
524
525		1. Refactor the EnhancedMatrixClient to use plugins
526		2. Implement the MatrixAgentAdapterPlugin
527		3. Update the matrix command handling to use the plugin registry
528
529		### Phase 4: Agent Integration
530
531		1. Refactor the ModularOllamaAgent to use plugins
532		2. Update the ExtendedModularAgent to leverage shared plugins
533		3. Create adapter plugins for different agent types
534
535		### Phase 5: Documentation and Examples
536
537		1. Create comprehensive documentation for the plugin system
538		2. Provide examples of creating custom plugins
539		3. Update existing examples to use the new plugin architecture
540
541		## Benefits
542
543		This unified plugin architecture provides several benefits:
544
545		1. **Reduced Code Duplication**: Common functionality is extracted into shared plugins
546		2. **Improved Extensibility**: New features can be added as plugins without modifying core code
547		3. **Better Integration**: Matrix clients and agents can share functionality through plugins
548		4. **Consistent Interfaces**: All plugins follow the same patterns and conventions
549		5. **Simplified Maintenance**: Each plugin has a clear, focused responsibility
550
551		## Conclusion
552
553		The proposed unified plugin architecture provides a solid foundation for better integration between the matrix, modular, and extended agent libraries in the OllamaTools project. By implementing this architecture, we can reduce code duplication, improve extensibility, and create a more cohesive user experience across all components.

ollamatools-project/ai_simplification_guides/00_INDEX.md +0 -103

…	…	@@ -1,103 +0,0 @@
1		# OllamaTools AI Simplification Guide - Index
2
3		This is the main index for the AI Simplification Guide, which provides detailed instructions for AI assistants to implement the OllamaTools simplification plan. The guide is broken down into multiple documents to make it more manageable and focused.
4
5		## Guide Structure
6
7		The AI Simplification Guide is organized into the following documents:
8
9		### Core Documents
10
11		1. **[00_INDEX.md](./00_INDEX.md)** - This index document
12		2. **[01_ANALYSIS.md](./01_ANALYSIS.md)** - Initial codebase analysis and setup
13		3. **[02_DIRECTORY_STRUCTURE.md](./02_DIRECTORY_STRUCTURE.md)** - Complete directory structure for the simplified project
14
15		### Phase 1: Core Infrastructure
16
17		4. **[10_CORE_OVERVIEW.md](./10_CORE_OVERVIEW.md)** - Overview of the Core Infrastructure phase
18		5. **[11_CORE_AGENT.md](./11_CORE_AGENT.md)** - Implementation of the Core Agent
19		6. **[12_CORE_CONFIG.md](./12_CORE_CONFIG.md)** - Implementation of the Configuration System
20		7. **[13_CORE_ERRORS.md](./13_CORE_ERRORS.md)** - Implementation of Error Handling
21		8. **[14_CORE_SECURITY.md](./14_CORE_SECURITY.md)** - Implementation of Security Features
22		9. **[15_CORE_UTILS.md](./15_CORE_UTILS.md)** - Implementation of Core Utilities
23
24		### Phase 2: Plugin System
25
26		10. **[20_PLUGIN_OVERVIEW.md](./20_PLUGIN_OVERVIEW.md)** - Overview of the Plugin System phase
27		11. **[21_PLUGIN_BASE.md](./21_PLUGIN_BASE.md)** - Implementation of Plugin Base Classes
28		12. **[22_PLUGIN_REGISTRY.md](./22_PLUGIN_REGISTRY.md)** - Implementation of Plugin Registry
29		13. **[23_PLUGIN_ERRORS.md](./23_PLUGIN_ERRORS.md)** - Implementation of Plugin Error Handling
30
31		### Phase 3: Standard Plugins
32
33		14. **[30_STANDARD_PLUGINS_OVERVIEW.md](./30_STANDARD_PLUGINS_OVERVIEW.md)** - Overview of Standard Plugins
34		15. **[31_SEARCH_PLUGIN.md](./31_SEARCH_PLUGIN.md)** - Implementation of Search Plugin
35		16. **[32_COMMANDS_PLUGIN.md](./32_COMMANDS_PLUGIN.md)** - Implementation of Commands Plugin
36		17. **[33_CONVERSATION_PLUGINS.md](./33_CONVERSATION_PLUGINS.md)** - Implementation of Conversation Plugins
37		18. **[34_FEEDBACK_PLUGIN.md](./34_FEEDBACK_PLUGIN.md)** - Implementation of Feedback Plugin
38		19. **[35_ISSUES_PLUGIN.md](./35_ISSUES_PLUGIN.md)** - Implementation of Issues Plugin
39		20. **[36_FUNCTION_CALLING_PLUGIN.md](./36_FUNCTION_CALLING_PLUGIN.md)** - Implementation of Function Calling Plugin
40
41		### Phase 4: Tool Plugins
42
43		21. **[40_TOOL_PLUGINS_OVERVIEW.md](./40_TOOL_PLUGINS_OVERVIEW.md)** - Overview of Tool Plugins
44		22. **[41_SYSTEM_TOOLS.md](./41_SYSTEM_TOOLS.md)** - Implementation of System Tools Plugin
45		23. **[42_WEB_TOOLS.md](./42_WEB_TOOLS.md)** - Implementation of Web Tools Plugin
46		24. **[43_DOC_TOOLS.md](./43_DOC_TOOLS.md)** - Implementation of Documentation Tools Plugin
47
48		### Phase 5: Integration Plugins
49
50		25. **[50_INTEGRATION_PLUGINS_OVERVIEW.md](./50_INTEGRATION_PLUGINS_OVERVIEW.md)** - Overview of Integration Plugins
51		26. **[51_MATRIX_INTEGRATION.md](./51_MATRIX_INTEGRATION.md)** - Implementation of Matrix Integration
52		27. **[52_OPENSEARCH_INTEGRATION.md](./52_OPENSEARCH_INTEGRATION.md)** - Implementation of OpenSearch Integration
53
54		### Phase 6: Agent Implementations
55
56		28. **[60_AGENT_IMPLEMENTATIONS_OVERVIEW.md](./60_AGENT_IMPLEMENTATIONS_OVERVIEW.md)** - Overview of Agent Implementations
57		29. **[61_BASIC_AGENT.md](./61_BASIC_AGENT.md)** - Implementation of Basic Agent
58		30. **[62_MODULAR_AGENT.md](./62_MODULAR_AGENT.md)** - Implementation of Modular Agent
59		31. **[63_EXTENDED_AGENT.md](./63_EXTENDED_AGENT.md)** - Implementation of Extended Agent
60
61		### Phase 7: Configuration System
62
63		32. **[70_CONFIG_SYSTEM_OVERVIEW.md](./70_CONFIG_SYSTEM_OVERVIEW.md)** - Overview of Configuration System
64		33. **[71_CONFIG_SCHEMAS.md](./71_CONFIG_SCHEMAS.md)** - Implementation of Configuration Schemas
65		34. **[72_CONFIG_LOADER.md](./72_CONFIG_LOADER.md)** - Implementation of Configuration Loader
66		35. **[73_CONFIG_DEFAULTS.md](./73_CONFIG_DEFAULTS.md)** - Implementation of Default Configurations
67
68		### Phase 8: Command-Line Interface
69
70		36. **[80_CLI_OVERVIEW.md](./80_CLI_OVERVIEW.md)** - Overview of Command-Line Interface
71		37. **[81_CLI_IMPLEMENTATION.md](./81_CLI_IMPLEMENTATION.md)** - Implementation of Command-Line Interface
72
73		### Migration and Testing
74
75		38. **[90_MIGRATION_OVERVIEW.md](./90_MIGRATION_OVERVIEW.md)** - Overview of Migration Strategy
76		39. **[91_MIGRATION_CORE.md](./91_MIGRATION_CORE.md)** - Migration of Core Components
77		40. **[92_MIGRATION_PLUGINS.md](./92_MIGRATION_PLUGINS.md)** - Migration of Plugins
78		41. **[93_MIGRATION_AGENTS.md](./93_MIGRATION_AGENTS.md)** - Migration of Agents
79		42. **[94_TESTING_STRATEGY.md](./94_TESTING_STRATEGY.md)** - Testing Strategy and Implementation
80
81		## How to Use This Guide
82
83		This guide is designed to be followed sequentially, starting with the analysis and directory structure, then implementing each phase in order. Each document provides detailed, step-by-step instructions with complete code examples.
84
85		For AI assistants:
86		1. Start by reading the analysis document to understand the current codebase
87		2. Review the directory structure document to understand the target architecture
88		3. Implement each phase in order, following the detailed instructions in each document
89		4. Use the migration documents to ensure backward compatibility
90		5. Follow the testing strategy to verify the implementation
91
92		For human developers:
93		1. Use this guide to understand the implementation plan
94		2. Assign specific documents or phases to AI assistants for implementation
95		3. Review the AI-generated code before committing
96		4. Provide feedback to improve the implementation
97
98		## Implementation Status
99
100		This guide is a work in progress. The following documents are currently complete:
101		- [00_INDEX.md](./00_INDEX.md) - This index document
102
103		The remaining documents will be created as the implementation progresses.

ollamatools-project/ai_simplification_guides/01_ANALYSIS.md +0 -234

…	…	@@ -1,234 +0,0 @@
1		# Initial Codebase Analysis and Setup
2
3		This document provides detailed instructions for analyzing the current OllamaTools codebase and setting up the environment for the simplification project.
4
5		## Prerequisites
6
7		Before beginning the analysis, ensure you have the following:
8
9		1. A local clone of the OllamaTools repository
10		2. Python 3.8 or higher installed
11		3. Git installed
12		4. A code editor (e.g., VS Code, PyCharm)
13
14		## Step 1: Clone the Repository and Create a Branch
15
16		```bash
17		# Clone the repository (if you haven't already)
18		git clone [repository-url]
19		cd ollamatools
20
21		# Create and checkout a new branch for the simplification project
22		git checkout -b simplify
23		```
24
25		## Step 2: Install Dependencies
26
27		```bash
28		# Install the package in development mode
29		pip install -e .
30
31		# Install development dependencies
32		pip install pytest pytest-cov black isort flake8
33		```
34
35		## Step 3: Analyze the Current Directory Structure
36
37		Run the following command to get a high-level overview of the directory structure:
38
39		```bash
40		find . -type d -not -path "*/\.*" -not -path "*/venv*" | sort
41		```
42
43		The current directory structure should look something like this:
44
45		```
46		.
47		├── build
48		│ └── lib
49		│ ├── ollamatools
50		│ │ ├── agent
51		│ │ │ ├── extended
52		│ │ │ ├── modular
53		│ │ │ ├── plugins
54		│ │ │ └── transparent
55		│ │ ├── config
56		│ │ ├── plugins
57		│ │ ├── tools
58		│ │ └── utils
59		│ └── tests
60		├── docs
61		├── examples
62		│ └── config
63		├── ollamatools
64		│ ├── agent
65		│ │ ├── extended
66		│ │ ├── modular
67		│ │ ├── plugins
68		│ │ └── transparent
69		│ ├── config
70		│ ├── plugins
71		│ ├── tools
72		│ └── utils
73		├── ollamatools.egg-info
74		├── scripts
75		└── tests
76		```
77
78		## Step 4: Identify Key Files
79
80		Identify the key files in the codebase that will need to be analyzed in detail:
81
82		```bash
83		# Find all Python files
84		find . -name "*.py" -not -path "*/\.*" -not -path "*/venv*" | sort
85		```
86
87		Pay special attention to the following files:
88
89		1. **Core Agent Files**:
90		- `ollamatools/agent.py`
91		- `ollamatools/agent/core.py`
92
93		2. **Modular Agent Files**:
94		- `ollamatools/agent/modular/agent.py`
95		- `ollamatools/agent/modular/commands.py`
96		- `ollamatools/agent/modular/search.py`
97		- `ollamatools/agent/modular/issues.py`
98		- `ollamatools/agent/modular/feedback.py`
99		- `ollamatools/agent/modular/utils.py`
100
101		3. **Extended Agent Files**:
102		- `ollamatools/agent/extended/agent.py`
103
104		4. **Plugin System Files**:
105		- `ollamatools/plugins/registry.py`
106		- `ollamatools/plugins/errors.py`
107		- `ollamatools/agent/plugins/*.py`
108
109		5. **Tool Files**:
110		- `ollamatools/tools/system_tools.py`
111		- `ollamatools/tools/web_tools.py`
112		- `ollamatools/tools/doc_tools.py`
113		- `ollamatools/tools/matrix_tools.py`
114
115		6. **Configuration Files**:
116		- `ollamatools/config/core_config.py`
117		- `ollamatools/config/plugin_config.py`
118		- `ollamatools/config/schemas.py`
119
120		## Step 5: Analyze Dependencies Between Components
121
122		Create a dependency graph to understand how the different components interact with each other:
123
124		```bash
125		# Install pipdeptree if you don't have it
126		pip install pipdeptree
127
128		# Generate a dependency graph
129		pipdeptree -p ollamatools
130		```
131
132		Additionally, analyze the import statements in the codebase to identify internal dependencies:
133
134		```bash
135		# Find all import statements
136		grep -r "^import " --include="*.py" . | grep -v "__pycache__" | sort
137		grep -r "^from " --include="*.py" . | grep -v "__pycache__" | sort
138		```
139
140		Look for circular dependencies, which will need to be resolved in the simplified architecture.
141
142		## Step 6: Analyze the Current Plugin System
143
144		The current plugin system is implemented across multiple directories:
145
146		- `ollamatools/plugins/` - Contains the plugin registry and base classes
147		- `ollamatools/agent/plugins/` - Contains agent-specific plugins
148		- `ollamatools/tools/` - Contains tool implementations that may be used by plugins
149
150		Analyze how plugins are currently registered, loaded, and used:
151
152		```bash
153		# Find all plugin-related code
154		grep -r "Plugin" --include="*.py" . | grep -v "__pycache__" | sort
155		```
156
157		## Step 7: Analyze the Matrix Integration
158
159		The Matrix integration is implemented in:
160
161		- `ollamatools/tools/matrix_tools.py`
162		- `ollamatools/tools/enhanced_matrix_tools.py`
163		- `ollamatools/tools/enhanced_matrix_core.py`
164		- `ollamatools/tools/enhanced_matrix_history.py`
165		- `ollamatools/tools/enhanced_matrix_search.py`
166
167		Analyze how the Matrix integration interacts with the agent and plugin system:
168
169		```bash
170		# Find all Matrix-related code
171		grep -r "Matrix" --include="*.py" . | grep -v "__pycache__" | sort
172		```
173
174		## Step 8: Analyze the Test Suite
175
176		Analyze the test suite to understand what functionality needs to be preserved:
177
178		```bash
179		# Find all test files
180		find . -name "test_*.py" -not -path "*/\.*" -not -path "*/venv*" | sort
181
182		# Run the tests to see what's currently working
183		pytest
184		```
185
186		## Step 9: Identify Redundancies and Overlaps
187
188		Look for redundant code and overlapping functionality:
189
190		```bash
191		# Find similar function names across files
192		grep -r "def " --include="*.py" . | grep -v "__pycache__" | sort | uniq -c | sort -nr | head -20
193		```
194
195		Common areas of redundancy include:
196		- Configuration handling
197		- Error handling
198		- Logging
199		- Permission checking
200		- Command execution
201		- File operations
202		- Web requests
203
204		## Step 10: Document Findings
205
206		Document your findings in a structured way, focusing on:
207
208		1. **Current Architecture**:
209		- Main components and their responsibilities
210		- Interactions between components
211		- Dependencies between components
212
213		2. **Redundancies and Overlaps**:
214		- Duplicate functionality
215		- Similar implementations with slight variations
216		- Overlapping responsibilities
217
218		3. **Circular Dependencies**:
219		- Components that depend on each other
220		- Import cycles that need to be broken
221
222		4. **Inconsistencies**:
223		- Inconsistent naming conventions
224		- Inconsistent error handling
225		- Inconsistent configuration handling
226
227		5. **Strengths to Preserve**:
228		- Well-designed components
229		- Clean interfaces
230		- Comprehensive test coverage
231
232		## Next Steps
233
234		After completing the analysis, proceed to the [Directory Structure](./02_DIRECTORY_STRUCTURE.md) document to understand the target architecture for the simplified project.

ollamatools-project/ai_simplification_guides/02_DIRECTORY_STRUCTURE.md +0 -412

…	…	@@ -1,412 +0,0 @@
1		# Complete Directory Structure for the Simplified Project
2
3		This document provides the complete directory structure for the simplified OllamaTools project. This structure serves as a blueprint for the implementation and should be followed closely to ensure consistency.
4
5		## Target Directory Structure
6
7		```
8		ollamatools/
9		├── core/ # Core functionality
10		│ ├── __init__.py
11		│ ├── agent.py # Base agent implementation
12		│ ├── config.py # Configuration management
13		│ ├── errors.py # Error definitions and handling
14		│ ├── security.py # Security utilities and permission system
15		│ └── utils/
16		│ ├── __init__.py
17		│ ├── logger.py # Logging utilities
18		│ ├── validators.py # Input validation (combines command/path/url validators)
19		│ └── formatting.py # Text formatting utilities
20		│
21		├── plugins/ # Plugin system and implementations
22		│ ├── __init__.py
23		│ ├── base.py # Base plugin interfaces
24		│ ├── registry.py # Plugin discovery and management
25		│ ├── errors.py # Plugin-specific error handling
26		│ ├── standard/ # Standard plugins
27		│ │ ├── __init__.py
28		│ │ ├── search.py # Search functionality
29		│ │ ├── commands.py # Command management
30		│ │ ├── conversation/
31		│ │ │ ├── __init__.py
32		│ │ │ ├── history.py # Conversation history tracking
33		│ │ │ ├── context.py # Conversation context management
34		│ │ │ └── analysis.py # Topic analysis and metrics
35		│ │ ├── feedback.py # Feedback collection
36		│ │ ├── issues.py # Issue tracking
37		│ │ └── function_calling.py # Semantic function calling
38		│ │
39		│ ├── tools/ # Tool plugins
40		│ │ ├── __init__.py
41		│ │ ├── system.py # System tools (file operations, commands)
42		│ │ ├── web.py # Web tools (HTTP requests, web searches)
43		│ │ └── doc.py # Documentation tools
44		│ │
45		│ └── integrations/ # Integration plugins
46		│ ├── __init__.py
47		│ ├── matrix/
48		│ │ ├── __init__.py
49		│ │ ├── client.py # Matrix client core
50		│ │ ├── commands.py # Matrix-specific commands
51		│ │ ├── config.py # Matrix configuration
52		│ │ ├── search.py # Matrix search integration
53		│ │ ├── history.py # Matrix history tracking
54		│ │ └── rate_limit.py # Rate limiting for Matrix
55		│ └── opensearch/
56		│ ├── __init__.py
57		│ ├── client.py # OpenSearch client
58		│ └── tools.py # OpenSearch web tools
59		│
60		├── agents/ # Agent implementations
61		│ ├── __init__.py
62		│ ├── basic.py # Basic agent implementation
63		│ ├── modular.py # Modular agent implementation
64		│ └── extended.py # Extended agent implementation
65		│
66		├── config/ # Configuration system
67		│ ├── __init__.py
68		│ ├── schemas.py # Configuration schemas
69		│ ├── loader.py # Configuration loading/validation
70		│ └── defaults.py # Default configurations
71		│
72		├── cli/ # Command-line interface
73		│ ├── __init__.py
74		│ └── main.py # Command-line interface
75		│
76		├── __init__.py # Package initialization
77		└── version.py # Version information
78		```
79
80		## File Descriptions
81
82		### Core Module
83
84		#### `core/__init__.py`
85		- Exports core components
86		- Defines version information
87		- Provides convenience imports
88
89		#### `core/agent.py`
90		- Defines the base `Agent` class
91		- Implements core agent functionality
92		- Provides plugin management
93		- Handles basic chat processing
94		- Implements error handling
95		- Manages lifecycle (startup/shutdown)
96
97		#### `core/config.py`
98		- Implements configuration loading and saving
99		- Provides configuration validation
100		- Handles configuration merging
101		- Defines default configuration values
102
103		#### `core/errors.py`
104		- Defines error types
105		- Implements error handling utilities
106		- Provides error formatting
107
108		#### `core/security.py`
109		- Implements permission system
110		- Provides security validation
111		- Handles secure defaults
112
113		#### `core/utils/__init__.py`
114		- Exports utility functions
115		- Provides convenience imports
116
117		#### `core/utils/logger.py`
118		- Implements logging configuration
119		- Provides logging utilities
120		- Handles log formatting
121
122		#### `core/utils/validators.py`
123		- Implements input validation
124		- Combines command/path/url validators
125		- Provides security checks
126
127		#### `core/utils/formatting.py`
128		- Implements text formatting utilities
129		- Provides markdown generation
130		- Handles output formatting
131
132		### Plugin System
133
134		#### `plugins/__init__.py`
135		- Exports plugin components
136		- Provides convenience imports
137
138		#### `plugins/base.py`
139		- Defines base plugin interfaces
140		- Implements plugin lifecycle methods
141		- Provides plugin metadata
142
143		#### `plugins/registry.py`
144		- Implements plugin discovery
145		- Handles plugin registration
146		- Manages plugin dependencies
147		- Provides plugin initialization
148
149		#### `plugins/errors.py`
150		- Defines plugin-specific error types
151		- Implements plugin error handling
152		- Provides plugin error reporting
153
154		### Standard Plugins
155
156		#### `plugins/standard/__init__.py`
157		- Exports standard plugins
158		- Provides convenience imports
159
160		#### `plugins/standard/search.py`
161		- Implements search functionality
162		- Provides intent detection
163		- Handles search result processing
164
165		#### `plugins/standard/commands.py`
166		- Implements command management
167		- Provides command registration
168		- Handles command execution
169
170		#### `plugins/standard/conversation/__init__.py`
171		- Exports conversation plugins
172		- Provides convenience imports
173
174		#### `plugins/standard/conversation/history.py`
175		- Implements conversation history tracking
176		- Provides history storage and retrieval
177		- Handles history summarization
178
179		#### `plugins/standard/conversation/context.py`
180		- Implements conversation context management
181		- Provides context tracking per user/room
182		- Handles context window management
183
184		#### `plugins/standard/conversation/analysis.py`
185		- Implements topic analysis
186		- Provides performance metrics
187		- Handles usage statistics
188
189		#### `plugins/standard/feedback.py`
190		- Implements feedback collection
191		- Provides feedback analysis
192		- Handles feedback-based improvements
193
194		#### `plugins/standard/issues.py`
195		- Implements issue tracking
196		- Provides issue creation and management
197		- Handles issue prioritization
198
199		#### `plugins/standard/function_calling.py`
200		- Implements semantic function calling
201		- Provides function detection
202		- Handles function parameter extraction
203
204		### Tool Plugins
205
206		#### `plugins/tools/__init__.py`
207		- Exports tool plugins
208		- Provides convenience imports
209
210		#### `plugins/tools/system.py`
211		- Implements system command execution
212		- Provides file operations
213		- Handles directory operations
214
215		#### `plugins/tools/web.py`
216		- Implements HTTP requests
217		- Provides web searches
218		- Handles API integrations
219
220		#### `plugins/tools/doc.py`
221		- Implements markdown generation
222		- Provides document summarization
223		- Handles context extraction
224
225		### Integration Plugins
226
227		#### `plugins/integrations/__init__.py`
228		- Exports integration plugins
229		- Provides convenience imports
230
231		#### `plugins/integrations/matrix/__init__.py`
232		- Exports Matrix integration components
233		- Provides convenience imports
234
235		#### `plugins/integrations/matrix/client.py`
236		- Implements Matrix client
237		- Provides connection management
238		- Handles message sending/receiving
239
240		#### `plugins/integrations/matrix/commands.py`
241		- Implements Matrix-specific commands
242		- Provides command parsing
243		- Handles command execution
244
245		#### `plugins/integrations/matrix/config.py`
246		- Implements Matrix configuration
247		- Provides configuration loading/saving
248		- Handles configuration validation
249
250		#### `plugins/integrations/matrix/search.py`
251		- Implements Matrix search integration
252		- Provides search command handling
253		- Handles search result formatting
254
255		#### `plugins/integrations/matrix/history.py`
256		- Implements Matrix history tracking
257		- Provides history storage
258		- Handles history command handling
259
260		#### `plugins/integrations/matrix/rate_limit.py`
261		- Implements rate limiting for Matrix
262		- Provides backoff strategies
263		- Handles error handling for rate limits
264
265		#### `plugins/integrations/opensearch/__init__.py`
266		- Exports OpenSearch integration components
267		- Provides convenience imports
268
269		#### `plugins/integrations/opensearch/client.py`
270		- Implements OpenSearch client
271		- Provides connection management
272		- Handles query execution
273
274		#### `plugins/integrations/opensearch/tools.py`
275		- Implements OpenSearch web tools
276		- Provides search query formatting
277		- Handles result processing
278
279		### Agent Implementations
280
281		#### `agents/__init__.py`
282		- Exports agent implementations
283		- Provides convenience imports
284
285		#### `agents/basic.py`
286		- Implements basic agent
287		- Provides minimal feature set
288		- Handles simple chat processing
289
290		#### `agents/modular.py`
291		- Implements modular agent
292		- Provides plugin-based architecture
293		- Handles advanced chat processing
294
295		#### `agents/extended.py`
296		- Implements extended agent
297		- Provides advanced features
298		- Handles comprehensive functionality
299
300		### Configuration System
301
302		#### `config/__init__.py`
303		- Exports configuration components
304		- Provides convenience imports
305
306		#### `config/schemas.py`
307		- Implements configuration schemas
308		- Provides schema validation
309		- Handles schema versioning
310
311		#### `config/loader.py`
312		- Implements configuration loading
313		- Provides environment variable integration
314		- Handles configuration merging
315
316		#### `config/defaults.py`
317		- Implements default configurations
318		- Provides sensible defaults
319		- Handles documentation of configuration options
320
321		### Command-Line Interface
322
323		#### `cli/__init__.py`
324		- Exports CLI components
325		- Provides convenience imports
326
327		#### `cli/main.py`
328		- Implements command-line interface
329		- Provides command parsing
330		- Handles interactive mode
331
332		### Package Root
333
334		#### `__init__.py`
335		- Exports main components
336		- Provides convenience imports
337		- Handles package initialization
338
339		#### `version.py`
340		- Defines version information
341		- Provides version checking
342		- Handles compatibility information
343
344		## Implementation Strategy
345
346		When implementing the simplified project, follow these guidelines:
347
348		1. **Create the directory structure first**:
349		```bash
350		mkdir -p ollamatools/core/utils
351		mkdir -p ollamatools/plugins/standard/conversation
352		mkdir -p ollamatools/plugins/tools
353		mkdir -p ollamatools/plugins/integrations/matrix
354		mkdir -p ollamatools/plugins/integrations/opensearch
355		mkdir -p ollamatools/agents
356		mkdir -p ollamatools/config
357		mkdir -p ollamatools/cli
358		```
359
360		2. **Create empty `__init__.py` files**:
361		```bash
362		touch ollamatools/__init__.py
363		touch ollamatools/core/__init__.py
364		touch ollamatools/core/utils/__init__.py
365		touch ollamatools/plugins/__init__.py
366		touch ollamatools/plugins/standard/__init__.py
367		touch ollamatools/plugins/standard/conversation/__init__.py
368		touch ollamatools/plugins/tools/__init__.py
369		touch ollamatools/plugins/integrations/__init__.py
370		touch ollamatools/plugins/integrations/matrix/__init__.py
371		touch ollamatools/plugins/integrations/opensearch/__init__.py
372		touch ollamatools/agents/__init__.py
373		touch ollamatools/config/__init__.py
374		touch ollamatools/cli/__init__.py
375		```
376
377		3. **Implement core components first**:
378		- Start with `core/errors.py` as other components depend on it
379		- Then implement `core/utils/` modules
380		- Next, implement `core/config.py` and `core/security.py`
381		- Finally, implement `core/agent.py`
382
383		4. **Implement the plugin system**:
384		- Start with `plugins/errors.py`
385		- Then implement `plugins/base.py`
386		- Finally, implement `plugins/registry.py`
387
388		5. **Implement plugins in order of dependency**:
389		- Start with tool plugins
390		- Then implement standard plugins
391		- Finally, implement integration plugins
392
393		6. **Implement agent implementations**:
394		- Start with the basic agent
395		- Then implement the modular agent
396		- Finally, implement the extended agent
397
398		7. **Implement the configuration system**:
399		- Start with `config/schemas.py`
400		- Then implement `config/defaults.py`
401		- Finally, implement `config/loader.py`
402
403		8. **Implement the command-line interface**:
404		- Implement `cli/main.py`
405
406		9. **Implement package root files**:
407		- Implement `__init__.py`
408		- Implement `version.py`
409
410		## Next Steps
411
412		After understanding the target directory structure, proceed to the [Core Overview](./10_CORE_OVERVIEW.md) document to begin implementing the core infrastructure.

ollamatools-project/ai_simplification_guides/10_CORE_OVERVIEW.md +0 -82

…	…	@@ -1,82 +0,0 @@
1		# Core Infrastructure Overview
2
3		This document provides an overview of the Core Infrastructure phase of the OllamaTools simplification project. The Core Infrastructure forms the foundation of the simplified architecture and includes the essential components that all other parts of the system depend on.
4
5		## Core Components
6
7		The Core Infrastructure consists of the following components:
8
9		1. **Error Handling System** (`core/errors.py`)
10		- Base error classes
11		- Error formatting utilities
12		- Error handling functions
13
14		2. **Utility Functions** (`core/utils/`)
15		- Logging utilities (`logger.py`)
16		- Input validation (`validators.py`)
17		- Text formatting (`formatting.py`)
18
19		3. **Configuration Management** (`core/config.py`)
20		- Configuration loading and saving
21		- Configuration validation
22		- Default configuration values
23
24		4. **Security System** (`core/security.py`)
25		- Permission management
26		- Security validation
27		- Secure defaults
28
29		5. **Base Agent** (`core/agent.py`)
30		- Plugin management
31		- Basic chat processing
32		- Lifecycle management
33
34		## Implementation Order
35
36		The Core Infrastructure components should be implemented in the following order:
37
38		1. **Error Handling System** - This is the most fundamental component that other components depend on.
39		2. **Utility Functions** - These provide common functionality used by other components.
40		3. **Configuration Management** - This provides configuration handling used by other components.
41		4. **Security System** - This builds on the error handling and configuration systems.
42		5. **Base Agent** - This brings together all the other components.
43
44		## Dependencies
45
46		The Core Infrastructure components have the following dependencies:
47
48		- **Error Handling System**: No dependencies
49		- **Utility Functions**: Depends on Error Handling System
50		- **Configuration Management**: Depends on Error Handling System
51		- **Security System**: Depends on Error Handling System and Configuration Management
52		- **Base Agent**: Depends on all other Core Infrastructure components
53
54		## Implementation Guidelines
55
56		When implementing the Core Infrastructure components, follow these guidelines:
57
58		1. **Keep it minimal**: The Core Infrastructure should contain only essential functionality that is needed by multiple other components.
59
60		2. **Avoid circular dependencies**: Ensure that dependencies flow in one direction, from higher-level components to lower-level components.
61
62		3. **Use clear interfaces**: Define clear interfaces for each component to make them easy to use and understand.
63
64		4. **Document thoroughly**: Include comprehensive docstrings for all classes, methods, and functions.
65
66		5. **Write tests**: Include unit tests for all components to ensure they work correctly.
67
68		6. **Consider backward compatibility**: Where possible, maintain backward compatibility with the existing codebase to ease migration.
69
70		## Implementation Details
71
72		Each Core Infrastructure component is detailed in its own document:
73
74		- [Core Agent](./11_CORE_AGENT.md)
75		- [Core Configuration](./12_CORE_CONFIG.md)
76		- [Core Errors](./13_CORE_ERRORS.md)
77		- [Core Security](./14_CORE_SECURITY.md)
78		- [Core Utilities](./15_CORE_UTILS.md)
79
80		## Next Steps
81
82		After reviewing this overview, proceed to the [Core Errors](./13_CORE_ERRORS.md) document to begin implementing the Error Handling System, which is the foundation of the Core Infrastructure.

ollamatools-project/command_validator.py +0 -326

…	…	@@ -1,326 +0,0 @@
1		"""
2		Command validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing shell commands
5		to prevent command injection and other security vulnerabilities.
6		"""
7
8		import os
9		import re
10		import shlex
11		import logging
12		import pathlib
13		from typing import List, Optional, Dict, Set, Tuple, Union
14
15		# Set up logging
16		logger = logging.getLogger(__name__)
17
18		# Default allowlist of commands
19		DEFAULT_ALLOWLIST = {
20		# File operations
21		'ls', 'dir', 'find', 'grep', 'cat', 'less', 'head', 'tail',
22		# Directory operations
23		'pwd', 'cd', 'mkdir', 'rmdir',
24		# System information
25		'uname', 'hostname', 'whoami', 'id', 'ps', 'top', 'df', 'du', 'free',
26		# Network utilities
27		'ping', 'traceroute', 'curl', 'wget', 'netstat', 'dig', 'nslookup',
28		# Text processing
29		'echo', 'printf', 'sed', 'awk', 'cut', 'sort', 'uniq', 'wc',
30		# File manipulation
31		'cp', 'mv', 'rm', 'touch', 'chmod', 'chown', 'chgrp', 'ln',
32		# Compression
33		'tar', 'gzip', 'gunzip', 'zip', 'unzip',
34		# Package management
35		'apt-get', 'apt', 'yum', 'dnf', 'brew',
36		# Process control
37		'kill', 'pkill', 'pgrep',
38		# Admin
39		'sudo',
40		}
41
42		# Commands that require special validation
43		HIGH_RISK_COMMANDS = {
44		'rm', 'mv', 'cp', # File operations that can cause data loss
45		'chmod', 'chown', 'chgrp', # Permission changes
46		'sudo', # Privilege escalation
47		'wget', 'curl', # Network operations
48		}
49
50		# Paths that should be protected (not allowed to be modified)
51		PROTECTED_PATHS = {
52		'/etc', '/bin', '/sbin', '/usr/bin', '/usr/sbin',
53		'/boot', '/lib', '/lib64', '/usr/lib', '/usr/lib64',
54		'/sys', '/proc', '/dev'
55		}
56
57		# Regular expression patterns for dangerous argument patterns
58		DANGEROUS_ARG_PATTERNS = [
59		r'[|;&`$()]', # Shell metacharacters
60		r'>(>)?[^>]*', # Redirections
61		r'<(<)?[^<]*', # Input redirections
62		r'\$\([^)]*\)', # Command substitution
63		r'`[^`]*`', # Backtick command substitution
64		r'\$\{[^}]*\}', # Variable expansion
65		r'\.\.', # Path traversal
66		r'~[^/]*', # Home directory reference
67		r'\\\\', # Double backslash
68		]
69
70		class CommandValidationError(Exception):
71		"""Exception raised for command validation errors."""
72		pass
73
74
75		def is_command_in_allowlist(command: str, custom_allowlist: Optional[Set[str]] = None) -> bool:
76		"""
77		Check if a command is in the allowlist.
78
79		Args:
80		command: Command to check
81		custom_allowlist: Optional custom allowlist to use instead of the default
82
83		Returns:
84		bool: True if command is in allowlist, False otherwise
85		"""
86		allowlist = custom_allowlist if custom_allowlist is not None else DEFAULT_ALLOWLIST
87
88		# Strip path information for the check
89		cmd_name = os.path.basename(command)
90
91		return cmd_name in allowlist
92
93
94		def validate_command_path(command: str) -> str:
95		"""
96		Validate that a command's path is safe.
97
98		Args:
99		command: Command to validate
100
101		Returns:
102		str: Validated command path
103
104		Raises:
105		CommandValidationError: If the command path is not valid
106		"""
107		# Check for absolute path
108		if os.path.isabs(command):
109		cmd_path = pathlib.Path(command).resolve()
110		cmd_name = os.path.basename(command)
111
112		# Ensure the command exists
113		if not cmd_path.exists():
114		raise CommandValidationError(f"Command '{command}' not found")
115
116		# Ensure the path is to a file not a directory
117		if not cmd_path.is_file():
118		raise CommandValidationError(f"Command path '{command}' is not a file")
119
120		# Check if command in a standard location
121		std_paths = ['/bin', '/usr/bin', '/usr/local/bin', '/sbin', '/usr/sbin']
122		parent_path = str(cmd_path.parent)
123		if not any(parent_path.startswith(p) for p in std_paths):
124		logger.warning(f"Command '{command}' is outside standard paths: {parent_path}")
125
126		# Additional permission check
127		if not os.access(cmd_path, os.X_OK):
128		raise CommandValidationError(f"Command '{command}' is not executable")
129
130		return str(cmd_path)
131		else:
132		# For non-absolute paths, we'll rely on the allowlist check
133		return command
134
135
136		def validate_command_args(args: List[str], command: str) -> List[str]:
137		"""
138		Validate command arguments for security issues.
139
140		Args:
141		args: Command arguments to validate
142		command: The command these args apply to (for context-specific checks)
143
144		Returns:
145		List[str]: Validated arguments
146
147		Raises:
148		CommandValidationError: If any arguments are not valid
149		"""
150		validated_args = []
151
152		# Check if command is high risk
153		is_high_risk = os.path.basename(command) in HIGH_RISK_COMMANDS
154
155		for arg in args:
156		# Skip empty args
157		if not arg:
158		continue
159
160		# Check for dangerous patterns
161		for pattern in DANGEROUS_ARG_PATTERNS:
162		if re.search(pattern, arg):
163		raise CommandValidationError(
164		f"Argument '{arg}' contains dangerous pattern matching '{pattern}'"
165		)
166
167		# Special checks for specific commands
168		cmd_name = os.path.basename(command)
169
170		# Special checks for rm command
171		if cmd_name == 'rm' and (arg == '-rf' or arg == '-fr' or arg == '-r' or arg == '-f'):
172		# These arguments need special attention with protected paths
173		if '-r' in arg or '-f' in arg:
174		logger.warning(f"High-risk argument '{arg}' detected for '{command}'")
175
176		# Check file paths in arguments for protected paths
177		if is_high_risk and arg.startswith('/'):
178		arg_path = pathlib.Path(arg).resolve()
179		arg_path_str = str(arg_path)
180
181		# Check against protected paths
182		for protected in PROTECTED_PATHS:
183		if arg_path_str.startswith(protected):
184		raise CommandValidationError(
185		f"Argument '{arg}' references protected path '{protected}'"
186		)
187
188		validated_args.append(arg)
189
190		return validated_args
191
192
193		def sanitize_command(command: str) -> str:
194		"""
195		Sanitize a command string to prevent command injection.
196
197		Args:
198		command: Command string to sanitize
199
200		Returns:
201		str: Sanitized command string
202		"""
203		# Convert to string and trim
204		if not isinstance(command, str):
205		command = str(command)
206
207		sanitized = command.strip()
208
209		# Remove any null bytes or control characters
210		sanitized = re.sub(r'[\x00-\x1F\x7F]', '', sanitized)
211
212		# Remove shell metacharacters
213		dangerous_chars = r'[;&|`$()"\'\\]'
214		sanitized = re.sub(dangerous_chars, '', sanitized)
215
216		return sanitized
217
218
219		def validate_and_sanitize_command(
220		command: str,
221		args: Optional[List[str]] = None,
222		custom_allowlist: Optional[Set[str]] = None
223		) -> Tuple[str, List[str]]:
224		"""
225		Validate and sanitize a command and its arguments.
226
227		Args:
228		command: Command to validate and sanitize
229		args: Optional command arguments
230		custom_allowlist: Optional custom allowlist
231
232		Returns:
233		tuple: (sanitized_command, sanitized_args)
234
235		Raises:
236		CommandValidationError: If the command or arguments are not valid
237		"""
238		# Handle None args
239		if args is None:
240		args = []
241
242		# Handle case where command contains both command and arguments as a string
243		if isinstance(command, str) and ' ' in command and args is None:
244		try:
245		# Split using shlex to handle quotes properly
246		command_parts = shlex.split(command)
247		command = command_parts[0]
248		args = command_parts[1:] if len(command_parts) > 1 else []
249		logger.debug(f"Split command into cmd='{command}', args={args}")
250		except ValueError as e:
251		raise CommandValidationError(f"Invalid command syntax: {str(e)}")
252
253		# Check if command is in allowlist
254		if not is_command_in_allowlist(command, custom_allowlist):
255		raise CommandValidationError(
256		f"Command '{command}' not in allowed command list"
257		)
258
259		# Validate command path
260		validated_command = validate_command_path(command)
261
262		# Validate and sanitize args
263		validated_args = validate_command_args(args, validated_command)
264
265		return validated_command, validated_args
266
267
268		def is_sudo_command(command: str, args: Optional[List[str]] = None) -> bool:
269		"""
270		Check if a command uses sudo.
271
272		Args:
273		command: Command to check
274		args: Optional command arguments
275
276		Returns:
277		bool: True if the command uses sudo, False otherwise
278		"""
279		if args is None:
280		args = []
281
282		# Check if command is sudo directly
283		if os.path.basename(command) == 'sudo':
284		return True
285
286		# Check if command path contains sudo
287		if 'sudo' in command.split(os.path.sep):
288		return True
289
290		# Check if sudo appears in the full command string
291		full_command = f"{command} {' '.join(args)}"
292		if full_command.lstrip().startswith('sudo '):
293		return True
294
295		return False
296
297
298		def get_secure_env() -> Dict[str, str]:
299		"""
300		Get a sanitized environment dictionary for command execution.
301
302		Returns:
303		dict: Sanitized environment variables
304		"""
305		# Start with a minimal environment
306		secure_env = {
307		'PATH': '/usr/local/bin:/usr/bin:/bin',
308		'LANG': os.environ.get('LANG', 'C.UTF-8'),
309		'LC_ALL': os.environ.get('LC_ALL', 'C.UTF-8'),
310		'TERM': os.environ.get('TERM', 'xterm'),
311		'HOME': os.environ.get('HOME', '/tmp'),
312		'USER': os.environ.get('USER', 'nobody'),
313		}
314
315		# Copy allowed environment variables
316		allowed_vars = {
317		'DISPLAY', 'EDITOR', 'PAGER', 'TZ',
318		'HTTP_PROXY', 'HTTPS_PROXY', 'NO_PROXY',
319		'http_proxy', 'https_proxy', 'no_proxy'
320		}
321
322		for var in allowed_vars:
323		if var in os.environ:
324		secure_env[var] = os.environ[var]
325
326		return secure_env

ollamatools-project/copy_refactor_file.sh +0 -81

…	…	@@ -1,81 +0,0 @@
1		#!/bin/bash
2
3		# Script to help copy files from the original codebase to the new structure
4		# This script copies a file from the original codebase to the new structure
5		# and opens it in the editor for refactoring
6
7		# Define the parent directory and project directories
8		PARENT_DIR=".."
9		ORIGINAL_DIR="$PARENT_DIR/ollamatools-project"
10		NEW_DIR="$PARENT_DIR/ollamatools-new"
11
12		# Function to display usage information
13		usage() {
14		echo "Usage: $0 <source_file> <destination_file>"
15		echo ""
16		echo "Arguments:"
17		echo " <source_file> Path to the source file in the original codebase (relative to $ORIGINAL_DIR)"
18		echo " <destination_file> Path to the destination file in the new codebase (relative to $NEW_DIR)"
19		echo ""
20		echo "Example:"
21		echo " $0 ollamatools/agent.py ollamatools/core/agent.py"
22		echo ""
23		echo "This will copy ollamatools/agent.py from the original codebase to ollamatools/core/agent.py in the new codebase"
24		echo "and open it in the editor for refactoring."
25		exit 1
26		}
27
28		# Check if the correct number of arguments is provided
29		if [ $# -ne 2 ]; then
30		usage
31		fi
32
33		SOURCE_FILE="$1"
34		DEST_FILE="$2"
35
36		# Check if the source file exists
37		if [ ! -f "$ORIGINAL_DIR/$SOURCE_FILE" ]; then
38		echo "Error: Source file $ORIGINAL_DIR/$SOURCE_FILE does not exist."
39		exit 1
40		fi
41
42		# Check if the destination directory exists, create it if it doesn't
43		DEST_DIR=$(dirname "$NEW_DIR/$DEST_FILE")
44		if [ ! -d "$DEST_DIR" ]; then
45		echo "Creating destination directory: $DEST_DIR"
46		mkdir -p "$DEST_DIR"
47		fi
48
49		# Copy the file
50		echo "Copying $ORIGINAL_DIR/$SOURCE_FILE to $NEW_DIR/$DEST_FILE"
51		cp "$ORIGINAL_DIR/$SOURCE_FILE" "$NEW_DIR/$DEST_FILE"
52
53		# Check if the copy was successful
54		if [ $? -ne 0 ]; then
55		echo "Error: Failed to copy file."
56		exit 1
57		fi
58
59		# Open the file in the editor
60		if command -v code &> /dev/null; then
61		echo "Opening file in VS Code..."
62		code "$NEW_DIR/$DEST_FILE"
63		elif command -v vim &> /dev/null; then
64		echo "Opening file in Vim..."
65		vim "$NEW_DIR/$DEST_FILE"
66		elif command -v nano &> /dev/null; then
67		echo "Opening file in Nano..."
68		nano "$NEW_DIR/$DEST_FILE"
69		else
70		echo "No suitable editor found. File has been copied but not opened."
71		fi
72
73		echo ""
74		echo "File copied successfully!"
75		echo "You can now refactor the file according to the new architecture."
76		echo ""
77		echo "Remember to:"
78		echo "1. Update imports to use the new package structure"
79		echo "2. Adapt the code to the new architecture"
80		echo "3. Remove any redundant or deprecated functionality"
81		echo "4. Add proper documentation"

ollamatools-project/docs/matrix_integration.md +0 -546

…	…	@@ -1,546 +0,0 @@
1		# Matrix Integration for OllamaTools
2
3		This document provides information on how to use the Matrix integration in OllamaTools to connect your Ollama-powered applications to Matrix chat rooms.
4
5		There are two Matrix client implementations available:
6		1. **Basic Matrix Client**: The standard Matrix client with basic functionality
7		2. **Enhanced Matrix Client**: An extended client with advanced features from modular and extended agents
8
9		## Overview
10
11		The Matrix integration allows you to:
12
13		- Connect to a Matrix homeserver
14		- Join Matrix rooms
15		- Send messages to rooms
16		- Listen for incoming messages
17		- Process messages with Ollama and send responses
18		- Switch between different Ollama models
19		- Maintain conversation context for each user
20
21		## Requirements
22
23		To use the Matrix integration, you need to install the following dependencies:
24
25		```bash
26		pip install matrix-nio[e2e] # For Matrix protocol support with end-to-end encryption
27		pip install ollama # For Ollama API access
28		pip install pyyaml # For configuration handling
29		```
30
31		## Getting Started
32
33		### 1. Import the Matrix Tools
34
35		For the basic Matrix client:
36
37		```python
38		from ollamatools.tools.matrix_tools import (
39		MatrixClient,
40		connect_to_matrix,
41		join_matrix_room,
42		send_matrix_message,
43		listen_for_matrix_messages
44		)
45		```
46
47		For the enhanced Matrix client:
48
49		```python
50		from ollamatools.tools.enhanced_matrix_tools import (
51		EnhancedMatrixClient,
52		connect_enhanced_matrix,
53		join_enhanced_matrix_room,
54		send_enhanced_matrix_message,
55		listen_for_enhanced_matrix_messages,
56		create_enhanced_matrix_config,
57		disconnect_enhanced_matrix,
58		process_with_enhanced_ollama,
59		MODULAR_AGENT_AVAILABLE,
60		OPENSEARCH_AVAILABLE
61		)
62		```
63
64		### 2. Create a Matrix Client
65
66		You can create a Matrix client in two ways:
67
68		#### Option 1: Using the MatrixClient class directly
69
70		```python
71		from ollamatools.tools.matrix_tools import MatrixClient
72
73		client = MatrixClient(
74		homeserver="https://matrix.example.org",
75		user_id="@user:example.org",
76		password="your-password",
77		device_id="ollamatools-matrix", # Optional, for E2E encryption
78		ollama_host="localhost:11434", # Optional, defaults to localhost:11434
79		ollama_model="llama3:latest", # Optional, defaults to llama3:latest
80		ollama_prompt="You are a helpful assistant." # Optional
81		)
82
83		# Connect to the Matrix homeserver
84		import asyncio
85		async def connect():
86		success = await client.connect()
87		if success:
88		print("Connected to Matrix homeserver")
89		else:
90		print("Failed to connect to Matrix homeserver")
91
92		asyncio.run(connect())
93		```
94
95		#### Option 2: Using the connect_to_matrix function
96
97		```python
98		from ollamatools.tools.matrix_tools import connect_to_matrix
99
100		client = connect_to_matrix(
101		homeserver="https://matrix.example.org",
102		user_id="@user:example.org",
103		password="your-password",
104		device_id="ollamatools-matrix" # Optional
105		)
106
107		if client:
108		print("Connected to Matrix homeserver")
109		else:
110		print("Failed to connect to Matrix homeserver")
111		```
112
113		### 3. Join a Matrix Room
114
115		```python
116		# Using the MatrixClient class
117		async def join_room_async():
118		success = await client.join_room("#room:example.org")
119		if success:
120		print("Joined room")
121		else:
122		print("Failed to join room")
123
124		asyncio.run(join_room_async())
125
126		# Or using the join_matrix_room function
127		from ollamatools.tools.matrix_tools import join_matrix_room
128
129		success = join_matrix_room(client, "#room:example.org")
130		if success:
131		print("Joined room")
132		else:
133		print("Failed to join room")
134		```
135
136		### 4. Send a Message to a Room
137
138		```python
139		# Using the MatrixClient class
140		async def send_message_async():
141		success = await client.send_message("#room:example.org", "Hello from OllamaTools!")
142		if success:
143		print("Message sent")
144		else:
145		print("Failed to send message")
146
147		asyncio.run(send_message_async())
148
149		# Or using the send_matrix_message function
150		from ollamatools.tools.matrix_tools import send_matrix_message
151
152		success = send_matrix_message(client, "#room:example.org", "Hello from OllamaTools!")
153		if success:
154		print("Message sent")
155		else:
156		print("Failed to send message")
157		```
158
159		### 5. Listen for Messages
160
161		By default, the Matrix client will process incoming messages with Ollama and send the responses back to the room. You can start listening for messages like this:
162
163		```python
164		# Using the MatrixClient class
165		async def listen_async():
166		await client.listen()
167
168		asyncio.run(listen_async())
169
170		# Or using the listen_for_matrix_messages function
171		from ollamatools.tools.matrix_tools import listen_for_matrix_messages
172
173		listen_for_matrix_messages(client)
174		```
175
176		### 6. Custom Message Handling
177
178		You can also provide a custom message handler to process messages in your own way:
179
180		```python
181		async def custom_message_handler(room, event, client):
182		message = event.body
183		room_id = room.room_id
184		sender = event.sender
185
186		print(f"Received message in room {room_id} from {sender}: {message}")
187
188		# Process the message with Ollama
189		response = await client.process_with_ollama(message)
190
191		# Add a prefix to the response
192		response = f"Response from Ollama: {response}"
193
194		# Send the response back to the room
195		await client.send_message(room_id, response)
196
197		# Set the custom message handler
198		client.set_message_callback(custom_message_handler)
199
200		# Start listening for messages
201		asyncio.run(client.listen())
202		```
203
204		## Configuration
205
206		You can use the MatrixConfig class to load and save configuration from YAML files:
207
208		```python
209		from ollamatools.tools.matrix_tools import (
210		MatrixConfig,
211		load_matrix_config,
212		save_matrix_config,
213		create_default_matrix_config
214		)
215
216		# Create a default configuration
217		config = create_default_matrix_config()
218
219		# Save the configuration to a file
220		save_matrix_config(config, "config.yaml")
221
222		# Load the configuration from a file
223		config = load_matrix_config("config.yaml")
224		```
225
226		A typical configuration file looks like this:
227
228		```yaml
229		MATRIX:
230		HOMESERVER: "https://matrix.example.org"
231		USER: "@user:example.org"
232		PASSWORD: "your-password"
233		DEVICE_ID: "ollamatools-matrix"
234		ROOMS:
235		- "#room:example.org"
236		- "#another-room:example.org"
237
238		OLLAMA:
239		HOST: "localhost:11434"
240		MODEL: "llama3:latest"
241		PROMPT: "You are a helpful assistant."
242		```
243
244		## Example Scripts
245
246		### Basic Matrix Client
247
248		See the `examples/matrix_client_example.py` script for a complete example of how to use the basic Matrix integration.
249
250		To run the example:
251
252		1. Create a configuration file:
253
254		```bash
255		python examples/matrix_client_example.py --create-config
256		```
257
258		2. Edit the configuration file at `examples/config/matrix_config.yaml` with your Matrix credentials and Ollama settings.
259
260		3. Run the example:
261
262		```bash
263		python examples/matrix_client_example.py
264		```
265
266		### Enhanced Matrix Client
267
268		See the `examples/enhanced_matrix_client_example.py` script for a complete example of how to use the enhanced Matrix integration with advanced features.
269
270		To run the example:
271
272		1. Create a configuration file:
273
274		```bash
275		python examples/enhanced_matrix_client_example.py --create-config
276		```
277
278		2. Edit the configuration file at `examples/config/enhanced_matrix_config.yaml` with your Matrix credentials, Ollama settings, and enhanced options.
279
280		3. Run the example:
281
282		```bash
283		python examples/enhanced_matrix_client_example.py
284		```
285
286		## Advanced Usage
287
288		### Basic Matrix Client Features
289
290		#### Model Selection
291
292		The Matrix client supports switching between different Ollama models. Users can change the model they're using with the `/model` command:
293
294		```
295		/model llama3:latest
296		```
297
298		To enable model selection:
299
300		1. Specify available models in your configuration:
301
302		```yaml
303		OLLAMA:
304		AVAILABLE_MODELS:
305		- "llama3:latest"
306		- "mistral:latest"
307		- "gemma:latest"
308		```
309
310		2. Or provide them when creating the client:
311
312		```python
313		client = MatrixClient(
314		# ... other parameters ...
315		available_models=["llama3:latest", "mistral:latest", "gemma:latest"]
316		)
317		```
318
319		3. Users can list available models with the `/models` command and get help with the `/help` command.
320
321		4. You can also update available models from the Ollama server:
322
323		```python
324		await client.update_available_models()
325		```
326
327		#### User Context Tracking
328
329		The Matrix client maintains conversation context for each user in each room. This allows for more coherent conversations as the model can remember previous messages.
330
331		Context tracking is enabled by default and works automatically when you pass the sender and room_id to the `process_with_ollama` method:
332
333		```python
334		response = await client.process_with_ollama(message, sender, room_id)
335		```
336
337		You can control the maximum context length by setting:
338
339		```python
340		client.max_context_length = 15 # Default is 10
341		```
342
343		#### Admin Users
344
345		You can designate certain Matrix users as admins, giving them special privileges:
346
347		```python
348		client = MatrixClient(
349		# ... other parameters ...
350		admin_users=["@admin:example.org", "@moderator:example.org"]
351		)
352		```
353
354		Or add them later:
355
356		```python
357		client.add_admin_user("@admin:example.org")
358		client.remove_admin_user("@admin:example.org")
359		```
360
361		### Enhanced Matrix Client Features
362
363		The enhanced Matrix client includes all the features of the basic client, plus the following advanced features:
364
365		#### Modular Structure
366
367		The enhanced Matrix client is organized into multiple modules for better maintainability and extensibility:
368
369		1. **enhanced_matrix_core.py**: Contains the core `EnhancedMatrixClient` class and basic functionality
370		2. **enhanced_matrix_search.py**: Contains search-related functionality
371		3. **enhanced_matrix_history.py**: Contains conversation history tracking and analytics
372		4. **enhanced_matrix_tools.py**: Main module that imports and re-exports everything
373
374		This modular structure makes it easier to:
375		- Understand and maintain the codebase
376		- Extend specific functionality without affecting other parts
377		- Import only the components you need
378		- Avoid circular dependencies
379
380		You can import from the main module to get all functionality:
381
382		```python
383		from ollamatools.tools.enhanced_matrix_tools import (
384		EnhancedMatrixClient,
385		connect_enhanced_matrix,
386		# ... other imports
387		)
388		```
389
390		Or import directly from specific modules if you only need certain components:
391
392		```python
393		from ollamatools.tools.enhanced_matrix_core import EnhancedMatrixClient
394		from ollamatools.tools.enhanced_matrix_search import execute_search_query
395		from ollamatools.tools.enhanced_matrix_history import analyze_conversation_topics
396		```
397
398		#### Improved Message Response Logic
399
400		The enhanced client has improved message detection logic:
401
402		- Case-insensitive username detection
403		- Direct mention detection (e.g., @username:server.com)
404		- Option to respond to all messages (not just mentions)
405
406		```python
407		client = EnhancedMatrixClient(
408		# ... other parameters ...
409		respond_to_all_messages=True # Respond to all messages, not just mentions
410		)
411		```
412
413		#### Search Integration
414
415		The enhanced client can search the web for information using OpenSearchNG:
416
417		```python
418		client = EnhancedMatrixClient(
419		# ... other parameters ...
420		enable_search=True,
421		opensearch_config={
422		"BASE_URL": "http://localhost:8081",
423		"ENGINES": ["google", "bing", "duckduckgo", "brave"],
424		"NUM_RESULTS": 5
425		}
426		)
427		```
428
429		Users can search directly with the `/search` command:
430
431		```
432		/search what is the capital of France?
433		```
434
435		The client can also automatically detect when a message might benefit from search and augment the message with search results.
436
437		#### Conversation History Tracking
438
439		The enhanced client can track conversation history and provide analytics:
440
441		```python
442		client = EnhancedMatrixClient(
443		# ... other parameters ...
444		enable_conversation_tracking=True,
445		max_conversation_history=100
446		)
447		```
448
449		Users can access conversation history with the following commands:
450
451		- `/history [count]` - Show recent conversation history
452		- `/topics` - Show conversation topics
453		- `/metrics` - Show performance metrics
454
455		#### Configuration
456
457		You can configure the enhanced client using the configuration file:
458
459		```yaml
460		ENHANCED:
461		# Enable search functionality (requires OpenSearchNG)
462		ENABLE_SEARCH: true
463
464		# Enable conversation tracking and history
465		ENABLE_CONVERSATION_TRACKING: true
466
467		# Maximum number of messages to keep in context for each user
468		MAX_CONTEXT_LENGTH: 10
469
470		# Maximum number of conversations to keep in history
471		MAX_CONVERSATION_HISTORY: 100
472
473		# Whether to respond to all messages (not just mentions)
474		RESPOND_TO_ALL_MESSAGES: false
475
476		# OpenSearchNG configuration
477		OPENSEARCH:
478		BASE_URL: "http://localhost:8081"
479		ENGINES:
480		- "google"
481		- "bing"
482		- "duckduckgo"
483		- "brave"
484		NUM_RESULTS: 5
485		```
486
487		### End-to-End Encryption
488
489		The Matrix integration supports end-to-end encryption through the matrix-nio[e2e] package. To use E2E encryption, you need to:
490
491		1. Install the E2E dependencies:
492
493		```bash
494		pip install matrix-nio[e2e]
495		```
496
497		2. Provide a device ID when creating the Matrix client:
498
499		```python
500		client = MatrixClient(
501		homeserver="https://matrix.example.org",
502		user_id="@user:example.org",
503		password="your-password",
504		device_id="ollamatools-matrix" # Required for E2E encryption
505		)
506		```
507
508		### Streaming Responses
509
510		The Matrix client currently doesn't support streaming responses from Ollama, but this feature may be added in the future.
511
512		## Troubleshooting
513
514		### Connection Issues
515
516		If you're having trouble connecting to the Matrix homeserver, check:
517
518		- The homeserver URL is correct and includes the protocol (https://)
519		- Your user ID and password are correct
520		- The Matrix server is reachable from your network
521
522		### Message Processing Issues
523
524		If messages aren't being processed correctly:
525
526		- Check that Ollama is running and accessible at the configured host
527		- Verify that the specified model is available in your Ollama installation
528		- Enable debug logging to see more detailed information:
529
530		```python
531		import logging
532		logging.basicConfig(level=logging.DEBUG)
533		```
534
535		## API Reference
536
537		For detailed API documentation, see the docstrings in the following files:
538
539		- **Basic Matrix Client**: `ollamatools/tools/matrix_tools.py`
540		- **Enhanced Matrix Client**:
541		- Main module: `ollamatools/tools/enhanced_matrix_tools.py`
542		- Core functionality: `ollamatools/tools/enhanced_matrix_core.py`
543		- Search functionality: `ollamatools/tools/enhanced_matrix_search.py`
544		- History tracking: `ollamatools/tools/enhanced_matrix_history.py`
545
546		Each module contains comprehensive docstrings for all classes, methods, and functions.

ollamatools-project/docs/migration_guide.md +0 -248

…	…	@@ -1,248 +0,0 @@
1		# Migration to ModularOllamaAgent
2
3		This guide provides instructions for migrating to the ModularOllamaAgent.
4
5		## Overview
6
7		The ModularOllamaAgent is the primary agent implementation in OllamaTools, with a modular and maintainable architecture. This guide will help you migrate your code to use the ModularOllamaAgent if you were previously using the now-removed TransparentOllamaAgent.
8
9		## Basic Migration
10
11		### Import Changes
12
13		Change your imports from:
14
15		```python
16		from ollamatools.agent.transparent import TransparentOllamaAgent
17		```
18
19		To:
20
21		```python
22		from ollamatools.agent import ModularOllamaAgent
23		```
24
25		### Class Name Changes
26
27		Change instances of `TransparentOllamaAgent` to `ModularOllamaAgent`:
28
29		```python
30		# Old code
31		agent = TransparentOllamaAgent(model="llama3.1:latest")
32
33		# New code
34		agent = ModularOllamaAgent(model="llama3.1:latest")
35		```
36
37		## Utility Function Migration
38
39		If you're using utility functions from the transparent agent, update your imports:
40
41		### Old imports:
42
43		```python
44		from ollamatools.agent.transparent import (
45		print_status, print_success, print_error,
46		print_warning, print_info
47		)
48		```
49
50		### New imports:
51
52		```python
53		from ollamatools.agent.modular import (
54		print_status, print_success, print_error,
55		print_warning, print_info, suggest_fix
56		)
57		```
58
59		## Feedback System Migration
60
61		If you're using the feedback system, update your imports:
62
63		### Old imports:
64
65		```python
66		from ollamatools.agent.transparent import FunctionSelectionFeedback
67		from ollamatools.agent.transparent.feedback import (
68		detect_command_intent,
69		extract_command_from_similar_queries,
70		format_command_result
71		)
72		```
73
74		### New imports:
75
76		```python
77		from ollamatools.agent.modular import FunctionSelectionFeedback
78		from ollamatools.agent.modular.feedback import (
79		detect_command_intent,
80		extract_command_from_similar_queries,
81		format_command_result
82		)
83		```
84
85		## Complete Example Migration
86
87		### Old code:
88
89		```python
90		from ollamatools.agent.transparent import (
91		TransparentOllamaAgent,
92		print_status, print_success, print_info
93		)
94		from ollamatools.opensearch_web_tools import DEFAULT_OPENSEARCH_CONFIG
95		from ollamatools.command_validator import DEFAULT_ALLOWLIST
96
97		# Extend the default command allowlist
98		EXTENDED_ALLOWLIST = DEFAULT_ALLOWLIST.union({
99		"python", "python3", "calc", "find", "grep", "cat", "echo", "pwd"
100		})
101
102		# Initialize the agent
103		agent = TransparentOllamaAgent(
104		model="llama3.1:latest",
105		permission_prompts=True,
106		opensearch_config=DEFAULT_OPENSEARCH_CONFIG,
107		command_allowlist=EXTENDED_ALLOWLIST
108		)
109
110		# Use the agent
111		response = agent.chat("Hello, how can you help me?")
112		print(response)
113		```
114
115		### New code:
116
117		```python
118		from ollamatools.agent import ModularOllamaAgent
119		from ollamatools.agent.modular import (
120		print_status, print_success, print_info
121		)
122		from ollamatools.opensearch_web_tools import DEFAULT_OPENSEARCH_CONFIG
123		from ollamatools.command_validator import DEFAULT_ALLOWLIST
124
125		# Extend the default command allowlist
126		EXTENDED_ALLOWLIST = DEFAULT_ALLOWLIST.union({
127		"python", "python3", "calc", "find", "grep", "cat", "echo", "pwd"
128		})
129
130		# Initialize the agent
131		agent = ModularOllamaAgent(
132		model="llama3.1:latest",
133		permission_prompts=True,
134		opensearch_config=DEFAULT_OPENSEARCH_CONFIG,
135		command_allowlist=EXTENDED_ALLOWLIST
136		)
137
138		# Use the agent
139		response = agent.chat("Hello, how can you help me?")
140		print(response)
141		```
142
143		## Extending the Agent
144
145		If you have code that extends the TransparentOllamaAgent, update your class inheritance:
146
147		### Old code:
148
149		```python
150		from ollamatools.agent.transparent import TransparentOllamaAgent
151
152		class CustomAgent(TransparentOllamaAgent):
153		def __init__(self, *args, **kwargs):
154		super().__init__(*args, **kwargs)
155		# Custom initialization
156
157		def custom_method(self):
158		# Custom implementation
159		pass
160		```
161
162		### New code:
163
164		```python
165		from ollamatools.agent import ModularOllamaAgent
166
167		class CustomAgent(ModularOllamaAgent):
168		def __init__(self, *args, **kwargs):
169		super().__init__(*args, **kwargs)
170		# Custom initialization
171
172		def custom_method(self):
173		# Custom implementation
174		pass
175		```
176
177		## Breaking Changes
178
179		The TransparentOllamaAgent class and the entire transparent agent implementation have been removed from the codebase. If your code was using the TransparentOllamaAgent, you must update it to use the ModularOllamaAgent instead.
180
181		This is a breaking change that was made to simplify the codebase and focus development efforts on the modular implementation.
182
183		## New Features in ModularOllamaAgent
184
185		The ModularOllamaAgent includes several new utility functions that weren't available in the TransparentOllamaAgent:
186
187		- `suggest_fix`: Suggests a fix for an error
188		- `format_code_block`: Formats text as a code block
189		- `format_table`: Formats data as a table
190		- `truncate_text`: Truncates text to a maximum length
191		- `format_execution_time`: Formats execution time in a human-readable format
192
193		Example usage:
194
195		```python
196		from ollamatools.agent.modular import (
197		suggest_fix, format_code_block, format_table,
198		truncate_text, format_execution_time
199		)
200
201		# Suggest a fix
202		suggest_fix("Make sure your OpenSearchNG instance is running")
203
204		# Format code
205		code = format_code_block("print('Hello, world!')", language="python")
206
207		# Format a table
208		table = format_table(
209		headers=["Name", "Age", "City"],
210		rows=[
211		["Alice", 30, "New York"],
212		["Bob", 25, "San Francisco"]
213		]
214		)
215
216		# Truncate text
217		short_text = truncate_text("This is a very long text...", max_length=10)
218
219		# Format execution time
220		formatted_time = format_execution_time(0.123) # "123.00ms"
221		```
222
223		## Troubleshooting
224
225		### Import Errors
226
227		If you encounter import errors after migration, make sure you're using the correct import paths:
228
229		```python
230		# For the main agent class
231		from ollamatools.agent import ModularOllamaAgent
232
233		# For utility functions
234		from ollamatools.agent.modular import print_status, print_success, print_error
235		```
236
237		### Method Not Found Errors
238
239		If you encounter "method not found" errors, check if you're using methods that have been renamed or moved. Refer to the [ModularOllamaAgent Guide](modular_agent_guide.md) for a complete list of available methods.
240
241
242		## Getting Help
243
244		If you encounter any issues during migration, please:
245
246		1. Check the [ModularOllamaAgent Guide](modular_agent_guide.md) for detailed documentation
247		2. Look at the example scripts in the `examples/` directory
248		3. File an issue on the GitHub repository if you encounter a bug

ollamatools-project/docs/modular_agent_guide.md +0 -274

…	…	@@ -1,274 +0,0 @@
1		# ModularOllamaAgent Guide
2
3		This guide provides comprehensive documentation for the ModularOllamaAgent, a modular implementation of the OllamaAgent with enhanced capabilities.
4
5		## Overview
6
7		The ModularOllamaAgent is a feature-rich agent built on top of the base OllamaAgent, designed with a modular architecture that separates functionality into logical components for better maintainability and extensibility. It includes features like:
8
9		- Search integration with OpenSearchNG
10		- Command management
11		- Issue tracking
12		- Semantic function selection feedback
13		- Detailed execution history
14
15		## Installation
16
17		The ModularOllamaAgent is included in the OllamaTools package. To install:
18
19		```bash
20		pip install ollamatools
21		```
22
23		## Basic Usage
24
25		Here's a simple example of how to use the ModularOllamaAgent:
26
27		```python
28		from ollamatools.agent import ModularOllamaAgent
29
30		# Initialize the agent
31		agent = ModularOllamaAgent(
32		model="llama3.1:latest",
33		permission_prompts=True
34		)
35
36		# Chat with the agent
37		response = agent.chat("Hello, how can you help me?")
38		print(response)
39		```
40
41		## Configuration Options
42
43		The ModularOllamaAgent accepts several configuration options:
44
45		| Parameter | Type | Description | Default |
46		|-----------|------|-------------|---------|
47		| `model` | str | Ollama model to use | "llama3.1:latest" |
48		| `permission_prompts` | bool | Whether to display permission prompts | True |
49		| `opensearch_config` | Dict | OpenSearchNG configuration | Default config |
50		| `command_allowlist` | Set | Custom command allowlist | Extended allowlist |
51		| `feedback_file_path` | str | Path to the feedback file | ~/.ollamatools/function_selection_feedback.jsonl |
52
53		## Features
54
55		### Search Integration
56
57		The ModularOllamaAgent can automatically detect when a query might benefit from search and perform web searches using OpenSearchNG:
58
59		```python
60		# Explicit search
61		results = agent.execute_search("What is the capital of France?")
62
63		# Or through natural language
64		response = agent.chat("Can you tell me about the history of Python programming language?")
65		# The agent will automatically search if needed
66		```
67
68		### Command Management
69
70		The agent provides methods to manage the list of allowed commands:
71
72		```python
73		# List allowed commands
74		agent.list_allowed_commands()
75
76		# Add a command to the allowlist
77		agent.add_allowed_command("npm")
78
79		# Remove a command from the allowlist
80		agent.remove_allowed_command("rm")
81		```
82
83		### Issue Tracking
84
85		The agent includes a built-in issue tracking system:
86
87		```python
88		# Add an issue
89		issue_id = agent.add_issue(
90		title="Search not working",
91		description="The search functionality fails when using Brave engine",
92		severity="high"
93		)
94
95		# List all issues
96		agent.list_issues()
97
98		# Get issue details
99		agent.get_issue(issue_id)
100
101		# Update an issue
102		agent.update_issue(issue_id, status="resolved")
103		```
104
105		### Semantic Function Selection Feedback
106
107		The agent includes a feedback system that improves its ability to understand natural language queries and map them to appropriate functions:
108
109		```python
110		# Provide explicit feedback
111		agent.provide_function_feedback(
112		query="what directory am I in",
113		correct_function="execute_command",
114		actual_function="read_file"
115		)
116
117		# Analyze feedback patterns
118		analysis = agent.function_feedback.analyze_function_selection_patterns()
119		```
120
121		### Execution History
122
123		The agent keeps track of executed functions and commands:
124
125		```python
126		# Show execution history
127		agent.show_execution_history(max_entries=10)
128		```
129
130		## Advanced Usage
131
132		### Custom System Prompt
133
134		You can customize the system prompt used by the agent:
135
136		```python
137		agent = ModularOllamaAgent(model="llama3.1:latest")
138		agent.system_prompt = """
139		You are an AI assistant specialized in Python programming.
140		Always provide code examples when explaining programming concepts.
141		"""
142		response = agent.chat("How do I use list comprehensions in Python?")
143		```
144
145		### Directory Structure Analysis
146
147		The agent can analyze directory structures:
148
149		```python
150		structure = agent.get_directory_structure(path=".", max_depth=2)
151		```
152
153		## Extending the ModularOllamaAgent
154
155		The modular architecture makes it easy to extend the agent with custom functionality:
156
157		```python
158		from ollamatools.agent import ModularOllamaAgent
159
160		class CustomAgent(ModularOllamaAgent):
161		"""A custom agent with additional capabilities."""
162
163		def __init__(self, *args, **kwargs):
164		super().__init__(*args, **kwargs)
165		# Add custom initialization
166
167		def custom_method(self, param):
168		"""Implement a custom method."""
169		# Your implementation here
170		return f"Processed {param}"
171		```
172
173		## Module Structure
174
175		The ModularOllamaAgent is organized into several modules:
176
177		- `agent.py`: Main agent implementation
178		- `search.py`: Search functionality
179		- `commands.py`: Command management
180		- `issues.py`: Issue tracking
181		- `feedback.py`: Semantic function selection feedback
182		- `utils.py`: Utility functions
183
184		## Complete Example
185
186		Here's a complete example that demonstrates various features of the ModularOllamaAgent:
187
188		```python
189		import os
190		import sys
191		from ollamatools.agent import ModularOllamaAgent
192		from ollamatools.agent.modular import print_info
193		from ollamatools.opensearch_web_tools import DEFAULT_OPENSEARCH_CONFIG
194		from ollamatools.command_validator import DEFAULT_ALLOWLIST
195
196		# Extend the default command allowlist
197		EXTENDED_ALLOWLIST = DEFAULT_ALLOWLIST.union({
198		"python", "python3", "calc", "find", "grep", "cat", "echo", "pwd"
199		})
200
201		# Initialize the agent
202		agent = ModularOllamaAgent(
203		model="llama3.1:latest",
204		permission_prompts=True,
205		opensearch_config=DEFAULT_OPENSEARCH_CONFIG,
206		command_allowlist=EXTENDED_ALLOWLIST
207		)
208
209		# Simple chat loop
210		while True:
211		user_input = input("\nYou: ").strip()
212
213		if user_input.lower() in ['exit', 'quit', 'bye']:
214		print("\nExiting. Goodbye!")
215		break
216
217		# Process as normal chat message
218		response = agent.chat(user_input)
219		print(f"\nAssistant: {response}")
220		```
221
222		## Migrating from TransparentOllamaAgent
223
224		If you're currently using the TransparentOllamaAgent, migrating to the ModularOllamaAgent is straightforward:
225
226		```python
227		# Old code
228		from ollamatools.agent.transparent import TransparentOllamaAgent
229		agent = TransparentOllamaAgent(model="llama3.1:latest")
230
231		# New code
232		from ollamatools.agent import ModularOllamaAgent
233		agent = ModularOllamaAgent(model="llama3.1:latest")
234		```
235
236		The ModularOllamaAgent is a drop-in replacement for the TransparentOllamaAgent, with the same API but a more modular and maintainable implementation.
237
238		## Troubleshooting
239
240		### OpenSearchNG Not Available
241
242		If OpenSearchNG is not available, the agent will fall back to standard search methods:
243
244		```
245		❌ OpenSearchNG not available: Could not connect to OpenSearchNG server at http://localhost:8081
246		⚠️ Falling back to regular search methods
247		💡 Suggested fix: Make sure your OpenSearchNG instance is running at http://localhost:8081
248		```
249
250		### Command Not in Allowlist
251
252		If a command is not in the allowlist, you'll see an error:
253
254		```
255		❌ Command 'npm' is not in the allowed command list.
256		```
257
258		You can add it to the allowlist:
259
260		```python
261		agent.add_allowed_command("npm")
262		```
263
264		## Best Practices
265
266		1. **Use the ModularOllamaAgent for new projects**: The ModularOllamaAgent is the recommended agent for new projects.
267
268		2. **Leverage the feedback system**: The feedback system can significantly improve the agent's ability to understand natural language queries.
269
270		3. **Customize the command allowlist**: Tailor the command allowlist to your specific needs to balance functionality and security.
271
272		4. **Extend rather than modify**: When adding custom functionality, extend the ModularOllamaAgent rather than modifying its core implementation.
273
274		5. **Use the issue tracking system**: The built-in issue tracking system can help you manage issues and track their resolution.

ollamatools-project/docs/user_guide.md +0 -518

…	…	@@ -1,518 +0,0 @@
1		# OllamaTools User Guide
2
3		This guide provides detailed instructions on using OllamaTools, a Python library that extends Ollama with semantic function calling capabilities.
4
5		## Table of Contents
6
7		1. [Installation](#installation)
8		2. [Getting Started](#getting-started)
9		3. [Using OllamaAgent](#using-ollamaagent)
10		4. [Working with System Tools](#working-with-system-tools)
11		5. [Working with Web Tools](#working-with-web-tools)
12		6. [Working with Documentation Tools](#working-with-documentation-tools)
13		7. [Security and Permissions](#security-and-permissions)
14		8. [Configuration Options](#configuration-options)
15		9. [Best Practices](#best-practices)
16		10. [Troubleshooting](#troubleshooting)
17
18		## Installation
19
20		### Prerequisites
21
22		- Python 3.8 or higher
23		- Ollama 0.4.0 or higher with function calling support
24		- Linux-based operating system (primary target)
25
26		### Installation Steps
27
28		```bash
29		# Install from PyPI
30		pip install ollamatools
31
32		# Or install from source
33		git clone https://github.com/yourusername/ollamatools.git
34		cd ollamatools
35		pip install -e .
36		```
37
38		### Verifying Installation
39
40		To verify that OllamaTools is installed correctly, run:
41
42		```bash
43		python -c "import ollamatools; print(ollamatools.__version__)"
44		```
45
46		## Getting Started
47
48		### Basic Setup
49
50		```python
51		from ollamatools import OllamaAgent
52
53		# Create an agent with default settings
54		agent = OllamaAgent()
55
56		# Send a message to the agent
57		response = agent.chat("Hello, Ollama!")
58		print(response)
59		```
60
61		### Running Your First Function
62
63		```python
64		# Execute a system command through the agent
65		response = agent.chat("What files are in the current directory?")
66
67		# This will trigger a permission prompt
68		# After approving, you'll see the list of files
69		print(response)
70		```
71
72		## Using OllamaAgent
73
74		### Initialization Options
75
76		```python
77		from ollamatools import OllamaAgent, Config
78
79		# Create a custom configuration
80		config = Config(
81		model="llama3.1",
82		host="http://localhost:11434",
83		permission_prompts=True,
84		sudo_always_prompt=True,
85		timeout_seconds=60,
86		max_file_size=1024 * 1024 # 1 MB
87		)
88
89		# Create an agent with the custom configuration
90		agent = OllamaAgent(config=config)
91		```
92
93		### Chat Method
94
95		The `chat` method is the primary way to interact with the agent:
96
97		```python
98		response = agent.chat(
99		message="What's the weather like in San Francisco?",
100		system_prompt="You are a helpful assistant.",
101		context=[
102		{"role": "user", "content": "I need help with the weather."},
103		{"role": "assistant", "content": "I'd be happy to help. What city are you interested in?"}
104		]
105		)
106		```
107
108		### Function Calling
109
110		Function calling happens automatically when the agent detects that a function should be called based on the conversation context:
111
112		```python
113		# This will trigger the execute_command function
114		response = agent.chat("Show me the largest files in the current directory")
115
116		# This will trigger the fetch_url function
117		response = agent.chat("What's on the front page of example.com?")
118
119		# This will trigger the generate_markdown function
120		response = agent.chat("Create documentation for a Python function that calculates prime numbers")
121		```
122
123		## Working with System Tools
124
125		### Executing Commands
126
127		```python
128		from ollamatools.security import PermissionManager
129		from ollamatools.tools.system_tools import execute_command
130
131		# Create a permission manager
132		permission_manager = PermissionManager(prompt_enabled=True)
133
134		# Execute a command
135		result = execute_command(
136		command="ls",
137		args=["-la"],
138		permission_manager=permission_manager
139		)
140
141		# Print the result
142		print(f"Command output: {result['stdout']}")
143		print(f"Error output: {result['stderr']}")
144		print(f"Exit code: {result['returncode']}")
145		```
146
147		### Reading Files
148
149		```python
150		from ollamatools.tools.system_tools import read_file
151
152		# Read a file
153		content = read_file(
154		path="example.txt",
155		permission_manager=permission_manager
156		)
157
158		print(f"File content: {content}")
159		```
160
161		### Writing Files
162
163		```python
164		from ollamatools.tools.system_tools import write_file
165
166		# Write to a file
167		success = write_file(
168		path="output.txt",
169		content="Hello, OllamaTools!",
170		permission_manager=permission_manager
171		)
172
173		print(f"Write success: {success}")
174		```
175
176		### Finding Files
177
178		```python
179		from ollamatools.tools.system_tools import find_files
180
181		# Find files matching a pattern
182		py_files = find_files(
183		pattern="*.py",
184		directory=".",
185		recursive=True,
186		permission_manager=permission_manager
187		)
188
189		print(f"Found {len(py_files)} Python files")
190		for file in py_files[:5]: # Show the first 5 files
191		print(f"- {file}")
192		```
193
194		## Working with Web Tools
195
196		### Fetching URLs
197
198		```python
199		from ollamatools.tools.web_tools import fetch_url
200
201		# Fetch a URL
202		result = fetch_url(
203		url="https://example.com",
204		method="GET",
205		params=None,
206		headers={"User-Agent": "OllamaTools/0.1.0"},
207		permission_manager=permission_manager
208		)
209
210		print(f"Status: {result['status']}")
211		print(f"Content type: {result['headers'].get('Content-Type')}")
212		print(f"Content snippet: {result['content'][:100]}...")
213		```
214
215		### Getting Weather Information
216
217		```python
218		from ollamatools.tools.web_tools import get_weather
219
220		# Get weather information (requires API key configuration)
221		weather = get_weather(
222		location="San Francisco",
223		days=3,
224		units="metric",
225		permission_manager=permission_manager
226		)
227
228		print(f"Current temperature: {weather['current']['temperature']}°C")
229		print(f"Conditions: {weather['current']['conditions']}")
230		print(f"Forecast:")
231		for day in weather.get('forecast', []):
232		print(f"- {day['day']}: {day['conditions']}, High: {day['high']}°C, Low: {day['low']}°C")
233		```
234
235		### Searching the Web
236
237		```python
238		from ollamatools.tools.web_tools import search_web
239
240		# Search the web (requires API key configuration)
241		search_results = search_web(
242		query="Python programming",
243		num_results=3,
244		permission_manager=permission_manager
245		)
246
247		print(f"Search results:")
248		for i, result in enumerate(search_results):
249		print(f"{i+1}. {result['title']} - {result['url']}")
250		print(f" {result['snippet']}")
251		```
252
253		## Working with Documentation Tools
254
255		### Generating Markdown
256
257		```python
258		from ollamatools.tools.doc_tools import generate_markdown
259
260		# Generate markdown
261		content = """# Sample Document
262
263		This is a sample document created by OllamaTools.
264
265		## Features
266		- Easy to use
267		- Secure by default
268		- Extensible
269		"""
270
271		markdown = generate_markdown(
272		content=content,
273		title="Sample Document",
274		permission_manager=permission_manager
275		)
276
277		print(f"Generated markdown:\n{markdown}")
278
279		# Write the markdown to a file
280		from ollamatools.tools.system_tools import write_file
281		write_file("sample_doc.md", markdown, permission_manager=permission_manager)
282		```
283
284		### Extracting Context
285
286		```python
287		from ollamatools.tools.doc_tools import extract_context
288
289		# Define some documents
290		documents = [
291		"OllamaTools is a Python library for Ollama LLMs.",
292		"It provides semantic function calling capabilities.",
293		"Python is a popular programming language.",
294		"JavaScript is another programming language."
295		]
296
297		# Extract context for a query
298		context = extract_context(
299		query="Ollama functions",
300		documents=documents,
301		permission_manager=permission_manager
302		)
303
304		print(f"Extracted context: {context}")
305		```
306
307		### Summarizing Text
308
309		```python
310		from ollamatools.tools.doc_tools import summarize_text
311
312		# Define some text to summarize
313		text = """
314		OllamaTools is a Python library that extends Ollama with semantic function calling capabilities.
315		It enables automatic invocation of system commands, web requests, and documentation utilities based on conversational context.
316		The primary goal is to allow Ollama LLMs to execute external functions based on semantic understanding of user queries.
317		The library includes several components such as OllamaAgent, PermissionManager, and various tool libraries.
318		"""
319
320		# Summarize the text
321		summary = summarize_text(
322		text=text,
323		permission_manager=permission_manager
324		)
325
326		print(f"Summary: {summary}")
327		```
328
329		## Security and Permissions
330
331		### Permission System
332
333		OllamaTools uses a permission system to ensure that functions are executed securely:
334
335		```python
336		from ollamatools.security import PermissionManager
337
338		# Create a permission manager with prompts enabled
339		permission_manager = PermissionManager(
340		prompt_enabled=True, # Show permission prompts
341		sudo_always_prompt=True # Always prompt for sudo commands
342		)
343
344		# Request permission for a function
345		permission_granted = permission_manager.request_permission(
346		function_name="execute_command",
347		args={"command": "ls", "args": ["-la"]},
348		is_sudo=False
349		)
350
351		if permission_granted:
352		print("Permission granted")
353		else:
354		print("Permission denied")
355		```
356
357		### Permission Response Options
358
359		When prompted for permission, you have several options:
360
361		1. **Allow**: Grant permission for this operation
362		2. **Allow for session**: Grant permission for similar operations in this session
363		3. **Deny**: Reject the operation
364		4. **Modify**: Allow modifying the operation before execution
365
366		### Handling Sudo Commands
367
368		Sudo commands receive special handling:
369
370		```python
371		# Check if a command uses sudo
372		is_sudo = permission_manager.detect_sudo("sudo apt-get update")
373		print(f"Is sudo command: {is_sudo}") # True
374
375		# This will always prompt for permission if sudo_always_prompt is True
376		permission_granted = permission_manager.request_permission(
377		function_name="execute_command",
378		args={"command": "sudo", "args": ["apt-get", "update"]},
379		is_sudo=True
380		)
381		```
382
383		## Configuration Options
384
385		### Available Options
386
387		OllamaTools provides several configuration options:
388
389		```python
390		from ollamatools.config import Config
391
392		# Create a configuration with default values
393		default_config = Config()
394		print(f"Default config: {default_config}")
395
396		# Create a custom configuration
397		custom_config = Config(
398		model="llama3.1",
399		host="http://localhost:11434",
400		permission_prompts=True,
401		sudo_always_prompt=True,
402		timeout_seconds=30,
403		max_file_size=1024 * 1024 # 1 MB
404		)
405		print(f"Custom config: {custom_config}")
406		```
407
408		### Saving and Loading Configuration
409
410		You can save and load configuration to/from a file:
411
412		```python
413		# Save configuration to a file
414		config_file = "ollamatools_config.json"
415		custom_config.save(config_file)
416		print(f"Saved configuration to {config_file}")
417
418		# Load configuration from a file
419		loaded_config = Config.load(config_file)
420		print(f"Loaded config: {loaded_config}")
421		```
422
423		## Best Practices
424
425		### Security Best Practices
426
427		1. **Keep permission prompts enabled** for sensitive operations
428		2. **Always prompt for sudo commands** to prevent unauthorized elevation
429		3. **Use resource limits** to prevent excessive resource usage
430		4. **Validate inputs** before passing them to system functions
431		5. **Review function calls** carefully before approving them
432
433		### Performance Best Practices
434
435		1. **Use caching** for frequently accessed resources
436		2. **Use asynchronous operations** for I/O-bound tasks
437		3. **Limit resource usage** for long-running operations
438		4. **Use session permissions** to avoid repeated prompts for similar operations
439
440		### Usage Best Practices
441
442		1. **Start with simple queries** and gradually increase complexity
443		2. **Use specific language** for clearer function detection
444		3. **Handle errors gracefully** by checking result objects
445		4. **Use appropriate timeout values** for different operations
446		5. **Provide feedback** to improve the agent's understanding
447
448		## Troubleshooting
449
450		### Common Issues and Solutions
451
452		#### Permission Issues
453
454		**Issue**: Permission prompts not appearing or always denied.
455
456		**Solution**: Ensure the permission manager is properly initialized and that `prompt_enabled` is set to `True`.
457
458		```python
459		permission_manager = PermissionManager(prompt_enabled=True)
460		```
461
462		#### Command Execution Issues
463
464		**Issue**: Commands fail to execute.
465
466		**Solution**: Check if the command exists and is in the PATH. Try running the command directly in the terminal to see if it works.
467
468		```python
469		# Check if a command exists
470		import shutil
471		command_exists = shutil.which("your_command") is not None
472		```
473
474		#### Timeout Issues
475
476		**Issue**: Operations time out.
477
478		**Solution**: Increase the timeout value for long-running operations.
479
480		```python
481		result = execute_command(
482		command="long_running_command",
483		timeout=300, # 5 minutes
484		permission_manager=permission_manager
485		)
486		```
487
488		#### API Key Issues
489
490		**Issue**: Weather or search functions fail.
491
492		**Solution**: Ensure API keys are properly configured.
493
494		#### Ollama Server Issues
495
496		**Issue**: Cannot connect to Ollama server.
497
498		**Solution**: Ensure Ollama is running and accessible at the configured host address.
499
500		```bash
501		# Start Ollama server
502		ollama serve
503
504		# Check if Ollama is running
505		curl http://localhost:11434/api/version
506		```
507
508		### Getting Help
509
510		If you encounter issues not covered in this guide, please:
511
512		1. Check the [GitHub repository](https://github.com/yourusername/ollamatools) for known issues
513		2. Submit a new issue if your problem is not already reported
514		3. Join the community discussion on Discord or other platforms
515
516		---
517
518		This user guide is a living document and will be updated as OllamaTools evolves. Your feedback is valuable in improving both the library and this documentation.

ollamatools-project/example_doc.html +0 -27

…	…	@@ -1,27 +0,0 @@
1		<!DOCTYPE html>
2		<html>
3		<head>
4		<title>OllamaTools Documentation Example</title>
5		<meta charset="UTF-8">
6		<meta name="viewport" content="width=device-width, initial-scale=1.0">
7		</head>
8		<body>
9		<h1>OllamaTools Documentation Example</h1>
10		<h2>Introduction</h2>
11		<p>This is a sample document generated using OllamaTools' documentation utilities.</p>
12		<h2>Features</h2>
13		<ul>
14		<li>Markdown formatting</li>
15		<li>HTML conversion</li>
16		<li>Simple and easy to use</li>
17		</ul>
18		<h2>Usage</h2>
19		<p>```python
20		from ollamatools.tools.doc_tools import generate_markdown</p>
21		<h1>Generate markdown</h1>
22		<p>doc = generate_markdown(content, title, format_type='markdown')</p>
23		<h1>Generate HTML</h1>
24		<p>html = generate_markdown(content, title, format_type='html')
25		```</p>
26		</body>
27		</html>

ollamatools-project/example_doc.md +0 -24

…	…	@@ -1,24 +0,0 @@
1		# OllamaTools Documentation Example
2
3
4		## Introduction
5
6		This is a sample document generated using OllamaTools' documentation utilities.
7
8		## Features
9
10		- Markdown formatting
11		- HTML conversion
12		- Simple and easy to use
13
14		## Usage
15
16		```python
17		from ollamatools.tools.doc_tools import generate_markdown
18
19		# Generate markdown
20		doc = generate_markdown(content, title, format_type='markdown')
21
22		# Generate HTML
23		html = generate_markdown(content, title, format_type='html')
24		```

ollamatools-project/examples/README_MODULAR_AGENT.md +0 -235

…	…	@@ -1,235 +0,0 @@
1		# Modular Ollama Agent Examples
2
3		This directory contains examples of the Modular Ollama Agent:
4
5		1. `improved_transparent_agent.py` - An implementation using the ModularOllamaAgent
6		2. `extended_transparent_agent.py` - An example of extending the ModularOllamaAgent with custom functionality
7
8		## Comparison of Implementations
9
10		### Original Implementation (`transparent_ollama_agent.py`)
11
12		The original implementation:
13		- Contains all functionality in a single file (~1000+ lines)
14		- Defines the `TransparentOllamaAgent` class with all methods directly in the class
15		- Handles search, command management, issue tracking, and other functionality in the same file
16		- Is more difficult to maintain and extend due to its monolithic nature
17
18		### Improved Implementation (`improved_transparent_agent.py`)
19
20		The improved implementation:
21		- Uses the modular agent code from `ollamatools/agent/modular/`
22		- Is much shorter (~150 lines) because it leverages the modular components
23		- Imports the `ModularOllamaAgent` class and utility functions from the modular code
24		- Has comprehensive functionality with better code organization
25
26		## Benefits of the Modular Approach
27
28		1. **Better Code Organization**: Functionality is logically separated into different modules:
29		- `agent.py`: Main agent implementation
30		- `search.py`: Search functionality
31		- `commands.py`: Command management
32		- `issues.py`: Issue tracking
33		- `utils.py`: Utility functions
34		- `feedback.py`: Semantic function selection feedback
35
36		2. **Easier Maintenance**: Changes to one aspect (e.g., search) don't require modifying the entire agent
37
38		3. **Improved Reusability**: Components can be reused in other agents or scripts
39
40		4. **Cleaner Implementation**: The main agent class delegates to specialized modules
41
42		5. **Better Testing**: Modules can be tested independently
43
44		## How to Use the Modular Agent
45
46		To use the modular agent in your own scripts:
47
48		```python
49		from ollamatools.agent import ModularOllamaAgent
50		from ollamatools.agent.modular import (
51		print_status, print_success, print_info
52		)
53
54		# Initialize the agent
55		agent = ModularOllamaAgent(
56		model="llama3.1:latest",
57		permission_prompts=True
58		)
59
60		# Use the agent
61		response = agent.chat("Hello, how can you help me?")
62		print(response)
63		```
64
65		## Extending the Modular Agent
66
67		If you want to extend the functionality of the modular agent:
68
69		1. Create a new module in `ollamatools/agent/` for your specific functionality
70		2. Import and use the existing modules as needed
71		3. Create a new agent class that inherits from `ModularOllamaAgent` if necessary
72
73		Example of extending the agent:
74
75		```python
76		from ollamatools.agent import ModularOllamaAgent
77
78		class EnhancedAgent(ModularOllamaAgent):
79		"""An enhanced version of the ModularOllamaAgent with additional capabilities."""
80
81		def __init__(self, *args, **kwargs):
82		super().__init__(*args, **kwargs)
83		# Add your custom initialization here
84
85		def new_feature(self):
86		"""Implement a new feature."""
87		# Your implementation here
88		pass
89		```
90
91		## Running the Examples
92
93		To run the improved implementation:
94		```
95		python improved_transparent_agent.py
96		```
97
98		To run the extended implementation with custom features:
99		```
100		python extended_transparent_agent.py
101		```
102
103		The extended implementation demonstrates how to add custom features by inheriting from the modular agent.
104
105		## Semantic Function Selection Feedback
106
107		The improved agent now includes a feedback system that enhances its ability to understand natural language queries and map them to appropriate functions. This system:
108
109		1. **Learns from User Interactions**: Records when functions are used correctly or incorrectly
110		2. **Improves Over Time**: Gets better at selecting the right function for similar queries
111		3. **Adapts to User Language**: Learns the specific ways users phrase their requests
112		4. **Provides Analytics**: Offers insights into common patterns and mismatches
113
114		### How the Feedback System Works
115
116		1. **Data Collection**: The system records:
117		- Which queries led to which function calls
118		- Whether the function calls were successful
119		- User corrections when the wrong function was used
120		- Similarity scores between current and past queries
121
122		2. **Pattern Recognition**: The system analyzes:
123		- Common words and phrases in similar queries
124		- Which functions work best for different types of questions
125		- Patterns of mismatches between expected and actual functions
126		- Implicit feedback from execution results
127
128		3. **Improved Selection**: When a new query comes in:
129		- The system compares it to previous similar queries
130		- It suggests the function that worked best for similar queries
131		- It calculates confidence scores for function suggestions
132		- It applies this knowledge before falling back to default behavior
133
134		### Technical Implementation
135
136		The feedback system is implemented in the `ollamatools/agent/modular/feedback.py` module through the `FunctionSelectionFeedback` class:
137
138		1. **Feedback Storage**:
139		- Feedback is stored in a JSONL file at `~/.ollamatools/function_selection_feedback.jsonl`
140		- Recent feedback is cached in memory for quick access
141		- Each feedback entry includes query, function information, timestamp, and similarity data
142
143		2. **Similarity Calculation**:
144		- Uses Jaccard similarity between query word sets
145		- Compares new queries against previously recorded queries
146		- Filters out common stop words to focus on meaningful terms
147		- Returns top similar queries with their similarity scores
148
149		3. **Function Suggestion**:
150		- Weights function suggestions by similarity scores
151		- Calculates confidence based on consistency of function usage
152		- Returns the best function match with confidence score
153
154		4. **Pattern Generation**:
155		- Automatically generates regex patterns from feedback data
156		- Creates patterns based on common words in similar queries
157		- Identifies consistent query patterns that map to specific functions
158		- Suggests improvements for function selection logic
159
160		5. **Command Intent Detection**:
161		- Specialized support for detecting system command intents
162		- Maps natural language questions to appropriate shell commands
163		- Uses both predefined patterns and learned patterns from feedback
164		- Formats command results in a user-friendly way
165
166		### Using the Feedback System
167
168		The improved agent includes commands for interacting with the feedback system:
169
170		- `feedback <query> | <correct_function>` - Provide explicit feedback on function selection
171		- `analyze_feedback` - Show analysis of function selection patterns and improvement suggestions
172
173		Example:
174		```
175		You: what directory am I in
176		Assistant: [uses read_file incorrectly]
177
178		You: feedback 'what directory am I in' | execute_command
179		Assistant: Thank you for your feedback! I'll remember to use execute_command for similar queries in the future.
180
181		[Later]
182		You: where am I located?
183		Assistant: [now correctly uses pwd command]
184		```
185
186		### Advanced Usage
187
188		Developers can leverage the feedback system in their own agents:
189
190		```python
191		from ollamatools.agent.modular.feedback import FunctionSelectionFeedback
192
193		# Initialize the feedback system
194		feedback_system = FunctionSelectionFeedback()
195
196		# Get function suggestions for a query
197		function, confidence, similar_queries = feedback_system.get_function_suggestion("what files are in this folder?")
198
199		# Record explicit feedback
200		feedback_system.record_explicit_feedback(
201		query="what files are in this folder?",
202		intended_function="execute_command",
203		actual_function="read_file",
204		user_correction=True
205		)
206
207		# Record implicit feedback from execution results
208		feedback_system.record_implicit_feedback(
209		query="show me the files",
210		function="execute_command",
211		confidence_score=0.8,
212		execution_result={"returncode": 0, "stdout": "file1.txt file2.txt", "stderr": ""}
213		)
214
215		# Generate improved patterns for function selection
216		improved_patterns = feedback_system.generate_improved_patterns()
217
218		# Analyze function selection patterns
219		analysis = feedback_system.analyze_function_selection_patterns()
220		```
221
222		## Extended Agent Features
223
224		The `extended_transparent_agent.py` example demonstrates:
225
226		1. **Conversation History Tracking**: Saves all conversations to a JSON file
227		2. **Conversation Summarization**: Provides summaries of recent conversations
228		3. **Topic Analysis**: Analyzes conversation topics based on keywords
229		4. **Performance Metrics**: Tracks response times and estimated token usage
230
231		This example shows how to:
232		- Inherit from the `ModularOllamaAgent` class
233		- Override methods to add custom functionality
234		- Add new methods for additional features
235		- Maintain compatibility with the base agent's functionality

ollamatools-project/examples/Untitled File +0 -0

(no content changes — binary or rename)

ollamatools-project/examples/agent_debug.py +0 -68

…	…	@@ -1,68 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Debugging script for OllamaAgent to test command execution.
4
5		This script provides a simple way to test command execution and file operations
6		directly without going through the agent's chat interface.
7
8		Usage:
9		python agent_debug.py
10		"""
11
12		import os
13		import sys
14		import logging
15
16		# Add the parent directory to the Python path to import ollamatools
17		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
18
19		from ollamatools.tools.system_tools import execute_command, find_files
20		from ollamatools.security import PermissionManager
21		from ollamatools.utils.logger import setup_logger
22
23		# Set up logging
24		logger = setup_logger("agent_debug", level="INFO")
25
26		def main():
27		"""Run the debug tests."""
28		try:
29		# Create a permission manager with no prompts for testing
30		permission_manager = PermissionManager(prompt_enabled=False)
31
32		print("\n=== Command Execution Tests ===")
33
34		# Test 1: List files with ls -l
35		print("\nTest 1: 'ls -l' command")
36		result = execute_command("ls -l", permission_manager=permission_manager)
37		print(f"Command: {result.get('command', 'unknown')}")
38		print(f"Exit code: {result.get('returncode', 'unknown')}")
39		print(f"Output: {result.get('stdout', 'none')}")
40		print(f"Error: {result.get('stderr', 'none')}")
41
42		# Test 2: Execute pwd
43		print("\nTest 2: 'pwd' command")
44		result = execute_command("pwd", permission_manager=permission_manager)
45		print(f"Command: {result.get('command', 'unknown')}")
46		print(f"Exit code: {result.get('returncode', 'unknown')}")
47		print(f"Output: {result.get('stdout', 'none')}")
48		print(f"Error: {result.get('stderr', 'none')}")
49
50		# Test 3: Find all Python files in the current directory
51		print("\nTest 3: Find all Python files")
52		files = find_files("*.py", directory=".", recursive=True, permission_manager=permission_manager)
53		print(f"Found {len(files)} Python files:")
54		for i, file in enumerate(files[:10]): # Print first 10 files
55		print(f" {i+1}. {file}")
56		if len(files) > 10:
57		print(f" ... and {len(files) - 10} more")
58
59		print("\n=== All tests completed ===")
60		return 0
61
62		except Exception as e:
63		logger.error(f"Error in debug tests: {str(e)}")
64		print(f"Error: {str(e)}")
65		return 1
66
67		if __name__ == "__main__":
68		sys.exit(main())

ollamatools-project/examples/basic_agent.py +0 -100

…	…	@@ -1,100 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Basic example of using the refactored OllamaAgent.
4
5		This example demonstrates how to initialize the OllamaAgent
6		and use it for basic conversation with function calling capabilities.
7		The agent has been refactored to work correctly with Ollama's API.
8
9		Usage:
10		python basic_agent.py
11		"""
12
13		import os
14		import sys
15		import logging
16
17		# Add the parent directory to the Python path to import ollamatools
18		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
19
20		from ollamatools import OllamaAgent, Config
21		from ollamatools.utils.logger import setup_logger
22
23		# Set up logging
24		logger = setup_logger("basic_agent", level="INFO")
25
26		def main():
27		"""Run the example."""
28		try:
29		# Initialize the agent with default settings
30		logger.info("Initializing OllamaAgent...")
31		agent = OllamaAgent(
32		model="llama3.1:latest", # Using your available model
33		permission_prompts=True # Ask for permission before executing functions
34		)
35
36		logger.info("Agent initialized. Enter 'exit' to quit.")
37		print("\nOllamaTools Chat Example")
38		print("------------------------")
39		print("Type 'exit' to quit")
40		print("Try asking the agent to perform tasks like:")
41		print(" - What files are in my current directory?")
42		print(" - Can you get the weather in New York?")
43		print(" - Generate a markdown document about climate change")
44		print(" - What's 2+2?\n")
45
46		# Simple chat loop
47		context = []
48
49		while True:
50		# Get user input
51		user_input = input("\nYou: ")
52
53		# Check for exit command
54		if user_input.lower() in ['exit', 'quit', 'bye']:
55		print("\nExiting. Goodbye!")
56		break
57
58		# Process message with function calling
59		try:
60		# System message that informs the model about its capabilities
61		system_prompt = """
62		You are an AI assistant with access to various tools to help the user.
63		You can execute system commands, fetch data from the web, and generate documentation.
64		When a user asks for something that requires using one of your tools,
65		use the appropriate function to complete the task.
66		Be helpful, concise, and accurate.
67		"""
68
69		# Send message to agent
70		response = agent.chat(
71		message=user_input,
72		system_prompt=system_prompt,
73		context=context
74		)
75
76		# Update conversation context
77		context.append({"role": "user", "content": user_input})
78		context.append({"role": "assistant", "content": response})
79
80		# Limit context size to avoid exceeding token limits
81		if len(context) > 10:
82		# Keep only the last 10 messages
83		context = context[-10:]
84
85		# Display response
86		print(f"\nAssistant: {response}")
87
88		except Exception as e:
89		logger.error(f"Error processing message: {str(e)}")
90		print(f"\nError: {str(e)}")
91
92		except Exception as e:
93		logger.error(f"Error in main function: {str(e)}")
94		print(f"Error: {str(e)}")
95		return 1
96
97		return 0
98
99		if __name__ == "__main__":
100		sys.exit(main())

ollamatools-project/examples/basic_agent_improved.py +0 -128

…	…	@@ -1,128 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Improved version of the basic agent example with better system prompts.
4
5		This example demonstrates how to initialize the OllamaAgent
6		and use it for basic conversation with function calling capabilities,
7		with improved system instructions for better tool usage.
8
9		Usage:
10		python basic_agent_improved.py
11		"""
12
13		import os
14		import sys
15		import logging
16
17		# Add the parent directory to the Python path to import ollamatools
18		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
19
20		from ollamatools import OllamaAgent, Config
21		from ollamatools.utils.logger import setup_logger
22
23		# Set up logging
24		logger = setup_logger("basic_agent_improved", level="INFO")
25
26		def main():
27		"""Run the example."""
28		try:
29		# Initialize the agent with default settings
30		logger.info("Initializing OllamaAgent...")
31		agent = OllamaAgent(
32		model="llama3.1:latest", # Using your available model
33		permission_prompts=True # Ask for permission before executing functions
34		)
35
36		logger.info("Agent initialized. Enter 'exit' to quit.")
37		print("\nOllamaTools Chat Example (Improved)")
38		print("----------------------------------")
39		print("Type 'exit' to quit")
40		print("Try asking the agent to perform tasks like:")
41		print(" - What files are in my current directory?")
42		print(" - Show me the 5 largest Python files")
43		print(" - What's my current working directory?")
44		print(" - Generate a markdown document about climate change")
45		print(" - What's 2+2?\n")
46
47		# Simple chat loop
48		context = []
49
50		while True:
51		# Get user input
52		user_input = input("\nYou: ")
53
54		# Check for exit command
55		if user_input.lower() in ['exit', 'quit', 'bye']:
56		print("\nExiting. Goodbye!")
57		break
58
59		# Process message with function calling
60		try:
61		# Improved system prompt with better tool usage instructions
62		system_prompt = """
63		You are an AI assistant with access to various tools to help the user. You have the following tools available:
64
65		1. System Tools:
66		- execute_command: Execute a shell command. For commands like 'ls', 'pwd', etc., use this tool.
67		- find_files: Find files matching a pattern in a directory. Always use the current directory ('.') as the directory parameter unless specified otherwise.
68		- read_file: Read the contents of a file.
69		- write_file: Write content to a file.
70
71		2. Web Tools:
72		- fetch_url: Fetch content from a URL.
73		- get_weather: Get weather information for a location.
74		- search_web: Search the web for information.
75
76		3. Documentation Tools:
77		- generate_markdown: Generate a formatted markdown document.
78		- extract_context: Extract relevant context from documents.
79		- summarize_text: Generate a summary of text.
80
81		When a user asks for something that requires using one of your tools, use the appropriate function to complete the task.
82		For file operations, always use the current directory ('.') as the directory parameter unless specified otherwise.
83		For executing commands, use the execute_command function with the full command as a string.
84
85		For example, if a user asks "What files are in my current directory?", respond with:
86		1. Use find_files(pattern="*", directory=".", recursive=False)
87		2. Format the results in a readable way
88		3. Explain what you found
89
90		If a user asks "What's my current working directory?", respond with:
91		1. Use execute_command(command="pwd")
92		2. Show the output
93
94		Be helpful, concise, and accurate. When you use a tool, explain what you're doing.
95		"""
96
97		# Send message to agent
98		response = agent.chat(
99		message=user_input,
100		system_prompt=system_prompt,
101		context=context
102		)
103
104		# Update conversation context
105		context.append({"role": "user", "content": user_input})
106		context.append({"role": "assistant", "content": response})
107
108		# Limit context size to avoid exceeding token limits
109		if len(context) > 10:
110		# Keep only the last 10 messages
111		context = context[-10:]
112
113		# Display response
114		print(f"\nAssistant: {response}")
115
116		except Exception as e:
117		logger.error(f"Error processing message: {str(e)}")
118		print(f"\nError: {str(e)}")
119
120		except Exception as e:
121		logger.error(f"Error in main function: {str(e)}")
122		print(f"Error: {str(e)}")
123		return 1
124
125		return 0
126
127		if __name__ == "__main__":
128		sys.exit(main())

ollamatools-project/examples/basic_usage.py +0 -24

…	…	@@ -1,24 +0,0 @@
1		"""
2		Basic usage example for OllamaTools.
3		"""
4
5		from ollamatools import OllamaAgent
6
7		# Create agent with default settings
8		agent = OllamaAgent(model="llama3.1")
9
10		# Chat with automatic function detection
11		response = agent.chat("Hello, what can you do?")
12		print(response)
13
14		# Example of system command execution
15		response = agent.chat("Can you list the files in the current directory?")
16		print(response)
17
18		# Example of web request
19		response = agent.chat("What's the weather like in San Francisco?")
20		print(response)
21
22		# Example of documentation generation
23		response = agent.chat("Create documentation for a function that calculates prime numbers")
24		print(response)

ollamatools-project/examples/command_execution.py +0 -96

…	…	@@ -1,96 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example script demonstrating command execution with OllamaTools.
4
5		This script shows how to use the command execution function with
6		various examples and security features.
7		"""
8
9		import sys
10		import os
11		import time
12
13		# Add the parent directory to the path so we can import the package
14		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
15
16		from ollamatools.tools.system_tools import execute_command
17		from ollamatools.security import PermissionManager
18
19
20		def print_result(result):
21		"""Print command execution result in a formatted way."""
22		print("\nCommand Result:")
23		print(f"Command: {result['command']}")
24		print(f"Exit Code: {result['exit_code']}")
25
26		if result['stdout']:
27		print("\nStandard Output:")
28		print("-" * 40)
29		print(result['stdout'])
30		print("-" * 40)
31
32		if result['stderr']:
33		print("\nStandard Error:")
34		print("-" * 40)
35		print(result['stderr'])
36		print("-" * 40)
37
38
39		def main():
40		print("=== Command Execution Example ===\n")
41
42		# Create a permission manager for secure command execution
43		pm = PermissionManager()
44
45		try:
46		# Example 1: Simple command without arguments
47		print("\n--- Example 1: Simple Command ---")
48		print("Executing: 'pwd'")
49		result = execute_command("pwd", permission_manager=pm)
50		print_result(result)
51
52		# Example 2: Command with arguments
53		print("\n--- Example 2: Command with Arguments ---")
54		print("Executing: 'ls -l'")
55		result = execute_command("ls", ["-l"], permission_manager=pm)
56		print_result(result)
57
58		# Example 3: Command with an error
59		print("\n--- Example 3: Command with Error ---")
60		print("Executing: 'ls --invalid-option'")
61		result = execute_command("ls", ["--invalid-option"], permission_manager=pm)
62		print_result(result)
63
64		# Example 4: Command with timeout
65		print("\n--- Example 4: Command with Timeout ---")
66		print("Executing: 'sleep 10' with a 2 second timeout")
67		try:
68		result = execute_command("sleep", ["10"], timeout=2, permission_manager=pm)
69		print_result(result)
70		except TimeoutError as e:
71		print(f"Caught timeout error: {e}")
72
73		# Example 5: Sudo command
74		print("\n--- Example 5: Sudo Command ---")
75		print("Executing: 'sudo echo \"Hello from root\"'")
76		try:
77		result = execute_command("sudo", ["echo", "Hello from root"], permission_manager=pm)
78		print_result(result)
79		except PermissionError as e:
80		print(f"Permission error: {e}")
81
82		# Example 6: Potentially dangerous command (should be blocked)
83		print("\n--- Example 6: Dangerous Command ---")
84		print("Attempting to execute a dangerous command with shell redirection")
85		try:
86		result = execute_command("echo 'Hello' > /tmp/test.txt", permission_manager=pm)
87		print_result(result)
88		except ValueError as e:
89		print(f"Caught security error: {e}")
90
91		except Exception as e:
92		print(f"An error occurred: {e}")
93
94
95		if __name__ == "__main__":
96		main()

ollamatools-project/examples/complete_example.py +0 -0

(no content changes — binary or rename)

ollamatools-project/examples/config/enhanced_matrix_config.yaml +0 -50

…	…	@@ -1,50 +0,0 @@
1		MATRIX:
2		HOMESERVER: "https://prsm.space"
3		USER: "@chat7rn:prsm.space"
4		PASSWORD: "chat7rn"
5		DEVICE_ID: "ollamatools-matrix"
6		ROOMS:
7		# - "#general:prsm.space"
8		- "#ai_testing:prsm.space"
9		ADMIN_USERS:
10		- "@chat7rn:prsm.space"
11
12		OLLAMA:
13		HOST: "localhost:11434"
14		MODEL: "llama3.1:latest"
15		PROMPT: "You are a helpful assistant in a Matrix chat room. Be concise and friendly in your responses."
16		AVAILABLE_MODELS:
17		- "deepseek-r1:7b"
18		- "llama3.2:1b"
19		- "llama3.1:latest"
20
21		ENHANCED:
22		# Enable search functionality
23		# Note: Requires both OpenSearchNG and modular agent components to be installed
24		# If these components are not available, search will be automatically disabled
25		ENABLE_SEARCH: false
26
27		# Enable conversation tracking and history
28		# This feature works without modular agent components
29		ENABLE_CONVERSATION_TRACKING: true
30
31		# Maximum number of messages to keep in context for each user
32		MAX_CONTEXT_LENGTH: 10
33
34		# Maximum number of conversations to keep in history
35		MAX_CONVERSATION_HISTORY: 100
36
37		# Whether to respond to all messages (not just mentions)
38		# This feature works without modular agent components
39		RESPOND_TO_ALL_MESSAGES: false
40
41		# OpenSearchNG configuration
42		# Note: Only used if search is enabled and OpenSearchNG is available
43		OPENSEARCH:
44		BASE_URL: "http://localhost:8081"
45		ENGINES:
46		- "google"
47		- "bing"
48		- "duckduckgo"
49		- "brave"
50		NUM_RESULTS: 5

ollamatools-project/examples/config/matrix_config.yaml +0 -11

…	…	@@ -1,11 +0,0 @@
1		MATRIX:
2		HOMESERVER: "https://prsm.space"
3		USER: "@chat7rn:prsm.space"
4		PASSWORD: "chat7rn"
5		DEVICE_ID: "ollamatools-matrix"
6		ROOMS:
7		- "#ai_testing:prsm.space"
8		OLLAMA:
9		HOST: "localhost:11434"
10		MODEL: "llama3.1:latest"
11		PROMPT: "You are a helpful assistant in a Matrix chat room. Be concise and friendly in your responses."

ollamatools-project/examples/documentation_tools.py +0 -111

…	…	@@ -1,111 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example script demonstrating the use of documentation tools.
4
5		This script shows how to use the documentation tools to generate
6		formatted documentation in both Markdown and HTML formats, as well
7		as extracting context and summarizing text.
8		"""
9
10		from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
11
12
13		def main():
14		"""Run the documentation tools example."""
15		print("=== OllamaTools Documentation Tools Example ===\n")
16
17		# Part 1: Document Generation
18		print("\n--- Document Generation Example ---\n")
19
20		# Sample content for documentation
21		content = """
22		## Introduction
23
24		This is a sample document generated using OllamaTools' documentation utilities.
25
26		## Features
27
28		- Markdown formatting
29		- HTML conversion
30		- Simple and easy to use
31
32		## Usage
33
34		```python
35		from ollamatools.tools.doc_tools import generate_markdown
36
37		# Generate markdown
38		doc = generate_markdown(content, title, format_type='markdown')
39
40		# Generate HTML
41		html = generate_markdown(content, title, format_type='html')
42		```
43		"""
44
45		# Generate markdown document
46		title = "OllamaTools Documentation Example"
47		markdown_doc = generate_markdown(content, title, format_type='markdown')
48		print("Markdown Document:")
49		print("-----------------")
50		print(markdown_doc[:200] + "...\n")
51
52		# Generate HTML document
53		html_doc = generate_markdown(content, title, format_type='html')
54		print("HTML Document (first 200 chars):")
55		print("--------------------------------")
56		print(html_doc[:200] + "...\n")
57
58		# Save the documents to files
59		with open("example_doc.md", "w") as f:
60		f.write(markdown_doc)
61		print("Saved markdown document to 'example_doc.md'")
62
63		with open("example_doc.html", "w") as f:
64		f.write(html_doc)
65		print("Saved HTML document to 'example_doc.html'\n")
66
67		# Part 2: Context Extraction
68		print("\n--- Context Extraction Example ---\n")
69
70		# Sample documents
71		documents = [
72		"Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.",
73		"JavaScript is a programming language that is one of the core technologies of the World Wide Web, alongside HTML and CSS.",
74		"Rust is a multi-paradigm, general-purpose programming language that emphasizes performance, type safety, and concurrency.",
75		"OllamaTools is a Python library that extends Ollama with semantic function calling capabilities."
76		]
77
78		# Extract context based on a query
79		query = "programming language features"
80		context = extract_context(query, documents)
81
82		print(f"Query: '{query}'")
83		print("Extracted Context:")
84		print("-----------------")
85		print(context)
86		print()
87
88		# Part 3: Text Summarization
89		print("\n--- Text Summarization Example ---\n")
90
91		# Sample text to summarize
92		long_text = """
93		Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically-typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming. It is often described as a "batteries included" language due to its comprehensive standard library.
94
95		Guido van Rossum began working on Python in the late 1980s, as a successor to the ABC programming language, and first released it in 1991 as Python 0.9.0. Python 2.0 was released in 2000 and introduced new features, such as list comprehensions and a garbage collection system using reference counting. Python 3.0 was released in 2008 and was a major revision of the language that is not completely backward-compatible. Python 2 was discontinued with version 2.7.18 in 2020.
96
97		Python consistently ranks as one of the most popular programming languages. Python is used in many application domains, such as web development, artificial intelligence, machine learning, data science, scientific computing, computer vision, computational statistics, computational biology, computer-aided design, computational economics, computational physics, SEO, automation, and information security.
98		"""
99
100		# Summarize the text
101		summary = summarize_text(long_text, max_length=200)
102
103		print("Original Text Length:", len(long_text))
104		print("Summary Length:", len(summary))
105		print("Summary:")
106		print("--------")
107		print(summary)
108
109
110		if __name__ == "__main__":
111		main()

ollamatools-project/examples/enhanced_matrix_client_example.py +0 -330

…	…	@@ -1,330 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Enhanced Matrix Client Example for OllamaTools
4
5		This example demonstrates how to use the enhanced Matrix integration in OllamaTools
6		to connect to a Matrix homeserver, join rooms, and process messages with Ollama.
7		It showcases advanced features like improved message response logic, search integration,
8		conversation tracking, and more.
9
10		Requirements:
11		- matrix-nio: `pip install matrix-nio[e2e]`
12		- ollama: `pip install ollama`
13		- pyyaml: `pip install pyyaml`
14
15		Optional Dependencies (for advanced features):
16		- ollamatools.agent.modular: For search integration and natural language understanding
17		- ollamatools.opensearch_web_tools: For web search functionality
18
19		Note: If the modular agent components are not available, some advanced features
20		like search integration will be automatically disabled, but the client will still
21		function with basic features like improved message response logic and conversation tracking.
22
23		Usage:
24		1. Edit the configuration below or create a config.yaml file
25		2. Run the script: `python enhanced_matrix_client_example.py`
26		"""
27
28		import os
29		import sys
30		import yaml
31		import logging
32		import asyncio
33		import argparse
34		import random
35		from typing import Dict, List, Optional
36
37		# Add the parent directory to the path so we can import ollamatools
38		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
39
40		from ollamatools.tools.enhanced_matrix_tools import (
41		EnhancedMatrixClient,
42		connect_enhanced_matrix,
43		create_enhanced_matrix_config,
44		load_matrix_config,
45		save_matrix_config,
46		disconnect_enhanced_matrix,
47		MODULAR_AGENT_AVAILABLE,
48		OPENSEARCH_AVAILABLE
49		)
50
51		# Configure logging
52		logging.basicConfig(
53		level=logging.INFO,
54		format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
55		)
56		logger = logging.getLogger(__name__)
57
58		# Default configuration file path
59		DEFAULT_CONFIG_PATH = os.path.join(os.path.dirname(__file__), 'config', 'enhanced_matrix_config.yaml')
60
61		def parse_arguments():
62		"""Parse command line arguments."""
63		parser = argparse.ArgumentParser(description='Enhanced Matrix client example for OllamaTools')
64		parser.add_argument(
65		'-c', '--config',
66		help='Path to the configuration file',
67		default=DEFAULT_CONFIG_PATH
68		)
69		parser.add_argument(
70		'--create-config',
71		help='Create a default configuration file',
72		action='store_true'
73		)
74		parser.add_argument(
75		'-d', '--debug',
76		help='Enable debug logging',
77		action='store_true'
78		)
79		parser.add_argument(
80		'--update-models',
81		help='Update available models from Ollama server',
82		action='store_true'
83		)
84		return parser.parse_args()
85
86		def create_config_file(config_path: str) -> None:
87		"""Create a default configuration file."""
88		config = create_enhanced_matrix_config()
89
90		# Create the directory if it doesn't exist
91		os.makedirs(os.path.dirname(config_path), exist_ok=True)
92
93		# Save the configuration
94		save_matrix_config(config, config_path)
95
96		logger.info(f"Created default configuration file at {config_path}")
97		logger.info("Please edit this file with your Matrix credentials and Ollama settings.")
98
99		sys.exit(0)
100
101		def load_config(config_path: str) -> Dict:
102		"""Load the configuration from a file."""
103		try:
104		config = load_matrix_config(config_path)
105		return config
106		except FileNotFoundError:
107		logger.error(f"Configuration file not found: {config_path}")
108		logger.info("You can create a default configuration file with --create-config")
109		sys.exit(1)
110		except Exception as e:
111		logger.error(f"Error loading configuration: {e}")
112		sys.exit(1)
113
114		async def custom_message_handler(room, event, client):
115		"""
116		Custom message handler for Matrix events.
117
118		This is an example of how to create a custom handler for Matrix messages.
119		It processes the message with Ollama and sends the response back to the room.
120
121		Args:
122		room: The Matrix room where the message was received
123		event: The Matrix event (message)
124		client: The MatrixClient instance
125		"""
126		message = event.body
127		room_id = room.room_id
128		sender = event.sender
129
130		logger.info(f"Custom handler received message in room {room_id} from {sender}: {message}")
131
132		# Process the message with Ollama, passing sender and room_id for context tracking
133		response = await client.process_with_ollama(message, sender, room_id)
134
135		# Add a prefix to the response
136		response = f"Response from Ollama: {response}"
137
138		# Send the response back to the room
139		await client.send_message(room_id, response)
140
141		def get_random_greeting() -> str:
142		"""Generate a random greeting message."""
143		greetings = [
144		"Hello everyone! I'm here to assist with your questions.",
145		"Greetings! I'm your friendly Matrix bot powered by Ollama.",
146		"Hi there! I'm online and ready to chat.",
147		"Good day! I'm here to help with anything you need.",
148		"Hey folks! I've just connected and I'm ready to assist.",
149		"Hello! I'm your AI assistant, ready to help with your questions.",
150		"Greetings, humans! AI assistant at your service.",
151		"Hi everyone! I'm here and ready to chat.",
152		"Hello Matrix world! How can I assist you today?",
153		"Greetings! Your friendly neighborhood AI has arrived."
154		]
155		return random.choice(greetings)
156
157		async def main_async(config: Dict, update_models: bool = False) -> None:
158		"""
159		Main async function to run the Enhanced Matrix client.
160
161		Args:
162		config: The configuration dictionary
163		update_models: Whether to update available models from Ollama server
164		"""
165		# Create a client variable in the outer scope so we can access it in the finally block
166		client = None
167		# Extract configuration
168		matrix_config = config.get('MATRIX', {})
169		ollama_config = config.get('OLLAMA', {})
170		enhanced_config = config.get('ENHANCED', {})
171
172		homeserver = matrix_config.get('HOMESERVER')
173		user_id = matrix_config.get('USER')
174		password = matrix_config.get('PASSWORD')
175		device_id = matrix_config.get('DEVICE_ID')
176		rooms = matrix_config.get('ROOMS', [])
177		admin_users = matrix_config.get('ADMIN_USERS', [])
178
179		ollama_host = ollama_config.get('HOST', 'localhost:11434')
180		ollama_model = ollama_config.get('MODEL', 'llama3:latest')
181		ollama_prompt = ollama_config.get('PROMPT')
182		available_models = ollama_config.get('AVAILABLE_MODELS', [ollama_model])
183
184		# Extract enhanced configuration
185		enable_search = enhanced_config.get('ENABLE_SEARCH', True)
186		enable_conversation_tracking = enhanced_config.get('ENABLE_CONVERSATION_TRACKING', True)
187		max_context_length = enhanced_config.get('MAX_CONTEXT_LENGTH', 10)
188		max_conversation_history = enhanced_config.get('MAX_CONVERSATION_HISTORY', 100)
189		respond_to_all_messages = enhanced_config.get('RESPOND_TO_ALL_MESSAGES', False)
190		opensearch_config = enhanced_config.get('OPENSEARCH', None)
191
192		try:
193		# Create and connect the Enhanced Matrix client
194		client = EnhancedMatrixClient(
195		homeserver=homeserver,
196		user_id=user_id,
197		password=password,
198		device_id=device_id,
199		ollama_host=ollama_host,
200		ollama_model=ollama_model,
201		ollama_prompt=ollama_prompt,
202		available_models=available_models,
203		admin_users=admin_users,
204		opensearch_config=opensearch_config,
205		enable_search=enable_search,
206		enable_conversation_tracking=enable_conversation_tracking,
207		max_context_length=max_context_length,
208		max_conversation_history=max_conversation_history,
209		respond_to_all_messages=respond_to_all_messages
210		)
211
212		# Connect to the Matrix homeserver
213		success = await client.connect()
214
215		if not success:
216		logger.error("Failed to connect to Matrix homeserver")
217		return
218
219		# Update available models if requested
220		if update_models:
221		logger.info("Updating available models from Ollama server...")
222		await client.update_available_models()
223		logger.info(f"Available models: {', '.join(client.available_models)}")
224
225		# Join rooms and track successful joins
226		successful_joins = []
227		for room in rooms:
228		if isinstance(room, dict) and 'ROOM' in room:
229		room_id = room['ROOM']
230		else:
231		room_id = room
232
233		join_success = await client.join_room(room_id)
234		if join_success:
235		successful_joins.append(room_id)
236		logger.info(f"Successfully joined room {room_id}")
237
238		# Send a single greeting to the first successfully joined room
239		if successful_joins:
240		first_room = successful_joins[0]
241		greeting = get_random_greeting()
242		await client.send_message(first_room, greeting)
243		logger.info(f"Sent greeting to room {first_room}: {greeting}")
244
245		# Optional: Set a custom message handler
246		# client.set_message_callback(custom_message_handler)
247
248		# Start listening for messages
249		logger.info("Starting to listen for messages...")
250		logger.info("The bot will respond to the following commands:")
251		logger.info(" /help - Show help message")
252		logger.info(" /models - List available models")
253		logger.info(" /model <model_name> - Change to a different model")
254
255		# Log enhanced commands if enabled
256		if enable_search:
257		if MODULAR_AGENT_AVAILABLE and OPENSEARCH_AVAILABLE:
258		logger.info(" /search <query> - Search the web for information")
259		else:
260		logger.warning("Search functionality is disabled because required components are not available")
261		logger.info("To enable search functionality, make sure the modular agent components and OpenSearchNG are available")
262
263		if enable_conversation_tracking:
264		logger.info(" /history [count] - Show recent conversation history")
265		logger.info(" /topics - Show conversation topics")
266		logger.info(" /metrics - Show performance metrics")
267
268		# Extract username for logging
269		username = user_id.split(':')[0].lstrip('@')
270		logger.info(f"The bot will also respond when mentioned by username: {username}")
271
272		if respond_to_all_messages:
273		logger.info("The bot will respond to all messages (not just mentions)")
274
275		# Log availability of advanced features
276		if not MODULAR_AGENT_AVAILABLE:
277		if enhanced_config.get('ENABLE_SEARCH', True):
278		logger.warning("Modular agent components are not available. Some advanced features will be disabled.")
279		else:
280		logger.info("Modular agent components are not available, but search is disabled in config.")
281		logger.info("Basic features like improved message response logic and conversation tracking will still work.")
282
283		if not OPENSEARCH_AVAILABLE:
284		logger.warning("OpenSearchNG is not available. Search functionality will be disabled.")
285
286		# Print instructions for stopping the client
287		logger.info("Press Ctrl+C to stop the client")
288
289		# Start listening for messages
290		await client.listen()
291		except KeyboardInterrupt:
292		logger.info("Interrupted by user")
293		except Exception as e:
294		logger.error(f"Error in main_async: {e}")
295		finally:
296		# Clean up resources
297		if client:
298		logger.info("Disconnecting from Matrix homeserver...")
299		try:
300		await client.disconnect()
301		logger.info("Disconnected from Matrix homeserver")
302		except Exception as e:
303		logger.error(f"Error disconnecting from Matrix homeserver: {e}")
304
305		def main():
306		"""Main function to run the Matrix client."""
307		args = parse_arguments()
308
309		# Set log level
310		if args.debug:
311		logging.getLogger().setLevel(logging.DEBUG)
312
313		# Create a default configuration file if requested
314		if args.create_config:
315		create_config_file(args.config)
316
317		# Load the configuration
318		config = load_config(args.config)
319
320		# Run the async main function
321		try:
322		# Use asyncio.run to handle the event loop properly
323		asyncio.run(main_async(config, args.update_models))
324		except KeyboardInterrupt:
325		logger.info("Interrupted by user")
326		except Exception as e:
327		logger.error(f"Error: {e}")
328
329		if __name__ == '__main__':
330		main()

ollamatools-project/examples/enhanced_permission_example.py +0 -106

…	…	@@ -1,106 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example script demonstrating the enhanced permission system.
4
5		This script shows how to use the interactive permission prompts
6		and the session-based permission system.
7		"""
8
9		import sys
10		import os
11		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
12
13		from ollamatools.security import PermissionManager
14
15
16		def simulate_function_execution(function_name, args, is_sudo=False):
17		"""Simulate the execution of a function."""
18		if is_sudo:
19		print(f"[SIMULATED] Executing sudo operation: {function_name}")
20		else:
21		print(f"[SIMULATED] Executing: {function_name}")
22
23		print(f"[SIMULATED] Arguments: {args}")
24		print(f"[SIMULATED] Operation completed successfully")
25		print()
26
27
28		def main():
29		# Create a permission manager with default settings
30		pm = PermissionManager()
31
32		print("\n=== Enhanced Permission System Example ===\n")
33		print("This example demonstrates the interactive permission system.")
34		print("You will be prompted to allow or deny various operations.")
35		print("Try different responses (A, S, M, D) to see how they work.")
36		print("\nPress Enter to continue...")
37		input()
38
39		# Example 1: Basic function permission
40		print("\n--- Example 1: Basic Function Permission ---")
41		print("A simple read operation that requires permission.")
42		if pm.request_permission("read_file", {"path": "/etc/hosts"}):
43		simulate_function_execution("read_file", {"path": "/etc/hosts"})
44		else:
45		print("Operation denied by user.")
46
47		# Example 2: Sudo command permission
48		print("\n--- Example 2: Sudo Command Permission ---")
49		print("An operation that requires elevated permissions (sudo).")
50		if pm.request_permission("execute_command", {"command": "sudo apt update"}, is_sudo=True):
51		simulate_function_execution("execute_command", {"command": "sudo apt update"}, is_sudo=True)
52		else:
53		print("Sudo operation denied by user.")
54
55		# Example 3: Session permission
56		print("\n--- Example 3: Session Permission ---")
57		print("Try using the 'S' option to allow for session.")
58		print("The same operation will then be allowed without prompting again.")
59
60		# First request - will prompt
61		print("\nFirst request:")
62		if pm.request_permission("read_config", {"file": "settings.json"}):
63		simulate_function_execution("read_config", {"file": "settings.json"})
64		else:
65		print("Operation denied by user.")
66
67		# Second request - may not prompt depending on user's choice
68		print("\nSecond request (same operation):")
69		if pm.request_permission("read_config", {"file": "settings.json"}):
70		simulate_function_execution("read_config", {"file": "settings.json"})
71		else:
72		print("Operation denied by user.")
73
74		# Example 4: Modify operation
75		print("\n--- Example 4: Modify Operation ---")
76		print("Try using the 'M' option to modify the operation before execution.")
77
78		if pm.request_permission("write_file", {"path": "/tmp/example.txt", "content": "Hello, World!"}):
79		simulate_function_execution("write_file", {"path": "/tmp/example.txt", "content": "Hello, World!"})
80		else:
81		print("Operation denied or modification cancelled by user.")
82
83		# Example 5: Disabled prompts but sudo always prompts
84		print("\n--- Example 5: Disabled Prompts but Sudo Always Prompts ---")
85		print("Creating a permission manager with regular prompts disabled but sudo prompts enabled.")
86
87		pm_custom = PermissionManager(prompt_enabled=False, sudo_always_prompt=True)
88
89		# Regular operation - should not prompt
90		print("\nRegular operation (should execute without prompting):")
91		if pm_custom.request_permission("list_files", {"directory": "/tmp"}):
92		simulate_function_execution("list_files", {"directory": "/tmp"})
93		else:
94		print("Operation denied.")
95
96		# Sudo operation - should still prompt
97		print("\nSudo operation (should still prompt):")
98		if pm_custom.request_permission("execute_command", {"command": "sudo systemctl status"}, is_sudo=True):
99		simulate_function_execution("execute_command", {"command": "sudo systemctl status"}, is_sudo=True)
100		else:
101		print("Sudo operation denied by user.")
102
103
104		if __name__ == "__main__":
105		main()
106

ollamatools-project/examples/error_handling.py +0 -323

…	…	@@ -1,323 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example demonstrating error handling in OllamaTools.
4		"""
5
6		import os
7		import sys
8		import time
9		from typing import Dict, Any, List, Optional
10
11		# Add the parent directory to sys.path for imports
12		parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
13		if parent_dir not in sys.path:
14		sys.path.insert(0, parent_dir)
15
16		from ollamatools.security import PermissionManager
17		from ollamatools.tools.system_tools import execute_command, read_file, write_file
18		from ollamatools.tools.web_tools import fetch_url
19		from ollamatools.tools.doc_tools import generate_markdown
20		from ollamatools.utils.logger import setup_logger
21
22		# Setup logging
23		logger = setup_logger("error_handling", level="DEBUG")
24
25		def handle_permission_errors():
26		"""
27		Demonstrate handling permission errors.
28		"""
29		logger.info("=== Permission Error Handling Demo ===")
30
31		# Create a permission manager that will deny all requests
32		permission_manager = PermissionManager(prompt_enabled=True)
33
34		try:
35		# Attempt to execute a command with a permission manager that will prompt
36		print("This will display a permission prompt. Enter 'n' to deny permission.")
37		result = execute_command(
38		command="ls",
39		args=["-la"],
40		permission_manager=permission_manager
41		)
42
43		logger.info(f"Command result: {result}")
44		except Exception as e:
45		logger.error(f"Error executing command: {str(e)}")
46
47		logger.info("Permission error handling completed")
48
49		def handle_command_execution_errors():
50		"""
51		Demonstrate handling command execution errors.
52		"""
53		logger.info("\n=== Command Execution Error Handling Demo ===")
54
55		# Create a permission manager with prompts disabled for demo
56		permission_manager = PermissionManager(prompt_enabled=False)
57
58		# Invalid command
59		logger.info("Executing an invalid command...")
60		result = execute_command(
61		command="nonexistent_command",
62		permission_manager=permission_manager
63		)
64
65		if "error" in result:
66		logger.error(f"Command error: {result['error']}")
67		else:
68		logger.info(f"Command result: {result}")
69
70		# Command with very short timeout
71		logger.info("\nExecuting a command with a very short timeout...")
72		result = execute_command(
73		command="sleep",
74		args=["10"],
75		timeout=1,
76		permission_manager=permission_manager
77		)
78
79		if "error" in result:
80		logger.error(f"Timeout error: {result['error']}")
81		else:
82		logger.info(f"Command result: {result}")
83
84		logger.info("Command execution error handling completed")
85
86		def handle_file_operation_errors():
87		"""
88		Demonstrate handling file operation errors.
89		"""
90		logger.info("\n=== File Operation Error Handling Demo ===")
91
92		# Create a permission manager with prompts disabled for demo
93		permission_manager = PermissionManager(prompt_enabled=False)
94
95		# Read a non-existent file
96		non_existent_file = "non_existent_file.txt"
97
98		logger.info(f"Reading non-existent file: {non_existent_file}...")
99		try:
100		content = read_file(
101		path=non_existent_file,
102		permission_manager=permission_manager
103		)
104		logger.info(f"File content: {content}")
105		except Exception as e:
106		logger.error(f"Error reading file: {str(e)}")
107
108		# Write to a directory (which should fail)
109		logger.info("\nWriting to a directory (should fail)...")
110		try:
111		success = write_file(
112		path=".", # Current directory
113		content="This should fail",
114		permission_manager=permission_manager
115		)
116		logger.info(f"Write success: {success}")
117		except Exception as e:
118		logger.error(f"Error writing to directory: {str(e)}")
119
120		# Write to a file with invalid permissions
121		if os.name == "posix": # Unix-like systems only
122		restricted_file = "/etc/test_file.txt"
123
124		logger.info(f"\nWriting to a restricted file: {restricted_file}...")
125		try:
126		success = write_file(
127		path=restricted_file,
128		content="This should fail",
129		permission_manager=permission_manager
130		)
131		logger.info(f"Write success: {success}")
132		except Exception as e:
133		logger.error(f"Error writing to restricted file: {str(e)}")
134
135		logger.info("File operation error handling completed")
136
137		def handle_web_request_errors():
138		"""
139		Demonstrate handling web request errors.
140		"""
141		logger.info("\n=== Web Request Error Handling Demo ===")
142
143		# Create a permission manager with prompts disabled for demo
144		permission_manager = PermissionManager(prompt_enabled=False)
145
146		# Invalid URL
147		invalid_url = "http://nonexistent.domain.123"
148
149		logger.info(f"Fetching invalid URL: {invalid_url}...")
150		try:
151		result = fetch_url(
152		url=invalid_url,
153		permission_manager=permission_manager
154		)
155		logger.info(f"Result: {result}")
156		except Exception as e:
157		logger.error(f"Error fetching invalid URL: {str(e)}")
158
159		# URL with very short timeout
160		logger.info("\nFetching URL with very short timeout...")
161		try:
162		result = fetch_url(
163		url="https://example.com",
164		timeout=0.001, # Very short timeout
165		permission_manager=permission_manager
166		)
167		logger.info(f"Result: {result}")
168		except Exception as e:
169		logger.error(f"Error fetching URL with short timeout: {str(e)}")
170
171		logger.info("Web request error handling completed")
172
173		def handle_doc_tools_errors():
174		"""
175		Demonstrate handling documentation tools errors.
176		"""
177		logger.info("\n=== Documentation Tools Error Handling Demo ===")
178
179		# Create a permission manager with prompts disabled for demo
180		permission_manager = PermissionManager(prompt_enabled=False)
181
182		# Generate markdown with empty content
183		logger.info("Generating markdown with empty content...")
184		try:
185		result = generate_markdown(
186		content="",
187		title="Empty Document",
188		permission_manager=permission_manager
189		)
190		logger.info(f"Generated markdown: {result}")
191		except Exception as e:
192		logger.error(f"Error generating markdown: {str(e)}")
193
194		# Extract context with empty documents
195		logger.info("\nExtracting context with empty documents...")
196		try:
197		context = extract_context(
198		query="test",
199		documents=[],
200		permission_manager=permission_manager
201		)
202		logger.info(f"Extracted context: {context}")
203		except Exception as e:
204		logger.error(f"Error extracting context: {str(e)}")
205
206		# Summarize empty text
207		logger.info("\nSummarizing empty text...")
208		try:
209		summary = summarize_text(
210		text="",
211		permission_manager=permission_manager
212		)
213		logger.info(f"Summary: {summary}")
214		except Exception as e:
215		logger.error(f"Error summarizing text: {str(e)}")
216
217		logger.info("Documentation tools error handling completed")
218
219		def handle_custom_errors():
220		"""
221		Demonstrate handling custom errors with try-except blocks.
222		"""
223		logger.info("\n=== Custom Error Handling Demo ===")
224
225		# Wrap command execution in a try-except block
226		logger.info("Custom error handling for command execution...")
227		try:
228		# This will fail because the command doesn't exist
229		result = execute_command(
230		command="nonexistent_command",
231		permission_manager=PermissionManager(prompt_enabled=False)
232		)
233
234		# Check for error in the result
235		if "error" in result:
236		logger.warning(f"Command failed: {result['error']}")
237		print("Showing user-friendly error message instead of technical details.")
238		print("The command could not be executed. Please check if it's installed.")
239		else:
240		logger.info(f"Command succeeded: {result}")
241		except Exception as e:
242		logger.error(f"Unexpected error: {str(e)}")
243		print("An unexpected error occurred. Please try again later.")
244
245		# Create a function that retries web requests
246		def fetch_with_retry(url, max_retries=3, delay=1):
247		"""Fetch a URL with retries."""
248		permission_manager = PermissionManager(prompt_enabled=False)
249
250		for attempt in range(max_retries):
251		try:
252		logger.info(f"Attempt {attempt + 1}/{max_retries} to fetch {url}")
253		result = fetch_url(
254		url=url,
255		permission_manager=permission_manager
256		)
257		return result
258		except Exception as e:
259		logger.warning(f"Attempt {attempt + 1} failed: {str(e)}")
260		if attempt < max_retries - 1:
261		logger.info(f"Retrying in {delay} seconds...")
262		time.sleep(delay)
263		delay *= 2 # Exponential backoff
264		else:
265		logger.error("All retry attempts failed")
266		raise
267
268		# Test the retry function
269		logger.info("\nTesting fetch with retry (with invalid URL)...")
270		try:
271		result = fetch_with_retry("http://nonexistent.domain.123")
272		logger.info(f"Result: {result}")
273		except Exception as e:
274		logger.error(f"Final error after retries: {str(e)}")
275
276		logger.info("Custom error handling completed")
277
278		def main():
279		"""Run the error handling examples."""
280		logger.info("OllamaTools Error Handling Examples")
281		logger.info("==================================")
282
283		try:
284		# Handle permission errors
285		handle_permission_errors()
286
287		# Handle command execution errors
288		handle_command_execution_errors()
289
290		# Handle file operation errors
291		handle_file_operation_errors()
292
293		# Handle web request errors
294		handle_web_request_errors()
295
296		# Handle documentation tools errors
297		handle_doc_tools_errors()
298
299		# Handle custom errors
300		handle_custom_errors()
301
302		logger.info("\n=== Error Handling Examples Complete ===")
303		except Exception as e:
304		logger.error(f"Unexpected error in examples: {str(e)}")
305		return 1
306
307		return 0
308
309		if __name__ == "__main__":
310		# Import extract_context and summarize_text here to avoid import errors if they're not implemented
311		try:
312		from ollamatools.tools.doc_tools import extract_context, summarize_text
313		except ImportError:
314		# Define placeholder functions if not implemented
315		, documents, max_length=500):
316		"""Placeholder for extract_context if not implemented."""
317		return f"Context for query '{query}' (placeholder)"
318
319		def summarize_text(text, max_length=200):
320		"""Placeholder for summarize_text if not implemented."""
321		return "Summary placeholder"
322
323		sys.exit(main())

ollamatools-project/examples/extended_modular_agent.py +0 -309

…	…	@@ -1,309 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Extended ModularOllamaAgent Example.
4
5		This script demonstrates how to extend the ModularOllamaAgent
6		with custom functionality like conversation history tracking,
7		conversation summarization, and topic analysis.
8		"""
9
10		import os
11		import sys
12		import json
13		import time
14		import logging
15		from typing import Dict, Any, List, Optional
16
17		# Import the ModularOllamaAgent
18		from ollamatools.agent import ModularOllamaAgent
19		from ollamatools.agent.modular import (
20		print_status, print_success, print_info, print_error
21		)
22		from ollamatools.opensearch_web_tools import DEFAULT_OPENSEARCH_CONFIG
23		from ollamatools.command_validator import DEFAULT_ALLOWLIST
24		from ollamatools.utils.logger import setup_logger
25
26		# Set up logging
27		logger = setup_logger("extended_modular_agent", level="INFO")
28
29		class ExtendedModularAgent(ModularOllamaAgent):
30		"""
31		Extended version of the ModularOllamaAgent with additional features.
32
33		This class demonstrates how to extend the ModularOllamaAgent with custom
34		functionality while leveraging the existing modular architecture.
35		"""
36
37		def __init__(self, *args, **kwargs):
38		"""Initialize the extended agent."""
39		# Initialize the base class
40		super().__init__(*args, **kwargs)
41
42		# Add custom attributes
43		self.conversation_history_file = os.path.join(
44		os.path.expanduser('~'),
45		'.ollamatools',
46		'conversation_history.json'
47		)
48		self.conversation_history = []
49
50		# Load conversation history if it exists
51		self._load_conversation_history()
52
53		print_info("Extended agent initialized with conversation history tracking")
54
55		def _load_conversation_history(self) -> None:
56		"""Load conversation history from file."""
57		if os.path.exists(self.conversation_history_file):
58		try:
59		with open(self.conversation_history_file, 'r') as f:
60		self.conversation_history = json.load(f)
61		print_info(f"Loaded {len(self.conversation_history)} conversations from history")
62		except Exception as e:
63		print_error(f"Failed to load conversation history: {str(e)}")
64		self.conversation_history = []
65
66		def _save_conversation_history(self) -> None:
67		"""Save conversation history to file."""
68		try:
69		# Create directory if it doesn't exist
70		os.makedirs(os.path.dirname(self.conversation_history_file), exist_ok=True)
71
72		with open(self.conversation_history_file, 'w') as f:
73		json.dump(self.conversation_history, f, indent=2)
74		print_info(f"Saved {len(self.conversation_history)} conversations to history")
75		except Exception as e:
76		print_error(f"Failed to save conversation history: {str(e)}")
77
78		def chat(self, message: str) -> str:
79		"""
80		Enhanced chat method that tracks conversation history.
81
82		Args:
83		message: User message
84
85		Returns:
86		str: Response message
87		"""
88		# Record start time for performance tracking
89		start_time = time.time()
90
91		# Process the message using the parent class method
92		response = super().chat(message)
93
94		# Calculate response time
95		response_time = time.time() - start_time
96
97		# Add to conversation history
98		conversation = {
99		"timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
100		"user_message": message,
101		"assistant_response": response,
102		"response_time": response_time,
103		"tokens_estimated": len(message.split()) + len(response.split())
104		}
105
106		self.conversation_history.append(conversation)
107		self._save_conversation_history()
108
109		return response
110
111		def summarize_conversations(self, n: int = 5) -> str:
112		"""
113		Summarize recent conversations.
114
115		Args:
116		n: Number of recent conversations to summarize
117
118		Returns:
119		str: Summary of recent conversations
120		"""
121		if not self.conversation_history:
122		return "No conversation history available."
123
124		# Get the most recent conversations
125		recent = self.conversation_history[-n:] if len(self.conversation_history) >= n else self.conversation_history
126
127		# Create a summary
128		summary = ["Recent Conversations:"]
129		for i, conv in enumerate(recent, 1):
130		summary.append(f"\n{i}. {conv['timestamp']}")
131		summary.append(f"User: {conv['user_message'][:100]}..." if len(conv['user_message']) > 100 else f"User: {conv['user_message']}")
132		summary.append(f"Response time: {conv['response_time']:.2f}s")
133
134		return "\n".join(summary)
135
136		def analyze_conversation_topics(self) -> Dict[str, int]:
137		"""
138		Analyze common topics in conversation history.
139
140		Returns:
141		Dict[str, int]: Dictionary of topics and their frequency
142		"""
143		if not self.conversation_history:
144		return {}
145
146		# Simple keyword-based topic analysis
147		topics = {
148		"help": 0,
149		"search": 0,
150		"file": 0,
151		"command": 0,
152		"code": 0,
153		"error": 0,
154		"issue": 0,
155		"question": 0,
156		"other": 0
157		}
158
159		for conv in self.conversation_history:
160		message = conv["user_message"].lower()
161
162		# Check for keywords
163		if "help" in message or "how" in message:
164		topics["help"] += 1
165		if "search" in message or "find" in message or "look" in message:
166		topics["search"] += 1
167		if "file" in message or "read" in message or "write" in message:
168		topics["file"] += 1
169		if "command" in message or "run" in message or "execute" in message:
170		topics["command"] += 1
171		if "code" in message or "program" in message or "script" in message:
172		topics["code"] += 1
173		if "error" in message or "bug" in message or "fix" in message:
174		topics["error"] += 1
175		if "issue" in message or "problem" in message:
176		topics["issue"] += 1
177		if "?" in message:
178		topics["question"] += 1
179
180		# If no keywords matched, count as other
181		if not any(count > 0 for count in topics.values()):
182		topics["other"] += 1
183
184		# Remove topics with zero count
185		return {k: v for k, v in topics.items() if v > 0}
186
187		def get_performance_metrics(self) -> Dict[str, Any]:
188		"""
189		Get performance metrics from conversation history.
190
191		Returns:
192		Dict[str, Any]: Performance metrics
193		"""
194		if not self.conversation_history:
195		return {"error": "No conversation history available"}
196
197		# Calculate metrics
198		total_conversations = len(self.conversation_history)
199		total_response_time = sum(conv["response_time"] for conv in self.conversation_history)
200		avg_response_time = total_response_time / total_conversations if total_conversations > 0 else 0
201
202		# Calculate token usage
203		total_tokens = sum(conv.get("tokens_estimated", 0) for conv in self.conversation_history)
204		avg_tokens_per_conversation = total_tokens / total_conversations if total_conversations > 0 else 0
205
206		# Find slowest and fastest responses
207		slowest = max(self.conversation_history, key=lambda x: x["response_time"]) if self.conversation_history else None
208		fastest = min(self.conversation_history, key=lambda x: x["response_time"]) if self.conversation_history else None
209
210		return {
211		"total_conversations": total_conversations,
212		"total_response_time": total_response_time,
213		"avg_response_time": avg_response_time,
214		"total_tokens_estimated": total_tokens,
215		"avg_tokens_per_conversation": avg_tokens_per_conversation,
216		"slowest_response": {
217		"time": slowest["response_time"] if slowest else 0,
218		"message": slowest["user_message"][:50] + "..." if slowest and len(slowest["user_message"]) > 50 else slowest["user_message"] if slowest else ""
219		},
220		"fastest_response": {
221		"time": fastest["response_time"] if fastest else 0,
222		"message": fastest["user_message"][:50] + "..." if fastest and len(fastest["user_message"]) > 50 else fastest["user_message"] if fastest else ""
223		}
224		}
225
226
227		def main():
228		"""Run the extended ModularOllamaAgent."""
229		try:
230		# Initialize the extended agent
231		logger.info("Initializing ExtendedModularAgent...")
232		agent = ExtendedModularAgent(
233		model="llama3.1:latest",
234		permission_prompts=True
235		)
236
237		logger.info("Agent initialized. Enter 'exit' to quit.")
238		print("\nExtended ModularOllamaAgent")
239		print("===========================")
240		print("Type 'exit' to quit")
241		print("Type 'commands' to see available commands")
242		print("Type 'history' to see execution history")
243		print("Type 'conversations' to see recent conversations")
244		print("Type 'analyze' to analyze conversation topics")
245		print("Type 'metrics' to see performance metrics")
246
247		# Simple chat loop
248		while True:
249		# Get user input
250		user_input = input("\nYou: ").strip()
251
252		# Check for exit command
253		if user_input.lower() in ['exit', 'quit', 'bye']:
254		print("\nExiting. Goodbye!")
255		break
256
257		# Handle special commands
258		if user_input.lower() == 'commands':
259		agent.list_allowed_commands()
260		continue
261
262		if user_input.lower() == 'history':
263		agent.show_execution_history()
264		continue
265
266		if user_input.lower() == 'conversations':
267		summary = agent.summarize_conversations()
268		print(summary)
269		continue
270
271		if user_input.lower() == 'analyze':
272		topics = agent.analyze_conversation_topics()
273		if topics:
274		print("\nConversation Topic Analysis:")
275		for topic, count in sorted(topics.items(), key=lambda x: x[1], reverse=True):
276		print(f" {topic}: {count}")
277		else:
278		print("\nNo conversation data to analyze.")
279		continue
280
281		if user_input.lower() == 'metrics':
282		metrics = agent.get_performance_metrics()
283		if "error" in metrics:
284		print(f"\n{metrics['error']}")
285		else:
286		print("\nPerformance Metrics:")
287		print(f" Total conversations: {metrics['total_conversations']}")
288		print(f" Average response time: {metrics['avg_response_time']:.2f}s")
289		print(f" Estimated total tokens: {metrics['total_tokens_estimated']}")
290		print(f" Average tokens per conversation: {metrics['avg_tokens_per_conversation']:.1f}")
291		print(f" Slowest response: {metrics['slowest_response']['time']:.2f}s for '{metrics['slowest_response']['message']}'")
292		print(f" Fastest response: {metrics['fastest_response']['time']:.2f}s for '{metrics['fastest_response']['message']}'")
293		continue
294
295		# Process as normal chat message
296		response = agent.chat(user_input)
297		print(f"\nAssistant: {response}")
298
299		except KeyboardInterrupt:
300		print("\n\nExiting due to keyboard interrupt.")
301		except Exception as e:
302		logger.error(f"Unexpected error in main function: {str(e)}")
303		print(f"\nUnexpected error: {str(e)}")
304		print("Check the logs for details.")
305
306		return 0
307
308		if __name__ == "__main__":
309		sys.exit(main())

ollamatools-project/examples/extended_transparent_agent.py +0 -254

…	…	@@ -1,254 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Extended Modular Ollama Agent Example.
4
5		This script demonstrates how to extend the ModularOllamaAgent
6		with custom functionality.
7		"""
8
9		import os
10		import sys
11		import json
12		import time
13		import logging
14		from typing import Dict, Any, List, Optional
15
16		# Import the modular agent
17		from ollamatools.agent import ModularOllamaAgent
18		from ollamatools.agent.modular import (
19		print_status, print_success, print_info, print_error
20		)
21		from ollamatools.opensearch_web_tools import DEFAULT_OPENSEARCH_CONFIG
22		from ollamatools.command_validator import DEFAULT_ALLOWLIST
23		from ollamatools.utils.logger import setup_logger
24
25		# Set up logging
26		logger = setup_logger("extended_agent", level="INFO")
27
28		class ExtendedModularAgent(ModularOllamaAgent):
29		"""
30		Extended version of the ModularOllamaAgent with additional features.
31
32		This class demonstrates how to extend the modular agent with custom
33		functionality while leveraging the existing features.
34		"""
35
36		def __init__(self, *args, **kwargs):
37		"""Initialize the extended agent."""
38		# Initialize the base class
39		super().__init__(*args, **kwargs)
40
41		# Add custom attributes
42		self.conversation_history_file = os.path.join(
43		os.path.expanduser('~'),
44		'.ollamatools',
45		'conversation_history.json'
46		)
47		self.conversation_history = []
48
49		# Load conversation history if it exists
50		self._load_conversation_history()
51
52		print_info("Extended agent initialized with conversation history tracking")
53
54		def _load_conversation_history(self) -> None:
55		"""Load conversation history from file."""
56		if os.path.exists(self.conversation_history_file):
57		try:
58		with open(self.conversation_history_file, 'r') as f:
59		self.conversation_history = json.load(f)
60		print_info(f"Loaded {len(self.conversation_history)} conversations from history")
61		except Exception as e:
62		print_error(f"Failed to load conversation history: {str(e)}")
63		self.conversation_history = []
64
65		def _save_conversation_history(self) -> None:
66		"""Save conversation history to file."""
67		try:
68		# Create directory if it doesn't exist
69		os.makedirs(os.path.dirname(self.conversation_history_file), exist_ok=True)
70
71		with open(self.conversation_history_file, 'w') as f:
72		json.dump(self.conversation_history, f, indent=2)
73		print_info(f"Saved {len(self.conversation_history)} conversations to history")
74		except Exception as e:
75		print_error(f"Failed to save conversation history: {str(e)}")
76
77		def chat(self, message: str) -> str:
78		"""
79		Enhanced chat method that tracks conversation history.
80
81		Args:
82		message: User message
83
84		Returns:
85		str: Response message
86		"""
87		# Record start time for performance tracking
88		start_time = time.time()
89
90		# Process the message using the parent class method
91		response = super().chat(message)
92
93		# Calculate response time
94		response_time = time.time() - start_time
95
96		# Add to conversation history
97		conversation = {
98		"timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
99		"user_message": message,
100		"assistant_response": response,
101		"response_time": response_time,
102		"tokens_estimated": len(message.split()) + len(response.split())
103		}
104
105		self.conversation_history.append(conversation)
106		self._save_conversation_history()
107
108		return response
109
110		def summarize_conversations(self, n: int = 5) -> str:
111		"""
112		Summarize recent conversations.
113
114		Args:
115		n: Number of recent conversations to summarize
116
117		Returns:
118		str: Summary of recent conversations
119		"""
120		if not self.conversation_history:
121		return "No conversation history available."
122
123		# Get the most recent conversations
124		recent = self.conversation_history[-n:] if len(self.conversation_history) >= n else self.conversation_history
125
126		# Create a summary
127		summary = ["Recent Conversations:"]
128		for i, conv in enumerate(recent, 1):
129		summary.append(f"\n{i}. {conv['timestamp']}")
130		summary.append(f"User: {conv['user_message'][:100]}..." if len(conv['user_message']) > 100 else f"User: {conv['user_message']}")
131		summary.append(f"Response time: {conv['response_time']:.2f}s")
132
133		return "\n".join(summary)
134
135		def analyze_conversation_topics(self) -> Dict[str, int]:
136		"""
137		Analyze common topics in conversation history.
138
139		Returns:
140		Dict[str, int]: Dictionary of topics and their frequency
141		"""
142		if not self.conversation_history:
143		return {}
144
145		# Simple keyword-based topic analysis
146		topics = {
147		"help": 0,
148		"search": 0,
149		"file": 0,
150		"command": 0,
151		"code": 0,
152		"error": 0,
153		"issue": 0,
154		"question": 0,
155		"other": 0
156		}
157
158		for conv in self.conversation_history:
159		message = conv["user_message"].lower()
160
161		# Check for keywords
162		if "help" in message or "how" in message:
163		topics["help"] += 1
164		if "search" in message or "find" in message or "look" in message:
165		topics["search"] += 1
166		if "file" in message or "read" in message or "write" in message:
167		topics["file"] += 1
168		if "command" in message or "run" in message or "execute" in message:
169		topics["command"] += 1
170		if "code" in message or "program" in message or "script" in message:
171		topics["code"] += 1
172		if "error" in message or "bug" in message or "fix" in message:
173		topics["error"] += 1
174		if "issue" in message or "problem" in message:
175		topics["issue"] += 1
176		if "?" in message:
177		topics["question"] += 1
178
179		# If no keywords matched, count as other
180		if not any(count > 0 for count in topics.values()):
181		topics["other"] += 1
182
183		# Remove topics with zero count
184		return {k: v for k, v in topics.items() if v > 0}
185
186
187		def main():
188		"""Run the extended transparent Ollama agent."""
189		try:
190		# Initialize the extended agent
191		logger.info("Initializing ExtendedModularAgent...")
192		agent = ExtendedModularAgent(
193		model="llama3.1:latest",
194		permission_prompts=True
195		)
196
197		logger.info("Agent initialized. Enter 'exit' to quit.")
198		print("\nExtended Modular Ollama Agent")
199		print("=================================")
200		print("Type 'exit' to quit")
201		print("Type 'commands' to see available commands")
202		print("Type 'history' to see execution history")
203		print("Type 'conversations' to see recent conversations")
204		print("Type 'analyze' to analyze conversation topics")
205
206		# Simple chat loop
207		while True:
208		# Get user input
209		user_input = input("\nYou: ").strip()
210
211		# Check for exit command
212		if user_input.lower() in ['exit', 'quit', 'bye']:
213		print("\nExiting. Goodbye!")
214		break
215
216		# Handle special commands
217		if user_input.lower() == 'commands':
218		agent.list_allowed_commands()
219		continue
220
221		if user_input.lower() == 'history':
222		agent.show_execution_history()
223		continue
224
225		if user_input.lower() == 'conversations':
226		summary = agent.summarize_conversations()
227		print(summary)
228		continue
229
230		if user_input.lower() == 'analyze':
231		topics = agent.analyze_conversation_topics()
232		if topics:
233		print("\nConversation Topic Analysis:")
234		for topic, count in sorted(topics.items(), key=lambda x: x[1], reverse=True):
235		print(f" {topic}: {count}")
236		else:
237		print("\nNo conversation data to analyze.")
238		continue
239
240		# Process as normal chat message
241		response = agent.chat(user_input)
242		print(f"\nAssistant: {response}")
243
244		except KeyboardInterrupt:
245		print("\n\nExiting due to keyboard interrupt.")
246		except Exception as e:
247		logger.error(f"Unexpected error in main function: {str(e)}")
248		print(f"\nUnexpected error: {str(e)}")
249		print("Check the logs for details.")
250
251		return 0
252
253		if __name__ == "__main__":
254		sys.exit(main())

ollamatools-project/examples/file_operations.py +0 -172

…	…	@@ -1,172 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example script demonstrating file operations with OllamaTools.
4
5		This script shows how to use the file operation functions with
6		various examples and security features.
7		"""
8
9		import sys
10		import os
11		import tempfile
12		import shutil
13
14		# Add the parent directory to the path so we can import the package
15		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
16
17		from ollamatools.tools.system_tools import read_file, write_file, find_files
18		from ollamatools.security import PermissionManager
19
20
21		def create_sample_files(directory):
22		"""Create sample files for demonstration."""
23		# Create a few text files
24		write_file(os.path.join(directory, "notes.txt"),
25		"This is a sample text file for demonstration.\n"
26		"It contains multiple lines of text.\n"
27		"This can be used to test the read_file function.")
28
29		write_file(os.path.join(directory, "data.txt"),
30		"Sample data file with numbers:\n"
31		"1, 2, 3, 4, 5\n"
32		"10, 20, 30, 40, 50")
33
34		# Create a markdown file
35		write_file(os.path.join(directory, "readme.md"),
36		"# Sample Markdown File\n\n"
37		"This is a *markdown* file with some **formatting**.\n\n"
38		"- Item 1\n"
39		"- Item 2\n"
40		"- Item 3\n")
41
42		# Create a file in a subdirectory
43		os.makedirs(os.path.join(directory, "subdir"), exist_ok=True)
44		write_file(os.path.join(directory, "subdir", "nested.txt"),
45		"This file is in a subdirectory.")
46
47
48		def main():
49		print("=== File Operations Example ===\n")
50
51		# Create a permission manager for secure file operations
52		pm = PermissionManager()
53
54		# Create a temporary directory for demonstration
55		temp_dir = tempfile.mkdtemp()
56		print(f"Created temporary directory: {temp_dir}")
57
58		try:
59		# Create sample files
60		print("\nCreating sample files...")
61		create_sample_files(temp_dir)
62		print("Sample files created successfully.")
63
64		# Example 1: Reading a file
65		print("\n--- Example 1: Reading a File ---")
66		file_path = os.path.join(temp_dir, "notes.txt")
67		print(f"Reading file: {file_path}")
68
69		try:
70		content = read_file(file_path, permission_manager=pm)
71		print("\nFile content:")
72		print("-" * 40)
73		print(content)
74		print("-" * 40)
75		except Exception as e:
76		print(f"Error reading file: {e}")
77
78		# Example 2: Writing to a file
79		print("\n--- Example 2: Writing to a File ---")
80		new_file_path = os.path.join(temp_dir, "new_file.txt")
81		new_content = "This is a new file created using write_file.\n" \
82		"It demonstrates how to write content to a file."
83
84		print(f"Writing to file: {new_file_path}")
85
86		try:
87		success = write_file(new_file_path, new_content, permission_manager=pm)
88		if success:
89		print("File written successfully.")
90
91		# Read back the file to verify
92		content = read_file(new_file_path, permission_manager=pm)
93		print("\nVerifying file content:")
94		print("-" * 40)
95		print(content)
96		print("-" * 40)
97		except Exception as e:
98		print(f"Error writing file: {e}")
99
100		# Example 3: Appending to a file
101		print("\n--- Example 3: Appending to a File ---")
102		append_content = "\nThis content was appended to the file."
103
104		print(f"Appending to file: {new_file_path}")
105
106		try:
107		success = write_file(new_file_path, append_content, mode='a', permission_manager=pm)
108		if success:
109		print("Content appended successfully.")
110
111		# Read back the file to verify
112		content = read_file(new_file_path, permission_manager=pm)
113		print("\nVerifying file content after append:")
114		print("-" * 40)
115		print(content)
116		print("-" * 40)
117		except Exception as e:
118		print(f"Error appending to file: {e}")
119
120		# Example 4: Finding files
121		print("\n--- Example 4: Finding Files ---")
122
123		try:
124		# Find all text files
125		print("Finding all .txt files (recursive):")
126		txt_files = find_files("*.txt", temp_dir, recursive=True, permission_manager=pm)
127		for file in txt_files:
128		print(f" - {file}")
129
130		# Find all markdown files
131		print("\nFinding all .md files:")
132		md_files = find_files("*.md", temp_dir, recursive=True, permission_manager=pm)
133		for file in md_files:
134		print(f" - {file}")
135
136		# Find files in the root directory only (non-recursive)
137		print("\nFinding files in the root directory only (non-recursive):")
138		root_files = find_files("*", temp_dir, recursive=False, permission_manager=pm)
139		for file in root_files:
140		print(f" - {file}")
141		except Exception as e:
142		print(f"Error finding files: {e}")
143
144		# Example 5: Handling errors
145		print("\n--- Example 5: Handling Errors ---")
146
147		print("Attempting to read a non-existent file:")
148		try:
149		content = read_file(os.path.join(temp_dir, "nonexistent.txt"), permission_manager=pm)
150		print("File content (should not reach here):", content)
151		except Exception as e:
152		print(f"Error correctly caught: {e}")
153
154		print("\nAttempting to read a file with size limit:")
155		try:
156		# Get the size of notes.txt and try to read with a smaller limit
157		file_path = os.path.join(temp_dir, "notes.txt")
158		file_size = os.path.getsize(file_path)
159		content = read_file(file_path, max_size=file_size - 10, permission_manager=pm)
160		print("File content (should not reach here):", content)
161		except Exception as e:
162		print(f"Error correctly caught: {e}")
163
164		finally:
165		# Clean up the temporary directory
166		print(f"\nCleaning up temporary directory: {temp_dir}")
167		shutil.rmtree(temp_dir)
168		print("Cleanup complete.")
169
170
171		if __name__ == "__main__":
172		main()

ollamatools-project/examples/final_demo.py +0 -191

…	…	@@ -1,191 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Final demonstration of OllamaTools functionality.
4		This example shows the complete functionality of the library.
5		"""
6
7		import os
8		import sys
9		import time
10		import logging
11
12		# Add the parent directory to sys.path for imports
13		parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
14		if parent_dir not in sys.path:
15		sys.path.insert(0, parent_dir)
16
17		from ollamatools.config import Config
18		from ollamatools.security import PermissionManager
19		from ollamatools.agent import OllamaAgent
20		from ollamatools.tools.system_tools import execute_command, read_file, write_file, find_files
21		from ollamatools.tools.web_tools import fetch_url
22		from ollamatools.tools.doc_tools import generate_markdown
23		from ollamatools.utils.logger import setup_logger
24
25		# Setup logging
26		logger = setup_logger("final_demo", level="INFO")
27
28		def main():
29		"""Run the final demonstration."""
30		print("=== OllamaTools Final Demonstration ===")
31		print("This demo will show the core functionality of OllamaTools.")
32
33		# Step 1: Create a configuration
34		print("\n--- Step 1: Configuration ---")
35		config = Config(
36		model="llama3.1",
37		host="http://localhost:11434",
38		permission_prompts=True,
39		sudo_always_prompt=True,
40		timeout_seconds=30
41		)
42		print(f"Created configuration: {config}")
43
44		# Step 2: Create a permission manager
45		print("\n--- Step 2: Permission Management ---")
46		permission_manager = PermissionManager(
47		prompt_enabled=True,
48		sudo_always_prompt=True
49		)
50		print("Created permission manager with interactive prompts enabled.")
51
52		# Step 3: System Tools
53		print("\n--- Step 3: System Tools ---")
54
55		# For demo purposes, we'll temporarily disable permission prompts
56		permission_manager.prompt_enabled = False
57
58		# Execute a command
59		print("\n• Executing 'ls -la' command...")
60		result = execute_command(
61		command="ls",
62		args=["-la"],
63		permission_manager=permission_manager
64		)
65
66		if 'stdout' in result:
67		print(f"Command result: stdout={result['stdout'][:100]}...")
68		else:
69		print(f"Command result: {result}")
70
71		# Read a file
72		readme_path = os.path.join(parent_dir, "README.md")
73		if os.path.exists(readme_path):
74		print(f"\n• Reading file: {readme_path}")
75		content = read_file(
76		path=readme_path,
77		permission_manager=permission_manager
78		)
79		print(f"File content (first 100 chars): {content[:100]}...")
80		else:
81		print(f"\n• File {readme_path} not found, skipping reading demo")
82
83		# Write a file
84		test_file = "test_output.txt"
85		print(f"\n• Writing to file: {test_file}")
86		content_to_write = "This is a test file created by OllamaTools demo."
87		success = write_file(
88		path=test_file,
89		content=content_to_write,
90		permission_manager=permission_manager
91		)
92		print(f"Write success: {success}")
93
94		# Find files
95		print("\n• Finding Python files in the current directory...")
96		py_files = find_files(
97		pattern="*.py",
98		directory=".",
99		recursive=True,
100		permission_manager=permission_manager
101		)
102		print(f"Found {len(py_files)} Python files. First few: {py_files[:3] if py_files else []}")
103
104		# Step 4: Documentation Tools
105		print("\n--- Step 4: Documentation Tools ---")
106
107		# Generate markdown
108		print("\n• Generating markdown document...")
109		markdown_content = """# Sample Document
110
111		This is a sample document created by OllamaTools.
112
113		## Features
114		- Easy to use
115		- Secure by default
116		- Extensible
117		"""
118
119		try:
120		markdown_result = generate_markdown(
121		content=markdown_content,
122		title="Sample Document",
123		permission_manager=permission_manager
124		)
125		print(f"Generated markdown: {markdown_result[:100]}...")
126
127		# Write markdown to a file
128		markdown_file = "sample_doc.md"
129		print(f"\n• Writing markdown to file: {markdown_file}")
130		write_file(
131		path=markdown_file,
132		content=markdown_result,
133		permission_manager=permission_manager
134		)
135		except Exception as e:
136		print(f"Error generating markdown: {str(e)}")
137		markdown_file = None
138
139		# Step 5: OllamaAgent (simulated interaction as we don't have an Ollama server running)
140		print("\n--- Step 5: OllamaAgent Simulation ---")
141
142		print("\n• Creating OllamaAgent...")
143		try:
144		# Create an agent with permission prompts disabled for demo
145		agent = OllamaAgent(
146		model="llama3.1",
147		permission_prompts=False
148		)
149
150		# For a real example, you would use:
151		# response = agent.chat("What's the weather like in San Francisco?")
152
153		# Here we'll simulate a response
154		print("\n• Simulated chat interaction:")
155		print("User: What files are in the current directory?")
156		print("Agent: Let me check that for you.")
157
158		# Simulate function execution
159		files_result = execute_command(
160		command="ls",
161		args=["-la", "."],
162		permission_manager=permission_manager
163		)
164
165		if 'stdout' in files_result:
166		print(f"Agent: Here are the files in the current directory:\n{files_result['stdout']}")
167		else:
168		print(f"Agent: I encountered an error: {files_result}")
169		except Exception as e:
170		print(f"Error creating or using OllamaAgent: {str(e)}")
171
172		# Step 6: Clean up
173		print("\n--- Step 6: Clean Up ---")
174
175		# Remove test files
176		files_to_remove = [test_file]
177		if markdown_file:
178		files_to_remove.append(markdown_file)
179
180		for file in files_to_remove:
181		if os.path.exists(file):
182		os.remove(file)
183		print(f"Removed {file}")
184
185		print("\n=== Demonstration Complete ===")
186		print("OllamaTools provides a powerful interface for LLMs to interact with the system")
187		print("securely and effectively. This demo showed just a small sample of what's possible.")
188		return 0
189
190		if __name__ == "__main__":
191		sys.exit(main())

ollamatools-project/examples/function_execution.py +0 -106

…	…	@@ -1,106 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example showing function detection and execution with OllamaAgent.
4
5		This example demonstrates how the OllamaAgent automatically detects
6		and executes functions based on the conversation context.
7
8		Usage:
9		python function_execution.py
10		"""
11
12		import os
13		import sys
14		import json
15		import logging
16
17		# Add the parent directory to the Python path to import ollamatools
18		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
19
20		from ollamatools import OllamaAgent, Config, get_logger
21		from ollamatools.tools import system_tools, web_tools, doc_tools
22
23		# Set up logging
24		logger = get_logger(__name__, level=logging.INFO)
25
26		def run_example(agent, prompt, description):
27		"""Run an example prompt and display results."""
28		print(f"\n{'=' * 50}")
29		print(f"Example: {description}")
30		print(f"{'=' * 50}")
31
32		print(f"User: {prompt}")
33
34		# Send message to agent
35		response = agent.chat(
36		message=prompt,
37		system_prompt="""
38		You are an AI assistant with access to various tools to help the user.
39		When a user asks for something that requires using one of your tools,
40		automatically use the appropriate function to complete the task.
41		Be helpful, concise, and accurate.
42		"""
43		)
44
45		print(f"Assistant: {response}")
46
47		def main():
48		"""Run the function execution examples."""
49		try:
50		# Initialize the agent
51		logger.info("Initializing OllamaAgent...")
52		agent = OllamaAgent(
53		model="llama3:latest", # Adjust according to your installed models
54		permission_prompts=True
55		)
56
57		print("\nOllamaTools Function Execution Example")
58		print("--------------------------------------")
59		print("This example demonstrates automatic function detection and execution.")
60
61		# Example 1: System tools - List files
62		run_example(
63		agent,
64		"What files are in the current directory?",
65		"System Tools - List Files"
66		)
67
68		# Example 2: System tools - Execute command
69		run_example(
70		agent,
71		"How much disk space is available on my system?",
72		"System Tools - Disk Space Check"
73		)
74
75		# Example 3: Web tools - Weather
76		run_example(
77		agent,
78		"What's the weather like in San Francisco?",
79		"Web Tools - Weather Information"
80		)
81
82		# Example 4: Documentation tools - Generate markdown
83		run_example(
84		agent,
85		"Generate a short markdown document about Python functions",
86		"Documentation Tools - Generate Markdown"
87		)
88
89		# Example 5: Multiple functions
90		run_example(
91		agent,
92		"Find all Python files in the current directory and create a markdown document listing them",
93		"Multiple Functions - File Search and Document Generation"
94		)
95
96		print("\nExamples completed successfully!")
97
98		except Exception as e:
99		logger.error(f"Error in examples: {str(e)}")
100		print(f"Error: {str(e)}")
101		return 1
102
103		return 0
104
105		if __name__ == "__main__":
106		sys.exit(main())

ollamatools-project/examples/improved_transparent_agent.py +0 -229

…	…	@@ -1,229 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Improved Modular Ollama Agent Example.
4
5		This script demonstrates how to use the ModularOllamaAgent
6		to create a transparent, feature-rich agent with search capabilities,
7		command management, issue tracking, and semantic function selection feedback.
8		"""
9
10		import os
11		import sys
12		import logging
13		from typing import Dict, Any
14
15		# Import the modular agent
16		from ollamatools.agent import ModularOllamaAgent
17		from ollamatools.agent.modular import (
18		print_status, print_success, print_info,
19		FunctionSelectionFeedback
20		)
21		from ollamatools.opensearch_web_tools import DEFAULT_OPENSEARCH_CONFIG
22		from ollamatools.command_validator import DEFAULT_ALLOWLIST
23		from ollamatools.utils.logger import setup_logger
24
25		# Set up logging
26		logger = setup_logger("improved_agent", level="INFO")
27
28		# Extend the default command allowlist
29		EXTENDED_ALLOWLIST = DEFAULT_ALLOWLIST.union({
30		"python", "python3", "calc", "bc", "find", "grep", "awk", "sed",
31		"head", "tail", "more", "less", "cat", "echo", "cd", "pwd"
32		})
33
34		# Configuration for OpenSearchNG
35		OPENSEARCH_CONFIG = DEFAULT_OPENSEARCH_CONFIG.copy()
36		OPENSEARCH_CONFIG.update({
37		"base_url": "http://localhost:8081",
38		"engines": ["google", "bing", "duckduckgo", "brave"],
39		"num_results": 5
40		})
41
42		def main():
43		"""Run the improved transparent Ollama agent."""
44		try:
45		# Initialize the agent using the modular implementation
46		logger.info("Initializing ModularOllamaAgent...")
47		agent = ModularOllamaAgent(
48		model="llama3.1:latest",
49		permission_prompts=True,
50		opensearch_config=OPENSEARCH_CONFIG,
51		command_allowlist=EXTENDED_ALLOWLIST
52		)
53
54		logger.info("Agent initialized. Enter 'exit' to quit.")
55		print("\nImproved Modular Ollama Agent")
56		print("================================")
57		print("Type 'exit' to quit")
58		print("Type 'commands' to see available commands")
59		print("Type 'add_command <name>' to add a command to the allowed list")
60		print("Type 'remove_command <name>' to remove a command from the allowed list")
61		print("Type 'search <query>' to search directly")
62		print("Type 'dir' to show current directory structure")
63		print("Type 'history' to see execution history")
64
65		print("\nIssue Tracking:")
66		print(" issues - List all issues")
67		print(" issue <id> - Show issue details")
68		print(" add_issue <title> | <description> | [severity] - Add a new issue")
69		print(" resolve_issue <id> - Mark an issue as resolved")
70
71		print("\nFunction Selection Feedback:")
72		print(" feedback <query> | <correct_function> - Provide feedback on function selection")
73		print(" analyze_feedback - Show analysis of function selection patterns")
74
75		# Simple chat loop
76		while True:
77		# Get user input
78		user_input = input("\nYou: ").strip()
79
80		# Check for exit command
81		if user_input.lower() in ['exit', 'quit', 'bye']:
82		print("\nExiting. Goodbye!")
83		break
84
85		# Handle special commands
86		if user_input.lower() == 'commands':
87		agent.list_allowed_commands()
88		continue
89
90		if user_input.lower().startswith('add_command '):
91		command = user_input[len('add_command '):].strip()
92		agent.add_allowed_command(command)
93		continue
94
95		if user_input.lower().startswith('remove_command '):
96		command = user_input[len('remove_command '):].strip()
97		agent.remove_allowed_command(command)
98		continue
99
100		if user_input.lower() == 'history':
101		agent.show_execution_history()
102		continue
103
104		if user_input.lower().startswith('search '):
105		query = user_input[7:].strip()
106		if query:
107		try:
108		results = agent.execute_search(query)
109		# Show the top 3 results
110		if results.get("results"):
111		print("\nTop Results:")
112		for i, result in enumerate(results["results"][:3], 1):
113		print(f"{i}. {result['title']}")
114		print(f" URL: {result['url']}")
115		print(f" {result['content'][:100]}...")
116		print()
117		except Exception as e:
118		print(f"Search failed: {str(e)}")
119		else:
120		print("Please provide a search query.")
121		continue
122
123		if user_input.lower().startswith('dir'):
124		parts = user_input.split()
125		path = parts[1] if len(parts) > 1 else '.'
126		depth = int(parts[2]) if len(parts) > 2 and parts[2].isdigit() else 2
127
128		try:
129		agent.get_directory_structure(path, depth)
130		except Exception as e:
131		print(f"Failed to get directory structure: {str(e)}")
132		continue
133
134		# Handle issue tracking commands
135		if user_input.lower() == 'issues':
136		agent.list_issues()
137		continue
138
139		if user_input.lower().startswith('issue '):
140		try:
141		issue_id = int(user_input[len('issue '):].strip())
142		agent.get_issue(issue_id)
143		except ValueError:
144		print("Invalid issue ID. Use 'issue <number>'.")
145		continue
146
147		if user_input.lower().startswith('add_issue '):
148		# Parse command: add_issue Title | Description | [severity]
149		parts = user_input[len('add_issue '):].split('|')
150		if len(parts) >= 2:
151		title = parts[0].strip()
152		description = parts[1].strip()
153		severity = parts[2].strip().lower() if len(parts) > 2 else 'medium'
154		agent.add_issue(title, description, severity)
155		else:
156		print("Usage: add_issue Title | Description | [severity]")
157		continue
158
159		if user_input.lower().startswith('resolve_issue '):
160		try:
161		issue_id = int(user_input[len('resolve_issue '):].strip())
162		agent.update_issue(issue_id, status='resolved')
163		except ValueError:
164		print("Invalid issue ID. Use 'resolve_issue <number>'.")
165		continue
166
167		# Handle function selection feedback commands
168		if user_input.lower().startswith('feedback '):
169		# Parse command: feedback query | correct_function
170		parts = user_input[len('feedback '):].split('|')
171		if len(parts) >= 2:
172		query = parts[0].strip()
173		correct_function = parts[1].strip()
174		agent.provide_function_feedback(query, correct_function)
175		else:
176		print("Usage: feedback query | correct_function")
177		print("Example: feedback 'what directory am I in' | execute_command")
178		continue
179
180		if user_input.lower() == 'analyze_feedback':
181		try:
182		analysis = agent.function_feedback.analyze_function_selection_patterns()
183		if "error" in analysis:
184		print(f"Error analyzing feedback: {analysis['error']}")
185		else:
186		print("\nFunction Selection Feedback Analysis:")
187		print(f"Total feedback entries: {analysis['total_feedback_entries']}")
188
189		if analysis.get("top_mismatches"):
190		print("\nTop Function Mismatches:")
191		for i, (mismatch, data) in enumerate(analysis["top_mismatches"][:5], 1):
192		print(f"{i}. {mismatch} (count: {data['count']})")
193		print(f" Examples: {', '.join(data['examples'][:3])}")
194
195		if analysis.get("top_query_patterns"):
196		print("\nTop Query Patterns:")
197		for i, (pattern, data) in enumerate(analysis["top_query_patterns"][:5], 1):
198		print(f"{i}. '{pattern}' (count: {data['count']})")
199		if data.get("functions"):
200		top_function = max(data["functions"].items(), key=lambda x: x[1])
201		print(f" Most common function: {top_function[0]} ({top_function[1]} occurrences)")
202
203		if analysis.get("improvement_suggestions"):
204		print("\nImprovement Suggestions:")
205		for i, suggestion in enumerate(analysis["improvement_suggestions"][:3], 1):
206		print(f"{i}. {suggestion['description']}")
207		if suggestion.get("examples"):
208		print(f" Examples: {', '.join(suggestion['examples'])}")
209		if suggestion.get("suggested_keywords"):
210		print(f" Keywords: {', '.join(suggestion['suggested_keywords'])}")
211		except Exception as e:
212		print(f"Error analyzing feedback: {str(e)}")
213		continue
214
215		# Process as normal chat message
216		response = agent.chat(user_input)
217		print(f"\nAssistant: {response}")
218
219		except KeyboardInterrupt:
220		print("\n\nExiting due to keyboard interrupt.")
221		except Exception as e:
222		logger.error(f"Unexpected error in main function: {str(e)}")
223		print(f"\nUnexpected error: {str(e)}")
224		print("Check the logs for details.")
225
226		return 0
227
228		if __name__ == "__main__":
229		sys.exit(main())

ollamatools-project/examples/matrix_client_example.py +0 -227

…	…	@@ -1,227 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Matrix Client Example for OllamaTools
4
5		This example demonstrates how to use the Matrix integration in OllamaTools
6		to connect to a Matrix homeserver, join rooms, and process messages with Ollama.
7
8		Requirements:
9		- matrix-nio: `pip install matrix-nio[e2e]`
10		- ollama: `pip install ollama`
11		- pyyaml: `pip install pyyaml`
12
13		Usage:
14		1. Edit the configuration below or create a config.yaml file
15		2. Run the script: `python matrix_client_example.py`
16		"""
17
18		import os
19		import sys
20		import yaml
21		import logging
22		import asyncio
23		import argparse
24		import random
25		from typing import Dict, Optional
26
27		# Add the parent directory to the path so we can import ollamatools
28		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
29
30		from ollamatools.tools.matrix_tools import (
31		MatrixClient,
32		MatrixConfig,
33		connect_to_matrix,
34		join_matrix_room,
35		send_matrix_message,
36		listen_for_matrix_messages,
37		load_matrix_config,
38		save_matrix_config,
39		create_default_matrix_config
40		)
41
42		# Configure logging
43		logging.basicConfig(
44		level=logging.INFO,
45		format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
46		)
47		logger = logging.getLogger(__name__)
48
49		# Default configuration file path
50		DEFAULT_CONFIG_PATH = os.path.join(os.path.dirname(__file__), 'config', 'matrix_config.yaml')
51
52		def parse_arguments():
53		"""Parse command line arguments."""
54		parser = argparse.ArgumentParser(description='Matrix client example for OllamaTools')
55		parser.add_argument(
56		'-c', '--config',
57		help='Path to the configuration file',
58		default=DEFAULT_CONFIG_PATH
59		)
60		parser.add_argument(
61		'--create-config',
62		help='Create a default configuration file',
63		action='store_true'
64		)
65		parser.add_argument(
66		'-d', '--debug',
67		help='Enable debug logging',
68		action='store_true'
69		)
70		return parser.parse_args()
71
72		def create_config_file(config_path: str) -> None:
73		"""Create a default configuration file."""
74		config = create_default_matrix_config()
75
76		# Create the directory if it doesn't exist
77		os.makedirs(os.path.dirname(config_path), exist_ok=True)
78
79		# Save the configuration
80		save_matrix_config(config, config_path)
81
82		logger.info(f"Created default configuration file at {config_path}")
83		logger.info("Please edit this file with your Matrix credentials and Ollama settings.")
84
85		sys.exit(0)
86
87		def load_config(config_path: str) -> Dict:
88		"""Load the configuration from a file."""
89		try:
90		config = load_matrix_config(config_path)
91		return config
92		except FileNotFoundError:
93		logger.error(f"Configuration file not found: {config_path}")
94		logger.info("You can create a default configuration file with --create-config")
95		sys.exit(1)
96		except Exception as e:
97		logger.error(f"Error loading configuration: {e}")
98		sys.exit(1)
99
100		async def custom_message_handler(room, event, client):
101		"""
102		Custom message handler for Matrix events.
103
104		This is an example of how to create a custom handler for Matrix messages.
105		It processes the message with Ollama and sends the response back to the room.
106
107		Args:
108		room: The Matrix room where the message was received
109		event: The Matrix event (message)
110		client: The MatrixClient instance
111		"""
112		message = event.body
113		room_id = room.room_id
114		sender = event.sender
115
116		logger.info(f"Custom handler received message in room {room_id} from {sender}: {message}")
117
118		# Process the message with Ollama
119		response = await client.process_with_ollama(message)
120
121		# Add a prefix to the response
122		response = f"Response from Ollama: {response}"
123
124		# Send the response back to the room
125		await client.send_message(room_id, response)
126
127		def get_random_greeting() -> str:
128		"""Generate a random greeting message."""
129		greetings = [
130		"Hello everyone! I'm here to assist with your questions.",
131		"Greetings! I'm your friendly Matrix bot powered by Ollama.",
132		"Hi there! I'm online and ready to chat.",
133		"Good day! I'm here to help with anything you need.",
134		"Hey folks! I've just connected and I'm ready to assist.",
135		"Hello! I'm your AI assistant, ready to help with your questions.",
136		"Greetings, humans! AI assistant at your service.",
137		"Hi everyone! I'm here and ready to chat.",
138		"Hello Matrix world! How can I assist you today?",
139		"Greetings! Your friendly neighborhood AI has arrived."
140		]
141		return random.choice(greetings)
142
143		async def main_async(config: Dict) -> None:
144		"""
145		Main async function to run the Matrix client.
146
147		Args:
148		config: The configuration dictionary
149		"""
150		# Extract configuration
151		matrix_config = config.get('MATRIX', {})
152		ollama_config = config.get('OLLAMA', {})
153
154		homeserver = matrix_config.get('HOMESERVER')
155		user_id = matrix_config.get('USER')
156		password = matrix_config.get('PASSWORD')
157		device_id = matrix_config.get('DEVICE_ID')
158		rooms = matrix_config.get('ROOMS', [])
159
160		ollama_host = ollama_config.get('HOST', 'localhost:11434')
161		ollama_model = ollama_config.get('MODEL', 'llama3:latest')
162		ollama_prompt = ollama_config.get('PROMPT')
163
164		# Create and connect the Matrix client
165		client = MatrixClient(
166		homeserver=homeserver,
167		user_id=user_id,
168		password=password,
169		device_id=device_id,
170		ollama_host=ollama_host,
171		ollama_model=ollama_model,
172		ollama_prompt=ollama_prompt
173		)
174
175		# Connect to the Matrix homeserver
176		success = await client.connect()
177
178		if not success:
179		logger.error("Failed to connect to Matrix homeserver")
180		return
181
182		# Join rooms and send a random greeting
183		for room in rooms:
184		if isinstance(room, dict) and 'ROOM' in room:
185		room_id = room['ROOM']
186		else:
187		room_id = room
188
189		join_success = await client.join_room(room_id)
190		if join_success:
191		# Send a random greeting message
192		greeting = get_random_greeting()
193		await client.send_message(room_id, greeting)
194		logger.info(f"Sent greeting to room {room_id}: {greeting}")
195
196		# Optional: Set a custom message handler
197		# client.set_message_callback(custom_message_handler)
198
199		# Start listening for messages
200		logger.info("Starting to listen for messages...")
201		await client.listen()
202
203		def main():
204		"""Main function to run the Matrix client."""
205		args = parse_arguments()
206
207		# Set log level
208		if args.debug:
209		logging.getLogger().setLevel(logging.DEBUG)
210
211		# Create a default configuration file if requested
212		if args.create_config:
213		create_config_file(args.config)
214
215		# Load the configuration
216		config = load_config(args.config)
217
218		# Run the async main function
219		try:
220		asyncio.run(main_async(config))
221		except KeyboardInterrupt:
222		logger.info("Interrupted by user")
223		except Exception as e:
224		logger.error(f"Error: {e}")
225
226		if __name__ == '__main__':
227		main()

ollamatools-project/examples/modular_agent_example.py +0 -228

…	…	@@ -1,228 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		ModularOllamaAgent Example.
4
5		This script demonstrates how to use the ModularOllamaAgent
6		to create a feature-rich agent with search capabilities,
7		command management, issue tracking, and semantic function selection feedback.
8		"""
9
10		import os
11		import sys
12		import logging
13		from typing import Dict, Any
14
15		# Import the ModularOllamaAgent
16		from ollamatools.agent import ModularOllamaAgent
17		from ollamatools.agent.modular import (
18		print_status, print_success, print_info
19		)
20		from ollamatools.opensearch_web_tools import DEFAULT_OPENSEARCH_CONFIG
21		from ollamatools.command_validator import DEFAULT_ALLOWLIST
22		from ollamatools.utils.logger import setup_logger
23
24		# Set up logging
25		logger = setup_logger("modular_agent_example", level="INFO")
26
27		# Extend the default command allowlist
28		EXTENDED_ALLOWLIST = DEFAULT_ALLOWLIST.union({
29		"python", "python3", "calc", "bc", "find", "grep", "awk", "sed",
30		"head", "tail", "more", "less", "cat", "echo", "cd", "pwd"
31		})
32
33		# Configuration for OpenSearchNG
34		OPENSEARCH_CONFIG = DEFAULT_OPENSEARCH_CONFIG.copy()
35		OPENSEARCH_CONFIG.update({
36		"base_url": "http://localhost:8081",
37		"engines": ["google", "bing", "duckduckgo", "brave"],
38		"num_results": 5
39		})
40
41		def main():
42		"""Run the ModularOllamaAgent example."""
43		try:
44		# Initialize the agent
45		logger.info("Initializing ModularOllamaAgent...")
46		agent = ModularOllamaAgent(
47		model="llama3.1:latest",
48		permission_prompts=True,
49		opensearch_config=OPENSEARCH_CONFIG,
50		command_allowlist=EXTENDED_ALLOWLIST
51		)
52
53		logger.info("Agent initialized. Enter 'exit' to quit.")
54		print("\nModularOllamaAgent Example")
55		print("=========================")
56		print("Type 'exit' to quit")
57		print("Type 'commands' to see available commands")
58		print("Type 'add_command <name>' to add a command to the allowed list")
59		print("Type 'remove_command <name>' to remove a command from the allowed list")
60		print("Type 'search <query>' to search directly")
61		print("Type 'dir' to show current directory structure")
62		print("Type 'history' to see execution history")
63
64		print("\nIssue Tracking:")
65		print(" issues - List all issues")
66		print(" issue <id> - Show issue details")
67		print(" add_issue <title> | <description> | [severity] - Add a new issue")
68		print(" resolve_issue <id> - Mark an issue as resolved")
69
70		print("\nFunction Selection Feedback:")
71		print(" feedback <query> | <correct_function> - Provide feedback on function selection")
72		print(" analyze_feedback - Show analysis of function selection patterns")
73
74		# Simple chat loop
75		while True:
76		# Get user input
77		user_input = input("\nYou: ").strip()
78
79		# Check for exit command
80		if user_input.lower() in ['exit', 'quit', 'bye']:
81		print("\nExiting. Goodbye!")
82		break
83
84		# Handle special commands
85		if user_input.lower() == 'commands':
86		agent.list_allowed_commands()
87		continue
88
89		if user_input.lower().startswith('add_command '):
90		command = user_input[len('add_command '):].strip()
91		agent.add_allowed_command(command)
92		continue
93
94		if user_input.lower().startswith('remove_command '):
95		command = user_input[len('remove_command '):].strip()
96		agent.remove_allowed_command(command)
97		continue
98
99		if user_input.lower() == 'history':
100		agent.show_execution_history()
101		continue
102
103		if user_input.lower().startswith('search '):
104		query = user_input[7:].strip()
105		if query:
106		try:
107		results = agent.execute_search(query)
108		# Show the top 3 results
109		if results.get("results"):
110		print("\nTop Results:")
111		for i, result in enumerate(results["results"][:3], 1):
112		print(f"{i}. {result['title']}")
113		print(f" URL: {result['url']}")
114		print(f" {result['content'][:100]}...")
115		print()
116		except Exception as e:
117		print(f"Search failed: {str(e)}")
118		else:
119		print("Please provide a search query.")
120		continue
121
122		if user_input.lower().startswith('dir'):
123		parts = user_input.split()
124		path = parts[1] if len(parts) > 1 else '.'
125		depth = int(parts[2]) if len(parts) > 2 and parts[2].isdigit() else 2
126
127		try:
128		agent.get_directory_structure(path, depth)
129		except Exception as e:
130		print(f"Failed to get directory structure: {str(e)}")
131		continue
132
133		# Handle issue tracking commands
134		if user_input.lower() == 'issues':
135		agent.list_issues()
136		continue
137
138		if user_input.lower().startswith('issue '):
139		try:
140		issue_id = int(user_input[len('issue '):].strip())
141		agent.get_issue(issue_id)
142		except ValueError:
143		print("Invalid issue ID. Use 'issue <number>'.")
144		continue
145
146		if user_input.lower().startswith('add_issue '):
147		# Parse command: add_issue Title | Description | [severity]
148		parts = user_input[len('add_issue '):].split('|')
149		if len(parts) >= 2:
150		title = parts[0].strip()
151		description = parts[1].strip()
152		severity = parts[2].strip().lower() if len(parts) > 2 else 'medium'
153		agent.add_issue(title, description, severity)
154		else:
155		print("Usage: add_issue Title | Description | [severity]")
156		continue
157
158		if user_input.lower().startswith('resolve_issue '):
159		try:
160		issue_id = int(user_input[len('resolve_issue '):].strip())
161		agent.update_issue(issue_id, status='resolved')
162		except ValueError:
163		print("Invalid issue ID. Use 'resolve_issue <number>'.")
164		continue
165
166		# Handle function selection feedback commands
167		if user_input.lower().startswith('feedback '):
168		# Parse command: feedback query | correct_function
169		parts = user_input[len('feedback '):].split('|')
170		if len(parts) >= 2:
171		query = parts[0].strip()
172		correct_function = parts[1].strip()
173		agent.provide_function_feedback(query, correct_function)
174		else:
175		print("Usage: feedback query | correct_function")
176		print("Example: feedback 'what directory am I in' | execute_command")
177		continue
178
179		if user_input.lower() == 'analyze_feedback':
180		try:
181		analysis = agent.function_feedback.analyze_function_selection_patterns()
182		if "error" in analysis:
183		print(f"Error analyzing feedback: {analysis['error']}")
184		else:
185		print("\nFunction Selection Feedback Analysis:")
186		print(f"Total feedback entries: {analysis['total_feedback_entries']}")
187
188		if analysis.get("top_mismatches"):
189		print("\nTop Function Mismatches:")
190		for i, (mismatch, data) in enumerate(analysis["top_mismatches"][:5], 1):
191		print(f"{i}. {mismatch} (count: {data['count']})")
192		print(f" Examples: {', '.join(data['examples'][:3])}")
193
194		if analysis.get("top_query_patterns"):
195		print("\nTop Query Patterns:")
196		for i, (pattern, data) in enumerate(analysis["top_query_patterns"][:5], 1):
197		print(f"{i}. '{pattern}' (count: {data['count']})")
198		if data.get("functions"):
199		top_function = max(data["functions"].items(), key=lambda x: x[1])
200		print(f" Most common function: {top_function[0]} ({top_function[1]} occurrences)")
201
202		if analysis.get("improvement_suggestions"):
203		print("\nImprovement Suggestions:")
204		for i, suggestion in enumerate(analysis["improvement_suggestions"][:3], 1):
205		print(f"{i}. {suggestion['description']}")
206		if suggestion.get("examples"):
207		print(f" Examples: {', '.join(suggestion['examples'])}")
208		if suggestion.get("suggested_keywords"):
209		print(f" Keywords: {', '.join(suggestion['suggested_keywords'])}")
210		except Exception as e:
211		print(f"Error analyzing feedback: {str(e)}")
212		continue
213
214		# Process as normal chat message
215		response = agent.chat(user_input)
216		print(f"\nAssistant: {response}")
217
218		except KeyboardInterrupt:
219		print("\n\nExiting due to keyboard interrupt.")
220		except Exception as e:
221		logger.error(f"Unexpected error in main function: {str(e)}")
222		print(f"\nUnexpected error: {str(e)}")
223		print("Check the logs for details.")
224
225		return 0
226
227		if __name__ == "__main__":
228		sys.exit(main())

ollamatools-project/examples/performance_optimization.py +0 -323

…	…	@@ -1,323 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example demonstrating error handling in OllamaTools.
4		"""
5
6		import os
7		import sys
8		import time
9		from typing import Dict, Any, List, Optional
10
11		# Add the parent directory to sys.path for imports
12		parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
13		if parent_dir not in sys.path:
14		sys.path.insert(0, parent_dir)
15
16		from ollamatools.security import PermissionManager
17		from ollamatools.tools.system_tools import execute_command, read_file, write_file
18		from ollamatools.tools.web_tools import fetch_url
19		from ollamatools.tools.doc_tools import generate_markdown
20		from ollamatools.utils.logger import setup_logger
21
22		# Setup logging
23		logger = setup_logger("error_handling", level="DEBUG")
24
25		def handle_permission_errors():
26		"""
27		Demonstrate handling permission errors.
28		"""
29		logger.info("=== Permission Error Handling Demo ===")
30
31		# Create a permission manager that will deny all requests
32		permission_manager = PermissionManager(prompt_enabled=True)
33
34		try:
35		# Attempt to execute a command with a permission manager that will prompt
36		print("This will display a permission prompt. Enter 'n' to deny permission.")
37		result = execute_command(
38		command="ls",
39		args=["-la"],
40		permission_manager=permission_manager
41		)
42
43		logger.info(f"Command result: {result}")
44		except Exception as e:
45		logger.error(f"Error executing command: {str(e)}")
46
47		logger.info("Permission error handling completed")
48
49		def handle_command_execution_errors():
50		"""
51		Demonstrate handling command execution errors.
52		"""
53		logger.info("\n=== Command Execution Error Handling Demo ===")
54
55		# Create a permission manager with prompts disabled for demo
56		permission_manager = PermissionManager(prompt_enabled=False)
57
58		# Invalid command
59		logger.info("Executing an invalid command...")
60		result = execute_command(
61		command="nonexistent_command",
62		permission_manager=permission_manager
63		)
64
65		if "error" in result:
66		logger.error(f"Command error: {result['error']}")
67		else:
68		logger.info(f"Command result: {result}")
69
70		# Command with very short timeout
71		logger.info("\nExecuting a command with a very short timeout...")
72		result = execute_command(
73		command="sleep",
74		args=["10"],
75		timeout=1,
76		permission_manager=permission_manager
77		)
78
79		if "error" in result:
80		logger.error(f"Timeout error: {result['error']}")
81		else:
82		logger.info(f"Command result: {result}")
83
84		logger.info("Command execution error handling completed")
85
86		def handle_file_operation_errors():
87		"""
88		Demonstrate handling file operation errors.
89		"""
90		logger.info("\n=== File Operation Error Handling Demo ===")
91
92		# Create a permission manager with prompts disabled for demo
93		permission_manager = PermissionManager(prompt_enabled=False)
94
95		# Read a non-existent file
96		non_existent_file = "non_existent_file.txt"
97
98		logger.info(f"Reading non-existent file: {non_existent_file}...")
99		try:
100		content = read_file(
101		path=non_existent_file,
102		permission_manager=permission_manager
103		)
104		logger.info(f"File content: {content}")
105		except Exception as e:
106		logger.error(f"Error reading file: {str(e)}")
107
108		# Write to a directory (which should fail)
109		logger.info("\nWriting to a directory (should fail)...")
110		try:
111		success = write_file(
112		path=".", # Current directory
113		content="This should fail",
114		permission_manager=permission_manager
115		)
116		logger.info(f"Write success: {success}")
117		except Exception as e:
118		logger.error(f"Error writing to directory: {str(e)}")
119
120		# Write to a file with invalid permissions
121		if os.name == "posix": # Unix-like systems only
122		restricted_file = "/etc/test_file.txt"
123
124		logger.info(f"\nWriting to a restricted file: {restricted_file}...")
125		try:
126		success = write_file(
127		path=restricted_file,
128		content="This should fail",
129		permission_manager=permission_manager
130		)
131		logger.info(f"Write success: {success}")
132		except Exception as e:
133		logger.error(f"Error writing to restricted file: {str(e)}")
134
135		logger.info("File operation error handling completed")
136
137		def handle_web_request_errors():
138		"""
139		Demonstrate handling web request errors.
140		"""
141		logger.info("\n=== Web Request Error Handling Demo ===")
142
143		# Create a permission manager with prompts disabled for demo
144		permission_manager = PermissionManager(prompt_enabled=False)
145
146		# Invalid URL
147		invalid_url = "http://nonexistent.domain.123"
148
149		logger.info(f"Fetching invalid URL: {invalid_url}...")
150		try:
151		result = fetch_url(
152		url=invalid_url,
153		permission_manager=permission_manager
154		)
155		logger.info(f"Result: {result}")
156		except Exception as e:
157		logger.error(f"Error fetching invalid URL: {str(e)}")
158
159		# URL with very short timeout
160		logger.info("\nFetching URL with very short timeout...")
161		try:
162		result = fetch_url(
163		url="https://example.com",
164		timeout=0.001, # Very short timeout
165		permission_manager=permission_manager
166		)
167		logger.info(f"Result: {result}")
168		except Exception as e:
169		logger.error(f"Error fetching URL with short timeout: {str(e)}")
170
171		logger.info("Web request error handling completed")
172
173		def handle_doc_tools_errors():
174		"""
175		Demonstrate handling documentation tools errors.
176		"""
177		logger.info("\n=== Documentation Tools Error Handling Demo ===")
178
179		# Create a permission manager with prompts disabled for demo
180		permission_manager = PermissionManager(prompt_enabled=False)
181
182		# Generate markdown with empty content
183		logger.info("Generating markdown with empty content...")
184		try:
185		result = generate_markdown(
186		content="",
187		title="Empty Document",
188		permission_manager=permission_manager
189		)
190		logger.info(f"Generated markdown: {result}")
191		except Exception as e:
192		logger.error(f"Error generating markdown: {str(e)}")
193
194		# Extract context with empty documents
195		logger.info("\nExtracting context with empty documents...")
196		try:
197		context = extract_context(
198		query="test",
199		documents=[],
200		permission_manager=permission_manager
201		)
202		logger.info(f"Extracted context: {context}")
203		except Exception as e:
204		logger.error(f"Error extracting context: {str(e)}")
205
206		# Summarize empty text
207		logger.info("\nSummarizing empty text...")
208		try:
209		summary = summarize_text(
210		text="",
211		permission_manager=permission_manager
212		)
213		logger.info(f"Summary: {summary}")
214		except Exception as e:
215		logger.error(f"Error summarizing text: {str(e)}")
216
217		logger.info("Documentation tools error handling completed")
218
219		def handle_custom_errors():
220		"""
221		Demonstrate handling custom errors with try-except blocks.
222		"""
223		logger.info("\n=== Custom Error Handling Demo ===")
224
225		# Wrap command execution in a try-except block
226		logger.info("Custom error handling for command execution...")
227		try:
228		# This will fail because the command doesn't exist
229		result = execute_command(
230		command="nonexistent_command",
231		permission_manager=PermissionManager(prompt_enabled=False)
232		)
233
234		# Check for error in the result
235		if "error" in result:
236		logger.warning(f"Command failed: {result['error']}")
237		print("Showing user-friendly error message instead of technical details.")
238		print("The command could not be executed. Please check if it's installed.")
239		else:
240		logger.info(f"Command succeeded: {result}")
241		except Exception as e:
242		logger.error(f"Unexpected error: {str(e)}")
243		print("An unexpected error occurred. Please try again later.")
244
245		# Create a function that retries web requests
246		def fetch_with_retry(url, max_retries=3, delay=1):
247		"""Fetch a URL with retries."""
248		permission_manager = PermissionManager(prompt_enabled=False)
249
250		for attempt in range(max_retries):
251		try:
252		logger.info(f"Attempt {attempt + 1}/{max_retries} to fetch {url}")
253		result = fetch_url(
254		url=url,
255		permission_manager=permission_manager
256		)
257		return result
258		except Exception as e:
259		logger.warning(f"Attempt {attempt + 1} failed: {str(e)}")
260		if attempt < max_retries - 1:
261		logger.info(f"Retrying in {delay} seconds...")
262		time.sleep(delay)
263		delay *= 2 # Exponential backoff
264		else:
265		logger.error("All retry attempts failed")
266		raise
267
268		# Test the retry function
269		logger.info("\nTesting fetch with retry (with invalid URL)...")
270		try:
271		result = fetch_with_retry("http://nonexistent.domain.123")
272		logger.info(f"Result: {result}")
273		except Exception as e:
274		logger.error(f"Final error after retries: {str(e)}")
275
276		logger.info("Custom error handling completed")
277
278		def main():
279		"""Run the error handling examples."""
280		logger.info("OllamaTools Error Handling Examples")
281		logger.info("==================================")
282
283		try:
284		# Handle permission errors
285		handle_permission_errors()
286
287		# Handle command execution errors
288		handle_command_execution_errors()
289
290		# Handle file operation errors
291		handle_file_operation_errors()
292
293		# Handle web request errors
294		handle_web_request_errors()
295
296		# Handle documentation tools errors
297		handle_doc_tools_errors()
298
299		# Handle custom errors
300		handle_custom_errors()
301
302		logger.info("\n=== Error Handling Examples Complete ===")
303		except Exception as e:
304		logger.error(f"Unexpected error in examples: {str(e)}")
305		return 1
306
307		return 0
308
309		if __name__ == "__main__":
310		# Import extract_context and summarize_text here to avoid import errors if they're not implemented
311		try:
312		from ollamatools.tools.doc_tools import extract_context, summarize_text
313		except ImportError:
314		# Define placeholder functions if not implemented
315		, documents, max_length=500):
316		"""Placeholder for extract_context if not implemented."""
317		return f"Context for query '{query}' (placeholder)"
318
319		def summarize_text(text, max_length=200):
320		"""Placeholder for summarize_text if not implemented."""
321		return "Summary placeholder"
322
323		sys.exit(main())

ollamatools-project/examples/permission_example.py +0 -78

…	…	@@ -1,78 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example script demonstrating the basic usage of the PermissionManager.
4
5		This script shows how to initialize the PermissionManager and request
6		permissions for various operations.
7		"""
8
9		# Add parent directory to path to allow importing the package
10		import sys
11		import os
12		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
13
14		# Now import from the package
15		from ollamatools.security import PermissionManager
16
17
18		def main():
19		# Create a permission manager with default settings
20		# (prompts enabled, sudo always prompts)
21		pm = PermissionManager()
22
23		print("=== Basic Permission Example ===")
24
25		# Example 1: Request permission for a simple function
26		print("\nExample 1: Simple function")
27		result = pm.request_permission(
28		"read_file",
29		{"path": "/etc/hosts"}
30		)
31		print(f"Permission granted: {result}")
32
33		# Example 2: Request permission for a function with sudo
34		print("\nExample 2: Function with sudo")
35		result = pm.request_permission(
36		"execute_command",
37		{"command": "apt update"},
38		is_sudo=True
39		)
40		print(f"Permission granted: {result}")
41
42		# Example 3: Detect sudo in a command
43		print("\nExample 3: Sudo detection")
44		commands = [
45		"ls -la",
46		"sudo apt update",
47		"echo 'This is not sudo'",
48		"sudo reboot"
49		]
50
51		for cmd in commands:
52		is_sudo = pm.detect_sudo(cmd)
53		print(f"Command: {cmd}")
54		print(f"Uses sudo: {is_sudo}")
55		print()
56
57		# Example 4: Create a permission manager with prompts disabled
58		print("\nExample 4: Disabled prompts")
59		pm_no_prompts = PermissionManager(prompt_enabled=False)
60		result = pm_no_prompts.request_permission(
61		"execute_command",
62		{"command": "ls -la"}
63		)
64		print(f"Permission granted: {result}")
65
66		# Example 5: Create a permission manager with sudo prompts disabled
67		print("\nExample 5: Disabled sudo prompts")
68		pm_no_sudo_prompts = PermissionManager(sudo_always_prompt=False)
69		result = pm_no_sudo_prompts.request_permission(
70		"execute_command",
71		{"command": "sudo apt update"},
72		is_sudo=True
73		)
74		print(f"Permission granted: {result}")
75
76
77		if __name__ == "__main__":
78		main()

ollamatools-project/examples/plugin_system_example.py +0 -181

…	…	@@ -1,181 +0,0 @@
1		"""
2		Example demonstrating the OllamaTools plugin system.
3
4		This example shows how to create, register, and use plugins with the OllamaTools
5		plugin system.
6		"""
7
8		import os
9		import sys
10		import logging
11		from typing import Dict, Any, List
12
13		# Configure logging
14		logging.basicConfig(
15		level=logging.INFO,
16		format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
17		)
18		logger = logging.getLogger(__name__)
19
20		# Import plugin system components using relative imports
21		import sys
22		import os
23
24		# Add the parent directory to the Python path
25		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
26
27		from ollamatools.plugins import OllamaToolsPlugin, SharedPlugin, MatrixPlugin, AgentPlugin
28		from ollamatools.plugins.registry import PluginRegistry
29		from ollamatools.plugins.errors import PluginError, handle_plugin_exception
30		from ollamatools.config import PluginConfig, load_plugin_config
31
32
33		# Define a simple plugin
34		class GreetingPlugin(SharedPlugin):
35		"""Plugin that provides greeting capabilities."""
36
37		def __init__(self, config=None):
38		super().__init__(config)
39		logger.info("GreetingPlugin created")
40
41		def initialize(self, context):
42		logger.info("GreetingPlugin initialized with context")
43		self.language = self.config.get("language", "en")
44
45		def shutdown(self):
46		logger.info("GreetingPlugin shutting down")
47
48		@property
49		def version(self):
50		return "1.0.0"
51
52		def get_capabilities(self):
53		return ["greeting"]
54
55		def greet(self, name):
56		"""
57		Greet a person by name.
58
59		Args:
60		name: The name of the person to greet
61
62		Returns:
63		str: The greeting message
64		"""
65		if self.language == "en":
66		return f"Hello, {name}!"
67		elif self.language == "es":
68		return f"¡Hola, {name}!"
69		elif self.language == "fr":
70		return f"Bonjour, {name}!"
71		else:
72		return f"Hello, {name}!"
73
74
75		# Define a plugin that depends on another plugin
76		class AdvancedGreetingPlugin(SharedPlugin):
77		"""Plugin that extends the greeting capabilities."""
78
79		@property
80		def dependencies(self):
81		return ["GreetingPlugin"]
82
83		def initialize(self, context):
84		logger.info("AdvancedGreetingPlugin initialized")
85		self.context = context
86		self.formal = self.config.get("formal", False)
87
88		def get_capabilities(self):
89		return ["advanced_greeting"]
90
91		def formal_greet(self, name, title=""):
92		"""
93		Greet a person formally.
94
95		Args:
96		name: The name of the person to greet
97		title: Optional title (Mr., Ms., Dr., etc.)
98
99		Returns:
100		str: The formal greeting message
101		"""
102		# Get the basic greeting plugin
103		greeting_plugin = self.context.get("registry").get_plugin("GreetingPlugin")
104
105		# Get the basic greeting
106		basic_greeting = greeting_plugin.greet(f"{title} {name}").strip("!")
107
108		if self.formal:
109		return f"{basic_greeting}, it's a pleasure to meet you."
110		else:
111		return f"{basic_greeting}, how are you doing today?"
112
113
114		def main():
115		"""Run the plugin system example."""
116		try:
117		# Create a plugin registry
118		registry = PluginRegistry()
119		logger.info("Created plugin registry")
120
121		# Create plugin configurations
122		greeting_config = PluginConfig({
123		"language": "en"
124		})
125
126		advanced_config = PluginConfig({
127		"formal": True
128		})
129
130		# Register plugins with configurations
131		registry.register_plugin(GreetingPlugin, greeting_config.to_dict())
132		registry.register_plugin(AdvancedGreetingPlugin, advanced_config.to_dict())
133		logger.info("Registered plugins")
134
135		# Create a context for plugin initialization
136		context = {
137		"registry": registry,
138		"app_name": "PluginExample"
139		}
140
141		# Initialize all plugins
142		initialized_plugins = registry.initialize_all(context)
143		logger.info(f"Initialized {len(initialized_plugins)} plugins")
144
145		# Use the plugins
146		greeting_plugin = registry.get_plugin("GreetingPlugin")
147		advanced_plugin = registry.get_plugin("AdvancedGreetingPlugin")
148
149		# Basic greeting
150		greeting = greeting_plugin.greet("World")
151		logger.info(f"Basic greeting: {greeting}")
152
153		# Advanced greeting
154		formal_greeting = advanced_plugin.formal_greet("Smith", "Mr.")
155		logger.info(f"Advanced greeting: {formal_greeting}")
156
157		# Change configuration and see the effect
158		greeting_plugin.config["language"] = "es"
159		spanish_greeting = greeting_plugin.greet("Mundo")
160		logger.info(f"Spanish greeting: {spanish_greeting}")
161
162		# Shutdown plugins
163		registry.shutdown()
164		logger.info("Plugins shut down")
165
166		return 0
167
168		except PluginError as e:
169		# Handle plugin-specific errors
170		error_details = handle_plugin_exception(e)
171		logger.error(f"Plugin error: {error_details['error_message']}")
172		return 1
173
174		except Exception as e:
175		# Handle other errors
176		logger.exception("Unexpected error")
177		return 1
178
179
180		if __name__ == "__main__":
181		sys.exit(main())

ollamatools-project/examples/secure_doc_tools.py +0 -161

…	…	@@ -1,161 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example script demonstrating secure documentation tools with permission system integration.
4
5		This script shows how to use the documentation tools with the permission system
6		to ensure security when processing potentially sensitive documents.
7		"""
8
9		from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
10		from ollamatools.security import PermissionManager
11
12
13		def main():
14		"""Run the secure documentation tools example."""
15		print("=== OllamaTools Secure Documentation Tools Example ===\n")
16
17		# Initialize the permission manager
18		permission_manager = PermissionManager(
19		prompt_enabled=True,
20		sudo_always_prompt=True
21		)
22
23		print("Permission manager initialized. All documentation operations will require permission.\n")
24
25		# Part 1: Secure Document Generation
26		print("\n--- Secure Document Generation Example ---\n")
27
28		# Sample content for documentation
29		content = """
30		## Secure Document Generation
31
32		This document demonstrates the integration of OllamaTools' documentation
33		utilities with the permission system for enhanced security.
34
35		## Security Features
36
37		- Permission prompts before document generation
38		- Secure handling of potentially sensitive content
39		- Integration with OllamaTools permission system
40
41		## Usage with Permission System
42
43		```python
44		from ollamatools.tools.doc_tools import generate_markdown
45		from ollamatools.security import PermissionManager
46
47		# Initialize permission manager
48		permission_manager = PermissionManager()
49
50		# Generate markdown with permission checks
51		doc = generate_markdown(content, title,
52		format_type='markdown',
53		permission_manager=permission_manager)
54		```
55		"""
56
57		# Generate markdown document with permission check
58		print("Attempting to generate markdown document...\n")
59		print("You will be prompted for permission to generate the document.")
60		print("(Choose 'y' to allow, 'n' to deny)\n")
61
62		try:
63		title = "Secure Documentation Example"
64		markdown_doc = generate_markdown(
65		content,
66		title,
67		format_type='markdown',
68		permission_manager=permission_manager
69		)
70		print("\nPermission granted! Document generated successfully.\n")
71		print("First 200 characters of the document:")
72		print("-" * 50)
73		print(markdown_doc[:200] + "...")
74
75		# Save the document if permission was granted
76		with open("secure_doc.md", "w") as f:
77		f.write(markdown_doc)
78		print("\nSaved markdown document to 'secure_doc.md'")
79
80		except PermissionError as e:
81		print(f"\nDocument generation failed: {e}")
82
83		# Part 2: Secure Context Extraction
84		print("\n\n--- Secure Context Extraction Example ---\n")
85
86		# Sample documents (potentially sensitive information)
87		documents = [
88		"The system password is 'secure123'. It should be changed every 30 days.",
89		"Python is a programming language that is widely used in data science.",
90		"The admin credentials are stored in the config file at /etc/app/config.json.",
91		"JavaScript is a programming language used for web development."
92		]
93
94		query = "programming language"
95
96		print(f"Attempting to extract context about '{query}' from documents...\n")
97		print("You will be prompted for permission to extract context from documents.")
98		print("(Choose 'y' to allow, 'n' to deny)\n")
99
100		try:
101		context = extract_context(
102		query,
103		documents,
104		max_length=500,
105		permission_manager=permission_manager
106		)
107		print("\nPermission granted! Context extracted successfully.\n")
108		print("Extracted Context:")
109		print("-" * 50)
110		print(context)
111
112		except PermissionError as e:
113		print(f"\nContext extraction failed: {e}")
114
115		# Part 3: Secure Text Summarization
116		print("\n\n--- Secure Text Summarization Example ---\n")
117
118		# Sample text to summarize (potentially sensitive information)
119		long_text = """
120		Project XYZ Security Analysis Report - CONFIDENTIAL
121
122		The security audit of Project XYZ revealed several critical vulnerabilities that need immediate attention.
123		The most severe issue is the unencrypted storage of user credentials in the database. This violates our security policy and industry standards.
124		We also found that the API endpoints are not properly protected against SQL injection attacks. This could allow malicious users to access sensitive data.
125		The authentication mechanism uses outdated hashing algorithms (MD5) which are considered insecure by modern standards.
126		The session management system doesn't implement proper timeout controls, allowing sessions to remain active indefinitely.
127		All API requests are transmitted over HTTP rather than HTTPS, exposing data to potential man-in-the-middle attacks.
128
129		We recommend implementing the following changes immediately:
130		1. Encrypt all stored credentials using industry-standard algorithms.
131		2. Implement parameterized queries for all database operations.
132		3. Update the password hashing to use bcrypt or Argon2.
133		4. Implement a session timeout of 30 minutes for inactive users.
134		5. Enforce HTTPS for all API communications.
135
136		These issues must be addressed before the product can be deployed to production.
137		"""
138
139		print("Attempting to summarize confidential text...\n")
140		print("You will be prompted for permission to summarize the text.")
141		print("(Choose 'y' to allow, 'n' to deny)\n")
142
143		try:
144		summary = summarize_text(
145		long_text,
146		max_length=150,
147		permission_manager=permission_manager
148		)
149		print("\nPermission granted! Text summarized successfully.\n")
150		print("Original Text Length:", len(long_text))
151		print("Summary Length:", len(summary))
152		print("Summary:")
153		print("-" * 50)
154		print(summary)
155
156		except PermissionError as e:
157		print(f"\nText summarization failed: {e}")
158
159
160		if __name__ == "__main__":
161		main()

ollamatools-project/examples/secure_web_tools.py +0 -100

…	…	@@ -1,100 +0,0 @@
1		"""
2		Example demonstrating the use of web tools with permission system in OllamaTools.
3
4		This script shows how to securely use the web tools with the permission system.
5		"""
6
7		from ollamatools.tools.web_tools import fetch_url, get_weather, search_web
8		from ollamatools.security import PermissionManager
9		import json
10
11		def main():
12		"""Run the secure web tools example."""
13		print("OllamaTools Secure Web Tools Example")
14		print("===================================")
15
16		# Create a permission manager with prompting enabled
17		permission_manager = PermissionManager(prompt_enabled=True)
18		print("\nCreated permission manager with prompting enabled.")
19		print("You will be prompted for permission before each web request.")
20
21		# Example 1: Basic fetch_url with permission
22		print("\n1. Secure URL Fetch")
23		print("-----------------")
24		try:
25		print("Attempting to fetch https://httpbin.org/get...")
26		result = fetch_url(
27		"https://httpbin.org/get",
28		permission_manager=permission_manager
29		)
30		print(f"Request successful! Status: {result['status']}")
31		print(f"Response content type: {result['headers'].get('Content-Type', 'Unknown')}")
32		print(f"Response URL: {result['url']}")
33		except PermissionError as e:
34		print(f"Permission denied: {e}")
35		except Exception as e:
36		print(f"Error: {e}")
37
38		# Example 2: fetch_url to a different domain
39		print("\n2. Fetch from a Different Domain")
40		print("------------------------------")
41		try:
42		print("Attempting to fetch https://example.com...")
43		result = fetch_url(
44		"https://example.com",
45		permission_manager=permission_manager
46		)
47		print(f"Request successful! Status: {result['status']}")
48		print(f"Response content type: {result['headers'].get('Content-Type', 'Unknown')}")
49		print(f"Response content length: {len(str(result['content']))} characters")
50		except PermissionError as e:
51		print(f"Permission denied: {e}")
52		except Exception as e:
53		print(f"Error: {e}")
54
55		# Example 3: Search web with permission
56		print("\n3. Secure Web Search")
57		print("-----------------")
58		try:
59		print("Attempting to search for 'Python programming'...")
60		results = search_web(
61		"Python programming",
62		num_results=3,
63		permission_manager=permission_manager
64		)
65		print(f"Search successful! Found {len(results)} results")
66		for i, result in enumerate(results[:2]): # Show first 2 results
67		print(f"\nResult {i+1}:")
68		print(f" Title: {result['title']}")
69		print(f" URL: {result['url']}")
70		except PermissionError as e:
71		print(f"Permission denied: {e}")
72		except Exception as e:
73		print(f"Error: {e}")
74
75		# Example 4: POST request with permission
76		print("\n4. Secure POST Request")
77		print("--------------------")
78		try:
79		print("Attempting to make a POST request to https://httpbin.org/post...")
80		data = {"name": "OllamaTools", "purpose": "security_demo"}
81		result = fetch_url(
82		"https://httpbin.org/post",
83		method="POST",
84		data=data,
85		permission_manager=permission_manager
86		)
87		print(f"Request successful! Status: {result['status']}")
88		print(f"Server received our data: {json.dumps(result['content'].get('json', {}))}")
89		except PermissionError as e:
90		print(f"Permission denied: {e}")
91		except Exception as e:
92		print(f"Error: {e}")
93
94		print("\nExample completed. The permission system allows users to control which")
95		print("web requests are allowed, providing an additional layer of security.")
96		print("This can be especially important when functions are automatically triggered")
97		print("by an AI model.")
98
99		if __name__ == "__main__":
100		main()

ollamatools-project/examples/security_example.py +0 -554

…	…	@@ -1,554 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Example demonstrating the enhanced security features in OllamaTools.
4
5		This script shows how to use the command validation, path validation,
6		URL validation, and error handling features safely.
7		"""
8
9		import os
10		import sys
11		import logging
12		from typing import Set, Dict, Any
13
14		# Add the parent directory to the path so we can import the package
15		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
16
17		from ollamatools.security import PermissionManager
18		from ollamatools.command_validator import (
19		validate_and_sanitize_command,
20		CommandValidationError,
21		DEFAULT_ALLOWLIST
22		)
23		from ollamatools.path_validator import (
24		validate_path_for_reading,
25		validate_path_for_writing,
26		validate_path_for_searching,
27		PathValidationError,
28		configure_path_sandbox
29		)
30		from ollamatools.url_validator import (
31		validate_url,
32		URLValidationError,
33		configure_domain_lists
34		)
35		from ollamatools.errors import (
36		OllamaToolsError,
37		handle_exception,
38		format_error_response
39		)
40		from ollamatools.tools.system_tools import (
41		execute_command,
42		read_file,
43		write_file,
44		find_files
45		)
46		from ollamatools.tools.web_tools import fetch_url
47		from ollamatools.utils.logger import setup_logger
48
49		# Set up logging
50		logger = setup_logger("security_example", level="INFO")
51
52		def demonstrate_command_validation():
53		"""Demonstrate the command validation features."""
54		print("\n=== Command Validation Demo ===")
55
56		try:
57		# Example 1: Basic validation and execution
58		print("\n--- Example 1: Valid Command ---")
59		command = "ls"
60		args = ["-la"]
61
62		print(f"Validating command: {command} {' '.join(args)}")
63		validated_command, validated_args = validate_and_sanitize_command(command, args)
64		print(f"Validated command: {validated_command} {' '.join(validated_args)}")
65
66		result = execute_command(command, args)
67		print(f"Command executed with result code: {result.get('returncode')}")
68		print(f"First few lines of output: {result.get('stdout', '')[:200]}...")
69
70		# Example 2: Command not in allowlist
71		print("\n--- Example 2: Command Not in Allowlist ---")
72		command = "malicious_command"
73		args = []
74
75		print(f"Validating command: {command}")
76		try:
77		validated_command, validated_args = validate_and_sanitize_command(command, args)
78		print("This should not be reached")
79		except CommandValidationError as e:
80		print(f"Validation error as expected: {e}")
81
82		# Example 3: Custom allowlist
83		print("\n--- Example 3: Custom Allowlist ---")
84		custom_allowlist = DEFAULT_ALLOWLIST.copy()
85		custom_allowlist.add("custom_command")
86
87		command = "custom_command"
88		args = ["--option", "value"]
89
90		print(f"Validating command with custom allowlist: {command}")
91		try:
92		validated_command, validated_args = validate_and_sanitize_command(
93		command, args, custom_allowlist=custom_allowlist
94		)
95		print(f"Command validated successfully: {validated_command} {' '.join(validated_args)}")
96		except CommandValidationError as e:
97		print(f"Validation error: {e}")
98
99		# Example 4: Command with dangerous arguments
100		print("\n--- Example 4: Command with Dangerous Arguments ---")
101		command = "ls"
102		args = ["|", "rm", "-rf", "/"] # Dangerous pipe to rm
103
104		print(f"Validating command with dangerous args: {command} {' '.join(args)}")
105		try:
106		validated_command, validated_args = validate_and_sanitize_command(command, args)
107		print("This should not be reached")
108		except CommandValidationError as e:
109		print(f"Validation error as expected: {e}")
110
111		except Exception as e:
112		error_details = handle_exception(e)
113		print(f"Unexpected error: {error_details['error_message']}")
114
115		def demonstrate_path_validation():
116		"""Demonstrate the path validation features."""
117		print("\n=== Path Validation Demo ===")
118
119		try:
120		# Example 1: Reading a valid file
121		print("\n--- Example 1: Reading a Valid File ---")
122		# Use README.md which should exist in the project
123		file_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), "README.md")
124
125		print(f"Validating path for reading: {file_path}")
126		try:
127		validated_path = validate_path_for_reading(file_path)
128		print(f"Path validated successfully: {validated_path}")
129
130		content = read_file(file_path)
131		print(f"First 100 characters of file content: {content[:100]}...")
132		except PathValidationError as e:
133		print(f"Path validation error: {e}")
134
135		# Example 2: Writing to a safe path
136		print("\n--- Example 2: Writing to a Safe Path ---")
137		# Use a temporary file in the current directory
138		write_path = os.path.join(os.getcwd(), "temp_test_file.txt")
139
140		print(f"Validating path for writing: {write_path}")
141		try:
142		validated_path = validate_path_for_writing(write_path)
143		print(f"Path validated successfully: {validated_path}")
144
145		content = "This is a test file created by the security example."
146		success = write_file(write_path, content)
147		print(f"Write success: {success}")
148
149		# Clean up
150		if os.path.exists(write_path):
151		os.remove(write_path)
152		print(f"Cleaned up test file: {write_path}")
153		except PathValidationError as e:
154		print(f"Path validation error: {e}")
155
156		# Example 3: Path traversal attempt
157		print("\n--- Example 3: Path Traversal Attempt ---")
158		traversal_path = "../../../etc/passwd"
159
160		print(f"Validating path with traversal: {traversal_path}")
161		try:
162		validated_path = validate_path_for_reading(traversal_path)
163		print("This should not be reached")
164		except PathValidationError as e:
165		print(f"Path validation error as expected: {e}")
166
167		# Example 4: Protected path write attempt
168		print("\n--- Example 4: Protected Path Write Attempt ---")
169		protected_path = "/etc/hosts"
170
171		print(f"Validating protected path for writing: {protected_path}")
172		try:
173		validated_path = validate_path_for_writing(protected_path)
174		print("This should not be reached")
175		except PathValidationError as e:
176		print(f"Path validation error as expected: {e}")
177
178		# Example 5: Custom sandbox configuration
179		print("\n--- Example 5: Custom Sandbox Configuration ---")
180		# Create a temporary directory for the sandbox
181		sandbox_dir = os.path.join(os.getcwd(), "sandbox_test")
182		os.makedirs(sandbox_dir, exist_ok=True)
183
184		# Configure a custom sandbox
185		sandbox_config = configure_path_sandbox(
186		sandbox_path=sandbox_dir,
187		additional_allowed_paths={os.path.join(os.getcwd(), "README.md")},
188		additional_protected_paths=set()
189		)
190
191		print(f"Configured custom sandbox at: {sandbox_dir}")
192
193		# Try to write to the sandbox
194		sandbox_file = os.path.join(sandbox_dir, "test.txt")
195		print(f"Writing to file in custom sandbox: {sandbox_file}")
196
197		try:
198		validated_path = validate_path_for_writing(
199		sandbox_file,
200		sandbox_path=sandbox_config['sandbox_path']
201		)
202		print(f"Path validated successfully: {validated_path}")
203
204		content = "This is a test file in the custom sandbox."
205		success = write_file(
206		sandbox_file,
207		content,
208		sandbox_path=sandbox_config['sandbox_path']
209		)
210		print(f"Write success: {success}")
211		except PathValidationError as e:
212		print(f"Path validation error: {e}")
213
214		# Clean up the sandbox
215		if os.path.exists(sandbox_file):
216		os.remove(sandbox_file)
217		if os.path.exists(sandbox_dir):
218		os.rmdir(sandbox_dir)
219		print(f"Cleaned up sandbox directory: {sandbox_dir}")
220
221		except Exception as e:
222		error_details = handle_exception(e)
223		print(f"Unexpected error: {error_details['error_message']}")
224
225		def demonstrate_url_validation():
226		"""Demonstrate the URL validation features."""
227		print("\n=== URL Validation Demo ===")
228
229		try:
230		# Example 1: Valid URL
231		print("\n--- Example 1: Valid URL ---")
232		url = "https://www.example.com/path?param=value"
233
234		print(f"Validating URL: {url}")
235		try:
236		validated_url = validate_url(url)
237		print(f"URL validated successfully: {validated_url}")
238		except URLValidationError as e:
239		print(f"URL validation error: {e}")
240
241		# Example 2: Blocked domain
242		print("\n--- Example 2: Blocked Domain ---")
243		blocked_url = "http://localhost:8080/path"
244
245		print(f"Validating blocked URL: {blocked_url}")
246		try:
247		validated_url = validate_url(blocked_url)
248		print("This should not be reached")
249		except URLValidationError as e:
250		print(f"URL validation error as expected: {e}")
251
252		# Example 3: Disallowed scheme
253		print("\n--- Example 3: Disallowed Scheme ---")
254		bad_scheme_url = "file:///etc/passwd"
255
256		print(f"Validating URL with disallowed scheme: {bad_scheme_url}")
257		try:
258		validated_url = validate_url(bad_scheme_url)
259		print("This should not be reached")
260		except URLValidationError as e:
261		print(f"URL validation error as expected: {e}")
262
263		# Example 4: Custom domain configuration
264		print("\n--- Example 4: Custom Domain Configuration ---")
265
266		# Configure custom domain lists
267		domain_allowlist, domain_blocklist = configure_domain_lists(
268		additional_allowlist={"trusted-site.com"},
269		additional_blocklist={"malicious-site.com"}
270		)
271
272		# Test a custom allowed domain
273		allowed_url = "https://trusted-site.com/path"
274		print(f"Validating URL with custom allowlist: {allowed_url}")
275
276		try:
277		validated_url = validate_url(
278		allowed_url,
279		domain_allowlist=domain_allowlist,
280		domain_blocklist=domain_blocklist
281		)
282		print(f"URL validated successfully: {validated_url}")
283		except URLValidationError as e:
284		print(f"URL validation error: {e}")
285
286		# Test a custom blocked domain
287		blocked_url = "https://malicious-site.com/path"
288		print(f"Validating URL with custom blocklist: {blocked_url}")
289
290		try:
291		validated_url = validate_url(
292		blocked_url,
293		domain_allowlist=domain_allowlist,
294		domain_blocklist=domain_blocklist
295		)
296		print("This should not be reached")
297		except URLValidationError as e:
298		print(f"URL validation error as expected: {e}")
299
300		# Example 5: URL normalization
301		print("\n--- Example 5: URL Normalization ---")
302		unnormalized_url = "HTTP://Example.COM/path/../other/./file.html?param=value#fragment"
303
304		print(f"Normalizing URL: {unnormalized_url}")
305		try:
306		validated_url = validate_url(unnormalized_url)
307		print(f"URL normalized to: {validated_url}")
308		except URLValidationError as e:
309		print(f"URL validation error: {e}")
310
311		# Example 6: Real HTTP request
312		print("\n--- Example 6: Real HTTP Request ---")
313		request_url = "https://httpbin.org/get"
314
315		print(f"Making request to: {request_url}")
316		try:
317		result = fetch_url(request_url)
318		print(f"Request successful with status: {result.get('status')}")
319		print(f"Response snippet: {str(result.get('content'))[:100]}...")
320		except Exception as e:
321		print(f"Request failed: {e}")
322
323		except Exception as e:
324		error_details = handle_exception(e)
325		print(f"Unexpected error: {error_details['error_message']}")
326
327		def demonstrate_error_handling():
328		"""Demonstrate the standardized error handling."""
329		print("\n=== Error Handling Demo ===")
330
331		try:
332		# Example 1: Basic error handling
333		print("\n--- Example 1: Basic Error Handling ---")
334		try:
335		# Force an error
336		result = 1 / 0
337		except Exception as e:
338		# Handle the error
339		error_response = format_error_response(e)
340		print(f"Error type: {error_response['error_type']}")
341		print(f"Error message: {error_response['error_message']}")
342
343		# Example 2: OllamaToolsError
344		print("\n--- Example 2: OllamaToolsError ---")
345		try:
346		# Simulate a command error
347		from ollamatools.errors import CommandError, ERROR_CODES
348		raise CommandError(
349		message="Command failed to execute",
350		command="test_command",
351		args=["--option", "value"],
352		exit_code=1,
353		stderr="Error: invalid option",
354		code=ERROR_CODES["COMMAND_EXECUTION_ERROR"]
355		)
356		except OllamaToolsError as e:
357		# Handle the error with the utility function
358		error_details = handle_exception(e, log_error=True)
359		print(f"Error code: {error_details.get('error_code')}")
360		print(f"Error message: {error_details['error_message']}")
361
362		# Example 3: Nested error handling
363		print("\n--- Example 3: Nested Error Handling ---")
364		try:
365		# Outer try block
366		try:
367		# Inner operation that fails
368		from ollamatools.errors import FileError
369		raise FileError(
370		message="File could not be read",
371		path="/path/to/nonexistent/file.txt",
372		code=ERROR_CODES["FILE_NOT_FOUND"]
373		)
374		except FileError as e:
375		# Convert to a different error type
376		from ollamatools.errors import ApiError
377		raise ApiError(
378		message=f"API failed due to file error: {e}",
379		api_name="file_api",
380		status_code=404,
381		code=ERROR_CODES["API_ERROR"]
382		) from e
383		except OllamaToolsError as e:
384		# Handle the final error
385		error_details = handle_exception(e, include_traceback=True)
386		print(f"Final error message: {error_details['error_message']}")
387		if 'traceback' in error_details:
388		print(f"Error has traceback with {len(error_details['traceback'])} frames")
389
390		except Exception as e:
391		error_details = handle_exception(e)
392		print(f"Unexpected error: {error_details['error_message']}")
393
394		def demonstrate_permission_system():
395		"""Demonstrate the enhanced permission system."""
396		print("\n=== Permission System Demo ===")
397		print("This demo will prompt for permissions interactively.")
398		print("You'll need to respond to permission requests.")
399
400		# Create a permission manager with prompts enabled
401		permission_manager = PermissionManager(
402		prompt_enabled=True,
403		sudo_always_prompt=True
404		)
405
406		try:
407		# Example 1: Basic permission request
408		print("\n--- Example 1: Basic Permission Request ---")
409		command = "ls"
410		args = ["-la"]
411
412		print(f"Requesting permission to execute: {command} {' '.join(args)}")
413		permission_granted = permission_manager.request_permission(
414		function_name="execute_command",
415		args={"command": command, "args": args}
416		)
417
418		if permission_granted:
419		print("Permission granted! Executing command...")
420		result = execute_command(command, args, permission_manager=permission_manager)
421		print(f"Command executed with result code: {result.get('returncode')}")
422		else:
423		print("Permission denied. Command not executed.")
424
425		# Example 2: Sudo command
426		print("\n--- Example 2: Sudo Command ---")
427		sudo_command = "sudo"
428		sudo_args = ["ls", "/root"]
429
430		print(f"Requesting permission to execute sudo command: {sudo_command} {' '.join(sudo_args)}")
431		# Note: is_sudo will be auto-detected, but we set it explicitly here for the example
432		permission_granted = permission_manager.request_permission(
433		function_name="execute_command",
434		args={"command": sudo_command, "args": sudo_args},
435		is_sudo=True
436		)
437
438		if permission_granted:
439		print("Permission granted! Executing sudo command...")
440		# Note: This may fail if the user doesn't have sudo privileges
441		result = execute_command(sudo_command, sudo_args, permission_manager=permission_manager)
442		print(f"Command executed with result code: {result.get('returncode')}")
443		else:
444		print("Permission denied. Sudo command not executed.")
445
446		# Example 3: Session permissions
447		print("\n--- Example 3: Session Permissions ---")
448		print("When prompted, choose 'S' to allow for the session.")
449
450		# First request - will prompt
451		file_path = "README.md"
452		print(f"Requesting permission to read file: {file_path}")
453		permission_granted = permission_manager.request_permission(
454		function_name="read_file",
455		args={"path": file_path}
456		)
457
458		if permission_granted:
459		print("Permission granted! Reading file...")
460		content = read_file(file_path, permission_manager=permission_manager)
461		print(f"File read successfully, first 50 chars: {content[:50]}...")
462		else:
463		print("Permission denied. File not read.")
464
465		# Second request - should not prompt if session permission was granted
466		print("\nMaking a second request for the same operation...")
467		print(f"Requesting permission to read file: {file_path}")
468		permission_granted = permission_manager.request_permission(
469		function_name="read_file",
470		args={"path": file_path}
471		)
472
473		if permission_granted:
474		print("Permission granted without prompting! Reading file...")
475		content = read_file(file_path, permission_manager=permission_manager)
476		print(f"File read successfully, first 50 chars: {content[:50]}...")
477		else:
478		print("Permission denied. File not read.")
479
480		# Example 4: Command modification
481		print("\n--- Example 4: Command Modification ---")
482		print("When prompted, choose 'M' to modify the operation.")
483
484		command = "cat"
485		args = ["/etc/passwd"] # This would be a sensitive file
486
487		print(f"Requesting permission to execute: {command} {' '.join(args)}")
488		permission_granted = permission_manager.request_permission(
489		function_name="execute_command",
490		args={"command": command, "args": args}
491		)
492
493		if permission_granted:
494		print("Permission granted (after modification)! Executing command...")
495		result = execute_command(command, args, permission_manager=permission_manager)
496		print(f"Command executed with result code: {result.get('returncode')}")
497		else:
498		print("Permission denied or modification cancelled. Command not executed.")
499
500		except Exception as e:
501		error_details = handle_exception(e)
502		print(f"Unexpected error: {error_details['error_message']}")
503
504		def main():
505		"""Run all the security demonstration examples."""
506		print("=== OllamaTools Security Features Demonstration ===")
507		print("This script demonstrates the enhanced security features in OllamaTools.")
508
509		try:
510		# Choose which demos to run
511		demos = {
512		"1": ("Command Validation", demonstrate_command_validation),
513		"2": ("Path Validation", demonstrate_path_validation),
514		"3": ("URL Validation", demonstrate_url_validation),
515		"4": ("Error Handling", demonstrate_error_handling),
516		"5": ("Permission System", demonstrate_permission_system),
517		"all": ("All Demos", None)
518		}
519
520		print("\nAvailable demos:")
521		for key, (name, _) in demos.items():
522		if key != "all":
523		print(f" {key}: {name}")
524		print(" all: Run all demos")
525
526		choice = input("\nEnter demo number(s) to run (e.g. '1', '1 3 5', or 'all'): ").strip().lower()
527
528		if choice == "all":
529		# Run all demos
530		demonstrate_command_validation()
531		demonstrate_path_validation()
532		demonstrate_url_validation()
533		demonstrate_error_handling()
534		demonstrate_permission_system()
535		else:
536		# Run selected demos
537		selected = choice.split()
538		for demo_id in selected:
539		if demo_id in demos and demo_id != "all":
540		print(f"\nRunning demo: {demos[demo_id][0]}")
541		demos[demo_id][1]()
542		else:
543		print(f"Unknown demo ID: {demo_id}")
544
545		print("\n=== Security Demonstration Completed ===")
546
547		except KeyboardInterrupt:
548		print("\n\nDemonstration interrupted by user.")
549		except Exception as e:
550		error_details = handle_exception(e, log_error=True)
551		print(f"\nUnexpected error in main function: {error_details['error_message']}")
552
553		if __name__ == "__main__":
554		main()

ollamatools-project/examples/test_commands.py +0 -89

…	…	@@ -1,89 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Test script for system tools command execution.
4
5		This script provides a simple way to test the system tools directly.
6		"""
7
8		import os
9		import sys
10		import shlex
11		import subprocess
12
13		# Add the parent directory to the path
14		parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
15		if parent_dir not in sys.path:
16		sys.path.insert(0, parent_dir)
17
18		# Function to directly execute a command using subprocess
19		def direct_execute_command(command):
20		"""Execute a command directly using subprocess."""
21		print(f"\nExecuting: '{command}'")
22
23		if isinstance(command, str) and ' ' in command:
24		# Split the command with shlex
25		parts = shlex.split(command)
26		cmd = parts[0]
27		args = parts[1:] if len(parts) > 1 else []
28		print(f"Split into cmd='{cmd}', args={args}")
29		else:
30		cmd = command
31		args = []
32		print(f"Using cmd='{cmd}', args={args}")
33
34		try:
35		result = subprocess.run(
36		[cmd] + args,
37		capture_output=True,
38		text=True
39		)
40		print(f"Exit code: {result.returncode}")
41		print(f"Stdout: {result.stdout}")
42		print(f"Stderr: {result.stderr}")
43		return result
44		except Exception as e:
45		print(f"Error: {e}")
46		return None
47
48		# Function to find files using glob
49		def direct_find_files(pattern, directory='.'):
50		"""Find files using glob."""
51		import glob
52		print(f"\nFinding files with pattern '{pattern}' in directory '{directory}'")
53		try:
54		files = glob.glob(os.path.join(directory, pattern))
55		print(f"Found {len(files)} files")
56		for i, file in enumerate(files[:5]):
57		print(f" {i+1}. {file}")
58		if len(files) > 5:
59		print(f" ... and {len(files) - 5} more")
60		return files
61		except Exception as e:
62		print(f"Error: {e}")
63		return []
64
65		def main():
66		"""Run the tests."""
67		print("\n=== Testing Command Execution ===")
68
69		# Test 1: Direct execution with 'ls -la'
70		direct_execute_command("ls -la")
71
72		# Test 2: Direct execution with 'pwd'
73		direct_execute_command("pwd")
74
75		# Test 3: Direct execution with 'echo "Hello World"'
76		direct_execute_command('echo "Hello World"')
77
78		print("\n=== Testing File Finding ===")
79
80		# Test 4: Find Python files in current directory
81		direct_find_files("*.py")
82
83		# Test 5: Find markdown files in root directory
84		direct_find_files("*.md", parent_dir)
85
86		print("\n=== Tests Complete ===")
87
88		if __name__ == "__main__":
89		main()

ollamatools-project/examples/test_ollama_integration.py +0 -84

…	…	@@ -1,84 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Integration test script for the refactored OllamaAgent with Ollama's API.
4
5		This script tests the integration between the refactored OllamaAgent
6		and Ollama's function calling API. It verifies that the agent can
7		properly pass tools to Ollama and execute function calls.
8
9		Usage:
10		python test_ollama_integration.py
11		"""
12
13		import os
14		import sys
15		import json
16		import logging
17		from typing import Dict, Any, List, Optional
18
19		# Add the parent directory to the path
20		parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
21		if parent_dir not in sys.path:
22		sys.path.insert(0, parent_dir)
23
24		from ollamatools import OllamaAgent, Config
25		from ollamatools.security import PermissionManager
26		from ollamatools.utils.logger import setup_logger
27
28		# Configure logging
29		logger = setup_logger("ollama_integration", level="INFO")
30
31		def main():
32		"""Run the integration test."""
33		# Test with permission prompts disabled for automation
34		config = Config(
35		model="llama3.1:latest", # Use your available model
36		host="http://localhost:11434",
37		permission_prompts=False,
38		sudo_always_prompt=False
39		)
40
41		try:
42		# 1. Test agent initialization
43		logger.info("Testing agent initialization...")
44		agent = OllamaAgent(config=config)
45		logger.info("Agent initialized successfully")
46
47		# 2. Test basic chat without function calls
48		logger.info("Testing basic chat without function calls...")
49		response = agent.chat("Hello, what's your name?")
50		logger.info(f"Response: {response}")
51
52		# 3. Test chat that should trigger system commands
53		logger.info("Testing chat that should trigger system commands...")
54		response = agent.chat("What files are in the current directory?")
55		logger.info(f"Response: {response}")
56
57		# 4. Test chat that should trigger web requests
58		logger.info("Testing chat that should trigger web requests...")
59		response = agent.chat("What's the weather like in New York?")
60		logger.info(f"Response: {response}")
61
62		# 5. Test chat that should trigger documentation tools
63		logger.info("Testing chat that should trigger documentation tools...")
64		response = agent.chat("Create a markdown document about Python programming.")
65		logger.info(f"Response: {response}")
66
67		# 6. Test chat with conversation context
68		logger.info("Testing chat with conversation context...")
69		context = [
70		{"role": "user", "content": "I want to learn about file operations."},
71		{"role": "assistant", "content": "I'd be happy to help you learn about file operations in various programming languages. What specific aspect are you interested in?"}
72		]
73		response = agent.chat("Can you write a Python function to count lines in a file?", context=context)
74		logger.info(f"Response: {response}")
75
76		logger.info("All tests completed successfully!")
77		return 0
78
79		except Exception as e:
80		logger.error(f"Error during integration test: {str(e)}")
81		return 1
82
83		if __name__ == "__main__":
84		sys.exit(main())

ollamatools-project/examples/test_performance.py +0 -188

…	…	@@ -1,188 +0,0 @@
1		import os
2		import time
3		import unittest
4		import tempfile
5		import random
6		import string
7		from unittest.mock import patch, MagicMock, PropertyMock
8
9		import pytest
10
11		from ollamatools.config import Config
12		from ollamatools.security import PermissionManager
13		from ollamatools.agent import OllamaAgent
14		from ollamatools.tools.system_tools import execute_command, read_file, write_file, find_files
15
16
17		class TestPerformance(unittest.TestCase):
18		def setUp(self):
19		self.permission_manager = PermissionManager(
20		prompt_enabled=False # Disable for testing
21		)
22
23		# Create temp directory for file operations
24		self.temp_dir = tempfile.mkdtemp()
25
26		# Create a number of test files
27		self.num_files = 10 # Reduced for faster testing
28		self.test_files = []
29
30		for i in range(self.num_files):
31		filename = os.path.join(self.temp_dir, f"test_file_{i}.txt")
32		content = ''.join(random.choices(string.ascii_letters + string.digits, k=1000))
33		with open(filename, "w") as f:
34		f.write(content)
35		self.test_files.append(filename)
36
37		def tearDown(self):
38		# Clean up the test files
39		for filename in self.test_files:
40		if os.path.exists(filename):
41		os.remove(filename)
42
43		# Remove the temp directory
44		if os.path.exists(self.temp_dir):
45		os.rmdir(self.temp_dir)
46
47		def test_file_operations_performance(self):
48		"""Test the performance of file operations."""
49		# Measure time to read all test files
50		start_time = time.time()
51
52		for filename in self.test_files:
53		content = read_file(
54		path=filename,
55		permission_manager=self.permission_manager
56		)
57		self.assertIsNotNone(content)
58
59		read_duration = time.time() - start_time
60		print(f"Read {self.num_files} files in {read_duration:.2f} seconds")
61
62		# Measure time to write all test files
63		start_time = time.time()
64
65		for i, filename in enumerate(self.test_files):
66		content = f"New content for file {i}"
67		success = write_file(
68		path=filename,
69		content=content,
70		permission_manager=self.permission_manager
71		)
72		self.assertTrue(success)
73
74		write_duration = time.time() - start_time
75		print(f"Wrote {self.num_files} files in {write_duration:.2f} seconds")
76
77		# Measure time to find all test files
78		start_time = time.time()
79
80		found_files = find_files(
81		pattern="*.txt",
82		directory=self.temp_dir,
83		recursive=True,
84		permission_manager=self.permission_manager
85		)
86
87		find_duration = time.time() - start_time
88		print(f"Found {len(found_files)} files in {find_duration:.2f} seconds")
89
90		# Assert all files were found
91		self.assertEqual(len(found_files), self.num_files)
92
93		# Assert reasonable performance
94		# These thresholds are arbitrary and may need adjustment
95		self.assertLess(read_duration, 2.0, f"Reading {self.num_files} files took too long")
96		self.assertLess(write_duration, 2.0, f"Writing {self.num_files} files took too long")
97		self.assertLess(find_duration, 1.0, f"Finding {self.num_files} files took too long")
98
99		@patch("ollamatools.tools.system_tools.subprocess.run")
100		def test_command_execution_performance(self, mock_run):
101		"""Test the performance of command execution."""
102		# Mock subprocess.run to return quickly
103		mock_process = MagicMock()
104		mock_process.stdout = "command output" # Using string instead of bytes
105		mock_process.stderr = ""
106		mock_process.returncode = 0
107		mock_run.return_value = mock_process
108
109		# Execute a command multiple times and measure performance
110		num_executions = 10 # Reduced for faster testing
111		start_time = time.time()
112
113		for i in range(num_executions):
114		result = execute_command(
115		command="echo",
116		args=[f"test {i}"],
117		permission_manager=self.permission_manager
118		)
119		self.assertEqual(result["returncode"], 0)
120
121		duration = time.time() - start_time
122		print(f"Executed {num_executions} commands in {duration:.2f} seconds")
123
124		# Assert reasonable performance
125		# This threshold is arbitrary and may need adjustment
126		self.assertLess(duration, 2.0, f"Executing {num_executions} commands took too long")
127
128		# Test the throughput
129		throughput = num_executions / duration
130		print(f"Command execution throughput: {throughput:.2f} commands/second")
131
132		# Assert reasonable throughput
133		# This threshold is arbitrary and may need adjustment
134		self.assertGreater(throughput, 5.0, f"Command execution throughput is too low")
135
136		@patch("ollamatools.agent.ollama")
137		def test_agent_performance(self, mock_ollama):
138		"""Test the performance of the OllamaAgent."""
139		# Set up mock client
140		mock_client = MagicMock()
141		mock_ollama.Client.return_value = mock_client
142
143		# Create mock response object
144		mock_response = MagicMock()
145
146		# Set up mock message with content property that returns a string
147		mock_message = MagicMock()
148		type(mock_message).content = PropertyMock(return_value="This is a test response")
149		type(mock_message).tool_calls = PropertyMock(return_value=None)
150
151		# Set up mock response with message property that returns mock_message
152		type(mock_response).message = PropertyMock(return_value=mock_message)
153
154		# Configure mock client's chat method to return mock response
155		mock_client.chat.return_value = mock_response
156
157		# Create the agent
158		agent = OllamaAgent(
159		model="llama3.1",
160		permission_prompts=False
161		)
162
163		# Make multiple chat requests and measure performance
164		num_chats = 10
165		start_time = time.time()
166
167		for i in range(num_chats):
168		response = agent.chat(f"Test message {i}")
169		self.assertEqual(response, "This is a test response")
170
171		duration = time.time() - start_time
172		print(f"Made {num_chats} chat requests in {duration:.2f} seconds")
173
174		# Assert reasonable performance
175		# This threshold is arbitrary and may need adjustment
176		self.assertLess(duration, 2.0, f"Making {num_chats} chat requests took too long")
177
178		# Test the throughput
179		throughput = num_chats / duration
180		print(f"Chat request throughput: {throughput:.2f} chats/second")
181
182		# Assert reasonable throughput
183		# This threshold is arbitrary and may need adjustment
184		self.assertGreater(throughput, 5.0, f"Chat request throughput is too low")
185
186
187		if __name__ == "__main__":
188		unittest.main()

ollamatools-project/examples/web_requests.py +0 -75

…	…	@@ -1,75 +0,0 @@
1		"""
2		Example demonstrating the use of web tools in OllamaTools.
3
4		This script shows how to make web requests using the fetch_url function.
5		"""
6
7		from ollamatools.tools.web_tools import fetch_url
8		from ollamatools.security import PermissionManager
9		import json
10
11		def main():
12		"""Run the example."""
13		print("OllamaTools Web Request Example")
14		print("===============================")
15
16		# Create a permission manager
17		permission_manager = PermissionManager(prompt_enabled=True)
18
19		print("\n1. Simple GET request")
20		try:
21		result = fetch_url(
22		"https://httpbin.org/get",
23		permission_manager=permission_manager
24		)
25		print(f"Status: {result['status']}")
26		print(f"Response content type: {result['headers'].get('Content-Type', 'Unknown')}")
27		print(f"Response snippet: {json.dumps(result['content'], indent=2)[:200]}...")
28		except Exception as e:
29		print(f"Error: {e}")
30
31		print("\n2. POST request with data")
32		try:
33		post_data = {"name": "OllamaTools", "version": "0.1.0"}
34		result = fetch_url(
35		"https://httpbin.org/post",
36		method="POST",
37		data=post_data,
38		permission_manager=permission_manager
39		)
40		print(f"Status: {result['status']}")
41		print(f"Response content type: {result['headers'].get('Content-Type', 'Unknown')}")
42		print(f"Response snippet: {json.dumps(result['content'], indent=2)[:200]}...")
43		except Exception as e:
44		print(f"Error: {e}")
45
46		print("\n3. GET request with parameters")
47		try:
48		params = {"query": "ollama", "limit": "5"}
49		result = fetch_url(
50		"https://httpbin.org/get",
51		params=params,
52		permission_manager=permission_manager
53		)
54		print(f"Status: {result['status']}")
55		print(f"Response content type: {result['headers'].get('Content-Type', 'Unknown')}")
56		print(f"Response snippet: {json.dumps(result['content'], indent=2)[:200]}...")
57		except Exception as e:
58		print(f"Error: {e}")
59
60		print("\n4. Request with custom headers")
61		try:
62		headers = {"X-API-Key": "demo-key", "User-Agent": "OllamaTools/0.1.0"}
63		result = fetch_url(
64		"https://httpbin.org/headers",
65		headers=headers,
66		permission_manager=permission_manager
67		)
68		print(f"Status: {result['status']}")
69		print(f"Response content type: {result['headers'].get('Content-Type', 'Unknown')}")
70		print(f"Response snippet: {json.dumps(result['content'], indent=2)[:200]}...")
71		except Exception as e:
72		print(f"Error: {e}")
73
74		if __name__ == "__main__":
75		main()

ollamatools-project/examples/web_tools_examples.py +0 -105

…	…	@@ -1,105 +0,0 @@
1		"""
2		Examples demonstrating the use of advanced web tools in OllamaTools.
3
4		This script shows how to use the get_weather and search_web functions.
5		"""
6
7		from ollamatools.tools.web_tools import fetch_url, get_weather, search_web
8		from ollamatools.security import PermissionManager
9		import json
10
11		def main():
12		"""Run the example."""
13		print("OllamaTools Advanced Web Tools Example")
14		print("=====================================")
15
16		# Create a permission manager
17		permission_manager = PermissionManager(prompt_enabled=True)
18
19		# Example 1: Get Weather
20		print("\n1. Weather Information")
21		print("---------------------")
22
23		try:
24		# Replace 'YOUR_API_KEY' with an actual OpenWeatherMap API key
25		api_key = "YOUR_API_KEY"
26
27		# Skip this example if no API key is provided
28		if api_key == "YOUR_API_KEY":
29		print("Skipping weather example - Please add your OpenWeatherMap API key to use this example.")
30		else:
31		print("Getting weather for London...")
32		weather_info = get_weather(
33		"London",
34		days=3,
35		units="metric",
36		api_key=api_key,
37		permission_manager=permission_manager
38		)
39
40		# Display current weather
41		print(f"Current weather in {weather_info['location']['name']}, {weather_info['location']['country']}:")
42		print(f" Temperature: {weather_info['current']['temperature']}°C")
43		print(f" Description: {weather_info['current']['description']}")
44		print(f" Humidity: {weather_info['current']['humidity']}%")
45
46		# Display forecast summary
47		print("\nForecast:")
48		for i, item in enumerate(weather_info['forecast'][:3]): # Show first 3 forecast items
49		print(f" Item {i+1}: {item['temperature']}°C, {item['description']}")
50
51		except Exception as e:
52		print(f"Error getting weather: {e}")
53
54		# Example 2: Web Search
55		print("\n2. Web Search")
56		print("------------")
57
58		try:
59		# Replace 'YOUR_API_KEY' with an actual Search API key if you have one
60		# This example will use the fallback search method if no API key is provided
61		api_key = "YOUR_API_KEY"
62
63		print("Searching for 'Python programming'...")
64		search_results = search_web(
65		"Python programming",
66		num_results=3,
67		api_key=api_key if api_key != "YOUR_API_KEY" else None,
68		permission_manager=permission_manager
69		)
70
71		# Display search results
72		print(f"Found {len(search_results)} results:")
73		for i, result in enumerate(search_results):
74		print(f"\nResult {i+1}:")
75		print(f" Title: {result['title']}")
76		print(f" URL: {result['url']}")
77		print(f" Snippet: {result['snippet'][:100]}...")
78
79		except Exception as e:
80		print(f"Error searching web: {e}")
81
82		# Example 3: Combined Usage
83		print("\n3. Combined Usage Example")
84		print("-----------------------")
85
86		try:
87		print("This example would fetch weather data for a city from a search result.")
88		print("(Skipped in this demo to avoid API key requirements)")
89
90		# In a real implementation, you might do something like:
91		# 1. Search for "best cities to visit in Europe"
92		# 2. Extract a city from the results
93		# 3. Get weather for that city
94
95		print("\nPseudo-code:")
96		print(" results = search_web('best cities to visit in Europe')")
97		print(" city = extract_city_from_results(results)")
98		print(" weather = get_weather(city)")
99		print(" print(f'Weather in {city}: {weather['current']['temperature']}°C')")
100
101		except Exception as e:
102		print(f"Error in combined example: {e}")
103
104		if __name__ == "__main__":
105		main()

ollamatools-project/file_mapping.md +0 -41

…	…	@@ -1,41 +0,0 @@
1		# OllamaTools File Mapping
2
3		This document provides a mapping between the original codebase and the new refactored structure.
4		Use this as a guide for the refactoring process to ensure all functionality is migrated correctly.
5
6		## File Mapping
7
8		| Original File | New File | Status | Notes |
9		|---------------|----------|--------|-------|
10		| `ollamatools/agent.py` | `ollamatools/core/agent.py` | Completed | Base agent implementation |
11		| `ollamatools/config.py` | `ollamatools/core/config.py` | Completed | Configuration management |
12		| `ollamatools/errors.py` | `ollamatools/core/errors.py` | Completed | Error definitions and handling |
13		| `ollamatools/security.py` | `ollamatools/core/security.py` | Completed | Security utilities and permission system |
14		| `ollamatools/logger.py` | `ollamatools/core/utils/logger.py` | Completed | Logging utilities |
15		| `ollamatools/command_validator.py` | `ollamatools/core/utils/validators/command.py` | Completed | Command validation |
16		| `ollamatools/path_validator.py` | `ollamatools/core/utils/validators/path.py` | Completed | Path validation |
17		| `ollamatools/url_validator.py` | `ollamatools/core/utils/validators/url.py` | Completed | URL validation |
18		| `ollamatools/plugins/__init__.py` | `ollamatools/plugins/__init__.py` | Pending | Plugin system initialization |
19		| `ollamatools/plugins/registry.py` | `ollamatools/plugins/registry.py` | Pending | Plugin discovery and management |
20		| `ollamatools/plugins/errors.py` | `ollamatools/plugins/errors.py` | Pending | Plugin-specific error handling |
21		| `ollamatools/agent/modular/search.py` | `ollamatools/plugins/standard/search.py` | Pending | Search functionality |
22		| `ollamatools/agent/modular/commands.py` | `ollamatools/plugins/standard/commands.py` | Pending | Command management |
23		| `ollamatools/agent/modular/feedback.py` | `ollamatools/plugins/standard/feedback.py` | Pending | Feedback collection |
24		| `ollamatools/agent/modular/issues.py` | `ollamatools/plugins/standard/issues.py` | Pending | Issue tracking |
25		| `ollamatools/plugins/conversation_history.py` | `ollamatools/plugins/standard/conversation/history.py` | Pending | Conversation history tracking |
26		| `ollamatools/plugins/conversation_context.py` | `ollamatools/plugins/standard/conversation/context.py` | Pending | Conversation context management |
27		| `ollamatools/tools/system_tools.py` | `ollamatools/plugins/tools/system.py` | Pending | System tools (file operations, commands) |
28		| `ollamatools/tools/web_tools.py` | `ollamatools/plugins/tools/web.py` | Pending | Web tools (HTTP requests, web searches) |
29		| `ollamatools/tools/doc_tools.py` | `ollamatools/plugins/tools/doc.py` | Pending | Documentation tools |
30		| `ollamatools/opensearch_web_tools.py` | `ollamatools/plugins/integrations/opensearch/tools.py` | Pending | OpenSearch web tools |
31		| `ollamatools/tools/matrix_tools.py` | `ollamatools/plugins/integrations/matrix/client.py` | Pending | Matrix client core |
32		| `ollamatools/tools/enhanced_matrix_core.py` | `ollamatools/plugins/integrations/matrix/client.py` | Pending | Enhanced Matrix client core |
33		| `ollamatools/tools/enhanced_matrix_history.py` | `ollamatools/plugins/integrations/matrix/history.py` | Pending | Matrix history tracking |
34		| `ollamatools/tools/enhanced_matrix_search.py` | `ollamatools/plugins/integrations/matrix/search.py` | Pending | Matrix search integration |
35		| `ollamatools/tools/matrix_rate_limit_patch.py` | `ollamatools/plugins/integrations/matrix/rate_limit.py` | Pending | Rate limiting for Matrix |
36		| `examples/basic_agent.py` | `ollamatools/agents/basic.py` | Pending | Basic agent implementation |
37		| `ollamatools/agent/modular/agent.py` | `ollamatools/agents/modular.py` | Pending | Modular agent implementation |
38		| `ollamatools/agent/extended/agent.py` | `ollamatools/agents/extended.py` | Pending | Extended agent implementation |
39		| `ollamatools/config/schemas.py` | `ollamatools/config/schemas.py` | Pending | Configuration schemas |
40		| `ollamatools/config/core_config.py` | `ollamatools/config/loader.py` | Pending | Configuration loading/validation |
41		| `ollamatools/config/plugin_config.py` | `ollamatools/config/defaults.py` | Pending | Default configurations |

ollamatools-project/finalize_refactoring.sh +0 -74

…	…	@@ -1,74 +0,0 @@
1		#!/bin/bash
2
3		# Script to finalize the refactoring process by:
4		# 1. Creating a backup of the original codebase
5		# 2. Renaming the original directory to ollamatools-legacy
6		# 3. Renaming the new directory to ollamatools
7
8		# Define the parent directory and project directories
9		PARENT_DIR=".."
10		ORIGINAL_DIR="$PARENT_DIR/ollamatools-project"
11		NEW_DIR="$PARENT_DIR/ollamatools-new"
12		LEGACY_DIR="$PARENT_DIR/ollamatools-legacy"
13		BACKUP_DIR="$PARENT_DIR/ollamatools-backup-$(date +%Y%m%d%H%M%S)"
14
15		# Check if the directories exist
16		if [ ! -d "$ORIGINAL_DIR" ]; then
17		echo "Error: Original directory $ORIGINAL_DIR does not exist."
18		exit 1
19		fi
20
21		if [ ! -d "$NEW_DIR" ]; then
22		echo "Error: New directory $NEW_DIR does not exist."
23		exit 1
24		fi
25
26		# Confirm with the user
27		echo "This script will finalize the refactoring process by:"
28		echo "1. Creating a backup of the original codebase at $BACKUP_DIR"
29		echo "2. Renaming $ORIGINAL_DIR to $LEGACY_DIR"
30		echo "3. Renaming $NEW_DIR to $ORIGINAL_DIR"
31		echo ""
32		read -p "Are you sure you want to proceed? (y/n): " -n 1 -r
33		echo ""
34
35		if [[ ! $REPLY =~ ^[Yy]$ ]]; then
36		echo "Operation cancelled."
37		exit 0
38		fi
39
40		# Create a backup of the original codebase
41		echo "Creating backup of the original codebase..."
42		cp -r "$ORIGINAL_DIR" "$BACKUP_DIR"
43		if [ $? -ne 0 ]; then
44		echo "Error: Failed to create backup. Aborting."
45		exit 1
46		fi
47		echo "Backup created at $BACKUP_DIR"
48
49		# Rename the original directory to ollamatools-legacy
50		echo "Renaming original directory to ollamatools-legacy..."
51		mv "$ORIGINAL_DIR" "$LEGACY_DIR"
52		if [ $? -ne 0 ]; then
53		echo "Error: Failed to rename original directory. Aborting."
54		exit 1
55		fi
56		echo "Original directory renamed to ollamatools-legacy"
57
58		# Rename the new directory to ollamatools
59		echo "Renaming new directory to ollamatools-project..."
60		mv "$NEW_DIR" "$ORIGINAL_DIR"
61		if [ $? -ne 0 ]; then
62		echo "Error: Failed to rename new directory. Restoring original directory."
63		mv "$LEGACY_DIR" "$ORIGINAL_DIR"
64		exit 1
65		fi
66		echo "New directory renamed to ollamatools-project"
67
68		echo ""
69		echo "Refactoring finalization complete!"
70		echo "- Original codebase backup: $BACKUP_DIR"
71		echo "- Legacy codebase: $LEGACY_DIR"
72		echo "- New codebase: $ORIGINAL_DIR"
73		echo ""
74		echo "Note: You may need to update any references to the project paths in your environment."

ollamatools-project/generate_file_mapping.sh +0 -98

…	…	@@ -1,98 +0,0 @@
1		#!/bin/bash
2
3		# Script to generate a mapping between the old and new file structures
4		# This helps guide the refactoring process by showing which files from the original codebase
5		# should be moved to which locations in the new structure
6
7		# Define the output file
8		OUTPUT_FILE="file_mapping.md"
9
10		# Create the output file with a header
11		cat > "$OUTPUT_FILE" << 'EOF'
12		# OllamaTools File Mapping
13
14		This document provides a mapping between the original codebase and the new refactored structure.
15		Use this as a guide for the refactoring process to ensure all functionality is migrated correctly.
16
17		## File Mapping
18
19		| Original File | New File | Status | Notes |
20		|---------------|----------|--------|-------|
21		EOF
22
23		# Function to add a mapping entry
24		add_mapping() {
25		local original_file="$1"
26		local new_file="$2"
27		local status="$3"
28		local notes="$4"
29
30		echo "| \`$original_file\` | \`$new_file\` | $status | $notes |" >> "$OUTPUT_FILE"
31		}
32
33		# Core module mappings
34		echo "Adding Core module mappings..."
35		add_mapping "ollamatools/agent.py" "ollamatools/core/agent.py" "Pending" "Base agent implementation"
36		add_mapping "ollamatools/config.py" "ollamatools/core/config.py" "Pending" "Configuration management"
37		add_mapping "ollamatools/errors.py" "ollamatools/core/errors.py" "Pending" "Error definitions and handling"
38		add_mapping "ollamatools/security.py" "ollamatools/core/security.py" "Pending" "Security utilities and permission system"
39		add_mapping "ollamatools/logger.py" "ollamatools/core/utils/logger.py" "Pending" "Logging utilities"
40		add_mapping "ollamatools/command_validator.py" "ollamatools/core/utils/validators.py" "Pending" "Command validation (to be merged with other validators)"
41		add_mapping "ollamatools/path_validator.py" "ollamatools/core/utils/validators.py" "Pending" "Path validation (to be merged with other validators)"
42		add_mapping "ollamatools/url_validator.py" "ollamatools/core/utils/validators.py" "Pending" "URL validation (to be merged with other validators)"
43
44		# Plugin system mappings
45		echo "Adding Plugin system mappings..."
46		add_mapping "ollamatools/plugins/__init__.py" "ollamatools/plugins/__init__.py" "Pending" "Plugin system initialization"
47		add_mapping "ollamatools/plugins/registry.py" "ollamatools/plugins/registry.py" "Pending" "Plugin discovery and management"
48		add_mapping "ollamatools/plugins/errors.py" "ollamatools/plugins/errors.py" "Pending" "Plugin-specific error handling"
49
50		# Standard plugins mappings
51		echo "Adding Standard plugins mappings..."
52		add_mapping "ollamatools/agent/modular/search.py" "ollamatools/plugins/standard/search.py" "Pending" "Search functionality"
53		add_mapping "ollamatools/agent/modular/commands.py" "ollamatools/plugins/standard/commands.py" "Pending" "Command management"
54		add_mapping "ollamatools/agent/modular/feedback.py" "ollamatools/plugins/standard/feedback.py" "Pending" "Feedback collection"
55		add_mapping "ollamatools/agent/modular/issues.py" "ollamatools/plugins/standard/issues.py" "Pending" "Issue tracking"
56		add_mapping "ollamatools/plugins/conversation_history.py" "ollamatools/plugins/standard/conversation/history.py" "Pending" "Conversation history tracking"
57		add_mapping "ollamatools/plugins/conversation_context.py" "ollamatools/plugins/standard/conversation/context.py" "Pending" "Conversation context management"
58
59		# Tool plugins mappings
60		echo "Adding Tool plugins mappings..."
61		add_mapping "ollamatools/tools/system_tools.py" "ollamatools/plugins/tools/system.py" "Pending" "System tools (file operations, commands)"
62		add_mapping "ollamatools/tools/web_tools.py" "ollamatools/plugins/tools/web.py" "Pending" "Web tools (HTTP requests, web searches)"
63		add_mapping "ollamatools/tools/doc_tools.py" "ollamatools/plugins/tools/doc.py" "Pending" "Documentation tools"
64		add_mapping "ollamatools/opensearch_web_tools.py" "ollamatools/plugins/integrations/opensearch/tools.py" "Pending" "OpenSearch web tools"
65
66		# Integration plugins mappings
67		echo "Adding Integration plugins mappings..."
68		add_mapping "ollamatools/tools/matrix_tools.py" "ollamatools/plugins/integrations/matrix/client.py" "Pending" "Matrix client core"
69		add_mapping "ollamatools/tools/enhanced_matrix_core.py" "ollamatools/plugins/integrations/matrix/client.py" "Pending" "Enhanced Matrix client core"
70		add_mapping "ollamatools/tools/enhanced_matrix_history.py" "ollamatools/plugins/integrations/matrix/history.py" "Pending" "Matrix history tracking"
71		add_mapping "ollamatools/tools/enhanced_matrix_search.py" "ollamatools/plugins/integrations/matrix/search.py" "Pending" "Matrix search integration"
72		add_mapping "ollamatools/tools/matrix_rate_limit_patch.py" "ollamatools/plugins/integrations/matrix/rate_limit.py" "Pending" "Rate limiting for Matrix"
73
74		# Agent implementations mappings
75		echo "Adding Agent implementations mappings..."
76		add_mapping "examples/basic_agent.py" "ollamatools/agents/basic.py" "Pending" "Basic agent implementation"
77		add_mapping "ollamatools/agent/modular/agent.py" "ollamatools/agents/modular.py" "Pending" "Modular agent implementation"
78		add_mapping "ollamatools/agent/extended/agent.py" "ollamatools/agents/extended.py" "Pending" "Extended agent implementation"
79
80		# Configuration system mappings
81		echo "Adding Configuration system mappings..."
82		add_mapping "ollamatools/config/schemas.py" "ollamatools/config/schemas.py" "Pending" "Configuration schemas"
83		add_mapping "ollamatools/config/core_config.py" "ollamatools/config/loader.py" "Pending" "Configuration loading/validation"
84		add_mapping "ollamatools/config/plugin_config.py" "ollamatools/config/defaults.py" "Pending" "Default configurations"
85
86		echo ""
87		echo "File mapping generated: $OUTPUT_FILE"
88		echo ""
89		echo "This mapping provides a guide for the refactoring process."
90		echo "For each file in the original codebase, it shows where the functionality should be moved in the new structure."
91		echo ""
92		echo "To use this mapping with the copy_refactor_file.sh script:"
93		echo " ./copy_refactor_file.sh <original_file> <new_file>"
94		echo ""
95		echo "For example:"
96		echo " ./copy_refactor_file.sh ollamatools/agent.py ollamatools/core/agent.py"
97		echo ""
98		echo "After refactoring each file, update the Status column in the mapping file to track progress."

ollamatools-project/make_executable.sh +0 -5

…	…	@@ -1,5 +0,0 @@
1		#!/bin/bash
2		# Make scripts executable
3		chmod +x mkdir_command.sh
4		chmod +x scripts/install_and_verify.sh
5		chmod +x examples/final_demo.py

ollamatools-project/matrix_agent_plugin.py +0 -753

…	…	@@ -1,753 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Matrix Agent Plugin - Proof of Concept
4
5		This file demonstrates how the matrix client could be integrated with the modular agent
6		using the proposed unified plugin architecture. It implements a plugin that connects
7		a Matrix client to a ModularOllamaAgent, allowing the agent to process messages from
8		Matrix rooms.
9
10		This is a proof-of-concept implementation and is not intended for production use.
11		"""
12
13		import os
14		import sys
15		import json
16		import time
17		import logging
18		import asyncio
19		from typing import Dict, Any, List, Optional, Union, Set, Tuple
20
21		# Base plugin interface
22		class OllamaToolsPlugin:
23		"""Base interface for all OllamaTools plugins."""
24
25		def __init__(self, config=None):
26		"""Initialize the plugin with optional configuration."""
27		self.config = config or {}
28
29		def initialize(self, context):
30		"""Initialize the plugin with the runtime context."""
31		pass
32
33		def shutdown(self):
34		"""Clean up resources when shutting down."""
35		pass
36
37		@property
38		def name(self):
39		"""Get the plugin name."""
40		return self.__class__.__name__
41
42		@property
43		def version(self):
44		"""Get the plugin version."""
45		return "1.0.0"
46
47		@property
48		def dependencies(self):
49		"""Get plugin dependencies."""
50		return []
51
52		# Matrix plugin interface
53		class MatrixPlugin(OllamaToolsPlugin):
54		"""Base class for Matrix plugins."""
55
56		async def on_message(self, room, event, client):
57		"""Handle an incoming Matrix message."""
58		return False
59
60		def get_commands(self):
61		"""Get Matrix commands provided by this plugin."""
62		return {}
63
64		async def on_room_join(self, room, client):
65		"""Handle room join event."""
66		pass
67
68		async def on_client_start(self, client):
69		"""Handle client start event."""
70		pass
71
72		async def on_client_stop(self, client):
73		"""Handle client stop event."""
74		pass
75
76		# Agent plugin interface
77		class AgentPlugin(OllamaToolsPlugin):
78		"""Base class for agent plugins."""
79
80		def process_message(self, message, context):
81		"""Process a message and return a response."""
82		return None
83
84		def get_commands(self):
85		"""Get commands provided by this plugin."""
86		return {}
87
88		def on_function_call(self, function_name, args, context):
89		"""Handle a function call."""
90		return None
91
92		# Shared plugin interface
93		class SharedPlugin(OllamaToolsPlugin):
94		"""Base class for plugins that can be used by multiple components."""
95
96		def get_capabilities(self):
97		"""Get capabilities provided by this plugin."""
98		return []
99
100		# Plugin registry
101		class PluginRegistry:
102		"""Registry for OllamaTools plugins."""
103
104		def __init__(self):
105		"""Initialize the plugin registry."""
106		self.plugins = {}
107		self.initialized_plugins = {}
108
109		def register_plugin(self, plugin_class, config=None):
110		"""Register a plugin class."""
111		plugin_name = plugin_class.__name__
112		self.plugins[plugin_name] = {
113		"class": plugin_class,
114		"config": config
115		}
116		return self
117
118		def initialize_plugin(self, plugin_name, context):
119		"""Initialize a plugin and its dependencies."""
120		if plugin_name in self.initialized_plugins:
121		return self.initialized_plugins[plugin_name]
122
123		if plugin_name not in self.plugins:
124		raise ValueError(f"Plugin {plugin_name} not found")
125
126		plugin_info = self.plugins[plugin_name]
127		plugin_class = plugin_info["class"]
128		plugin_config = plugin_info["config"]
129
130		# Create plugin instance
131		plugin_instance = plugin_class(plugin_config)
132
133		# Initialize dependencies first
134		for dependency in plugin_instance.dependencies:
135		self.initialize_plugin(dependency, context)
136
137		# Initialize the plugin
138		plugin_instance.initialize(context)
139		self.initialized_plugins[plugin_name] = plugin_instance
140
141		return plugin_instance
142
143		def initialize_all(self, context):
144		"""Initialize all registered plugins."""
145		for plugin_name in list(self.plugins.keys()):
146		self.initialize_plugin(plugin_name, context)
147
148		def get_plugin(self, plugin_name):
149		"""Get an initialized plugin."""
150		return self.initialized_plugins.get(plugin_name)
151
152		def shutdown(self):
153		"""Shutdown all initialized plugins."""
154		for plugin_name, plugin in list(self.initialized_plugins.items()):
155		try:
156		plugin.shutdown()
157		except Exception as e:
158		print(f"Error shutting down plugin {plugin_name}: {e}")
159
160		self.initialized_plugins = {}
161
162		# Conversation history plugin
163		class ConversationHistoryPlugin(SharedPlugin):
164		"""Plugin for conversation history tracking."""
165
166		def __init__(self, config=None):
167		super().__init__(config)
168		self.history_file = self.config.get("history_file")
169		self.max_history = self.config.get("max_history", 100)
170		self.conversation_history = []
171
172		def initialize(self, context):
173		"""Initialize the conversation history plugin."""
174		if not self.history_file:
175		# Set default history file
176		self.history_file = os.path.join(
177		os.path.expanduser('~'),
178		'.ollamatools',
179		'conversation_history.json'
180		)
181
182		self._load_conversation_history()
183
184		def _load_conversation_history(self):
185		"""Load conversation history from file."""
186		if not os.path.exists(self.history_file):
187		return
188
189		try:
190		with open(self.history_file, 'r') as f:
191		self.conversation_history = json.load(f)
192		print(f"Loaded {len(self.conversation_history)} conversations from history")
193		except Exception as e:
194		print(f"Error loading conversation history: {e}")
195		self.conversation_history = []
196
197		def _save_conversation_history(self):
198		"""Save conversation history to file."""
199		try:
200		# Create directory if it doesn't exist
201		os.makedirs(os.path.dirname(self.history_file), exist_ok=True)
202
203		with open(self.history_file, 'w') as f:
204		json.dump(self.conversation_history, f, indent=2)
205		print(f"Saved {len(self.conversation_history)} conversations to history")
206		except Exception as e:
207		print(f"Error saving conversation history: {e}")
208
209		def record_conversation(self, user_message, assistant_response, metadata=None):
210		"""Record a conversation."""
211		conversation = {
212		"timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
213		"user_message": user_message,
214		"assistant_response": assistant_response,
215		"metadata": metadata or {}
216		}
217
218		self.conversation_history.append(conversation)
219
220		# Trim conversation history if it gets too long
221		if len(self.conversation_history) > self.max_history:
222		self.conversation_history = self.conversation_history[-self.max_history:]
223
224		self._save_conversation_history()
225
226		return conversation
227
228		def get_recent_conversations(self, n=5):
229		"""Get recent conversations."""
230		return self.conversation_history[-n:] if len(self.conversation_history) >= n else self.conversation_history
231
232		def analyze_topics(self):
233		"""Analyze conversation topics."""
234		if not self.conversation_history:
235		return {}
236
237		# Simple keyword-based topic analysis
238		topics = {
239		"help": 0,
240		"search": 0,
241		"file": 0,
242		"command": 0,
243		"code": 0,
244		"error": 0,
245		"issue": 0,
246		"question": 0,
247		"other": 0
248		}
249
250		for conv in self.conversation_history:
251		message = conv["user_message"].lower()
252
253		# Check for keywords
254		if "help" in message or "how" in message:
255		topics["help"] += 1
256		if "search" in message or "find" in message or "look" in message:
257		topics["search"] += 1
258		if "file" in message or "read" in message or "write" in message:
259		topics["file"] += 1
260		if "command" in message or "run" in message or "execute" in message:
261		topics["command"] += 1
262		if "code" in message or "program" in message or "script" in message:
263		topics["code"] += 1
264		if "error" in message or "bug" in message or "fix" in message:
265		topics["error"] += 1
266		if "issue" in message or "problem" in message:
267		topics["issue"] += 1
268		if "?" in message:
269		topics["question"] += 1
270
271		# If no keywords matched, count as other
272		if not any(count > 0 for count in topics.values()):
273		topics["other"] += 1
274
275		# Remove topics with zero count
276		return {k: v for k, v in topics.items() if v > 0}
277
278		def get_metrics(self):
279		"""Get performance metrics."""
280		if not self.conversation_history:
281		return {"error": "No conversation history available"}
282
283		# Calculate metrics
284		total_conversations = len(self.conversation_history)
285
286		# Calculate response times if available
287		response_times = []
288		for conv in self.conversation_history:
289		if "metadata" in conv and "response_time" in conv["metadata"]:
290		response_times.append(conv["metadata"]["response_time"])
291
292		avg_response_time = sum(response_times) / len(response_times) if response_times else 0
293
294		# Calculate token usage if available
295		total_tokens = 0
296		for conv in self.conversation_history:
297		if "metadata" in conv and "tokens" in conv["metadata"]:
298		total_tokens += conv["metadata"]["tokens"]
299
300		avg_tokens_per_conversation = total_tokens / total_conversations if total_tokens > 0 else 0
301
302		return {
303		"total_conversations": total_conversations,
304		"avg_response_time": avg_response_time,
305		"total_tokens": total_tokens,
306		"avg_tokens_per_conversation": avg_tokens_per_conversation
307		}
308
309		def get_capabilities(self):
310		"""Get capabilities provided by this plugin."""
311		return ["conversation_history", "topic_analysis", "performance_metrics"]
312
313		def shutdown(self):
314		"""Save conversation history when shutting down."""
315		self._save_conversation_history()
316
317		# Matrix agent adapter plugin
318		class MatrixAgentAdapterPlugin(MatrixPlugin):
319		"""Plugin that adapts an agent for use with Matrix."""
320
321		def __init__(self, config=None):
322		super().__init__(config)
323		self.agent_class = self.config.get("agent_class")
324		self.agent_config = self.config.get("agent_config", {})
325		self.agent = None
326		self.registry = None
327
328		@property
329		def dependencies(self):
330		"""Get plugin dependencies."""
331		return ["ConversationHistoryPlugin"]
332
333		def initialize(self, context):
334		"""Initialize the plugin with an agent instance."""
335		if not self.agent_class:
336		raise ValueError("Agent class not specified")
337
338		# Import the agent class if it's a string
339		if isinstance(self.agent_class, str):
340		module_path, class_name = self.agent_class.rsplit('.', 1)
341		module = __import__(module_path, fromlist=[class_name])
342		self.agent_class = getattr(module, class_name)
343
344		# Create the agent instance
345		self.agent = self.agent_class(**self.agent_config)
346
347		# Get the plugin registry
348		self.registry = context.get("plugin_registry")
349
350		print(f"Initialized MatrixAgentAdapterPlugin with {self.agent_class.__name__}")
351
352		async def on_message(self, room, event, client):
353		"""Handle an incoming Matrix message using the agent."""
354		# Skip messages from the bot itself
355		if event.sender == client.user_id:
356		return False
357
358		message = event.body
359		sender = event.sender
360		room_id = room.room_id
361
362		print(f"Processing message from {sender} in room {room_id}: {message}")
363
364		# Record start time for performance tracking
365		start_time = time.time()
366
367		# Process the message with the agent
368		try:
369		response = self.agent.chat(message)
370
371		# Calculate response time
372		response_time = time.time() - start_time
373
374		# Record the conversation if history plugin is available
375		history_plugin = self.registry.get_plugin("ConversationHistoryPlugin")
376		if history_plugin:
377		metadata = {
378		"sender": sender,
379		"room_id": room_id,
380		"timestamp": event.server_timestamp,
381		"response_time": response_time,
382		"tokens": len(message.split()) + len(response.split()) # Rough estimate
383		}
384		history_plugin.record_conversation(message, response, metadata)
385
386		# Send the response
387		await client.send_message(room_id, response)
388
389		return True
390		except Exception as e:
391		print(f"Error processing message with agent: {e}")
392		await client.send_message(room_id, f"Error processing message: {str(e)}")
393		return True
394
395		async def on_client_start(self, client):
396		"""Handle client start event."""
397		print(f"Matrix client started with user ID: {client.user_id}")
398
399		def get_commands(self):
400		"""Get commands from the agent."""
401		commands = {}
402
403		# Add history commands
404		commands["/history"] = self._handle_history_command
405		commands["/topics"] = self._handle_topics_command
406		commands["/metrics"] = self._handle_metrics_command
407
408		# Add agent commands if available
409		if hasattr(self.agent, "get_commands"):
410		agent_commands = self.agent.get_commands()
411		commands.update(agent_commands)
412
413		return commands
414
415		async def _handle_history_command(self, client, room_id, args):
416		"""Handle the /history command."""
417		history_plugin = self.registry.get_plugin("ConversationHistoryPlugin")
418		if not history_plugin:
419		await client.send_message(room_id, "Conversation history tracking is not available.")
420		return
421
422		# Parse count argument
423		count = 5
424		if args and args[0].isdigit():
425		count = int(args[0])
426
427		# Get recent conversations
428		conversations = history_plugin.get_recent_conversations(count)
429
430		if not conversations:
431		await client.send_message(room_id, "No conversation history available.")
432		return
433
434		# Format the history
435		formatted_history = ["Recent Conversations:"]
436
437		for i, conv in enumerate(conversations, 1):
438		timestamp = conv.get("timestamp", "Unknown time")
439		user_message = conv.get("user_message", "No message")
440		metadata = conv.get("metadata", {})
441		sender = metadata.get("sender", "Unknown sender")
442
443		# Truncate message if it's too long
444		if len(user_message) > 100:
445		user_message = user_message[:100] + "..."
446
447		formatted_history.append(f"\n{i}. {timestamp}")
448		formatted_history.append(f" From: {sender}")
449		formatted_history.append(f" Message: {user_message}")
450
451		# Send the formatted history
452		await client.send_message(room_id, "\n".join(formatted_history))
453
454		async def _handle_topics_command(self, client, room_id, args):
455		"""Handle the /topics command."""
456		history_plugin = self.registry.get_plugin("ConversationHistoryPlugin")
457		if not history_plugin:
458		await client.send_message(room_id, "Conversation history tracking is not available.")
459		return
460
461		# Analyze topics
462		topics = history_plugin.analyze_topics()
463
464		if not topics:
465		await client.send_message(room_id, "No topics found in conversation history.")
466		return
467
468		# Format the topics
469		formatted_topics = ["Conversation Topics:"]
470
471		for topic, count in sorted(topics.items(), key=lambda x: x[1], reverse=True):
472		formatted_topics.append(f"- {topic}: {count} mentions")
473
474		# Send the formatted topics
475		await client.send_message(room_id, "\n".join(formatted_topics))
476
477		async def _handle_metrics_command(self, client, room_id, args):
478		"""Handle the /metrics command."""
479		history_plugin = self.registry.get_plugin("ConversationHistoryPlugin")
480		if not history_plugin:
481		await client.send_message(room_id, "Conversation history tracking is not available.")
482		return
483
484		# Get metrics
485		metrics = history_plugin.get_metrics()
486
487		if "error" in metrics:
488		await client.send_message(room_id, metrics["error"])
489		return
490
491		# Format the metrics
492		formatted_metrics = ["Performance Metrics:"]
493		formatted_metrics.append(f"- Total conversations: {metrics['total_conversations']}")
494		formatted_metrics.append(f"- Average response time: {metrics['avg_response_time']:.2f}s")
495		formatted_metrics.append(f"- Estimated total tokens: {metrics['total_tokens']}")
496		formatted_metrics.append(f"- Average tokens per conversation: {metrics['avg_tokens_per_conversation']:.1f}")
497
498		# Send the formatted metrics
499		await client.send_message(room_id, "\n".join(formatted_metrics))
500
501		def shutdown(self):
502		"""Clean up the agent."""
503		if hasattr(self.agent, "shutdown"):
504		self.agent.shutdown()
505
506		# Enhanced Matrix client with plugin support
507		class PluginEnabledMatrixClient:
508		"""Enhanced Matrix client with plugin support."""
509
510		def __init__(self, homeserver, user_id, password, **kwargs):
511		"""Initialize the Matrix client with plugins."""
512		self.homeserver = homeserver
513		self.user_id = user_id
514		self.password = password
515		self.kwargs = kwargs
516
517		# Initialize plugin registry
518		self.plugin_registry = PluginRegistry()
519
520		# Initialize client
521		self.client = None
522		self.connected = False
523
524		def register_plugin(self, plugin_class, config=None):
525		"""Register a custom plugin."""
526		self.plugin_registry.register_plugin(plugin_class, config)
527		return self
528
529		async def connect(self):
530		"""Connect to the Matrix homeserver and initialize plugins."""
531		try:
532		# In a real implementation, this would connect to Matrix
533		# For this proof of concept, we'll just simulate it
534		print(f"Connecting to Matrix homeserver: {self.homeserver}")
535		print(f"User ID: {self.user_id}")
536
537		# Simulate connection delay
538		await asyncio.sleep(1)
539
540		# Set connected state
541		self.connected = True
542
543		# Initialize plugins
544		context = {
545		"client": self,
546		"plugin_registry": self.plugin_registry
547		}
548
549		self.plugin_registry.initialize_all(context)
550
551		# Notify plugins that client has started
552		for plugin_name, plugin in self.plugin_registry.initialized_plugins.items():
553		if isinstance(plugin, MatrixPlugin):
554		await plugin.on_client_start(self)
555
556		print(f"Connected to Matrix homeserver: {self.homeserver}")
557		return True
558		except Exception as e:
559		print(f"Error connecting to Matrix homeserver: {e}")
560		return False
561
562		async def join_room(self, room_id):
563		"""Join a Matrix room."""
564		if not self.connected:
565		print("Not connected to Matrix homeserver")
566		return False
567
568		try:
569		# In a real implementation, this would join a room
570		# For this proof of concept, we'll just simulate it
571		print(f"Joining room: {room_id}")
572
573		# Simulate join delay
574		await asyncio.sleep(0.5)
575
576		# Notify plugins that room has been joined
577		for plugin_name, plugin in self.plugin_registry.initialized_plugins.items():
578		if isinstance(plugin, MatrixPlugin):
579		await plugin.on_room_join({"room_id": room_id}, self)
580
581		print(f"Joined room: {room_id}")
582		return True
583		except Exception as e:
584		print(f"Error joining room: {e}")
585		return False
586
587		async def send_message(self, room_id, message):
588		"""Send a message to a Matrix room."""
589		if not self.connected:
590		print("Not connected to Matrix homeserver")
591		return False
592
593		try:
594		# In a real implementation, this would send a message
595		# For this proof of concept, we'll just simulate it
596		print(f"Sending message to room {room_id}: {message}")
597
598		# Simulate send delay
599		await asyncio.sleep(0.2)
600
601		print(f"Message sent to room {room_id}")
602		return True
603		except Exception as e:
604		print(f"Error sending message: {e}")
605		return False
606
607		async def process_message(self, room_id, sender, message):
608		"""Process an incoming message."""
609		if not self.connected:
610		print("Not connected to Matrix homeserver")
611		return
612
613		# Create a simulated event
614		event = type('Event', (), {
615		'body': message,
616		'sender': sender,
617		'server_timestamp': int(time.time() * 1000)
618		})
619
620		# Create a simulated room
621		room = type('Room', (), {
622		'room_id': room_id
623		})
624
625		# Process the message with plugins
626		for plugin_name, plugin in self.plugin_registry.initialized_plugins.items():
627		if isinstance(plugin, MatrixPlugin):
628		if await plugin.on_message(room, event, self):
629		return
630
631		# If no plugin handled the message, print a default response
632		print(f"No plugin handled message from {sender} in room {room_id}: {message}")
633
634		async def process_command(self, room_id, sender, command):
635		"""Process a command."""
636		if not self.connected:
637		print("Not connected to Matrix homeserver")
638		return
639
640		# Parse the command
641		parts = command.split()
642		cmd = parts[0].lower()
643		args = parts[1:] if len(parts) > 1 else []
644
645		# Get commands from plugins
646		all_commands = {}
647		for plugin_name, plugin in self.plugin_registry.initialized_plugins.items():
648		if isinstance(plugin, MatrixPlugin):
649		commands = plugin.get_commands()
650		all_commands.update(commands)
651
652		# Check if the command exists
653		if cmd in all_commands:
654		# Execute the command
655		handler = all_commands[cmd]
656		await handler(self, room_id, args)
657		else:
658		# Unknown command
659		await self.send_message(room_id, f"Unknown command: {cmd}")
660
661		async def disconnect(self):
662		"""Disconnect from Matrix and shutdown plugins."""
663		if not self.connected:
664		print("Not connected to Matrix homeserver")
665		return
666
667		try:
668		# In a real implementation, this would disconnect from Matrix
669		# For this proof of concept, we'll just simulate it
670		print(f"Disconnecting from Matrix homeserver: {self.homeserver}")
671
672		# Notify plugins that client is stopping
673		for plugin_name, plugin in self.plugin_registry.initialized_plugins.items():
674		if isinstance(plugin, MatrixPlugin):
675		await plugin.on_client_stop(self)
676
677		# Shutdown plugins
678		self.plugin_registry.shutdown()
679
680		# Set connected state
681		self.connected = False
682
683		print(f"Disconnected from Matrix homeserver: {self.homeserver}")
684		except Exception as e:
685		print(f"Error disconnecting from Matrix homeserver: {e}")
686
687		# Example usage
688		async def main():
689		"""Run the example."""
690		try:
691		# Create a Matrix client with plugins
692		client = PluginEnabledMatrixClient(
693		homeserver="https://matrix.example.org",
694		user_id="@ollamabot:example.org",
695		password="password"
696		)
697
698		# Register plugins
699		client.register_plugin(
700		ConversationHistoryPlugin,
701		{
702		"history_file": "matrix_conversation_history.json",
703		"max_history": 100
704		}
705		)
706
707		client.register_plugin(
708		MatrixAgentAdapterPlugin,
709		{
710		"agent_class": "ollamatools.agent.ModularOllamaAgent",
711		"agent_config": {
712		"model": "llama3:latest",
713		"permission_prompts": True
714		}
715		}
716		)
717
718		# Connect to Matrix
719		success = await client.connect()
720		if not success:
721		print("Failed to connect to Matrix homeserver")
722		return
723
724		# Join a room
725		room_id = "#ollamatools:example.org"
726		success = await client.join_room(room_id)
727		if not success:
728		print(f"Failed to join room: {room_id}")
729		await client.disconnect()
730		return
731
732		# Send a welcome message
733		await client.send_message(room_id, "Hello! I'm the OllamaTools Matrix bot. Type /help for available commands.")
734
735		# Simulate some messages
736		await client.process_message(room_id, "@user1:example.org", "Hello bot!")
737		await client.process_message(room_id, "@user2:example.org", "What can you do?")
738		await client.process_message(room_id, "@user1:example.org", "Tell me about Python.")
739
740		# Simulate some commands
741		await client.process_command(room_id, "@user1:example.org", "/history")
742		await client.process_command(room_id, "@user2:example.org", "/topics")
743		await client.process_command(room_id, "@user1:example.org", "/metrics")
744
745		# Disconnect from Matrix
746		await client.disconnect()
747
748		except Exception as e:
749		print(f"Error in main function: {e}")
750
751		if __name__ == "__main__":
752		# Run the example
753		asyncio.run(main())

ollamatools-project/matrix_code_review.md +0 -467

…	…	@@ -1,467 +0,0 @@
1		# Code Review: OllamaTools Matrix Integration
2
3		## Overview
4
5		This document provides a comprehensive code review of the Matrix integration tools in the OllamaTools project. The review covers the architecture, code quality, error handling, security, performance, and documentation of the codebase.
6
7		## Files Reviewed
8
9		1. `examples/enhanced_matrix_client_example.py`
10		2. `ollamatools/tools/matrix_tools.py`
11		3. `ollamatools/tools/enhanced_matrix_tools.py`
12		4. `ollamatools/tools/enhanced_matrix_core.py`
13		5. `ollamatools/tools/enhanced_matrix_search.py`
14		6. `ollamatools/tools/enhanced_matrix_history.py`
15
16		## 1. Architecture and Design
17
18		### 1.1 Overall Architecture
19
20		The Matrix integration is well-structured with a clear separation of concerns:
21
22		- Base functionality in `matrix_tools.py`
23		- Enhanced functionality split across multiple modules:
24		- Core client functionality in `enhanced_matrix_core.py`
25		- Search functionality in `enhanced_matrix_search.py`
26		- History and analytics in `enhanced_matrix_history.py`
27		- Public API in `enhanced_matrix_tools.py`
28
29		This modular approach allows for good maintainability and extensibility.
30
31		### 1.2 Design Patterns
32
33		The codebase uses several good design patterns:
34
35		- **Inheritance**: `EnhancedMatrixClient` extends `MatrixClient` to add functionality
36		- **Dependency Injection**: Configuration is passed to clients rather than hardcoded
37		- **Facade Pattern**: `enhanced_matrix_tools.py` provides a simplified interface to the underlying modules
38		- **Optional Dependencies**: The code gracefully handles missing optional dependencies
39
40		## 2. Code Quality and Best Practices
41
42		### 2.1 `matrix_tools.py`
43
44		**Strengths:**
45
46		- Well-documented classes and methods with clear docstrings
47		- Consistent error handling with try/except blocks
48		- Good separation of concerns between client functionality and configuration
49
50		**Issues:**
51
52		1. **Lines 196-204**: The `process_with_ollama` method doesn't handle rate limiting or timeout scenarios.
53
54		```python
55		async def process_with_ollama(self, message: str, sender: str = None, room_id: str = None) -> str:
56		if not OLLAMA_AVAILABLE:
57		return "Ollama is not available. Please install the 'ollama' package with 'pip install ollama'."
58
59		try:
60		ollama_client = OllamaAsync(host=self.ollama_host)
61		```
62
63		2. **Lines 246-250**: The context management could be improved with a more sophisticated approach than a simple list.
64
65		```python
66		# Store user message in context
67		if sender and room_id:
68		context_key = f"{sender}:{room_id}"
69		self.user_contexts[context_key].append(user_message)
70		# Trim context if it gets too long
71		```
72
73		3. **Lines 307-315**: The username detection is simplistic and might fail with certain username patterns.
74
75		```python
76		# Extract username from user_id (e.g., "@user:example.com" -> "user")
77		username = self.user_id.split(':')[0].lstrip('@')
78
79		# Check if the message contains the bot's username (case insensitive)
80		if username.lower() in message.lower():
81		logger.info(f"Username mention detected in message: {message}")
82		# Process with Ollama and send response
83		response = await self.process_with_ollama(message, sender, room_id)
84		await self.send_message(room_id, response)
85		return
86		```
87
88		### 2.2 `enhanced_matrix_core.py`
89
90		**Strengths:**
91
92		- Good extension of the base class with clear additional functionality
93		- Improved username detection with multiple methods
94		- Well-structured command handling
95
96		**Issues:**
97
98		1. **Lines 123-126**: The initialization of `function_feedback` is conditional but there's no check before using it elsewhere.
99
100		```python
101		# Initialize function selection feedback
102		if MODULAR_AGENT_AVAILABLE:
103		from ollamatools.agent.modular.feedback import FunctionSelectionFeedback
104		self.function_feedback = FunctionSelectionFeedback()
105		```
106
107		2. **Lines 186-193**: The search context handling could be more robust with validation of the returned context.
108
109		```python
110		# Check if the message might benefit from search
111		search_context = ""
112		if self.enable_search and MODULAR_AGENT_AVAILABLE and OPENSEARCH_AVAILABLE:
113		from ollamatools.tools.enhanced_matrix_search import execute_search_for_message
114		search_context = await execute_search_for_message(message, self.opensearch_config)
115		```
116
117		3. **Lines 196-204**: Error handling in the message processing could be more granular to provide better error messages.
118
119		```python
120		# Process with Ollama
121		try:
122		if search_context:
123		# Add search context to the message
124		augmented_message = f"{message}\n\nSearch context: {search_context}"
125		response = await self.process_with_ollama(augmented_message, sender, room_id)
126		else:
127		response = await self.process_with_ollama(message, sender, room_id)
128		```
129
130		### 2.3 `enhanced_matrix_search.py`
131
132		**Strengths:**
133
134		- Good error handling with informative error messages
135		- Clean separation of search detection and execution
136		- Graceful degradation when dependencies are missing
137
138		**Issues:**
139
140		1. **Lines 67-77**: The factual intent detection doesn't have a fallback mechanism if the detection fails.
141
142		```python
143		try:
144		factual_intent = detect_factual_intent(message)
145		if factual_intent:
146		logger.info(f"Factual intent detected in message: {message}")
147		try:
148		results = await execute_search_query(message, opensearch_config)
149		search_context = extract_search_context(results)
150		logger.info(f"Search context: {search_context[:100]}...")
151		return search_context
152		except Exception as e:
153		logger.error(f"Error executing search: {e}")
154		return ""
155		```
156
157		2. **Lines 95-107**: The search results handling doesn't validate the structure of the returned results.
158
159		```python
160		try:
161		# Execute the search
162		results = opensearch_query(
163		query=query,
164		config=opensearch_config
165		)
166
167		return results
168		except Exception as e:
169		logger.error(f"Error executing search: {e}")
170		return {"error": str(e), "results": []}
171		```
172
173		### 2.4 `enhanced_matrix_history.py`
174
175		**Strengths:**
176
177		- Good file I/O handling with appropriate error handling
178		- Well-structured analytics functions
179		- Clear separation between data storage and analysis
180
181		**Issues:**
182
183		1. **Lines 48-67**: The conversation history is stored as plain JSON without any encryption or privacy considerations.
184
185		```python
186		def save_conversation_history(history: List[Dict[str, Any]], history_file: str) -> None:
187		"""
188		Save conversation history to file.
189
190		Args:
191		history: The conversation history
192		history_file: Path to save the history to
193		"""
194		try:
195		# Create directory if it doesn't exist
196		os.makedirs(os.path.dirname(history_file), exist_ok=True)
197
198		with open(history_file, 'w') as f:
199		json.dump(history, f, indent=2)
200		logger.debug(f"Saved {len(history)} conversations to history")
201		except Exception as e:
202		logger.error(f"Failed to save conversation history: {str(e)}")
203		```
204
205		2. **Lines 95-133**: The topic analysis is very simplistic and could benefit from more sophisticated NLP techniques.
206
207		```python
208		def analyze_conversation_topics(conversations: List[Dict[str, Any]]) -> Dict[str, int]:
209		"""
210		Analyze common topics in conversation history.
211
212		Args:
213		conversations: List of conversation records
214
215		Returns:
216		Dict[str, int]: Dictionary of topics and their frequency
217		"""
218		if not conversations:
219		return {}
220
221		# Simple keyword-based topic analysis
222		topics = {
223		"help": 0,
224		"search": 0,
225		"file": 0,
226		"command": 0,
227		"code": 0,
228		"error": 0,
229		"issue": 0,
230		"question": 0,
231		"other": 0
232		}
233		```
234
235		3. **Lines 135-183**: The performance metrics calculation doesn't handle edge cases like empty conversations.
236
237		```python
238		def get_performance_metrics(conversations: List[Dict[str, Any]]) -> Dict[str, Any]:
239		"""
240		Get performance metrics from conversation history.
241
242		Args:
243		conversations: List of conversation records
244
245		Returns:
246		Dict[str, Any]: Performance metrics
247		"""
248		if not conversations:
249		return {"error": "No conversation history available"}
250
251		# Calculate metrics
252		total_conversations = len(conversations)
253		total_response_time = sum(conv.get("response_time", 0) for conv in conversations)
254		avg_response_time = total_response_time / total_conversations if total_conversations > 0 else 0
255		```
256
257		### 2.5 `enhanced_matrix_tools.py`
258
259		**Strengths:**
260
261		- Clean API design with well-documented functions
262		- Good encapsulation of the underlying modules
263		- Consistent error handling approach
264
265		**Issues:**
266
267		1. **Lines 50-51**: The imports from other modules could lead to circular dependencies.
268
269		```python
270		# Import from core module
271		from ollamatools.tools.enhanced_matrix_core import (
272		EnhancedMatrixClient,
273		_connect_enhanced_matrix
274		)
275		```
276
277		2. **Lines 124-125**: The configuration creation doesn't validate the values.
278
279		```python
280		def create_enhanced_matrix_config() -> Dict:
281		"""
282		Create a default Enhanced Matrix configuration.
283		```
284
285		### 2.6 `enhanced_matrix_client_example.py`
286
287		**Strengths:**
288
289		- Well-documented example with clear instructions
290		- Good command-line argument handling
291		- Comprehensive error handling
292
293		**Issues:**
294
295		1. **Lines 36-37**: The sys.path manipulation is not ideal for production code.
296
297		```python
298		# Add the parent directory to the path so we can import ollamatools
299		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
300		```
301
302		2. **Lines 155-166**: The custom message handler is defined but never used in the example.
303
304		```python
305		async def custom_message_handler(room, event, client):
306		"""
307		Custom message handler for Matrix events.
308
309		This is an example of how to create a custom handler for Matrix messages.
310		It processes the message with Ollama and sends the response back to the room.
311
312		Args:
313		room: The Matrix room where the message was received
314		event: The Matrix event (message)
315		client: The MatrixClient instance
316		"""
317		```
318
319		3. **Lines 168-183**: The random greeting function could be improved with more context-aware greetings.
320
321		```python
322		def get_random_greeting() -> str:
323		"""Generate a random greeting message."""
324		greetings = [
325		"Hello everyone! I'm here to assist with your questions.",
326		"Greetings! I'm your friendly Matrix bot powered by Ollama.",
327		"Hi there! I'm online and ready to chat.",
328		"Good day! I'm here to help with anything you need.",
329		"Hey folks! I've just connected and I'm ready to assist.",
330		"Hello! I'm your AI assistant, ready to help with your questions.",
331		"Greetings, humans! AI assistant at your service.",
332		"Hi everyone! I'm here and ready to chat.",
333		"Hello Matrix world! How can I assist you today?",
334		"Greetings! Your friendly neighborhood AI has arrived."
335		]
336		return random.choice(greetings)
337		```
338
339		## 3. Error Handling and Robustness
340
341		### 3.1 Exception Handling
342
343		The codebase generally has good exception handling with try/except blocks around critical operations. However, there are some areas for improvement:
344
345		1. **In `matrix_tools.py`, lines 196-204**: The exception handling in `process_with_ollama` could be more specific to handle different types of errors (network issues, API errors, etc.).
346
347		2. **In `enhanced_matrix_core.py`, lines 196-217**: The error handling in the message processing could provide more specific error messages based on the type of exception.
348
349		3. **In `enhanced_matrix_search.py`, lines 67-77**: The nested try/except blocks could be simplified for better readability.
350
351		### 3.2 Input Validation
352
353		Input validation is generally good, but there are some areas that could be improved:
354
355		1. **In `matrix_tools.py`, lines 307-315**: The username detection logic could be more robust with regex pattern matching.
356
357		2. **In `enhanced_matrix_history.py`, lines 185-208**: The history command handler doesn't validate that the count is positive.
358
359		### 3.3 Graceful Degradation
360
361		The codebase handles missing dependencies well with feature flags and graceful degradation:
362
363		1. **In `enhanced_matrix_search.py`, lines 15-36**: Good handling of optional dependencies with clear logging.
364
365		2. **In `enhanced_matrix_core.py`, lines 123-126**: Conditional initialization of features based on available dependencies.
366
367		## 4. Security Considerations
368
369		### 4.1 Authentication and Authorization
370
371		1. **In `matrix_tools.py`, lines 196-204**: The Ollama API doesn't appear to have authentication, which could be a security concern.
372
373		2. **In `matrix_tools.py`, lines 339-358**: The model change command checks if the model is available but doesn't validate if the user has permission to use it.
374
375		### 4.2 Data Privacy
376
377		1. **In `enhanced_matrix_history.py`, lines 48-67**: Conversation history is stored as plain JSON without encryption, which could expose sensitive information.
378
379		2. **In `enhanced_matrix_core.py`, lines 186-193**: Search queries might contain sensitive information that is sent to external services.
380
381		### 4.3 Input Sanitization
382
383		1. **In `matrix_tools.py`, lines 307-315**: The username detection doesn't sanitize input, which could lead to false positives.
384
385		2. **In `enhanced_matrix_search.py`, lines 95-107**: The search query is passed directly to the search function without sanitization.
386
387		## 5. Performance Considerations
388
389		### 5.1 Resource Usage
390
391		1. **In `matrix_tools.py`, lines 246-250**: The context management could lead to memory issues with long-running bots.
392
393		2. **In `enhanced_matrix_history.py`, lines 48-67**: The conversation history is loaded entirely into memory, which could be problematic with large histories.
394
395		### 5.2 Efficiency
396
397		1. **In `enhanced_matrix_core.py`, lines 186-193**: The search is performed synchronously, which could block the message handling loop.
398
399		2. **In `enhanced_matrix_history.py`, lines 95-133**: The topic analysis is performed on every request, which could be cached for better performance.
400
401		## 6. Documentation and Comments
402
403		### 6.1 Code Documentation
404
405		The codebase has excellent documentation with clear docstrings for classes and methods. Examples:
406
407		1. **In `matrix_tools.py`, lines 31-59**: Good class documentation with clear explanation of purpose and parameters.
408
409		2. **In `enhanced_matrix_core.py`, lines 19-33**: Clear module documentation with dependency information.
410
411		### 6.2 Comments
412
413		The code includes helpful comments that explain complex logic:
414
415		1. **In `enhanced_matrix_core.py`, lines 186-193**: Good comments explaining the search context handling.
416
417		2. **In `enhanced_matrix_history.py`, lines 95-133**: Clear comments explaining the topic analysis approach.
418
419		### 6.3 Examples
420
421		The example code is well-documented and provides clear usage instructions:
422
423		1. **In `enhanced_matrix_client_example.py`, lines 1-29**: Excellent module documentation with requirements and usage instructions.
424
425		## 7. Recommendations
426
427		### 7.1 Architecture and Design
428
429		1. Consider using a more formal plugin architecture for the enhanced features to make it easier to add new capabilities.
430
431		2. Implement a more sophisticated context management system that can handle longer conversations more efficiently.
432
433		### 7.2 Code Quality
434
435		1. Add more specific exception handling for different types of errors (network, API, authentication, etc.).
436
437		2. Improve the username detection with regex pattern matching to avoid false positives.
438
439		3. Add validation for configuration values to ensure they are within acceptable ranges.
440
441		### 7.3 Security
442
443		1. Add encryption for the conversation history to protect sensitive information.
444
445		2. Implement more robust authentication and authorization for model changes and other administrative commands.
446
447		3. Add input sanitization for search queries and other user inputs.
448
449		### 7.4 Performance
450
451		1. Implement caching for frequently used data like topic analysis results.
452
453		2. Consider using a database for conversation history instead of JSON files for better scalability.
454
455		3. Make search operations asynchronous to avoid blocking the message handling loop.
456
457		### 7.5 Documentation
458
459		1. Add more examples of custom message handlers and how to extend the functionality.
460
461		2. Document the security considerations and best practices for deploying the bot in production.
462
463		## 8. Conclusion
464
465		The OllamaTools Matrix integration is a well-designed and well-documented codebase with good separation of concerns and error handling. The modular architecture makes it easy to understand and extend. The main areas for improvement are in security, performance optimization, and more robust error handling.
466
467		The codebase demonstrates good software engineering practices and provides a solid foundation for building Matrix bots with Ollama integration. With the recommended improvements, it could be even more robust and secure for production use.

ollamatools-project/mkdir_command.sh +0 -3

…	…	@@ -1,3 +0,0 @@
1		#!/bin/bash
2		# Create scripts directory if it doesn't exist
3		mkdir -p scripts

ollamatools-project/ollama-matrix-client +0 -1

…	…	@@ -1 +0,0 @@
1		Subproject commit 44f8aafb2644ad801e94178ae51d66b3ecf2ac32

ollamatools-project/ollamatools/__init__.py +0 -134

…	…	@@ -1,134 +0,0 @@
1		"""
2		OllamaTools: Function Library for Ollama LLMs.
3
4		This package extends Ollama with semantic function calling capabilities,
5		enabling automatic invocation of system commands, web requests, and
6		documentation utilities based on conversational context.
7		"""
8
9		__version__ = "0.2.0"
10
11		from .config import Config
12		from .security import PermissionManager
13
14		# Import OllamaAgent directly from agent.py
15		import importlib.util
16		import os
17		import sys
18
19		# Get the absolute path to agent.py
20		agent_path = os.path.join(os.path.dirname(__file__), 'agent.py')
21
22		# Load the module directly
23		spec = importlib.util.spec_from_file_location("agent_module", agent_path)
24		agent_module = importlib.util.module_from_spec(spec)
25		spec.loader.exec_module(agent_module)
26
27		# Get the OllamaAgent class
28		OllamaAgent = agent_module.OllamaAgent
29
30		from .utils.logger import (
31		setup_logger,
32		get_logger,
33		enable_debug_logging,
34		disable_logging,
35		get_ollamatools_logger
36		)
37
38		# Import new security modules
39		from .command_validator import (
40		validate_and_sanitize_command,
41		CommandValidationError,
42		get_secure_env,
43		is_command_in_allowlist,
44		DEFAULT_ALLOWLIST
45		)
46
47		from .url_validator import (
48		validate_url,
49		URLValidationError,
50		configure_domain_lists,
51		DEFAULT_ALLOWED_SCHEMES,
52		DEFAULT_DOMAIN_ALLOWLIST,
53		DEFAULT_DOMAIN_BLOCKLIST
54		)
55
56		from .path_validator import (
57		validate_path_for_reading,
58		validate_path_for_writing,
59		validate_path_for_searching,
60		PathValidationError,
61		configure_path_sandbox,
62		DEFAULT_SANDBOX_PATH,
63		DEFAULT_ALLOWED_PATHS,
64		DEFAULT_PROTECTED_PATHS
65		)
66
67		# Import error handling utilities
68		from .errors import (
69		OllamaToolsError,
70		CommandError,
71		FileError,
72		WebError,
73		ApiError,
74		PermissionError,
75		TimeoutError,
76		ConfigurationError,
77		OllamaClientError,
78		ERROR_CODES,
79		handle_exception,
80		format_error_response
81		)
82
83		__all__ = [
84		# Core components
85		'Config',
86		'PermissionManager',
87		'OllamaAgent',
88
89		# Logger utilities
90		'setup_logger',
91		'get_logger',
92		'enable_debug_logging',
93		'disable_logging',
94		'get_ollamatools_logger',
95
96		# Command validation
97		'validate_and_sanitize_command',
98		'CommandValidationError',
99		'get_secure_env',
100		'is_command_in_allowlist',
101		'DEFAULT_ALLOWLIST',
102
103		# URL validation
104		'validate_url',
105		'URLValidationError',
106		'configure_domain_lists',
107		'DEFAULT_ALLOWED_SCHEMES',
108		'DEFAULT_DOMAIN_ALLOWLIST',
109		'DEFAULT_DOMAIN_BLOCKLIST',
110
111		# Path validation
112		'validate_path_for_reading',
113		'validate_path_for_writing',
114		'validate_path_for_searching',
115		'PathValidationError',
116		'configure_path_sandbox',
117		'DEFAULT_SANDBOX_PATH',
118		'DEFAULT_ALLOWED_PATHS',
119		'DEFAULT_PROTECTED_PATHS',
120
121		# Error handling
122		'OllamaToolsError',
123		'CommandError',
124		'FileError',
125		'WebError',
126		'ApiError',
127		'PermissionError',
128		'TimeoutError',
129		'ConfigurationError',
130		'OllamaClientError',
131		'ERROR_CODES',
132		'handle_exception',
133		'format_error_response'
134		]

ollamatools-project/ollamatools/agent.py +0 -455

…	…	@@ -1,455 +0,0 @@
1		"""
2		Refactored OllamaAgent implementation for OllamaTools.
3
4		This module provides the OllamaAgent class which serves as the
5		primary interface for interacting with Ollama and executing functions.
6		It has been updated to correctly work with Ollama's function calling API.
7		"""
8
9		import os
10		import re
11		import json
12		import inspect
13		import logging
14		from typing import Dict, Any, List, Optional, Callable, Union, Tuple
15
16		try:
17		import ollama
18		except ImportError:
19		print("Warning: ollama-python not installed. OllamaAgent will not function properly.")
20
21		from ollamatools.config import Config
22		from ollamatools.security import PermissionManager
23		from ollamatools.tools.system_tools import execute_command, read_file, write_file, find_files
24		from ollamatools.tools.web_tools import fetch_url, get_weather, search_web
25		from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
26
27		# Set up logging
28		logger = logging.getLogger(__name__)
29
30		class OllamaAgent:
31		"""
32		Agent for interacting with Ollama and executing functions.
33
34		This class serves as the main interface for OllamaTools, providing methods
35		to chat with Ollama and automatically execute functions based on the
36		conversation context.
37		"""
38
39		def __init__(
40		self,
41		model: str = "llama3:latest",
42		host: str = "http://localhost:11434",
43		permission_prompts: bool = True,
44		sudo_always_prompt: bool = True,
45		config: Optional[Config] = None
46		):
47		"""
48		Initialize the OllamaAgent with specified configuration.
49
50		Args:
51		model: Name of the Ollama model to use
52		host: Ollama server address
53		permission_prompts: Whether to prompt for function execution permission
54		sudo_always_prompt: Whether to always prompt for sudo commands
55		config: Configuration object (overrides individual parameters if provided)
56		"""
57		if config:
58		self.config = config
59		else:
60		self.config = Config(
61		model=model,
62		host=host,
63		permission_prompts=permission_prompts,
64		sudo_always_prompt=sudo_always_prompt
65		)
66
67		# Initialize the permission manager
68		self.permission_manager = PermissionManager(
69		prompt_enabled=self.config.permission_prompts,
70		sudo_always_prompt=self.config.sudo_always_prompt
71		)
72
73		# Set up the Ollama client
74		self.client = ollama.Client(host=self.config.host)
75		logger.info(f"Initialized Ollama client with host: {self.config.host}")
76
77		# Initialize registered functions dictionary and collect available tools
78		self.registered_functions = {}
79		self.collect_tools()
80
81		logger.info(f"OllamaAgent initialized with model={self.config.model}, host={self.config.host}")
82
83		def chat(
84		self,
85		message: str,
86		system_prompt: Optional[str] = None,
87		context: Optional[List[Dict[str, str]]] = None
88		) -> str:
89		"""
90		Send a message to Ollama, automatically detecting and executing functions.
91
92		Args:
93		message: User message
94		system_prompt: Optional system prompt
95		context: Optional conversation history
96
97		Returns:
98		str: Response from Ollama, potentially after executing functions
99		"""
100		if context is None:
101		context = []
102
103		logger.info(f"Sending message to Ollama: {message[:50]}...")
104
105		try:
106		# Prepare the chat request
107		messages = context + [{"role": "user", "content": message}]
108		options = {}
109
110		# Add system prompt if provided
111		if system_prompt:
112		options["system"] = system_prompt
113
114		# Generate tool definitions
115		tools = self.generate_tool_definitions()
116
117		# Send the request to Ollama with tools
118		response = self.client.chat(
119		model=self.config.model,
120		messages=messages,
121		options=options,
122		tools=tools, # Pass tools with each request
123		stream=False
124		)
125
126		# Extract the assistant's response
127		assistant_content = ""
128		tool_calls = []
129
130		# Handle different response formats based on Ollama version/client
131		if hasattr(response, 'message'):
132		# Direct object access (newer Ollama client)
133		assistant_content = response.message.content
134		tool_calls = getattr(response.message, 'tool_calls', []) or []
135		else:
136		# Dictionary access (older Ollama client)
137		assistant_message = response.get("message", {})
138		assistant_content = assistant_message.get("content", "")
139		tool_calls = assistant_message.get("tool_calls", [])
140
141		# Check if we have tool calls to process
142		if tool_calls and len(tool_calls) > 0:
143		logger.info(f"Detected {len(tool_calls)} tool calls")
144
145		# Process the function calls
146		tool_results = []
147		for tool_call in tool_calls:
148		# Extract function name and arguments based on response structure
149		if hasattr(tool_call, 'function'):
150		# Direct object access
151		function_name = tool_call.function.name
152		function_args = tool_call.function.arguments
153		if isinstance(function_args, str):
154		function_args = json.loads(function_args)
155		else:
156		# Dictionary access
157		function_data = tool_call.get('function', {})
158		function_name = function_data.get('name', '')
159		function_args = function_data.get('arguments', {})
160		if isinstance(function_args, str):
161		function_args = json.loads(function_args)
162
163		# Check if function exists
164		if function_name not in self.registered_functions:
165		logger.error(f"Function '{function_name}' not found in registered functions")
166		tool_results.append({
167		"error": f"Function '{function_name}' not found"
168		})
169		continue
170
171		# Get the function
172		func = self.registered_functions[function_name]
173
174		# Request permission
175		is_sudo = False
176		if function_name == "execute_command" and "command" in function_args:
177		is_sudo = self.permission_manager.detect_sudo(function_args["command"])
178
179		permission_granted = self.permission_manager.request_permission(
180		function_name=function_name,
181		args=function_args,
182		is_sudo=is_sudo
183		)
184
185		# Execute function if permission granted
186		if permission_granted:
187		try:
188		result = func(**function_args)
189		tool_results.append(result)
190		logger.info(f"Successfully executed {function_name}")
191		except Exception as e:
192		logger.error(f"Error executing {function_name}: {str(e)}")
193		tool_results.append({
194		"error": f"Error: {str(e)}"
195		})
196		else:
197		logger.warning(f"Permission denied for {function_name}")
198		tool_results.append({
199		"error": "Permission denied"
200		})
201
202		# Construct tool messages for the follow-up request
203		tool_messages = []
204		for i, result in enumerate(tool_results):
205		# Get function name from original tool call
206		if hasattr(tool_calls[i], 'function'):
207		function_name = tool_calls[i].function.name
208		else:
209		function_name = tool_calls[i].get('function', {}).get('name', '')
210
211		# Add tool message
212		tool_messages.append({
213		"role": "tool",
214		"name": function_name,
215		"content": json.dumps(result)
216		})
217
218		# Update conversation with function results
219		updated_messages = messages + [
220		{"role": "assistant", "content": assistant_content, "tool_calls": tool_calls}
221		] + tool_messages
222
223		# Send follow-up request with tool results
224		try:
225		follow_up_response = self.client.chat(
226		model=self.config.model,
227		messages=updated_messages,
228		options=options,
229		stream=False
230		)
231
232		# Extract final response
233		if hasattr(follow_up_response, 'message'):
234		# Direct object access
235		final_content = follow_up_response.message.content
236		else:
237		# Dictionary access
238		final_content = follow_up_response.get("message", {}).get("content", "")
239
240		logger.info(f"Received final response after tool execution")
241		return final_content
242		except Exception as e:
243		logger.error(f"Error in follow-up request: {str(e)}")
244		return f"Error processing function results: {str(e)}"
245
246		# Return the assistant's response if no tool calls
247		logger.info(f"Received response from Ollama")
248		return assistant_content
249
250		except Exception as e:
251		logger.error(f"Error in chat: {str(e)}")
252		return f"Error communicating with Ollama: {str(e)}"
253
254		def collect_tools(self):
255		"""
256		Collect all available tools from the tool modules.
257
258		This method scans the tool modules and collects function references
259		to be used when executing tools.
260		"""
261		logger.info("Collecting available tools...")
262
263		# Collect tool functions
264		tool_modules = [
265		("system_tools", [execute_command, read_file, write_file, find_files]),
266		("web_tools", [fetch_url, get_weather, search_web]),
267		("doc_tools", [generate_markdown, extract_context, summarize_text])
268		]
269
270		# Register functions
271		for module_name, functions in tool_modules:
272		for func in functions:
273		if func.__name__.startswith('_'):
274		continue
275
276		# Store the function reference
277		self.registered_functions[func.__name__] = func
278
279		logger.info(f"Collected {len(self.registered_functions)} tools")
280
281		def generate_tool_definitions(self) -> List[Dict[str, Any]]:
282		"""
283		Generate tool definitions for all registered functions.
284
285		Returns:
286		List[Dict[str, Any]]: List of tool definitions in Ollama's format
287		"""
288		tools = []
289
290		for func_name, func in self.registered_functions.items():
291		# Get function docstring
292		func_doc = inspect.getdoc(func) or "No description available"
293
294		# Generate tool definition
295		tool_def = self._generate_tool_definition(func_name, func, func_doc)
296		tools.append(tool_def)
297
298		return tools
299
300		def _generate_tool_definition(self, name: str, func: Callable, docstring: str) -> Dict[str, Any]:
301		"""
302		Generate a tool definition from a function according to Ollama's format.
303
304		Args:
305		name: Function name
306		func: Function object
307		docstring: Function docstring
308
309		Returns:
310		dict: Tool definition compatible with Ollama
311		"""
312		# Extract function signature
313		sig = inspect.signature(func)
314
315		# Generate description
316		description = self._extract_description_from_docstring(docstring)
317
318		# Generate parameters
319		parameters = {
320		"type": "object",
321		"properties": {},
322		"required": []
323		}
324
325		for param_name, param in sig.parameters.items():
326		# Skip permission_manager parameter
327		if param_name == "permission_manager":
328		continue
329
330		# Get parameter type
331		param_type = self._get_param_type(param)
332
333		# Get parameter description
334		param_desc = self._extract_param_description(docstring, param_name)
335
336		# Add parameter to properties
337		parameters["properties"][param_name] = {
338		"type": param_type,
339		"description": param_desc
340		}
341
342		# Add to required list if no default value
343		if param.default is inspect.Parameter.empty:
344		parameters["required"].append(param_name)
345
346		# Create tool definition in Ollama's format
347		tool_def = {
348		"type": "function",
349		"function": {
350		"name": name,
351		"description": description,
352		"parameters": parameters
353		}
354		}
355
356		return tool_def
357
358		def _extract_description_from_docstring(self, docstring: str) -> str:
359		"""
360		Extract the main description from a docstring.
361
362		Args:
363		docstring: Function docstring
364
365		Returns:
366		str: Function description
367		"""
368		if not docstring:
369		return "No description available"
370
371		# Split the docstring into lines
372		lines = docstring.strip().split('\n')
373
374		# Extract the first paragraph
375		description = []
376		for line in lines:
377		line = line.strip()
378		if not line or line.startswith('Args:') or line.startswith('Returns:'):
379		break
380		description.append(line)
381
382		return ' '.join(description) if description else "No description available"
383
384		def _extract_param_description(self, docstring: str, param_name: str) -> str:
385		"""
386		Extract parameter description from docstring.
387
388		Args:
389		docstring: Function docstring
390		param_name: Parameter name
391
392		Returns:
393		str: Parameter description
394		"""
395		if not docstring:
396		return "No description available"
397
398		# Look for parameters section
399		pattern = rf"\s+{re.escape(param_name)}:\s*(.*?)(?:\n\s+\w+:|$)"
400		match = re.search(pattern, docstring, re.DOTALL)
401
402		if match:
403		description = match.group(1).strip()
404		return description
405
406		return "No description available"
407
408		def _get_param_type(self, param: inspect.Parameter) -> str:
409		"""
410		Get parameter type for Ollama tool definition.
411
412		Args:
413		param: Parameter object
414
415		Returns:
416		str: Parameter type for Ollama
417		"""
418		annotation = param.annotation
419
420		if annotation is inspect.Parameter.empty:
421		return "string"
422
423		# Convert type annotation to string
424		annotation_str = str(annotation)
425
426		# Determine parameter type
427		if "str" in annotation_str:
428		return "string"
429		elif "int" in annotation_str:
430		return "integer"
431		elif "float" in annotation_str:
432		return "number"
433		elif "bool" in annotation_str:
434		return "boolean"
435		elif "List" in annotation_str or "list" in annotation_str:
436		return "array"
437		elif "Dict" in annotation_str or "dict" in annotation_str:
438		return "object"
439		elif "Optional" in annotation_str:
440		# Handle Optional types
441		if "str" in annotation_str:
442		return "string"
443		elif "int" in annotation_str:
444		return "integer"
445		elif "float" in annotation_str:
446		return "number"
447		elif "bool" in annotation_str:
448		return "boolean"
449		elif "List" in annotation_str or "list" in annotation_str:
450		return "array"
451		elif "Dict" in annotation_str or "dict" in annotation_str:
452		return "object"
453
454		# Default to string for unknown types
455		return "string"

ollamatools-project/ollamatools/agent/Untitled File +0 -0

(no content changes — binary or rename)

ollamatools-project/ollamatools/agent/__init__.py +0 -16

…	…	@@ -1,16 +0,0 @@
1		"""
2		OllamaTools Agent module initialization.
3
4		This package provides agent implementations for OllamaTools.
5		"""
6
7		# Import the ModularOllamaAgent
8		from .modular import ModularOllamaAgent
9
10		# Import the ExtendedModularAgent
11		from .extended import ExtendedModularAgent
12
13		__all__ = [
14		'ModularOllamaAgent',
15		'ExtendedModularAgent'
16		]

ollamatools-project/ollamatools/agent/extended/README.md +0 -109

…	…	@@ -1,109 +0,0 @@
1		# ExtendedModularAgent
2
3		This package provides extensions to the ModularOllamaAgent with additional features like conversation history tracking, conversation summarization, topic analysis, and performance metrics.
4
5		## Overview
6
7		The ExtendedModularAgent builds on the modular architecture of the ModularOllamaAgent, adding features that enhance its capabilities for tracking and analyzing conversations.
8
9		## Features
10
11		### Conversation History Tracking
12
13		The ExtendedModularAgent automatically tracks all conversations, including:
14		- User messages
15		- Agent responses
16		- Response times
17		- Estimated token usage
18
19		This history is saved to a JSON file for persistence across sessions.
20
21		### Conversation Summarization
22
23		The agent can summarize recent conversations, providing a quick overview of:
24		- Timestamps
25		- User messages (truncated for readability)
26		- Response times
27
28		### Topic Analysis
29
30		The agent can analyze conversation topics based on keywords, categorizing conversations into topics like:
31		- Help requests
32		- Search queries
33		- File operations
34		- Command executions
35		- Code-related questions
36		- Error reports
37		- Issue discussions
38		- General questions
39
40		### Performance Metrics
41
42		The agent tracks performance metrics, including:
43		- Total number of conversations
44		- Average response time
45		- Total estimated token usage
46		- Average tokens per conversation
47		- Slowest and fastest responses
48
49		## Usage
50
51		To use the ExtendedModularAgent:
52
53		```python
54		from ollamatools.agent.extended import ExtendedModularAgent
55
56		# Initialize the agent
57		agent = ExtendedModularAgent(
58		model="llama3.1:latest",
59		permission_prompts=True
60		)
61
62		# Chat with the agent
63		response = agent.chat("Hello, how can you help me?")
64		print(response)
65
66		# Get conversation summary
67		summary = agent.summarize_conversations(n=5)
68		print(summary)
69
70		# Analyze conversation topics
71		topics = agent.analyze_conversation_topics()
72		print(topics)
73
74		# Get performance metrics
75		metrics = agent.get_performance_metrics()
76		print(metrics)
77		```
78
79		## Example Script
80
81		See the [extended_modular_agent.py](../../../../examples/extended_modular_agent.py) example script for a complete demonstration of the ExtendedModularAgent's capabilities.
82
83		## Extending Further
84
85		The ExtendedModularAgent can be further extended with additional functionality:
86
87		```python
88		from ollamatools.agent.extended import ExtendedModularAgent
89
90		class SuperExtendedAgent(ExtendedModularAgent):
91		"""An even more extended agent with additional capabilities."""
92
93		def __init__(self, *args, **kwargs):
94		super().__init__(*args, **kwargs)
95		# Add custom initialization
96
97		def custom_method(self):
98		"""Implement a custom method."""
99		# Your implementation here
100		pass
101		```
102
103		## Documentation
104
105		For more detailed documentation, see:
106
107		- [ModularOllamaAgent Guide](../../../../docs/modular_agent_guide.md): Comprehensive guide to using the ModularOllamaAgent
108		- [Migration Guide](../../../../docs/migration_guide.md): Guide for migrating from TransparentOllamaAgent to ModularOllamaAgent
109		- [Example Scripts](../../../../examples/): Example scripts demonstrating the ModularOllamaAgent and ExtendedModularAgent

ollamatools-project/ollamatools/agent/extended/__init__.py +0 -13

…	…	@@ -1,13 +0,0 @@
1		"""
2		Extended ModularOllamaAgent package.
3
4		This package provides extensions to the ModularOllamaAgent with additional
5		features like conversation history tracking, conversation summarization,
6		topic analysis, and performance metrics.
7		"""
8
9		# Import the ExtendedModularAgent from the example script
10		# In a real implementation, this would be defined in a proper module file
11		from examples.extended_modular_agent import ExtendedModularAgent
12
13		__all__ = ['ExtendedModularAgent']

ollamatools-project/ollamatools/agent/modular/README.md +0 -97

…	…	@@ -1,97 +0,0 @@
1		# ModularOllamaAgent
2
3		This package provides a modular implementation of the OllamaAgent with enhanced capabilities like search integration, command management, issue tracking, and semantic function selection feedback.
4
5		## Module Structure
6
7		The ModularOllamaAgent is organized into several modules:
8
9		- `agent.py`: Main agent implementation
10		- `search.py`: Search functionality
11		- `commands.py`: Command management
12		- `issues.py`: Issue tracking
13		- `feedback.py`: Semantic function selection feedback
14		- `utils.py`: Utility functions
15
16		## Architecture
17
18		The ModularOllamaAgent follows a modular architecture where functionality is separated into logical components:
19
20		```
21		ModularOllamaAgent
22		├── Core Agent (agent.py)
23		│ ├── Initialization
24		│ ├── Chat Processing
25		│ └── Error Handling
26		│
27		├── Search Module (search.py)
28		│ ├── Intent Detection
29		│ ├── Search Execution
30		│ └── Result Processing
31		│
32		├── Commands Module (commands.py)
33		│ ├── Command Listing
34		│ ├── Command Addition
35		│ └── Command Removal
36		│
37		├── Issues Module (issues.py)
38		│ ├── Issue Loading/Saving
39		│ ├── Issue Creation
40		│ └── Issue Management
41		│
42		├── Feedback Module (feedback.py)
43		│ ├── Feedback Collection
44		│ ├── Pattern Analysis
45		│ └── Command Intent Detection
46		│
47		└── Utilities (utils.py)
48		├── Status Messages
49		├── Formatting Functions
50		└── Error Suggestions
51		```
52
53		## Usage
54
55		To use the ModularOllamaAgent:
56
57		```python
58		from ollamatools.agent import ModularOllamaAgent
59
60		# Initialize the agent
61		agent = ModularOllamaAgent(
62		model="llama3.1:latest",
63		permission_prompts=True
64		)
65
66		# Chat with the agent
67		response = agent.chat("Hello, how can you help me?")
68		print(response)
69		```
70
71		## Extending the Agent
72
73		The modular architecture makes it easy to extend the agent with custom functionality:
74
75		```python
76		from ollamatools.agent import ModularOllamaAgent
77
78		class CustomAgent(ModularOllamaAgent):
79		"""A custom agent with additional capabilities."""
80
81		def __init__(self, *args, **kwargs):
82		super().__init__(*args, **kwargs)
83		# Add custom initialization
84
85		def custom_method(self, param):
86		"""Implement a custom method."""
87		# Your implementation here
88		return f"Processed {param}"
89		```
90
91		## Documentation
92
93		For more detailed documentation, see:
94
95		- [ModularOllamaAgent Guide](../../../../docs/modular_agent_guide.md): Comprehensive guide to using the ModularOllamaAgent
96		- [Migration Guide](../../../../docs/migration_guide.md): Guide for migrating from TransparentOllamaAgent to ModularOllamaAgent
97		- [Example Scripts](../../../../examples/): Example scripts demonstrating the ModularOllamaAgent

ollamatools-project/ollamatools/agent/modular/__init__.py +0 -42

…	…	@@ -1,42 +0,0 @@
1		"""
2		ModularOllamaAgent package.
3
4		This package provides a modular implementation of the OllamaAgent with
5		enhanced capabilities like search integration, command management,
6		issue tracking, and semantic function selection feedback.
7		"""
8
9		from .agent import ModularOllamaAgent
10		from .utils import (
11		print_status, print_success, print_error,
12		print_warning, print_info, suggest_fix,
13		format_code_block, format_table, truncate_text,
14		format_execution_time
15		)
16		from .feedback import FunctionSelectionFeedback
17		from .search import detect_factual_intent, extract_search_context, execute_search
18		from .commands import list_allowed_commands, add_allowed_command, remove_allowed_command
19		from .issues import load_issues, save_issues, add_issue, list_issues, get_issue, update_issue
20
21		__all__ = [
22		# Main agent class
23		'ModularOllamaAgent',
24
25		# Utility functions
26		'print_status', 'print_success', 'print_error',
27		'print_warning', 'print_info', 'suggest_fix',
28		'format_code_block', 'format_table', 'truncate_text',
29		'format_execution_time',
30
31		# Feedback system
32		'FunctionSelectionFeedback',
33
34		# Search functions
35		'detect_factual_intent', 'extract_search_context', 'execute_search',
36
37		# Command management functions
38		'list_allowed_commands', 'add_allowed_command', 'remove_allowed_command',
39
40		# Issue tracking functions
41		'load_issues', 'save_issues', 'add_issue', 'list_issues', 'get_issue', 'update_issue'
42		]

ollamatools-project/ollamatools/agent/modular/agent.py +0 -702

…	…	@@ -1,702 +0,0 @@
1		"""
2		ModularOllamaAgent implementation.
3
4		This module provides the ModularOllamaAgent class, which extends
5		the base OllamaAgent with modular components, detailed feedback, and
6		additional features like search integration, command management,
7		issue tracking, and semantic function selection feedback.
8		"""
9
10		import os
11		import sys
12		import json
13		import time
14		import logging
15		import re
16		from typing import Dict, Any, List, Optional, Set, Union, Tuple
17
18		from ollamatools import OllamaAgent, Config
19		from ollamatools.command_validator import DEFAULT_ALLOWLIST
20		from ollamatools.utils.logger import setup_logger
21		from ollamatools.opensearch_web_tools import (
22		opensearch_query,
23		opensearch_fetch_url,
24		get_opensearch_info,
25		DEFAULT_OPENSEARCH_CONFIG
26		)
27		from ollamatools.errors import (
28		OllamaToolsError,
29		CommandError,
30		FileError,
31		WebError,
32		handle_exception
33		)
34		from ollamatools.tools.system_tools import execute_command
35
36		# Import agent components
37		from .search import detect_factual_intent, extract_search_context, execute_search
38		from .commands import list_allowed_commands, add_allowed_command, remove_allowed_command
39		from .issues import load_issues, save_issues, add_issue, list_issues, get_issue, update_issue
40		from .utils import print_status, print_success, print_error, print_warning, print_info, suggest_fix
41		from .feedback import (
42		FunctionSelectionFeedback,
43		detect_command_intent,
44		extract_command_from_similar_queries,
45		format_command_result
46		)
47
48		# Set up logging
49		logger = setup_logger("modular_ollama_agent", level="INFO")
50
51		# Extend the default command allowlist with useful commands
52		EXTENDED_ALLOWLIST = DEFAULT_ALLOWLIST.union({
53		"python", "python3", "calc", "bc", "find", "grep", "awk", "sed",
54		"head", "tail", "more", "less", "cat", "echo", "cd", "pwd"
55		})
56
57		# Configuration for OpenSearchNG
58		OPENSEARCH_CONFIG = DEFAULT_OPENSEARCH_CONFIG.copy()
59		OPENSEARCH_CONFIG.update({
60		"base_url": "http://localhost:8081", # Updated to your container's port
61		"engines": ["google", "bing", "duckduckgo", "brave"],
62		"num_results": 5
63		})
64
65		class ModularOllamaAgent:
66		"""
67		A modular OllamaAgent that provides transparent execution and better feedback.
68
69		This agent is designed with a modular architecture that separates functionality
70		into logical components for better maintainability and extensibility. It includes
71		features like search integration, command management, issue tracking, and
72		semantic function selection feedback.
73		"""
74
75		def __init__(
76		self,
77		model: str = "llama3.1:latest",
78		permission_prompts: bool = True,
79		opensearch_config: Optional[Dict[str, Any]] = None,
80		command_allowlist: Optional[Set[str]] = None,
81		feedback_file_path: Optional[str] = None
82		):
83		"""
84		Initialize the agent.
85
86		Args:
87		model: Ollama model to use
88		permission_prompts: Whether to display permission prompts
89		opensearch_config: OpenSearchNG configuration
90		command_allowlist: Custom command allowlist
91		feedback_file_path: Path to the feedback file (default: ~/.ollamatools/function_selection_feedback.jsonl)
92		"""
93		# Initialize OllamaAgent
94		self.agent = OllamaAgent(
95		model=model,
96		permission_prompts=permission_prompts
97		)
98
99		# Store configuration
100		self.opensearch_config = opensearch_config or OPENSEARCH_CONFIG
101		self.command_allowlist = command_allowlist or EXTENDED_ALLOWLIST
102
103		# Initialize state
104		self.context = []
105		self.execution_history = []
106
107		# Initialize issue tracking
108		self.issues = []
109		self.issue_id_counter = 1
110
111		# Path for saving issues
112		self.issues_file = os.path.join(
113		os.path.expanduser('~'),
114		'.ollamatools',
115		'issues.json'
116		)
117
118		# Create directory if it doesn't exist
119		os.makedirs(os.path.dirname(self.issues_file), exist_ok=True)
120
121		# Load existing issues if any
122		self.load_issues()
123
124		# Initialize function selection feedback system
125		self.function_feedback = FunctionSelectionFeedback(feedback_file_path)
126		logger.info("Initialized function selection feedback system")
127
128		# Define system prompt
129		self.system_prompt = f"""
130		You are an AI assistant with modular access to various tools.
131		You always explain what you're doing step by step and show your work.
132
133		When a user asks you a question, you should:
134		1. Identify the most appropriate tool for the task
135		2. Describe what you're going to do before you do it
136		3. Show the results of any commands or functions you execute
137		4. Explain how you interpret these results
138
139		MOST IMPORTANTLY: When calculating math problems, ALWAYS use the 'calc' command rather than
140		doing the calculation yourself or using python -c. For example: calc 2+2
141
142		For searching the web:
143		- Use the 'opensearch_query' function to perform web searches
144		- Use the 'opensearch_fetch_url' function to fetch web page content safely
145		- Specify search engines if needed (available: {', '.join(self.opensearch_config['engines'])})
146
147		For file and system operations:
148		- Use 'find_files' to locate files
149		- Use 'read_file' to view file content
150		- Use 'execute_command' for running shell commands like 'ls', 'pwd', etc.
151
152		For system information queries:
153		- When a user asks about the current directory, use 'pwd' command
154		- When a user asks about file listings, use 'ls' command
155		- When a user asks about the current user, use 'whoami' command
156		- When a user asks about system information, prefer shell commands over other methods
157
158		Always check command permissions carefully and explain what each command does.
159		For directory structure, use 'find .' or 'ls -la' commands.
160
161		When a user asks for information that might require searching the web or retrieving external data,
162		automatically determine if you should use the 'opensearch_query' function instead of requiring explicit search commands.
163
164		For example, if a user asks "can you give me a recipe for chilli sauce?", you should:
165		1. Recognize this as a request that requires external information
166		2. Use the opensearch_query function with an appropriate search query
167		3. Present the results in a helpful format
168
169		Only use tools when necessary - for factual questions you can answer directly, don't use search.
170		"""
171
172		# Check if OpenSearchNG is available
173		try:
174		self.print_status("Checking OpenSearchNG availability...")
175		info = get_opensearch_info(config=self.opensearch_config)
176		self.print_success(f"OpenSearchNG running at {self.opensearch_config['base_url']}")
177
178		# Log available engines
179		if 'engines' in info and info['engines']:
180		self.print_info(f"Available search engines: {', '.join(info['engines'])}")
181		except Exception as e:
182		self.print_error(f"OpenSearchNG not available: {str(e)}")
183		self.print_warning("Falling back to regular search methods")
184		self.suggest_fix("Make sure your OpenSearchNG instance is running at " +
185		f"{self.opensearch_config['base_url']}")
186
187		# Utility methods redirected to utils module
188		def print_status(self, message: str) -> None:
189		print_status(message)
190
191		def print_success(self, message: str) -> None:
192		print_success(message)
193
194		def print_error(self, message: str) -> None:
195		print_error(message)
196
197		def print_warning(self, message: str) -> None:
198		print_warning(message)
199
200		def print_info(self, message: str) -> None:
201		print_info(message)
202
203		def suggest_fix(self, suggestion: str) -> None:
204		suggest_fix(suggestion)
205
206		# Search methods redirected to search module
207		def detect_factual_intent(self, message: str) -> bool:
208		return detect_factual_intent(message)
209
210		def extract_search_context(self, search_results: Dict[str, Any]) -> str:
211		return extract_search_context(search_results)
212
213		def execute_search(self, query: str) -> Dict[str, Any]:
214		return execute_search(query, self.opensearch_config, self.agent.permission_manager, self.execution_history)
215
216		# Command management methods redirected to commands module
217		def list_allowed_commands(self) -> List[str]:
218		return list_allowed_commands(self.command_allowlist)
219
220		def add_allowed_command(self, command: str) -> bool:
221		return add_allowed_command(command, self.command_allowlist)
222
223		def remove_allowed_command(self, command: str) -> bool:
224		return remove_allowed_command(command, self.command_allowlist)
225
226		# Issue tracking methods redirected to issues module
227		def load_issues(self) -> None:
228		load_issues(self)
229
230		def save_issues(self) -> None:
231		save_issues(self)
232
233		def add_issue(self, title: str, description: str, severity: str = 'medium') -> int:
234		return add_issue(self, title, description, severity)
235
236		def list_issues(self, status: Optional[str] = None) -> List[Dict[str, Any]]:
237		return list_issues(self, status)
238
239		def get_issue(self, issue_id: int) -> Optional[Dict[str, Any]]:
240		return get_issue(self, issue_id)
241
242		def update_issue(self, issue_id: int, status: Optional[str] = None,
243		severity: Optional[str] = None, description: Optional[str] = None) -> bool:
244		return update_issue(self, issue_id, status, severity, description)
245
246		def show_execution_history(self, max_entries: int = 10) -> None:
247		"""
248		Display the execution history.
249
250		Args:
251		max_entries: Maximum number of history entries to show
252		"""
253		if not self.execution_history:
254		print("\n📜 Execution history is empty.")
255		return
256
257		# Get the most recent entries
258		recent_entries = self.execution_history[-max_entries:]
259
260		print("\n📜 Execution History:")
261		print("=" * 60)
262
263		for i, entry in enumerate(recent_entries, 1):
264		# Format timestamp
265		timestamp = time.strftime("%H:%M:%S", time.localtime(entry["timestamp"]))
266
267		# Format function and args
268		function_name = entry["function"]
269		args_str = ", ".join(f"{k}={v!r}" for k, v in entry["args"].items())
270
271		# Format status
272		status = "✅ Success" if entry.get("success", False) else "❌ Failed"
273
274		# Print the entry
275		print(f"{i}. [{timestamp}] {function_name}({args_str}) -> {status}")
276
277		# Show additional details for errors
278		if not entry.get("success", False) and "error" in entry:
279		print(f" Error: {entry['error']}")
280
281		# Show additional details for commands
282		if function_name == "execute_command" and "returncode" in entry:
283		print(f" Exit code: {entry['returncode']}")
284
285		# Show additional details for search
286		if function_name == "opensearch_query" and "num_results" in entry:
287		print(f" Results: {entry['num_results']}")
288
289		print("=" * 60)
290
291		def get_directory_structure(self, path: str = '.', max_depth: int = 2) -> Dict[str, Any]:
292		"""
293		Get a formatted directory structure.
294
295		Args:
296		path: Starting directory path
297		max_depth: Maximum directory depth to show
298
299		Returns:
300		dict: Directory structure information
301		"""
302		try:
303		self.print_status(f"Getting directory structure for: {path} (depth: {max_depth})")
304
305		# Execute find command to get the structure
306		from ollamatools.tools.system_tools import execute_command
307		result = execute_command(
308		command="find",
309		args=[path, "-maxdepth", str(max_depth), "-type", "f", "-o", "-type", "d"],
310		permission_manager=self.agent.permission_manager
311		)
312
313		if "error" in result:
314		return result
315
316		# Process the output into a more structured format
317		lines = result.get("stdout", "").strip().split('\n')
318
319		structure = {
320		"path": path,
321		"depth": max_depth,
322		"entries": lines,
323		"files": [line for line in lines if os.path.isfile(line)],
324		"directories": [line for line in lines if os.path.isdir(line)],
325		"count": {
326		"total": len(lines),
327		"files": len([line for line in lines if os.path.isfile(line)]),
328		"directories": len([line for line in lines if os.path.isdir(line)])
329		}
330		}
331
332		self.print_success(f"Found {structure['count']['total']} entries "
333		f"({structure['count']['files']} files, "
334		f"{structure['count']['directories']} directories)")
335
336		return structure
337
338		except Exception as e:
339		self.print_error(f"Failed to get directory structure: {str(e)}")
340		self.suggest_fix(self._generate_error_fix_suggestion(e))
341
342		return {
343		"error": str(e),
344		"path": path,
345		"depth": max_depth
346		}
347
348		def _generate_error_fix_suggestion(self, error: Exception) -> str:
349		"""
350		Generate a fix suggestion based on the error type.
351
352		Args:
353		error: The exception
354
355		Returns:
356		str: Suggested fix
357		"""
358		if isinstance(error, CommandError):
359		return self._generate_command_fix_suggestion(error)
360		elif isinstance(error, FileError):
361		return self._generate_file_fix_suggestion(error)
362		elif isinstance(error, WebError):
363		return self._generate_web_fix_suggestion(error)
364		else:
365		# Generic suggestion
366		return f"Try a different approach or check if the required resources are available."
367
368		def _generate_command_fix_suggestion(self, error: Union[CommandError, Dict[str, Any]]) -> str:
369		"""
370		Generate a fix suggestion for command errors.
371
372		Args:
373		error: Error object or dictionary
374
375		Returns:
376		str: Suggested fix
377		"""
378		# Extract error details
379		if isinstance(error, CommandError):
380		error_msg = str(error)
381		details = error.details or {}
382		command = details.get('command', '')
383		stderr = details.get('stderr', '')
384		else:
385		error_msg = error.get('error', '')
386		command = error.get('command', '')
387		stderr = error.get('stderr', '')
388
389		# Check for common error patterns
390		if "not in allowed command list" in error_msg:
391		return (f"The command '{command}' is not in the allowed command list. "
392		f"Try using one of these commands instead: {', '.join(sorted(list(self.command_allowlist)[:10]))}... "
393		f"or add this command to the allowlist when initializing the agent.")
394
395		elif "dangerous pattern" in error_msg:
396		pattern = error_msg.split("pattern matching '")[1].split("'")[0] if "pattern matching '" in error_msg else ""
397		return (f"The command contains potentially dangerous pattern: {pattern}. "
398		f"Try reformulating the command to avoid shell metacharacters.")
399
400		elif "not found" in error_msg or "command not found" in stderr:
401		return f"The command '{command}' does not exist on this system. Check the spelling or install the command."
402
403		elif "permission denied" in stderr.lower():
404		return f"Permission denied when executing '{command}'. Try running with appropriate permissions."
405
406		else:
407		return f"Try a different command syntax or check command documentation."
408
409		def _generate_file_fix_suggestion(self, error: FileError) -> str:
410		"""
411		Generate a fix suggestion for file errors.
412
413		Args:
414		error: File error
415
416		Returns:
417		str: Suggested fix
418		"""
419		error_msg = str(error)
420		details = error.details or {}
421		path = details.get('path', '')
422
423		if "not found" in error_msg:
424		return f"The file '{path}' does not exist. Check the path or create the file."
425
426		elif "is a directory" in error_msg:
427		return f"'{path}' is a directory, not a file. Try specifying a file path instead."
428
429		elif "permission denied" in error_msg.lower():
430		return f"Permission denied when accessing '{path}'. Check file permissions."
431
432		elif "exceeds maximum size" in error_msg:
433		return f"The file '{path}' is too large. Try reading a smaller file or increasing max_size."
434
435		elif "outside the allowed directories" in error_msg:
436		return f"The path '{path}' is outside the allowed directories. Try a path within the sandbox."
437
438		else:
439		return f"Check if the file exists and you have appropriate permissions."
440
441		def _generate_web_fix_suggestion(self, error: WebError) -> str:
442		"""
443		Generate a fix suggestion for web errors.
444
445		Args:
446		error: Web error
447
448		Returns:
449		str: Suggested fix
450		"""
451		error_msg = str(error)
452		details = error.details or {}
453		url = details.get('url', '')
454
455		if "domain" in error_msg and "blocked" in error_msg:
456		return f"The domain in '{url}' is blocked for security reasons. Try a different domain."
457
458		elif "scheme" in error_msg and "not allowed" in error_msg:
459		return f"The URL scheme is not allowed. Use http:// or https:// instead."
460
461		elif "Connection refused" in error_msg:
462		return f"Could not connect to '{url}'. Check if the service is running and accessible."
463
464		elif "timeout" in error_msg.lower():
465		return f"The request to '{url}' timed out. Try again later or check your internet connection."
466
467		elif "Status code:" in error_msg and "404" in error_msg:
468		return f"The resource at '{url}' was not found (404). Check the URL for typos."
469
470		elif "Status code:" in error_msg and "403" in error_msg:
471		return f"Access to '{url}' is forbidden (403). The site may require authentication."
472
473		else:
474		return f"Try a different URL or check your internet connection."
475
476		def detect_command_intent(self, message: str) -> Tuple[bool, Optional[str]]:
477		"""
478		Detect if a message is asking for system information that requires a command.
479
480		Args:
481		message: User message
482
483		Returns:
484		Tuple[bool, Optional[str]]: (has_command_intent, suggested_command)
485		"""
486		return detect_command_intent(message, self.function_feedback)
487
488		def provide_function_feedback(self, query: str, correct_function: str, actual_function: str = None) -> bool:
489		"""
490		Allow users to provide feedback on function selection.
491
492		Args:
493		query: The original query
494		correct_function: The function that should have been used
495		actual_function: The function that was actually used (if known)
496
497		Returns:
498		bool: True if feedback was recorded successfully
499		"""
500		if actual_function:
501		# Record explicit feedback
502		self.function_feedback.record_explicit_feedback(
503		query, correct_function, actual_function, user_correction=True
504		)
505		self.print_success(f"Thank you for your feedback! I'll remember to use {correct_function} for similar queries in the future.")
506		return True
507		else:
508		# Try to find the most recent function used for this query
509		for entry in reversed(self.execution_history):
510		if entry.get("query", "") == query:
511		actual_function = entry.get("function")
512		if actual_function and actual_function != correct_function:
513		self.function_feedback.record_explicit_feedback(
514		query, correct_function, actual_function, user_correction=True
515		)
516		self.print_success(f"Thank you for your feedback! I'll remember to use {correct_function} for similar queries in the future.")
517		return True
518
519		# If we couldn't find a matching entry, just record the correct function
520		self.function_feedback.record_explicit_feedback(
521		query, correct_function, "unknown", user_correction=True
522		)
523		self.print_success(f"Thank you for your feedback! I'll remember to use {correct_function} for similar queries in the future.")
524		return True
525
526		def chat(self, message: str) -> str:
527		"""
528		Process a chat message, showing the execution steps transparently.
529
530		Args:
531		message: User message
532
533		Returns:
534		str: Response message
535		"""
536		try:
537		# First check if the message is asking for system information
538		has_command_intent, suggested_command = self.detect_command_intent(message)
539		if has_command_intent and suggested_command:
540		self.print_info(f"This appears to be a question about system information. Using '{suggested_command}' command...")
541		try:
542		# Execute the command
543		result = execute_command(
544		command=suggested_command,
545		permission_manager=self.agent.permission_manager
546		)
547
548		# Record this in execution history
549		self.execution_history.append({
550		"function": "execute_command",
551		"args": {"command": suggested_command},
552		"timestamp": time.time(),
553		"success": "error" not in result,
554		"returncode": result.get("returncode"),
555		"query": message
556		})
557
558		# Record implicit feedback
559		self.function_feedback.record_implicit_feedback(
560		message, "execute_command", 1.0, result
561		)
562
563		if "error" in result:
564		self.print_error(f"Command execution failed: {result['error']}")
565		return f"I tried to run the '{suggested_command}' command to answer your question, but encountered an error: {result['error']}"
566		else:
567		# Format the response based on the command
568		formatted_result = format_command_result(suggested_command, result)
569		return formatted_result
570		except Exception as e:
571		self.print_error(f"Command execution failed: {str(e)}")
572		return f"I tried to run the '{suggested_command}' command to answer your question, but encountered an error: {str(e)}"
573
574		# If not a command, check if the message likely requires a search but doesn't explicitly ask for one
575		if not message.lower().startswith('search '):
576		search_indicators = [
577		"what is", "how to", "who is", "where can", "when did", "why does",
578		"recipe for", "information about", "tell me about", "find", "look up",
579		"latest", "news on", "current", "recent"
580		]
581
582		# Check if the message contains search indicators
583		needs_search = any(indicator in message.lower() for indicator in search_indicators)
584
585		# Check for question structure
586		question_structure = message.lower().startswith(("what", "how", "who", "where", "when", "why", "can", "could"))
587
588		if needs_search or question_structure:
589		# Check if message is asking for external information we likely don't have
590		factual_intent = self.detect_factual_intent(message)
591		if factual_intent:
592		self.print_info("This appears to be a question that might benefit from search. Searching automatically...")
593		try:
594		results = self.execute_search(message)
595		# Process search results before continuing with normal chat flow
596		context = self.extract_search_context(results)
597		# Add context to the user message
598		message = f"{message}\n\nSearch context: {context}"
599
600		# Record implicit feedback
601		self.function_feedback.record_implicit_feedback(
602		message, "opensearch_query", 0.9, results
603		)
604		except Exception as e:
605		self.print_error(f"Automatic search failed: {str(e)}")
606		self.print_info("Continuing without search results")
607
608		# Print what we're doing
609		self.print_status(f"Processing message: {message}")
610
611		# Start timer for perf measurement
612		start_time = time.time()
613
614		# Process LLM response
615		try:
616		response = self.agent.chat(
617		message=message,
618		system_prompt=self.system_prompt,
619		context=self.context
620		)
621
622		# Check if the response looks like a JSON function call
623		if response.strip().startswith('{') and ('function' in response or 'name' in response):
624		self.print_info("Detected potential function call in response")
625		try:
626		# Parse the JSON response
627		func_data = json.loads(response)
628
629		# Extract function name and args
630		function_name = None
631		args = {}
632
633		# Handle different JSON formats
634		if 'function' in func_data:
635		function_name = func_data['function'].get('name')
636		args_str = func_data['function'].get('arguments', '{}')
637		if isinstance(args_str, str):
638		args = json.loads(args_str)
639		else:
640		args = args_str
641		elif 'name' in func_data:
642		function_name = func_data.get('name')
643		args = func_data.get('arguments', {})
644
645		if function_name:
646		self.print_status(f"Executing function: {function_name} with args: {args}")
647
648		# Execute the function
649		if function_name == 'opensearch_query' or function_name == 'search_web':
650		results = self.execute_search(args.get('query', message))
651		context = self.extract_search_context(results)
652		response = f"Here's what I found about '{args.get('query', message)}':\n\n{context}"
653		elif hasattr(self, function_name) and callable(getattr(self, function_name)):
654		# Execute a method of this class
655		method = getattr(self, function_name)
656		result = method(**args)
657		response = f"Here are the results for '{function_name}':\n\n{result}"
658		elif function_name in self.agent.registered_functions:
659		# Execute a registered function
660		result = self.agent.registered_functions[function_name](**args)
661		response = f"Here are the results for '{function_name}':\n\n{result}"
662		else:
663		self.print_error(f"Function '{function_name}' not found")
664		response = f"I tried to call a function '{function_name}' but it's not available. I'll try to help you directly instead."
665		except json.JSONDecodeError:
666		self.print_info("Response looked like JSON but couldn't be parsed as a function call")
667		except Exception as e:
668		self.print_error(f"Error executing function from response: {str(e)}")
669		response = f"I tried to execute a function to help with your request, but encountered an error: {str(e)}"
670		except Exception as e:
671		self.print_error(f"Error communicating with Ollama: {str(e)}")
672		response = f"I encountered an error communicating with the language model: {str(e)}"
673
674		# Update context
675		self.context.append({"role": "user", "content": message})
676		self.context.append({"role": "assistant", "content": response})
677
678		# Limit context size
679		if len(self.context) > 10:
680		self.context = self.context[-10:]
681
682		# Calculate execution time
683		execution_time = time.time() - start_time
684		self.print_success(f"Response generated in {execution_time:.2f} seconds")
685
686		return response
687
688		except OllamaToolsError as e:
689		error_details = handle_exception(e, log_error=True)
690
691		self.print_error(f"Error: {error_details['error_message']}")
692		self.suggest_fix(self._generate_error_fix_suggestion(e))
693
694		return f"I encountered an error: {error_details['error_message']}\n\n" + \
695		f"Suggested fix: {self._generate_error_fix_suggestion(e)}"
696
697		except Exception as e:
698		error_details = handle_exception(e, log_error=True)
699
700		self.print_error(f"Unexpected error: {str(e)}")
701
702		return f"I encountered an unexpected error: {str(e)}"

ollamatools-project/ollamatools/agent/modular/commands.py +0 -122

…	…	@@ -1,122 +0,0 @@
1		"""
2		Command management functionality for the ModularOllamaAgent.
3
4		This module provides functions for managing the command allowlist,
5		including listing, adding, and removing commands.
6		"""
7
8		from typing import List, Set
9		from .utils import print_info, print_error, print_success, print_warning
10
11		def list_allowed_commands(command_allowlist: Set[str]) -> List[str]:
12		"""
13		List all currently allowed commands.
14
15		Args:
16		command_allowlist: Set of currently allowed commands
17
18		Returns:
19		List[str]: List of allowed commands
20		"""
21		# Get the current allowlist
22		all_commands = sorted(list(command_allowlist))
23
24		# Print the commands
25		print_info(f"Currently allowed commands ({len(all_commands)}):")
26
27		# Format into categories for better readability
28		categories = {
29		"File Operations": ["ls", "cat", "find", "grep", "head", "tail"],
30		"Directory Operations": ["pwd", "cd", "mkdir", "rmdir"],
31		"System Information": ["uname", "hostname", "whoami", "ps", "top", "df", "du", "free"],
32		"Network Utilities": ["ping", "curl", "wget", "netstat", "dig", "nslookup"],
33		"Text Processing": ["echo", "printf", "sed", "awk", "cut", "sort", "uniq", "wc"],
34		"File Manipulation": ["cp", "mv", "rm", "touch", "chmod", "chown", "chgrp", "ln"],
35		"Other": []
36		}
37
38		# Categorize commands
39		for cmd in all_commands:
40		categorized = False
41		for category, commands in categories.items():
42		if cmd in commands:
43		categorized = True
44		break
45		if not categorized:
46		categories["Other"].append(cmd)
47
48		# Print by category
49		for category, commands in categories.items():
50		if commands:
51		print(f"\n{category}:")
52		for i, cmd in enumerate(sorted(commands)):
53		print(f" {cmd}", end="\t")
54		# New line every 5 commands for readability
55		if (i + 1) % 5 == 0:
56		print()
57		print() # Ensure we end with a newline
58
59		return all_commands
60
61		def add_allowed_command(command: str, command_allowlist: Set[str]) -> bool:
62		"""
63		Add a command to the allowed list.
64
65		Args:
66		command: Command to add
67		command_allowlist: Set of currently allowed commands
68
69		Returns:
70		bool: True if added successfully
71		"""
72		if not command or not isinstance(command, str):
73		print_error("Invalid command name")
74		return False
75
76		# Clean up the command name
77		command = command.strip().lower()
78
79		# Check if already in allowlist
80		if command in command_allowlist:
81		print_info(f"Command '{command}' is already allowed")
82		return True
83
84		# Add to custom allowlist
85		command_allowlist.add(command)
86		print_success(f"Command '{command}' added to allowed list")
87
88		# Show warning for potentially dangerous commands
89		dangerous_commands = {"rm", "mv", "chmod", "chown", "dd", "mkfs", "iptables"}
90		if command in dangerous_commands:
91		print_warning(f"'{command}' is a potentially dangerous command. Use with caution!")
92
93		return True
94
95		def remove_allowed_command(command: str, command_allowlist: Set[str]) -> bool:
96		"""
97		Remove a command from the allowed list.
98
99		Args:
100		command: Command to remove
101		command_allowlist: Set of currently allowed commands
102
103		Returns:
104		bool: True if removed successfully
105		"""
106		if not command or not isinstance(command, str):
107		print_error("Invalid command name")
108		return False
109
110		# Clean up the command name
111		command = command.strip().lower()
112
113		# Check if in allowlist
114		if command not in command_allowlist:
115		print_error(f"Command '{command}' is not in the allowed list")
116		return False
117
118		# Remove from custom allowlist
119		command_allowlist.remove(command)
120		print_success(f"Command '{command}' removed from allowed list")
121
122		return True

ollamatools-project/ollamatools/agent/modular/feedback.py +0 -586

…	…	@@ -1,586 +0,0 @@
1		"""
2		Feedback system for the ModularOllamaAgent.
3
4		This module provides a feedback loop system that improves the agent's
5		semantic understanding of user queries by learning from past interactions.
6		It helps map natural language queries to appropriate functions and commands.
7		"""
8
9		import os
10		import re
11		import json
12		import time
13		import logging
14		from typing import Dict, Any, List, Optional, Tuple, Set, Union
15
16		# Set up logging
17		logger = logging.getLogger(__name__)
18
19		class FunctionSelectionFeedback:
20		"""Manages feedback on function selection for continuous improvement."""
21
22		def __init__(self, feedback_file_path=None):
23		"""
24		Initialize the feedback system.
25
26		Args:
27		feedback_file_path: Path to the feedback file (default: ~/.ollamatools/function_selection_feedback.jsonl)
28		"""
29		if feedback_file_path is None:
30		self.feedback_file_path = os.path.join(
31		os.path.expanduser('~'),
32		'.ollamatools',
33		'function_selection_feedback.jsonl'
34		)
35		else:
36		self.feedback_file_path = feedback_file_path
37
38		# Create directory if it doesn't exist
39		os.makedirs(os.path.dirname(self.feedback_file_path), exist_ok=True)
40
41		# Cache of recent feedback for quick lookup
42		self.recent_feedback = []
43		self.max_cache_size = 1000
44
45		# Load existing feedback into cache
46		self._load_recent_feedback()
47
48		def _load_recent_feedback(self):
49		"""Load recent feedback entries into memory cache."""
50		try:
51		if os.path.exists(self.feedback_file_path):
52		with open(self.feedback_file_path, 'r') as f:
53		# Load the most recent entries (up to max_cache_size)
54		lines = f.readlines()
55		for line in lines[-self.max_cache_size:]:
56		try:
57		entry = json.loads(line.strip())
58		self.recent_feedback.append(entry)
59		except json.JSONDecodeError:
60		continue
61		logger.info(f"Loaded {len(self.recent_feedback)} feedback entries from {self.feedback_file_path}")
62		else:
63		logger.info(f"No feedback file found at {self.feedback_file_path}, starting with empty feedback")
64		except Exception as e:
65		logger.error(f"Error loading feedback data: {str(e)}")
66
67		def record_explicit_feedback(self, query, intended_function, actual_function, user_correction=True):
68		"""
69		Record explicit feedback when a function selection was incorrect.
70
71		Args:
72		query: The original user query
73		intended_function: The function that should have been used
74		actual_function: The function that was actually used
75		user_correction: Whether this feedback came directly from a user
76
77		Returns:
78		dict: The recorded feedback entry
79		"""
80		feedback_entry = {
81		"query": query,
82		"intended_function": intended_function,
83		"actual_function": actual_function,
84		"timestamp": time.time(),
85		"source": "user_correction" if user_correction else "system_detection",
86		"similarity_score": self._calculate_query_similarity(query)
87		}
88
89		# Add to cache
90		self.recent_feedback.append(feedback_entry)
91		if len(self.recent_feedback) > self.max_cache_size:
92		self.recent_feedback.pop(0)
93
94		# Write to file
95		try:
96		with open(self.feedback_file_path, 'a') as f:
97		f.write(json.dumps(feedback_entry) + '\n')
98		logger.info(f"Recorded explicit feedback for query: {query}")
99		except Exception as e:
100		logger.error(f"Error recording feedback: {str(e)}")
101
102		return feedback_entry
103
104		def record_implicit_feedback(self, query, function, confidence_score, execution_result):
105		"""
106		Record implicit feedback based on execution results.
107
108		Args:
109		query: The original user query
110		function: The function that was used
111		confidence_score: Confidence score for the function selection (0-1)
112		execution_result: Result of executing the function
113
114		Returns:
115		dict: The recorded feedback entry
116		"""
117		# Determine if the execution was likely successful
118		success_indicators = {
119		"execute_command": lambda r: r.get("returncode", 1) == 0 and not r.get("stderr"),
120		"read_file": lambda r: not (isinstance(r, str) and (r.startswith("Error") or r.startswith("Path validation error"))),
121		"search_web": lambda r: len(r) > 0 if isinstance(r, list) else True
122		}
123
124		likely_success = False
125		if function in success_indicators:
126		likely_success = success_indicators[function](execution_result)
127
128		feedback_entry = {
129		"query": query,
130		"function": function,
131		"timestamp": time.time(),
132		"source": "implicit_feedback",
133		"confidence_score": confidence_score,
134		"likely_success": likely_success,
135		"similarity_score": self._calculate_query_similarity(query)
136		}
137
138		# Only record if confidence was low or execution likely failed
139		if confidence_score < 0.7 or not likely_success:
140		# Add to cache
141		self.recent_feedback.append(feedback_entry)
142		if len(self.recent_feedback) > self.max_cache_size:
143		self.recent_feedback.pop(0)
144
145		# Write to file
146		try:
147		with open(self.feedback_file_path, 'a') as f:
148		f.write(json.dumps(feedback_entry) + '\n')
149		logger.info(f"Recorded implicit feedback for query: {query}")
150		except Exception as e:
151		logger.error(f"Error recording feedback: {str(e)}")
152
153		return feedback_entry
154
155		def _calculate_query_similarity(self, query):
156		"""
157		Calculate similarity between this query and previous queries.
158
159		Args:
160		query: The query to compare
161
162		Returns:
163		list: List of similar queries with their similarity scores
164		"""
165		# Simple word-based similarity for now
166		query_words = set(query.lower().split())
167
168		similarities = []
169		for entry in self.recent_feedback:
170		past_query = entry.get("query", "")
171		past_words = set(past_query.lower().split())
172
173		# Calculate Jaccard similarity
174		if not query_words or not past_words:
175		similarity = 0
176		else:
177		intersection = len(query_words.intersection(past_words))
178		union = len(query_words.union(past_words))
179		similarity = intersection / union if union > 0 else 0
180
181		if similarity > 0.3: # Only include reasonably similar queries
182		similarities.append({
183		"query": past_query,
184		"similarity": similarity,
185		"intended_function": entry.get("intended_function") or entry.get("function"),
186		"timestamp": entry.get("timestamp")
187		})
188
189		# Sort by similarity score
190		similarities.sort(key=lambda x: x["similarity"], reverse=True)
191		return similarities[:5] # Return top 5 similar queries
192
193		def get_function_suggestion(self, query):
194		"""
195		Get function suggestion based on feedback history.
196
197		Args:
198		query: The user query
199
200		Returns:
201		tuple: (suggested_function, confidence_score, similar_queries)
202		"""
203		# Get similar queries
204		similar_queries = self._calculate_query_similarity(query)
205
206		if not similar_queries:
207		return None, 0, []
208
209		# Count function occurrences weighted by similarity
210		function_scores = {}
211		for item in similar_queries:
212		function = item["intended_function"]
213		similarity = item["similarity"]
214
215		if function not in function_scores:
216		function_scores[function] = 0
217
218		function_scores[function] += similarity
219
220		# Find the function with the highest score
221		best_function = None
222		highest_score = 0
223
224		for function, score in function_scores.items():
225		if score > highest_score:
226		highest_score = score
227		best_function = function
228
229		# Calculate confidence based on the highest score and number of similar queries
230		confidence = highest_score / len(similar_queries) if similar_queries else 0
231
232		return best_function, confidence, similar_queries
233
234		def generate_improved_patterns(self):
235		"""
236		Generate improved regex patterns based on feedback data.
237
238		Returns:
239		dict: Dictionary mapping regex patterns to functions
240		"""
241		# Analyze the feedback data
242		analysis = self.analyze_function_selection_patterns()
243
244		if "error" in analysis:
245		return {}
246
247		improved_patterns = {}
248
249		# Process mismatches to generate patterns
250		for mismatch_info in analysis.get("top_mismatches", []):
251		mismatch, data = mismatch_info
252		actual, intended = mismatch.split(" → ")
253
254		# Skip if we don't have enough examples
255		if len(data["examples"]) < 2:
256		continue
257
258		# Generate a pattern from the examples
259		pattern = self._generate_regex_from_examples(data["examples"], intended)
260		if pattern:
261		improved_patterns[pattern] = intended
262
263		# Process query patterns
264		for pattern_info in analysis.get("top_query_patterns", []):
265		pattern_text, data = pattern_info
266		functions = data["functions"]
267
268		if not functions:
269		continue
270
271		# Find the most common function
272		most_common = max(functions.items(), key=lambda x: x[1])
273		function_name, count = most_common
274
275		# Only include if it's a strong pattern (used for the same function >70% of the time)
276		confidence = count / data["count"]
277		if confidence > 0.7:
278		# Create a regex pattern from the text pattern
279		regex_pattern = r"\b" + re.escape(pattern_text) + r".*"
280		improved_patterns[regex_pattern] = function_name
281
282		return improved_patterns
283
284		def _generate_regex_from_examples(self, examples, function):
285		"""
286		Generate a regex pattern from example queries.
287
288		Args:
289		examples: List of example queries
290		function: The target function
291
292		Returns:
293		str: Regex pattern
294		"""
295		if not examples:
296		return None
297
298		# Extract common words across examples
299		common_words = set(examples[0].lower().split())
300		for example in examples[1:]:
301		common_words &= set(example.lower().split())
302
303		# Remove very common words
304		stop_words = {"what", "is", "the", "a", "an", "in", "on", "at", "to", "for", "of", "with", "by"}
305		common_words -= stop_words
306
307		if not common_words:
308		# If no common words, try to find common starting words
309		starts_with = []
310		for example in examples:
311		words = example.lower().split()
312		if words:
313		starts_with.append(words[0])
314
315		# If we have consistent starting words
316		if len(set(starts_with)) == 1 and starts_with[0] not in stop_words:
317		return r"\b" + re.escape(starts_with[0]) + r".*"
318
319		return None
320
321		# Create a pattern with the common words
322		pattern_parts = []
323		for word in common_words:
324		pattern_parts.append(r"\b" + re.escape(word) + r"\b")
325
326		# Combine with lookaheads for flexibility in word order
327		pattern = "".join(f"(?=.*{part})" for part in pattern_parts) + ".*"
328		return pattern
329
330		def analyze_function_selection_patterns(self):
331		"""
332		Analyze patterns in function selection feedback.
333
334		Returns:
335		dict: Analysis results
336		"""
337		try:
338		# Load all feedback data
339		all_feedback = []
340		if os.path.exists(self.feedback_file_path):
341		with open(self.feedback_file_path, 'r') as f:
342		for line in f:
343		try:
344		entry = json.loads(line.strip())
345		all_feedback.append(entry)
346		except json.JSONDecodeError:
347		continue
348
349		if not all_feedback:
350		return {"error": "No feedback data available"}
351
352		# Analyze common mismatches
353		mismatches = {}
354		for entry in all_feedback:
355		if "intended_function" in entry and "actual_function" in entry:
356		if entry["intended_function"] != entry["actual_function"]:
357		key = f"{entry['actual_function']} → {entry['intended_function']}"
358		if key not in mismatches:
359		mismatches[key] = {
360		"count": 0,
361		"examples": []
362		}
363
364		mismatches[key]["count"] += 1
365		if len(mismatches[key]["examples"]) < 5: # Limit examples
366		mismatches[key]["examples"].append(entry["query"])
367
368		# Sort mismatches by frequency
369		sorted_mismatches = sorted(
370		[(k, v) for k, v in mismatches.items()],
371		key=lambda x: x[1]["count"],
372		reverse=True
373		)
374
375		# Analyze query patterns
376		query_patterns = {}
377		for entry in all_feedback:
378		query = entry.get("query", "")
379		words = query.lower().split()
380
381		# Extract potential patterns (first 2-3 words)
382		if len(words) >= 2:
383		pattern = " ".join(words[:2])
384		if pattern not in query_patterns:
385		query_patterns[pattern] = {
386		"count": 0,
387		"functions": {}
388		}
389
390		query_patterns[pattern]["count"] += 1
391
392		function = entry.get("intended_function") or entry.get("function")
393		if function:
394		if function not in query_patterns[pattern]["functions"]:
395		query_patterns[pattern]["functions"][function] = 0
396		query_patterns[pattern]["functions"][function] += 1
397
398		# Sort patterns by frequency
399		sorted_patterns = sorted(
400		[(k, v) for k, v in query_patterns.items()],
401		key=lambda x: x[1]["count"],
402		reverse=True
403		)
404
405		return {
406		"total_feedback_entries": len(all_feedback),
407		"top_mismatches": sorted_mismatches[:10],
408		"top_query_patterns": sorted_patterns[:10],
409		"improvement_suggestions": self._generate_improvement_suggestions(sorted_mismatches, sorted_patterns)
410		}
411		except Exception as e:
412		logger.error(f"Error analyzing feedback: {str(e)}")
413		return {"error": f"Error analyzing feedback: {str(e)}"}
414
415		def _generate_improvement_suggestions(self, mismatches, patterns):
416		"""
417		Generate suggestions for improving function selection.
418
419		Args:
420		mismatches: List of function mismatches
421		patterns: List of query patterns
422
423		Returns:
424		list: Improvement suggestions
425		"""
426		suggestions = []
427
428		# Suggest improvements based on common mismatches
429		if mismatches:
430		for mismatch, data in mismatches[:5]:
431		actual, intended = mismatch.split(" → ")
432		examples = data["examples"]
433
434		if examples:
435		pattern_words = set()
436		for example in examples:
437		pattern_words.update(example.lower().split())
438
439		# Filter out common words
440		common_words = {"what", "is", "the", "a", "an", "in", "on", "at", "to", "for", "of", "with", "by"}
441		pattern_words = pattern_words - common_words
442
443		if pattern_words:
444		suggestion = {
445		"type": "function_mapping",
446		"description": f"Improve mapping from '{actual}' to '{intended}'",
447		"examples": examples[:3],
448		"suggested_keywords": list(pattern_words)[:5],
449		"implementation_hint": f"Add pattern matching for these keywords to select '{intended}'"
450		}
451		suggestions.append(suggestion)
452
453		# Suggest improvements based on query patterns
454		if patterns:
455		for pattern, data in patterns[:5]:
456		functions = data["functions"]
457		if functions:
458		# Find the most common function for this pattern
459		most_common = max(functions.items(), key=lambda x: x[1])
460		function_name, count = most_common
461
462		suggestion = {
463		"type": "query_pattern",
464		"description": f"Add explicit handling for queries starting with '{pattern}'",
465		"suggested_function": function_name,
466		"confidence": count / data["count"],
467		"implementation_hint": f"Add pattern matching for queries starting with '{pattern}' to select '{function_name}'"
468		}
469		suggestions.append(suggestion)
470
471		return suggestions
472
473		def detect_command_intent(message, feedback_system=None):
474		"""
475		Detect if a message is asking for system information that requires a command.
476
477		Args:
478		message: User message
479		feedback_system: Optional FunctionSelectionFeedback instance
480
481		Returns:
482		Tuple[bool, Optional[str]]: (has_command_intent, suggested_command)
483		"""
484		# First check feedback-based patterns if available
485		if feedback_system:
486		suggested_function, confidence, similar_queries = feedback_system.get_function_suggestion(message)
487
488		if suggested_function == "execute_command" and confidence > 0.7:
489		# Extract command from similar queries
490		command = extract_command_from_similar_queries(message, similar_queries)
491		if command:
492		return True, command
493
494		# Map common questions to shell commands
495		command_patterns = {
496		r"(?:what|which)(?:\s+is)?(?:\s+the)?(?:\s+current)?\s+directory": "pwd",
497		r"where\s+am\s+i": "pwd",
498		r"list\s+(?:the\s+)?(?:files|contents)": "ls",
499		r"show\s+(?:the\s+)?(?:files|contents)": "ls",
500		r"(?:what|which)(?:\s+is)?(?:\s+the)?(?:\s+current)?\s+user": "whoami",
501		r"who\s+am\s+i": "whoami",
502		r"(?:what|which)(?:\s+is)?(?:\s+the)?(?:\s+current)?\s+date": "date",
503		r"(?:what|which)(?:\s+is)?(?:\s+the)?(?:\s+current)?\s+time": "date",
504		# Add more patterns as needed
505		}
506
507		# Check if message matches any pattern
508		for pattern, command in command_patterns.items():
509		if re.search(pattern, message.lower()):
510		return True, command
511
512		return False, None
513
514		def extract_command_from_similar_queries(message, similar_queries):
515		"""
516		Extract the most appropriate command from similar queries.
517
518		Args:
519		message: Current user message
520		similar_queries: List of similar queries with their metadata
521
522		Returns:
523		str: Extracted command or None
524		"""
525		# Count command occurrences in similar queries
526		command_counts = {}
527
528		for query_info in similar_queries:
529		# Skip if not execute_command
530		if query_info.get("intended_function") != "execute_command":
531		continue
532
533		# Try to extract the command from the query
534		query = query_info.get("query", "")
535
536		# Check for common command patterns
537		command_mapping = {
538		r"directory|where\s+am\s+i": "pwd",
539		r"list\s+files|show\s+files|contents": "ls",
540		r"user|who\s+am\s+i": "whoami",
541		r"date|time": "date"
542		}
543
544		for pattern, cmd in command_mapping.items():
545		if re.search(pattern, query.lower()):
546		if cmd not in command_counts:
547		command_counts[cmd] = 0
548		command_counts[cmd] += query_info.get("similarity", 0.5)
549
550		# Return the most common command
551		if command_counts:
552		return max(command_counts.items(), key=lambda x: x[1])[0]
553
554		return None
555
556		def format_command_result(command, result):
557		"""
558		Format the result of a command execution in a user-friendly way.
559
560		Args:
561		command: The command that was executed
562		result: The command execution result
563
564		Returns:
565		str: Formatted result
566		"""
567		if "error" in result:
568		return f"Error executing command: {result['error']}"
569
570		stdout = result.get("stdout", "").strip()
571		stderr = result.get("stderr", "").strip()
572
573		if command == "pwd":
574		return f"You are currently in: {stdout}"
575		elif command == "ls":
576		return f"Files in the current directory:\n{stdout}"
577		elif command == "whoami":
578		return f"Current user: {stdout}"
579		elif command == "date":
580		return f"Current date and time: {stdout}"
581		else:
582		# Generic formatting
583		response = f"Command result:\n{stdout}"
584		if stderr:
585		response += f"\nErrors/Warnings:\n{stderr}"
586		return response

ollamatools-project/ollamatools/agent/modular/issues.py +0 -189

…	…	@@ -1,189 +0,0 @@
1		"""
2		Issue tracking functionality for the ModularOllamaAgent.
3
4		This module provides functions for managing issues, including
5		loading, saving, adding, listing, and updating issues.
6		"""
7
8		import os
9		import json
10		import time
11		from typing import Dict, Any, List, Optional
12
13		from .utils import print_info, print_error, print_success
14
15		def load_issues(agent) -> None:
16		"""
17		Load saved issues from file.
18
19		Args:
20		agent: The agent instance
21		"""
22		if os.path.exists(agent.issues_file):
23		try:
24		with open(agent.issues_file, 'r') as f:
25		data = json.load(f)
26		agent.issues = data.get('issues', [])
27		agent.issue_id_counter = data.get('next_id', 1)
28		agent.print_info(f"Loaded {len(agent.issues)} issues from {agent.issues_file}")
29		except Exception as e:
30		agent.print_error(f"Failed to load issues: {str(e)}")
31		agent.issues = []
32		agent.issue_id_counter = 1
33
34		def save_issues(agent) -> None:
35		"""
36		Save issues to file.
37
38		Args:
39		agent: The agent instance
40		"""
41		try:
42		with open(agent.issues_file, 'w') as f:
43		json.dump({
44		'issues': agent.issues,
45		'next_id': agent.issue_id_counter
46		}, f, indent=2)
47		agent.print_info(f"Saved {len(agent.issues)} issues to {agent.issues_file}")
48		except Exception as e:
49		agent.print_error(f"Failed to save issues: {str(e)}")
50
51		def add_issue(agent, title: str, description: str, severity: str = 'medium') -> int:
52		"""
53		Add a new issue to the tracker.
54
55		Args:
56		agent: The agent instance
57		title: Issue title
58		description: Issue description
59		severity: Issue severity (low, medium, high)
60
61		Returns:
62		int: Issue ID
63		"""
64		# Validate severity
65		if severity not in ['low', 'medium', 'high']:
66		severity = 'medium'
67
68		# Create issue
69		issue = {
70		'id': agent.issue_id_counter,
71		'title': title,
72		'description': description,
73		'severity': severity,
74		'status': 'open',
75		'created_at': time.strftime('%Y-%m-%d %H:%M:%S'),
76		'updated_at': time.strftime('%Y-%m-%d %H:%M:%S')
77		}
78
79		# Add to list and increment counter
80		agent.issues.append(issue)
81		agent.issue_id_counter += 1
82
83		# Save issues
84		save_issues(agent)
85
86		agent.print_success(f"Issue #{issue['id']} added: {title}")
87		return issue['id']
88
89		def list_issues(agent, status: Optional[str] = None) -> List[Dict[str, Any]]:
90		"""
91		List all issues, optionally filtered by status.
92
93		Args:
94		agent: The agent instance
95		status: Optional status filter (open, resolved)
96
97		Returns:
98		List[Dict[str, Any]]: List of issues
99		"""
100		# Filter issues if status provided
101		filtered_issues = agent.issues
102		if status:
103		filtered_issues = [i for i in agent.issues if i['status'] == status]
104
105		# Print issues
106		if not filtered_issues:
107		agent.print_info("No issues found")
108		return []
109
110		agent.print_info(f"Issues ({len(filtered_issues)}):")
111		print("\nID | Severity | Status | Title")
112		print("----|----------|----------|-------------------")
113
114		for issue in filtered_issues:
115		print(f"{issue['id']:<4}| {issue['severity']:<9}| {issue['status']:<9}| {issue['title']}")
116
117		return filtered_issues
118
119		def get_issue(agent, issue_id: int) -> Optional[Dict[str, Any]]:
120		"""
121		Get details of a specific issue.
122
123		Args:
124		agent: The agent instance
125		issue_id: Issue ID
126
127		Returns:
128		Optional[Dict[str, Any]]: Issue details or None if not found
129		"""
130		# Find issue
131		for issue in agent.issues:
132		if issue['id'] == issue_id:
133		# Print issue details
134		agent.print_info(f"Issue #{issue['id']}: {issue['title']}")
135		print(f"Status: {issue['status']}")
136		print(f"Severity: {issue['severity']}")
137		print(f"Created: {issue['created_at']}")
138		print(f"Updated: {issue['updated_at']}")
139		print("\nDescription:")
140		print(issue['description'])
141		return issue
142
143		agent.print_error(f"Issue #{issue_id} not found")
144		return None
145
146		def update_issue(
147		agent,
148		issue_id: int,
149		status: Optional[str] = None,
150		severity: Optional[str] = None,
151		description: Optional[str] = None
152		) -> bool:
153		"""
154		Update an existing issue.
155
156		Args:
157		agent: The agent instance
158		issue_id: Issue ID
159		status: New status (open, resolved)
160		severity: New severity (low, medium, high)
161		description: New description
162
163		Returns:
164		bool: True if updated successfully
165		"""
166		# Find issue
167		for i, issue in enumerate(agent.issues):
168		if issue['id'] == issue_id:
169		# Update fields
170		if status and status in ['open', 'resolved']:
171		agent.issues[i]['status'] = status
172
173		if severity and severity in ['low', 'medium', 'high']:
174		agent.issues[i]['severity'] = severity
175
176		if description:
177		agent.issues[i]['description'] = description
178
179		# Update timestamp
180		agent.issues[i]['updated_at'] = time.strftime('%Y-%m-%d %H:%M:%S')
181
182		# Save issues
183		save_issues(agent)
184
185		agent.print_success(f"Issue #{issue_id} updated")
186		return True
187
188		agent.print_error(f"Issue #{issue_id} not found")
189		return False

ollamatools-project/ollamatools/agent/modular/search.py +0 -208

…	…	@@ -1,208 +0,0 @@
1		"""
2		Search functionality for the ModularOllamaAgent.
3
4		This module provides search-related functionality for the ModularOllamaAgent,
5		including natural language detection of search intent, search execution, and
6		search results processing.
7		"""
8
9		import time
10		import logging
11		from typing import Dict, Any, List, Optional, Set
12
13		from ollamatools.tools.web_tools import search_web
14		from ollamatools.errors import WebError
15		from ollamatools.opensearch_web_tools import opensearch_query
16		from .utils import print_status, print_success, print_error, print_warning, print_info
17
18		# Set up logger
19		logger = logging.getLogger(__name__)
20
21		def detect_factual_intent(message: str) -> bool:
22		"""
23		Detect if a message is likely asking for factual information
24		that would benefit from search.
25
26		Args:
27		message: User message
28
29		Returns:
30		bool: True if message likely needs external information
31		"""
32		# Basic topics that often require external information
33		external_topics = [
34		"recipe", "weather", "news", "sports", "movie", "book", "author",
35		"product", "review", "price", "stock", "company", "history",
36		"person", "celebrity", "politician", "event", "scientific",
37		"research", "study", "statistic", "data", "definition"
38		]
39
40		# Check if message contains any of these topics
41		contains_topic = any(topic in message.lower() for topic in external_topics)
42
43		# Check if it's a complex question (longer questions tend to be more specific)
44		is_complex = len(message.split()) > 6
45
46		# Simple heuristic: if it contains a topic of interest and is reasonably complex
47		return contains_topic and is_complex
48
49		def extract_search_context(search_results: Dict[str, Any]) -> str:
50		"""
51		Extract useful context from search results.
52
53		Args:
54		search_results: Results from opensearch_query
55
56		Returns:
57		str: Formatted context for the agent
58		"""
59		context_parts = []
60
61		# Add direct answer if available
62		if search_results.get("answer"):
63		try:
64		# Handle both string and dict answers
65		answer = search_results["answer"]
66		if isinstance(answer, dict):
67		answer_text = answer.get("content", str(answer))
68		else:
69		answer_text = str(answer)
70		context_parts.append(f"Direct answer: {answer_text}")
71		except Exception as e:
72		logger.warning(f"Error extracting answer for context: {str(e)}")
73
74		# Add snippets from results
75		if search_results.get("results"):
76		context_parts.append("Information from search results:")
77		for i, result in enumerate(search_results["results"][:3]): # Limit to 3 results
78		title = result.get("title", "Untitled")
79		content = result.get("content", "No content")
80		url = result.get("url", "")
81		context_parts.append(f"- {title}: {content} (Source: {url})")
82
83		# Join everything with newlines
84		return "\n".join(context_parts)
85
86		def execute_search(
87		query: str,
88		opensearch_config: Dict[str, Any],
89		permission_manager: Any,
90		execution_history: List[Dict[str, Any]]
91		) -> Dict[str, Any]:
92		"""
93		Execute a search using OpenSearchNG or fall back to standard search.
94
95		Args:
96		query: Search query
97		opensearch_config: OpenSearchNG configuration
98		permission_manager: Permission manager instance
99		execution_history: History of executed functions
100
101		Returns:
102		dict: Search results
103		"""
104		try:
105		print_status(f"Searching for: {query}")
106
107		# Try to use OpenSearchNG first
108		try:
109		# Execute the search with OpenSearchNG
110		results = opensearch_query(
111		query=query,
112		config=opensearch_config,
113		permission_manager=permission_manager
114		)
115
116		# Show the number of results
117		num_results = len(results.get("results", []))
118		print_success(f"Found {num_results} results using OpenSearchNG")
119
120		# Show suggested answer if available
121		if results.get("answer"):
122		try:
123		# Handle both string and dict answers
124		answer = results["answer"]
125		if isinstance(answer, dict):
126		answer_text = answer.get("content", str(answer))
127		else:
128		answer_text = str(answer)
129		print_info(f"Direct answer: {answer_text}")
130		except Exception as e:
131		logger.warning(f"Error displaying answer: {str(e)}")
132
133		# Register in execution history
134		execution_history.append({
135		"function": "opensearch_query",
136		"args": {"query": query},
137		"timestamp": time.time(),
138		"success": True,
139		"num_results": num_results
140		})
141
142		return results
143
144		except (WebError, ImportError, TypeError) as e:
145		# If OpenSearchNG fails, fall back to standard web search
146		if "Could not connect to OpenSearchNG server" in str(e) or isinstance(e, ImportError):
147		print_warning("OpenSearchNG is not available. Using fallback search method.")
148
149		# Use the fallback search method from web_tools
150		search_results = search_web(
151		query,
152		num_results=5,
153		permission_manager=permission_manager
154		)
155
156		# Convert results to the expected format
157		fallback_results = {
158		"query": query,
159		"results": [],
160		"suggestion": None,
161		"answer": None,
162		"engines": ["fallback"],
163		"timing": {"total": 0}
164		}
165
166		for result in search_results:
167		fallback_results["results"].append({
168		"title": result.get("title", ""),
169		"url": result.get("url", ""),
170		"content": result.get("snippet", ""),
171		"engine": "fallback"
172		})
173
174		num_results = len(fallback_results["results"])
175		print_success(f"Found {num_results} results using fallback search")
176
177		# Register in execution history
178		execution_history.append({
179		"function": "search_web_fallback",
180		"args": {"query": query},
181		"timestamp": time.time(),
182		"success": True,
183		"num_results": num_results
184		})
185
186		return fallback_results
187		else:
188		# Re-raise other WebError exceptions
189		raise
190
191		except Exception as e:
192		print_error(f"Search failed: {str(e)}")
193
194		# Register in execution history
195		execution_history.append({
196		"function": "execute_search",
197		"args": {"query": query},
198		"timestamp": time.time(),
199		"success": False,
200		"error": str(e)
201		})
202
203		# Return empty results
204		return {
205		"query": query,
206		"results": [],
207		"error": str(e)
208		}

ollamatools-project/ollamatools/agent/modular/utils.py +0 -136

…	…	@@ -1,136 +0,0 @@
1		"""
2		Utility functions for the ModularOllamaAgent.
3
4		This module provides utility functions for displaying status messages,
5		formatting output, and suggesting fixes for errors.
6		"""
7
8		def print_status(message: str) -> None:
9		"""
10		Print a status message.
11
12		Args:
13		message: Status message to print
14		"""
15		print(f"\n🔄 {message}")
16
17		def print_success(message: str) -> None:
18		"""
19		Print a success message.
20
21		Args:
22		message: Success message to print
23		"""
24		print(f"✅ {message}")
25
26		def print_error(message: str) -> None:
27		"""
28		Print an error message.
29
30		Args:
31		message: Error message to print
32		"""
33		print(f"❌ {message}")
34
35		def print_warning(message: str) -> None:
36		"""
37		Print a warning message.
38
39		Args:
40		message: Warning message to print
41		"""
42		print(f"⚠️ {message}")
43
44		def print_info(message: str) -> None:
45		"""
46		Print an information message.
47
48		Args:
49		message: Information message to print
50		"""
51		print(f"ℹ️ {message}")
52
53		def suggest_fix(suggestion: str) -> None:
54		"""
55		Print a fix suggestion.
56
57		Args:
58		suggestion: Fix suggestion to print
59		"""
60		print(f"💡 Suggested fix: {suggestion}")
61
62		def format_code_block(code: str, language: str = "python") -> str:
63		"""
64		Format text as a code block.
65
66		Args:
67		code: Code to format
68		language: Programming language for syntax highlighting
69
70		Returns:
71		str: Formatted code block
72		"""
73		return f"```{language}\n{code}\n```"
74
75		def format_table(headers: list, rows: list) -> str:
76		"""
77		Format data as a table.
78
79		Args:
80		headers: Table headers
81		rows: Table rows
82
83		Returns:
84		str: Formatted table
85		"""
86		# Determine column widths
87		col_widths = [len(h) for h in headers]
88		for row in rows:
89		for i, cell in enumerate(row):
90		col_widths[i] = max(col_widths[i], len(str(cell)))
91
92		# Create the header
93		header = "| " + " | ".join(h.ljust(w) for h, w in zip(headers, col_widths)) + " |"
94		separator = "|-" + "-|-".join("-" * w for w in col_widths) + "-|"
95
96		# Create the rows
97		formatted_rows = []
98		for row in rows:
99		formatted_row = "| " + " | ".join(str(cell).ljust(w) for cell, w in zip(row, col_widths)) + " |"
100		formatted_rows.append(formatted_row)
101
102		# Combine everything
103		return "\n".join([header, separator] + formatted_rows)
104
105		def truncate_text(text: str, max_length: int = 100, suffix: str = "...") -> str:
106		"""
107		Truncate text to a maximum length.
108
109		Args:
110		text: Text to truncate
111		max_length: Maximum length
112		suffix: Suffix to add to truncated text
113
114		Returns:
115		str: Truncated text
116		"""
117		if len(text) <= max_length:
118		return text
119		return text[:max_length - len(suffix)] + suffix
120
121		def format_execution_time(seconds: float) -> str:
122		"""
123		Format execution time in a human-readable format.
124
125		Args:
126		seconds: Execution time in seconds
127
128		Returns:
129		str: Formatted execution time
130		"""
131		if seconds < 0.001:
132		return f"{seconds * 1000000:.2f}μs"
133		elif seconds < 1:
134		return f"{seconds * 1000:.2f}ms"
135		else:
136		return f"{seconds:.2f}s"

ollamatools-project/ollamatools/command_validator.py +0 -326

…	…	@@ -1,326 +0,0 @@
1		"""
2		Command validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing shell commands
5		to prevent command injection and other security vulnerabilities.
6		"""
7
8		import os
9		import re
10		import shlex
11		import logging
12		import pathlib
13		from typing import List, Optional, Dict, Set, Tuple, Union
14
15		# Set up logging
16		logger = logging.getLogger(__name__)
17
18		# Default allowlist of commands
19		DEFAULT_ALLOWLIST = {
20		# File operations
21		'ls', 'dir', 'find', 'grep', 'cat', 'less', 'head', 'tail',
22		# Directory operations
23		'pwd', 'cd', 'mkdir', 'rmdir',
24		# System information
25		'uname', 'hostname', 'whoami', 'id', 'ps', 'top', 'df', 'du', 'free',
26		# Network utilities
27		'ping', 'traceroute', 'curl', 'wget', 'netstat', 'dig', 'nslookup',
28		# Text processing
29		'echo', 'printf', 'sed', 'awk', 'cut', 'sort', 'uniq', 'wc',
30		# File manipulation
31		'cp', 'mv', 'rm', 'touch', 'chmod', 'chown', 'chgrp', 'ln',
32		# Compression
33		'tar', 'gzip', 'gunzip', 'zip', 'unzip',
34		# Package management
35		'apt-get', 'apt', 'yum', 'dnf', 'brew',
36		# Process control
37		'kill', 'pkill', 'pgrep',
38		# Admin
39		'sudo',
40		}
41
42		# Commands that require special validation
43		HIGH_RISK_COMMANDS = {
44		'rm', 'mv', 'cp', # File operations that can cause data loss
45		'chmod', 'chown', 'chgrp', # Permission changes
46		'sudo', # Privilege escalation
47		'wget', 'curl', # Network operations
48		}
49
50		# Paths that should be protected (not allowed to be modified)
51		PROTECTED_PATHS = {
52		'/etc', '/bin', '/sbin', '/usr/bin', '/usr/sbin',
53		'/boot', '/lib', '/lib64', '/usr/lib', '/usr/lib64',
54		'/sys', '/proc', '/dev'
55		}
56
57		# Regular expression patterns for dangerous argument patterns
58		DANGEROUS_ARG_PATTERNS = [
59		r'[|;&`$()]', # Shell metacharacters
60		r'>(>)?[^>]*', # Redirections
61		r'<(<)?[^<]*', # Input redirections
62		r'\$\([^)]*\)', # Command substitution
63		r'`[^`]*`', # Backtick command substitution
64		r'\$\{[^}]*\}', # Variable expansion
65		r'\.\.', # Path traversal
66		r'~[^/]*', # Home directory reference
67		r'\\\\', # Double backslash
68		]
69
70		class CommandValidationError(Exception):
71		"""Exception raised for command validation errors."""
72		pass
73
74
75		def is_command_in_allowlist(command: str, custom_allowlist: Optional[Set[str]] = None) -> bool:
76		"""
77		Check if a command is in the allowlist.
78
79		Args:
80		command: Command to check
81		custom_allowlist: Optional custom allowlist to use instead of the default
82
83		Returns:
84		bool: True if command is in allowlist, False otherwise
85		"""
86		allowlist = custom_allowlist if custom_allowlist is not None else DEFAULT_ALLOWLIST
87
88		# Strip path information for the check
89		cmd_name = os.path.basename(command)
90
91		return cmd_name in allowlist
92
93
94		def validate_command_path(command: str) -> str:
95		"""
96		Validate that a command's path is safe.
97
98		Args:
99		command: Command to validate
100
101		Returns:
102		str: Validated command path
103
104		Raises:
105		CommandValidationError: If the command path is not valid
106		"""
107		# Check for absolute path
108		if os.path.isabs(command):
109		cmd_path = pathlib.Path(command).resolve()
110		cmd_name = os.path.basename(command)
111
112		# Ensure the command exists
113		if not cmd_path.exists():
114		raise CommandValidationError(f"Command '{command}' not found")
115
116		# Ensure the path is to a file not a directory
117		if not cmd_path.is_file():
118		raise CommandValidationError(f"Command path '{command}' is not a file")
119
120		# Check if command in a standard location
121		std_paths = ['/bin', '/usr/bin', '/usr/local/bin', '/sbin', '/usr/sbin']
122		parent_path = str(cmd_path.parent)
123		if not any(parent_path.startswith(p) for p in std_paths):
124		logger.warning(f"Command '{command}' is outside standard paths: {parent_path}")
125
126		# Additional permission check
127		if not os.access(cmd_path, os.X_OK):
128		raise CommandValidationError(f"Command '{command}' is not executable")
129
130		return str(cmd_path)
131		else:
132		# For non-absolute paths, we'll rely on the allowlist check
133		return command
134
135
136		def validate_command_args(args: List[str], command: str) -> List[str]:
137		"""
138		Validate command arguments for security issues.
139
140		Args:
141		args: Command arguments to validate
142		command: The command these args apply to (for context-specific checks)
143
144		Returns:
145		List[str]: Validated arguments
146
147		Raises:
148		CommandValidationError: If any arguments are not valid
149		"""
150		validated_args = []
151
152		# Check if command is high risk
153		is_high_risk = os.path.basename(command) in HIGH_RISK_COMMANDS
154
155		for arg in args:
156		# Skip empty args
157		if not arg:
158		continue
159
160		# Check for dangerous patterns
161		for pattern in DANGEROUS_ARG_PATTERNS:
162		if re.search(pattern, arg):
163		raise CommandValidationError(
164		f"Argument '{arg}' contains dangerous pattern matching '{pattern}'"
165		)
166
167		# Special checks for specific commands
168		cmd_name = os.path.basename(command)
169
170		# Special checks for rm command
171		if cmd_name == 'rm' and (arg == '-rf' or arg == '-fr' or arg == '-r' or arg == '-f'):
172		# These arguments need special attention with protected paths
173		if '-r' in arg or '-f' in arg:
174		logger.warning(f"High-risk argument '{arg}' detected for '{command}'")
175
176		# Check file paths in arguments for protected paths
177		if is_high_risk and arg.startswith('/'):
178		arg_path = pathlib.Path(arg).resolve()
179		arg_path_str = str(arg_path)
180
181		# Check against protected paths
182		for protected in PROTECTED_PATHS:
183		if arg_path_str.startswith(protected):
184		raise CommandValidationError(
185		f"Argument '{arg}' references protected path '{protected}'"
186		)
187
188		validated_args.append(arg)
189
190		return validated_args
191
192
193		def sanitize_command(command: str) -> str:
194		"""
195		Sanitize a command string to prevent command injection.
196
197		Args:
198		command: Command string to sanitize
199
200		Returns:
201		str: Sanitized command string
202		"""
203		# Convert to string and trim
204		if not isinstance(command, str):
205		command = str(command)
206
207		sanitized = command.strip()
208
209		# Remove any null bytes or control characters
210		sanitized = re.sub(r'[\x00-\x1F\x7F]', '', sanitized)
211
212		# Remove shell metacharacters
213		dangerous_chars = r'[;&|`$()"\'\\]'
214		sanitized = re.sub(dangerous_chars, '', sanitized)
215
216		return sanitized
217
218
219		def validate_and_sanitize_command(
220		command: str,
221		args: Optional[List[str]] = None,
222		custom_allowlist: Optional[Set[str]] = None
223		) -> Tuple[str, List[str]]:
224		"""
225		Validate and sanitize a command and its arguments.
226
227		Args:
228		command: Command to validate and sanitize
229		args: Optional command arguments
230		custom_allowlist: Optional custom allowlist
231
232		Returns:
233		tuple: (sanitized_command, sanitized_args)
234
235		Raises:
236		CommandValidationError: If the command or arguments are not valid
237		"""
238		# Handle None args
239		if args is None:
240		args = []
241
242		# Handle case where command contains both command and arguments as a string
243		if isinstance(command, str) and ' ' in command and args is None:
244		try:
245		# Split using shlex to handle quotes properly
246		command_parts = shlex.split(command)
247		command = command_parts[0]
248		args = command_parts[1:] if len(command_parts) > 1 else []
249		logger.debug(f"Split command into cmd='{command}', args={args}")
250		except ValueError as e:
251		raise CommandValidationError(f"Invalid command syntax: {str(e)}")
252
253		# Check if command is in allowlist
254		if not is_command_in_allowlist(command, custom_allowlist):
255		raise CommandValidationError(
256		f"Command '{command}' not in allowed command list"
257		)
258
259		# Validate command path
260		validated_command = validate_command_path(command)
261
262		# Validate and sanitize args
263		validated_args = validate_command_args(args, validated_command)
264
265		return validated_command, validated_args
266
267
268		def is_sudo_command(command: str, args: Optional[List[str]] = None) -> bool:
269		"""
270		Check if a command uses sudo.
271
272		Args:
273		command: Command to check
274		args: Optional command arguments
275
276		Returns:
277		bool: True if the command uses sudo, False otherwise
278		"""
279		if args is None:
280		args = []
281
282		# Check if command is sudo directly
283		if os.path.basename(command) == 'sudo':
284		return True
285
286		# Check if command path contains sudo
287		if 'sudo' in command.split(os.path.sep):
288		return True
289
290		# Check if sudo appears in the full command string
291		full_command = f"{command} {' '.join(args)}"
292		if full_command.lstrip().startswith('sudo '):
293		return True
294
295		return False
296
297
298		def get_secure_env() -> Dict[str, str]:
299		"""
300		Get a sanitized environment dictionary for command execution.
301
302		Returns:
303		dict: Sanitized environment variables
304		"""
305		# Start with a minimal environment
306		secure_env = {
307		'PATH': '/usr/local/bin:/usr/bin:/bin',
308		'LANG': os.environ.get('LANG', 'C.UTF-8'),
309		'LC_ALL': os.environ.get('LC_ALL', 'C.UTF-8'),
310		'TERM': os.environ.get('TERM', 'xterm'),
311		'HOME': os.environ.get('HOME', '/tmp'),
312		'USER': os.environ.get('USER', 'nobody'),
313		}
314
315		# Copy allowed environment variables
316		allowed_vars = {
317		'DISPLAY', 'EDITOR', 'PAGER', 'TZ',
318		'HTTP_PROXY', 'HTTPS_PROXY', 'NO_PROXY',
319		'http_proxy', 'https_proxy', 'no_proxy'
320		}
321
322		for var in allowed_vars:
323		if var in os.environ:
324		secure_env[var] = os.environ[var]
325
326		return secure_env

ollamatools-project/ollamatools/config.py +0 -107

…	…	@@ -1,107 +0,0 @@
1		"""
2		Configuration management for OllamaTools.
3
4		This module contains the Config class, which handles configuration settings for
5		the OllamaTools library.
6		"""
7
8		import json
9		import os
10		from typing import Optional, Dict, Any
11
12		from pydantic import BaseModel, validator, Field
13
14
15		class Config(BaseModel):
16		"""
17		Configuration class for OllamaTools.
18
19		This class manages configuration settings for the OllamaTools library using
20		Pydantic for validation. It provides methods for loading and saving configuration
21		to a file.
22		"""
23
24		model: str = "llama3.1"
25		host: str = "http://localhost:11434"
26		permission_prompts: bool = True
27		sudo_always_prompt: bool = True
28		timeout_seconds: int = 60
29		max_file_size: Optional[int] = None
30
31		@validator('timeout_seconds')
32		def validate_timeout(cls, v):
33		"""Validate that timeout is positive."""
34		if v <= 0:
35		raise ValueError("Timeout must be positive")
36		return v
37
38		@validator('max_file_size')
39		def validate_max_file_size(cls, v):
40		"""Validate that max_file_size is positive if provided."""
41		if v is not None and v <= 0:
42		raise ValueError("Max file size must be positive")
43		return v
44
45		@classmethod
46		def load_from_file(cls, file_path: str) -> "Config":
47		"""
48		Load configuration from a JSON file.
49
50		Args:
51		file_path: Path to the configuration file
52
53		Returns:
54		Config: Configuration object
55
56		Raises:
57		FileNotFoundError: If the file does not exist
58		json.JSONDecodeError: If the file contains invalid JSON
59		ValueError: If the configuration is invalid
60		"""
61		if not os.path.exists(file_path):
62		raise FileNotFoundError(f"Configuration file not found: {file_path}")
63
64		with open(file_path, 'r') as f:
65		try:
66		config_data = json.load(f)
67		return cls(**config_data)
68		except json.JSONDecodeError:
69		raise json.JSONDecodeError(f"Invalid JSON in configuration file: {file_path}", "", 0)
70
71		def save_to_file(self, file_path: str) -> None:
72		"""
73		Save configuration to a JSON file.
74
75		Args:
76		file_path: Path to the configuration file
77
78		Raises:
79		PermissionError: If the file cannot be written
80		"""
81		try:
82		# Create directory if it doesn't exist
83		os.makedirs(os.path.dirname(os.path.abspath(file_path)), exist_ok=True)
84
85		with open(file_path, 'w') as f:
86		json.dump(self.dict(), f, indent=2)
87		except PermissionError:
88		raise PermissionError(f"Cannot write to configuration file: {file_path}")
89
90		def to_dict(self) -> Dict[str, Any]:
91		"""
92		Convert configuration to a dictionary.
93
94		Returns:
95		Dict[str, Any]: Configuration as a dictionary
96		"""
97		return self.dict(exclude_none=True)
98
99		@classmethod
100		def default_config_path(cls) -> str:
101		"""
102		Get the default configuration file path.
103
104		Returns:
105		str: Default configuration file path
106		"""
107		return os.path.expanduser("~/.config/ollamatools/config.json")

ollamatools-project/ollamatools/config/__init__.py +0 -40

…	…	@@ -1,40 +0,0 @@
1		"""
2		OllamaTools Configuration System.
3
4		This package provides configuration management for OllamaTools and its plugins.
5		"""
6
7		import os
8		import logging
9		from typing import Dict, Any, Optional
10
11		# Configure logging
12		logger = logging.getLogger(__name__)
13
14		# Default configuration directory
15		DEFAULT_CONFIG_DIR = os.path.join(
16		os.path.expanduser('~'),
17		'.ollamatools'
18		)
19
20		# Create default configuration directory if it doesn't exist
21		os.makedirs(DEFAULT_CONFIG_DIR, exist_ok=True)
22		os.makedirs(os.path.join(DEFAULT_CONFIG_DIR, 'plugins'), exist_ok=True)
23
24		# Import the Config class from core_config.py
25		from ollamatools.config.core_config import Config
26
27		# Import key components
28		from ollamatools.config.plugin_config import PluginConfig, load_plugin_config, create_default_config
29		from ollamatools.config.schemas import get_schema_for_plugin, SCHEMA_REGISTRY
30
31		# Export all components
32		__all__ = [
33		'Config',
34		'PluginConfig',
35		'load_plugin_config',
36		'create_default_config',
37		'get_schema_for_plugin',
38		'SCHEMA_REGISTRY',
39		'DEFAULT_CONFIG_DIR'
40		]

ollamatools-project/ollamatools/config/core_config.py +0 -107

…	…	@@ -1,107 +0,0 @@
1		"""
2		Core configuration management for OllamaTools.
3
4		This module contains the Config class, which handles configuration settings for
5		the OllamaTools library.
6		"""
7
8		import json
9		import os
10		from typing import Optional, Dict, Any
11
12		from pydantic import BaseModel, validator, Field
13
14
15		class Config(BaseModel):
16		"""
17		Configuration class for OllamaTools.
18
19		This class manages configuration settings for the OllamaTools library using
20		Pydantic for validation. It provides methods for loading and saving configuration
21		to a file.
22		"""
23
24		model: str = "llama3.1"
25		host: str = "http://localhost:11434"
26		permission_prompts: bool = True
27		sudo_always_prompt: bool = True
28		timeout_seconds: int = 60
29		max_file_size: Optional[int] = None
30
31		@validator('timeout_seconds')
32		def validate_timeout(cls, v):
33		"""Validate that timeout is positive."""
34		if v <= 0:
35		raise ValueError("Timeout must be positive")
36		return v
37
38		@validator('max_file_size')
39		def validate_max_file_size(cls, v):
40		"""Validate that max_file_size is positive if provided."""
41		if v is not None and v <= 0:
42		raise ValueError("Max file size must be positive")
43		return v
44
45		@classmethod
46		def load_from_file(cls, file_path: str) -> "Config":
47		"""
48		Load configuration from a JSON file.
49
50		Args:
51		file_path: Path to the configuration file
52
53		Returns:
54		Config: Configuration object
55
56		Raises:
57		FileNotFoundError: If the file does not exist
58		json.JSONDecodeError: If the file contains invalid JSON
59		ValueError: If the configuration is invalid
60		"""
61		if not os.path.exists(file_path):
62		raise FileNotFoundError(f"Configuration file not found: {file_path}")
63
64		with open(file_path, 'r') as f:
65		try:
66		config_data = json.load(f)
67		return cls(**config_data)
68		except json.JSONDecodeError:
69		raise json.JSONDecodeError(f"Invalid JSON in configuration file: {file_path}", "", 0)
70
71		def save_to_file(self, file_path: str) -> None:
72		"""
73		Save configuration to a JSON file.
74
75		Args:
76		file_path: Path to the configuration file
77
78		Raises:
79		PermissionError: If the file cannot be written
80		"""
81		try:
82		# Create directory if it doesn't exist
83		os.makedirs(os.path.dirname(os.path.abspath(file_path)), exist_ok=True)
84
85		with open(file_path, 'w') as f:
86		json.dump(self.dict(), f, indent=2)
87		except PermissionError:
88		raise PermissionError(f"Cannot write to configuration file: {file_path}")
89
90		def to_dict(self) -> Dict[str, Any]:
91		"""
92		Convert configuration to a dictionary.
93
94		Returns:
95		Dict[str, Any]: Configuration as a dictionary
96		"""
97		return self.dict(exclude_none=True)
98
99		@classmethod
100		def default_config_path(cls) -> str:
101		"""
102		Get the default configuration file path.
103
104		Returns:
105		str: Default configuration file path
106		"""
107		return os.path.expanduser("~/.config/ollamatools/config.json")

ollamatools-project/ollamatools/config/plugin_config.py +0 -344

…	…	@@ -1,344 +0,0 @@
1		"""
2		Configuration system for the OllamaTools plugin architecture.
3
4		This module provides classes and functions for managing plugin configuration.
5		"""
6
7		import os
8		import sys
9		import yaml
10		import json
11		import logging
12		from typing import Dict, Any, List, Optional, Set, Union, Callable
13
14		from ollamatools.plugins.errors import PluginConfigurationError
15
16		# Configure logging
17		logger = logging.getLogger(__name__)
18
19		class PluginConfig:
20		"""Configuration manager for plugins."""
21
22		def __init__(self, config=None, schema=None):
23		"""
24		Initialize the configuration manager.
25
26		Args:
27		config: Initial configuration dictionary or path to config file
28		schema: Optional schema for validation
29		"""
30		self.schema = schema or {}
31		self.config = {}
32
33		if config:
34		self.load_config(config)
35
36		def load_config(self, config):
37		"""
38		Load configuration from a dictionary, JSON, or YAML file.
39
40		Args:
41		config: Configuration source (dict, path to JSON/YAML file)
42
43		Returns:
44		PluginConfig: Self for method chaining
45
46		Raises:
47		PluginConfigurationError: If configuration loading fails
48		"""
49		try:
50		if isinstance(config, dict):
51		# Load from dictionary
52		self.config = config
53		elif isinstance(config, str) and os.path.exists(config):
54		# Load from file
55		with open(config, 'r') as f:
56		if config.endswith('.json'):
57		self.config = json.load(f)
58		elif config.endswith(('.yaml', '.yml')):
59		self.config = yaml.safe_load(f)
60		else:
61		raise PluginConfigurationError(
62		f"Unsupported configuration file format: {config}",
63		details={"file": config}
64		)
65		else:
66		raise PluginConfigurationError(
67		f"Invalid configuration source: {config}",
68		details={"source": str(config)}
69		)
70
71		# Validate the configuration
72		if self.schema:
73		self.validate()
74
75		return self
76
77		except PluginConfigurationError:
78		# Re-raise plugin configuration errors
79		raise
80		except Exception as e:
81		# Wrap other exceptions
82		raise PluginConfigurationError(
83		f"Failed to load configuration: {str(e)}",
84		details={"source": str(config)}
85		) from e
86
87		def validate(self):
88		"""
89		Validate the configuration against the schema.
90
91		Returns:
92		bool: True if valid
93
94		Raises:
95		PluginConfigurationError: If validation fails
96		"""
97		# Simple schema validation
98		for key, schema_item in self.schema.items():
99		# Check required fields
100		if schema_item.get('required', False) and key not in self.config:
101		raise PluginConfigurationError(
102		f"Missing required configuration key: {key}",
103		details={"key": key, "schema": schema_item}
104		)
105
106		# Check type if the key exists
107		if key in self.config:
108		value = self.config[key]
109		expected_type = schema_item.get('type')
110
111		if expected_type and not isinstance(value, expected_type):
112		raise PluginConfigurationError(
113		f"Invalid type for configuration key {key}. Expected {expected_type.__name__}, got {type(value).__name__}",
114		details={"key": key, "expected_type": expected_type.__name__, "actual_type": type(value).__name__}
115		)
116
117		# Check enum values
118		enum_values = schema_item.get('enum')
119		if enum_values and value not in enum_values:
120		raise PluginConfigurationError(
121		f"Invalid value for configuration key {key}. Expected one of {enum_values}, got {value}",
122		details={"key": key, "expected_values": enum_values, "actual_value": value}
123		)
124
125		# Check custom validation
126		validator = schema_item.get('validator')
127		if validator and callable(validator):
128		if not validator(value):
129		raise PluginConfigurationError(
130		f"Validation failed for configuration key {key}",
131		details={"key": key, "value": value}
132		)
133
134		return True
135
136		def get(self, key, default=None):
137		"""
138		Get a configuration value.
139
140		Args:
141		key: Configuration key
142		default: Default value if key is not found
143
144		Returns:
145		Any: Configuration value or default
146		"""
147		return self.config.get(key, default)
148
149		def set(self, key, value):
150		"""
151		Set a configuration value.
152
153		Args:
154		key: Configuration key
155		value: Configuration value
156
157		Returns:
158		PluginConfig: Self for method chaining
159
160		Raises:
161		PluginConfigurationError: If validation fails
162		"""
163		self.config[key] = value
164
165		# Validate if schema exists for this key
166		if self.schema and key in self.schema:
167		schema_item = self.schema[key]
168
169		# Check type
170		expected_type = schema_item.get('type')
171		if expected_type and not isinstance(value, expected_type):
172		raise PluginConfigurationError(
173		f"Invalid type for configuration key {key}. Expected {expected_type.__name__}, got {type(value).__name__}",
174		details={"key": key, "expected_type": expected_type.__name__, "actual_type": type(value).__name__}
175		)
176
177		# Check enum values
178		enum_values = schema_item.get('enum')
179		if enum_values and value not in enum_values:
180		raise PluginConfigurationError(
181		f"Invalid value for configuration key {key}. Expected one of {enum_values}, got {value}",
182		details={"key": key, "expected_values": enum_values, "actual_value": value}
183		)
184
185		# Check custom validation
186		validator = schema_item.get('validator')
187		if validator and callable(validator):
188		if not validator(value):
189		raise PluginConfigurationError(
190		f"Validation failed for configuration key {key}",
191		details={"key": key, "value": value}
192		)
193
194		return self
195
196		def merge(self, other_config):
197		"""
198		Merge another configuration into this one.
199
200		Args:
201		other_config: Another PluginConfig or dictionary
202
203		Returns:
204		PluginConfig: Self for method chaining
205		"""
206		if isinstance(other_config, PluginConfig):
207		other_dict = other_config.to_dict()
208		else:
209		other_dict = other_config
210
211		# Deep merge
212		self._deep_merge(self.config, other_dict)
213
214		# Validate the merged configuration
215		if self.schema:
216		self.validate()
217
218		return self
219
220		def _deep_merge(self, target, source):
221		"""
222		Deep merge two dictionaries.
223
224		Args:
225		target: Target dictionary
226		source: Source dictionary
227		"""
228		for key, value in source.items():
229		if key in target and isinstance(target[key], dict) and isinstance(value, dict):
230		self._deep_merge(target[key], value)
231		else:
232		target[key] = value
233
234		def to_dict(self):
235		"""
236		Convert the configuration to a dictionary.
237
238		Returns:
239		Dict[str, Any]: Configuration dictionary
240		"""
241		return self.config.copy()
242
243		def save(self, file_path, format='yaml'):
244		"""
245		Save the configuration to a file.
246
247		Args:
248		file_path: Path to save the configuration to
249		format: File format ('json' or 'yaml')
250
251		Returns:
252		PluginConfig: Self for method chaining
253
254		Raises:
255		PluginConfigurationError: If saving fails
256		"""
257		try:
258		# Create directory if it doesn't exist
259		os.makedirs(os.path.dirname(file_path), exist_ok=True)
260
261		with open(file_path, 'w') as f:
262		if format.lower() == 'json':
263		json.dump(self.config, f, indent=2)
264		elif format.lower() in ('yaml', 'yml'):
265		yaml.dump(self.config, f, default_flow_style=False)
266		else:
267		raise PluginConfigurationError(
268		f"Unsupported configuration format: {format}",
269		details={"format": format}
270		)
271
272		return self
273
274		except PluginConfigurationError:
275		# Re-raise plugin configuration errors
276		raise
277		except Exception as e:
278		# Wrap other exceptions
279		raise PluginConfigurationError(
280		f"Failed to save configuration: {str(e)}",
281		details={"file_path": file_path, "format": format}
282		) from e
283
284
285		def create_default_config(plugin_name, config_type='shared'):
286		"""
287		Create a default configuration for a plugin.
288
289		Args:
290		plugin_name: Name of the plugin
291		config_type: Type of configuration ('shared', 'matrix', 'agent')
292
293		Returns:
294		Dict[str, Any]: Default configuration
295		"""
296		# Base configuration
297		config = {
298		"enabled": True,
299		"name": plugin_name
300		}
301
302		# Add type-specific defaults
303		if config_type == 'shared':
304		pass # No additional defaults
305		elif config_type == 'matrix':
306		config.update({
307		"commands_enabled": True,
308		"command_prefix": "/"
309		})
310		elif config_type == 'agent':
311		config.update({
312		"functions_enabled": True
313		})
314
315		return config
316
317
318		def load_plugin_config(plugin_name, config_dir=None):
319		"""
320		Load configuration for a plugin from the default location.
321
322		Args:
323		plugin_name: Name of the plugin
324		config_dir: Optional custom configuration directory
325
326		Returns:
327		PluginConfig: Plugin configuration
328		"""
329		if not config_dir:
330		# Use default location
331		config_dir = os.path.join(
332		os.path.expanduser('~'),
333		'.ollamatools',
334		'plugins'
335		)
336
337		# Check for plugin-specific config file
338		config_file = os.path.join(config_dir, f"{plugin_name.lower()}.yaml")
339
340		if os.path.exists(config_file):
341		return PluginConfig(config_file)
342		else:
343		# Return default configuration
344		return PluginConfig(create_default_config(plugin_name))

ollamatools-project/ollamatools/config/schemas.py +0 -163

…	…	@@ -1,163 +0,0 @@
1		"""
2		Configuration schemas for OllamaTools plugins.
3
4		This module provides schema definitions for validating plugin configurations.
5		"""
6
7		from typing import Dict, Any, List, Optional, Set, Union, Callable
8
9		# Schema for the ConversationHistoryPlugin
10		CONVERSATION_HISTORY_SCHEMA = {
11		"history_file": {
12		"type": str,
13		"required": False,
14		"description": "Path to the conversation history file"
15		},
16		"max_history": {
17		"type": int,
18		"required": False,
19		"default": 100,
20		"description": "Maximum number of conversations to keep in history"
21		},
22		"enable_topic_analysis": {
23		"type": bool,
24		"required": False,
25		"default": True,
26		"description": "Whether to enable topic analysis"
27		},
28		"enable_metrics": {
29		"type": bool,
30		"required": False,
31		"default": True,
32		"description": "Whether to enable performance metrics"
33		}
34		}
35
36		# Schema for the SearchPlugin
37		SEARCH_PLUGIN_SCHEMA = {
38		"base_url": {
39		"type": str,
40		"required": False,
41		"default": "http://localhost:8081",
42		"description": "Base URL for the OpenSearchNG service"
43		},
44		"engines": {
45		"type": list,
46		"required": False,
47		"default": ["google", "bing", "duckduckgo", "brave"],
48		"description": "List of search engines to use"
49		},
50		"num_results": {
51		"type": int,
52		"required": False,
53		"default": 5,
54		"description": "Number of search results to return"
55		},
56		"enable_auto_search": {
57		"type": bool,
58		"required": False,
59		"default": True,
60		"description": "Whether to enable automatic search for factual queries"
61		}
62		}
63
64		# Schema for the MatrixAgentAdapterPlugin
65		MATRIX_AGENT_ADAPTER_SCHEMA = {
66		"agent_class": {
67		"type": (str, object), # Can be a string path or a class object
68		"required": True,
69		"description": "Agent class to use (string path or class object)"
70		},
71		"agent_config": {
72		"type": dict,
73		"required": False,
74		"default": {},
75		"description": "Configuration for the agent"
76		}
77		}
78
79		# Schema for the MatrixCorePlugin
80		MATRIX_CORE_PLUGIN_SCHEMA = {
81		"homeserver": {
82		"type": str,
83		"required": True,
84		"description": "URL of the Matrix homeserver"
85		},
86		"user_id": {
87		"type": str,
88		"required": True,
89		"description": "Matrix user ID"
90		},
91		"password": {
92		"type": str,
93		"required": True,
94		"description": "Matrix password"
95		},
96		"device_id": {
97		"type": str,
98		"required": False,
99		"description": "Device ID for E2E encryption"
100		},
101		"ssl": {
102		"type": bool,
103		"required": False,
104		"default": True,
105		"description": "Whether to use SSL for the Matrix connection"
106		},
107		"respond_to_all_messages": {
108		"type": bool,
109		"required": False,
110		"default": False,
111		"description": "Whether to respond to all messages (not just mentions)"
112		}
113		}
114
115		# Schema for the ConversationContextPlugin
116		CONVERSATION_CONTEXT_SCHEMA = {
117		"max_context_length": {
118		"type": int,
119		"required": False,
120		"default": 10,
121		"description": "Maximum number of messages to keep in context"
122		},
123		"context_strategy": {
124		"type": str,
125		"required": False,
126		"default": "sliding_window",
127		"enum": ["sliding_window", "token_based", "time_based", "relevance_based"],
128		"description": "Strategy for managing conversation context"
129		},
130		"token_limit": {
131		"type": int,
132		"required": False,
133		"default": 4096,
134		"description": "Maximum number of tokens to keep in context (for token_based strategy)"
135		},
136		"time_window": {
137		"type": int,
138		"required": False,
139		"default": 3600,
140		"description": "Time window in seconds for context (for time_based strategy)"
141		}
142		}
143
144		# Schema registry
145		SCHEMA_REGISTRY = {
146		"ConversationHistoryPlugin": CONVERSATION_HISTORY_SCHEMA,
147		"SearchPlugin": SEARCH_PLUGIN_SCHEMA,
148		"MatrixAgentAdapterPlugin": MATRIX_AGENT_ADAPTER_SCHEMA,
149		"MatrixCorePlugin": MATRIX_CORE_PLUGIN_SCHEMA,
150		"ConversationContextPlugin": CONVERSATION_CONTEXT_SCHEMA
151		}
152
153		def get_schema_for_plugin(plugin_name):
154		"""
155		Get the configuration schema for a plugin.
156
157		Args:
158		plugin_name: Name of the plugin
159
160		Returns:
161		Dict[str, Any]: Schema dictionary or empty dict if not found
162		"""
163		return SCHEMA_REGISTRY.get(plugin_name, {})

ollamatools-project/ollamatools/errors.py +0 -384

…	…	@@ -1,384 +0,0 @@
1		"""
2		Error definitions for OllamaTools.
3
4		This module defines custom exceptions and error utilities for standardized
5		error handling across the OllamaTools library.
6		"""
7
8		from typing import Dict, Any, Optional, List, Tuple, Union
9		import logging
10		import sys
11		import traceback
12
13		# Set up logging
14		logger = logging.getLogger(__name__)
15
16		# Define error codes
17		ERROR_CODES = {
18		# General errors
19		'GENERAL_ERROR': 100,
20		'NOT_IMPLEMENTED': 101,
21		'CONFIGURATION_ERROR': 102,
22		'TIMEOUT_ERROR': 103,
23
24		# Permission errors
25		'PERMISSION_DENIED': 200,
26		'PERMISSION_PROMPT_CANCELLED': 201,
27
28		# Command execution errors
29		'COMMAND_VALIDATION_ERROR': 300,
30		'COMMAND_EXECUTION_ERROR': 301,
31		'COMMAND_NOT_FOUND': 302,
32		'COMMAND_TIMEOUT': 303,
33		'COMMAND_NOT_ALLOWED': 304,
34
35		# File operation errors
36		'FILE_NOT_FOUND': 400,
37		'FILE_READ_ERROR': 401,
38		'FILE_WRITE_ERROR': 402,
39		'FILE_PERMISSION_ERROR': 403,
40		'FILE_SIZE_EXCEEDED': 404,
41		'PATH_TRAVERSAL_ERROR': 405,
42		'PATH_VALIDATION_ERROR': 406,
43		'PATH_OUTSIDE_SANDBOX': 407,
44
45		# Web request errors
46		'URL_VALIDATION_ERROR': 500,
47		'URL_FETCH_ERROR': 501,
48		'URL_CONNECTION_ERROR': 502,
49		'URL_RESPONSE_ERROR': 503,
50		'URL_BLOCKED': 504,
51		'URL_TIMEOUT': 505,
52
53		# API errors
54		'API_ERROR': 600,
55		'API_KEY_MISSING': 601,
56		'API_KEY_INVALID': 602,
57		'API_LIMIT_EXCEEDED': 603,
58		'API_SERVICE_UNAVAILABLE': 604,
59
60		# Ollama client errors
61		'OLLAMA_CONNECTION_ERROR': 700,
62		'OLLAMA_MODEL_ERROR': 701,
63		'OLLAMA_API_ERROR': 702,
64		'OLLAMA_PARSING_ERROR': 703,
65		}
66
67
68		class OllamaToolsError(Exception):
69		"""Base exception class for all OllamaTools errors."""
70
71		def __init__(
72		self,
73		message: str,
74		code: Optional[int] = None,
75		details: Optional[Dict[str, Any]] = None
76		):
77		"""
78		Initialize the exception.
79
80		Args:
81		message: Error message
82		code: Error code
83		details: Additional error details
84		"""
85		self.message = message
86		self.code = code or ERROR_CODES.get('GENERAL_ERROR', 100)
87		self.details = details or {}
88
89		# Construct the full error message
90		full_message = f"[Error {self.code}] {message}"
91		if details:
92		full_message += f" - Details: {details}"
93
94		super().__init__(full_message)
95
96
97		class ConfigurationError(OllamaToolsError):
98		"""Error raised for configuration issues."""
99
100		def __init__(
101		self,
102		message: str,
103		details: Optional[Dict[str, Any]] = None
104		):
105		super().__init__(
106		message,
107		code=ERROR_CODES.get('CONFIGURATION_ERROR'),
108		details=details
109		)
110
111
112		class TimeoutError(OllamaToolsError):
113		"""Error raised when an operation times out."""
114
115		def __init__(
116		self,
117		message: str,
118		timeout: Optional[int] = None,
119		details: Optional[Dict[str, Any]] = None
120		):
121		if timeout is not None:
122		details = details or {}
123		details['timeout'] = timeout
124
125		super().__init__(
126		message,
127		code=ERROR_CODES.get('TIMEOUT_ERROR'),
128		details=details
129		)
130
131
132		class PermissionError(OllamaToolsError):
133		"""Error raised when permission is denied for an operation."""
134
135		def __init__(
136		self,
137		message: str,
138		function_name: Optional[str] = None,
139		args: Optional[Dict[str, Any]] = None,
140		details: Optional[Dict[str, Any]] = None
141		):
142		combined_details = details or {}
143		if function_name:
144		combined_details['function'] = function_name
145		if args:
146		combined_details['args'] = args
147
148		super().__init__(
149		message,
150		code=ERROR_CODES.get('PERMISSION_DENIED'),
151		details=combined_details
152		)
153
154
155		class CommandError(OllamaToolsError):
156		"""Error raised for command execution issues."""
157
158		def __init__(
159		self,
160		message: str,
161		command: Optional[str] = None,
162		args: Optional[List[str]] = None,
163		exit_code: Optional[int] = None,
164		stderr: Optional[str] = None,
165		code: Optional[int] = None,
166		details: Optional[Dict[str, Any]] = None
167		):
168		error_code = code or ERROR_CODES.get('COMMAND_EXECUTION_ERROR')
169
170		combined_details = details or {}
171		if command:
172		combined_details['command'] = command
173		if args:
174		combined_details['args'] = args
175		if exit_code is not None:
176		combined_details['exit_code'] = exit_code
177		if stderr:
178		combined_details['stderr'] = stderr
179
180		super().__init__(
181		message,
182		code=error_code,
183		details=combined_details
184		)
185
186
187		class FileError(OllamaToolsError):
188		"""Error raised for file operation issues."""
189
190		def __init__(
191		self,
192		message: str,
193		path: Optional[str] = None,
194		code: Optional[int] = None,
195		details: Optional[Dict[str, Any]] = None
196		):
197		error_code = code or ERROR_CODES.get('FILE_READ_ERROR')
198
199		combined_details = details or {}
200		if path:
201		combined_details['path'] = path
202
203		super().__init__(
204		message,
205		code=error_code,
206		details=combined_details
207		)
208
209
210		class WebError(OllamaToolsError):
211		"""Error raised for web request issues."""
212
213		def __init__(
214		self,
215		message: str,
216		url: Optional[str] = None,
217		status_code: Optional[int] = None,
218		code: Optional[int] = None,
219		details: Optional[Dict[str, Any]] = None
220		):
221		error_code = code or ERROR_CODES.get('URL_FETCH_ERROR')
222
223		combined_details = details or {}
224		if url:
225		combined_details['url'] = url
226		if status_code is not None:
227		combined_details['status_code'] = status_code
228
229		super().__init__(
230		message,
231		code=error_code,
232		details=combined_details
233		)
234
235
236		class ApiError(OllamaToolsError):
237		"""Error raised for API-related issues."""
238
239		def __init__(
240		self,
241		message: str,
242		api_name: Optional[str] = None,
243		status_code: Optional[int] = None,
244		code: Optional[int] = None,
245		details: Optional[Dict[str, Any]] = None
246		):
247		error_code = code or ERROR_CODES.get('API_ERROR')
248
249		combined_details = details or {}
250		if api_name:
251		combined_details['api'] = api_name
252		if status_code is not None:
253		combined_details['status_code'] = status_code
254
255		super().__init__(
256		message,
257		code=error_code,
258		details=combined_details
259		)
260
261
262		class OllamaClientError(OllamaToolsError):
263		"""Error raised for Ollama client issues."""
264
265		def __init__(
266		self,
267		message: str,
268		model: Optional[str] = None,
269		code: Optional[int] = None,
270		details: Optional[Dict[str, Any]] = None
271		):
272		error_code = code or ERROR_CODES.get('OLLAMA_API_ERROR')
273
274		combined_details = details or {}
275		if model:
276		combined_details['model'] = model
277
278		super().__init__(
279		message,
280		code=error_code,
281		details=combined_details
282		)
283
284
285		def get_error_details(exception: Exception) -> Dict[str, Any]:
286		"""
287		Extract detailed information from an exception.
288
289		Args:
290		exception: The exception to extract details from
291
292		Returns:
293		dict: Error details
294		"""
295		error_type = type(exception).__name__
296		error_message = str(exception)
297
298		# Get the traceback
299		tb = traceback.extract_tb(sys.exc_info()[2])
300		tb_formatted = []
301
302		for frame in tb:
303		tb_formatted.append({
304		'file': frame.filename,
305		'line': frame.lineno,
306		'function': frame.name,
307		'code': frame.line
308		})
309
310		# Create the error details
311		details = {
312		'type': error_type,
313		'message': error_message,
314		'traceback': tb_formatted
315		}
316
317		# Add additional details if it's an OllamaToolsError
318		if isinstance(exception, OllamaToolsError):
319		details['code'] = exception.code
320		if exception.details:
321		details['additional_details'] = exception.details
322
323		return details
324
325
326		def format_error_response(exception: Exception) -> Dict[str, Any]:
327		"""
328		Format an exception as a standardized error response.
329
330		Args:
331		exception: The exception to format
332
333		Returns:
334		dict: Formatted error response
335		"""
336		details = get_error_details(exception)
337
338		# Create a user-friendly error response
339		error_response = {
340		'error': True,
341		'error_type': details['type'],
342		'error_message': details['message']
343		}
344
345		# Add code if available
346		if 'code' in details:
347		error_response['error_code'] = details['code']
348
349		return error_response
350
351
352		def handle_exception(
353		exception: Exception,
354		log_error: bool = True,
355		include_traceback: bool = False
356		) -> Dict[str, Any]:
357		"""
358		Handle an exception and return a standardized error response.
359
360		Args:
361		exception: The exception to handle
362		log_error: Whether to log the error
363		include_traceback: Whether to include traceback in the response
364
365		Returns:
366		dict: Error response
367		"""
368		# Log the error if requested
369		if log_error:
370		error_details = get_error_details(exception)
371		logger.error(
372		f"Error: {error_details['type']}: {error_details['message']}",
373		exc_info=True
374		)
375
376		# Format the error response
377		error_response = format_error_response(exception)
378
379		# Include traceback if requested
380		if include_traceback:
381		error_details = get_error_details(exception)
382		error_response['traceback'] = error_details['traceback']
383
384		return error_response

ollamatools-project/ollamatools/logger.py +0 -144

…	…	@@ -1,144 +0,0 @@
1		"""
2		Logging utilities for OllamaTools.
3
4		This module contains logging utilities used by the OllamaTools library.
5		"""
6
7		import logging
8		import os
9		import sys
10		from typing import Optional, Union, Dict, Any, List
11
12
13		def get_logger(
14		name: str,
15		level: Union[int, str] = logging.INFO,
16		log_file: Optional[str] = None,
17		log_format: Optional[str] = None,
18		propagate: bool = False,
19		additional_handlers: Optional[List[logging.Handler]] = None,
20		extra: Optional[Dict[str, Any]] = None
21		) -> logging.Logger:
22		"""
23		Get a configured logger instance.
24
25		Args:
26		name: Logger name
27		level: Log level (can be integer or string like 'INFO', 'DEBUG')
28		log_file: Optional log file path
29		log_format: Optional custom log format
30		propagate: Whether to propagate logs to parent loggers
31		additional_handlers: Optional list of additional handlers to add
32		extra: Optional extra fields to add to all log records
33
34		Returns:
35		logging.Logger: Configured logger instance
36		"""
37		# Convert string level to integer if needed
38		if isinstance(level, str):
39		level = getattr(logging, level.upper())
40
41		# Create or get the logger
42		logger = logging.getLogger(name)
43		logger.setLevel(level)
44		logger.propagate = propagate
45
46		# Remove existing handlers to avoid duplicates if the logger already exists
47		for handler in logger.handlers[:]:
48		logger.removeHandler(handler)
49
50		# Default format if none provided
51		if log_format is None:
52		log_format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
53
54		formatter = logging.Formatter(log_format)
55
56		# Add console handler
57		console_handler = logging.StreamHandler(stream=sys.stdout)
58		console_handler.setLevel(level)
59		console_handler.setFormatter(formatter)
60		logger.addHandler(console_handler)
61
62		# Add file handler if specified
63		if log_file:
64		# Create directory if it doesn't exist
65		log_dir = os.path.dirname(os.path.abspath(log_file))
66		if log_dir:
67		os.makedirs(log_dir, exist_ok=True)
68
69		file_handler = logging.FileHandler(log_file)
70		file_handler.setLevel(level)
71		file_handler.setFormatter(formatter)
72		logger.addHandler(file_handler)
73
74		# Add any additional handlers
75		if additional_handlers:
76		for handler in additional_handlers:
77		logger.addHandler(handler)
78
79		# Add a filter for extra fields if provided
80		if extra:
81		class ExtraFilter(logging.Filter):
82		def filter(self, record):
83		for key, value in extra.items():
84		setattr(record, key, value)
85		return True
86
87		extra_filter = ExtraFilter()
88		logger.addFilter(extra_filter)
89
90		return logger
91
92
93		def enable_debug_logging(logger_names: Optional[List[str]] = None) -> None:
94		"""
95		Enable debug logging for specified loggers or all loggers.
96
97		Args:
98		logger_names: Optional list of logger names to enable debug logging for,
99		if None, enables for root logger
100		"""
101		if logger_names is None:
102		# Enable debug logging for the root logger
103		logging.getLogger().setLevel(logging.DEBUG)
104		else:
105		# Enable debug logging for specific loggers
106		for name in logger_names:
107		logging.getLogger(name).setLevel(logging.DEBUG)
108
109
110		def disable_logging(logger_names: Optional[List[str]] = None) -> None:
111		"""
112		Disable logging for specified loggers or all loggers.
113
114		Args:
115		logger_names: Optional list of logger names to disable logging for,
116		if None, disables for root logger
117		"""
118		if logger_names is None:
119		# Disable logging for the root logger
120		logging.getLogger().setLevel(logging.CRITICAL + 1)
121		else:
122		# Disable logging for specific loggers
123		for name in logger_names:
124		logging.getLogger(name).setLevel(logging.CRITICAL + 1)
125
126
127		def get_ollamatools_logger(
128		component: str,
129		level: Union[int, str] = logging.INFO,
130		log_file: Optional[str] = None
131		) -> logging.Logger:
132		"""
133		Get a logger for a specific OllamaTools component.
134
135		Args:
136		component: Component name (e.g., 'agent', 'system_tools')
137		level: Log level
138		log_file: Optional log file path
139
140		Returns:
141		logging.Logger: Configured logger instance
142		"""
143		logger_name = f"ollamatools.{component}"
144		return get_logger(logger_name, level, log_file)

ollamatools-project/ollamatools/opensearch_web_tools.py +0 -425

…	…	@@ -1,425 +0,0 @@
1		"""
2		OpenSearchNG integration for OllamaTools.
3
4		This module extends the web_tools capabilities by adding integration with
5		a locally running OpenSearchNG instance for improved search functionality.
6		"""
7
8		import requests
9		import json
10		import logging
11		from typing import Dict, Any, Optional, List, Union, Set
12		from urllib.parse import quote_plus
13
14		# Import URL validation utilities
15		from ollamatools.url_validator import (
16		validate_url,
17		URLValidationError,
18		configure_domain_lists,
19		)
20
21		# Import error handling utilities
22		from ollamatools.errors import (
23		WebError,
24		ERROR_CODES,
25		handle_exception
26		)
27
28		# Set up logging
29		logger = logging.getLogger(__name__)
30
31		# Default OpenSearchNG configuration
32		DEFAULT_OPENSEARCH_CONFIG = {
33		"base_url": "http://localhost:8081", # Default local OpenSearchNG URL
34		"search_path": "/search", # Path for search endpoint
35		"preferences_path": "/preferences", # Path for preferences endpoint
36		"categories": ["general"], # Default search categories
37		"engines": ["google", "bing", "duckduckgo"], # Default search engines
38		"language": "en-US", # Default language
39		"format": "json", # Response format (html, json, csv, rss)
40		"timeout": 10, # Default timeout in seconds
41		"num_results": 10, # Default number of results
42		"safe_search": 0 # 0=off, 1=moderate, 2=strict
43		}
44
45		def opensearch_query(
46		query: str,
47		config: Optional[Dict[str, Any]] = None,
48		permission_manager = None,
49		domain_allowlist: Optional[Set[str]] = None,
50		domain_blocklist: Optional[Set[str]] = None
51		) -> Dict[str, Any]:
52		"""
53		Perform a search query using a local OpenSearchNG instance.
54
55		Args:
56		query: Search query string
57		config: OpenSearchNG configuration (defaults used if None)
58		permission_manager: Optional permission manager to request permission
59		domain_allowlist: Optional set of allowed domains for results
60		domain_blocklist: Optional set of blocked domains for results
61
62		Returns:
63		dict: Search results with metadata
64
65		Raises:
66		ValueError: If parameters are invalid
67		WebError: If the search fails
68		PermissionError: If permission is denied
69		"""
70		# Validate input
71		if not query or not isinstance(query, str):
72		raise ValueError("Query must be a non-empty string")
73
74		# Use default config if none provided
75		search_config = DEFAULT_OPENSEARCH_CONFIG.copy()
76		if config:
77		search_config.update(config)
78
79		# Request permission if permission manager is provided
80		if permission_manager:
81		args = {
82		"query": query,
83		"config": search_config,
84		}
85
86		if not permission_manager.request_permission("opensearch_query", args):
87		logger.warning(f"Permission denied for OpenSearchNG query: {query}")
88		raise PermissionError(f"Permission denied for OpenSearchNG query: {query}")
89
90		# Construct the search URL
91		base_url = search_config["base_url"]
92		search_path = search_config["search_path"]
93		search_url = f"{base_url}{search_path}"
94
95		# Validate the OpenSearchNG instance URL
96		try:
97		# Allow localhost for OpenSearchNG access
98		local_allowlist = {"localhost", "127.0.0.1"}
99		if domain_allowlist:
100		local_allowlist.update(domain_allowlist)
101
102		# Extract the host from the base URL for validation
103		from urllib.parse import urlparse
104		parsed_url = urlparse(base_url)
105		hostname = parsed_url.netloc.split(':')[0]
106
107		if hostname not in local_allowlist:
108		validate_url(
109		base_url,
110		domain_allowlist=local_allowlist,
111		domain_blocklist=domain_blocklist
112		)
113		except URLValidationError as e:
114		# Make a special exception for localhost
115		if "localhost" not in base_url and "127.0.0.1" not in base_url:
116		raise ValueError(f"Invalid OpenSearchNG URL: {str(e)}")
117
118		# Prepare query parameters
119		params = {
120		"q": query,
121		"format": search_config["format"],
122		"categories": ",".join(search_config["categories"]),
123		"engines": ",".join(search_config["engines"]),
124		"language": search_config["language"],
125		"safesearch": search_config["safe_search"],
126		"time_range": "", # Optional time range
127		"pageno": 1, # First page of results
128		}
129
130		# Create session with appropriate headers
131		with requests.Session() as session:
132		headers = {
133		"User-Agent": "OllamaTools/0.2.0",
134		"Accept": "application/json"
135		}
136		session.headers.update(headers)
137
138		try:
139		logger.info(f"Making OpenSearchNG query: {query}")
140		response = session.get(
141		search_url,
142		params=params,
143		timeout=search_config["timeout"]
144		)
145
146		# Check for successful response
147		if response.status_code != 200:
148		raise WebError(
149		f"OpenSearchNG query failed with status code: {response.status_code}",
150		url=search_url,
151		status_code=response.status_code,
152		code=ERROR_CODES["URL_RESPONSE_ERROR"]
153		)
154
155		# Parse the response based on format
156		if search_config["format"] == "json":
157		try:
158		result_data = response.json()
159		except json.JSONDecodeError:
160		raise WebError(
161		"Failed to parse JSON response from OpenSearchNG",
162		url=search_url,
163		code=ERROR_CODES["URL_RESPONSE_ERROR"]
164		)
165		else:
166		# For other formats, return the raw content
167		result_data = {"raw_content": response.text}
168
169		# Validate and process the results
170		processed_results = process_opensearch_results(result_data, query)
171
172		return processed_results
173
174		except requests.RequestException as e:
175		raise WebError(
176		f"OpenSearchNG request failed: {str(e)}",
177		url=search_url,
178		code=ERROR_CODES["URL_CONNECTION_ERROR"]
179		)
180
181		def process_opensearch_results(
182		result_data: Dict[str, Any],
183		query: str
184		) -> Dict[str, Any]:
185		"""
186		Process and format the OpenSearchNG results.
187
188		Args:
189		result_data: Raw result data from OpenSearchNG
190		query: Original search query
191
192		Returns:
193		dict: Processed search results with metadata
194		"""
195		processed_results = {
196		"query": query,
197		"results": [],
198		"suggestion": None,
199		"answer": None,
200		"infobox": None,
201		"engines": [],
202		"timing": {}
203		}
204
205		# Extract search results
206		if "results" in result_data:
207		for result in result_data["results"]:
208		processed_result = {
209		"title": result.get("title", ""),
210		"url": result.get("url", ""),
211		"content": result.get("content", ""),
212		"engine": result.get("engine", ""),
213		"score": result.get("score", 0),
214		"category": result.get("category", ""),
215		"pretty_url": result.get("pretty_url", "")
216		}
217		processed_results["results"].append(processed_result)
218
219		# Extract suggestions
220		if "suggestions" in result_data and result_data["suggestions"]:
221		try:
222		suggestion = result_data["suggestions"][0]
223		# Handle case where suggestion is a dict instead of a string
224		if isinstance(suggestion, dict):
225		processed_results["suggestion"] = suggestion.get("value", str(suggestion))
226		else:
227		processed_results["suggestion"] = suggestion
228		except (IndexError, TypeError) as e:
229		logger.warning(f"Error processing suggestions: {str(e)}")
230		processed_results["suggestion"] = None
231
232		# Extract answer if available
233		if "answers" in result_data and result_data["answers"]:
234		try:
235		answer = result_data["answers"][0]
236		# Handle case where answer is a dict instead of a string
237		if isinstance(answer, dict):
238		processed_results["answer"] = answer.get("content", str(answer))
239		else:
240		processed_results["answer"] = answer
241		except (IndexError, TypeError) as e:
242		logger.warning(f"Error processing answers: {str(e)}")
243		processed_results["answer"] = None
244
245		# Extract infobox if available
246		if "infoboxes" in result_data and result_data["infoboxes"]:
247		try:
248		infobox = result_data["infoboxes"][0]
249		# Handle case where infobox is a dict (which is expected)
250		processed_results["infobox"] = infobox
251		except (IndexError, TypeError) as e:
252		logger.warning(f"Error processing infoboxes: {str(e)}")
253		processed_results["infobox"] = None
254
255		# Extract engines used
256		if "engines" in result_data:
257		processed_results["engines"] = result_data["engines"]
258
259		# Extract timing information
260		if "timing" in result_data:
261		processed_results["timing"] = result_data["timing"]
262
263		return processed_results
264
265		def get_opensearch_info(
266		config: Optional[Dict[str, Any]] = None,
267		permission_manager = None
268		) -> Dict[str, Any]:
269		"""
270		Get information about the OpenSearchNG instance.
271
272		Args:
273		config: OpenSearchNG configuration (defaults used if None)
274		permission_manager: Optional permission manager to request permission
275
276		Returns:
277		dict: Information about the OpenSearchNG instance
278
279		Raises:
280		WebError: If the request fails
281		PermissionError: If permission is denied
282		"""
283		# Use default config if none provided
284		search_config = DEFAULT_OPENSEARCH_CONFIG.copy()
285		if config:
286		search_config.update(config)
287
288		# Request permission if permission manager is provided
289		if permission_manager:
290		if not permission_manager.request_permission("get_opensearch_info", {"config": search_config}):
291		logger.warning("Permission denied for getting OpenSearchNG info")
292		raise PermissionError("Permission denied for getting OpenSearchNG info")
293
294		# Construct the URL
295		base_url = search_config["base_url"]
296		config_url = f"{base_url}/config"
297
298		try:
299		logger.info("Getting OpenSearchNG instance information")
300		response = requests.get(
301		config_url,
302		timeout=search_config["timeout"]
303		)
304
305		# Check for successful response
306		if response.status_code != 200:
307		raise WebError(
308		f"OpenSearchNG info request failed with status code: {response.status_code}",
309		url=config_url,
310		status_code=response.status_code,
311		code=ERROR_CODES["URL_RESPONSE_ERROR"]
312		)
313
314		# Parse the response
315		try:
316		config_data = response.json()
317		return config_data
318		except json.JSONDecodeError:
319		raise WebError(
320		"Failed to parse JSON response from OpenSearchNG",
321		url=config_url,
322		code=ERROR_CODES["URL_RESPONSE_ERROR"]
323		)
324
325		except requests.RequestException as e:
326		raise WebError(
327		f"OpenSearchNG info request failed: {str(e)}",
328		url=config_url,
329		code=ERROR_CODES["URL_CONNECTION_ERROR"]
330		)
331
332		def opensearch_fetch_url(
333		url: str,
334		config: Optional[Dict[str, Any]] = None,
335		permission_manager = None,
336		domain_allowlist: Optional[Set[str]] = None,
337		domain_blocklist: Optional[Set[str]] = None
338		) -> Dict[str, Any]:
339		"""
340		Fetch a URL through the OpenSearchNG proxy to avoid direct connections.
341		This provides an additional security layer for web requests.
342
343		Args:
344		url: URL to fetch
345		config: OpenSearchNG configuration (defaults used if None)
346		permission_manager: Optional permission manager to request permission
347		domain_allowlist: Optional set of allowed domains
348		domain_blocklist: Optional set of blocked domains
349
350		Returns:
351		dict: Response content and metadata
352
353		Raises:
354		ValueError: If URL is invalid
355		WebError: If the fetch fails
356		PermissionError: If permission is denied
357		"""
358		# Validate URL
359		try:
360		validated_url = validate_url(
361		url,
362		domain_allowlist=domain_allowlist,
363		domain_blocklist=domain_blocklist
364		)
365		except URLValidationError as e:
366		raise ValueError(f"URL validation failed: {str(e)}")
367
368		# Use default config if none provided
369		search_config = DEFAULT_OPENSEARCH_CONFIG.copy()
370		if config:
371		search_config.update(config)
372
373		# Request permission if permission manager is provided
374		if permission_manager:
375		args = {
376		"url": url,
377		"config": search_config
378		}
379
380		if not permission_manager.request_permission("opensearch_fetch_url", args):
381		logger.warning(f"Permission denied for fetching URL via OpenSearchNG: {url}")
382		raise PermissionError(f"Permission denied for fetching URL via OpenSearchNG: {url}")
383
384		# Construct the URL for the OpenSearchNG proxy
385		base_url = search_config["base_url"]
386		proxy_url = f"{base_url}/url_proxy"
387
388		# Prepare query parameters
389		params = {
390		"url": validated_url
391		}
392
393		try:
394		logger.info(f"Fetching URL via OpenSearchNG proxy: {validated_url}")
395		response = requests.get(
396		proxy_url,
397		params=params,
398		timeout=search_config["timeout"]
399		)
400
401		# Check for successful response
402		if response.status_code != 200:
403		raise WebError(
404		f"URL fetch via OpenSearchNG failed with status code: {response.status_code}",
405		url=proxy_url,
406		status_code=response.status_code,
407		code=ERROR_CODES["URL_RESPONSE_ERROR"]
408		)
409
410		# Create result with metadata
411		result = {
412		"status": response.status_code,
413		"headers": dict(response.headers),
414		"content": response.text,
415		"url": validated_url
416		}
417
418		return result
419
420		except requests.RequestException as e:
421		raise WebError(
422		f"URL fetch via OpenSearchNG failed: {str(e)}",
423		url=proxy_url,
424		code=ERROR_CODES["URL_CONNECTION_ERROR"]
425		)

ollamatools-project/ollamatools/path_validator.py +0 -419

…	…	@@ -1,419 +0,0 @@
1		"""
2		Path validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing file paths
5		to prevent path traversal attacks and other file path-related security issues.
6		"""
7
8		import os
9		import re
10		import pathlib
11		from typing import Optional, List, Set, Dict, Any, Union, Tuple
12		import logging
13
14		# Set up logging
15		logger = logging.getLogger(__name__)
16
17		# Default protected paths - should not be modified
18		DEFAULT_PROTECTED_PATHS = {
19		'/etc',
20		'/bin',
21		'/sbin',
22		'/usr/bin',
23		'/usr/sbin',
24		'/usr/lib',
25		'/usr/lib64',
26		'/boot',
27		'/dev',
28		'/proc',
29		'/sys',
30		'/var/lib',
31		'/var/log',
32		'/var/run',
33		'/var/lock',
34		}
35
36		# Default paths to allow even if they're in protected paths
37		DEFAULT_ALLOWED_PATHS = {
38		'/etc/hosts', # commonly read for network configuration
39		'/etc/hostname', # commonly read for system identification
40		'/etc/passwd', # commonly read for user information (non-sensitive part)
41		}
42
43		# Default sandbox path - where operations are restricted to
44		DEFAULT_SANDBOX_PATH = os.path.expanduser('~')
45
46		class PathValidationError(Exception):
47		"""Exception raised for path validation errors."""
48		pass
49
50
51		def normalize_path(path: str) -> str:
52		"""
53		Normalize a file path to resolve symlinks, relative paths, etc.
54
55		Args:
56		path: File path to normalize
57
58		Returns:
59		str: Normalized absolute path
60		"""
61		# Convert to string if it's not already
62		if not isinstance(path, str):
63		path = str(path)
64
65		# Remove any null bytes
66		path = path.replace('\0', '')
67
68		# Trim whitespace
69		path = path.strip()
70
71		# Create an absolute path
72		abs_path = os.path.abspath(path)
73
74		# Normalize the path (resolves ., .., symlinks)
75		try:
76		norm_path = os.path.normpath(abs_path)
77		return norm_path
78		except Exception as e:
79		raise PathValidationError(f"Path normalization error: {str(e)}")
80
81
82		def resolve_symlinks(path: str) -> str:
83		"""
84		Resolve symlinks in a path.
85
86		Args:
87		path: File path that might contain symlinks
88
89		Returns:
90		str: Path with all symlinks resolved
91		"""
92		try:
93		# Use pathlib to resolve symlinks
94		resolved_path = str(pathlib.Path(path).resolve())
95		return resolved_path
96		except Exception as e:
97		raise PathValidationError(f"Failed to resolve symlinks in path '{path}': {str(e)}")
98
99
100		def is_path_traversal_attempt(path: str) -> bool:
101		"""
102		Check if a path contains traversal patterns that might be used in attacks.
103
104		Args:
105		path: File path to check
106
107		Returns:
108		bool: True if the path contains suspicious patterns, False otherwise
109		"""
110		# Normalize for easy pattern matching
111		normalized = os.path.normpath(path)
112
113		# Check for common traversal patterns
114		suspicious_patterns = [
115		r'\.\./', # Parent directory references
116		r'/\.\./', # Parent directory references
117		r'/\.\.$', # Path ending with parent reference
118		r'^\.\./.*', # Path starting with parent reference
119		r'\\\\', # Double backslash (Windows)
120		r'\.\\', # Current directory backslash (Windows)
121		r'//', # Double forward slash
122		r'\0', # Null byte
123		r'%00', # URL-encoded null byte
124		r'%2e%2e%2f', # URL-encoded ../
125		r'%2e%2e/', # URL-encoded ../
126		r'\$\{.*?\}', # Shell variable expansion
127		r'`.*?`', # Command substitution
128		]
129
130		for pattern in suspicious_patterns:
131		if re.search(pattern, path):
132		return True
133
134		return False
135
136
137		def is_path_in_sandbox(
138		path: str,
139		sandbox_path: Optional[str] = None,
140		additional_allowed_paths: Optional[Set[str]] = None
141		) -> bool:
142		"""
143		Check if a path is within the sandbox path or explicitly allowed.
144
145		Args:
146		path: File path to check
147		sandbox_path: Base path that defines the sandbox (default is user's home)
148		additional_allowed_paths: Additional paths to allow
149
150		Returns:
151		bool: True if the path is in the sandbox or allowed, False otherwise
152		"""
153		# Resolve the sandbox path
154		if sandbox_path is None:
155		sandbox_path = DEFAULT_SANDBOX_PATH
156
157		# Normalize all paths for comparison
158		normalized_path = normalize_path(path)
159		normalized_sandbox = normalize_path(sandbox_path)
160
161		# Check if the path is explicitly allowed
162		allowed_paths = DEFAULT_ALLOWED_PATHS.copy()
163		if additional_allowed_paths:
164		allowed_paths.update(additional_allowed_paths)
165
166		if normalized_path in allowed_paths:
167		return True
168
169		# Check if the path is a subdirectory of the sandbox
170		return normalized_path.startswith(normalized_sandbox)
171
172
173		def is_path_protected(
174		path: str,
175		additional_protected_paths: Optional[Set[str]] = None,
176		allowed_paths: Optional[Set[str]] = None
177		) -> bool:
178		"""
179		Check if a path is in a protected area that should not be modified.
180
181		Args:
182		path: File path to check
183		additional_protected_paths: Additional paths to protect
184		allowed_paths: Paths explicitly allowed even if in protected areas
185
186		Returns:
187		bool: True if the path is protected, False otherwise
188		"""
189		# Normalize the path
190		normalized_path = normalize_path(path)
191
192		# Combine default and additional protected paths
193		protected_paths = DEFAULT_PROTECTED_PATHS.copy()
194		if additional_protected_paths:
195		protected_paths.update(additional_protected_paths)
196
197		# Combine default and additional allowed paths
198		all_allowed_paths = DEFAULT_ALLOWED_PATHS.copy()
199		if allowed_paths:
200		all_allowed_paths.update(allowed_paths)
201
202		# First check if the path is explicitly allowed
203		if normalized_path in all_allowed_paths:
204		return False
205
206		# Then check if the path is in a protected area
207		for protected in protected_paths:
208		if normalized_path == protected or normalized_path.startswith(protected + os.path.sep):
209		return True
210
211		return False
212
213
214		def validate_path_for_reading(
215		path: str,
216		sandbox_path: Optional[str] = None,
217		additional_allowed_paths: Optional[Set[str]] = None,
218		resolve_links: bool = True
219		) -> str:
220		"""
221		Validate a path for reading operations (e.g., read_file).
222
223		Args:
224		path: File path to validate
225		sandbox_path: Base path that defines the sandbox
226		additional_allowed_paths: Additional paths to allow
227		resolve_links: Whether to resolve symlinks
228
229		Returns:
230		str: Validated path
231
232		Raises:
233		PathValidationError: If the path does not pass validation
234		"""
235		# Check for empty path
236		if not path:
237		raise PathValidationError("Path cannot be empty")
238
239		# Normalize path
240		normalized_path = normalize_path(path)
241
242		# Check for path traversal attempts
243		if is_path_traversal_attempt(path):
244		raise PathValidationError(f"Path '{path}' contains suspicious traversal patterns")
245
246		# Resolve symlinks if requested
247		if resolve_links:
248		normalized_path = resolve_symlinks(normalized_path)
249
250		# Check if path exists (for reading, the file should exist)
251		if not os.path.exists(normalized_path):
252		raise PathValidationError(f"Path '{path}' does not exist")
253
254		# Check if path is a file (for reading, it should be a file not a directory)
255		if os.path.isdir(normalized_path):
256		raise PathValidationError(f"Path '{path}' is a directory, not a file")
257
258		# Check if path is in sandbox or explicitly allowed
259		if not is_path_in_sandbox(normalized_path, sandbox_path, additional_allowed_paths):
260		raise PathValidationError(f"Path '{path}' is outside the allowed directories")
261
262		return normalized_path
263
264
265		def validate_path_for_writing(
266		path: str,
267		sandbox_path: Optional[str] = None,
268		additional_allowed_paths: Optional[Set[str]] = None,
269		additional_protected_paths: Optional[Set[str]] = None,
270		resolve_links: bool = True
271		) -> str:
272		"""
273		Validate a path for writing operations (e.g., write_file).
274
275		Args:
276		path: File path to validate
277		sandbox_path: Base path that defines the sandbox
278		additional_allowed_paths: Additional paths to allow
279		additional_protected_paths: Additional paths to protect
280		resolve_links: Whether to resolve symlinks
281
282		Returns:
283		str: Validated path
284
285		Raises:
286		PathValidationError: If the path does not pass validation
287		"""
288		# Check for empty path
289		if not path:
290		raise PathValidationError("Path cannot be empty")
291
292		# Normalize path
293		normalized_path = normalize_path(path)
294
295		# Check for path traversal attempts
296		if is_path_traversal_attempt(path):
297		raise PathValidationError(f"Path '{path}' contains suspicious traversal patterns")
298
299		# Resolve symlinks if requested
300		if resolve_links:
301		normalized_path = resolve_symlinks(normalized_path)
302
303		# Check if path is protected (for writing, protected paths are off-limits)
304		if is_path_protected(normalized_path, additional_protected_paths, additional_allowed_paths):
305		raise PathValidationError(f"Path '{path}' is in a protected area")
306
307		# Check if path is in sandbox or explicitly allowed
308		if not is_path_in_sandbox(normalized_path, sandbox_path, additional_allowed_paths):
309		raise PathValidationError(f"Path '{path}' is outside the allowed directories")
310
311		# Ensure the directory exists
312		directory = os.path.dirname(normalized_path)
313		if not os.path.exists(directory):
314		try:
315		# Create the directory if it doesn't exist
316		os.makedirs(directory, exist_ok=True)
317		except Exception as e:
318		raise PathValidationError(f"Cannot create directory for path '{path}': {str(e)}")
319
320		return normalized_path
321
322
323		def validate_path_for_searching(
324		directory: str,
325		pattern: str,
326		sandbox_path: Optional[str] = None,
327		additional_allowed_paths: Optional[Set[str]] = None,
328		resolve_links: bool = True
329		) -> Tuple[str, str]:
330		"""
331		Validate a directory and pattern for file searching operations (e.g., find_files).
332
333		Args:
334		directory: Directory to search in
335		pattern: File pattern to search for
336		sandbox_path: Base path that defines the sandbox
337		additional_allowed_paths: Additional paths to allow
338		resolve_links: Whether to resolve symlinks
339
340		Returns:
341		tuple: (validated_directory, validated_pattern)
342
343		Raises:
344		PathValidationError: If the directory or pattern does not pass validation
345		"""
346		# Check for empty directory or pattern
347		if not directory:
348		raise PathValidationError("Directory cannot be empty")
349
350		if not pattern:
351		raise PathValidationError("Pattern cannot be empty")
352
353		# Normalize directory path
354		normalized_dir = normalize_path(directory)
355
356		# Check for path traversal attempts in directory
357		if is_path_traversal_attempt(directory):
358		raise PathValidationError(f"Directory '{directory}' contains suspicious traversal patterns")
359
360		# Resolve symlinks if requested
361		if resolve_links:
362		normalized_dir = resolve_symlinks(normalized_dir)
363
364		# Check if directory exists
365		if not os.path.exists(normalized_dir):
366		raise PathValidationError(f"Directory '{directory}' does not exist")
367
368		# Check if path is a directory
369		if not os.path.isdir(normalized_dir):
370		raise PathValidationError(f"Path '{directory}' is not a directory")
371
372		# Check if directory is in sandbox or explicitly allowed
373		if not is_path_in_sandbox(normalized_dir, sandbox_path, additional_allowed_paths):
374		raise PathValidationError(f"Directory '{directory}' is outside the allowed directories")
375
376		# Validate and sanitize the pattern
377		# Remove any path components from the pattern
378		sanitized_pattern = os.path.basename(pattern)
379
380		# Check for suspicious patterns in the pattern
381		suspicious_pattern_chars = r'[&|;`$()<>]'
382		if re.search(suspicious_pattern_chars, sanitized_pattern):
383		raise PathValidationError(f"Pattern '{pattern}' contains suspicious characters")
384
385		return normalized_dir, sanitized_pattern
386
387
388		def configure_path_sandbox(
389		sandbox_path: Optional[str] = None,
390		additional_allowed_paths: Optional[Set[str]] = None,
391		additional_protected_paths: Optional[Set[str]] = None
392		) -> Dict[str, Union[str, Set[str]]]:
393		"""
394		Configure the path sandbox with custom settings.
395
396		Args:
397		sandbox_path: Base path that defines the sandbox
398		additional_allowed_paths: Additional paths to allow
399		additional_protected_paths: Additional paths to protect
400
401		Returns:
402		dict: Configuration dictionary with sandbox settings
403		"""
404		# Start with default values
405		config = {
406		'sandbox_path': sandbox_path or DEFAULT_SANDBOX_PATH,
407		'allowed_paths': DEFAULT_ALLOWED_PATHS.copy(),
408		'protected_paths': DEFAULT_PROTECTED_PATHS.copy()
409		}
410
411		# Add additional allowed paths
412		if additional_allowed_paths:
413		config['allowed_paths'].update(additional_allowed_paths)
414
415		# Add additional protected paths
416		if additional_protected_paths:
417		config['protected_paths'].update(additional_protected_paths)
418
419		return config

ollamatools-project/ollamatools/plugins/README.md +0 -147

…	…	@@ -1,147 +0,0 @@
1		# OllamaTools Plugin System
2
3		This directory contains the core components of the OllamaTools plugin system.
4
5		## Overview
6
7		The plugin system provides a flexible way to extend OllamaTools functionality through plugins. Plugins can be used to add new features, integrate with external services, or customize behavior.
8
9		## Plugin Types
10
11		The plugin system supports several types of plugins:
12
13		- **SharedPlugin**: Plugins that can be used by multiple components
14		- **MatrixPlugin**: Plugins specifically for Matrix integration
15		- **AgentPlugin**: Plugins for extending agent functionality
16
17		## Creating a Plugin
18
19		To create a plugin, you need to:
20
21		1. Create a class that inherits from one of the plugin base classes
22		2. Implement the required methods
23		3. Register the plugin with the plugin registry
24
25		### Example Plugin
26
27		```python
28		from ollamatools.plugins import SharedPlugin
29
30		class MyPlugin(SharedPlugin):
31		"""Example plugin that demonstrates the plugin system."""
32
33		def __init__(self, config=None):
34		super().__init__(config)
35		# Initialize your plugin
36
37		def initialize(self, context):
38		# Set up your plugin with the runtime context
39		pass
40
41		def shutdown(self):
42		# Clean up resources
43		pass
44
45		@property
46		def version(self):
47		return "1.0.0"
48
49		@property
50		def dependencies(self):
51		# List any other plugins this plugin depends on
52		return []
53
54		def get_capabilities(self):
55		# Return a list of capabilities provided by this plugin
56		return ["my_capability"]
57		```
58
59		## Registering a Plugin
60
61		Plugins can be registered in several ways:
62
63		### 1. Direct Registration
64
65		```python
66		from ollamatools.plugins.registry import PluginRegistry
67		from my_plugin import MyPlugin
68
69		# Create a registry
70		registry = PluginRegistry()
71
72		# Register the plugin
73		registry.register_plugin(MyPlugin)
74
75		# Initialize the plugin
76		context = {"some_context": "value"}
77		registry.initialize_plugin("MyPlugin", context)
78		```
79
80		### 2. Entry Points
81
82		You can also register plugins using entry points in your `setup.py`:
83
84		```python
85		setup(
86		name="my_plugin_package",
87		# ...
88		entry_points={
89		"ollamatools.plugins": [
90		"my_plugin = my_plugin:MyPlugin",
91		],
92		},
93		)
94		```
95
96		Then discover plugins:
97
98		```python
99		registry = PluginRegistry()
100		registry.discover_plugins()
101		```
102
103		## Plugin Configuration
104
105		Plugins can be configured using the configuration system:
106
107		```python
108		from ollamatools.config import load_plugin_config
109
110		# Load configuration for a plugin
111		config = load_plugin_config("MyPlugin")
112
113		# Use the configuration
114		value = config.get("some_setting", "default_value")
115		```
116
117		## Error Handling
118
119		The plugin system includes comprehensive error handling:
120
121		```python
122		from ollamatools.plugins.errors import PluginError, handle_plugin_exception
123
124		try:
125		# Do something with a plugin
126		plugin.some_method()
127		except PluginError as e:
128		# Handle plugin-specific errors
129		error_details = handle_plugin_exception(e)
130		print(f"Plugin error: {error_details['error_message']}")
131		```
132
133		## Plugin Lifecycle
134
135		1. **Registration**: Plugin classes are registered with the registry
136		2. **Initialization**: Plugins are instantiated and initialized with context
137		3. **Operation**: Plugins provide functionality through their methods
138		4. **Shutdown**: Plugins clean up resources when no longer needed
139
140		## Best Practices
141
142		1. Keep plugins focused on a single responsibility
143		2. Document plugin capabilities and configuration options
144		3. Handle errors gracefully
145		4. Clean up resources in the shutdown method
146		5. Use dependency injection through the context parameter
147		6. Follow the plugin interface contract

ollamatools-project/ollamatools/plugins/__init__.py +0 -177

…	…	@@ -1,177 +0,0 @@
1		"""
2		OllamaTools Plugin System.
3
4		This module provides the base interfaces and registry for the OllamaTools plugin system.
5		"""
6
7		import os
8		import sys
9		import importlib
10		import logging
11		from typing import Dict, Any, List, Optional, Set, Type, Callable, Union
12
13		# Configure logging
14		logger = logging.getLogger(__name__)
15
16		class OllamaToolsPlugin:
17		"""Base interface for all OllamaTools plugins."""
18
19		def __init__(self, config=None):
20		"""
21		Initialize the plugin with optional configuration.
22
23		Args:
24		config: Optional configuration dictionary for the plugin
25		"""
26		self.config = config or {}
27
28		def initialize(self, context):
29		"""
30		Initialize the plugin with the runtime context.
31
32		Args:
33		context: Runtime context dictionary containing references to other components
34		"""
35		pass
36
37		def shutdown(self):
38		"""Clean up resources when shutting down."""
39		pass
40
41		@property
42		def name(self):
43		"""
44		Get the plugin name.
45
46		Returns:
47		str: The plugin name
48		"""
49		return self.__class__.__name__
50
51		@property
52		def version(self):
53		"""
54		Get the plugin version.
55
56		Returns:
57		str: The plugin version
58		"""
59		return "1.0.0"
60
61		@property
62		def dependencies(self):
63		"""
64		Get plugin dependencies.
65
66		Returns:
67		List[str]: List of plugin names that this plugin depends on
68		"""
69		return []
70
71
72		class MatrixPlugin(OllamaToolsPlugin):
73		"""Base class for Matrix plugins."""
74
75		async def on_message(self, room, event, client):
76		"""
77		Handle an incoming Matrix message.
78
79		Args:
80		room: The room the message was sent in
81		event: The message event
82		client: The Matrix client
83
84		Returns:
85		bool: True if the message was handled, False otherwise
86		"""
87		return False
88
89		def get_commands(self):
90		"""
91		Get Matrix commands provided by this plugin.
92
93		Returns:
94		Dict[str, Callable]: Dictionary of command names to handler functions
95		"""
96		return {}
97
98		async def on_room_join(self, room, client):
99		"""
100		Handle room join event.
101
102		Args:
103		room: The room that was joined
104		client: The Matrix client
105		"""
106		pass
107
108		async def on_client_start(self, client):
109		"""
110		Handle client start event.
111
112		Args:
113		client: The Matrix client
114		"""
115		pass
116
117		async def on_client_stop(self, client):
118		"""
119		Handle client stop event.
120
121		Args:
122		client: The Matrix client
123		"""
124		pass
125
126
127		class AgentPlugin(OllamaToolsPlugin):
128		"""Base class for agent plugins."""
129
130		def process_message(self, message, context):
131		"""
132		Process a message and return a response.
133
134		Args:
135		message: The user message
136		context: Processing context
137
138		Returns:
139		Optional[str]: Response message if handled, None otherwise
140		"""
141		return None
142
143		def get_commands(self):
144		"""
145		Get commands provided by this plugin.
146
147		Returns:
148		Dict[str, Callable]: Dictionary of command names to handler functions
149		"""
150		return {}
151
152		def on_function_call(self, function_name, args, context):
153		"""
154		Handle a function call.
155
156		Args:
157		function_name: The name of the function being called
158		args: The arguments to the function
159		context: Processing context
160
161		Returns:
162		Any: The result of the function call
163		"""
164		return None
165
166
167		class SharedPlugin(OllamaToolsPlugin):
168		"""Base class for plugins that can be used by multiple components."""
169
170		def get_capabilities(self):
171		"""
172		Get capabilities provided by this plugin.
173
174		Returns:
175		List[str]: List of capability names
176		"""
177		return []

ollamatools-project/ollamatools/plugins/errors.py +0 -142

…	…	@@ -1,142 +0,0 @@
1		"""
2		Error handling for the OllamaTools plugin system.
3
4		This module provides exception classes and utility functions for handling
5		errors in the OllamaTools plugin system.
6		"""
7
8		import sys
9		import logging
10		import traceback
11		from typing import Dict, Any, Optional
12
13		# Configure logging
14		logger = logging.getLogger(__name__)
15
16		class PluginError(Exception):
17		"""Base class for all plugin-related errors."""
18
19		def __init__(self, message, details=None):
20		"""
21		Initialize the error.
22
23		Args:
24		message: Error message
25		details: Optional dictionary with additional error details
26		"""
27		super().__init__(message)
28		self.details = details or {}
29
30
31		class PluginNotFoundError(PluginError):
32		"""Error raised when a plugin is not found."""
33		pass
34
35
36		class PluginInitializationError(PluginError):
37		"""Error raised when a plugin fails to initialize."""
38		pass
39
40
41		class PluginDependencyError(PluginError):
42		"""Error raised when a plugin dependency cannot be satisfied."""
43		pass
44
45
46		class PluginExecutionError(PluginError):
47		"""Error raised when a plugin operation fails during execution."""
48		pass
49
50
51		class PluginConfigurationError(PluginError):
52		"""Error raised when a plugin has invalid configuration."""
53		pass
54
55
56		def format_plugin_error(error: PluginError) -> str:
57		"""
58		Format a plugin error into a user-friendly message.
59
60		Args:
61		error: The plugin error
62
63		Returns:
64		str: Formatted error message
65		"""
66		message = str(error)
67
68		if isinstance(error, PluginNotFoundError):
69		return f"Plugin not found: {message}"
70
71		if isinstance(error, PluginInitializationError):
72		return f"Failed to initialize plugin: {message}"
73
74		if isinstance(error, PluginDependencyError):
75		dependency = error.details.get("dependency", "unknown")
76		return f"Plugin dependency error: {message} (missing: {dependency})"
77
78		if isinstance(error, PluginExecutionError):
79		plugin = error.details.get("plugin", "unknown")
80		operation = error.details.get("operation", "unknown operation")
81		return f"Plugin execution error in {plugin}.{operation}: {message}"
82
83		if isinstance(error, PluginConfigurationError):
84		plugin = error.details.get("plugin", "unknown")
85		return f"Invalid configuration for plugin {plugin}: {message}"
86
87		# Generic plugin error
88		return f"Plugin error: {message}"
89
90
91		def log_plugin_error(error: PluginError, level=logging.ERROR) -> None:
92		"""
93		Log a plugin error with appropriate context.
94
95		Args:
96		error: The plugin error
97		level: Logging level
98		"""
99		# Format the error message
100		message = format_plugin_error(error)
101
102		# Add details if available
103		if error.details:
104		details_str = ", ".join(f"{k}={v}" for k, v in error.details.items())
105		message = f"{message} [Details: {details_str}]"
106
107		# Log the error
108		logger.log(level, message, exc_info=True)
109
110
111		def handle_plugin_exception(error: Exception) -> Dict[str, Any]:
112		"""
113		Handle a plugin exception and return error details.
114
115		Args:
116		error: The exception to handle
117
118		Returns:
119		Dict[str, Any]: Error details
120		"""
121		error_type = type(error).__name__
122		error_message = str(error)
123		error_traceback = traceback.format_exc()
124
125		# Log the error
126		if isinstance(error, PluginError):
127		log_plugin_error(error)
128		else:
129		logger.error(f"Unexpected error: {error_type}: {error_message}", exc_info=True)
130
131		# Create error details
132		error_details = {
133		"error_type": error_type,
134		"error_message": error_message,
135		"traceback": error_traceback
136		}
137
138		# Add plugin-specific details
139		if isinstance(error, PluginError) and error.details:
140		error_details["details"] = error.details
141
142		return error_details

ollamatools-project/ollamatools/plugins/registry.py +0 -198

…	…	@@ -1,198 +0,0 @@
1		"""
2		Plugin Registry for OllamaTools.
3
4		This module provides the PluginRegistry class for managing OllamaTools plugins.
5		"""
6
7		import os
8		import sys
9		import importlib
10		import logging
11		import pkg_resources
12		from typing import Dict, Any, List, Optional, Set, Type, Callable, Union
13
14		from ollamatools.plugins import OllamaToolsPlugin
15		from ollamatools.plugins.errors import (
16		PluginError,
17		PluginInitializationError,
18		PluginDependencyError,
19		PluginNotFoundError,
20		handle_plugin_exception
21		)
22
23		# Configure logging
24		logger = logging.getLogger(__name__)
25
26		class PluginRegistry:
27		"""Registry for OllamaTools plugins."""
28
29		def __init__(self):
30		"""Initialize the plugin registry."""
31		self.plugins = {}
32		self.initialized_plugins = {}
33
34		def register_plugin(self, plugin_class, config=None):
35		"""
36		Register a plugin class.
37
38		Args:
39		plugin_class: The plugin class to register
40		config: Optional configuration for the plugin
41
42		Returns:
43		PluginRegistry: Self for method chaining
44		"""
45		plugin_name = plugin_class.__name__
46		self.plugins[plugin_name] = {
47		"class": plugin_class,
48		"config": config
49		}
50		logger.debug(f"Registered plugin: {plugin_name}")
51		return self
52
53		def discover_plugins(self):
54		"""
55		Discover plugins using entry points.
56
57		Returns:
58		PluginRegistry: Self for method chaining
59		"""
60		try:
61		for entry_point in pkg_resources.iter_entry_points('ollamatools.plugins'):
62		try:
63		plugin_class = entry_point.load()
64		self.register_plugin(plugin_class)
65		logger.info(f"Discovered plugin: {plugin_class.__name__} from entry point {entry_point.name}")
66		except Exception as e:
67		logger.error(f"Error loading plugin from entry point {entry_point.name}: {e}")
68		except Exception as e:
69		logger.error(f"Error discovering plugins: {e}")
70
71		return self
72
73		def initialize_plugin(self, plugin_name, context):
74		"""
75		Initialize a plugin and its dependencies.
76
77		Args:
78		plugin_name: The name of the plugin to initialize
79		context: Runtime context for the plugin
80
81		Returns:
82		OllamaToolsPlugin: The initialized plugin instance
83
84		Raises:
85		PluginNotFoundError: If the plugin is not found
86		PluginInitializationError: If the plugin fails to initialize
87		PluginDependencyError: If a dependency cannot be initialized
88		"""
89		# Return already initialized plugin
90		if plugin_name in self.initialized_plugins:
91		return self.initialized_plugins[plugin_name]
92
93		# Check if plugin exists
94		if plugin_name not in self.plugins:
95		raise PluginNotFoundError(f"Plugin {plugin_name} not found")
96
97		logger.debug(f"Initializing plugin: {plugin_name}")
98
99		plugin_info = self.plugins[plugin_name]
100		plugin_class = plugin_info["class"]
101		plugin_config = plugin_info["config"]
102
103		try:
104		# Create plugin instance
105		plugin_instance = plugin_class(plugin_config)
106
107		# Initialize dependencies first
108		for dependency in plugin_instance.dependencies:
109		try:
110		self.initialize_plugin(dependency, context)
111		except PluginError as e:
112		raise PluginDependencyError(
113		f"Failed to initialize dependency {dependency} for plugin {plugin_name}",
114		details={"plugin": plugin_name, "dependency": dependency, "original_error": str(e)}
115		) from e
116
117		# Initialize the plugin
118		plugin_instance.initialize(context)
119		self.initialized_plugins[plugin_name] = plugin_instance
120
121		logger.info(f"Initialized plugin: {plugin_name}")
122		return plugin_instance
123
124		except PluginError:
125		# Re-raise plugin errors
126		raise
127		except Exception as e:
128		# Wrap other exceptions
129		raise PluginInitializationError(
130		f"Failed to initialize plugin {plugin_name}: {str(e)}",
131		details={"plugin": plugin_name}
132		) from e
133
134		def initialize_all(self, context):
135		"""
136		Initialize all registered plugins.
137
138		Args:
139		context: Runtime context for the plugins
140
141		Returns:
142		Dict[str, OllamaToolsPlugin]: Dictionary of initialized plugins
143		"""
144		for plugin_name in list(self.plugins.keys()):
145		try:
146		self.initialize_plugin(plugin_name, context)
147		except PluginError as e:
148		logger.error(f"Error initializing plugin {plugin_name}: {e}")
149		handle_plugin_exception(e)
150		except Exception as e:
151		logger.error(f"Unexpected error initializing plugin {plugin_name}: {e}")
152
153		return self.initialized_plugins
154
155		def get_plugin(self, plugin_name):
156		"""
157		Get an initialized plugin.
158
159		Args:
160		plugin_name: The name of the plugin to get
161
162		Returns:
163		Optional[OllamaToolsPlugin]: The plugin instance or None if not found
164		"""
165		return self.initialized_plugins.get(plugin_name)
166
167		def get_plugins_by_type(self, plugin_type):
168		"""
169		Get all initialized plugins of a specific type.
170
171		Args:
172		plugin_type: The plugin type class
173
174		Returns:
175		Dict[str, OllamaToolsPlugin]: Dictionary of plugin names to instances
176		"""
177		return {
178		name: plugin for name, plugin in self.initialized_plugins.items()
179		if isinstance(plugin, plugin_type)
180		}
181
182		def shutdown(self):
183		"""
184		Shutdown all initialized plugins.
185
186		Returns:
187		PluginRegistry: Self for method chaining
188		"""
189		for plugin_name, plugin in list(self.initialized_plugins.items()):
190		try:
191		logger.debug(f"Shutting down plugin: {plugin_name}")
192		plugin.shutdown()
193		logger.info(f"Shut down plugin: {plugin_name}")
194		except Exception as e:
195		logger.error(f"Error shutting down plugin {plugin_name}: {e}")
196
197		self.initialized_plugins = {}
198		return self

ollamatools-project/ollamatools/security.py +0 -419

…	…	@@ -1,419 +0,0 @@
1		"""
2		Permission management system for OllamaTools.
3
4		This module provides security controls for executing functions,
5		with a focus on a prompt-based permission system.
6		"""
7		import os
8		from typing import Dict, Any, Optional, Tuple, Set
9		import logging
10		import re
11
12		from ollamatools.errors import PermissionError, ERROR_CODES
13
14		# Set up logging
15		logger = logging.getLogger(__name__)
16
17		class PermissionManager:
18		"""
19		Manages security permissions for function execution.
20
21		The PermissionManager controls access to potentially sensitive
22		operations such as executing system commands or making web requests.
23		It provides a prompt-based approach to security, allowing users to
24		approve or deny operations at runtime.
25
26		Attributes:
27		prompt_enabled (bool): Whether to prompt for permission before operations
28		sudo_always_prompt (bool): Whether to always prompt for sudo commands
29		session_permissions (Set[str]): Set of operations allowed for the session
30		"""
31
32		def __init__(
33		self,
34		prompt_enabled: bool = True,
35		sudo_always_prompt: bool = True
36		):
37		"""
38		Initialize the permission manager.
39
40		Args:
41		prompt_enabled: Whether to prompt for permissions
42		sudo_always_prompt: Whether to always prompt for sudo commands
43		"""
44		self.prompt_enabled = prompt_enabled
45		self.sudo_always_prompt = sudo_always_prompt
46		self.session_permissions = set() # Store allowed operations for the session
47		self.session_permission_patterns = {
48		"read_file": {}, # Directory-based permissions
49		"write_file": {}, # Directory-based permissions
50		"execute_command": {} # Command-based permissions
51		}
52
53		logger.debug("PermissionManager initialized with prompt_enabled=%s, sudo_always_prompt=%s",
54		prompt_enabled, sudo_always_prompt)
55
56		def _generate_permission_key(self, function_name: str, args: Dict[str, Any]) -> str:
57		"""
58		Generate a more flexible permission key that recognizes similar operations.
59
60		Args:
61		function_name: Name of the function
62		args: Function arguments
63
64		Returns:
65		str: Permission key
66		"""
67		# For commands, focus on the command name rather than all arguments
68		if function_name == "execute_command" and "command" in args:
69		command = args["command"]
70		# Just use the base command without all arguments for matching
71		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
72		return f"{function_name}:command={base_command}"
73
74		# For file operations, focus on the operation type and parent directory
75		if function_name in ("read_file", "write_file") and "path" in args:
76		path = args["path"]
77		# Use parent directory for path-based operations to match similar files
78		parent_dir = os.path.dirname(os.path.abspath(path))
79		return f"{function_name}:dir={parent_dir}"
80
81		# For other operations, use the full args as before
82		args_str = ', '.join(f"{k}={v!r}" for k, v in args.items())
83		return f"{function_name}:{args_str}"
84
85		def request_permission(
86		self,
87		function_name: str,
88		args: Dict[str, Any],
89		is_sudo: bool = False
90		) -> bool:
91		"""
92		Request permission to execute a function.
93
94		This method handles the permission request workflow, which may involve
95		prompting the user for approval depending on configuration settings.
96
97		Args:
98		function_name: Name of the function
99		args: Function arguments
100		is_sudo: Whether the command uses sudo
101
102		Returns:
103		bool: True if permission granted, False otherwise
104
105		Raises:
106		PermissionError: If there's an issue with the permission request
107		"""
108		# Format the arguments as a readable string
109		args_str = ', '.join(f"{k}={v!r}" for k, v in args.items())
110
111		# Generate a permission key for the operation
112		permission_key = self._generate_permission_key(function_name, args)
113
114		# Check if we need to prompt for permission
115		should_prompt = self.prompt_enabled
116
117		# Always prompt for sudo if sudo_always_prompt is True
118		if is_sudo and self.sudo_always_prompt:
119		should_prompt = True
120		logger.info("Forcing permission prompt for sudo command")
121
122		# Check if this operation is already allowed for the session
123		if permission_key in self.session_permissions:
124		logger.debug("Permission already granted for session: %s", permission_key)
125		return True
126
127		# Check specific session permission patterns
128		if function_name == "execute_command" and "command" in args:
129		command = args["command"]
130		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
131		if base_command in self.session_permission_patterns["execute_command"]:
132		logger.debug("Permission already granted for command type: %s", base_command)
133		return True
134
135		elif function_name in ("read_file", "write_file") and "path" in args:
136		path = args["path"]
137		parent_dir = os.path.dirname(os.path.abspath(path))
138		if parent_dir in self.session_permission_patterns[function_name]:
139		logger.debug("Permission already granted for directory: %s", parent_dir)
140		return True
141
142		# If prompting is disabled and not overridden by sudo, grant permission
143		if not should_prompt:
144		logger.debug("Permission prompts disabled, automatically granting permission for: %s", permission_key)
145		return True
146
147		# Display the permission prompt
148		print("\n" + "="*60)
149		print("PERMISSION REQUEST")
150		print("="*60)
151		print(f"Function: {function_name}")
152		print(f"Arguments: {args_str}")
153
154		if is_sudo:
155		print("\nWARNING: This operation requires elevated permissions (sudo)")
156		print(" Be cautious when granting permission for sudo operations")
157
158		print("\nOptions:")
159		print(" [A] Allow: Grant permission for this operation")
160		print(" [S] Allow for Session: Grant permission for similar operations in this session")
161		print(" [M] Modify: Modify the operation before execution")
162		print(" [D] Deny: Reject the operation")
163
164		while True:
165		try:
166		response = input("\nYour choice [A/S/M/D]: ").strip().upper()
167
168		if response == 'A': # Allow once
169		logger.info("Permission granted for: %s", permission_key)
170		return True
171
172		elif response == 'S': # Allow for session
173		self.session_permissions.add(permission_key)
174
175		# Store more specific pattern matches for better session handling
176		if function_name == "execute_command" and "command" in args:
177		command = args["command"]
178		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
179		self.session_permission_patterns["execute_command"][base_command] = True
180		logger.info(f"Command '{base_command}' allowed for session")
181
182		elif function_name in ("read_file", "write_file") and "path" in args:
183		path = args["path"]
184		parent_dir = os.path.dirname(os.path.abspath(path))
185		self.session_permission_patterns[function_name][parent_dir] = True
186		logger.info(f"{function_name} allowed for directory: {parent_dir}")
187
188		logger.info("Permission granted for session: %s", permission_key)
189		return True
190
191		elif response == 'M': # Modify
192		modified_args = self.modify_command(function_name, args)
193		if modified_args is not None:
194		# Generate a new permission key for the modified args
195		modified_args_str = ', '.join(f"{k}={v!r}" for k, v in modified_args.items())
196		modified_key = f"{function_name}:{modified_args_str}"
197
198		# Add to session permissions if it's a modification of an existing permission
199		if permission_key in self.session_permissions:
200		self.session_permissions.add(modified_key)
201
202		# Check if sudo status has changed
203		new_is_sudo = False
204		if function_name == "execute_command" and "command" in modified_args:
205		new_is_sudo = self.detect_sudo(modified_args["command"])
206
207		# Call request_permission again with modified arguments
208		logger.info("Requesting permission for modified operation: %s", modified_key)
209		return self.request_permission(
210		function_name,
211		modified_args,
212		is_sudo=new_is_sudo
213		)
214		else:
215		logger.info("Modification cancelled for: %s", permission_key)
216		print("Modification cancelled.")
217		return False
218
219		elif response == 'D': # Deny
220		logger.info("Permission denied for: %s", permission_key)
221		print("Permission denied.")
222		return False
223
224		else:
225		print("Invalid choice. Please enter A, S, M, or D.")
226		except KeyboardInterrupt:
227		logger.warning("Permission request interrupted")
228		print("\nOperation cancelled.")
229		raise PermissionError(
230		"Permission request interrupted by user",
231		function_name=function_name,
232		args=args,
233		code=ERROR_CODES.get('PERMISSION_PROMPT_CANCELLED')
234		)
235
236		def modify_command(
237		self,
238		function_name: str,
239		args: Dict[str, Any]
240		) -> Optional[Dict[str, Any]]:
241		"""
242		Allow the user to modify command arguments.
243
244		Args:
245		function_name: Name of the function
246		args: Current function arguments
247
248		Returns:
249		Optional[Dict[str, Any]]: Modified arguments or None if cancelled
250
251		Raises:
252		PermissionError: If there's an issue with the modification process
253		"""
254		print("\n" + "="*60)
255		print("MODIFY OPERATION")
256		print("="*60)
257		print(f"Function: {function_name}")
258		print("Current arguments:")
259
260		# Create a copy of the arguments that will be modified
261		modified_args = args.copy()
262
263		try:
264		# Present each argument for modification
265		for key, value in args.items():
266		while True:
267		new_value = input(f" {key} [{value}] (Press Enter to keep, 'C' to cancel): ")
268
269		if not new_value: # Keep current value
270		break
271		elif new_value.upper() == 'C': # Cancel the modification
272		return None
273		else:
274		# Try to convert to the same type as the original value
275		try:
276		# Handle different types appropriately
277		if isinstance(value, bool):
278		# Convert string to boolean
279		if new_value.lower() in ('true', 'yes', '1', 'y'):
280		modified_args[key] = True
281		elif new_value.lower() in ('false', 'no', '0', 'n'):
282		modified_args[key] = False
283		else:
284		print(f"Invalid boolean value. Use 'true', 'false', 'yes', 'no', '1', '0', 'y', or 'n'.")
285		continue
286		elif isinstance(value, int):
287		modified_args[key] = int(new_value)
288		elif isinstance(value, float):
289		modified_args[key] = float(new_value)
290		else:
291		# For strings and other types, use the new value as is
292		modified_args[key] = new_value
293		break
294		except ValueError:
295		print(f"Invalid value for {key}. Expected type: {type(value).__name__}")
296
297		# Show the modified arguments for confirmation
298		print("\nModified arguments:")
299		for key, value in modified_args.items():
300		print(f" {key}: {value}")
301
302		# Confirm the modifications
303		confirm = input("\nConfirm these modifications? [Y/n]: ").strip().lower()
304		if confirm in ('', 'y', 'yes'):
305		logger.info("Modified args confirmed for function: %s", function_name)
306		return modified_args
307		else:
308		logger.info("Modified args rejected for function: %s", function_name)
309		return None
310
311		except KeyboardInterrupt:
312		logger.warning("Command modification interrupted")
313		print("\nModification cancelled.")
314		return None
315		except Exception as e:
316		logger.error("Error modifying command: %s", str(e))
317		raise PermissionError(
318		f"Error modifying command: {str(e)}",
319		function_name=function_name,
320		args=args
321		)
322
323		def detect_sudo(self, command: str) -> bool:
324		"""
325		Detect if a command uses sudo.
326
327		Args:
328		command: Command string
329
330		Returns:
331		bool: True if command uses sudo
332		"""
333		# Basic detection for sudo command
334		if not command:
335		return False
336
337		# Split the command to handle commands with arguments
338		cmd_parts = command.strip().split()
339		if not cmd_parts:
340		return False
341
342		# Check if the command starts with sudo
343		if cmd_parts[0].lower() == "sudo":
344		logger.info("Sudo command detected: %s", command)
345		return True
346
347		# More advanced detection - check for sudo within a command string
348		sudo_pattern = r'(^|\s+)sudo(\s+|$)'
349		if re.search(sudo_pattern, command):
350		logger.info("Sudo pattern detected in command: %s", command)
351		return True
352
353		return False
354
355		def clear_session_permissions(self):
356		"""
357		Clear all session permissions.
358		"""
359		num_permissions = len(self.session_permissions)
360		self.session_permissions.clear()
361		# Clear pattern-based permissions too
362		for key in self.session_permission_patterns:
363		self.session_permission_patterns[key] = {}
364		logger.info("Cleared %d session permissions", num_permissions)
365
366		def add_session_permission(self, function_name: str, args: Dict[str, Any]):
367		"""
368		Add a specific operation to the session permissions.
369
370		Args:
371		function_name: Name of the function
372		args: Function arguments
373		"""
374		permission_key = self._generate_permission_key(function_name, args)
375		self.session_permissions.add(permission_key)
376
377		# Add to pattern-based permissions
378		if function_name == "execute_command" and "command" in args:
379		command = args["command"]
380		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
381		self.session_permission_patterns["execute_command"][base_command] = True
382
383		elif function_name in ("read_file", "write_file") and "path" in args:
384		path = args["path"]
385		parent_dir = os.path.dirname(os.path.abspath(path))
386		self.session_permission_patterns[function_name][parent_dir] = True
387
388		logger.info("Added session permission: %s", permission_key)
389
390		def has_session_permission(self, function_name: str, args: Dict[str, Any]) -> bool:
391		"""
392		Check if an operation has session permission.
393
394		Args:
395		function_name: Name of the function
396		args: Function arguments
397
398		Returns:
399		bool: True if the operation has session permission
400		"""
401		# Check specific permission key
402		permission_key = self._generate_permission_key(function_name, args)
403		if permission_key in self.session_permissions:
404		return True
405
406		# Check pattern-based permissions
407		if function_name == "execute_command" and "command" in args:
408		command = args["command"]
409		base_command = command.split()[0] if isinstance(command, str) and " " in command else command
410		if base_command in self.session_permission_patterns["execute_command"]:
411		return True
412
413		elif function_name in ("read_file", "write_file") and "path" in args:
414		path = args["path"]
415		parent_dir = os.path.dirname(os.path.abspath(path))
416		if parent_dir in self.session_permission_patterns[function_name]:
417		return True
418
419		return False

ollamatools-project/ollamatools/tools/__init__.py +0 -86

…	…	@@ -1,86 +0,0 @@
1		"""
2		OllamaTools function libraries.
3
4		This package contains various tool modules for use with OllamaTools:
5		- system_tools: Execute Linux commands and file operations
6		- web_tools: Make web requests and process responses
7		- doc_tools: Generate and process documentation
8		- matrix_tools: Connect to Matrix homeservers and chat rooms
9		- enhanced_matrix_tools: Enhanced Matrix client with modular agent functionality
10		"""
11
12		from ollamatools.tools.system_tools import (
13		execute_command,
14		read_file,
15		write_file,
16		find_files
17		)
18
19		from ollamatools.tools.web_tools import (
20		fetch_url,
21		get_weather,
22		search_web
23		)
24
25		from ollamatools.tools.doc_tools import (
26		generate_markdown,
27		extract_context,
28		summarize_text
29		)
30
31		from ollamatools.tools.matrix_tools import (
32		MatrixClient,
33		MatrixConfig,
34		connect_to_matrix,
35		join_matrix_room,
36		send_matrix_message,
37		listen_for_matrix_messages,
38		load_matrix_config,
39		save_matrix_config,
40		create_default_matrix_config
41		)
42
43		from ollamatools.tools.enhanced_matrix_tools import (
44		EnhancedMatrixClient,
45		connect_enhanced_matrix,
46		join_enhanced_matrix_room,
47		send_enhanced_matrix_message,
48		listen_for_enhanced_matrix_messages,
49		create_enhanced_matrix_config,
50		disconnect_enhanced_matrix,
51		process_with_enhanced_ollama,
52		MODULAR_AGENT_AVAILABLE,
53		OPENSEARCH_AVAILABLE
54		)
55
56		__all__ = [
57		'execute_command',
58		'read_file',
59		'write_file',
60		'find_files',
61		'fetch_url',
62		'get_weather',
63		'search_web',
64		'generate_markdown',
65		'extract_context',
66		'summarize_text',
67		'MatrixClient',
68		'MatrixConfig',
69		'connect_to_matrix',
70		'join_matrix_room',
71		'send_matrix_message',
72		'listen_for_matrix_messages',
73		'load_matrix_config',
74		'save_matrix_config',
75		'create_default_matrix_config',
76		'EnhancedMatrixClient',
77		'connect_enhanced_matrix',
78		'join_enhanced_matrix_room',
79		'send_enhanced_matrix_message',
80		'listen_for_enhanced_matrix_messages',
81		'create_enhanced_matrix_config',
82		'disconnect_enhanced_matrix',
83		'process_with_enhanced_ollama',
84		'MODULAR_AGENT_AVAILABLE',
85		'OPENSEARCH_AVAILABLE'
86		]

ollamatools-project/ollamatools/tools/doc_tools.py +0 -278

…	…	@@ -1,278 +0,0 @@
1		"""
2		Documentation Tools for OllamaTools.
3
4		This module provides utilities for generating and processing documentation.
5		It includes tools for creating formatted documentation, extracting relevant
6		context from documents, and summarizing text.
7
8		All functions support integration with the permission system for security.
9		"""
10
11		import markdown
12		import re
13		from collections import Counter
14		from typing import Optional, Dict, Any, List
15
16
17		def generate_markdown(
18		content: str,
19		title: str,
20		format_type: str = 'markdown',
21		permission_manager = None
22		) -> str:
23		"""
24		Generate formatted documentation.
25
26		Args:
27		content: Document content to format
28		title: Document title
29		format_type: Output format ('markdown' or 'html')
30		permission_manager: Optional permission manager for security checks
31
32		Returns:
33		str: Formatted documentation
34
35		Raises:
36		ValueError: If format_type is not supported
37		PermissionError: If permission is denied
38		"""
39		# Permission check
40		if permission_manager is not None:
41		args = {
42		'content_preview': content[:50] + ('...' if len(content) > 50 else ''),
43		'title': title,
44		'format_type': format_type
45		}
46		if not permission_manager.request_permission('generate_markdown', args):
47		raise PermissionError("Permission denied to generate markdown document")
48
49		# Input validation
50		if not content or not isinstance(content, str):
51		raise ValueError("Content must be a non-empty string")
52
53		if not title or not isinstance(title, str):
54		raise ValueError("Title must be a non-empty string")
55
56		if format_type not in ['markdown', 'html']:
57		raise ValueError("Format type must be 'markdown' or 'html'")
58
59		# Create the formatted document
60		if format_type == 'markdown':
61		# For markdown, we'll create a simple markdown document with a title
62		formatted_doc = f"# {title}\n\n{content}"
63		return formatted_doc
64		else: # html
65		# For HTML, we'll convert the markdown to HTML
66		html_content = markdown.markdown(content)
67		formatted_doc = f"""<!DOCTYPE html>
68		<html>
69		<head>
70		<title>{title}</title>
71		<meta charset="UTF-8">
72		<meta name="viewport" content="width=device-width, initial-scale=1.0">
73		</head>
74		<body>
75		<h1>{title}</h1>
76		{html_content}
77		</body>
78		</html>"""
79		return formatted_doc
80
81
82		def extract_context(
83		query: str,
84		documents: List[str],
85		max_length: int = 1000,
86		permission_manager = None
87		) -> str:
88		"""
89		Extract relevant context from documents based on a query.
90
91		This function extracts passages from documents that are most relevant to the query
92		based on keyword matching. It prioritizes sentences containing query keywords.
93
94		Args:
95		query: Search query to find relevant contexts
96		documents: List of document contents
97		max_length: Maximum total length of extracted context
98		permission_manager: Optional permission manager for security checks
99
100		Returns:
101		str: Extracted context
102
103		Raises:
104		ValueError: If inputs are invalid
105		PermissionError: If permission is denied
106		"""
107		# Permission check
108		if permission_manager is not None:
109		args = {
110		'query': query,
111		'documents_count': len(documents) if documents else 0,
112		'max_length': max_length
113		}
114		if not permission_manager.request_permission('extract_context', args):
115		raise PermissionError("Permission denied to extract context from documents")
116
117		# Input validation
118		if not query or not isinstance(query, str):
119		raise ValueError("Query must be a non-empty string")
120
121		if not documents or not isinstance(documents, list):
122		raise ValueError("Documents must be a non-empty list of strings")
123
124		if not all(isinstance(doc, str) for doc in documents):
125		raise ValueError("All documents must be strings")
126
127		if not max_length or not isinstance(max_length, int) or max_length <= 0:
128		raise ValueError("Max length must be a positive integer")
129
130		# Extract keywords from the query
131		# Normalize by removing common words and punctuation
132		query = query.lower()
133		query_words = re.findall(r'\b\w+\b', query)
134
135		# Remove common stop words
136		stop_words = {'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are', 'of', 'to', 'in',
137		'for', 'with', 'by', 'at', 'on', 'from'}
138		keywords = [word for word in query_words if word not in stop_words]
139
140		if not keywords:
141		# Use the original words if all were stop words
142		keywords = query_words
143
144		# Find sentences in documents containing keywords
145		relevant_sentences = []
146		for doc in documents:
147		# Split documents into sentences (simple heuristic)
148		sentences = re.split(r'(?<=[.!?])\s+', doc)
149
150		for sentence in sentences:
151		# Check for keyword presence
152		sentence_lower = sentence.lower()
153		sentence_words = set(re.findall(r'\b\w+\b', sentence_lower))
154
155		# Calculate relevance score as the number of keywords found
156		keywords_found = sum(1 for keyword in keywords if keyword in sentence_words)
157
158		if keywords_found > 0:
159		relevant_sentences.append((sentence, keywords_found))
160
161		# Sort sentences by relevance (number of keywords found)
162		relevant_sentences.sort(key=lambda x: x[1], reverse=True)
163
164		# Build context up to max_length
165		context = []
166		current_length = 0
167
168		for sentence, _ in relevant_sentences:
169		if current_length + len(sentence) + 1 <= max_length: # +1 for space
170		context.append(sentence)
171		current_length += len(sentence) + 1
172		else:
173		# If we can't fit the next sentence, stop
174		break
175
176		# If no relevant sentences found, return empty string
177		if not context:
178		return ""
179
180		return " ".join(context)
181
182
183		def summarize_text(
184		text: str,
185		max_length: int = 500,
186		permission_manager = None
187		) -> str:
188		"""
189		Generate a summary of text using extractive summarization.
190
191		This function extracts the most important sentences from the text
192		based on word frequency analysis.
193
194		Args:
195		text: Text to summarize
196		max_length: Maximum summary length
197		permission_manager: Optional permission manager for security checks
198
199		Returns:
200		str: Text summary
201
202		Raises:
203		ValueError: If inputs are invalid
204		PermissionError: If permission is denied
205		"""
206		# Permission check
207		if permission_manager is not None:
208		args = {
209		'text_preview': text[:50] + ('...' if len(text) > 50 else ''),
210		'text_length': len(text),
211		'max_length': max_length
212		}
213		if not permission_manager.request_permission('summarize_text', args):
214		raise PermissionError("Permission denied to summarize text")
215
216		# Input validation
217		if not text or not isinstance(text, str):
218		raise ValueError("Text must be a non-empty string")
219
220		if not max_length or not isinstance(max_length, int) or max_length <= 0:
221		raise ValueError("Max length must be a positive integer")
222
223		# If text is already shorter than max_length, return it as is
224		if len(text) <= max_length:
225		return text
226
227		# Split text into sentences
228		sentences = re.split(r'(?<=[.!?])\s+', text)
229
230		# Handle case with only one sentence
231		if len(sentences) <= 1:
232		return text[:max_length] + "..." if len(text) > max_length else text
233
234		# Extract all words, lowercase them, and remove stop words
235		stop_words = {'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are', 'of', 'to', 'in',
236		'for', 'with', 'by', 'at', 'on', 'from'}
237
238		words = []
239		for sentence in sentences:
240		sentence_words = re.findall(r'\b\w+\b', sentence.lower())
241		words.extend([word for word in sentence_words if word not in stop_words])
242
243		# Count word frequencies
244		word_freq = Counter(words)
245
246		# Score sentences based on word frequencies
247		sentence_scores = []
248		for i, sentence in enumerate(sentences):
249		sentence_words = re.findall(r'\b\w+\b', sentence.lower())
250		score = sum(word_freq[word] for word in sentence_words if word not in stop_words)
251		# Normalize by sentence length to avoid bias towards longer sentences
252		if len(sentence_words) > 0:
253		score = score / len(sentence_words)
254		sentence_scores.append((i, sentence, score))
255
256		# Sort sentences by score
257		sentence_scores.sort(key=lambda x: x[2], reverse=True)
258
259		# Select top sentences and restore original order
260		top_sentences = sentence_scores[:len(sentences)//3 + 1] # Take ~1/3 of sentences
261		top_sentences.sort(key=lambda x: x[0]) # Sort by original position
262
263		# Build summary up to max_length
264		summary = []
265		current_length = 0
266
267		for _, sentence, _ in top_sentences:
268		if current_length + len(sentence) + 1 <= max_length: # +1 for space
269		summary.append(sentence)
270		current_length += len(sentence) + 1
271		else:
272		break
273
274		# If no sentences fit, take the first sentence and truncate
275		if not summary:
276		return sentences[0][:max_length] + "..."
277
278		return " ".join(summary)

ollamatools-project/ollamatools/tools/enhanced_matrix_core.py +0 -466

…	…	@@ -1,466 +0,0 @@
1		"""
2		Core functionality for the Enhanced Matrix integration in OllamaTools.
3
4		This module provides the core EnhancedMatrixClient class and basic functionality
5		for connecting to Matrix homeservers, joining rooms, and handling messages.
6
7		Dependencies:
8		- matrix-nio: For Matrix protocol support
9		- matrix-nio[e2e]: For end-to-end encryption support (optional)
10		- pyyaml: For configuration handling
11		- ollama: For Ollama API access
12		"""
13
14		import os
15		import sys
16		import yaml
17		import json
18		import time
19		import logging
20		import asyncio
21		import re
22		from typing import Dict, List, Optional, Union, Callable, Any, Set, Tuple
23		from collections import defaultdict
24
25		# Import base matrix tools
26		from ollamatools.tools.matrix_tools import (
27		MatrixClient,
28		MatrixConfig,
29		RoomMessageText
30		)
31
32		# Configure logging
33		logger = logging.getLogger(__name__)
34
35		# Import flags from other modules
36		from ollamatools.tools.enhanced_matrix_search import MODULAR_AGENT_AVAILABLE, OPENSEARCH_AVAILABLE
37
38		class EnhancedMatrixClient(MatrixClient):
39		"""
40		Enhanced Matrix client with advanced features from modular and extended agents.
41
42		This client extends the base MatrixClient with:
43		- Improved message response logic
44		- Natural language understanding
45		- Search integration
46		- Conversation history tracking and summarization
47		- Topic analysis
48		- Performance metrics
49		"""
50
51		# Additional command patterns
52		SEARCH_COMMAND_PATTERN = re.compile(r'^/search\s+(.+)$')
53		HISTORY_COMMAND_PATTERN = re.compile(r'^/history(?:\s+(\d+))?$')
54		TOPICS_COMMAND_PATTERN = re.compile(r'^/topics$')
55		METRICS_COMMAND_PATTERN = re.compile(r'^/metrics$')
56
57		def __init__(
58		self,
59		homeserver: str,
60		user_id: str,
61		password: str,
62		device_id: str = None,
63		ollama_host: str = "localhost:11434",
64		ollama_model: str = "llama3:latest",
65		ollama_prompt: str = None,
66		available_models: List[str] = None,
67		admin_users: List[str] = None,
68		opensearch_config: Dict[str, Any] = None,
69		ssl: bool = True,
70		enable_search: bool = True,
71		enable_conversation_tracking: bool = True,
72		max_context_length: int = 10,
73		max_conversation_history: int = 100,
74		respond_to_all_messages: bool = False
75		):
76		"""
77		Initialize a new Enhanced Matrix client.
78
79		Args:
80		homeserver: The URL of the Matrix homeserver
81		user_id: The Matrix user ID (e.g., @user:example.com)
82		password: The password for the Matrix account
83		device_id: A unique identifier for this device (for E2E encryption)
84		ollama_host: The host address for the Ollama API
85		ollama_model: The Ollama model to use for generating responses
86		ollama_prompt: An optional system prompt for the Ollama model
87		available_models: List of available Ollama models
88		admin_users: List of Matrix user IDs that can change models for all users
89		opensearch_config: Configuration for OpenSearchNG
90		ssl: Whether to use SSL for the Matrix connection
91		enable_search: Whether to enable search functionality
92		enable_conversation_tracking: Whether to enable conversation tracking
93		max_context_length: Maximum number of messages to keep in context
94		max_conversation_history: Maximum number of conversations to keep in history
95		respond_to_all_messages: Whether to respond to all messages (not just mentions)
96		"""
97		# Initialize the base class
98		super().__init__(
99		homeserver=homeserver,
100		user_id=user_id,
101		password=password,
102		device_id=device_id,
103		ollama_host=ollama_host,
104		ollama_model=ollama_model,
105		ollama_prompt=ollama_prompt,
106		available_models=available_models,
107		admin_users=admin_users,
108		ssl=ssl
109		)
110
111		# Set enhanced options
112		self.enable_search = enable_search and OPENSEARCH_AVAILABLE and MODULAR_AGENT_AVAILABLE
113		self.enable_conversation_tracking = enable_conversation_tracking
114		self.max_context_length = max_context_length
115		self.max_conversation_history = max_conversation_history
116		self.respond_to_all_messages = respond_to_all_messages
117
118		# Initialize OpenSearch configuration
119		from ollamatools.tools.enhanced_matrix_search import get_default_opensearch_config
120		self.opensearch_config = opensearch_config or get_default_opensearch_config() if OPENSEARCH_AVAILABLE else None
121
122		# Initialize conversation history tracking
123		self.conversation_history = []
124		self.conversation_history_file = os.path.join(
125		os.path.expanduser('~'),
126		'.ollamatools',
127		'matrix_conversation_history.json'
128		)
129
130		# Initialize function selection feedback
131		if MODULAR_AGENT_AVAILABLE:
132		from ollamatools.agent.modular.feedback import FunctionSelectionFeedback
133		self.function_feedback = FunctionSelectionFeedback()
134
135		# Load conversation history if enabled
136		if self.enable_conversation_tracking:
137		from ollamatools.tools.enhanced_matrix_history import load_conversation_history
138		self.conversation_history = load_conversation_history(self.conversation_history_file)
139
140		async def _message_callback(self, room: Any, event: Any) -> None:
141		"""
142		Enhanced internal callback for handling incoming messages.
143
144		This extends the base callback with:
145		- Improved username detection
146		- Natural language understanding
147		- Search integration
148		- Conversation tracking
149
150		Args:
151		room: The room the message was sent in
152		event: The message event
153		"""
154		# Use the base class method to handle basic functionality
155		if not hasattr(event, 'body') or not hasattr(event, 'sender'):
156		return
157
158		# Get message timestamp
159		message_timestamp = event.server_timestamp / 1000
160
161		# Skip messages from before the client connected
162		if self.connection_time and message_timestamp < self.connection_time:
163		logger.debug("Skipping old message")
164		return
165
166		# Skip messages from the bot itself
167		if event.sender == self.user_id:
168		logger.debug("Skipping message from self")
169		return
170
171		message = event.body
172		room_id = room.room_id
173		sender = event.sender
174
175		logger.info(f"Received message in room {room_id} from {sender}: {message}")
176
177		# Record start time for performance tracking
178		start_time = time.time()
179
180		# If a custom callback is set, use it
181		if self.message_callback:
182		await self.message_callback(room, event, self)
183		return
184
185		# Check if this is a command
186		if message.startswith('/'):
187		await self._handle_command(message, room_id, sender)
188
189		# Record in conversation history
190		if self.enable_conversation_tracking:
191		response_time = time.time() - start_time
192		from ollamatools.tools.enhanced_matrix_history import record_conversation
193		record_conversation(
194		self.conversation_history,
195		message,
196		"Command processed",
197		sender,
198		room_id,
199		response_time,
200		self.conversation_history_file,
201		self.max_conversation_history
202		)
203
204		return
205
206		# Extract username from user_id (e.g., "@user:example.com" -> "user")
207		username = self.user_id.split(':')[0].lstrip('@')
208
209		# Check if the message contains the bot's username (case insensitive)
210		# or if it's a direct mention like @username:server.com
211		should_respond = False
212
213		# Check for username mention (case insensitive)
214		if username.lower() in message.lower():
215		logger.info(f"Username mention detected in message: {message}")
216		should_respond = True
217
218		# Check for direct mention like @username:server.com
219		elif self.user_id.lower() in message.lower():
220		logger.info(f"Direct mention detected in message: {message}")
221		should_respond = True
222
223		# Check if we should respond to all messages
224		elif self.respond_to_all_messages:
225		logger.info(f"Responding to all messages: {message}")
226		should_respond = True
227
228		# If we should respond, process with Ollama and send response
229		if should_respond:
230		# Check if the message might benefit from search
231		search_context = ""
232		if self.enable_search and MODULAR_AGENT_AVAILABLE and OPENSEARCH_AVAILABLE:
233		from ollamatools.tools.enhanced_matrix_search import execute_search_for_message
234		search_context = await execute_search_for_message(message, self.opensearch_config)
235
236		# Process with Ollama
237		try:
238		if search_context:
239		# Add search context to the message
240		augmented_message = f"{message}\n\nSearch context: {search_context}"
241		response = await self.process_with_ollama(augmented_message, sender, room_id)
242		else:
243		response = await self.process_with_ollama(message, sender, room_id)
244
245		# Send the response
246		await self.send_message(room_id, response)
247
248		# Record in conversation history
249		if self.enable_conversation_tracking:
250		response_time = time.time() - start_time
251		from ollamatools.tools.enhanced_matrix_history import record_conversation
252		record_conversation(
253		self.conversation_history,
254		message,
255		response,
256		sender,
257		room_id,
258		response_time,
259		self.conversation_history_file,
260		self.max_conversation_history
261		)
262		except Exception as e:
263		logger.error(f"Error processing message with Ollama: {e}")
264		error_message = f"I encountered an error processing your message: {str(e)}"
265		await self.send_message(room_id, error_message)
266
267		async def _handle_command(self, message: str, room_id: str, sender: str) -> None:
268		"""
269		Enhanced command handler with additional commands.
270
271		Args:
272		message: The command message
273		room_id: The ID of the room where the command was sent
274		sender: The Matrix user ID of the sender
275		"""
276		# Check for base commands first
277		model_match = self.MODEL_COMMAND_PATTERN.match(message)
278		if model_match:
279		model_name = model_match.group(1)
280		await self._change_model(model_name, room_id, sender)
281		return
282
283		if self.HELP_COMMAND_PATTERN.match(message):
284		await self._send_help(room_id)
285		return
286
287		if self.LIST_MODELS_COMMAND_PATTERN.match(message):
288		await self._list_models(room_id)
289		return
290
291		# Check for enhanced commands
292		search_match = self.SEARCH_COMMAND_PATTERN.match(message)
293		if search_match and self.enable_search:
294		query = search_match.group(1)
295		from ollamatools.tools.enhanced_matrix_search import handle_search_command
296		await handle_search_command(self, query, room_id)
297		return
298
299		history_match = self.HISTORY_COMMAND_PATTERN.match(message)
300		if history_match and self.enable_conversation_tracking:
301		count = int(history_match.group(1)) if history_match.group(1) else 5
302		from ollamatools.tools.enhanced_matrix_history import handle_history_command
303		await handle_history_command(self, count, room_id)
304		return
305
306		if self.TOPICS_COMMAND_PATTERN.match(message) and self.enable_conversation_tracking:
307		from ollamatools.tools.enhanced_matrix_history import handle_topics_command
308		await handle_topics_command(self, room_id)
309		return
310
311		if self.METRICS_COMMAND_PATTERN.match(message) and self.enable_conversation_tracking:
312		from ollamatools.tools.enhanced_matrix_history import handle_metrics_command
313		await handle_metrics_command(self, room_id)
314		return
315
316		# Unknown command
317		await self.send_message(room_id, f"Unknown command: {message}\nType /help for available commands.")
318
319		async def _send_help(self, room_id: str) -> None:
320		"""
321		Send enhanced help information to a room.
322
323		Args:
324		room_id: The ID of the room to send the help to
325		"""
326		help_text = """
327		Available commands:
328		- /help - Show this help message
329		- /models - List available models
330		- /model <model_name> - Change to a different model
331		"""
332
333		if self.enable_search:
334		help_text += "- /search <query> - Search the web for information\n"
335
336		if self.enable_conversation_tracking:
337		help_text += """- /history [count] - Show recent conversation history
338		- /topics - Show conversation topics
339		- /metrics - Show performance metrics
340		"""
341
342		help_text += f"""
343		You can also mention me by my username in your message and I'll respond.
344		For example: "Hey {self.user_id.split(':')[0].lstrip('@')}, what's the weather like today?"
345		"""
346
347		await self.send_message(room_id, help_text.strip())
348
349		async def listen(self) -> None:
350		"""
351		Start listening for messages in joined rooms.
352
353		This method will block until the client is stopped.
354		"""
355		try:
356		# Set up the message callback
357		self.client.add_event_callback(self._message_callback, RoomMessageText)
358
359		# Start syncing with the server
360		await self.client.sync_forever(timeout=30000, full_state=True)
361		except KeyboardInterrupt:
362		logger.info("Interrupted by user")
363		except Exception as e:
364		logger.error(f"Error in listen: {e}")
365		finally:
366		# Clean up resources
367		await self.disconnect()
368
369		async def disconnect(self) -> None:
370		"""
371		Disconnect from the Matrix homeserver and clean up resources.
372		"""
373		try:
374		logger.info(f"Disconnecting from {self.homeserver}...")
375
376		# Save conversation history before disconnecting
377		if self.enable_conversation_tracking:
378		from ollamatools.tools.enhanced_matrix_history import save_conversation_history
379		save_conversation_history(self.conversation_history, self.conversation_history_file)
380
381		# Logout and close the client
382		try:
383		await self.client.logout()
384		logger.info("Logged out from Matrix homeserver")
385		except Exception as e:
386		logger.error(f"Error logging out: {e}")
387
388		# Close the client session
389		try:
390		await self.client.close()
391		logger.info("Matrix client session closed")
392		except Exception as e:
393		logger.error(f"Error closing Matrix client session: {e}")
394
395		logger.info(f"Disconnected from {self.homeserver}")
396		except Exception as e:
397		logger.error(f"Error during disconnect: {e}")
398
399		# Async helper function for the module API
400		async def _connect_enhanced_matrix(
401		homeserver: str,
402		user_id: str,
403		password: str,
404		device_id: str = None,
405		ollama_host: str = "localhost:11434",
406		ollama_model: str = "llama3:latest",
407		ollama_prompt: str = None,
408		available_models: List[str] = None,
409		admin_users: List[str] = None,
410		opensearch_config: Dict[str, Any] = None,
411		ssl: bool = True,
412		enable_search: bool = True,
413		enable_conversation_tracking: bool = True,
414		max_context_length: int = 10,
415		max_conversation_history: int = 100,
416		respond_to_all_messages: bool = False
417		) -> Optional[EnhancedMatrixClient]:
418		"""
419		Connect to a Matrix homeserver with enhanced features.
420
421		Args:
422		homeserver: The URL of the Matrix homeserver
423		user_id: The Matrix user ID (e.g., @user:example.com)
424		password: The password for the Matrix account
425		device_id: A unique identifier for this device (for E2E encryption)
426		ollama_host: The host address for the Ollama API
427		ollama_model: The Ollama model to use for generating responses
428		ollama_prompt: An optional system prompt for the Ollama model
429		available_models: List of available Ollama models
430		admin_users: List of Matrix user IDs that can change models for all users
431		opensearch_config: Configuration for OpenSearchNG
432		ssl: Whether to use SSL for the Matrix connection
433		enable_search: Whether to enable search functionality
434		enable_conversation_tracking: Whether to enable conversation tracking
435		max_context_length: Maximum number of messages to keep in context
436		max_conversation_history: Maximum number of conversations to keep in history
437		respond_to_all_messages: Whether to respond to all messages (not just mentions)
438
439		Returns:
440		Optional[EnhancedMatrixClient]: A connected Enhanced Matrix client, or None if connection failed
441		"""
442		client = EnhancedMatrixClient(
443		homeserver=homeserver,
444		user_id=user_id,
445		password=password,
446		device_id=device_id,
447		ollama_host=ollama_host,
448		ollama_model=ollama_model,
449		ollama_prompt=ollama_prompt,
450		available_models=available_models,
451		admin_users=admin_users,
452		opensearch_config=opensearch_config,
453		ssl=ssl,
454		enable_search=enable_search,
455		enable_conversation_tracking=enable_conversation_tracking,
456		max_context_length=max_context_length,
457		max_conversation_history=max_conversation_history,
458		respond_to_all_messages=respond_to_all_messages
459		)
460
461		success = await client.connect()
462
463		if success:
464		return client
465		else:
466		return None

ollamatools-project/ollamatools/tools/enhanced_matrix_history.py +0 -329

…	…	@@ -1,329 +0,0 @@
1		"""
2		Conversation history and analytics for the Enhanced Matrix integration in OllamaTools.
3
4		This module provides functionality for tracking conversation history,
5		analyzing topics, and generating performance metrics for the Enhanced Matrix client.
6
7		Dependencies:
8		- matrix-nio: For Matrix protocol support
9		- ollama: For Ollama API access
10		"""
11
12		import os
13		import json
14		import time
15		import logging
16		from typing import Dict, List, Optional, Any, Set, Tuple
17		from collections import defaultdict
18
19		# Configure logging
20		logger = logging.getLogger(__name__)
21
22		def load_conversation_history(history_file: str) -> List[Dict[str, Any]]:
23		"""
24		Load conversation history from file.
25
26		Args:
27		history_file: Path to the history file
28
29		Returns:
30		List[Dict[str, Any]]: The conversation history
31		"""
32		if os.path.exists(history_file):
33		try:
34		with open(history_file, 'r') as f:
35		history = json.load(f)
36		logger.info(f"Loaded {len(history)} conversations from history")
37		return history
38		except Exception as e:
39		logger.error(f"Failed to load conversation history: {str(e)}")
40		return []
41		else:
42		logger.info("No conversation history file found")
43		return []
44
45		def save_conversation_history(history: List[Dict[str, Any]], history_file: str) -> None:
46		"""
47		Save conversation history to file.
48
49		Args:
50		history: The conversation history
51		history_file: Path to save the history to
52		"""
53		try:
54		# Create directory if it doesn't exist
55		os.makedirs(os.path.dirname(history_file), exist_ok=True)
56
57		with open(history_file, 'w') as f:
58		json.dump(history, f, indent=2)
59		logger.debug(f"Saved {len(history)} conversations to history")
60		except Exception as e:
61		logger.error(f"Failed to save conversation history: {str(e)}")
62
63		def record_conversation(
64		history: List[Dict[str, Any]],
65		user_message: str,
66		assistant_response: str,
67		sender: str,
68		room_id: str,
69		response_time: float,
70		history_file: str,
71		max_history: int = 100
72		) -> None:
73		"""
74		Record a conversation in the history.
75
76		Args:
77		history: The conversation history list to update
78		user_message: The user's message
79		assistant_response: The assistant's response
80		sender: The Matrix user ID of the sender
81		room_id: The ID of the room where the conversation took place
82		response_time: The time taken to generate the response
83		history_file: Path to save the history to
84		max_history: Maximum number of conversations to keep in history
85		"""
86		# Create a conversation record
87		conversation = {
88		"timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
89		"user_message": user_message,
90		"assistant_response": assistant_response,
91		"sender": sender,
92		"room_id": room_id,
93		"response_time": response_time,
94		"tokens_estimated": len(user_message.split()) + len(assistant_response.split())
95		}
96
97		# Add to conversation history
98		history.append(conversation)
99
100		# Trim conversation history if it gets too long
101		if len(history) > max_history:
102		history = history[-max_history:]
103
104		# Save conversation history
105		save_conversation_history(history, history_file)
106
107		def analyze_conversation_topics(conversations: List[Dict[str, Any]]) -> Dict[str, int]:
108		"""
109		Analyze common topics in conversation history.
110
111		Args:
112		conversations: List of conversation records
113
114		Returns:
115		Dict[str, int]: Dictionary of topics and their frequency
116		"""
117		if not conversations:
118		return {}
119
120		# Simple keyword-based topic analysis
121		topics = {
122		"help": 0,
123		"search": 0,
124		"file": 0,
125		"command": 0,
126		"code": 0,
127		"error": 0,
128		"issue": 0,
129		"question": 0,
130		"other": 0
131		}
132
133		for conv in conversations:
134		message = conv.get("user_message", "").lower()
135
136		# Check for keywords
137		if "help" in message or "how" in message:
138		topics["help"] += 1
139		if "search" in message or "find" in message or "look" in message:
140		topics["search"] += 1
141		if "file" in message or "read" in message or "write" in message:
142		topics["file"] += 1
143		if "command" in message or "run" in message or "execute" in message:
144		topics["command"] += 1
145		if "code" in message or "program" in message or "script" in message:
146		topics["code"] += 1
147		if "error" in message or "bug" in message or "fix" in message:
148		topics["error"] += 1
149		if "issue" in message or "problem" in message:
150		topics["issue"] += 1
151		if "?" in message:
152		topics["question"] += 1
153
154		# If no keywords matched, count as other
155		if not any(count > 0 for count in topics.values()):
156		topics["other"] += 1
157
158		# Remove topics with zero count
159		return {k: v for k, v in topics.items() if v > 0}
160
161		def get_performance_metrics(conversations: List[Dict[str, Any]]) -> Dict[str, Any]:
162		"""
163		Get performance metrics from conversation history.
164
165		Args:
166		conversations: List of conversation records
167
168		Returns:
169		Dict[str, Any]: Performance metrics
170		"""
171		if not conversations:
172		return {"error": "No conversation history available"}
173
174		# Calculate metrics
175		total_conversations = len(conversations)
176		total_response_time = sum(conv.get("response_time", 0) for conv in conversations)
177		avg_response_time = total_response_time / total_conversations if total_conversations > 0 else 0
178
179		# Calculate token usage
180		total_tokens = sum(conv.get("tokens_estimated", 0) for conv in conversations)
181		avg_tokens_per_conversation = total_tokens / total_conversations if total_conversations > 0 else 0
182
183		# Find slowest and fastest responses
184		slowest = max(conversations, key=lambda x: x.get("response_time", 0)) if conversations else None
185		fastest = min(conversations, key=lambda x: x.get("response_time", 0)) if conversations else None
186
187		return {
188		"total_conversations": total_conversations,
189		"total_response_time": total_response_time,
190		"avg_response_time": avg_response_time,
191		"total_tokens_estimated": total_tokens,
192		"avg_tokens_per_conversation": avg_tokens_per_conversation,
193		"slowest_response": {
194		"time": slowest.get("response_time", 0) if slowest else 0,
195		"message": slowest.get("user_message", "")[:50] + "..." if slowest and len(slowest.get("user_message", "")) > 50 else slowest.get("user_message", "") if slowest else ""
196		},
197		"fastest_response": {
198		"time": fastest.get("response_time", 0) if fastest else 0,
199		"message": fastest.get("user_message", "")[:50] + "..." if fastest and len(fastest.get("user_message", "")) > 50 else fastest.get("user_message", "") if fastest else ""
200		}
201		}
202
203		async def handle_history_command(client: Any, count: int, room_id: str) -> None:
204		"""
205		Handle the /history command.
206
207		Args:
208		client: The EnhancedMatrixClient instance
209		count: Number of recent conversations to show
210		room_id: The ID of the room where the command was sent
211		"""
212		if not client.enable_conversation_tracking:
213		await client.send_message(room_id, "Conversation tracking is not enabled.")
214		return
215
216		if not client.conversation_history:
217		await client.send_message(room_id, "No conversation history available.")
218		return
219
220		# Get the most recent conversations for this room
221		room_history = [conv for conv in client.conversation_history if conv.get("room_id") == room_id]
222
223		if not room_history:
224		await client.send_message(room_id, "No conversation history available for this room.")
225		return
226
227		# Get the most recent conversations
228		recent = room_history[-count:] if len(room_history) >= count else room_history
229
230		# Format the history
231		formatted_history = ["Recent Conversations:"]
232
233		for i, conv in enumerate(recent, 1):
234		timestamp = conv.get("timestamp", "Unknown time")
235		user_message = conv.get("user_message", "No message")
236		sender = conv.get("sender", "Unknown sender")
237
238		# Truncate message if it's too long
239		if len(user_message) > 100:
240		user_message = user_message[:100] + "..."
241
242		formatted_history.append(f"{i}. {timestamp}")
243		formatted_history.append(f" From: {sender}")
244		formatted_history.append(f" Message: {user_message}")
245		formatted_history.append("")
246
247		# Send the formatted history
248		await client.send_message(room_id, "\n".join(formatted_history))
249
250		async def handle_topics_command(client: Any, room_id: str) -> None:
251		"""
252		Handle the /topics command.
253
254		Args:
255		client: The EnhancedMatrixClient instance
256		room_id: The ID of the room where the command was sent
257		"""
258		if not client.enable_conversation_tracking:
259		await client.send_message(room_id, "Conversation tracking is not enabled.")
260		return
261
262		if not client.conversation_history:
263		await client.send_message(room_id, "No conversation history available.")
264		return
265
266		# Get the conversations for this room
267		room_history = [conv for conv in client.conversation_history if conv.get("room_id") == room_id]
268
269		if not room_history:
270		await client.send_message(room_id, "No conversation history available for this room.")
271		return
272
273		# Analyze topics
274		topics = analyze_conversation_topics(room_history)
275
276		if not topics:
277		await client.send_message(room_id, "No topics found in conversation history.")
278		return
279
280		# Format the topics
281		formatted_topics = ["Conversation Topics:"]
282
283		for topic, count in sorted(topics.items(), key=lambda x: x[1], reverse=True):
284		formatted_topics.append(f"- {topic}: {count} mentions")
285
286		# Send the formatted topics
287		await client.send_message(room_id, "\n".join(formatted_topics))
288
289		async def handle_metrics_command(client: Any, room_id: str) -> None:
290		"""
291		Handle the /metrics command.
292
293		Args:
294		client: The EnhancedMatrixClient instance
295		room_id: The ID of the room where the command was sent
296		"""
297		if not client.enable_conversation_tracking:
298		await client.send_message(room_id, "Conversation tracking is not enabled.")
299		return
300
301		if not client.conversation_history:
302		await client.send_message(room_id, "No conversation history available.")
303		return
304
305		# Get the conversations for this room
306		room_history = [conv for conv in client.conversation_history if conv.get("room_id") == room_id]
307
308		if not room_history:
309		await client.send_message(room_id, "No conversation history available for this room.")
310		return
311
312		# Calculate metrics
313		metrics = get_performance_metrics(room_history)
314
315		# Format the metrics
316		formatted_metrics = ["Performance Metrics:"]
317		formatted_metrics.append(f"- Total conversations: {metrics['total_conversations']}")
318		formatted_metrics.append(f"- Average response time: {metrics['avg_response_time']:.2f}s")
319		formatted_metrics.append(f"- Estimated total tokens: {metrics['total_tokens_estimated']}")
320		formatted_metrics.append(f"- Average tokens per conversation: {metrics['avg_tokens_per_conversation']:.1f}")
321
322		if metrics.get("slowest_response"):
323		formatted_metrics.append(f"- Slowest response: {metrics['slowest_response']['time']:.2f}s for '{metrics['slowest_response']['message']}'")
324
325		if metrics.get("fastest_response"):
326		formatted_metrics.append(f"- Fastest response: {metrics['fastest_response']['time']:.2f}s for '{metrics['fastest_response']['message']}'")
327
328		# Send the formatted metrics
329		await client.send_message(room_id, "\n".join(formatted_metrics))

ollamatools-project/ollamatools/tools/enhanced_matrix_search.py +0 -170

…	…	@@ -1,170 +0,0 @@
1		"""
2		Search functionality for the Enhanced Matrix integration in OllamaTools.
3
4		This module provides search-related functionality for the Enhanced Matrix client,
5		including integration with OpenSearchNG and the modular agent's search capabilities.
6
7		Dependencies:
8		- matrix-nio: For Matrix protocol support
9		- ollama: For Ollama API access
10		- ollamatools.agent.modular: For search integration and natural language understanding
11		- ollamatools.opensearch_web_tools: For web search functionality
12		"""
13
14		import logging
15		from typing import Dict, List, Optional, Any, Union
16
17		# Configure logging
18		logger = logging.getLogger(__name__)
19
20		# Import from modular agent (optional)
21		try:
22		from ollamatools.agent.modular.search import (
23		detect_factual_intent,
24		extract_search_context,
25		execute_search
26		)
27		MODULAR_AGENT_AVAILABLE = True
28		except ImportError:
29		MODULAR_AGENT_AVAILABLE = False
30		# Only log this as debug since search might be disabled in config
31		logging.debug(
32		"The modular agent components are not available. "
33		"Search functionality will be disabled."
34		)
35
36		# Import from opensearch web tools (optional)
37		try:
38		from ollamatools.opensearch_web_tools import (
39		opensearch_query,
40		opensearch_fetch_url,
41		get_opensearch_info,
42		DEFAULT_OPENSEARCH_CONFIG
43		)
44		OPENSEARCH_AVAILABLE = True
45		except ImportError:
46		OPENSEARCH_AVAILABLE = False
47		# Only log this as debug since search might be disabled in config
48		logging.debug(
49		"The opensearch_web_tools module is not available. "
50		"Search functionality will be disabled."
51		)
52
53		def get_default_opensearch_config() -> Dict[str, Any]:
54		"""
55		Get the default OpenSearchNG configuration.
56
57		Returns:
58		Dict[str, Any]: The default configuration
59		"""
60		if OPENSEARCH_AVAILABLE:
61		return DEFAULT_OPENSEARCH_CONFIG.copy()
62		else:
63		return {}
64
65		async def execute_search_for_message(message: str, opensearch_config: Dict[str, Any]) -> str:
66		"""
67		Execute a search for a message and return the search context.
68
69		Args:
70		message: The message to search for
71		opensearch_config: The OpenSearchNG configuration
72
73		Returns:
74		str: The search context, or an empty string if search failed
75		"""
76		if not MODULAR_AGENT_AVAILABLE or not OPENSEARCH_AVAILABLE:
77		logger.warning("Search functionality is not available.")
78		return ""
79
80		try:
81		factual_intent = detect_factual_intent(message)
82		if factual_intent:
83		logger.info(f"Factual intent detected in message: {message}")
84		try:
85		results = await execute_search_query(message, opensearch_config)
86		search_context = extract_search_context(results)
87		logger.info(f"Search context: {search_context[:100]}...")
88		return search_context
89		except Exception as e:
90		logger.error(f"Error executing search: {e}")
91		return ""
92		else:
93		logger.debug(f"No factual intent detected in message: {message}")
94		return ""
95		except Exception as e:
96		logger.error(f"Error detecting factual intent: {e}")
97		return ""
98
99		async def execute_search_query(query: str, opensearch_config: Dict[str, Any]) -> Dict[str, Any]:
100		"""
101		Execute a search query using OpenSearchNG.
102
103		Args:
104		query: The search query
105		opensearch_config: The OpenSearchNG configuration
106
107		Returns:
108		Dict[str, Any]: Search results
109		"""
110		if not OPENSEARCH_AVAILABLE:
111		logger.warning("OpenSearchNG is not available.")
112		return {"error": "Search functionality is not available.", "results": []}
113
114		try:
115		# Execute the search
116		results = opensearch_query(
117		query=query,
118		config=opensearch_config
119		)
120
121		return results
122		except Exception as e:
123		logger.error(f"Error executing search: {e}")
124		return {"error": str(e), "results": []}
125
126		async def handle_search_command(client: Any, query: str, room_id: str) -> None:
127		"""
128		Handle the /search command.
129
130		Args:
131		client: The EnhancedMatrixClient instance
132		query: The search query
133		room_id: The ID of the room where the command was sent
134		"""
135		if not client.enable_search:
136		await client.send_message(room_id, "Search functionality is not enabled.")
137		return
138
139		await client.send_message(room_id, f"Searching for: {query}...")
140
141		try:
142		results = await execute_search_query(query, client.opensearch_config)
143
144		if not results or not results.get("results"):
145		await client.send_message(room_id, f"No results found for: {query}")
146		return
147
148		# Format the results
149		formatted_results = ["Search Results:"]
150
151		for i, result in enumerate(results["results"][:5], 1):
152		title = result.get("title", "No title")
153		url = result.get("url", "No URL")
154		content = result.get("content", "No content")
155
156		# Truncate content if it's too long
157		if len(content) > 200:
158		content = content[:200] + "..."
159
160		formatted_results.append(f"{i}. {title}")
161		formatted_results.append(f" URL: {url}")
162		formatted_results.append(f" {content}")
163		formatted_results.append("")
164
165		# Send the formatted results
166		await client.send_message(room_id, "\n".join(formatted_results))
167
168		except Exception as e:
169		logger.error(f"Error executing search: {e}")
170		await client.send_message(room_id, f"Error executing search: {str(e)}")

ollamatools-project/ollamatools/tools/enhanced_matrix_tools.py +0 -226

…	…	@@ -1,226 +0,0 @@
1		"""
2		Enhanced Matrix integration tools for OllamaTools.
3
4		This module extends the basic Matrix integration with advanced features from
5		the modular and extended agents, including:
6		- Improved message response logic
7		- Natural language understanding
8		- Search integration
9		- Conversation history tracking and summarization
10		- Topic analysis
11		- Performance metrics
12
13		Dependencies:
14		- matrix-nio: For Matrix protocol support
15		- matrix-nio[e2e]: For end-to-end encryption support (optional)
16		- pyyaml: For configuration handling
17		- ollama: For Ollama API access
18		"""
19
20		import os
21		import sys
22		import yaml
23		import json
24		import asyncio
25		import logging
26		from typing import Dict, List, Optional, Union, Callable, Any, Set, Tuple
27
28		# Import from core module
29		from ollamatools.tools.enhanced_matrix_core import (
30		EnhancedMatrixClient,
31		_connect_enhanced_matrix
32		)
33
34		# Import from search module
35		from ollamatools.tools.enhanced_matrix_search import (
36		MODULAR_AGENT_AVAILABLE,
37		OPENSEARCH_AVAILABLE,
38		get_default_opensearch_config,
39		execute_search_for_message,
40		execute_search_query,
41		handle_search_command
42		)
43
44		# Import from history module
45		from ollamatools.tools.enhanced_matrix_history import (
46		load_conversation_history,
47		save_conversation_history,
48		record_conversation,
49		analyze_conversation_topics,
50		get_performance_metrics,
51		handle_history_command,
52		handle_topics_command,
53		handle_metrics_command
54		)
55
56		# Import from base matrix tools
57		from ollamatools.tools.matrix_tools import (
58		MatrixConfig,
59		load_matrix_config,
60		save_matrix_config,
61		create_default_matrix_config,
62		RoomMessageText
63		)
64
65		# Configure logging
66		logger = logging.getLogger(__name__)
67
68		# Synchronous wrapper functions for the module API
69
70		def connect_enhanced_matrix(
71		homeserver: str,
72		user_id: str,
73		password: str,
74		device_id: str = None,
75		ollama_host: str = "localhost:11434",
76		ollama_model: str = "llama3:latest",
77		ollama_prompt: str = None,
78		available_models: List[str] = None,
79		admin_users: List[str] = None,
80		opensearch_config: Dict[str, Any] = None,
81		ssl: bool = True,
82		enable_search: bool = True,
83		enable_conversation_tracking: bool = True,
84		max_context_length: int = 10,
85		max_conversation_history: int = 100,
86		respond_to_all_messages: bool = False
87		) -> Optional[EnhancedMatrixClient]:
88		"""
89		Connect to a Matrix homeserver with enhanced features.
90
91		Args:
92		homeserver: The URL of the Matrix homeserver
93		user_id: The Matrix user ID (e.g., @user:example.com)
94		password: The password for the Matrix account
95		device_id: A unique identifier for this device (for E2E encryption)
96		ollama_host: The host address for the Ollama API
97		ollama_model: The Ollama model to use for generating responses
98		ollama_prompt: An optional system prompt for the Ollama model
99		available_models: List of available Ollama models
100		admin_users: List of Matrix user IDs that can change models for all users
101		opensearch_config: Configuration for OpenSearchNG
102		ssl: Whether to use SSL for the Matrix connection
103		enable_search: Whether to enable search functionality
104		enable_conversation_tracking: Whether to enable conversation tracking
105		max_context_length: Maximum number of messages to keep in context
106		max_conversation_history: Maximum number of conversations to keep in history
107		respond_to_all_messages: Whether to respond to all messages (not just mentions)
108
109		Returns:
110		Optional[EnhancedMatrixClient]: A connected Enhanced Matrix client, or None if connection failed
111		"""
112		return asyncio.run(_connect_enhanced_matrix(
113		homeserver=homeserver,
114		user_id=user_id,
115		password=password,
116		device_id=device_id,
117		ollama_host=ollama_host,
118		ollama_model=ollama_model,
119		ollama_prompt=ollama_prompt,
120		available_models=available_models,
121		admin_users=admin_users,
122		opensearch_config=opensearch_config,
123		ssl=ssl,
124		enable_search=enable_search,
125		enable_conversation_tracking=enable_conversation_tracking,
126		max_context_length=max_context_length,
127		max_conversation_history=max_conversation_history,
128		respond_to_all_messages=respond_to_all_messages
129		))
130
131		def join_enhanced_matrix_room(client: EnhancedMatrixClient, room_id: str) -> bool:
132		"""
133		Join a Matrix room with an Enhanced Matrix client.
134
135		Args:
136		client: A connected Enhanced Matrix client
137		room_id: The ID of the room to join (e.g., #room:example.com)
138
139		Returns:
140		bool: True if joining was successful, False otherwise
141		"""
142		return asyncio.run(client.join_room(room_id))
143
144		def send_enhanced_matrix_message(client: EnhancedMatrixClient, room_id: str, message: str) -> bool:
145		"""
146		Send a message to a Matrix room with an Enhanced Matrix client.
147
148		Args:
149		client: A connected Enhanced Matrix client
150		room_id: The ID of the room to send the message to
151		message: The message to send
152
153		Returns:
154		bool: True if sending was successful, False otherwise
155		"""
156		return asyncio.run(client.send_message(room_id, message))
157
158		def listen_for_enhanced_matrix_messages(client: EnhancedMatrixClient) -> None:
159		"""
160		Start listening for messages in joined rooms with an Enhanced Matrix client.
161
162		This function will block until the client is stopped.
163
164		Args:
165		client: A connected Enhanced Matrix client
166		"""
167		asyncio.run(client.listen())
168
169		def create_enhanced_matrix_config() -> Dict:
170		"""
171		Create a default Enhanced Matrix configuration.
172
173		Returns:
174		Dict: The default configuration
175		"""
176		config = create_default_matrix_config()
177
178		# Add enhanced configuration
179		config["ENHANCED"] = {
180		"ENABLE_SEARCH": True,
181		"ENABLE_CONVERSATION_TRACKING": True,
182		"MAX_CONTEXT_LENGTH": 10,
183		"MAX_CONVERSATION_HISTORY": 100,
184		"RESPOND_TO_ALL_MESSAGES": False,
185		"OPENSEARCH": {
186		"BASE_URL": "http://localhost:8081",
187		"ENGINES": [
188		"google",
189		"bing",
190		"duckduckgo",
191		"brave"
192		],
193		"NUM_RESULTS": 5
194		}
195		}
196
197		return config
198
199		def disconnect_enhanced_matrix(client: EnhancedMatrixClient) -> None:
200		"""
201		Disconnect from the Matrix homeserver and clean up resources.
202
203		Args:
204		client: A connected Enhanced Matrix client
205		"""
206		asyncio.run(client.disconnect())
207
208		def process_with_enhanced_ollama(
209		client: EnhancedMatrixClient,
210		message: str,
211		sender: str = None,
212		room_id: str = None
213		) -> str:
214		"""
215		Process a message with Ollama using an Enhanced Matrix client.
216
217		Args:
218		client: A connected Enhanced Matrix client
219		message: The message to process
220		sender: The Matrix user ID of the sender
221		room_id: The ID of the room where the message was sent
222
223		Returns:
224		str: The response from Ollama
225		"""
226		return asyncio.run(client.process_with_ollama(message, sender, room_id))

ollamatools-project/ollamatools/tools/matrix_tools.py +0 -800

…	…	@@ -1,800 +0,0 @@
1		"""
2		Matrix integration tools for OllamaTools.
3
4		This module provides functionality to connect to Matrix homeservers,
5		join rooms, send messages, and listen for incoming messages.
6		It integrates with Ollama to process messages and generate responses.
7
8		Dependencies:
9		- matrix-nio: For Matrix protocol support
10		- matrix-nio[e2e]: For end-to-end encryption support (optional)
11		- pyyaml: For configuration handling
12		"""
13
14		import os
15		import sys
16		import yaml
17		import logging
18		import asyncio
19		import re
20		from typing import Dict, List, Optional, Union, Callable, Any, Set
21		from collections import defaultdict
22
23		# Configure logging
24		logger = logging.getLogger(__name__)
25
26		try:
27		from nio import (
28		AsyncClient,
29		MatrixRoom,
30		RoomMessageText,
31		LoginResponse,
32		JoinError
33		)
34		except ImportError:
35		raise ImportError(
36		"Matrix support requires the 'matrix-nio' package. "
37		"Please install it with 'pip install matrix-nio' or "
38		"'pip install matrix-nio[e2e]' for end-to-end encryption support."
39		)
40
41		# Make ollama import optional
42		try:
43		from ollama import AsyncClient as OllamaAsync
44		OLLAMA_AVAILABLE = True
45		except ImportError:
46		OLLAMA_AVAILABLE = False
47		logger.warning(
48		"The 'ollama' package is not installed. "
49		"Message processing with Ollama will not be available. "
50		"Install it with 'pip install ollama'."
51		)
52
53		class MatrixClient:
54		"""
55		A client for connecting to Matrix homeservers and interacting with rooms.
56
57		This client can connect to a Matrix homeserver, join rooms, send messages,
58		and listen for incoming messages. It integrates with Ollama to process
59		messages and generate responses.
60		"""
61
62		# Command patterns
63		MODEL_COMMAND_PATTERN = re.compile(r'^/model\s+([a-zA-Z0-9:_\-\.]+)$')
64		HELP_COMMAND_PATTERN = re.compile(r'^/help$')
65		LIST_MODELS_COMMAND_PATTERN = re.compile(r'^/models$')
66
67		def __init__(
68		self,
69		homeserver: str,
70		user_id: str,
71		password: str,
72		device_id: str = None,
73		ollama_host: str = "localhost:11434",
74		ollama_model: str = "llama3:latest",
75		ollama_prompt: str = None,
76		available_models: List[str] = None,
77		admin_users: List[str] = None,
78		ssl: bool = True
79		):
80		"""
81		Initialize a new Matrix client.
82
83		Args:
84		homeserver: The URL of the Matrix homeserver
85		user_id: The Matrix user ID (e.g., @user:example.com)
86		password: The password for the Matrix account
87		device_id: A unique identifier for this device (for E2E encryption)
88		ollama_host: The host address for the Ollama API
89		ollama_model: The Ollama model to use for generating responses
90		ollama_prompt: An optional system prompt for the Ollama model
91		available_models: List of available Ollama models
92		admin_users: List of Matrix user IDs that can change models for all users
93		ssl: Whether to use SSL for the Matrix connection
94		"""
95		self.homeserver = homeserver
96		self.user_id = user_id
97		self.password = password
98		self.device_id = device_id
99		self.ollama_host = ollama_host
100		self.ollama_model = ollama_model
101		self.ollama_prompt = ollama_prompt
102		self.ssl = ssl
103
104		# Initialize the Matrix client
105		self.client = AsyncClient(
106		homeserver=self.homeserver,
107		device_id=self.device_id,
108		ssl=self.ssl
109		)
110
111		# Track when the client connected to avoid processing old messages
112		self.connection_time = None
113
114		# Track joined rooms
115		self.rooms = []
116
117		# Message callback
118		self.message_callback = None
119
120		# User context tracking
121		self.user_contexts = defaultdict(list)
122		self.max_context_length = 10 # Maximum number of messages to keep in context
123
124		# User model preferences
125		self.user_models = {}
126
127		# Available models
128		self.available_models = available_models or [ollama_model]
129
130		# Admin users who can change models
131		self.admin_users = set(admin_users or [])
132
133		async def connect(self) -> bool:
134		"""
135		Connect to the Matrix homeserver and log in.
136
137		Returns:
138		bool: True if the connection was successful, False otherwise
139		"""
140		try:
141		# Extract username from user_id (e.g., "@user:example.com" -> "user")
142		username = self.user_id.split(':')[0].lstrip('@')
143
144		# Set the user ID for the client
145		self.client.user = username
146
147		response = await self.client.login(
148		password=self.password
149		)
150
151		if isinstance(response, LoginResponse):
152		logger.info(f"Successfully connected to {self.homeserver} as {self.user_id}")
153		self.connection_time = asyncio.get_event_loop().time()
154		return True
155		else:
156		logger.error(f"Failed to connect to {self.homeserver}: {response}")
157		return False
158
159		except Exception as e:
160		logger.error(f"Error connecting to Matrix server: {e}")
161		return False
162
163		async def join_room(self, room_id: str) -> bool:
164		"""
165		Join a Matrix room.
166
167		Args:
168		room_id: The ID of the room to join (e.g., #room:example.com)
169
170		Returns:
171		bool: True if joining was successful, False otherwise
172		"""
173		try:
174		response = await self.client.join(room_id)
175
176		if isinstance(response, JoinError):
177		logger.error(f"Failed to join room {room_id}: {response}")
178		return False
179
180		logger.info(f"Successfully joined room {room_id}")
181		self.rooms.append(room_id)
182		return True
183
184		except Exception as e:
185		logger.error(f"Error joining room {room_id}: {e}")
186		return False
187
188		async def send_message(self, room_id: str, message: str) -> bool:
189		"""
190		Send a message to a Matrix room.
191
192		Args:
193		room_id: The ID of the room to send the message to
194		message: The message to send
195
196		Returns:
197		bool: True if sending was successful, False otherwise
198		"""
199		try:
200		response = await self.client.room_send(
201		room_id=room_id,
202		message_type="m.room.message",
203		content={
204		"msgtype": "m.text",
205		"body": message
206		}
207		)
208
209		logger.debug(f"Sent message to room {room_id}")
210		return True
211
212		except Exception as e:
213		logger.error(f"Error sending message to room {room_id}: {e}")
214		return False
215
216		async def process_with_ollama(self, message: str, sender: str = None, room_id: str = None) -> str:
217		"""
218		Process a message with Ollama and return the response.
219
220		Args:
221		message: The message to process
222		sender: The Matrix user ID of the sender
223		room_id: The ID of the room where the message was sent
224
225		Returns:
226		str: The response from Ollama
227		"""
228		if not OLLAMA_AVAILABLE:
229		return "Ollama is not available. Please install the 'ollama' package with 'pip install ollama'."
230
231		try:
232		ollama_client = OllamaAsync(host=self.ollama_host)
233
234		# Determine which model to use
235		model = self.ollama_model
236		if sender and sender in self.user_models:
237		model = self.user_models[sender]
238
239		messages = []
240
241		# Add system prompt if provided
242		if self.ollama_prompt:
243		messages.append({
244		"role": "system",
245		"content": self.ollama_prompt
246		})
247
248		# Add conversation context if available
249		if sender and room_id:
250		context_key = f"{sender}:{room_id}"
251		for ctx_msg in self.user_contexts[context_key]:
252		messages.append(ctx_msg)
253
254		# Add user message
255		user_message = {
256		"role": "user",
257		"content": message
258		}
259		messages.append(user_message)
260
261		# Store user message in context
262		if sender and room_id:
263		context_key = f"{sender}:{room_id}"
264		self.user_contexts[context_key].append(user_message)
265		# Trim context if it gets too long
266		if len(self.user_contexts[context_key]) > self.max_context_length:
267		self.user_contexts[context_key] = self.user_contexts[context_key][-self.max_context_length:]
268
269		response = await ollama_client.chat(
270		model=model,
271		messages=messages
272		)
273
274		response_message = response["message"]
275		message_content = response_message["content"]
276
277		# Store assistant response in context
278		if sender and room_id:
279		context_key = f"{sender}:{room_id}"
280		self.user_contexts[context_key].append({
281		"role": "assistant",
282		"content": message_content
283		})
284		# Trim context if it gets too long
285		if len(self.user_contexts[context_key]) > self.max_context_length:
286		self.user_contexts[context_key] = self.user_contexts[context_key][-self.max_context_length:]
287
288		logger.debug(f"Received response from Ollama (model: {model}): {message_content}")
289		return message_content
290
291		except Exception as e:
292		logger.error(f"Error processing message with Ollama: {e}")
293		return f"Error processing message: {e}"
294
295		async def _message_callback(self, room: MatrixRoom, event: RoomMessageText) -> None:
296		"""
297		Internal callback for handling incoming messages.
298
299		Args:
300		room: The room the message was sent in
301		event: The message event
302		"""
303		if not isinstance(event, RoomMessageText):
304		return
305
306		# Get message timestamp
307		message_timestamp = event.server_timestamp / 1000
308
309		# Skip messages from before the client connected
310		if self.connection_time and message_timestamp < self.connection_time:
311		logger.debug("Skipping old message")
312		return
313
314		# Skip messages from the bot itself
315		if event.sender == self.user_id:
316		logger.debug("Skipping message from self")
317		return
318
319		message = event.body
320		room_id = room.room_id
321		sender = event.sender
322
323		logger.info(f"Received message in room {room_id} from {sender}: {message}")
324
325		# If a custom callback is set, use it
326		if self.message_callback:
327		await self.message_callback(room, event, self)
328		return
329
330		# Check if this is a command
331		if message.startswith('/'):
332		await self._handle_command(message, room_id, sender)
333		return
334
335		# Extract username from user_id (e.g., "@user:example.com" -> "user")
336		username = self.user_id.split(':')[0].lstrip('@')
337
338		# Check if the message contains the bot's username (case insensitive)
339		if username.lower() in message.lower():
340		logger.info(f"Username mention detected in message: {message}")
341		# Process with Ollama and send response
342		response = await self.process_with_ollama(message, sender, room_id)
343		await self.send_message(room_id, response)
344		return
345
346		# For other messages without username mention, don't automatically respond
347
348		async def _handle_command(self, message: str, room_id: str, sender: str) -> None:
349		"""
350		Handle commands sent to the bot.
351
352		Args:
353		message: The command message
354		room_id: The ID of the room where the command was sent
355		sender: The Matrix user ID of the sender
356		"""
357		# Check if this is a model change command
358		model_match = self.MODEL_COMMAND_PATTERN.match(message)
359		if model_match:
360		model_name = model_match.group(1)
361		await self._change_model(model_name, room_id, sender)
362		return
363
364		# Check if this is a help command
365		if self.HELP_COMMAND_PATTERN.match(message):
366		await self._send_help(room_id)
367		return
368
369		# Check if this is a list models command
370		if self.LIST_MODELS_COMMAND_PATTERN.match(message):
371		await self._list_models(room_id)
372		return
373
374		# Unknown command
375		await self.send_message(room_id, f"Unknown command: {message}\nType /help for available commands.")
376
377		async def _change_model(self, model_name: str, room_id: str, sender: str) -> None:
378		"""
379		Change the Ollama model for a user.
380
381		Args:
382		model_name: The name of the model to change to
383		room_id: The ID of the room where the command was sent
384		sender: The Matrix user ID of the sender
385		"""
386		# Check if the model is available
387		if model_name not in self.available_models:
388		await self.send_message(
389		room_id,
390		f"Model '{model_name}' is not available. Available models: {', '.join(self.available_models)}"
391		)
392		return
393
394		# Change the model for the user
395		self.user_models[sender] = model_name
396
397		# Clear the user's context
398		for key in list(self.user_contexts.keys()):
399		if key.startswith(f"{sender}:"):
400		self.user_contexts[key] = []
401
402		await self.send_message(room_id, f"Changed model to '{model_name}' for {sender}")
403		logger.info(f"Changed model to '{model_name}' for user {sender}")
404
405		async def _send_help(self, room_id: str) -> None:
406		"""
407		Send help information to a room.
408
409		Args:
410		room_id: The ID of the room to send the help to
411		"""
412		help_text = """
413		Available commands:
414		- /help - Show this help message
415		- /models - List available models
416		- /model <model_name> - Change to a different model
417
418		You can also mention me by my username in your message and I'll respond.
419		For example: "Hey {username}, what's the weather like today?"
420		""".format(username=self.user_id.split(':')[0].lstrip('@'))
421		await self.send_message(room_id, help_text.strip())
422
423		async def _list_models(self, room_id: str) -> None:
424		"""
425		List available models to a room.
426
427		Args:
428		room_id: The ID of the room to send the list to
429		"""
430		models_text = f"Available models: {', '.join(self.available_models)}"
431		await self.send_message(room_id, models_text)
432
433		async def get_available_models(self) -> List[str]:
434		"""
435		Get a list of available Ollama models.
436
437		Returns:
438		List[str]: A list of available model names
439		"""
440		if not OLLAMA_AVAILABLE:
441		return [self.ollama_model]
442
443		try:
444		ollama_client = OllamaAsync(host=self.ollama_host)
445		response = await ollama_client.list()
446		models = [model["name"] for model in response["models"]]
447		return models
448		except Exception as e:
449		logger.error(f"Error getting available models: {e}")
450		return [self.ollama_model]
451
452		async def update_available_models(self) -> None:
453		"""
454		Update the list of available models from the Ollama server.
455		"""
456		models = await self.get_available_models()
457		self.available_models = models
458		logger.info(f"Updated available models: {', '.join(models)}")
459
460		def add_admin_user(self, user_id: str) -> None:
461		"""
462		Add a user to the list of admin users.
463
464		Args:
465		user_id: The Matrix user ID to add as an admin
466		"""
467		self.admin_users.add(user_id)
468		logger.info(f"Added admin user: {user_id}")
469
470		def remove_admin_user(self, user_id: str) -> None:
471		"""
472		Remove a user from the list of admin users.
473
474		Args:
475		user_id: The Matrix user ID to remove as an admin
476		"""
477		if user_id in self.admin_users:
478		self.admin_users.remove(user_id)
479		logger.info(f"Removed admin user: {user_id}")
480
481		def set_message_callback(self, callback: Callable) -> None:
482		"""
483		Set a custom callback for handling incoming messages.
484
485		The callback should be an async function that takes three arguments:
486		- room: The MatrixRoom object
487		- event: The RoomMessageText event
488		- client: This MatrixClient instance
489
490		Args:
491		callback: The callback function
492		"""
493		self.message_callback = callback
494
495		async def listen(self) -> None:
496		"""
497		Start listening for messages in joined rooms.
498
499		This method will block until the client is stopped.
500		"""
501		# Set up the message callback
502		self.client.add_event_callback(self._message_callback, RoomMessageText)
503
504		# Start syncing with the server
505		await self.client.sync_forever(timeout=30000, full_state=True)
506
507		async def disconnect(self) -> None:
508		"""
509		Disconnect from the Matrix homeserver.
510		"""
511		await self.client.logout()
512		await self.client.close()
513		logger.info(f"Disconnected from {self.homeserver}")
514
515
516		class MatrixConfig:
517		"""
518		Configuration handler for Matrix integration.
519		"""
520
521		@staticmethod
522		def load_config(config_path: str) -> Dict:
523		"""
524		Load configuration from a YAML file.
525
526		Args:
527		config_path: Path to the configuration file
528
529		Returns:
530		Dict: The configuration as a dictionary
531		"""
532		try:
533		with open(config_path, 'r') as f:
534		config = yaml.safe_load(f)
535
536		logger.info(f"Loaded configuration from {config_path}")
537		return config
538
539		except Exception as e:
540		logger.error(f"Error loading configuration from {config_path}: {e}")
541		raise
542
543		@staticmethod
544		def save_config(config: Dict, config_path: str) -> None:
545		"""
546		Save configuration to a YAML file.
547
548		Args:
549		config: The configuration as a dictionary
550		config_path: Path to save the configuration to
551		"""
552		try:
553		# Create directory if it doesn't exist
554		os.makedirs(os.path.dirname(config_path), exist_ok=True)
555
556		with open(config_path, 'w') as f:
557		yaml.dump(config, f, default_flow_style=False)
558
559		logger.info(f"Saved configuration to {config_path}")
560
561		except Exception as e:
562		logger.error(f"Error saving configuration to {config_path}: {e}")
563		raise
564
565		@staticmethod
566		def create_default_config() -> Dict:
567		"""
568		Create a default configuration.
569
570		Returns:
571		Dict: The default configuration
572		"""
573		return {
574		"MATRIX": {
575		"HOMESERVER": "https://matrix.example.org",
576		"USER": "@user:example.org",
577		"PASSWORD": "your-password",
578		"DEVICE_ID": "ollamatools-matrix",
579		"ROOMS": [
580		"#room:example.org"
581		],
582		"ADMIN_USERS": [
583		"@admin:example.org"
584		]
585		},
586		"OLLAMA": {
587		"HOST": "localhost:11434",
588		"MODEL": "llama3:latest",
589		"PROMPT": "You are a helpful assistant.",
590		"AVAILABLE_MODELS": [
591		"llama3:latest",
592		"mistral:latest",
593		"gemma:latest"
594		]
595		}
596		}
597
598
599		# Async helper functions for the module API
600
601		async def _connect_to_matrix(
602		homeserver: str,
603		user_id: str,
604		password: str,
605		device_id: str = None,
606		ollama_host: str = "localhost:11434",
607		ollama_model: str = "llama3:latest",
608		ollama_prompt: str = None,
609		available_models: List[str] = None,
610		admin_users: List[str] = None,
611		ssl: bool = True
612		) -> Optional[MatrixClient]:
613		"""
614		Connect to a Matrix homeserver.
615
616		Args:
617		homeserver: The URL of the Matrix homeserver
618		user_id: The Matrix user ID (e.g., @user:example.com)
619		password: The password for the Matrix account
620		device_id: A unique identifier for this device (for E2E encryption)
621		ollama_host: The host address for the Ollama API
622		ollama_model: The Ollama model to use for generating responses
623		ollama_prompt: An optional system prompt for the Ollama model
624		available_models: List of available Ollama models
625		admin_users: List of Matrix user IDs that can change models for all users
626		ssl: Whether to use SSL for the Matrix connection
627
628		Returns:
629		Optional[MatrixClient]: A connected Matrix client, or None if connection failed
630		"""
631		client = MatrixClient(
632		homeserver=homeserver,
633		user_id=user_id,
634		password=password,
635		device_id=device_id,
636		ollama_host=ollama_host,
637		ollama_model=ollama_model,
638		ollama_prompt=ollama_prompt,
639		available_models=available_models,
640		admin_users=admin_users,
641		ssl=ssl
642		)
643
644		success = await client.connect()
645
646		if success:
647		return client
648		else:
649		return None
650
651		async def _join_matrix_room(client: MatrixClient, room_id: str) -> bool:
652		"""
653		Join a Matrix room.
654
655		Args:
656		client: A connected Matrix client
657		room_id: The ID of the room to join (e.g., #room:example.com)
658
659		Returns:
660		bool: True if joining was successful, False otherwise
661		"""
662		return await client.join_room(room_id)
663
664		async def _send_matrix_message(client: MatrixClient, room_id: str, message: str) -> bool:
665		"""
666		Send a message to a Matrix room.
667
668		Args:
669		client: A connected Matrix client
670		room_id: The ID of the room to send the message to
671		message: The message to send
672
673		Returns:
674		bool: True if sending was successful, False otherwise
675		"""
676		return await client.send_message(room_id, message)
677
678		async def _listen_for_matrix_messages(client: MatrixClient) -> None:
679		"""
680		Start listening for messages in joined rooms.
681
682		Args:
683		client: A connected Matrix client
684		"""
685		await client.listen()
686
687
688		# Synchronous wrapper functions for the module API
689
690		def connect_to_matrix(
691		homeserver: str,
692		user_id: str,
693		password: str,
694		device_id: str = None,
695		ollama_host: str = "localhost:11434",
696		ollama_model: str = "llama3:latest",
697		ollama_prompt: str = None,
698		available_models: List[str] = None,
699		admin_users: List[str] = None,
700		ssl: bool = True
701		) -> Optional[MatrixClient]:
702		"""
703		Connect to a Matrix homeserver.
704
705		Args:
706		homeserver: The URL of the Matrix homeserver
707		user_id: The Matrix user ID (e.g., @user:example.com)
708		password: The password for the Matrix account
709		device_id: A unique identifier for this device (for E2E encryption)
710		ollama_host: The host address for the Ollama API
711		ollama_model: The Ollama model to use for generating responses
712		ollama_prompt: An optional system prompt for the Ollama model
713		available_models: List of available Ollama models
714		admin_users: List of Matrix user IDs that can change models for all users
715		ssl: Whether to use SSL for the Matrix connection
716
717		Returns:
718		Optional[MatrixClient]: A connected Matrix client, or None if connection failed
719		"""
720		return asyncio.run(_connect_to_matrix(
721		homeserver=homeserver,
722		user_id=user_id,
723		password=password,
724		device_id=device_id,
725		ollama_host=ollama_host,
726		ollama_model=ollama_model,
727		ollama_prompt=ollama_prompt,
728		available_models=available_models,
729		admin_users=admin_users,
730		ssl=ssl
731		))
732
733		def join_matrix_room(client: MatrixClient, room_id: str) -> bool:
734		"""
735		Join a Matrix room.
736
737		Args:
738		client: A connected Matrix client
739		room_id: The ID of the room to join (e.g., #room:example.com)
740
741		Returns:
742		bool: True if joining was successful, False otherwise
743		"""
744		return asyncio.run(_join_matrix_room(client, room_id))
745
746		def send_matrix_message(client: MatrixClient, room_id: str, message: str) -> bool:
747		"""
748		Send a message to a Matrix room.
749
750		Args:
751		client: A connected Matrix client
752		room_id: The ID of the room to send the message to
753		message: The message to send
754
755		Returns:
756		bool: True if sending was successful, False otherwise
757		"""
758		return asyncio.run(_send_matrix_message(client, room_id, message))
759
760		def listen_for_matrix_messages(client: MatrixClient) -> None:
761		"""
762		Start listening for messages in joined rooms.
763
764		This function will block until the client is stopped.
765
766		Args:
767		client: A connected Matrix client
768		"""
769		asyncio.run(_listen_for_matrix_messages(client))
770
771		def load_matrix_config(config_path: str) -> Dict:
772		"""
773		Load Matrix configuration from a YAML file.
774
775		Args:
776		config_path: Path to the configuration file
777
778		Returns:
779		Dict: The configuration as a dictionary
780		"""
781		return MatrixConfig.load_config(config_path)
782
783		def save_matrix_config(config: Dict, config_path: str) -> None:
784		"""
785		Save Matrix configuration to a YAML file.
786
787		Args:
788		config: The configuration as a dictionary
789		config_path: Path to save the configuration to
790		"""
791		MatrixConfig.save_config(config, config_path)
792
793		def create_default_matrix_config() -> Dict:
794		"""
795		Create a default Matrix configuration.
796
797		Returns:
798		Dict: The default configuration
799		"""
800		return MatrixConfig.create_default_config()

ollamatools-project/ollamatools/tools/system_tools.py +0 -335

…	…	@@ -1,335 +0,0 @@
1		"""
2		System tools for OllamaTools.
3		This module provides functions for executing system commands and working with files.
4		"""
5
6		import os
7		import glob
8		import subprocess
9		import logging
10		import shlex
11		from typing import List, Optional, Dict, Any, Union, Set
12
13		# Import the command validation module
14		from ollamatools.command_validator import (
15		validate_and_sanitize_command,
16		CommandValidationError,
17		get_secure_env,
18		is_sudo_command
19		)
20
21		# Import the path validation module
22		from ollamatools.path_validator import (
23		validate_path_for_reading,
24		validate_path_for_writing,
25		validate_path_for_searching,
26		PathValidationError,
27		configure_path_sandbox
28		)
29
30		# Set up logging
31		logger = logging.getLogger(__name__)
32
33		def execute_command(
34		command: str,
35		args: Optional[List[str]] = None,
36		timeout: int = 60,
37		permission_manager = None,
38		custom_allowlist: Optional[set] = None
39		) -> Dict[str, Any]:
40		"""
41		Execute a shell command with arguments.
42
43		Args:
44		command: Command to execute. Can be a simple command or a full command line.
45		args: Command arguments (used only if command is a simple command)
46		timeout: Command timeout in seconds
47		permission_manager: Permission manager instance
48		custom_allowlist: Optional custom allowlist for commands
49
50		Returns:
51		dict: Result with stdout, stderr, and returncode
52		"""
53		# Handle case where command contains both command and arguments as a string
54		if isinstance(command, str) and ' ' in command and args is None:
55		# Split the command into parts using shlex to handle quotes properly
56		try:
57		command_parts = shlex.split(command)
58		command = command_parts[0]
59		args = command_parts[1:] if len(command_parts) > 1 else []
60		logger.info(f"Split command '{command}' with args {args}")
61		except ValueError as e:
62		logger.error(f"Invalid command syntax: {str(e)}")
63		return {"error": f"Invalid command syntax: {str(e)}"}
64
65		# Default args to empty list if None
66		if args is None:
67		args = []
68
69		# Convert args to strings if they aren't already
70		args = [str(arg) for arg in args]
71
72		# Detect sudo before validation to ensure we prompt correctly
73		is_sudo = is_sudo_command(command, args)
74		if is_sudo:
75		logger.warning(f"Sudo command detected: {command} {' '.join(args)}")
76
77		# Check permission if permission manager is provided
78		if permission_manager:
79		function_args = {"command": command, "args": args, "timeout": timeout}
80		permission_granted = permission_manager.request_permission(
81		function_name="execute_command",
82		args=function_args,
83		is_sudo=is_sudo
84		)
85
86		if not permission_granted:
87		logger.warning(f"Permission denied for command: {command} {' '.join(args)}")
88		return {"error": "Permission denied"}
89
90		try:
91		# Validate and sanitize the command and arguments
92		validated_command, validated_args = validate_and_sanitize_command(
93		command, args, custom_allowlist
94		)
95
96		# Construct the full command
97		full_command = [validated_command] + validated_args
98		logger.info(f"Executing validated command: {' '.join(full_command)}")
99
100		# Get secure environment variables
101		secure_env = get_secure_env()
102
103		# Execute the command
104		result = subprocess.run(
105		full_command,
106		capture_output=True,
107		check=False,
108		text=True,
109		timeout=timeout,
110		env=secure_env
111		)
112
113		# Return the result
114		return {
115		"stdout": result.stdout,
116		"stderr": result.stderr,
117		"returncode": result.returncode,
118		"command": ' '.join(full_command) # Include the executed command for reference
119		}
120		except CommandValidationError as e:
121		logger.error(f"Command validation error: {str(e)}")
122		return {
123		"error": f"Command validation error: {str(e)}",
124		"command": f"{command} {' '.join(args) if args else ''}"
125		}
126		except subprocess.TimeoutExpired:
127		logger.error(f"Command timed out after {timeout} seconds: {command} {' '.join(args)}")
128		return {
129		"error": f"Command timed out after {timeout} seconds",
130		"command": f"{command} {' '.join(args) if args else ''}"
131		}
132		except Exception as e:
133		logger.error(f"Error executing command: {str(e)}")
134		return {
135		"error": str(e),
136		"command": f"{command} {' '.join(args) if args else ''}"
137		}
138
139		def read_file(
140		path: str,
141		max_size: Optional[int] = None,
142		permission_manager = None,
143		sandbox_path: Optional[str] = None,
144		additional_allowed_paths: Optional[Set[str]] = None
145		) -> str:
146		"""
147		Read file contents.
148
149		Args:
150		path: File path
151		max_size: Maximum file size to read in bytes
152		permission_manager: Permission manager instance
153		sandbox_path: Base path that defines the sandbox
154		additional_allowed_paths: Additional paths to allow
155
156		Returns:
157		str: File contents
158		"""
159		# Check permission if permission manager is provided
160		if permission_manager:
161		function_args = {"path": path, "max_size": max_size}
162		permission_granted = permission_manager.request_permission(
163		function_name="read_file",
164		args=function_args
165		)
166
167		if not permission_granted:
168		logger.warning(f"Permission denied for reading file: {path}")
169		return "Permission denied"
170
171		try:
172		# Validate the path for reading
173		validated_path = validate_path_for_reading(
174		path,
175		sandbox_path=sandbox_path,
176		additional_allowed_paths=additional_allowed_paths
177		)
178
179		# Check the file size if max_size is specified
180		if max_size is not None:
181		file_size = os.path.getsize(validated_path)
182		if file_size > max_size:
183		logger.error(f"File size ({file_size} bytes) exceeds maximum size ({max_size} bytes): {validated_path}")
184		return f"File size ({file_size} bytes) exceeds maximum size ({max_size} bytes)"
185
186		# Read the file
187		with open(validated_path, "r") as f:
188		content = f.read()
189
190		return content
191		except PathValidationError as e:
192		logger.error(f"Path validation error: {str(e)}")
193		return f"Path validation error: {str(e)}"
194		except Exception as e:
195		logger.error(f"Error reading file: {str(e)}")
196		return f"Error reading file: {str(e)}"
197
198		def write_file(
199		path: str,
200		content: str,
201		mode: str = 'w',
202		permission_manager = None,
203		sandbox_path: Optional[str] = None,
204		additional_allowed_paths: Optional[Set[str]] = None,
205		additional_protected_paths: Optional[Set[str]] = None
206		) -> bool:
207		"""
208		Write content to a file.
209
210		Args:
211		path: File path
212		content: Content to write
213		mode: Write mode ('w' for write, 'a' for append)
214		permission_manager: Permission manager instance
215		sandbox_path: Base path that defines the sandbox
216		additional_allowed_paths: Additional paths to allow
217		additional_protected_paths: Additional paths to protect
218
219		Returns:
220		bool: True if successful
221		"""
222		# Validate the mode
223		if mode not in ['w', 'a']:
224		logger.error(f"Invalid mode: {mode}")
225		return False
226
227		# Check permission if permission manager is provided
228		if permission_manager:
229		function_args = {"path": path, "content": content, "mode": mode}
230		permission_granted = permission_manager.request_permission(
231		function_name="write_file",
232		args=function_args
233		)
234
235		if not permission_granted:
236		logger.warning(f"Permission denied for writing to file: {path}")
237		return False
238
239		try:
240		# Validate the path for writing
241		validated_path = validate_path_for_writing(
242		path,
243		sandbox_path=sandbox_path,
244		additional_allowed_paths=additional_allowed_paths,
245		additional_protected_paths=additional_protected_paths
246		)
247
248		# Create the directory if it doesn't exist
249		os.makedirs(os.path.dirname(validated_path), exist_ok=True)
250
251		# Write to the file
252		with open(validated_path, mode) as f:
253		f.write(content)
254
255		return True
256		except PathValidationError as e:
257		logger.error(f"Path validation error: {str(e)}")
258		return False
259		except Exception as e:
260		logger.error(f"Error writing to file: {str(e)}")
261		return False
262
263		def find_files(
264		pattern: str,
265		directory: str = '.',
266		recursive: bool = True,
267		permission_manager = None,
268		sandbox_path: Optional[str] = None,
269		additional_allowed_paths: Optional[Set[str]] = None
270		) -> List[str]:
271		"""
272		Find files matching a pattern.
273
274		Args:
275		pattern: File pattern
276		directory: Search directory
277		recursive: Whether to search recursively
278		permission_manager: Permission manager instance
279		sandbox_path: Base path that defines the sandbox
280		additional_allowed_paths: Additional paths to allow
281
282		Returns:
283		List[str]: Matching file paths
284		"""
285		# Check permission if permission manager is provided
286		if permission_manager:
287		function_args = {"pattern": pattern, "directory": directory, "recursive": recursive}
288		permission_granted = permission_manager.request_permission(
289		function_name="find_files",
290		args=function_args
291		)
292
293		if not permission_granted:
294		logger.warning(f"Permission denied for finding files in: {directory}")
295		return []
296
297		try:
298		# Validate the directory and pattern for searching
299		validated_dir, validated_pattern = validate_path_for_searching(
300		directory,
301		pattern,
302		sandbox_path=sandbox_path,
303		additional_allowed_paths=additional_allowed_paths
304		)
305
306		# Construct the search pattern
307		search_pattern = os.path.join(validated_dir, '**', validated_pattern) if recursive else os.path.join(validated_dir, validated_pattern)
308
309		# Find files
310		files = glob.glob(search_pattern, recursive=recursive)
311
312		# Additional validation of found files (check if they're still in the sandbox)
313		validated_files = []
314		for file_path in files:
315		try:
316		# Only include files that are within the sandbox
317		abs_path = os.path.abspath(file_path)
318		if os.path.exists(abs_path) and (
319		is_path_in_sandbox(
320		abs_path,
321		sandbox_path=sandbox_path,
322		additional_allowed_paths=additional_allowed_paths
323		)
324		):
325		validated_files.append(abs_path)
326		except Exception as e:
327		logger.warning(f"Skipping file {file_path}: {str(e)}")
328
329		return validated_files
330		except PathValidationError as e:
331		logger.error(f"Path validation error: {str(e)}")
332		return []
333		except Exception as e:
334		logger.error(f"Error finding files: {str(e)}")
335		return []

ollamatools-project/ollamatools/tools/web_tools.py +0 -484

…	…	@@ -1,484 +0,0 @@
1		"""
2		Web tools module for OllamaTools.
3
4		This module provides functions for making web requests and processing responses,
5		including weather information fetching and web search functionality.
6		"""
7
8		import requests
9		from typing import Dict, Any, Optional, Union, List, Set
10		import logging
11		import json
12		from urllib.parse import urlparse, quote
13		from bs4 import BeautifulSoup
14
15		# Import the URL validation module
16		from ollamatools.url_validator import (
17		validate_url,
18		URLValidationError,
19		configure_domain_lists,
20		create_url_validator
21		)
22
23		# Get logger
24		logger = logging.getLogger(__name__)
25
26		def fetch_url(
27		url: str,
28		method: str = 'GET',
29		params: Optional[Dict[str, str]] = None,
30		headers: Optional[Dict[str, str]] = None,
31		data: Optional[Dict[str, Any]] = None,
32		timeout: int = 30,
33		permission_manager = None,
34		allowed_schemes: Optional[Set[str]] = None,
35		domain_allowlist: Optional[Set[str]] = None,
36		domain_blocklist: Optional[Set[str]] = None
37		) -> Dict[str, Any]:
38		"""
39		Fetch a URL with specified parameters.
40
41		Args:
42		url: URL to fetch
43		method: HTTP method (GET, POST, PUT, DELETE, etc.)
44		params: URL parameters
45		headers: HTTP headers
46		data: Request data
47		timeout: Request timeout in seconds
48		permission_manager: Optional permission manager to request permission
49		allowed_schemes: Optional set of allowed URL schemes
50		domain_allowlist: Optional set of allowed domains
51		domain_blocklist: Optional set of blocked domains
52
53		Returns:
54		dict: Response with status, headers, and content
55
56		Raises:
57		ValueError: If URL or method is invalid
58		requests.RequestException: If the request fails
59		PermissionError: If permission is denied
60		"""
61		# Request permission if permission manager is provided
62		if permission_manager:
63		args = {
64		"url": url,
65		"method": method,
66		"params": params,
67		"headers": headers,
68		"data": data
69		}
70
71		if not permission_manager.request_permission("fetch_url", args):
72		logger.warning(f"Permission denied for web request to {url}")
73		raise PermissionError(f"Permission denied for web request to {url}")
74
75		# Validate and normalize the URL
76		try:
77		validated_url = validate_url(
78		url,
79		allowed_schemes=allowed_schemes,
80		domain_allowlist=domain_allowlist,
81		domain_blocklist=domain_blocklist
82		)
83		logger.debug(f"URL validated and normalized: {validated_url}")
84		except URLValidationError as e:
85		logger.error(f"URL validation error: {str(e)}")
86		raise ValueError(f"URL validation failed: {str(e)}")
87
88		# Validate method
89		valid_methods = ['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'HEAD', 'OPTIONS']
90		method = method.upper()
91		if method not in valid_methods:
92		raise ValueError(f"Invalid HTTP method: {method}. Must be one of {valid_methods}")
93
94		# Set default headers if none provided
95		if headers is None:
96		headers = {}
97
98		# Add default User-Agent if not present
99		if 'User-Agent' not in headers:
100		headers['User-Agent'] = 'OllamaTools/0.1.0 (https://github.com/yourusername/ollamatools)'
101
102		logger.info(f"Making {method} request to {validated_url}")
103
104		try:
105		# Create a session to manage cookies and connection pooling
106		with requests.Session() as session:
107		# Configure session with reasonable security defaults
108		session.verify = True # Verify SSL certificates
109
110		# Make the request
111		response = session.request(
112		method=method,
113		url=validated_url,
114		params=params,
115		headers=headers,
116		json=data if method in ['POST', 'PUT', 'PATCH'] else None,
117		timeout=timeout
118		)
119
120		# Try to parse response as JSON, fall back to text
121		content_type = response.headers.get('Content-Type', '')
122		if 'application/json' in content_type:
123		try:
124		content = response.json()
125		except ValueError:
126		content = response.text
127		else:
128		content = response.text
129
130		result = {
131		"status": response.status_code,
132		"headers": dict(response.headers),
133		"content": content,
134		"url": response.url
135		}
136
137		logger.debug(f"Request to {validated_url} completed with status {response.status_code}")
138		return result
139
140		except requests.RequestException as e:
141		logger.error(f"Request to {validated_url} failed: {str(e)}")
142		raise
143
144
145		def get_weather(
146		location: str,
147		days: int = 3,
148		units: str = 'metric',
149		api_key: Optional[str] = None,
150		permission_manager = None,
151		domain_allowlist: Optional[Set[str]] = None,
152		domain_blocklist: Optional[Set[str]] = None
153		) -> Dict[str, Any]:
154		"""
155		Get weather information for a location.
156
157		Args:
158		location: Location name or coordinates
159		days: Number of forecast days (1-7)
160		units: Temperature units ('metric' or 'imperial')
161		api_key: OpenWeatherMap API key (if None, uses config)
162		permission_manager: Optional permission manager to request permission
163		domain_allowlist: Optional set of allowed domains
164		domain_blocklist: Optional set of blocked domains
165
166		Returns:
167		dict: Weather information including current conditions and forecast
168
169		Raises:
170		ValueError: If parameters are invalid
171		requests.RequestException: If the API request fails
172		PermissionError: If permission is denied
173		"""
174		# Validate parameters
175		if not location:
176		raise ValueError("Location cannot be empty")
177
178		if days < 1 or days > 7:
179		raise ValueError("Days must be between 1 and 7")
180
181		valid_units = ['metric', 'imperial']
182		if units not in valid_units:
183		raise ValueError(f"Units must be one of {valid_units}")
184
185		# Request permission if permission manager is provided
186		if permission_manager:
187		args = {
188		"location": location,
189		"days": days,
190		"units": units
191		}
192
193		if not permission_manager.request_permission("get_weather", args):
194		logger.warning(f"Permission denied for weather request for {location}")
195		raise PermissionError(f"Permission denied for weather request for {location}")
196
197		# Check if API key is provided
198		if api_key is None:
199		# In a real implementation, this would get the API key from the config
200		# For now, we'll use a placeholder message
201		logger.error("No API key provided for weather API")
202		raise ValueError("Weather API key is required. Please provide an API key or set it in the configuration.")
203
204		logger.info(f"Fetching weather for {location}")
205
206		try:
207		# Ensure allowlist includes the weather API domain
208		weather_domain_allowlist = {"api.openweathermap.org"}
209		if domain_allowlist:
210		weather_domain_allowlist.update(domain_allowlist)
211
212		# Base URL for OpenWeatherMap API
213		base_url = "https://api.openweathermap.org/data/2.5"
214
215		# Get current weather
216		current_url = f"{base_url}/weather"
217		params = {
218		"q": location,
219		"appid": api_key,
220		"units": units
221		}
222
223		current_response = fetch_url(
224		current_url,
225		params=params,
226		timeout=10,
227		permission_manager=permission_manager,
228		domain_allowlist=weather_domain_allowlist,
229		domain_blocklist=domain_blocklist
230		)
231		current_data = current_response["content"]
232
233		# Get forecast
234		forecast_url = f"{base_url}/forecast"
235		params = {
236		"q": location,
237		"appid": api_key,
238		"units": units,
239		"cnt": days * 8 # 8 forecasts per day (every 3 hours)
240		}
241
242		forecast_response = fetch_url(
243		forecast_url,
244		params=params,
245		timeout=10,
246		permission_manager=permission_manager,
247		domain_allowlist=weather_domain_allowlist,
248		domain_blocklist=domain_blocklist
249		)
250		forecast_data = forecast_response["content"]
251
252		# Format the response
253		weather_info = {
254		"location": {
255		"name": current_data.get("name", ""),
256		"country": current_data.get("sys", {}).get("country", ""),
257		"coordinates": current_data.get("coord", {})
258		},
259		"current": {
260		"temperature": current_data.get("main", {}).get("temp"),
261		"feels_like": current_data.get("main", {}).get("feels_like"),
262		"humidity": current_data.get("main", {}).get("humidity"),
263		"pressure": current_data.get("main", {}).get("pressure"),
264		"wind_speed": current_data.get("wind", {}).get("speed"),
265		"wind_direction": current_data.get("wind", {}).get("deg"),
266		"description": current_data.get("weather", [{}])[0].get("description", ""),
267		"icon": current_data.get("weather", [{}])[0].get("icon", "")
268		},
269		"forecast": []
270		}
271
272		# Process forecast data
273		for item in forecast_data.get("list", [])[:days * 8]:
274		forecast_item = {
275		"timestamp": item.get("dt"),
276		"temperature": item.get("main", {}).get("temp"),
277		"feels_like": item.get("main", {}).get("feels_like"),
278		"humidity": item.get("main", {}).get("humidity"),
279		"pressure": item.get("main", {}).get("pressure"),
280		"wind_speed": item.get("wind", {}).get("speed"),
281		"wind_direction": item.get("wind", {}).get("deg"),
282		"description": item.get("weather", [{}])[0].get("description", ""),
283		"icon": item.get("weather", [{}])[0].get("icon", "")
284		}
285		weather_info["forecast"].append(forecast_item)
286
287		return weather_info
288
289		except Exception as e:
290		logger.error(f"Weather API request failed: {str(e)}")
291		raise
292
293
294		def search_web(
295		query: str,
296		num_results: int = 5,
297		api_key: Optional[str] = None,
298		permission_manager = None,
299		domain_allowlist: Optional[Set[str]] = None,
300		domain_blocklist: Optional[Set[str]] = None
301		) -> List[Dict[str, str]]:
302		"""
303		Search the web for a query.
304
305		Args:
306		query: Search query
307		num_results: Number of results to return (1-10)
308		api_key: Search API key (if None, uses config)
309		permission_manager: Optional permission manager to request permission
310		domain_allowlist: Optional set of allowed domains
311		domain_blocklist: Optional set of blocked domains
312
313		Returns:
314		List[dict]: Search results with title, URL, and snippet
315
316		Raises:
317		ValueError: If parameters are invalid
318		requests.RequestException: If the API request fails
319		PermissionError: If permission is denied
320		"""
321		# Validate parameters
322		if not query:
323		raise ValueError("Query cannot be empty")
324
325		if num_results < 1 or num_results > 10:
326		raise ValueError("Number of results must be between 1 and 10")
327
328		# Request permission if permission manager is provided
329		if permission_manager:
330		args = {
331		"query": query,
332		"num_results": num_results
333		}
334
335		if not permission_manager.request_permission("search_web", args):
336		logger.warning(f"Permission denied for web search: {query}")
337		raise PermissionError(f"Permission denied for web search: {query}")
338
339		# Check if API key is provided
340		if api_key is None:
341		# In a real implementation, this would get the API key from the config
342		# For now, we'll use a fallback to a simple search method that doesn't require API key
343		logger.warning("No API key provided for search API, using fallback method")
344		return _fallback_search(
345		query,
346		num_results,
347		permission_manager=permission_manager,
348		domain_allowlist=domain_allowlist,
349		domain_blocklist=domain_blocklist
350		)
351
352		logger.info(f"Searching the web for: {query}")
353
354		try:
355		# Ensure allowlist includes the search API domain
356		search_domain_allowlist = {"serpapi.com"}
357		if domain_allowlist:
358		search_domain_allowlist.update(domain_allowlist)
359
360		# Example using the Serpapi (Google Search API)
361		# In a real implementation, you would use a proper search API
362		url = "https://serpapi.com/search"
363		params = {
364		"q": query,
365		"api_key": api_key,
366		"num": num_results
367		}
368
369		response = fetch_url(
370		url,
371		params=params,
372		timeout=15,
373		permission_manager=permission_manager,
374		domain_allowlist=search_domain_allowlist,
375		domain_blocklist=domain_blocklist
376		)
377
378		data = response["content"]
379		results = []
380
381		# Process search results
382		for item in data.get("organic_results", [])[:num_results]:
383		result = {
384		"title": item.get("title", ""),
385		"url": item.get("link", ""),
386		"snippet": item.get("snippet", "")
387		}
388		results.append(result)
389
390		return results
391
392		except Exception as e:
393		logger.error(f"Search API request failed: {str(e)}")
394
395		# Fall back to the simple search method if API request fails
396		logger.info("Falling back to simple search method")
397		return _fallback_search(
398		query,
399		num_results,
400		permission_manager=permission_manager,
401		domain_allowlist=domain_allowlist,
402		domain_blocklist=domain_blocklist
403		)
404
405
406		def _fallback_search(
407		query: str,
408		num_results: int = 5,
409		permission_manager = None,
410		domain_allowlist: Optional[Set[str]] = None,
411		domain_blocklist: Optional[Set[str]] = None
412		) -> List[Dict[str, str]]:
413		"""
414		Fallback search method using DuckDuckGo HTML scraping when API is not available.
415
416		Note: This is a fallback method and not recommended for production use.
417		It may break if the search engine changes its HTML structure.
418
419		Args:
420		query: Search query
421		num_results: Number of results to return
422		permission_manager: Optional permission manager to request permission
423		domain_allowlist: Optional set of allowed domains
424		domain_blocklist: Optional set of blocked domains
425
426		Returns:
427		List[dict]: Search results with title, URL, and snippet
428		"""
429		results = []
430		encoded_query = quote(query)
431		url = f"https://html.duckduckgo.com/html/?q={encoded_query}"
432
433		try:
434		# Ensure allowlist includes the search domain
435		search_domain_allowlist = {"html.duckduckgo.com"}
436		if domain_allowlist:
437		search_domain_allowlist.update(domain_allowlist)
438
439		headers = {
440		"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
441		}
442
443		response = fetch_url(
444		url,
445		headers=headers,
446		timeout=15,
447		permission_manager=permission_manager,
448		domain_allowlist=search_domain_allowlist,
449		domain_blocklist=domain_blocklist
450		)
451
452		soup = BeautifulSoup(response["content"], 'html.parser')
453
454		# Find search results
455		result_elements = soup.select('.result')
456
457		for element in result_elements[:num_results]:
458		title_element = element.select_one('.result__title')
459		link_element = element.select_one('.result__url')
460		snippet_element = element.select_one('.result__snippet')
461
462		if title_element and link_element:
463		title = title_element.get_text(strip=True)
464		# The actual URL is in the href attribute of the <a> tag
465		link = title_element.find('a').get('href', '')
466		snippet = snippet_element.get_text(strip=True) if snippet_element else ""
467
468		# Validate URL
469		try:
470		validate_url(link)
471		results.append({
472		"title": title,
473		"url": link,
474		"snippet": snippet
475		})
476		except URLValidationError:
477		logger.warning(f"Skipping result with invalid URL: {link}")
478
479		return results
480
481		except Exception as e:
482		logger.error(f"Fallback search failed: {str(e)}")
483		# Return empty results if even the fallback fails
484		return []

ollamatools-project/ollamatools/url_validator.py +0 -321

…	…	@@ -1,321 +0,0 @@
1		"""
2		URL validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing URLs
5		to prevent security vulnerabilities like SSRF attacks and URL injection.
6		"""
7
8		import re
9		import ipaddress
10		from urllib.parse import urlparse, parse_qs, urlencode, urlunparse, unquote
11		import logging
12		from typing import Dict, Any, List, Set, Optional, Tuple, Union
13
14		# Set up logging
15		logger = logging.getLogger(__name__)
16
17		# Default allowed schemes
18		DEFAULT_ALLOWED_SCHEMES = {'http', 'https'}
19
20		# Default blocklist of domains
21		DEFAULT_DOMAIN_BLOCKLIST = {
22		'localhost', '127.0.0.1', '0.0.0.0', '::1', # Localhost variations
23		'169.254.169.254', # AWS metadata endpoint
24		'169.254.170.2', # AWS ECS metadata endpoint
25		'169.254.170.23', # AWS Lambda metadata endpoint
26		'10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16', # Private IP ranges
27		'fc00::/7', # Unique local addresses (IPv6)
28		'fd00::/8', # Unique local addresses (IPv6)
29		'fe80::/10', # Link-local addresses (IPv6)
30		}
31
32		# Default allowlist of domains (overrides blocklist)
33		DEFAULT_DOMAIN_ALLOWLIST = set()
34
35		# Suspicious URL patterns
36		SUSPICIOUS_URL_PATTERNS = [
37		r'(file|ftp|gopher|data|telnet|ldap|ssh)://', # Non-HTTP protocols
38		r'@', # URL with credentials
39		r'\.\.', # Directory traversal
40		r'<|>|\{|\}|\[|\]|\|', # Special characters
41		]
42
43		class URLValidationError(Exception):
44		"""Exception raised for URL validation errors."""
45		pass
46
47
48		def normalize_url(url: str) -> str:
49		"""
50		Normalize a URL to prevent bypass techniques.
51
52		Args:
53		url: URL to normalize
54
55		Returns:
56		str: Normalized URL
57		"""
58		# Handle empty URLs
59		if not url:
60		raise URLValidationError("URL cannot be empty")
61
62		# Ensure URL is a string
63		url = str(url).strip()
64
65		# Remove control characters and whitespace
66		url = re.sub(r'[\x00-\x20\x7F]', '', url)
67
68		try:
69		# Parse the URL
70		parsed = urlparse(url)
71
72		# Check if scheme and netloc exist
73		if not parsed.scheme or not parsed.netloc:
74		raise URLValidationError(f"Invalid URL: {url} - missing scheme or host")
75
76		# Decode any URL-encoded components that might be hiding malicious content
77		path = unquote(parsed.path)
78
79		# Normalize path to remove traversal sequences
80		path = normalize_path(path)
81
82		# Parse and rebuild query parameters to normalize
83		query_params = parse_qs(parsed.query, keep_blank_values=True)
84		normalized_query = urlencode(query_params, doseq=True)
85
86		# Rebuild the normalized URL
87		normalized_url = urlunparse((
88		parsed.scheme.lower(), # Normalize scheme to lowercase
89		parsed.netloc.lower(), # Normalize domain to lowercase
90		path,
91		parsed.params,
92		normalized_query,
93		'' # Remove fragment as it's client-side only
94		))
95
96		return normalized_url
97
98		except Exception as e:
99		raise URLValidationError(f"URL normalization error: {str(e)}")
100
101
102		def normalize_path(path: str) -> str:
103		"""
104		Normalize a URL path to remove traversal sequences and other issues.
105
106		Args:
107		path: URL path to normalize
108
109		Returns:
110		str: Normalized path
111		"""
112		# Remove consecutive slashes
113		path = re.sub(r'/+', '/', path)
114
115		# Remove trailing slash for consistency unless the path is just "/"
116		if path != '/' and path.endswith('/'):
117		path = path[:-1]
118
119		# Process path components to resolve traversal sequences
120		# This is complex, so we'll do it manually
121		components = path.split('/')
122		normalized_components = []
123
124		for component in components:
125		if component == '' or component == '.':
126		# Skip empty components and current directory references
127		continue
128		elif component == '..':
129		# Go up one directory level
130		if normalized_components:
131		normalized_components.pop()
132		else:
133		normalized_components.append(component)
134
135		# Rebuild the path
136		normalized_path = '/' + '/'.join(normalized_components)
137
138		# Ensure path starts with "/"
139		if not normalized_path.startswith('/'):
140		normalized_path = '/' + normalized_path
141
142		return normalized_path
143
144
145		def is_ip_address_in_blocklist(ip_str: str, blocklist: Set[str]) -> bool:
146		"""
147		Check if an IP address is in the blocklist.
148
149		Args:
150		ip_str: IP address string
151		blocklist: Set of blocked IP addresses or CIDR ranges
152
153		Returns:
154		bool: True if the IP is blocked, False otherwise
155		"""
156		try:
157		# Parse the IP address
158		ip = ipaddress.ip_address(ip_str)
159
160		# Check if the IP is directly in the blocklist
161		if ip_str in blocklist:
162		return True
163
164		# Check if the IP is in any of the blocklisted CIDR ranges
165		for block in blocklist:
166		if '/' in block: # It's a CIDR block
167		try:
168		network = ipaddress.ip_network(block, strict=False)
169		if ip in network:
170		return True
171		except ValueError:
172		continue
173
174		return False
175		except ValueError:
176		# Not a valid IP address
177		return False
178
179
180		def validate_url(
181		url: str,
182		allowed_schemes: Optional[Set[str]] = None,
183		domain_allowlist: Optional[Set[str]] = None,
184		domain_blocklist: Optional[Set[str]] = None,
185		) -> str:
186		"""
187		Validate a URL against security rules.
188
189		Args:
190		url: URL to validate
191		allowed_schemes: Optional set of allowed URL schemes (defaults to http, https)
192		domain_allowlist: Optional set of allowed domains
193		domain_blocklist: Optional set of blocked domains
194
195		Returns:
196		str: Validated and normalized URL
197
198		Raises:
199		URLValidationError: If the URL does not pass validation
200		"""
201		# Use default values if not provided
202		if allowed_schemes is None:
203		allowed_schemes = DEFAULT_ALLOWED_SCHEMES
204
205		if domain_blocklist is None:
206		domain_blocklist = DEFAULT_DOMAIN_BLOCKLIST
207
208		if domain_allowlist is None:
209		domain_allowlist = DEFAULT_DOMAIN_ALLOWLIST
210
211		# Normalize the URL
212		try:
213		normalized_url = normalize_url(url)
214		except URLValidationError as e:
215		raise URLValidationError(f"URL normalization failed: {str(e)}")
216
217		# Parse the normalized URL
218		parsed = urlparse(normalized_url)
219
220		# Check scheme
221		if parsed.scheme not in allowed_schemes:
222		raise URLValidationError(
223		f"URL scheme '{parsed.scheme}' not allowed. Allowed schemes: {allowed_schemes}"
224		)
225
226		# Extract hostname
227		hostname = parsed.netloc.split(':')[0].lower()
228
229		# Check allowlist first (override blocklist)
230		if domain_allowlist and hostname in domain_allowlist:
231		return normalized_url
232
233		# Check hostname against blocklist
234		if hostname in domain_blocklist:
235		raise URLValidationError(f"Domain '{hostname}' is blocked")
236
237		# Check for IP address
238		try:
239		ip = ipaddress.ip_address(hostname)
240
241		# Check if IP is in blocklist
242		if is_ip_address_in_blocklist(str(ip), domain_blocklist):
243		raise URLValidationError(f"IP address '{ip}' is blocked")
244
245		# Check for localhost/internal IP
246		if ip.is_loopback or ip.is_private or ip.is_reserved or ip.is_link_local:
247		raise URLValidationError(f"IP address '{ip}' is internal/reserved")
248		except ValueError:
249		# Not an IP address, continue with domain checks
250		pass
251
252		# Check for suspicious patterns
253		for pattern in SUSPICIOUS_URL_PATTERNS:
254		if re.search(pattern, normalized_url):
255		raise URLValidationError(f"URL contains suspicious pattern: {pattern}")
256
257		# Check port if specified
258		if ':' in parsed.netloc:
259		try:
260		port = int(parsed.netloc.split(':')[1])
261		if port < 1 or port > 65535:
262		raise URLValidationError(f"Invalid port number: {port}")
263		except (IndexError, ValueError):
264		raise URLValidationError(f"Invalid port specification in URL: {parsed.netloc}")
265
266		return normalized_url
267
268
269		def configure_domain_lists(
270		additional_allowlist: Optional[Set[str]] = None,
271		additional_blocklist: Optional[Set[str]] = None
272		) -> Tuple[Set[str], Set[str]]:
273		"""
274		Configure domain allowlist and blocklist by adding custom entries.
275
276		Args:
277		additional_allowlist: Additional domains to allow
278		additional_blocklist: Additional domains to block
279
280		Returns:
281		tuple: (configured_allowlist, configured_blocklist)
282		"""
283		# Start with default lists
284		allowlist = DEFAULT_DOMAIN_ALLOWLIST.copy()
285		blocklist = DEFAULT_DOMAIN_BLOCKLIST.copy()
286
287		# Add additional entries
288		if additional_allowlist:
289		allowlist.update(additional_allowlist)
290
291		if additional_blocklist:
292		blocklist.update(additional_blocklist)
293
294		return allowlist, blocklist
295
296
297		def create_url_validator(
298		allowed_schemes: Optional[Set[str]] = None,
299		domain_allowlist: Optional[Set[str]] = None,
300		domain_blocklist: Optional[Set[str]] = None,
301		) -> callable:
302		"""
303		Create a URL validator function with the specified configuration.
304
305		Args:
306		allowed_schemes: Set of allowed URL schemes
307		domain_allowlist: Set of allowed domains
308		domain_blocklist: Set of blocked domains
309
310		Returns:
311		callable: URL validator function
312		"""
313		def validate(url: str) -> str:
314		return validate_url(
315		url,
316		allowed_schemes=allowed_schemes,
317		domain_allowlist=domain_allowlist,
318		domain_blocklist=domain_blocklist
319		)
320
321		return validate

ollamatools-project/ollamatools/utils/__init__.py +0 -19

…	…	@@ -1,19 +0,0 @@
1		"""
2		Utility modules for OllamaTools.
3		"""
4
5		from .logger import (
6		setup_logger,
7		get_logger,
8		enable_debug_logging,
9		disable_logging,
10		get_ollamatools_logger
11		)
12
13		__all__ = [
14		'setup_logger',
15		'get_logger',
16		'enable_debug_logging',
17		'disable_logging',
18		'get_ollamatools_logger'
19		]

ollamatools-project/ollamatools/utils/logger.py +0 -110

…	…	@@ -1,110 +0,0 @@
1		"""
2		Logging utilities for OllamaTools.
3		This module provides functions for setting up and configuring logging.
4		"""
5
6		import logging
7		import sys
8		from typing import Optional
9
10		def setup_logger(
11		name: str,
12		level: str = "INFO",
13		log_file: Optional[str] = None,
14		console_output: bool = True,
15		format_string: Optional[str] = None
16		) -> logging.Logger:
17		"""
18		Set up a logger with the specified configuration.
19
20		Args:
21		name: Name of the logger
22		level: Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
23		log_file: Optional path to a log file
24		console_output: Whether to output logs to console
25		format_string: Optional custom format string for log messages
26
27		Returns:
28		logging.Logger: Configured logger instance
29		"""
30		# Create logger
31		logger = logging.getLogger(name)
32
33		# Set level
34		level_dict = {
35		"DEBUG": logging.DEBUG,
36		"INFO": logging.INFO,
37		"WARNING": logging.WARNING,
38		"ERROR": logging.ERROR,
39		"CRITICAL": logging.CRITICAL
40		}
41		logger.setLevel(level_dict.get(level.upper(), logging.INFO))
42
43		# Clear existing handlers
44		logger.handlers = []
45
46		# Set format
47		if format_string is None:
48		format_string = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
49		formatter = logging.Formatter(format_string)
50
51		# Add console handler if requested
52		if console_output:
53		console_handler = logging.StreamHandler(sys.stdout)
54		console_handler.setFormatter(formatter)
55		logger.addHandler(console_handler)
56
57		# Add file handler if requested
58		if log_file:
59		file_handler = logging.FileHandler(log_file)
60		file_handler.setFormatter(formatter)
61		logger.addHandler(file_handler)
62
63		return logger
64
65		def get_logger(name: str) -> logging.Logger:
66		"""
67		Get an existing logger or create a new one with default settings.
68
69		Args:
70		name: Name of the logger
71
72		Returns:
73		logging.Logger: Logger instance
74		"""
75		logger = logging.getLogger(name)
76
77		# If the logger doesn't have handlers, set up a default one
78		if not logger.handlers:
79		return setup_logger(name)
80
81		return logger
82
83		def enable_debug_logging(name: str) -> None:
84		"""
85		Enable debug logging for the specified logger.
86
87		Args:
88		name: Name of the logger
89		"""
90		logger = logging.getLogger(name)
91		logger.setLevel(logging.DEBUG)
92
93		def disable_logging(name: str) -> None:
94		"""
95		Disable logging for the specified logger.
96
97		Args:
98		name: Name of the logger
99		"""
100		logger = logging.getLogger(name)
101		logger.setLevel(logging.CRITICAL)
102
103		def get_ollamatools_logger() -> logging.Logger:
104		"""
105		Get the main OllamaTools logger.
106
107		Returns:
108		logging.Logger: OllamaTools logger instance
109		"""
110		return get_logger("ollamatools")

ollamatools-project/path_validator.py +0 -419

…	…	@@ -1,419 +0,0 @@
1		"""
2		Path validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing file paths
5		to prevent path traversal attacks and other file path-related security issues.
6		"""
7
8		import os
9		import re
10		import pathlib
11		from typing import Optional, List, Set, Dict, Any, Union, Tuple
12		import logging
13
14		# Set up logging
15		logger = logging.getLogger(__name__)
16
17		# Default protected paths - should not be modified
18		DEFAULT_PROTECTED_PATHS = {
19		'/etc',
20		'/bin',
21		'/sbin',
22		'/usr/bin',
23		'/usr/sbin',
24		'/usr/lib',
25		'/usr/lib64',
26		'/boot',
27		'/dev',
28		'/proc',
29		'/sys',
30		'/var/lib',
31		'/var/log',
32		'/var/run',
33		'/var/lock',
34		}
35
36		# Default paths to allow even if they're in protected paths
37		DEFAULT_ALLOWED_PATHS = {
38		'/etc/hosts', # commonly read for network configuration
39		'/etc/hostname', # commonly read for system identification
40		'/etc/passwd', # commonly read for user information (non-sensitive part)
41		}
42
43		# Default sandbox path - where operations are restricted to
44		DEFAULT_SANDBOX_PATH = os.path.expanduser('~')
45
46		class PathValidationError(Exception):
47		"""Exception raised for path validation errors."""
48		pass
49
50
51		def normalize_path(path: str) -> str:
52		"""
53		Normalize a file path to resolve symlinks, relative paths, etc.
54
55		Args:
56		path: File path to normalize
57
58		Returns:
59		str: Normalized absolute path
60		"""
61		# Convert to string if it's not already
62		if not isinstance(path, str):
63		path = str(path)
64
65		# Remove any null bytes
66		path = path.replace('\0', '')
67
68		# Trim whitespace
69		path = path.strip()
70
71		# Create an absolute path
72		abs_path = os.path.abspath(path)
73
74		# Normalize the path (resolves ., .., symlinks)
75		try:
76		norm_path = os.path.normpath(abs_path)
77		return norm_path
78		except Exception as e:
79		raise PathValidationError(f"Path normalization error: {str(e)}")
80
81
82		def resolve_symlinks(path: str) -> str:
83		"""
84		Resolve symlinks in a path.
85
86		Args:
87		path: File path that might contain symlinks
88
89		Returns:
90		str: Path with all symlinks resolved
91		"""
92		try:
93		# Use pathlib to resolve symlinks
94		resolved_path = str(pathlib.Path(path).resolve())
95		return resolved_path
96		except Exception as e:
97		raise PathValidationError(f"Failed to resolve symlinks in path '{path}': {str(e)}")
98
99
100		def is_path_traversal_attempt(path: str) -> bool:
101		"""
102		Check if a path contains traversal patterns that might be used in attacks.
103
104		Args:
105		path: File path to check
106
107		Returns:
108		bool: True if the path contains suspicious patterns, False otherwise
109		"""
110		# Normalize for easy pattern matching
111		normalized = os.path.normpath(path)
112
113		# Check for common traversal patterns
114		suspicious_patterns = [
115		r'\.\./', # Parent directory references
116		r'/\.\./', # Parent directory references
117		r'/\.\.$', # Path ending with parent reference
118		r'^\.\./.*', # Path starting with parent reference
119		r'\\\\', # Double backslash (Windows)
120		r'\.\\', # Current directory backslash (Windows)
121		r'//', # Double forward slash
122		r'\0', # Null byte
123		r'%00', # URL-encoded null byte
124		r'%2e%2e%2f', # URL-encoded ../
125		r'%2e%2e/', # URL-encoded ../
126		r'\$\{.*?\}', # Shell variable expansion
127		r'`.*?`', # Command substitution
128		]
129
130		for pattern in suspicious_patterns:
131		if re.search(pattern, path):
132		return True
133
134		return False
135
136
137		def is_path_in_sandbox(
138		path: str,
139		sandbox_path: Optional[str] = None,
140		additional_allowed_paths: Optional[Set[str]] = None
141		) -> bool:
142		"""
143		Check if a path is within the sandbox path or explicitly allowed.
144
145		Args:
146		path: File path to check
147		sandbox_path: Base path that defines the sandbox (default is user's home)
148		additional_allowed_paths: Additional paths to allow
149
150		Returns:
151		bool: True if the path is in the sandbox or allowed, False otherwise
152		"""
153		# Resolve the sandbox path
154		if sandbox_path is None:
155		sandbox_path = DEFAULT_SANDBOX_PATH
156
157		# Normalize all paths for comparison
158		normalized_path = normalize_path(path)
159		normalized_sandbox = normalize_path(sandbox_path)
160
161		# Check if the path is explicitly allowed
162		allowed_paths = DEFAULT_ALLOWED_PATHS.copy()
163		if additional_allowed_paths:
164		allowed_paths.update(additional_allowed_paths)
165
166		if normalized_path in allowed_paths:
167		return True
168
169		# Check if the path is a subdirectory of the sandbox
170		return normalized_path.startswith(normalized_sandbox)
171
172
173		def is_path_protected(
174		path: str,
175		additional_protected_paths: Optional[Set[str]] = None,
176		allowed_paths: Optional[Set[str]] = None
177		) -> bool:
178		"""
179		Check if a path is in a protected area that should not be modified.
180
181		Args:
182		path: File path to check
183		additional_protected_paths: Additional paths to protect
184		allowed_paths: Paths explicitly allowed even if in protected areas
185
186		Returns:
187		bool: True if the path is protected, False otherwise
188		"""
189		# Normalize the path
190		normalized_path = normalize_path(path)
191
192		# Combine default and additional protected paths
193		protected_paths = DEFAULT_PROTECTED_PATHS.copy()
194		if additional_protected_paths:
195		protected_paths.update(additional_protected_paths)
196
197		# Combine default and additional allowed paths
198		all_allowed_paths = DEFAULT_ALLOWED_PATHS.copy()
199		if allowed_paths:
200		all_allowed_paths.update(allowed_paths)
201
202		# First check if the path is explicitly allowed
203		if normalized_path in all_allowed_paths:
204		return False
205
206		# Then check if the path is in a protected area
207		for protected in protected_paths:
208		if normalized_path == protected or normalized_path.startswith(protected + os.path.sep):
209		return True
210
211		return False
212
213
214		def validate_path_for_reading(
215		path: str,
216		sandbox_path: Optional[str] = None,
217		additional_allowed_paths: Optional[Set[str]] = None,
218		resolve_links: bool = True
219		) -> str:
220		"""
221		Validate a path for reading operations (e.g., read_file).
222
223		Args:
224		path: File path to validate
225		sandbox_path: Base path that defines the sandbox
226		additional_allowed_paths: Additional paths to allow
227		resolve_links: Whether to resolve symlinks
228
229		Returns:
230		str: Validated path
231
232		Raises:
233		PathValidationError: If the path does not pass validation
234		"""
235		# Check for empty path
236		if not path:
237		raise PathValidationError("Path cannot be empty")
238
239		# Normalize path
240		normalized_path = normalize_path(path)
241
242		# Check for path traversal attempts
243		if is_path_traversal_attempt(path):
244		raise PathValidationError(f"Path '{path}' contains suspicious traversal patterns")
245
246		# Resolve symlinks if requested
247		if resolve_links:
248		normalized_path = resolve_symlinks(normalized_path)
249
250		# Check if path exists (for reading, the file should exist)
251		if not os.path.exists(normalized_path):
252		raise PathValidationError(f"Path '{path}' does not exist")
253
254		# Check if path is a file (for reading, it should be a file not a directory)
255		if os.path.isdir(normalized_path):
256		raise PathValidationError(f"Path '{path}' is a directory, not a file")
257
258		# Check if path is in sandbox or explicitly allowed
259		if not is_path_in_sandbox(normalized_path, sandbox_path, additional_allowed_paths):
260		raise PathValidationError(f"Path '{path}' is outside the allowed directories")
261
262		return normalized_path
263
264
265		def validate_path_for_writing(
266		path: str,
267		sandbox_path: Optional[str] = None,
268		additional_allowed_paths: Optional[Set[str]] = None,
269		additional_protected_paths: Optional[Set[str]] = None,
270		resolve_links: bool = True
271		) -> str:
272		"""
273		Validate a path for writing operations (e.g., write_file).
274
275		Args:
276		path: File path to validate
277		sandbox_path: Base path that defines the sandbox
278		additional_allowed_paths: Additional paths to allow
279		additional_protected_paths: Additional paths to protect
280		resolve_links: Whether to resolve symlinks
281
282		Returns:
283		str: Validated path
284
285		Raises:
286		PathValidationError: If the path does not pass validation
287		"""
288		# Check for empty path
289		if not path:
290		raise PathValidationError("Path cannot be empty")
291
292		# Normalize path
293		normalized_path = normalize_path(path)
294
295		# Check for path traversal attempts
296		if is_path_traversal_attempt(path):
297		raise PathValidationError(f"Path '{path}' contains suspicious traversal patterns")
298
299		# Resolve symlinks if requested
300		if resolve_links:
301		normalized_path = resolve_symlinks(normalized_path)
302
303		# Check if path is protected (for writing, protected paths are off-limits)
304		if is_path_protected(normalized_path, additional_protected_paths, additional_allowed_paths):
305		raise PathValidationError(f"Path '{path}' is in a protected area")
306
307		# Check if path is in sandbox or explicitly allowed
308		if not is_path_in_sandbox(normalized_path, sandbox_path, additional_allowed_paths):
309		raise PathValidationError(f"Path '{path}' is outside the allowed directories")
310
311		# Ensure the directory exists
312		directory = os.path.dirname(normalized_path)
313		if not os.path.exists(directory):
314		try:
315		# Create the directory if it doesn't exist
316		os.makedirs(directory, exist_ok=True)
317		except Exception as e:
318		raise PathValidationError(f"Cannot create directory for path '{path}': {str(e)}")
319
320		return normalized_path
321
322
323		def validate_path_for_searching(
324		directory: str,
325		pattern: str,
326		sandbox_path: Optional[str] = None,
327		additional_allowed_paths: Optional[Set[str]] = None,
328		resolve_links: bool = True
329		) -> Tuple[str, str]:
330		"""
331		Validate a directory and pattern for file searching operations (e.g., find_files).
332
333		Args:
334		directory: Directory to search in
335		pattern: File pattern to search for
336		sandbox_path: Base path that defines the sandbox
337		additional_allowed_paths: Additional paths to allow
338		resolve_links: Whether to resolve symlinks
339
340		Returns:
341		tuple: (validated_directory, validated_pattern)
342
343		Raises:
344		PathValidationError: If the directory or pattern does not pass validation
345		"""
346		# Check for empty directory or pattern
347		if not directory:
348		raise PathValidationError("Directory cannot be empty")
349
350		if not pattern:
351		raise PathValidationError("Pattern cannot be empty")
352
353		# Normalize directory path
354		normalized_dir = normalize_path(directory)
355
356		# Check for path traversal attempts in directory
357		if is_path_traversal_attempt(directory):
358		raise PathValidationError(f"Directory '{directory}' contains suspicious traversal patterns")
359
360		# Resolve symlinks if requested
361		if resolve_links:
362		normalized_dir = resolve_symlinks(normalized_dir)
363
364		# Check if directory exists
365		if not os.path.exists(normalized_dir):
366		raise PathValidationError(f"Directory '{directory}' does not exist")
367
368		# Check if path is a directory
369		if not os.path.isdir(normalized_dir):
370		raise PathValidationError(f"Path '{directory}' is not a directory")
371
372		# Check if directory is in sandbox or explicitly allowed
373		if not is_path_in_sandbox(normalized_dir, sandbox_path, additional_allowed_paths):
374		raise PathValidationError(f"Directory '{directory}' is outside the allowed directories")
375
376		# Validate and sanitize the pattern
377		# Remove any path components from the pattern
378		sanitized_pattern = os.path.basename(pattern)
379
380		# Check for suspicious patterns in the pattern
381		suspicious_pattern_chars = r'[&|;`$()<>]'
382		if re.search(suspicious_pattern_chars, sanitized_pattern):
383		raise PathValidationError(f"Pattern '{pattern}' contains suspicious characters")
384
385		return normalized_dir, sanitized_pattern
386
387
388		def configure_path_sandbox(
389		sandbox_path: Optional[str] = None,
390		additional_allowed_paths: Optional[Set[str]] = None,
391		additional_protected_paths: Optional[Set[str]] = None
392		) -> Dict[str, Union[str, Set[str]]]:
393		"""
394		Configure the path sandbox with custom settings.
395
396		Args:
397		sandbox_path: Base path that defines the sandbox
398		additional_allowed_paths: Additional paths to allow
399		additional_protected_paths: Additional paths to protect
400
401		Returns:
402		dict: Configuration dictionary with sandbox settings
403		"""
404		# Start with default values
405		config = {
406		'sandbox_path': sandbox_path or DEFAULT_SANDBOX_PATH,
407		'allowed_paths': DEFAULT_ALLOWED_PATHS.copy(),
408		'protected_paths': DEFAULT_PROTECTED_PATHS.copy()
409		}
410
411		# Add additional allowed paths
412		if additional_allowed_paths:
413		config['allowed_paths'].update(additional_allowed_paths)
414
415		# Add additional protected paths
416		if additional_protected_paths:
417		config['protected_paths'].update(additional_protected_paths)
418
419		return config

ollamatools-project/pytest.ini +0 -3

…	…	@@ -1,3 +0,0 @@
1		[pytest]
2		# Skip the agent tests for now since they don't match the current implementation
3		#addopts = --ignore=tests/test_agent.py

ollamatools-project/refactoring_guide.sh +0 -314

…	…	@@ -1,314 +0,0 @@
1		#!/bin/bash
2
3		# Master script to guide the refactoring process
4		# This script outlines the steps to follow for the refactoring process
5		# and provides commands to run at each step
6
7		# Define colors for better readability
8		RED='\033[0;31m'
9		GREEN='\033[0;32m'
10		YELLOW='\033[0;33m'
11		BLUE='\033[0;34m'
12		PURPLE='\033[0;35m'
13		CYAN='\033[0;36m'
14		NC='\033[0m' # No Color
15
16		# Function to print section headers
17		print_header() {
18		echo -e "\n${BLUE}========== $1 ==========${NC}\n"
19		}
20
21		# Function to print steps
22		print_step() {
23		echo -e "${GREEN}Step $1:${NC} $2"
24		}
25
26		# Function to print commands
27		print_command() {
28		echo -e " ${YELLOW}$1${NC}"
29		}
30
31		# Function to print notes
32		print_note() {
33		echo -e " ${PURPLE}Note:${NC} $1"
34		}
35
36		# Function to print warnings
37		print_warning() {
38		echo -e " ${RED}Warning:${NC} $1"
39		}
40
41		# Clear the screen
42		clear
43
44		# Print the title
45		echo -e "${CYAN}"
46		echo "======================================================"
47		echo " OllamaTools Refactoring Guide"
48		echo "======================================================"
49		echo -e "${NC}"
50		echo "This script will guide you through the refactoring process for the OllamaTools project."
51		echo "Follow the steps in order to ensure a smooth refactoring process."
52		echo ""
53		echo -e "${YELLOW}Note: This is a guide only. You will need to run the commands manually.${NC}"
54		echo ""
55
56		# Phase 1: Setup
57		print_header "Phase 1: Setup"
58
59		print_step "1" "Create the new directory structure"
60		print_command "./setup_refactored_structure.sh"
61		print_note "This will create a new directory structure at ../ollamatools-new"
62
63		print_step "2" "Generate the file mapping"
64		print_command "./generate_file_mapping.sh"
65		print_note "This will create a file_mapping.md file with a mapping between the original and new file structures"
66
67		# Phase 2: Core Infrastructure
68		print_header "Phase 2: Core Infrastructure"
69
70		print_step "1" "Implement the Error Handling System"
71		print_command "./copy_refactor_file.sh ollamatools/errors.py ollamatools/core/errors.py"
72		print_note "Refactor the error handling system according to the simplification plan"
73		print_command "./update_mapping_status.sh ollamatools/errors.py 'In Progress' 'Refactoring error handling system'"
74
75		print_step "2" "Implement the Utility Functions"
76		print_command "./copy_refactor_file.sh ollamatools/logger.py ollamatools/core/utils/logger.py"
77		print_note "Refactor the logger according to the simplification plan"
78		print_command "./update_mapping_status.sh ollamatools/logger.py 'In Progress' 'Refactoring logger'"
79
80		print_command "./copy_refactor_file.sh ollamatools/command_validator.py ollamatools/core/utils/validators.py"
81		print_note "Merge the command validator into the validators module"
82		print_command "./update_mapping_status.sh ollamatools/command_validator.py 'In Progress' 'Merging into validators module'"
83
84		print_command "./copy_refactor_file.sh ollamatools/path_validator.py ollamatools/core/utils/validators.py"
85		print_note "Merge the path validator into the validators module"
86		print_command "./update_mapping_status.sh ollamatools/path_validator.py 'In Progress' 'Merging into validators module'"
87
88		print_command "./copy_refactor_file.sh ollamatools/url_validator.py ollamatools/core/utils/validators.py"
89		print_note "Merge the URL validator into the validators module"
90		print_command "./update_mapping_status.sh ollamatools/url_validator.py 'In Progress' 'Merging into validators module'"
91
92		print_step "3" "Implement the Configuration Management"
93		print_command "./copy_refactor_file.sh ollamatools/config.py ollamatools/core/config.py"
94		print_note "Refactor the configuration management according to the simplification plan"
95		print_command "./update_mapping_status.sh ollamatools/config.py 'In Progress' 'Refactoring configuration management'"
96
97		print_step "4" "Implement the Security System"
98		print_command "./copy_refactor_file.sh ollamatools/security.py ollamatools/core/security.py"
99		print_note "Refactor the security system according to the simplification plan"
100		print_command "./update_mapping_status.sh ollamatools/security.py 'In Progress' 'Refactoring security system'"
101
102		print_step "5" "Implement the Base Agent"
103		print_command "./copy_refactor_file.sh ollamatools/agent.py ollamatools/core/agent.py"
104		print_note "Refactor the base agent according to the simplification plan"
105		print_command "./update_mapping_status.sh ollamatools/agent.py 'In Progress' 'Refactoring base agent'"
106
107		print_step "6" "Test the Core Infrastructure"
108		print_command "./test_refactored_code.sh test_agent.py"
109		print_note "Run tests for the agent to ensure it works correctly"
110		print_command "./test_refactored_code.sh test_config.py"
111		print_note "Run tests for the configuration to ensure it works correctly"
112		print_command "./test_refactored_code.sh test_security.py"
113		print_note "Run tests for the security system to ensure it works correctly"
114
115		# Phase 3: Plugin System
116		print_header "Phase 3: Plugin System"
117
118		print_step "1" "Implement the Plugin Base"
119		print_command "./copy_refactor_file.sh ollamatools/plugins/base.py ollamatools/plugins/base.py"
120		print_note "Create the plugin base interfaces"
121		print_command "./update_mapping_status.sh ollamatools/plugins/base.py 'In Progress' 'Creating plugin base interfaces'"
122
123		print_step "2" "Implement the Plugin Registry"
124		print_command "./copy_refactor_file.sh ollamatools/plugins/registry.py ollamatools/plugins/registry.py"
125		print_note "Refactor the plugin registry according to the simplification plan"
126		print_command "./update_mapping_status.sh ollamatools/plugins/registry.py 'In Progress' 'Refactoring plugin registry'"
127
128		print_step "3" "Implement the Plugin Error Handling"
129		print_command "./copy_refactor_file.sh ollamatools/plugins/errors.py ollamatools/plugins/errors.py"
130		print_note "Refactor the plugin error handling according to the simplification plan"
131		print_command "./update_mapping_status.sh ollamatools/plugins/errors.py 'In Progress' 'Refactoring plugin error handling'"
132
133		print_step "4" "Test the Plugin System"
134		print_command "./test_refactored_code.sh test_plugin_system.py"
135		print_note "Run tests for the plugin system to ensure it works correctly"
136
137		# Phase 4: Standard Plugins
138		print_header "Phase 4: Standard Plugins"
139
140		print_step "1" "Implement the Search Plugin"
141		print_command "./copy_refactor_file.sh ollamatools/agent/modular/search.py ollamatools/plugins/standard/search.py"
142		print_note "Refactor the search plugin according to the simplification plan"
143		print_command "./update_mapping_status.sh ollamatools/agent/modular/search.py 'In Progress' 'Refactoring search plugin'"
144
145		print_step "2" "Implement the Commands Plugin"
146		print_command "./copy_refactor_file.sh ollamatools/agent/modular/commands.py ollamatools/plugins/standard/commands.py"
147		print_note "Refactor the commands plugin according to the simplification plan"
148		print_command "./update_mapping_status.sh ollamatools/agent/modular/commands.py 'In Progress' 'Refactoring commands plugin'"
149
150		print_step "3" "Implement the Conversation Plugins"
151		print_command "./copy_refactor_file.sh ollamatools/plugins/conversation_history.py ollamatools/plugins/standard/conversation/history.py"
152		print_note "Refactor the conversation history plugin according to the simplification plan"
153		print_command "./update_mapping_status.sh ollamatools/plugins/conversation_history.py 'In Progress' 'Refactoring conversation history plugin'"
154
155		print_command "./copy_refactor_file.sh ollamatools/plugins/conversation_context.py ollamatools/plugins/standard/conversation/context.py"
156		print_note "Refactor the conversation context plugin according to the simplification plan"
157		print_command "./update_mapping_status.sh ollamatools/plugins/conversation_context.py 'In Progress' 'Refactoring conversation context plugin'"
158
159		print_step "4" "Implement the Feedback Plugin"
160		print_command "./copy_refactor_file.sh ollamatools/agent/modular/feedback.py ollamatools/plugins/standard/feedback.py"
161		print_note "Refactor the feedback plugin according to the simplification plan"
162		print_command "./update_mapping_status.sh ollamatools/agent/modular/feedback.py 'In Progress' 'Refactoring feedback plugin'"
163
164		print_step "5" "Implement the Issues Plugin"
165		print_command "./copy_refactor_file.sh ollamatools/agent/modular/issues.py ollamatools/plugins/standard/issues.py"
166		print_note "Refactor the issues plugin according to the simplification plan"
167		print_command "./update_mapping_status.sh ollamatools/agent/modular/issues.py 'In Progress' 'Refactoring issues plugin'"
168
169		print_step "6" "Test the Standard Plugins"
170		print_command "./test_refactored_code.sh test_search_plugin.py"
171		print_note "Run tests for the search plugin to ensure it works correctly"
172		print_command "./test_refactored_code.sh test_conversation_history_plugin.py"
173		print_note "Run tests for the conversation history plugin to ensure it works correctly"
174		print_command "./test_refactored_code.sh test_conversation_context_plugin.py"
175		print_note "Run tests for the conversation context plugin to ensure it works correctly"
176
177		# Phase 5: Tool Plugins
178		print_header "Phase 5: Tool Plugins"
179
180		print_step "1" "Implement the System Tools Plugin"
181		print_command "./copy_refactor_file.sh ollamatools/tools/system_tools.py ollamatools/plugins/tools/system.py"
182		print_note "Refactor the system tools plugin according to the simplification plan"
183		print_command "./update_mapping_status.sh ollamatools/tools/system_tools.py 'In Progress' 'Refactoring system tools plugin'"
184
185		print_step "2" "Implement the Web Tools Plugin"
186		print_command "./copy_refactor_file.sh ollamatools/tools/web_tools.py ollamatools/plugins/tools/web.py"
187		print_note "Refactor the web tools plugin according to the simplification plan"
188		print_command "./update_mapping_status.sh ollamatools/tools/web_tools.py 'In Progress' 'Refactoring web tools plugin'"
189
190		print_step "3" "Implement the Documentation Tools Plugin"
191		print_command "./copy_refactor_file.sh ollamatools/tools/doc_tools.py ollamatools/plugins/tools/doc.py"
192		print_note "Refactor the documentation tools plugin according to the simplification plan"
193		print_command "./update_mapping_status.sh ollamatools/tools/doc_tools.py 'In Progress' 'Refactoring documentation tools plugin'"
194
195		print_step "4" "Test the Tool Plugins"
196		print_command "./test_refactored_code.sh test_system_tools.py"
197		print_note "Run tests for the system tools plugin to ensure it works correctly"
198		print_command "./test_refactored_code.sh test_web_tools.py"
199		print_note "Run tests for the web tools plugin to ensure it works correctly"
200		print_command "./test_refactored_code.sh test_doc_tools.py"
201		print_note "Run tests for the documentation tools plugin to ensure it works correctly"
202
203		# Phase 6: Integration Plugins
204		print_header "Phase 6: Integration Plugins"
205
206		print_step "1" "Implement the Matrix Integration"
207		print_command "./copy_refactor_file.sh ollamatools/tools/matrix_tools.py ollamatools/plugins/integrations/matrix/client.py"
208		print_note "Refactor the Matrix client according to the simplification plan"
209		print_command "./update_mapping_status.sh ollamatools/tools/matrix_tools.py 'In Progress' 'Refactoring Matrix client'"
210
211		print_command "./copy_refactor_file.sh ollamatools/tools/enhanced_matrix_core.py ollamatools/plugins/integrations/matrix/client.py"
212		print_note "Merge the enhanced Matrix client into the Matrix client module"
213		print_command "./update_mapping_status.sh ollamatools/tools/enhanced_matrix_core.py 'In Progress' 'Merging into Matrix client module'"
214
215		print_command "./copy_refactor_file.sh ollamatools/tools/enhanced_matrix_history.py ollamatools/plugins/integrations/matrix/history.py"
216		print_note "Refactor the Matrix history tracking according to the simplification plan"
217		print_command "./update_mapping_status.sh ollamatools/tools/enhanced_matrix_history.py 'In Progress' 'Refactoring Matrix history tracking'"
218
219		print_command "./copy_refactor_file.sh ollamatools/tools/enhanced_matrix_search.py ollamatools/plugins/integrations/matrix/search.py"
220		print_note "Refactor the Matrix search integration according to the simplification plan"
221		print_command "./update_mapping_status.sh ollamatools/tools/enhanced_matrix_search.py 'In Progress' 'Refactoring Matrix search integration'"
222
223		print_command "./copy_refactor_file.sh ollamatools/tools/matrix_rate_limit_patch.py ollamatools/plugins/integrations/matrix/rate_limit.py"
224		print_note "Refactor the Matrix rate limiting according to the simplification plan"
225		print_command "./update_mapping_status.sh ollamatools/tools/matrix_rate_limit_patch.py 'In Progress' 'Refactoring Matrix rate limiting'"
226
227		print_step "2" "Implement the OpenSearch Integration"
228		print_command "./copy_refactor_file.sh ollamatools/opensearch_web_tools.py ollamatools/plugins/integrations/opensearch/tools.py"
229		print_note "Refactor the OpenSearch web tools according to the simplification plan"
230		print_command "./update_mapping_status.sh ollamatools/opensearch_web_tools.py 'In Progress' 'Refactoring OpenSearch web tools'"
231
232		print_step "3" "Test the Integration Plugins"
233		print_command "./test_refactored_code.sh test_matrix_tools.py"
234		print_note "Run tests for the Matrix integration to ensure it works correctly"
235		print_command "./test_refactored_code.sh test_opensearch_error_handling.py"
236		print_note "Run tests for the OpenSearch integration to ensure it works correctly"
237
238		# Phase 7: Agent Implementations
239		print_header "Phase 7: Agent Implementations"
240
241		print_step "1" "Implement the Basic Agent"
242		print_command "./copy_refactor_file.sh examples/basic_agent.py ollamatools/agents/basic.py"
243		print_note "Refactor the basic agent according to the simplification plan"
244		print_command "./update_mapping_status.sh examples/basic_agent.py 'In Progress' 'Refactoring basic agent'"
245
246		print_step "2" "Implement the Modular Agent"
247		print_command "./copy_refactor_file.sh ollamatools/agent/modular/agent.py ollamatools/agents/modular.py"
248		print_note "Refactor the modular agent according to the simplification plan"
249		print_command "./update_mapping_status.sh ollamatools/agent/modular/agent.py 'In Progress' 'Refactoring modular agent'"
250
251		print_step "3" "Implement the Extended Agent"
252		print_command "./copy_refactor_file.sh ollamatools/agent/extended/agent.py ollamatools/agents/extended.py"
253		print_note "Refactor the extended agent according to the simplification plan"
254		print_command "./update_mapping_status.sh ollamatools/agent/extended/agent.py 'In Progress' 'Refactoring extended agent'"
255
256		print_step "4" "Test the Agent Implementations"
257		print_command "./test_refactored_code.sh test_agent.py"
258		print_note "Run tests for the agents to ensure they work correctly"
259		print_command "./test_refactored_code.sh test_plugin_enabled_agent.py"
260		print_note "Run tests for the plugin-enabled agents to ensure they work correctly"
261
262		# Phase 8: Configuration System
263		print_header "Phase 8: Configuration System"
264
265		print_step "1" "Implement the Configuration Schemas"
266		print_command "./copy_refactor_file.sh ollamatools/config/schemas.py ollamatools/config/schemas.py"
267		print_note "Refactor the configuration schemas according to the simplification plan"
268		print_command "./update_mapping_status.sh ollamatools/config/schemas.py 'In Progress' 'Refactoring configuration schemas'"
269
270		print_step "2" "Implement the Configuration Loader"
271		print_command "./copy_refactor_file.sh ollamatools/config/core_config.py ollamatools/config/loader.py"
272		print_note "Refactor the configuration loader according to the simplification plan"
273		print_command "./update_mapping_status.sh ollamatools/config/core_config.py 'In Progress' 'Refactoring configuration loader'"
274
275		print_step "3" "Implement the Default Configurations"
276		print_command "./copy_refactor_file.sh ollamatools/config/plugin_config.py ollamatools/config/defaults.py"
277		print_note "Refactor the default configurations according to the simplification plan"
278		print_command "./update_mapping_status.sh ollamatools/config/plugin_config.py 'In Progress' 'Refactoring default configurations'"
279
280		print_step "4" "Test the Configuration System"
281		print_command "./test_refactored_code.sh test_config.py"
282		print_note "Run tests for the configuration system to ensure it works correctly"
283
284		# Phase 9: Final Testing
285		print_header "Phase 9: Final Testing"
286
287		print_step "1" "Run all tests"
288		print_command "cd ../ollamatools-new && python -m pytest"
289		print_note "Run all tests to ensure everything works correctly"
290
291		print_step "2" "Run integration tests"
292		print_command "./test_refactored_code.sh test_integration.py"
293		print_note "Run integration tests to ensure all components work together correctly"
294
295		print_step "3" "Run performance tests"
296		print_command "./test_refactored_code.sh test_performance.py"
297		print_note "Run performance tests to ensure the refactored code performs well"
298
299		# Phase 10: Finalization
300		print_header "Phase 10: Finalization"
301
302		print_step "1" "Finalize the refactoring"
303		print_command "./finalize_refactoring.sh"
304		print_note "This will create a backup of the original codebase, rename the original directory to ollamatools-legacy, and rename the new directory to ollamatools-project"
305		print_warning "Make sure all tests pass before running this command!"
306
307		print_header "Refactoring Complete!"
308		echo -e "${GREEN}Congratulations! You have successfully refactored the OllamaTools project.${NC}"
309		echo ""
310		echo "The refactored codebase is now available at ../ollamatools-project"
311		echo "The original codebase is available at ../ollamatools-legacy"
312		echo "A backup of the original codebase is available at ../ollamatools-backup-<timestamp>"
313		echo ""
314		echo -e "${YELLOW}Note: You may need to update any references to the project paths in your environment.${NC}"

ollamatools-project/requirements.txt +0 -16

…	…	@@ -1,16 +0,0 @@
1		# Core dependencies
2		requests>=2.28.0
3		beautifulsoup4>=4.11.0
4		pydantic>=1.10.0
5		markdown>=3.4.0
6		typing-extensions>=4.4.0
7
8		# Test dependencies
9		pytest>=7.0.0
10
11		# Ollama client dependency
12		ollama>=0.4.0
13
14		# Matrix integration dependencies
15		matrix-nio>=0.20.0
16		pyyaml>=6.0.0

ollamatools-project/scripts/install_and_verify.sh +0 -42

…	…	@@ -1,42 +0,0 @@
1		#!/bin/bash
2		# Installation and verification script for OllamaTools
3
4		set -e # Exit on any error
5
6		echo "=== OllamaTools Installation and Verification ==="
7
8		# Create and activate virtual environment
9		echo "Creating virtual environment..."
10		python3.11 -m venv venv
11		source venv/bin/activate
12
13		# Install the package in development mode
14		echo "Installing OllamaTools in development mode..."
15		pip install -e .
16
17		# Install test dependencies
18		echo "Installing test dependencies..."
19		pip install pytest pytest-cov
20
21		# Run the tests
22		echo "Running tests..."
23		pytest -v
24
25		# Run the final demo example
26		echo "Running final demo..."
27		python3.11 examples/final_demo.py
28
29		# Build distribution packages
30		echo "Building distribution packages..."
31		pip install build twine
32		python -m build
33
34		# Check the distributions
35		echo "Checking distribution packages..."
36		twine check dist/*
37
38		echo "=== Installation and Verification Complete ==="
39		echo "OllamaTools is now installed and verified."
40		echo "To run examples, activate the virtual environment with:"
41		echo "source venv/bin/activate"
42		echo "Then run any of the example scripts in the examples directory."

ollamatools-project/secure_doc.md +0 -28

…	…	@@ -1,28 +0,0 @@
1		# Secure Documentation Example
2
3
4		## Secure Document Generation
5
6		This document demonstrates the integration of OllamaTools' documentation
7		utilities with the permission system for enhanced security.
8
9		## Security Features
10
11		- Permission prompts before document generation
12		- Secure handling of potentially sensitive content
13		- Integration with OllamaTools permission system
14
15		## Usage with Permission System
16
17		```python
18		from ollamatools.tools.doc_tools import generate_markdown
19		from ollamatools.security import PermissionManager
20
21		# Initialize permission manager
22		permission_manager = PermissionManager()
23
24		# Generate markdown with permission checks
25		doc = generate_markdown(content, title,
26		format_type='markdown',
27		permission_manager=permission_manager)
28		```

ollamatools-project/setup.py +0 -40

…	…	@@ -1,40 +0,0 @@
1		"""
2		Setup script for the OllamaTools package.
3		"""
4
5		from setuptools import setup, find_packages
6
7		setup(
8		name="ollamatools",
9		version="0.1.0",
10		packages=find_packages(),
11		install_requires=[
12		"requests>=2.28.0",
13		"beautifulsoup4>=4.11.0",
14		"pydantic>=1.10.0",
15		"markdown>=3.4.0",
16		"typing-extensions>=4.4.0",
17		"ollama>=0.4.0",
18		],
19		extras_require={
20		"dev": [
21		"pytest>=7.0.0",
22		],
23		"matrix": [
24		"matrix-nio>=0.20.0",
25		"pyyaml>=6.0.0",
26		],
27		},
28		author="OllamaTools Team",
29		author_email="example@example.com",
30		description="Function Library for Ollama LLMs",
31		long_description=open("README.md").read(),
32		long_description_content_type="text/markdown",
33		url="https://github.com/example/ollamatools",
34		classifiers=[
35		"Programming Language :: Python :: 3",
36		"License :: OSI Approved :: MIT License",
37		"Operating System :: OS Independent",
38		],
39		python_requires=">=3.8",
40		)

ollamatools-project/setup_refactored_structure.sh +0 -232

…	…	@@ -1,232 +0,0 @@
1		#!/bin/bash
2
3		# Script to set up the directory structure for the refactored OllamaTools project
4		# This creates a parallel directory structure alongside the current one
5
6		# Define the parent directory and new project directory
7		PARENT_DIR=".."
8		NEW_PROJECT_DIR="$PARENT_DIR/ollamatools-new"
9
10		# Create the new project directory
11		echo "Creating new project directory: $NEW_PROJECT_DIR"
12		mkdir -p "$NEW_PROJECT_DIR"
13
14		# Copy essential files from the current project
15		echo "Copying essential files from current project..."
16		cp ../ollamatools-project/README.md "$NEW_PROJECT_DIR/"
17		cp ../ollamatools-project/setup.py "$NEW_PROJECT_DIR/"
18		cp ../ollamatools-project/requirements.txt "$NEW_PROJECT_DIR/"
19		cp ../ollamatools-project/CHANGELOG.md "$NEW_PROJECT_DIR/"
20		cp ../ollamatools-project/LICENSE "$NEW_PROJECT_DIR/" 2>/dev/null || echo "LICENSE file not found, skipping"
21		cp ../ollamatools-project/.gitignore "$NEW_PROJECT_DIR/" 2>/dev/null || echo ".gitignore file not found, skipping"
22
23		# Create the main package directory
24		mkdir -p "$NEW_PROJECT_DIR/ollamatools"
25		touch "$NEW_PROJECT_DIR/ollamatools/__init__.py"
26		touch "$NEW_PROJECT_DIR/ollamatools/version.py"
27
28		# Create the directory structure according to the simplification plan
29		echo "Creating directory structure according to the simplification plan..."
30
31		# Core module
32		mkdir -p "$NEW_PROJECT_DIR/ollamatools/core/utils"
33		touch "$NEW_PROJECT_DIR/ollamatools/core/__init__.py"
34		touch "$NEW_PROJECT_DIR/ollamatools/core/agent.py"
35		touch "$NEW_PROJECT_DIR/ollamatools/core/config.py"
36		touch "$NEW_PROJECT_DIR/ollamatools/core/errors.py"
37		touch "$NEW_PROJECT_DIR/ollamatools/core/security.py"
38		touch "$NEW_PROJECT_DIR/ollamatools/core/utils/__init__.py"
39		touch "$NEW_PROJECT_DIR/ollamatools/core/utils/logger.py"
40		touch "$NEW_PROJECT_DIR/ollamatools/core/utils/validators.py"
41		touch "$NEW_PROJECT_DIR/ollamatools/core/utils/formatting.py"
42
43		# Plugin system
44		mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins"
45		touch "$NEW_PROJECT_DIR/ollamatools/plugins/__init__.py"
46		touch "$NEW_PROJECT_DIR/ollamatools/plugins/base.py"
47		touch "$NEW_PROJECT_DIR/ollamatools/plugins/registry.py"
48		touch "$NEW_PROJECT_DIR/ollamatools/plugins/errors.py"
49
50		# Standard plugins
51		mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation"
52		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/__init__.py"
53		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/search.py"
54		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/commands.py"
55		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation/__init__.py"
56		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation/history.py"
57		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation/context.py"
58		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation/analysis.py"
59		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/feedback.py"
60		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/issues.py"
61		touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/function_calling.py"
62
63		# Tool plugins
64		mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins/tools"
65		touch "$NEW_PROJECT_DIR/ollamatools/plugins/tools/__init__.py"
66		touch "$NEW_PROJECT_DIR/ollamatools/plugins/tools/system.py"
67		touch "$NEW_PROJECT_DIR/ollamatools/plugins/tools/web.py"
68		touch "$NEW_PROJECT_DIR/ollamatools/plugins/tools/doc.py"
69
70		# Integration plugins
71		mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix"
72		mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/opensearch"
73		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/__init__.py"
74		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/__init__.py"
75		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/client.py"
76		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/commands.py"
77		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/config.py"
78		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/search.py"
79		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/history.py"
80		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/rate_limit.py"
81		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/opensearch/__init__.py"
82		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/opensearch/client.py"
83		touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/opensearch/tools.py"
84
85		# Agent implementations
86		mkdir -p "$NEW_PROJECT_DIR/ollamatools/agents"
87		touch "$NEW_PROJECT_DIR/ollamatools/agents/__init__.py"
88		touch "$NEW_PROJECT_DIR/ollamatools/agents/basic.py"
89		touch "$NEW_PROJECT_DIR/ollamatools/agents/modular.py"
90		touch "$NEW_PROJECT_DIR/ollamatools/agents/extended.py"
91
92		# Configuration system
93		mkdir -p "$NEW_PROJECT_DIR/ollamatools/config"
94		touch "$NEW_PROJECT_DIR/ollamatools/config/__init__.py"
95		touch "$NEW_PROJECT_DIR/ollamatools/config/schemas.py"
96		touch "$NEW_PROJECT_DIR/ollamatools/config/loader.py"
97		touch "$NEW_PROJECT_DIR/ollamatools/config/defaults.py"
98
99		# Command-line interface
100		mkdir -p "$NEW_PROJECT_DIR/ollamatools/cli"
101		touch "$NEW_PROJECT_DIR/ollamatools/cli/__init__.py"
102		touch "$NEW_PROJECT_DIR/ollamatools/cli/main.py"
103
104		# Tests directory
105		mkdir -p "$NEW_PROJECT_DIR/tests"
106		touch "$NEW_PROJECT_DIR/tests/__init__.py"
107
108		# Examples directory
109		mkdir -p "$NEW_PROJECT_DIR/examples"
110		touch "$NEW_PROJECT_DIR/examples/README.md"
111
112		# Docs directory
113		mkdir -p "$NEW_PROJECT_DIR/docs"
114		touch "$NEW_PROJECT_DIR/docs/README.md"
115
116		# Create a README file in the new project directory explaining the refactoring
117		cat > "$NEW_PROJECT_DIR/REFACTORING.md" << 'EOF'
118		# OllamaTools Refactoring
119
120		This directory contains the refactored version of the OllamaTools project, following the simplification plan outlined in the original project.
121
122		## Directory Structure
123
124		The refactored project follows this directory structure:
125
126		```
127		ollamatools/
128		├── core/ # Core functionality
129		│ ├── __init__.py
130		│ ├── agent.py # Base agent implementation
131		│ ├── config.py # Configuration management
132		│ ├── errors.py # Error definitions and handling
133		│ ├── security.py # Security utilities and permission system
134		│ └── utils/
135		│ ├── __init__.py
136		│ ├── logger.py # Logging utilities
137		│ ├── validators.py # Input validation (combines command/path/url validators)
138		│ └── formatting.py # Text formatting utilities
139		│
140		├── plugins/ # Plugin system and implementations
141		│ ├── __init__.py
142		│ ├── base.py # Base plugin interfaces
143		│ ├── registry.py # Plugin discovery and management
144		│ ├── errors.py # Plugin-specific error handling
145		│ ├── standard/ # Standard plugins
146		│ │ ├── __init__.py
147		│ │ ├── search.py # Search functionality
148		│ │ ├── commands.py # Command management
149		│ │ ├── conversation/
150		│ │ │ ├── __init__.py
151		│ │ │ ├── history.py # Conversation history tracking
152		│ │ │ ├── context.py # Conversation context management
153		│ │ │ └── analysis.py # Topic analysis and metrics
154		│ │ ├── feedback.py # Feedback collection
155		│ │ ├── issues.py # Issue tracking
156		│ │ └── function_calling.py # Semantic function calling
157		│ │
158		│ ├── tools/ # Tool plugins
159		│ │ ├── __init__.py
160		│ │ ├── system.py # System tools (file operations, commands)
161		│ │ ├── web.py # Web tools (HTTP requests, web searches)
162		│ │ └── doc.py # Documentation tools
163		│ │
164		│ └── integrations/ # Integration plugins
165		│ ├── __init__.py
166		│ ├── matrix/
167		│ │ ├── __init__.py
168		│ │ ├── client.py # Matrix client core
169		│ │ ├── commands.py # Matrix-specific commands
170		│ │ ├── config.py # Matrix configuration
171		│ │ ├── search.py # Matrix search integration
172		│ │ ├── history.py # Matrix history tracking
173		│ │ └── rate_limit.py # Rate limiting for Matrix
174		│ └── opensearch/
175		│ ├── __init__.py
176		│ ├── client.py # OpenSearch client
177		│ └── tools.py # OpenSearch web tools
178		│
179		├── agents/ # Agent implementations
180		│ ├── __init__.py
181		│ ├── basic.py # Basic agent implementation
182		│ ├── modular.py # Modular agent implementation
183		│ └── extended.py # Extended agent implementation
184		│
185		├── config/ # Configuration system
186		│ ├── __init__.py
187		│ ├── schemas.py # Configuration schemas
188		│ ├── loader.py # Configuration loading/validation
189		│ └── defaults.py # Default configurations
190		│
191		├── cli/ # Command-line interface
192		│ ├── __init__.py
193		│ └── main.py # Command-line interface
194		│
195		├── __init__.py # Package initialization
196		└── version.py # Version information
197		```
198
199		## Implementation Strategy
200
201		The implementation follows the phased approach outlined in the simplification plan:
202
203		1. Core Infrastructure
204		2. Plugin System
205		3. Standard Plugins
206		4. Tool Plugins
207		5. Integration Plugins
208		6. Agent Implementations
209		7. Configuration System
210		8. Command-Line Interface
211
212		## Using the Original Codebase as Reference
213
214		The original codebase (`../ollamatools-project/`) should be used as a reference when implementing the refactored version. This allows for:
215
216		- Understanding the current functionality
217		- Ensuring feature parity
218		- Reusing code where appropriate
219		- Verifying correctness of the refactored implementation
220
221		## Testing
222
223		Tests should be written for each component as it is implemented. The tests should verify that the refactored implementation maintains the same functionality as the original.
224
225		## Migration
226
227		Once the refactored implementation is complete and tested, the original codebase can be archived and the refactored version can be moved to the main project directory.
228		EOF
229
230		echo "Directory structure setup complete!"
231		echo "New project directory: $NEW_PROJECT_DIR"
232		echo "You can now begin implementing the refactored project using the original codebase as a reference."

ollamatools-project/simple_test_web_tools.py +0 -115

…	…	@@ -1,115 +0,0 @@
1		"""
2		Simple script to manually test the web tools functionality.
3		"""
4
5		import sys
6		import os
7		from unittest.mock import MagicMock
8
9		# Ensure the project directory is in the Python path
10		project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
11		if project_root not in sys.path:
12		sys.path.insert(0, project_root)
13
14		# Import the web tools
15		from ollamatools.tools.web_tools import fetch_url, get_weather, search_web
16
17		def test_fetch_url():
18		"""Test the fetch_url function."""
19		print("\nTesting fetch_url...")
20
21		# Test a simple GET request
22		try:
23		result = fetch_url("https://httpbin.org/get")
24		print(f"Status: {result['status']}")
25		print(f"Content type: {result['headers'].get('Content-Type')}")
26		print("Test passed!")
27		except Exception as e:
28		print(f"Test failed: {e}")
29
30		def test_search_web():
31		"""Test the search_web function."""
32		print("\nTesting search_web...")
33
34		# Test web search with fallback method (no API key)
35		try:
36		results = search_web("Python programming language")
37		print(f"Found {len(results)} results")
38		if results:
39		print(f"First result title: {results[0]['title']}")
40		print("Test passed!")
41		except Exception as e:
42		print(f"Test failed: {e}")
43
44		def test_get_weather_mock():
45		"""Test the get_weather function with mocked permission manager."""
46		print("\nTesting get_weather permission handling...")
47
48		# Create a mock permission manager
49		mock_permission_manager = MagicMock()
50
51		# Test 1: Permission granted
52		mock_permission_manager.request_permission.return_value = True
53
54		try:
55		# This should still fail because we don't have a valid API key,
56		# but it should get past the permission check
57		try:
58		get_weather(
59		"London",
60		api_key="fake_api_key",
61		permission_manager=mock_permission_manager
62		)
63		except ValueError as e:
64		if "API key" in str(e):
65		print("Permission check passed, failed with API key error as expected")
66		else:
67		raise
68
69		# Test that permission was requested
70		if mock_permission_manager.request_permission.called:
71		print("Permission request called correctly")
72		else:
73		print("Permission request was not called")
74		except Exception as e:
75		print(f"Test failed: {e}")
76
77		# Test 2: Permission denied
78		mock_permission_manager.request_permission.return_value = False
79		mock_permission_manager.request_permission.reset_mock()
80
81		try:
82		try:
83		get_weather(
84		"London",
85		api_key="fake_api_key",
86		permission_manager=mock_permission_manager
87		)
88		print("Test failed: Expected PermissionError but no exception was raised")
89		except PermissionError:
90		print("Permission denied test passed: PermissionError was raised")
91		except Exception as e:
92		print(f"Test failed: Expected PermissionError but got {type(e).__name__}: {e}")
93
94		# Test that permission was requested
95		if mock_permission_manager.request_permission.called:
96		print("Permission request called correctly")
97		else:
98		print("Permission request was not called")
99		except Exception as e:
100		print(f"Test failed: {e}")
101
102
103		def main():
104		"""Run all tests."""
105		print("Manual testing of web tools")
106		print("==========================")
107
108		test_fetch_url()
109		test_search_web()
110		test_get_weather_mock()
111
112		print("\nAll tests completed.")
113
114		if __name__ == "__main__":
115		main()

ollamatools-project/simplified_guides/00_INDEX.md +0 -47

…	…	@@ -1,47 +0,0 @@
1		# OllamaTools Simplified Refactoring Guide - Index
2
3		This directory contains a simplified approach to refactoring the OllamaTools project. The original refactoring plan was found to be too complex, particularly because merging files together made them too large. This guide presents a more manageable approach that maintains the target directory structure but avoids merging files.
4
5		## Guide Structure
6
7		The simplified refactoring guide is organized into the following documents:
8
9		1. **[00_INDEX.md](./00_INDEX.md)** - This index document
10		2. **[01_SIMPLIFIED_STRUCTURE.md](./01_SIMPLIFIED_STRUCTURE.md)** - The simplified file structure and file mapping strategy
11		3. **[02_IMPLEMENTATION_STEPS.md](./02_IMPLEMENTATION_STEPS.md)** - Detailed implementation steps for the simplified structure
12		4. **[03_FILE_MAPPING.md](./03_FILE_MAPPING.md)** - Complete file mapping table for reference
13
14		## Purpose
15
16		This guide aims to provide a clear, step-by-step approach to refactoring the OllamaTools project in a way that:
17
18		1. Maintains the target directory structure from the original AI simplification guides
19		2. Avoids creating files that are too large to manage
20		3. Preserves existing functionality
21		4. Minimizes the risk of introducing bugs during refactoring
22
23		## How to Use This Guide
24
25		This guide is designed to be followed sequentially:
26
27		1. Start by reviewing the simplified structure in [01_SIMPLIFIED_STRUCTURE.md](./01_SIMPLIFIED_STRUCTURE.md)
28		2. Follow the implementation steps in [02_IMPLEMENTATION_STEPS.md](./02_IMPLEMENTATION_STEPS.md)
29		3. Use the file mapping table in [03_FILE_MAPPING.md](./03_FILE_MAPPING.md) as a reference during implementation
30
31		For developers:
32		- Use this guide to understand the implementation plan
33		- Follow the steps to create the new directory structure and move files
34		- Update imports and references as described
35		- Test the refactored code to ensure it works correctly
36
37		## Implementation Timeline
38
39		The implementation can be broken down into manageable chunks:
40
41		1. **Week 1**: Create directory structure, copy files
42		2. **Week 2**: Update imports and references
43		3. **Week 3**: Implement new files
44		4. **Week 4**: Update package initialization, test code
45		5. **Week 5**: Documentation, migration guide, final steps
46
47		By following this phased approach, we can implement the simplified structure in a controlled manner, ensuring that each step is properly tested before moving on to the next.

ollamatools-project/simplified_guides/01_SIMPLIFIED_STRUCTURE.md +0 -172

…	…	@@ -1,172 +0,0 @@
1		# Simplified File Structure for OllamaTools Project
2
3		This document outlines a simplified approach to refactoring the OllamaTools project. The original refactoring plan was found to be too complex, particularly because merging files together made them too large. This document presents a more manageable approach that maintains the target directory structure but avoids merging files.
4
5		## Current Large Files (>500 lines)
6
7		Analysis of the current codebase reveals several files that are already quite large:
8
9		- `ollamatools/tools/matrix_tools.py` (800 lines)
10		- `ollamatools/agent/modular/agent.py` (702 lines)
11		- `ollamatools/agent/modular/feedback.py` (586 lines)
12		- `ollamatools/tools/web_tools.py` (484 lines)
13		- `ollamatools/tools/enhanced_matrix_core.py` (466 lines)
14		- `ollamatools/agent.py` (455 lines)
15
16		Merging these already large files would create files that are too difficult to maintain.
17
18		## Revised Simplified Structure
19
20		The target directory structure from the AI simplification guides provides a clear organization. Instead of merging files, we'll focus on organizing existing files into the appropriate directories while maintaining their individual functionality.
21
22		```
23		ollamatools/
24		├── core/ # Core functionality
25		│ ├── __init__.py
26		│ ├── agent.py # Base agent implementation (from ollamatools/agent.py)
27		│ ├── config.py # Configuration management (from ollamatools/config.py)
28		│ ├── errors.py # Error definitions and handling (from ollamatools/errors.py)
29		│ ├── security.py # Security utilities (from ollamatools/security.py)
30		│ └── utils/
31		│ ├── __init__.py
32		│ ├── logger.py # Logging utilities (from ollamatools/logger.py)
33		│ ├── validators/ # Keep validators separate to avoid large files
34		│ │ ├── __init__.py
35		│ │ ├── command.py # Command validation (from ollamatools/command_validator.py)
36		│ │ ├── path.py # Path validation (from ollamatools/path_validator.py)
37		│ │ └── url.py # URL validation (from ollamatools/url_validator.py)
38		│ └── formatting.py # Text formatting utilities (new file)
39		│
40		├── plugins/ # Plugin system and implementations
41		│ ├── __init__.py
42		│ ├── base.py # Base plugin interfaces (new file)
43		│ ├── registry.py # Plugin discovery (from ollamatools/plugins/registry.py)
44		│ ├── errors.py # Plugin-specific errors (from ollamatools/plugins/errors.py)
45		│ ├── standard/ # Standard plugins
46		│ │ ├── __init__.py
47		│ │ ├── search.py # Search functionality (from ollamatools/agent/modular/search.py)
48		│ │ ├── commands.py # Command management (from ollamatools/agent/modular/commands.py)
49		│ │ ├── conversation/
50		│ │ │ ├── __init__.py
51		│ │ │ ├── history.py # History tracking (from ollamatools/plugins/conversation_history.py)
52		│ │ │ ├── context.py # Context management (from ollamatools/plugins/conversation_context.py)
53		│ │ │ └── analysis.py # Topic analysis (new file)
54		│ │ ├── feedback.py # Feedback collection (from ollamatools/agent/modular/feedback.py)
55		│ │ ├── issues.py # Issue tracking (from ollamatools/agent/modular/issues.py)
56		│ │ └── function_calling.py # Semantic function calling (new file)
57		│ │
58		│ ├── tools/ # Tool plugins
59		│ │ ├── __init__.py
60		│ │ ├── system.py # System tools (from ollamatools/tools/system_tools.py)
61		│ │ ├── web.py # Web tools (from ollamatools/tools/web_tools.py)
62		│ │ └── doc.py # Documentation tools (from ollamatools/tools/doc_tools.py)
63		│ │
64		│ └── integrations/ # Integration plugins
65		│ ├── __init__.py
66		│ ├── matrix/ # Keep Matrix files separate to avoid large files
67		│ │ ├── __init__.py
68		│ │ ├── client.py # Matrix client (from ollamatools/tools/matrix_tools.py)
69		│ │ ├── commands.py # Matrix commands (from ollamatools/tools/enhanced_matrix_tools.py)
70		│ │ ├── config.py # Matrix configuration (new file)
71		│ │ ├── search.py # Matrix search (from ollamatools/tools/enhanced_matrix_search.py)
72		│ │ ├── history.py # Matrix history (from ollamatools/tools/enhanced_matrix_history.py)
73		│ │ └── rate_limit.py # Rate limiting (from ollamatools/tools/matrix_rate_limit_patch.py)
74		│ └── opensearch/
75		│ ├── __init__.py
76		│ ├── client.py # OpenSearch client (new file)
77		│ └── tools.py # OpenSearch tools (from ollamatools/opensearch_web_tools.py)
78		│
79		├── agents/ # Agent implementations
80		│ ├── __init__.py
81		│ ├── basic.py # Basic agent (from ollamatools/agent/core.py)
82		│ ├── modular.py # Modular agent (from ollamatools/agent/modular/agent.py)
83		│ └── extended.py # Extended agent (from ollamatools/agent/extended/agent.py)
84		│
85		├── config/ # Configuration system
86		│ ├── __init__.py
87		│ ├── schemas.py # Configuration schemas (from ollamatools/config/schemas.py)
88		│ ├── loader.py # Configuration loading (from ollamatools/config/core_config.py)
89		│ └── defaults.py # Default configurations (from ollamatools/config/plugin_config.py)
90		│
91		├── cli/ # Command-line interface
92		│ ├── __init__.py
93		│ └── main.py # Command-line interface (new file)
94		│
95		├── __init__.py # Package initialization
96		└── version.py # Version information
97		```
98
99		## Key Differences from Original Plan
100
101		1. **Validators Directory**: Instead of merging all validators into a single file, keep them separate to avoid creating another large file.
102
103		2. **Matrix Integration**: Keep Matrix-related files separate rather than merging them, as the original matrix_tools.py is already 800 lines.
104
105		3. **No File Merging**: Focus on organizing files into the appropriate directories without merging them, which avoids creating files that are too large to manage.
106
107		## File Mapping Strategy
108
109		### Core Module
110		- `ollamatools/agent.py` → `ollamatools/core/agent.py`
111		- `ollamatools/errors.py` → `ollamatools/core/errors.py`
112		- `ollamatools/config.py` → `ollamatools/core/config.py`
113		- `ollamatools/security.py` → `ollamatools/core/security.py`
114		- `ollamatools/logger.py` → `ollamatools/core/utils/logger.py`
115		- `ollamatools/command_validator.py` → `ollamatools/core/utils/validators/command.py`
116		- `ollamatools/path_validator.py` → `ollamatools/core/utils/validators/path.py`
117		- `ollamatools/url_validator.py` → `ollamatools/core/utils/validators/url.py`
118
119		### Plugin System
120		- `ollamatools/plugins/registry.py` → `ollamatools/plugins/registry.py` (same location)
121		- `ollamatools/plugins/errors.py` → `ollamatools/plugins/errors.py` (same location)
122		- Create new `ollamatools/plugins/base.py` from existing plugin interfaces
123
124		### Standard Plugins
125		- `ollamatools/agent/modular/search.py` → `ollamatools/plugins/standard/search.py`
126		- `ollamatools/agent/modular/commands.py` → `ollamatools/plugins/standard/commands.py`
127		- `ollamatools/plugins/conversation_history.py` → `ollamatools/plugins/standard/conversation/history.py`
128		- `ollamatools/plugins/conversation_context.py` → `ollamatools/plugins/standard/conversation/context.py`
129		- `ollamatools/agent/modular/feedback.py` → `ollamatools/plugins/standard/feedback.py`
130		- `ollamatools/agent/modular/issues.py` → `ollamatools/plugins/standard/issues.py`
131		- Create new `ollamatools/plugins/standard/function_calling.py`
132
133		### Tool Plugins
134		- `ollamatools/tools/system_tools.py` → `ollamatools/plugins/tools/system.py`
135		- `ollamatools/tools/web_tools.py` → `ollamatools/plugins/tools/web.py`
136		- `ollamatools/tools/doc_tools.py` → `ollamatools/plugins/tools/doc.py`
137
138		### Integration Plugins
139		- `ollamatools/tools/matrix_tools.py` → `ollamatools/plugins/integrations/matrix/client.py`
140		- `ollamatools/tools/enhanced_matrix_tools.py` → `ollamatools/plugins/integrations/matrix/commands.py`
141		- `ollamatools/tools/enhanced_matrix_search.py` → `ollamatools/plugins/integrations/matrix/search.py`
142		- `ollamatools/tools/enhanced_matrix_history.py` → `ollamatools/plugins/integrations/matrix/history.py`
143		- `ollamatools/tools/matrix_rate_limit_patch.py` → `ollamatools/plugins/integrations/matrix/rate_limit.py`
144		- `ollamatools/opensearch_web_tools.py` → `ollamatools/plugins/integrations/opensearch/tools.py`
145		- Create new `ollamatools/plugins/integrations/matrix/config.py`
146		- Create new `ollamatools/plugins/integrations/opensearch/client.py`
147
148		### Agent Implementations
149		- `ollamatools/agent/core.py` → `ollamatools/agents/basic.py`
150		- `ollamatools/agent/modular/agent.py` → `ollamatools/agents/modular.py`
151		- `ollamatools/agent/extended/agent.py` → `ollamatools/agents/extended.py`
152
153		### Configuration System
154		- `ollamatools/config/schemas.py` → `ollamatools/config/schemas.py` (same location)
155		- `ollamatools/config/core_config.py` → `ollamatools/config/loader.py`
156		- `ollamatools/config/plugin_config.py` → `ollamatools/config/defaults.py`
157
158		### CLI
159		- Create new `ollamatools/cli/main.py`
160
161		## Benefits of This Approach
162
163		1. **Maintains file granularity**: Avoids creating files that are too large to manage
164		2. **Preserves existing functionality**: Each file maintains its original purpose
165		3. **Follows the target architecture**: Organizes code according to the simplified structure
166		4. **Easier to implement**: Simpler to move files than to merge and refactor them
167		5. **Reduced risk**: Less chance of introducing bugs during the refactoring process
168		6. **Easier to review**: Changes are more isolated and easier to understand
169
170		## Next Steps
171
172		After understanding the simplified file structure and mapping strategy, proceed to the [Implementation Steps](./02_IMPLEMENTATION_STEPS.md) document to begin implementing the simplified structure.

ollamatools-project/simplified_guides/02_IMPLEMENTATION_STEPS.md +0 -457

…	…	@@ -1,457 +0,0 @@
1		# Detailed Implementation Steps for the Simplified Structure
2
3		This document provides a phased approach to implementing the simplified file structure for the OllamaTools project. By following these steps, you can refactor the codebase in a controlled manner, ensuring that each step is properly tested before moving on to the next.
4
5		## Phase 1: Create the Directory Structure
6
7		1. **Create all necessary directories**:
8		```bash
9		# Core structure
10		mkdir -p ollamatools-new/ollamatools/core/utils/validators
11		mkdir -p ollamatools-new/ollamatools/plugins/standard/conversation
12		mkdir -p ollamatools-new/ollamatools/plugins/tools
13		mkdir -p ollamatools-new/ollamatools/plugins/integrations/matrix
14		mkdir -p ollamatools-new/ollamatools/plugins/integrations/opensearch
15		mkdir -p ollamatools-new/ollamatools/agents
16		mkdir -p ollamatools-new/ollamatools/config
17		mkdir -p ollamatools-new/ollamatools/cli
18		```
19
20		2. **Create empty `__init__.py` files** in each directory to ensure proper Python package structure:
21		```bash
22		find ollamatools-new -type d -exec touch {}/__init__.py \;
23		```
24
25		## Phase 2: Copy Files to New Structure
26
27		1. **Create a file mapping document** that tracks the source and destination of each file:
28		```
29		# File Mapping Document (file_mapping.md)
30		| Source File | Destination File | Status |
31		|-------------|-----------------|--------|
32		| ollamatools/agent.py | ollamatools-new/ollamatools/core/agent.py | Pending |
33		| ollamatools/errors.py | ollamatools-new/ollamatools/core/errors.py | Pending |
34		...
35		```
36
37		2. **Copy files** to their new locations without modifying content initially:
38		```bash
39		# Core module
40		cp ollamatools/agent.py ollamatools-new/ollamatools/core/agent.py
41		cp ollamatools/errors.py ollamatools-new/ollamatools/core/errors.py
42		cp ollamatools/config.py ollamatools-new/ollamatools/core/config.py
43		cp ollamatools/security.py ollamatools-new/ollamatools/core/security.py
44		cp ollamatools/logger.py ollamatools-new/ollamatools/core/utils/logger.py
45		cp ollamatools/command_validator.py ollamatools-new/ollamatools/core/utils/validators/command.py
46		cp ollamatools/path_validator.py ollamatools-new/ollamatools/core/utils/validators/path.py
47		cp ollamatools/url_validator.py ollamatools-new/ollamatools/core/utils/validators/url.py
48
49		# Plugin system
50		cp ollamatools/plugins/registry.py ollamatools-new/ollamatools/plugins/registry.py
51		cp ollamatools/plugins/errors.py ollamatools-new/ollamatools/plugins/errors.py
52
53		# Standard plugins
54		cp ollamatools/agent/modular/search.py ollamatools-new/ollamatools/plugins/standard/search.py
55		cp ollamatools/agent/modular/commands.py ollamatools-new/ollamatools/plugins/standard/commands.py
56		cp ollamatools/plugins/conversation_history.py ollamatools-new/ollamatools/plugins/standard/conversation/history.py
57		cp ollamatools/plugins/conversation_context.py ollamatools-new/ollamatools/plugins/standard/conversation/context.py
58		cp ollamatools/agent/modular/feedback.py ollamatools-new/ollamatools/plugins/standard/feedback.py
59		cp ollamatools/agent/modular/issues.py ollamatools-new/ollamatools/plugins/standard/issues.py
60
61		# Tool plugins
62		cp ollamatools/tools/system_tools.py ollamatools-new/ollamatools/plugins/tools/system.py
63		cp ollamatools/tools/web_tools.py ollamatools-new/ollamatools/plugins/tools/web.py
64		cp ollamatools/tools/doc_tools.py ollamatools-new/ollamatools/plugins/tools/doc.py
65
66		# Integration plugins
67		cp ollamatools/tools/matrix_tools.py ollamatools-new/ollamatools/plugins/integrations/matrix/client.py
68		cp ollamatools/tools/enhanced_matrix_tools.py ollamatools-new/ollamatools/plugins/integrations/matrix/commands.py
69		cp ollamatools/tools/enhanced_matrix_search.py ollamatools-new/ollamatools/plugins/integrations/matrix/search.py
70		cp ollamatools/tools/enhanced_matrix_history.py ollamatools-new/ollamatools/plugins/integrations/matrix/history.py
71		cp ollamatools/tools/matrix_rate_limit_patch.py ollamatools-new/ollamatools/plugins/integrations/matrix/rate_limit.py
72		cp ollamatools/opensearch_web_tools.py ollamatools-new/ollamatools/plugins/integrations/opensearch/tools.py
73
74		# Agent implementations
75		cp ollamatools/agent/core.py ollamatools-new/ollamatools/agents/basic.py
76		cp ollamatools/agent/modular/agent.py ollamatools-new/ollamatools/agents/modular.py
77		cp ollamatools/agent/extended/agent.py ollamatools-new/ollamatools/agents/extended.py
78
79		# Configuration system
80		cp ollamatools/config/schemas.py ollamatools-new/ollamatools/config/schemas.py
81		cp ollamatools/config/core_config.py ollamatools-new/ollamatools/config/loader.py
82		cp ollamatools/config/plugin_config.py ollamatools-new/ollamatools/config/defaults.py
83		```
84
85		3. **Create new files** that don't exist in the current structure:
86		```bash
87		# Create new files with placeholder content
88		touch ollamatools-new/ollamatools/plugins/base.py
89		touch ollamatools-new/ollamatools/plugins/standard/function_calling.py
90		touch ollamatools-new/ollamatools/plugins/integrations/matrix/config.py
91		touch ollamatools-new/ollamatools/plugins/integrations/opensearch/client.py
92		touch ollamatools-new/ollamatools/core/utils/formatting.py
93		touch ollamatools-new/ollamatools/plugins/standard/conversation/analysis.py
94		touch ollamatools-new/ollamatools/cli/main.py
95		touch ollamatools-new/ollamatools/version.py
96		```
97
98		## Phase 3: Update Imports and References
99
100		1. **Create a script to update imports** in each file:
101		```python
102		# update_imports.py
103		import os
104		import re
105
106		def update_imports(file_path):
107		with open(file_path, 'r') as f:
108		content = f.read()
109
110		# Update import patterns
111		# from ollamatools.agent import X -> from ollamatools.core.agent import X
112		content = re.sub(r'from ollamatools\.agent import', 'from ollamatools.core.agent import', content)
113		# ... add more patterns based on the file mapping
114
115		with open(file_path, 'w') as f:
116		f.write(content)
117
118		# Process all Python files in the new structure
119		for root, dirs, files in os.walk('ollamatools-new'):
120		for file in files:
121		if file.endswith('.py'):
122		update_imports(os.path.join(root, file))
123		```
124
125		2. **Manually review and fix imports** in each file:
126		- Update relative imports to absolute imports
127		- Fix circular dependencies
128		- Ensure all imports reference the new file locations
129
130		3. **Update class and function references** to match the new structure:
131		- Update any hardcoded paths or module references
132		- Fix any references to module-level variables
133
134		## Phase 4: Implement New Files
135
136		1. **Implement `plugins/base.py`** with base plugin interfaces:
137		```python
138		# ollamatools-new/ollamatools/plugins/base.py
139		"""Base plugin interfaces for the OllamaTools plugin system."""
140
141		from abc import ABC, abstractmethod
142		from typing import Any, Dict, List, Optional
143
144		class Plugin(ABC):
145		"""Base class for all plugins."""
146
147		@property
148		@abstractmethod
149		def name(self) -> str:
150		"""Return the name of the plugin."""
151		pass
152
153		@property
154		@abstractmethod
155		def version(self) -> str:
156		"""Return the version of the plugin."""
157		pass
158
159		@abstractmethod
160		def initialize(self, config: Dict[str, Any]) -> None:
161		"""Initialize the plugin with the given configuration."""
162		pass
163
164		@abstractmethod
165		def shutdown(self) -> None:
166		"""Shutdown the plugin and release any resources."""
167		pass
168
169		# Add more base classes for specific plugin types
170		```
171
172		2. **Implement `plugins/standard/function_calling.py`** for semantic function calling:
173		```python
174		# ollamatools-new/ollamatools/plugins/standard/function_calling.py
175		"""Plugin for semantic function calling."""
176
177		from typing import Any, Dict, List, Optional
178
179		from ollamatools.plugins.base import Plugin
180
181		class FunctionCallingPlugin(Plugin):
182		"""Plugin for semantic function calling."""
183
184		@property
185		def name(self) -> str:
186		return "function_calling"
187
188		@property
189		def version(self) -> str:
190		return "0.1.0"
191
192		def initialize(self, config: Dict[str, Any]) -> None:
193		"""Initialize the plugin with the given configuration."""
194		self.config = config
195
196		def shutdown(self) -> None:
197		"""Shutdown the plugin and release any resources."""
198		pass
199
200		# Add function calling implementation
201		```
202
203		3. **Implement `plugins/integrations/matrix/config.py`** for Matrix configuration:
204		```python
205		# ollamatools-new/ollamatools/plugins/integrations/matrix/config.py
206		"""Configuration for Matrix integration."""
207
208		from typing import Any, Dict, Optional
209
210		def load_matrix_config(config_path: Optional[str] = None) -> Dict[str, Any]:
211		"""Load Matrix configuration from the given path."""
212		# Implementation
213		return {}
214
215		def validate_matrix_config(config: Dict[str, Any]) -> bool:
216		"""Validate Matrix configuration."""
217		# Implementation
218		return True
219		```
220
221		4. **Implement other new files** with appropriate content based on their purpose.
222
223		## Phase 5: Update Package Initialization
224
225		1. **Update `__init__.py` files** to expose the appropriate classes and functions:
226		```python
227		# ollamatools-new/ollamatools/__init__.py
228		"""OllamaTools package."""
229
230		from ollamatools.version import __version__
231
232		# Expose core components
233		from ollamatools.core.agent import Agent
234		from ollamatools.core.config import Config
235
236		# Expose agent implementations
237		from ollamatools.agents.basic import BasicAgent
238		from ollamatools.agents.modular import ModularAgent
239		from ollamatools.agents.extended import ExtendedAgent
240
241		# Expose plugin system
242		from ollamatools.plugins.base import Plugin
243		from ollamatools.plugins.registry import PluginRegistry
244		```
245
246		2. **Create a version file** with version information:
247		```python
248		# ollamatools-new/ollamatools/version.py
249		"""Version information for OllamaTools."""
250
251		__version__ = "0.1.0"
252		```
253
254		## Phase 6: Test the Refactored Code
255
256		1. **Create a test script** to verify the refactored code:
257		```python
258		# test_refactored_code.py
259		"""Test the refactored OllamaTools code."""
260
261		import sys
262		sys.path.insert(0, 'ollamatools-new')
263
264		# Import and test core components
265		from ollamatools.core.agent import Agent
266		print(f"Successfully imported Agent from core")
267
268		# Import and test plugin system
269		from ollamatools.plugins.registry import PluginRegistry
270		print(f"Successfully imported PluginRegistry")
271
272		# Import and test agent implementations
273		from ollamatools.agents.basic import BasicAgent
274		print(f"Successfully imported BasicAgent")
275
276		# Add more tests for other components
277		```
278
279		2. **Run existing tests** against the refactored code:
280		```bash
281		# Run tests with the refactored code
282		PYTHONPATH=ollamatools-new pytest tests/
283		```
284
285		3. **Fix any issues** discovered during testing:
286		- Update imports that were missed
287		- Fix any broken functionality
288		- Address any compatibility issues
289
290		## Phase 7: Documentation and Migration Guide
291
292		1. **Update documentation** to reflect the new structure:
293		- Update README.md with the new structure
294		- Create a migration guide for users
295		- Document the new plugin system
296
297		2. **Create a migration guide** for users:
298		```markdown
299		# Migration Guide
300
301		This guide helps you migrate from the old OllamaTools structure to the new simplified structure.
302
303		## Import Changes
304
305		Old imports:
306		```python
307		from ollamatools.agent import Agent
308		```
309
310		New imports:
311		```python
312		from ollamatools.core.agent import Agent
313		```
314
315		## Plugin System Changes
316
317		Old plugin usage:
318		```python
319		# Old code
320		```
321
322		New plugin usage:
323		```python
324		# New code
325		```
326		```
327
328		## Phase 8: Final Steps
329
330		1. **Create a setup.py file** for the new package:
331		```python
332		# ollamatools-new/setup.py
333		from setuptools import setup, find_packages
334
335		setup(
336		name="ollamatools",
337		version="0.1.0",
338		packages=find_packages(),
339		install_requires=[
340		# List dependencies
341		],
342		author="Your Name",
343		author_email="your.email@example.com",
344		description="OllamaTools - A toolkit for working with Ollama",
345		keywords="ollama, ai, tools",
346		url="https://github.com/yourusername/ollamatools",
347		)
348		```
349
350		2. **Create a deployment script** to install the new package:
351		```bash
352		# deploy.sh
353		#!/bin/bash
354
355		# Build and install the new package
356		cd ollamatools-new
357		pip install -e .
358		```
359
360		3. **Final verification**:
361		- Run all tests
362		- Verify all examples work
363		- Check documentation
364
365		## Implementation Timeline
366
367		This implementation can be broken down into manageable chunks:
368
369		1. **Week 1**: Phases 1-2 (Create directory structure, copy files)
370		2. **Week 2**: Phase 3 (Update imports and references)
371		3. **Week 3**: Phase 4 (Implement new files)
372		4. **Week 4**: Phases 5-6 (Update package initialization, test code)
373		5. **Week 5**: Phases 7-8 (Documentation, migration guide, final steps)
374
375		By following this phased approach, we can implement the simplified structure in a controlled manner, ensuring that each step is properly tested before moving on to the next.
376
377		## Automation Scripts
378
379		To facilitate the implementation, we can create several automation scripts:
380
381		1. **Directory Structure Creation Script**:
382		```bash
383		#!/bin/bash
384		# create_directory_structure.sh
385
386		# Create the directory structure
387		mkdir -p ollamatools-new/ollamatools/core/utils/validators
388		mkdir -p ollamatools-new/ollamatools/plugins/standard/conversation
389		mkdir -p ollamatools-new/ollamatools/plugins/tools
390		mkdir -p ollamatools-new/ollamatools/plugins/integrations/matrix
391		mkdir -p ollamatools-new/ollamatools/plugins/integrations/opensearch
392		mkdir -p ollamatools-new/ollamatools/agents
393		mkdir -p ollamatools-new/ollamatools/config
394		mkdir -p ollamatools-new/ollamatools/cli
395
396		# Create __init__.py files
397		find ollamatools-new -type d -exec touch {}/__init__.py \;
398
399		echo "Directory structure created successfully."
400		```
401
402		2. **File Copy Script**:
403		```bash
404		#!/bin/bash
405		# copy_files.sh
406
407		# Copy files to their new locations
408		# Core module
409		cp ollamatools/agent.py ollamatools-new/ollamatools/core/agent.py
410		cp ollamatools/errors.py ollamatools-new/ollamatools/core/errors.py
411		# ... add more copy commands
412
413		echo "Files copied successfully."
414		```
415
416		3. **Import Update Script**:
417		```python
418		# update_imports.py
419
420		import os
421		import re
422
423		def update_imports(file_path):
424		with open(file_path, 'r') as f:
425		content = f.read()
426
427		# Update import patterns
428		# ... add patterns
429
430		with open(file_path, 'w') as f:
431		f.write(content)
432
433		# Process all Python files in the new structure
434		for root, dirs, files in os.walk('ollamatools-new'):
435		for file in files:
436		if file.endswith('.py'):
437		update_imports(os.path.join(root, file))
438
439		print("Imports updated successfully.")
440		```
441
442		4. **Test Script**:
443		```bash
444		#!/bin/bash
445		# test_refactored_code.sh
446
447		# Run tests with the refactored code
448		PYTHONPATH=ollamatools-new pytest tests/
449
450		echo "Tests completed."
451		```
452
453		These scripts can be used to automate the implementation process and ensure consistency.
454
455		## Next Steps
456
457		After implementing the simplified structure, proceed to the [File Mapping](./03_FILE_MAPPING.md) document for a complete reference of the file mapping.

ollamatools-project/simplified_guides/03_FILE_MAPPING.md +0 -112

…	…	@@ -1,112 +0,0 @@
1		# Complete File Mapping for OllamaTools Refactoring
2
3		This document provides a complete reference of the file mapping for the OllamaTools refactoring project. Use this table to track the status of each file during the implementation process.
4
5		## File Mapping Table
6
7		| Source File | Destination File | Status | Notes |
8		|-------------|-----------------|--------|-------|
9		| **Core Module** | | | |
10		| `ollamatools/agent.py` | `ollamatools/core/agent.py` | Pending | Base agent implementation |
11		| `ollamatools/errors.py` | `ollamatools/core/errors.py` | Pending | Error definitions and handling |
12		| `ollamatools/config.py` | `ollamatools/core/config.py` | Pending | Configuration management |
13		| `ollamatools/security.py` | `ollamatools/core/security.py` | Pending | Security utilities |
14		| `ollamatools/logger.py` | `ollamatools/core/utils/logger.py` | Pending | Logging utilities |
15		| `ollamatools/command_validator.py` | `ollamatools/core/utils/validators/command.py` | Pending | Command validation |
16		| `ollamatools/path_validator.py` | `ollamatools/core/utils/validators/path.py` | Pending | Path validation |
17		| `ollamatools/url_validator.py` | `ollamatools/core/utils/validators/url.py` | Pending | URL validation |
18		| N/A | `ollamatools/core/utils/formatting.py` | Pending | New file for text formatting utilities |
19		| **Plugin System** | | | |
20		| `ollamatools/plugins/registry.py` | `ollamatools/plugins/registry.py` | Pending | Plugin discovery and management |
21		| `ollamatools/plugins/errors.py` | `ollamatools/plugins/errors.py` | Pending | Plugin-specific error handling |
22		| N/A | `ollamatools/plugins/base.py` | Pending | New file for base plugin interfaces |
23		| **Standard Plugins** | | | |
24		| `ollamatools/agent/modular/search.py` | `ollamatools/plugins/standard/search.py` | Pending | Search functionality |
25		| `ollamatools/agent/modular/commands.py` | `ollamatools/plugins/standard/commands.py` | Pending | Command management |
26		| `ollamatools/plugins/conversation_history.py` | `ollamatools/plugins/standard/conversation/history.py` | Pending | Conversation history tracking |
27		| `ollamatools/plugins/conversation_context.py` | `ollamatools/plugins/standard/conversation/context.py` | Pending | Conversation context management |
28		| N/A | `ollamatools/plugins/standard/conversation/analysis.py` | Pending | New file for topic analysis |
29		| `ollamatools/agent/modular/feedback.py` | `ollamatools/plugins/standard/feedback.py` | Pending | Feedback collection |
30		| `ollamatools/agent/modular/issues.py` | `ollamatools/plugins/standard/issues.py` | Pending | Issue tracking |
31		| N/A | `ollamatools/plugins/standard/function_calling.py` | Pending | New file for semantic function calling |
32		| **Tool Plugins** | | | |
33		| `ollamatools/tools/system_tools.py` | `ollamatools/plugins/tools/system.py` | Pending | System tools |
34		| `ollamatools/tools/web_tools.py` | `ollamatools/plugins/tools/web.py` | Pending | Web tools |
35		| `ollamatools/tools/doc_tools.py` | `ollamatools/plugins/tools/doc.py` | Pending | Documentation tools |
36		| **Matrix Integration** | | | |
37		| `ollamatools/tools/matrix_tools.py` | `ollamatools/plugins/integrations/matrix/client.py` | Pending | Matrix client core |
38		| `ollamatools/tools/enhanced_matrix_tools.py` | `ollamatools/plugins/integrations/matrix/commands.py` | Pending | Matrix-specific commands |
39		| `ollamatools/tools/enhanced_matrix_search.py` | `ollamatools/plugins/integrations/matrix/search.py` | Pending | Matrix search integration |
40		| `ollamatools/tools/enhanced_matrix_history.py` | `ollamatools/plugins/integrations/matrix/history.py` | Pending | Matrix history tracking |
41		| `ollamatools/tools/matrix_rate_limit_patch.py` | `ollamatools/plugins/integrations/matrix/rate_limit.py` | Pending | Rate limiting for Matrix |
42		| N/A | `ollamatools/plugins/integrations/matrix/config.py` | Pending | New file for Matrix configuration |
43		| **OpenSearch Integration** | | | |
44		| `ollamatools/opensearch_web_tools.py` | `ollamatools/plugins/integrations/opensearch/tools.py` | Pending | OpenSearch web tools |
45		| N/A | `ollamatools/plugins/integrations/opensearch/client.py` | Pending | New file for OpenSearch client |
46		| **Agent Implementations** | | | |
47		| `ollamatools/agent/core.py` | `ollamatools/agents/basic.py` | Pending | Basic agent implementation |
48		| `ollamatools/agent/modular/agent.py` | `ollamatools/agents/modular.py` | Pending | Modular agent implementation |
49		| `ollamatools/agent/extended/agent.py` | `ollamatools/agents/extended.py` | Pending | Extended agent implementation |
50		| **Configuration System** | | | |
51		| `ollamatools/config/schemas.py` | `ollamatools/config/schemas.py` | Pending | Configuration schemas |
52		| `ollamatools/config/core_config.py` | `ollamatools/config/loader.py` | Pending | Configuration loading/validation |
53		| `ollamatools/config/plugin_config.py` | `ollamatools/config/defaults.py` | Pending | Default configurations |
54		| **Command-Line Interface** | | | |
55		| N/A | `ollamatools/cli/main.py` | Pending | New file for command-line interface |
56		| **Package Root** | | | |
57		| `ollamatools/__init__.py` | `ollamatools/__init__.py` | Pending | Package initialization |
58		| N/A | `ollamatools/version.py` | Pending | New file for version information |
59
60		## Status Definitions
61
62		- **Pending**: File has not been copied or created yet
63		- **Copied**: File has been copied to the new location but imports have not been updated
64		- **Updated**: File has been copied and imports have been updated
65		- **Tested**: File has been tested and is working correctly
66		- **Completed**: File has been finalized and is ready for release
67
68		## Implementation Notes
69
70		### Large Files (>500 lines)
71
72		Special attention should be given to these large files during the refactoring process:
73
74		- `ollamatools/tools/matrix_tools.py` (800 lines)
75		- `ollamatools/agent/modular/agent.py` (702 lines)
76		- `ollamatools/agent/modular/feedback.py` (586 lines)
77		- `ollamatools/tools/web_tools.py` (484 lines)
78		- `ollamatools/tools/enhanced_matrix_core.py` (466 lines)
79		- `ollamatools/agent.py` (455 lines)
80
81		These files should be moved as-is without attempting to merge them with other files, as they are already quite large.
82
83		### New Files
84
85		The following new files need to be created from scratch:
86
87		- `ollamatools/core/utils/formatting.py`
88		- `ollamatools/plugins/base.py`
89		- `ollamatools/plugins/standard/conversation/analysis.py`
90		- `ollamatools/plugins/standard/function_calling.py`
91		- `ollamatools/plugins/integrations/matrix/config.py`
92		- `ollamatools/plugins/integrations/opensearch/client.py`
93		- `ollamatools/cli/main.py`
94		- `ollamatools/version.py`
95
96		### Files to Keep in Same Location
97
98		The following files will remain in the same location, but may need import updates:
99
100		- `ollamatools/plugins/registry.py`
101		- `ollamatools/plugins/errors.py`
102		- `ollamatools/config/schemas.py`
103
104		## Tracking Progress
105
106		Use this document to track the progress of the refactoring by updating the status of each file as you work through the implementation steps. This will help ensure that no files are missed and that the refactoring is completed systematically.
107
108		To update the status of a file, simply change the "Pending" status to the appropriate status (Copied, Updated, Tested, or Completed) as you progress through the implementation.
109
110		## Next Steps
111
112		After reviewing this file mapping, proceed to implement the refactoring according to the steps outlined in the [Implementation Steps](./02_IMPLEMENTATION_STEPS.md) document.

ollamatools-project/simplified_guides/MIGRATION_GUIDE.md +0 -363

…	…	@@ -1,363 +0,0 @@
1		# Migration Guide for OllamaTools Refactoring
2
3		This guide helps you migrate your code from the old OllamaTools structure to the new simplified structure. It provides examples of how to update imports and adapt your code to work with the refactored codebase.
4
5		## Import Changes
6
7		The most significant changes in the refactoring are the import paths. Here's how to update your imports:
8
9		### Core Module
10
11		**Old imports:**
12		```python
13		from ollamatools.agent import Agent
14		from ollamatools import agent
15		import ollamatools.agent
16		```
17
18		**New imports:**
19		```python
20		from ollamatools.core.agent import Agent
21		from ollamatools.core import agent
22		import ollamatools.core.agent
23		```
24
25		### Agent Implementations
26
27		**Old imports:**
28		```python
29		from ollamatools.agent.core import BasicAgent
30		from ollamatools.agent.modular.agent import ModularAgent
31		from ollamatools.agent.extended.agent import ExtendedAgent
32		```
33
34		**New imports:**
35		```python
36		from ollamatools.agents.basic import BasicAgent
37		from ollamatools.agents.modular import ModularAgent
38		from ollamatools.agents.extended import ExtendedAgent
39		```
40
41		### Standard Plugins
42
43		**Old imports:**
44		```python
45		from ollamatools.agent.modular.search import search_function
46		from ollamatools.agent.modular.commands import execute_command
47		from ollamatools.plugins.conversation_history import ConversationHistory
48		from ollamatools.plugins.conversation_context import ConversationContext
49		from ollamatools.agent.modular.feedback import collect_feedback
50		from ollamatools.agent.modular.issues import track_issue
51		```
52
53		**New imports:**
54		```python
55		from ollamatools.plugins.standard.search import search_function
56		from ollamatools.plugins.standard.commands import execute_command
57		from ollamatools.plugins.standard.conversation.history import ConversationHistory
58		from ollamatools.plugins.standard.conversation.context import ConversationContext
59		from ollamatools.plugins.standard.feedback import collect_feedback
60		from ollamatools.plugins.standard.issues import track_issue
61		```
62
63		### Tool Plugins
64
65		**Old imports:**
66		```python
67		from ollamatools.tools.system_tools import execute_system_command
68		from ollamatools.tools.web_tools import make_web_request
69		from ollamatools.tools.doc_tools import generate_documentation
70		```
71
72		**New imports:**
73		```python
74		from ollamatools.plugins.tools.system import execute_system_command
75		from ollamatools.plugins.tools.web import make_web_request
76		from ollamatools.plugins.tools.doc import generate_documentation
77		```
78
79		### Matrix Integration
80
81		**Old imports:**
82		```python
83		from ollamatools.tools.matrix_tools import MatrixClient
84		from ollamatools.tools.enhanced_matrix_tools import EnhancedMatrixClient
85		from ollamatools.tools.enhanced_matrix_search import MatrixSearch
86		from ollamatools.tools.enhanced_matrix_history import MatrixHistory
87		from ollamatools.tools.matrix_rate_limit_patch import apply_rate_limit_patch
88		```
89
90		**New imports:**
91		```python
92		from ollamatools.plugins.integrations.matrix.client import MatrixClient
93		from ollamatools.plugins.integrations.matrix.commands import EnhancedMatrixClient
94		from ollamatools.plugins.integrations.matrix.search import MatrixSearch
95		from ollamatools.plugins.integrations.matrix.history import MatrixHistory
96		from ollamatools.plugins.integrations.matrix.rate_limit import apply_rate_limit_patch
97		```
98
99		### OpenSearch Integration
100
101		**Old imports:**
102		```python
103		from ollamatools.opensearch_web_tools import OpenSearchWebTools
104		```
105
106		**New imports:**
107		```python
108		from ollamatools.plugins.integrations.opensearch.tools import OpenSearchWebTools
109		```
110
111		### Configuration System
112
113		**Old imports:**
114		```python
115		from ollamatools.config.core_config import load_config
116		from ollamatools.config.plugin_config import get_plugin_config
117		```
118
119		**New imports:**
120		```python
121		from ollamatools.config.loader import load_config
122		from ollamatools.config.defaults import get_plugin_config
123		```
124
125		## Creating Agents
126
127		### Basic Agent
128
129		**Old code:**
130		```python
131		from ollamatools.agent.core import BasicAgent
132
133		agent = BasicAgent(model="llama2")
134		response = agent.chat("Hello, world!")
135		```
136
137		**New code:**
138		```python
139		from ollamatools.agents.basic import BasicAgent
140
141		agent = BasicAgent(model="llama2")
142		response = agent.chat("Hello, world!")
143		```
144
145		### Modular Agent
146
147		**Old code:**
148		```python
149		from ollamatools.agent.modular.agent import ModularAgent
150		from ollamatools.agent.modular.search import SearchModule
151		from ollamatools.agent.modular.commands import CommandsModule
152
153		agent = ModularAgent(model="llama2")
154		agent.add_module(SearchModule())
155		agent.add_module(CommandsModule())
156		response = agent.chat("Search for information about Python")
157		```
158
159		**New code:**
160		```python
161		from ollamatools.agents.modular import ModularAgent
162		from ollamatools.plugins.standard.search import SearchPlugin
163		from ollamatools.plugins.standard.commands import CommandsPlugin
164
165		agent = ModularAgent(model="llama2")
166		agent.add_plugin(SearchPlugin())
167		agent.add_plugin(CommandsPlugin())
168		response = agent.chat("Search for information about Python")
169		```
170
171		### Extended Agent
172
173		**Old code:**
174		```python
175		from ollamatools.agent.extended.agent import ExtendedAgent
176
177		agent = ExtendedAgent(model="llama2")
178		response = agent.chat("Remember this information for later")
179		```
180
181		**New code:**
182		```python
183		from ollamatools.agents.extended import ExtendedAgent
184
185		agent = ExtendedAgent(model="llama2")
186		response = agent.chat("Remember this information for later")
187		```
188
189		## Using Plugins
190
191		### Search Plugin
192
193		**Old code:**
194		```python
195		from ollamatools.agent.modular.search import search_web
196
197		results = search_web("Python programming")
198		```
199
200		**New code:**
201		```python
202		from ollamatools.plugins.standard.search import search_web
203
204		results = search_web("Python programming")
205		```
206
207		### Commands Plugin
208
209		**Old code:**
210		```python
211		from ollamatools.agent.modular.commands import execute_command
212
213		result = execute_command("ls -la")
214		```
215
216		**New code:**
217		```python
218		from ollamatools.plugins.standard.commands import execute_command
219
220		result = execute_command("ls -la")
221		```
222
223		### Conversation History Plugin
224
225		**Old code:**
226		```python
227		from ollamatools.plugins.conversation_history import ConversationHistory
228
229		history = ConversationHistory()
230		history.add_message("user", "Hello")
231		history.add_message("assistant", "Hi there!")
232		```
233
234		**New code:**
235		```python
236		from ollamatools.plugins.standard.conversation.history import ConversationHistory
237
238		history = ConversationHistory()
239		history.add_message("user", "Hello")
240		history.add_message("assistant", "Hi there!")
241		```
242
243		## Using Tools
244
245		### System Tools
246
247		**Old code:**
248		```python
249		from ollamatools.tools.system_tools import execute_system_command, read_file
250
251		result = execute_system_command("ls -la")
252		content = read_file("example.txt")
253		```
254
255		**New code:**
256		```python
257		from ollamatools.plugins.tools.system import execute_system_command, read_file
258
259		result = execute_system_command("ls -la")
260		content = read_file("example.txt")
261		```
262
263		### Web Tools
264
265		**Old code:**
266		```python
267		from ollamatools.tools.web_tools import make_web_request
268
269		response = make_web_request("https://example.com")
270		```
271
272		**New code:**
273		```python
274		from ollamatools.plugins.tools.web import make_web_request
275
276		response = make_web_request("https://example.com")
277		```
278
279		## Using Integrations
280
281		### Matrix Integration
282
283		**Old code:**
284		```python
285		from ollamatools.tools.matrix_tools import MatrixClient
286
287		client = MatrixClient(homeserver="https://matrix.org", token="your_token")
288		client.send_message("!roomid:matrix.org", "Hello, Matrix!")
289		```
290
291		**New code:**
292		```python
293		from ollamatools.plugins.integrations.matrix.client import MatrixClient
294
295		client = MatrixClient(homeserver="https://matrix.org", token="your_token")
296		client.send_message("!roomid:matrix.org", "Hello, Matrix!")
297		```
298
299		### OpenSearch Integration
300
301		**Old code:**
302		```python
303		from ollamatools.opensearch_web_tools import OpenSearchWebTools
304
305		opensearch = OpenSearchWebTools(host="localhost", port=9200)
306		results = opensearch.search("example query")
307		```
308
309		**New code:**
310		```python
311		from ollamatools.plugins.integrations.opensearch.tools import OpenSearchWebTools
312
313		opensearch = OpenSearchWebTools(host="localhost", port=9200)
314		results = opensearch.search("example query")
315		```
316
317		## Configuration System
318
319		**Old code:**
320		```python
321		from ollamatools.config.core_config import load_config
322		from ollamatools.config.plugin_config import get_plugin_config
323
324		config = load_config("config.yaml")
325		plugin_config = get_plugin_config("search")
326		```
327
328		**New code:**
329		```python
330		from ollamatools.config.loader import load_config
331		from ollamatools.config.defaults import get_plugin_config
332
333		config = load_config("config.yaml")
334		plugin_config = get_plugin_config("search")
335		```
336
337		## Common Issues and Solutions
338
339		### Import Errors
340
341		If you encounter import errors, make sure you're using the new import paths as described above. The most common issue is forgetting to update imports to the new structure.
342
343		### Plugin Registration
344
345		In the new structure, plugins are registered using a more consistent API. If you were previously using modules with the modular agent, you'll need to update to use plugins instead.
346
347		**Old code:**
348		```python
349		agent.add_module(SearchModule())
350		```
351
352		**New code:**
353		```python
354		agent.add_plugin(SearchPlugin())
355		```
356
357		### Configuration Changes
358
359		The configuration system has been simplified, but the API remains largely the same. If you're having issues with configuration, make sure you're using the correct import paths.
360
361		## Getting Help
362
363		If you encounter any issues during migration, please refer to the documentation or open an issue on the GitHub repository.

ollamatools-project/simplified_guides/README.md +0 -75

…	…	@@ -1,75 +0,0 @@
1		# OllamaTools Simplified Refactoring Guide
2
3		This directory contains a simplified approach to refactoring the OllamaTools project. The original refactoring plan was found to be too complex, particularly because merging files together made them too large. This guide presents a more manageable approach that maintains the target directory structure but avoids merging files.
4
5		## Background
6
7		The OllamaTools project has evolved through several phases, resulting in a complex architecture with multiple overlapping components. The original AI simplification guides proposed a comprehensive refactoring that involved merging related functionality into consolidated files. However, analysis of the current codebase revealed that several files are already quite large (over 500 lines of code), making further merging impractical.
8
9		This simplified refactoring guide provides an alternative approach that:
10
11		1. Maintains the target directory structure from the original AI simplification guides
12		2. Avoids creating files that are too large to manage by keeping existing files separate
13		3. Preserves existing functionality
14		4. Minimizes the risk of introducing bugs during refactoring
15
16		## Guide Structure
17
18		The simplified refactoring guide is organized into the following documents:
19
20		1. **[00_INDEX.md](./00_INDEX.md)** - Index document with an overview of the guide
21		2. **[01_SIMPLIFIED_STRUCTURE.md](./01_SIMPLIFIED_STRUCTURE.md)** - The simplified file structure and file mapping strategy
22		3. **[02_IMPLEMENTATION_STEPS.md](./02_IMPLEMENTATION_STEPS.md)** - Detailed implementation steps for the simplified structure
23		4. **[03_FILE_MAPPING.md](./03_FILE_MAPPING.md)** - Complete file mapping table for reference
24
25		## Key Differences from Original Plan
26
27		1. **Validators Directory**: Instead of merging all validators into a single file, keep them separate to avoid creating another large file.
28
29		2. **Matrix Integration**: Keep Matrix-related files separate rather than merging them, as the original matrix_tools.py is already 800 lines.
30
31		3. **No File Merging**: Focus on organizing files into the appropriate directories without merging them, which avoids creating files that are too large to manage.
32
33		## Implementation Approach
34
35		The implementation is broken down into manageable phases:
36
37		1. **Phase 1**: Create the directory structure
38		2. **Phase 2**: Copy files to new structure
39		3. **Phase 3**: Update imports and references
40		4. **Phase 4**: Implement new files
41		5. **Phase 5**: Update package initialization
42		6. **Phase 6**: Test the refactored code
43		7. **Phase 7**: Documentation and migration guide
44		8. **Phase 8**: Final steps
45
46		Each phase is detailed in the [Implementation Steps](./02_IMPLEMENTATION_STEPS.md) document.
47
48		## Benefits of This Approach
49
50		1. **Maintains file granularity**: Avoids creating files that are too large to manage
51		2. **Preserves existing functionality**: Each file maintains its original purpose
52		3. **Follows the target architecture**: Organizes code according to the simplified structure
53		4. **Easier to implement**: Simpler to move files than to merge and refactor them
54		5. **Reduced risk**: Less chance of introducing bugs during the refactoring process
55		6. **Easier to review**: Changes are more isolated and easier to understand
56
57		## Getting Started
58
59		To get started with the refactoring, follow these steps:
60
61		1. Start by reviewing the simplified structure in [01_SIMPLIFIED_STRUCTURE.md](./01_SIMPLIFIED_STRUCTURE.md)
62		2. Follow the implementation steps in [02_IMPLEMENTATION_STEPS.md](./02_IMPLEMENTATION_STEPS.md)
63		3. Use the file mapping table in [03_FILE_MAPPING.md](./03_FILE_MAPPING.md) as a reference during implementation
64
65		## Implementation Timeline
66
67		The implementation can be broken down into manageable chunks:
68
69		1. **Week 1**: Phases 1-2 (Create directory structure, copy files)
70		2. **Week 2**: Phase 3 (Update imports and references)
71		3. **Week 3**: Phase 4 (Implement new files)
72		4. **Week 4**: Phases 5-6 (Update package initialization, test code)
73		5. **Week 5**: Phases 7-8 (Documentation, migration guide, final steps)
74
75		By following this phased approach, we can implement the simplified structure in a controlled manner, ensuring that each step is properly tested before moving on to the next.

ollamatools-project/simplified_guides/README_REFACTORED.md +0 -234

…	…	@@ -1,234 +0,0 @@
1		# OllamaTools
2
3		OllamaTools is a toolkit for working with Ollama, providing a range of utilities and plugins for building AI-powered applications.
4
5		## Features
6
7		- **Core Agent**: Base agent implementation with plugin support
8		- **Plugin System**: Extensible plugin architecture for adding new functionality
9		- **Standard Plugins**: Search, commands, conversation history, feedback, and more
10		- **Tool Plugins**: System tools, web tools, documentation tools
11		- **Integration Plugins**: Matrix chat integration, OpenSearch integration
12		- **Agent Implementations**: Basic, modular, and extended agent implementations
13		- **Configuration System**: Flexible configuration management
14		- **Command-Line Interface**: Easy-to-use CLI for common tasks
15
16		## Installation
17
18		You can install OllamaTools using pip:
19
20		```bash
21		pip install ollamatools
22		```
23
24		For development:
25
26		```bash
27		pip install -e ".[dev]"
28		```
29
30		For Matrix integration:
31
32		```bash
33		pip install -e ".[matrix]"
34		```
35
36		For OpenSearch integration:
37
38		```bash
39		pip install -e ".[opensearch]"
40		```
41
42		## Quick Start
43
44		### Basic Agent
45
46		```python
47		from ollamatools.agents.basic import BasicAgent
48
49		agent = BasicAgent(model="llama2")
50		response = agent.chat("Hello, world!")
51		print(response)
52		```
53
54		### Modular Agent with Plugins
55
56		```python
57		from ollamatools.agents.modular import ModularAgent
58		from ollamatools.plugins.standard.search import SearchPlugin
59		from ollamatools.plugins.standard.commands import CommandsPlugin
60
61		agent = ModularAgent(model="llama2")
62		agent.add_plugin(SearchPlugin())
63		agent.add_plugin(CommandsPlugin())
64		response = agent.chat("Search for information about Python")
65		print(response)
66		```
67
68		### Extended Agent with Conversation History
69
70		```python
71		from ollamatools.agents.extended import ExtendedAgent
72
73		agent = ExtendedAgent(model="llama2")
74		agent.chat("Remember this information for later")
75		response = agent.chat("What did I ask you to remember?")
76		print(response)
77		```
78
79		## Using Plugins
80
81		### Search Plugin
82
83		```python
84		from ollamatools.plugins.standard.search import search_web
85
86		results = search_web("Python programming")
87		print(results)
88		```
89
90		### Commands Plugin
91
92		```python
93		from ollamatools.plugins.standard.commands import execute_command
94
95		result = execute_command("ls -la")
96		print(result)
97		```
98
99		### Conversation History Plugin
100
101		```python
102		from ollamatools.plugins.standard.conversation.history import ConversationHistory
103
104		history = ConversationHistory()
105		history.add_message("user", "Hello")
106		history.add_message("assistant", "Hi there!")
107		print(history.get_messages())
108		```
109
110		## Using Tools
111
112		### System Tools
113
114		```python
115		from ollamatools.plugins.tools.system import execute_system_command, read_file
116
117		result = execute_system_command("ls -la")
118		content = read_file("example.txt")
119		print(result)
120		print(content)
121		```
122
123		### Web Tools
124
125		```python
126		from ollamatools.plugins.tools.web import make_web_request
127
128		response = make_web_request("https://example.com")
129		print(response.text)
130		```
131
132		## Using Integrations
133
134		### Matrix Integration
135
136		```python
137		from ollamatools.plugins.integrations.matrix.client import MatrixClient
138
139		client = MatrixClient(homeserver="https://matrix.org", token="your_token")
140		client.send_message("!roomid:matrix.org", "Hello, Matrix!")
141		```
142
143		### OpenSearch Integration
144
145		```python
146		from ollamatools.plugins.integrations.opensearch.tools import OpenSearchWebTools
147
148		opensearch = OpenSearchWebTools(host="localhost", port=9200)
149		results = opensearch.search("example query")
150		print(results)
151		```
152
153		## Configuration
154
155		OllamaTools uses a flexible configuration system that supports YAML files, environment variables, and programmatic configuration.
156
157		```python
158		from ollamatools.config.loader import load_config
159
160		config = load_config("config.yaml")
161		print(config)
162		```
163
164		Example configuration file:
165
166		```yaml
167		# config.yaml
168		agent:
169		model: llama2
170		temperature: 0.7
171		max_tokens: 1024
172
173		plugins:
174		search:
175		enabled: true
176		engine: duckduckgo
177		commands:
178		enabled: true
179		allow_system_commands: false
180		```
181
182		## Command-Line Interface
183
184		OllamaTools includes a command-line interface for common tasks:
185
186		```bash
187		# Start a chat session
188		ollamatools chat --model llama2
189
190		# Run a script or command
191		ollamatools run "print('Hello, world!')" --model llama2
192
193		# Show version information
194		ollamatools version
195		```
196
197		## Development
198
199		### Setup
200
201		```bash
202		# Clone the repository
203		git clone https://github.com/yourusername/ollamatools.git
204		cd ollamatools
205
206		# Install development dependencies
207		pip install -e ".[dev]"
208
209		# Run tests
210		pytest
211		```
212
213		### Code Style
214
215		OllamaTools uses Black for code formatting, isort for import sorting, and flake8 for linting:
216
217		```bash
218		# Format code
219		black ollamatools tests
220
221		# Sort imports
222		isort ollamatools tests
223
224		# Lint code
225		flake8 ollamatools tests
226		```
227
228		## Contributing
229
230		Contributions are welcome! Please feel free to submit a Pull Request.
231
232		## License
233
234		This project is licensed under the MIT License - see the LICENSE file for details.

ollamatools-project/simplified_guides/copy_files.sh +0 -413

…	…	@@ -1,413 +0,0 @@
1		#!/bin/bash
2		# copy_files.sh
3		#
4		# This script copies files from the original OllamaTools structure to the new simplified structure
5		# according to the file mapping defined in 03_FILE_MAPPING.md.
6
7		# Set the base directory for the new structure
8		BASE_DIR="ollamatools-new"
9
10		# Check if the directory structure exists
11		if [ ! -d "$BASE_DIR" ]; then
12		echo "Error: Directory structure not found. Please run create_directory_structure.sh first."
13		exit 1
14		fi
15
16		echo "Copying files to new structure..."
17
18		# Core Module
19		echo "Copying Core Module files..."
20		cp ollamatools/agent.py $BASE_DIR/ollamatools/core/agent.py
21		cp ollamatools/errors.py $BASE_DIR/ollamatools/core/errors.py
22		cp ollamatools/config.py $BASE_DIR/ollamatools/core/config.py
23		cp ollamatools/security.py $BASE_DIR/ollamatools/core/security.py
24		cp ollamatools/logger.py $BASE_DIR/ollamatools/core/utils/logger.py
25		cp ollamatools/command_validator.py $BASE_DIR/ollamatools/core/utils/validators/command.py
26		cp ollamatools/path_validator.py $BASE_DIR/ollamatools/core/utils/validators/path.py
27		cp ollamatools/url_validator.py $BASE_DIR/ollamatools/core/utils/validators/url.py
28
29		# Create new formatting.py file with placeholder content
30		echo "Creating new formatting.py file..."
31		cat > $BASE_DIR/ollamatools/core/utils/formatting.py << 'EOF'
32		"""Text formatting utilities for OllamaTools."""
33
34		def format_markdown(text):
35		"""Format text as markdown."""
36		return text
37
38		def format_code(code, language="python"):
39		"""Format code with syntax highlighting."""
40		return f"```{language}\n{code}\n```"
41
42		def format_table(headers, rows):
43		"""Format data as a markdown table."""
44		if not headers or not rows:
45		return ""
46
47		result = []
48
49		# Add headers
50		result.append("| " + " | ".join(headers) + " |")
51
52		# Add separator
53		result.append("| " + " | ".join(["---"] * len(headers)) + " |")
54
55		# Add rows
56		for row in rows:
57		result.append("| " + " | ".join(str(cell) for cell in row) + " |")
58
59		return "\n".join(result)
60		EOF
61
62		# Plugin System
63		echo "Copying Plugin System files..."
64		cp ollamatools/plugins/registry.py $BASE_DIR/ollamatools/plugins/registry.py
65		cp ollamatools/plugins/errors.py $BASE_DIR/ollamatools/plugins/errors.py
66
67		# Create new base.py file with placeholder content
68		echo "Creating new base.py file..."
69		cat > $BASE_DIR/ollamatools/plugins/base.py << 'EOF'
70		"""Base plugin interfaces for the OllamaTools plugin system."""
71
72		from abc import ABC, abstractmethod
73		from typing import Any, Dict, List, Optional
74
75		class Plugin(ABC):
76		"""Base class for all plugins."""
77
78		@property
79		@abstractmethod
80		def name(self) -> str:
81		"""Return the name of the plugin."""
82		pass
83
84		@property
85		@abstractmethod
86		def version(self) -> str:
87		"""Return the version of the plugin."""
88		pass
89
90		@abstractmethod
91		def initialize(self, config: Dict[str, Any]) -> None:
92		"""Initialize the plugin with the given configuration."""
93		pass
94
95		@abstractmethod
96		def shutdown(self) -> None:
97		"""Shutdown the plugin and release any resources."""
98		pass
99
100		class ToolPlugin(Plugin):
101		"""Base class for tool plugins."""
102
103		@abstractmethod
104		def get_tools(self) -> List[Dict[str, Any]]:
105		"""Return the list of tools provided by this plugin."""
106		pass
107
108		class StandardPlugin(Plugin):
109		"""Base class for standard plugins."""
110
111		@abstractmethod
112		def process_message(self, message: Dict[str, Any]) -> Dict[str, Any]:
113		"""Process a message and return the result."""
114		pass
115
116		class IntegrationPlugin(Plugin):
117		"""Base class for integration plugins."""
118
119		@abstractmethod
120		def connect(self) -> bool:
121		"""Connect to the external service."""
122		pass
123
124		@abstractmethod
125		def disconnect(self) -> None:
126		"""Disconnect from the external service."""
127		pass
128		EOF
129
130		# Standard Plugins
131		echo "Copying Standard Plugins files..."
132		cp ollamatools/agent/modular/search.py $BASE_DIR/ollamatools/plugins/standard/search.py
133		cp ollamatools/agent/modular/commands.py $BASE_DIR/ollamatools/plugins/standard/commands.py
134		cp ollamatools/plugins/conversation_history.py $BASE_DIR/ollamatools/plugins/standard/conversation/history.py
135		cp ollamatools/plugins/conversation_context.py $BASE_DIR/ollamatools/plugins/standard/conversation/context.py
136		cp ollamatools/agent/modular/feedback.py $BASE_DIR/ollamatools/plugins/standard/feedback.py
137		cp ollamatools/agent/modular/issues.py $BASE_DIR/ollamatools/plugins/standard/issues.py
138
139		# Create new conversation analysis file with placeholder content
140		echo "Creating new conversation analysis file..."
141		cat > $BASE_DIR/ollamatools/plugins/standard/conversation/analysis.py << 'EOF'
142		"""Conversation topic analysis and metrics."""
143
144		from typing import Any, Dict, List, Optional
145
146		def analyze_topics(messages: List[Dict[str, Any]]) -> List[str]:
147		"""Analyze the topics in a conversation."""
148		# Implementation
149		return ["topic1", "topic2"]
150
151		def calculate_metrics(messages: List[Dict[str, Any]]) -> Dict[str, Any]:
152		"""Calculate metrics for a conversation."""
153		# Implementation
154		return {
155		"message_count": len(messages),
156		"average_length": sum(len(m.get("content", "")) for m in messages) / max(1, len(messages)),
157		"user_messages": sum(1 for m in messages if m.get("role") == "user"),
158		"assistant_messages": sum(1 for m in messages if m.get("role") == "assistant"),
159		}
160		EOF
161
162		# Create new function calling file with placeholder content
163		echo "Creating new function calling file..."
164		cat > $BASE_DIR/ollamatools/plugins/standard/function_calling.py << 'EOF'
165		"""Plugin for semantic function calling."""
166
167		from typing import Any, Dict, List, Optional
168
169		from ollamatools.plugins.base import StandardPlugin
170
171		class FunctionCallingPlugin(StandardPlugin):
172		"""Plugin for semantic function calling."""
173
174		@property
175		def name(self) -> str:
176		return "function_calling"
177
178		@property
179		def version(self) -> str:
180		return "0.1.0"
181
182		def initialize(self, config: Dict[str, Any]) -> None:
183		"""Initialize the plugin with the given configuration."""
184		self.config = config
185		self.functions = {}
186
187		def shutdown(self) -> None:
188		"""Shutdown the plugin and release any resources."""
189		pass
190
191		def register_function(self, name: str, function: callable, description: str, parameters: Dict[str, Any]) -> None:
192		"""Register a function that can be called semantically."""
193		self.functions[name] = {
194		"function": function,
195		"description": description,
196		"parameters": parameters,
197		}
198
199		def process_message(self, message: Dict[str, Any]) -> Dict[str, Any]:
200		"""Process a message and extract function calls."""
201		# Implementation
202		return message
203		EOF
204
205		# Tool Plugins
206		echo "Copying Tool Plugins files..."
207		cp ollamatools/tools/system_tools.py $BASE_DIR/ollamatools/plugins/tools/system.py
208		cp ollamatools/tools/web_tools.py $BASE_DIR/ollamatools/plugins/tools/web.py
209		cp ollamatools/tools/doc_tools.py $BASE_DIR/ollamatools/plugins/tools/doc.py
210
211		# Integration Plugins
212		echo "Copying Integration Plugins files..."
213		cp ollamatools/tools/matrix_tools.py $BASE_DIR/ollamatools/plugins/integrations/matrix/client.py
214		cp ollamatools/tools/enhanced_matrix_tools.py $BASE_DIR/ollamatools/plugins/integrations/matrix/commands.py
215		cp ollamatools/tools/enhanced_matrix_search.py $BASE_DIR/ollamatools/plugins/integrations/matrix/search.py
216		cp ollamatools/tools/enhanced_matrix_history.py $BASE_DIR/ollamatools/plugins/integrations/matrix/history.py
217		cp ollamatools/tools/matrix_rate_limit_patch.py $BASE_DIR/ollamatools/plugins/integrations/matrix/rate_limit.py
218		cp ollamatools/opensearch_web_tools.py $BASE_DIR/ollamatools/plugins/integrations/opensearch/tools.py
219
220		# Create new Matrix config file with placeholder content
221		echo "Creating new Matrix config file..."
222		cat > $BASE_DIR/ollamatools/plugins/integrations/matrix/config.py << 'EOF'
223		"""Configuration for Matrix integration."""
224
225		from typing import Any, Dict, Optional
226
227		def load_matrix_config(config_path: Optional[str] = None) -> Dict[str, Any]:
228		"""Load Matrix configuration from the given path."""
229		# Implementation
230		return {}
231
232		def validate_matrix_config(config: Dict[str, Any]) -> bool:
233		"""Validate Matrix configuration."""
234		# Implementation
235		return True
236
237		def get_default_matrix_config() -> Dict[str, Any]:
238		"""Return the default Matrix configuration."""
239		return {
240		"homeserver": "https://matrix.org",
241		"user_id": "@ollamatools:matrix.org",
242		"access_token": "",
243		"device_id": "ollamatools",
244		"room_ids": [],
245		"rate_limit": {
246		"max_requests": 5,
247		"time_window": 1,
248		"backoff_factor": 2,
249		"max_retries": 5,
250		},
251		}
252		EOF
253
254		# Create new OpenSearch client file with placeholder content
255		echo "Creating new OpenSearch client file..."
256		cat > $BASE_DIR/ollamatools/plugins/integrations/opensearch/client.py << 'EOF'
257		"""OpenSearch client for OllamaTools."""
258
259		from typing import Any, Dict, List, Optional
260
261		class OpenSearchClient:
262		"""Client for interacting with OpenSearch."""
263
264		def __init__(self, host: str, port: int, username: Optional[str] = None, password: Optional[str] = None):
265		"""Initialize the OpenSearch client."""
266		self.host = host
267		self.port = port
268		self.username = username
269		self.password = password
270		self.connected = False
271
272		def connect(self) -> bool:
273		"""Connect to the OpenSearch server."""
274		# Implementation
275		self.connected = True
276		return True
277
278		def disconnect(self) -> None:
279		"""Disconnect from the OpenSearch server."""
280		# Implementation
281		self.connected = False
282
283		def search(self, index: str, query: Dict[str, Any]) -> List[Dict[str, Any]]:
284		"""Search for documents in the given index."""
285		# Implementation
286		return []
287
288		def index_document(self, index: str, document: Dict[str, Any], id: Optional[str] = None) -> bool:
289		"""Index a document in the given index."""
290		# Implementation
291		return True
292
293		def delete_document(self, index: str, id: str) -> bool:
294		"""Delete a document from the given index."""
295		# Implementation
296		return True
297		EOF
298
299		# Agent Implementations
300		echo "Copying Agent Implementations files..."
301		cp ollamatools/agent/core.py $BASE_DIR/ollamatools/agents/basic.py
302		cp ollamatools/agent/modular/agent.py $BASE_DIR/ollamatools/agents/modular.py
303		cp ollamatools/agent/extended/agent.py $BASE_DIR/ollamatools/agents/extended.py
304
305		# Configuration System
306		echo "Copying Configuration System files..."
307		cp ollamatools/config/schemas.py $BASE_DIR/ollamatools/config/schemas.py
308		cp ollamatools/config/core_config.py $BASE_DIR/ollamatools/config/loader.py
309		cp ollamatools/config/plugin_config.py $BASE_DIR/ollamatools/config/defaults.py
310
311		# Create new CLI main file with placeholder content
312		echo "Creating new CLI main file..."
313		cat > $BASE_DIR/ollamatools/cli/main.py << 'EOF'
314		"""Command-line interface for OllamaTools."""
315
316		import argparse
317		import sys
318		from typing import List, Optional
319
320		def parse_args(args: Optional[List[str]] = None) -> argparse.Namespace:
321		"""Parse command-line arguments."""
322		parser = argparse.ArgumentParser(description="OllamaTools - A toolkit for working with Ollama")
323
324		# Add subparsers for different commands
325		subparsers = parser.add_subparsers(dest="command", help="Command to execute")
326
327		# Chat command
328		chat_parser = subparsers.add_parser("chat", help="Start a chat session")
329		chat_parser.add_argument("--model", "-m", default="llama2", help="Model to use for chat")
330		chat_parser.add_argument("--config", "-c", help="Path to configuration file")
331
332		# Run command
333		run_parser = subparsers.add_parser("run", help="Run a script or command")
334		run_parser.add_argument("script", help="Script or command to run")
335		run_parser.add_argument("--model", "-m", default="llama2", help="Model to use for execution")
336
337		# Version command
338		subparsers.add_parser("version", help="Show version information")
339
340		return parser.parse_args(args)
341
342		def main(args: Optional[List[str]] = None) -> int:
343		"""Main entry point for the CLI."""
344		parsed_args = parse_args(args)
345
346		if parsed_args.command == "chat":
347		# Start chat session
348		print(f"Starting chat session with model {parsed_args.model}")
349		# Implementation
350		return 0
351		elif parsed_args.command == "run":
352		# Run script or command
353		print(f"Running {parsed_args.script} with model {parsed_args.model}")
354		# Implementation
355		return 0
356		elif parsed_args.command == "version":
357		# Show version information
358		from ollamatools.version import __version__
359		print(f"OllamaTools version {__version__}")
360		return 0
361		else:
362		# No command specified, show help
363		parse_args(["--help"])
364		return 1
365
366		if __name__ == "__main__":
367		sys.exit(main())
368		EOF
369
370		# Create new version file with placeholder content
371		echo "Creating new version file..."
372		cat > $BASE_DIR/ollamatools/version.py << 'EOF'
373		"""Version information for OllamaTools."""
374
375		__version__ = "0.1.0"
376		EOF
377
378		# Update package initialization
379		echo "Updating package initialization..."
380		cat > $BASE_DIR/ollamatools/__init__.py << 'EOF'
381		"""OllamaTools package."""
382
383		from ollamatools.version import __version__
384
385		# Expose core components
386		from ollamatools.core.agent import Agent
387		from ollamatools.core.config import Config
388
389		# Expose agent implementations
390		from ollamatools.agents.basic import BasicAgent
391		from ollamatools.agents.modular import ModularAgent
392		from ollamatools.agents.extended import ExtendedAgent
393
394		# Expose plugin system
395		from ollamatools.plugins.base import Plugin
396		from ollamatools.plugins.registry import PluginRegistry
397
398		__all__ = [
399		"__version__",
400		"Agent",
401		"Config",
402		"BasicAgent",
403		"ModularAgent",
404		"ExtendedAgent",
405		"Plugin",
406		"PluginRegistry",
407		]
408		EOF
409
410		echo "Files copied successfully."
411		echo "Next steps:"
412		echo "1. Update imports in the copied files"
413		echo "2. Test the refactored code"

ollamatools-project/simplified_guides/copy_readme.sh +0 -29

…	…	@@ -1,29 +0,0 @@
1		#!/bin/bash
2		# copy_readme.sh
3		#
4		# This script copies the README_REFACTORED.md file to the refactored project directory
5		# as README.md.
6
7		# Set the base directory for the new structure
8		BASE_DIR="ollamatools-new"
9
10		# Check if the directory structure exists
11		if [ ! -d "$BASE_DIR" ]; then
12		echo "Error: Directory structure not found. Please run create_directory_structure.sh first."
13		exit 1
14		fi
15
16		# Copy the README_REFACTORED.md file to the refactored project directory
17		echo "Copying README_REFACTORED.md to $BASE_DIR/README.md..."
18		cp simplified_guides/README_REFACTORED.md $BASE_DIR/README.md
19
20		# Copy the setup.py file to the refactored project directory
21		echo "Copying setup.py to $BASE_DIR/setup.py..."
22		cp simplified_guides/setup.py $BASE_DIR/setup.py
23
24		# Copy the MIGRATION_GUIDE.md file to the refactored project directory
25		echo "Copying MIGRATION_GUIDE.md to $BASE_DIR/MIGRATION_GUIDE.md..."
26		cp simplified_guides/MIGRATION_GUIDE.md $BASE_DIR/MIGRATION_GUIDE.md
27
28		echo "Files copied successfully."
29		echo "The refactored project now has a README.md, setup.py, and MIGRATION_GUIDE.md file."

ollamatools-project/simplified_guides/create_directory_structure.sh +0 -47

…	…	@@ -1,47 +0,0 @@
1		#!/bin/bash
2		# create_directory_structure.sh
3		#
4		# This script creates the directory structure for the simplified OllamaTools refactoring.
5		# It creates all necessary directories and initializes them with empty __init__.py files.
6
7		# Set the base directory for the new structure
8		BASE_DIR="ollamatools-new"
9
10		# Create the base directory if it doesn't exist
11		mkdir -p $BASE_DIR
12
13		# Create the directory structure
14		echo "Creating directory structure..."
15
16		# Core module
17		mkdir -p $BASE_DIR/ollamatools/core/utils/validators
18		mkdir -p $BASE_DIR/ollamatools/core/utils
19
20		# Plugin system
21		mkdir -p $BASE_DIR/ollamatools/plugins/standard/conversation
22		mkdir -p $BASE_DIR/ollamatools/plugins/tools
23		mkdir -p $BASE_DIR/ollamatools/plugins/integrations/matrix
24		mkdir -p $BASE_DIR/ollamatools/plugins/integrations/opensearch
25
26		# Agent implementations
27		mkdir -p $BASE_DIR/ollamatools/agents
28
29		# Configuration system
30		mkdir -p $BASE_DIR/ollamatools/config
31
32		# Command-line interface
33		mkdir -p $BASE_DIR/ollamatools/cli
34
35		# Create __init__.py files in all directories
36		echo "Creating __init__.py files..."
37		find $BASE_DIR -type d -exec touch {}/__init__.py \;
38
39		echo "Directory structure created successfully."
40		echo "The following directories have been created:"
41		find $BASE_DIR -type d | sort
42
43		echo "Next steps:"
44		echo "1. Copy files to their new locations using the file mapping in 03_FILE_MAPPING.md"
45		echo "2. Update imports in the copied files"
46		echo "3. Implement new files"
47		echo "4. Test the refactored code"

ollamatools-project/simplified_guides/make_scripts_executable.sh +0 -19

…	…	@@ -1,19 +0,0 @@
1		#!/bin/bash
2		# make_scripts_executable.sh
3		#
4		# This script makes all shell scripts in the simplified_guides directory executable.
5
6		# Make all shell scripts executable
7		echo "Making shell scripts executable..."
8		chmod +x simplified_guides/*.sh
9
10		echo "Shell scripts are now executable."
11		echo "You can now run the following scripts:"
12		echo " - simplified_guides/create_directory_structure.sh"
13		echo " - simplified_guides/copy_files.sh"
14		echo " - simplified_guides/make_scripts_executable.sh"
15
16		echo "To execute the scripts, run:"
17		echo " ./simplified_guides/create_directory_structure.sh"
18		echo " ./simplified_guides/copy_files.sh"
19		echo " python simplified_guides/test_refactored_code.py"

ollamatools-project/simplified_guides/run_all_steps.sh +0 -50

…	…	@@ -1,50 +0,0 @@
1		#!/bin/bash
2		# run_all_steps.sh
3		#
4		# This script runs all the steps in the simplified refactoring process in sequence.
5		# It creates the directory structure, copies files, updates imports, and tests the refactored code.
6
7		# Set the base directory for the new structure
8		BASE_DIR="ollamatools-new"
9
10		# Function to check if a command succeeded
11		check_status() {
12		if [ $? -ne 0 ]; then
13		echo "Error: $1 failed. Exiting."
14		exit 1
15		fi
16		}
17
18		# Make all scripts executable
19		echo "Making scripts executable..."
20		chmod +x simplified_guides/*.sh
21		check_status "Making scripts executable"
22
23		# Step 1: Create the directory structure
24		echo -e "\n=== Step 1: Creating Directory Structure ===\n"
25		./simplified_guides/create_directory_structure.sh
26		check_status "Creating directory structure"
27
28		# Step 2: Copy files to new structure
29		echo -e "\n=== Step 2: Copying Files ===\n"
30		./simplified_guides/copy_files.sh
31		check_status "Copying files"
32
33		# Step 3: Update imports
34		echo -e "\n=== Step 3: Updating Imports ===\n"
35		python simplified_guides/update_imports.py
36		check_status "Updating imports"
37
38		# Step 4: Test the refactored code
39		echo -e "\n=== Step 4: Testing Refactored Code ===\n"
40		python simplified_guides/test_refactored_code.py
41		check_status "Testing refactored code"
42
43		# Step 5: Copy README and other files
44		echo -e "\n=== Step 5: Copying README and Other Files ===\n"
45		./simplified_guides/copy_readme.sh
46		check_status "Copying README and other files"
47
48		echo -e "\n=== All Steps Completed Successfully ===\n"
49		echo "The refactored code is now available in the '$BASE_DIR' directory."
50		echo "You can now proceed with further testing and integration."

ollamatools-project/simplified_guides/setup.py +0 -69

…	…	@@ -1,69 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Setup script for the refactored OllamaTools package.
4
5		This script installs the refactored OllamaTools package and its dependencies.
6		It should be placed in the root directory of the refactored project.
7		"""
8
9		from setuptools import setup, find_packages
10
11		with open("README.md", "r", encoding="utf-8") as fh:
12		long_description = fh.read()
13
14		setup(
15		name="ollamatools",
16		version="0.1.0",
17		author="OllamaTools Team",
18		author_email="info@example.com",
19		description="A toolkit for working with Ollama",
20		long_description=long_description,
21		long_description_content_type="text/markdown",
22		url="https://github.com/yourusername/ollamatools",
23		packages=find_packages(),
24		classifiers=[
25		"Programming Language :: Python :: 3",
26		"License :: OSI Approved :: MIT License",
27		"Operating System :: OS Independent",
28		],
29		python_requires=">=3.8",
30		install_requires=[
31		"requests>=2.25.0",
32		"pyyaml>=5.4.0",
33		"typing-extensions>=4.0.0",
34		"aiohttp>=3.7.4",
35		"websockets>=10.0",
36		"pydantic>=1.8.0",
37		"numpy>=1.20.0",
38		"pandas>=1.3.0",
39		"matplotlib>=3.4.0",
40		"pytest>=6.2.0",
41		"pytest-cov>=2.12.0",
42		"black>=21.5b2",
43		"isort>=5.9.0",
44		"flake8>=3.9.0",
45		],
46		extras_require={
47		"dev": [
48		"pytest>=6.2.0",
49		"pytest-cov>=2.12.0",
50		"black>=21.5b2",
51		"isort>=5.9.0",
52		"flake8>=3.9.0",
53		"mypy>=0.812",
54		"sphinx>=4.0.0",
55		"sphinx-rtd-theme>=0.5.2",
56		],
57		"matrix": [
58		"matrix-nio>=0.19.0",
59		],
60		"opensearch": [
61		"opensearch-py>=1.0.0",
62		],
63		},
64		entry_points={
65		"console_scripts": [
66		"ollamatools=ollamatools.cli.main:main",
67		],
68		},
69		)

ollamatools-project/simplified_guides/test_refactored_code.py +0 -200

…	…	@@ -1,200 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Test script for the refactored OllamaTools code.
4
5		This script tests the basic functionality of the refactored OllamaTools code
6		by importing key components and verifying that they can be instantiated.
7
8		Usage:
9		python test_refactored_code.py
10
11		Note:
12		This script assumes that the refactored code is in the 'ollamatools-new' directory.
13		Make sure to run create_directory_structure.sh and copy_files.sh before running this script.
14		"""
15
16		import os
17		import sys
18		import importlib
19		import traceback
20
21		# Add the refactored code to the Python path
22		REFACTORED_DIR = "ollamatools-new"
23		sys.path.insert(0, REFACTORED_DIR)
24
25		def test_import(module_name, class_name=None):
26		"""Test importing a module and optionally a class from that module."""
27		try:
28		module = importlib.import_module(module_name)
29		print(f"✅ Successfully imported {module_name}")
30
31		if class_name:
32		cls = getattr(module, class_name)
33		print(f"✅ Successfully imported {class_name} from {module_name}")
34		return cls
35		return module
36		except Exception as e:
37		print(f"❌ Failed to import {module_name}{' - ' + class_name if class_name else ''}")
38		print(f" Error: {str(e)}")
39		traceback.print_exc()
40		return None
41
42		def test_core_components():
43		"""Test core components."""
44		print("\n=== Testing Core Components ===")
45
46		# Test core agent
47		Agent = test_import("ollamatools.core.agent", "Agent")
48
49		# Test core config
50		Config = test_import("ollamatools.core.config", "Config")
51
52		# Test core errors
53		test_import("ollamatools.core.errors")
54
55		# Test core security
56		test_import("ollamatools.core.security")
57
58		# Test core utils
59		test_import("ollamatools.core.utils.logger")
60		test_import("ollamatools.core.utils.validators.command")
61		test_import("ollamatools.core.utils.validators.path")
62		test_import("ollamatools.core.utils.validators.url")
63		test_import("ollamatools.core.utils.formatting")
64
65		def test_plugin_system():
66		"""Test plugin system."""
67		print("\n=== Testing Plugin System ===")
68
69		# Test plugin base
70		Plugin = test_import("ollamatools.plugins.base", "Plugin")
71
72		# Test plugin registry
73		PluginRegistry = test_import("ollamatools.plugins.registry", "PluginRegistry")
74
75		# Test plugin errors
76		test_import("ollamatools.plugins.errors")
77
78		def test_standard_plugins():
79		"""Test standard plugins."""
80		print("\n=== Testing Standard Plugins ===")
81
82		# Test search plugin
83		test_import("ollamatools.plugins.standard.search")
84
85		# Test commands plugin
86		test_import("ollamatools.plugins.standard.commands")
87
88		# Test conversation plugins
89		test_import("ollamatools.plugins.standard.conversation.history")
90		test_import("ollamatools.plugins.standard.conversation.context")
91		test_import("ollamatools.plugins.standard.conversation.analysis")
92
93		# Test feedback plugin
94		test_import("ollamatools.plugins.standard.feedback")
95
96		# Test issues plugin
97		test_import("ollamatools.plugins.standard.issues")
98
99		# Test function calling plugin
100		test_import("ollamatools.plugins.standard.function_calling")
101
102		def test_tool_plugins():
103		"""Test tool plugins."""
104		print("\n=== Testing Tool Plugins ===")
105
106		# Test system tools
107		test_import("ollamatools.plugins.tools.system")
108
109		# Test web tools
110		test_import("ollamatools.plugins.tools.web")
111
112		# Test doc tools
113		test_import("ollamatools.plugins.tools.doc")
114
115		def test_integration_plugins():
116		"""Test integration plugins."""
117		print("\n=== Testing Integration Plugins ===")
118
119		# Test Matrix integration
120		test_import("ollamatools.plugins.integrations.matrix.client")
121		test_import("ollamatools.plugins.integrations.matrix.commands")
122		test_import("ollamatools.plugins.integrations.matrix.search")
123		test_import("ollamatools.plugins.integrations.matrix.history")
124		test_import("ollamatools.plugins.integrations.matrix.rate_limit")
125		test_import("ollamatools.plugins.integrations.matrix.config")
126
127		# Test OpenSearch integration
128		test_import("ollamatools.plugins.integrations.opensearch.tools")
129		test_import("ollamatools.plugins.integrations.opensearch.client")
130
131		def test_agent_implementations():
132		"""Test agent implementations."""
133		print("\n=== Testing Agent Implementations ===")
134
135		# Test basic agent
136		test_import("ollamatools.agents.basic", "BasicAgent")
137
138		# Test modular agent
139		test_import("ollamatools.agents.modular", "ModularAgent")
140
141		# Test extended agent
142		test_import("ollamatools.agents.extended", "ExtendedAgent")
143
144		def test_configuration_system():
145		"""Test configuration system."""
146		print("\n=== Testing Configuration System ===")
147
148		# Test schemas
149		test_import("ollamatools.config.schemas")
150
151		# Test loader
152		test_import("ollamatools.config.loader")
153
154		# Test defaults
155		test_import("ollamatools.config.defaults")
156
157		def test_cli():
158		"""Test command-line interface."""
159		print("\n=== Testing Command-Line Interface ===")
160
161		# Test CLI main
162		test_import("ollamatools.cli.main")
163
164		def test_package_root():
165		"""Test package root."""
166		print("\n=== Testing Package Root ===")
167
168		# Test package initialization
169		test_import("ollamatools")
170
171		# Test version
172		test_import("ollamatools.version")
173
174		def main():
175		"""Main entry point."""
176		if not os.path.exists(REFACTORED_DIR):
177		print(f"Error: Directory '{REFACTORED_DIR}' not found.")
178		print("Please run create_directory_structure.sh and copy_files.sh first.")
179		return 1
180
181		print(f"Testing refactored code in '{REFACTORED_DIR}'...")
182
183		# Test all components
184		test_core_components()
185		test_plugin_system()
186		test_standard_plugins()
187		test_tool_plugins()
188		test_integration_plugins()
189		test_agent_implementations()
190		test_configuration_system()
191		test_cli()
192		test_package_root()
193
194		print("\n=== Test Summary ===")
195		print("All tests completed. Check the output above for any failures.")
196
197		return 0
198
199		if __name__ == "__main__":
200		sys.exit(main())

ollamatools-project/simplified_guides/update_imports.py +0 -182

…	…	@@ -1,182 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Script to update imports in the refactored OllamaTools code.
4
5		This script updates import statements in the refactored OllamaTools code to reflect
6		the new directory structure. It replaces old import paths with new ones based on
7		the file mapping defined in the simplified refactoring guide.
8
9		Usage:
10		python update_imports.py
11
12		Note:
13		This script assumes that the refactored code is in the 'ollamatools-new' directory.
14		Make sure to run create_directory_structure.sh and copy_files.sh before running this script.
15		"""
16
17		import os
18		import re
19		import sys
20
21		# Set the base directory for the new structure
22		BASE_DIR = "ollamatools-new"
23
24		# Define import mappings (old import -> new import)
25		IMPORT_MAPPINGS = [
26		# Core module
27		(r'from ollamatools\.agent import', 'from ollamatools.core.agent import'),
28		(r'from ollamatools import agent', 'from ollamatools.core import agent'),
29		(r'import ollamatools\.agent', 'import ollamatools.core.agent'),
30
31		(r'from ollamatools\.errors import', 'from ollamatools.core.errors import'),
32		(r'from ollamatools import errors', 'from ollamatools.core import errors'),
33		(r'import ollamatools\.errors', 'import ollamatools.core.errors'),
34
35		(r'from ollamatools\.config import', 'from ollamatools.core.config import'),
36		(r'from ollamatools import config', 'from ollamatools.core import config'),
37		(r'import ollamatools\.config', 'import ollamatools.core.config'),
38
39		(r'from ollamatools\.security import', 'from ollamatools.core.security import'),
40		(r'from ollamatools import security', 'from ollamatools.core import security'),
41		(r'import ollamatools\.security', 'import ollamatools.core.security'),
42
43		(r'from ollamatools\.logger import', 'from ollamatools.core.utils.logger import'),
44		(r'from ollamatools import logger', 'from ollamatools.core.utils import logger'),
45		(r'import ollamatools\.logger', 'import ollamatools.core.utils.logger'),
46
47		(r'from ollamatools\.command_validator import', 'from ollamatools.core.utils.validators.command import'),
48		(r'from ollamatools import command_validator', 'from ollamatools.core.utils.validators import command'),
49		(r'import ollamatools\.command_validator', 'import ollamatools.core.utils.validators.command'),
50
51		(r'from ollamatools\.path_validator import', 'from ollamatools.core.utils.validators.path import'),
52		(r'from ollamatools import path_validator', 'from ollamatools.core.utils.validators import path'),
53		(r'import ollamatools\.path_validator', 'import ollamatools.core.utils.validators.path'),
54
55		(r'from ollamatools\.url_validator import', 'from ollamatools.core.utils.validators.url import'),
56		(r'from ollamatools import url_validator', 'from ollamatools.core.utils.validators import url'),
57		(r'import ollamatools\.url_validator', 'import ollamatools.core.utils.validators.url'),
58
59		# Agent implementations
60		(r'from ollamatools\.agent\.core import', 'from ollamatools.agents.basic import'),
61		(r'from ollamatools\.agent import core', 'from ollamatools.agents import basic'),
62		(r'import ollamatools\.agent\.core', 'import ollamatools.agents.basic'),
63
64		(r'from ollamatools\.agent\.modular\.agent import', 'from ollamatools.agents.modular import'),
65		(r'from ollamatools\.agent\.modular import agent', 'from ollamatools.agents import modular'),
66		(r'import ollamatools\.agent\.modular\.agent', 'import ollamatools.agents.modular'),
67
68		(r'from ollamatools\.agent\.extended\.agent import', 'from ollamatools.agents.extended import'),
69		(r'from ollamatools\.agent\.extended import agent', 'from ollamatools.agents import extended'),
70		(r'import ollamatools\.agent\.extended\.agent', 'import ollamatools.agents.extended'),
71
72		# Standard plugins
73		(r'from ollamatools\.agent\.modular\.search import', 'from ollamatools.plugins.standard.search import'),
74		(r'from ollamatools\.agent\.modular import search', 'from ollamatools.plugins.standard import search'),
75		(r'import ollamatools\.agent\.modular\.search', 'import ollamatools.plugins.standard.search'),
76
77		(r'from ollamatools\.agent\.modular\.commands import', 'from ollamatools.plugins.standard.commands import'),
78		(r'from ollamatools\.agent\.modular import commands', 'from ollamatools.plugins.standard import commands'),
79		(r'import ollamatools\.agent\.modular\.commands', 'import ollamatools.plugins.standard.commands'),
80
81		(r'from ollamatools\.plugins\.conversation_history import', 'from ollamatools.plugins.standard.conversation.history import'),
82		(r'from ollamatools\.plugins import conversation_history', 'from ollamatools.plugins.standard.conversation import history'),
83		(r'import ollamatools\.plugins\.conversation_history', 'import ollamatools.plugins.standard.conversation.history'),
84
85		(r'from ollamatools\.plugins\.conversation_context import', 'from ollamatools.plugins.standard.conversation.context import'),
86		(r'from ollamatools\.plugins import conversation_context', 'from ollamatools.plugins.standard.conversation import context'),
87		(r'import ollamatools\.plugins\.conversation_context', 'import ollamatools.plugins.standard.conversation.context'),
88
89		(r'from ollamatools\.agent\.modular\.feedback import', 'from ollamatools.plugins.standard.feedback import'),
90		(r'from ollamatools\.agent\.modular import feedback', 'from ollamatools.plugins.standard import feedback'),
91		(r'import ollamatools\.agent\.modular\.feedback', 'import ollamatools.plugins.standard.feedback'),
92
93		(r'from ollamatools\.agent\.modular\.issues import', 'from ollamatools.plugins.standard.issues import'),
94		(r'from ollamatools\.agent\.modular import issues', 'from ollamatools.plugins.standard import issues'),
95		(r'import ollamatools\.agent\.modular\.issues', 'import ollamatools.plugins.standard.issues'),
96
97		# Tool plugins
98		(r'from ollamatools\.tools\.system_tools import', 'from ollamatools.plugins.tools.system import'),
99		(r'from ollamatools\.tools import system_tools', 'from ollamatools.plugins.tools import system'),
100		(r'import ollamatools\.tools\.system_tools', 'import ollamatools.plugins.tools.system'),
101
102		(r'from ollamatools\.tools\.web_tools import', 'from ollamatools.plugins.tools.web import'),
103		(r'from ollamatools\.tools import web_tools', 'from ollamatools.plugins.tools import web'),
104		(r'import ollamatools\.tools\.web_tools', 'import ollamatools.plugins.tools.web'),
105
106		(r'from ollamatools\.tools\.doc_tools import', 'from ollamatools.plugins.tools.doc import'),
107		(r'from ollamatools\.tools import doc_tools', 'from ollamatools.plugins.tools import doc'),
108		(r'import ollamatools\.tools\.doc_tools', 'import ollamatools.plugins.tools.doc'),
109
110		# Matrix integration
111		(r'from ollamatools\.tools\.matrix_tools import', 'from ollamatools.plugins.integrations.matrix.client import'),
112		(r'from ollamatools\.tools import matrix_tools', 'from ollamatools.plugins.integrations.matrix import client'),
113		(r'import ollamatools\.tools\.matrix_tools', 'import ollamatools.plugins.integrations.matrix.client'),
114
115		(r'from ollamatools\.tools\.enhanced_matrix_tools import', 'from ollamatools.plugins.integrations.matrix.commands import'),
116		(r'from ollamatools\.tools import enhanced_matrix_tools', 'from ollamatools.plugins.integrations.matrix import commands'),
117		(r'import ollamatools\.tools\.enhanced_matrix_tools', 'import ollamatools.plugins.integrations.matrix.commands'),
118
119		(r'from ollamatools\.tools\.enhanced_matrix_search import', 'from ollamatools.plugins.integrations.matrix.search import'),
120		(r'from ollamatools\.tools import enhanced_matrix_search', 'from ollamatools.plugins.integrations.matrix import search'),
121		(r'import ollamatools\.tools\.enhanced_matrix_search', 'import ollamatools.plugins.integrations.matrix.search'),
122
123		(r'from ollamatools\.tools\.enhanced_matrix_history import', 'from ollamatools.plugins.integrations.matrix.history import'),
124		(r'from ollamatools\.tools import enhanced_matrix_history', 'from ollamatools.plugins.integrations.matrix import history'),
125		(r'import ollamatools\.tools\.enhanced_matrix_history', 'import ollamatools.plugins.integrations.matrix.history'),
126
127		(r'from ollamatools\.tools\.matrix_rate_limit_patch import', 'from ollamatools.plugins.integrations.matrix.rate_limit import'),
128		(r'from ollamatools\.tools import matrix_rate_limit_patch', 'from ollamatools.plugins.integrations.matrix import rate_limit'),
129		(r'import ollamatools\.tools\.matrix_rate_limit_patch', 'import ollamatools.plugins.integrations.matrix.rate_limit'),
130
131		# OpenSearch integration
132		(r'from ollamatools\.opensearch_web_tools import', 'from ollamatools.plugins.integrations.opensearch.tools import'),
133		(r'from ollamatools import opensearch_web_tools', 'from ollamatools.plugins.integrations.opensearch import tools'),
134		(r'import ollamatools\.opensearch_web_tools', 'import ollamatools.plugins.integrations.opensearch.tools'),
135
136		# Configuration system
137		(r'from ollamatools\.config\.core_config import', 'from ollamatools.config.loader import'),
138		(r'from ollamatools\.config import core_config', 'from ollamatools.config import loader'),
139		(r'import ollamatools\.config\.core_config', 'import ollamatools.config.loader'),
140
141		(r'from ollamatools\.config\.plugin_config import', 'from ollamatools.config.defaults import'),
142		(r'from ollamatools\.config import plugin_config', 'from ollamatools.config import defaults'),
143		(r'import ollamatools\.config\.plugin_config', 'import ollamatools.config.defaults'),
144		]
145
146		def update_imports(file_path):
147		"""Update import statements in a file."""
148		with open(file_path, 'r') as f:
149		content = f.read()
150
151		# Apply import mappings
152		for old_import, new_import in IMPORT_MAPPINGS:
153		content = re.sub(old_import, new_import, content)
154
155		with open(file_path, 'w') as f:
156		f.write(content)
157
158		def main():
159		"""Main entry point."""
160		if not os.path.exists(BASE_DIR):
161		print(f"Error: Directory '{BASE_DIR}' not found.")
162		print("Please run create_directory_structure.sh and copy_files.sh first.")
163		return 1
164
165		print(f"Updating imports in '{BASE_DIR}'...")
166
167		# Process all Python files in the new structure
168		for root, dirs, files in os.walk(BASE_DIR):
169		for file in files:
170		if file.endswith('.py'):
171		file_path = os.path.join(root, file)
172		print(f"Updating imports in {file_path}")
173		update_imports(file_path)
174
175		print("Imports updated successfully.")
176		print("Next steps:")
177		print("1. Test the refactored code with: python simplified_guides/test_refactored_code.py")
178
179		return 0
180
181		if __name__ == "__main__":
182		sys.exit(main())

ollamatools-project/test_dict_handling.py +0 -275

…	…	@@ -1,275 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Simplified test script for dict handling in search results.
4
5		This script tests the handling of dictionary values in search results,
6		particularly for the error "sequence item 0: expected str instance, dict found".
7
8		Usage:
9		python test_dict_handling.py
10		"""
11
12		import json
13
14		def process_search_results(result_data):
15		"""
16		Process and format search results.
17
18		Args:
19		result_data: Raw result data
20
21		Returns:
22		dict: Processed search results with metadata
23		"""
24		processed_results = {
25		"query": "test query",
26		"results": [],
27		"suggestion": None,
28		"answer": None,
29		"infobox": None,
30		"engines": [],
31		"timing": {}
32		}
33
34		# Extract search results
35		if "results" in result_data:
36		for result in result_data["results"]:
37		processed_result = {
38		"title": result.get("title", ""),
39		"url": result.get("url", ""),
40		"content": result.get("content", ""),
41		"engine": result.get("engine", "")
42		}
43		processed_results["results"].append(processed_result)
44
45		# Extract suggestions
46		if "suggestions" in result_data and result_data["suggestions"]:
47		try:
48		suggestion = result_data["suggestions"][0]
49		# Handle case where suggestion is a dict instead of a string
50		if isinstance(suggestion, dict):
51		processed_results["suggestion"] = suggestion.get("value", str(suggestion))
52		else:
53		processed_results["suggestion"] = suggestion
54		except (IndexError, TypeError) as e:
55		print(f"Error processing suggestions: {str(e)}")
56		processed_results["suggestion"] = None
57
58		# Extract answer if available
59		if "answers" in result_data and result_data["answers"]:
60		try:
61		answer = result_data["answers"][0]
62		# Handle case where answer is a dict instead of a string
63		if isinstance(answer, dict):
64		processed_results["answer"] = answer.get("content", str(answer))
65		else:
66		processed_results["answer"] = answer
67		except (IndexError, TypeError) as e:
68		print(f"Error processing answers: {str(e)}")
69		processed_results["answer"] = None
70
71		# Extract infobox if available
72		if "infoboxes" in result_data and result_data["infoboxes"]:
73		try:
74		infobox = result_data["infoboxes"][0]
75		# Handle case where infobox is a dict (which is expected)
76		processed_results["infobox"] = infobox
77		except (IndexError, TypeError) as e:
78		print(f"Error processing infoboxes: {str(e)}")
79		processed_results["infobox"] = None
80
81		return processed_results
82
83		def extract_search_context(search_results):
84		"""
85		Extract useful context from search results.
86
87		Args:
88		search_results: Search results
89
90		Returns:
91		str: Formatted context
92		"""
93		context_parts = []
94
95		# Add direct answer if available
96		if search_results.get("answer"):
97		try:
98		# Handle both string and dict answers
99		answer = search_results["answer"]
100		if isinstance(answer, dict):
101		answer_text = answer.get("content", str(answer))
102		else:
103		answer_text = str(answer)
104		context_parts.append(f"Direct answer: {answer_text}")
105		except Exception as e:
106		print(f"Error extracting answer for context: {str(e)}")
107
108		# Add snippets from results
109		if search_results.get("results"):
110		context_parts.append("Information from search results:")
111		for i, result in enumerate(search_results["results"][:3]): # Limit to 3 results
112		title = result.get("title", "Untitled")
113		content = result.get("content", "No content")
114		url = result.get("url", "")
115		context_parts.append(f"- {title}: {content} (Source: {url})")
116
117		# Join everything with newlines
118		return "\n".join(context_parts)
119
120		def test_dict_answer_handling():
121		"""Test handling of dict answers in search results."""
122		print("Testing dict answer handling...")
123
124		# Create a sample result with a dict answer
125		result_data = {
126		"results": [
127		{
128		"title": "Test Result",
129		"url": "https://example.com",
130		"content": "This is a test result",
131		"engine": "test"
132		}
133		],
134		"answers": [
135		{
136		"content": "This is a test answer",
137		"engine": "test"
138		}
139		],
140		"suggestions": [
141		{
142		"value": "test suggestion",
143		"engine": "test"
144		}
145		],
146		"infoboxes": [
147		{
148		"title": "Test Infobox",
149		"content": "This is a test infobox",
150		"engine": "test"
151		}
152		]
153		}
154
155		# Process the results
156		try:
157		processed_results = process_search_results(result_data)
158		print("Successfully processed results with dict answer")
159
160		# Check if the answer was extracted correctly
161		if processed_results.get("answer") == "This is a test answer":
162		print("SUCCESS: Answer extracted correctly from dict")
163		else:
164		print(f"FAILURE: Answer not extracted correctly. Got: {processed_results.get('answer')}")
165
166		# Check if the suggestion was extracted correctly
167		if processed_results.get("suggestion") == "test suggestion":
168		print("SUCCESS: Suggestion extracted correctly from dict")
169		else:
170		print(f"FAILURE: Suggestion not extracted correctly. Got: {processed_results.get('suggestion')}")
171
172		# Check if the infobox was extracted correctly
173		if isinstance(processed_results.get("infobox"), dict):
174		print("SUCCESS: Infobox extracted correctly as dict")
175		else:
176		print(f"FAILURE: Infobox not extracted correctly. Got: {type(processed_results.get('infobox'))}")
177		except Exception as e:
178		print(f"ERROR: Failed to process results: {str(e)}")
179
180		def test_extract_search_context():
181		"""Test extraction of context from search results with dict answer."""
182		print("\nTesting extract_search_context with dict answer...")
183
184		# Create a sample search result with a dict answer
185		search_results = {
186		"query": "test query",
187		"results": [
188		{
189		"title": "Test Result",
190		"url": "https://example.com",
191		"content": "This is a test result",
192		"engine": "test"
193		}
194		],
195		"answer": {
196		"content": "This is a test answer",
197		"engine": "test"
198		}
199		}
200
201		# Extract context
202		try:
203		context = extract_search_context(search_results)
204		print("Successfully extracted context from search results with dict answer")
205
206		# Check if the answer was included in the context
207		if "This is a test answer" in context:
208		print("SUCCESS: Answer included in context")
209		else:
210		print(f"FAILURE: Answer not included in context. Context: {context}")
211		except Exception as e:
212		print(f"ERROR: Failed to extract context: {str(e)}")
213
214		def test_sequence_item_error():
215		"""Test the specific error case: sequence item 0: expected str instance, dict found."""
216		print("\nTesting sequence item error case...")
217
218		# Create a sample result that would cause the error
219		result_data = {
220		"results": [],
221		"answers": [
222		{
223		"content": "This is a test answer",
224		"engine": "test"
225		}
226		]
227		}
228
229		# Original problematic code (commented out)
230		"""
231		processed_results = {
232		"answer": None
233		}
234
235		# This would cause the error: sequence item 0: expected str instance, dict found
236		if "answers" in result_data and result_data["answers"]:
237		processed_results["answer"] = result_data["answers"][0]
238
239		# Try to use the answer in a string operation
240		if processed_results["answer"]:
241		context = "Direct answer: " + processed_results["answer"]
242		"""
243
244		# Fixed code
245		try:
246		processed_results = {
247		"answer": None
248		}
249
250		# Extract answer with proper type handling
251		if "answers" in result_data and result_data["answers"]:
252		try:
253		answer = result_data["answers"][0]
254		# Handle case where answer is a dict instead of a string
255		if isinstance(answer, dict):
256		processed_results["answer"] = answer.get("content", str(answer))
257		else:
258		processed_results["answer"] = answer
259		except (IndexError, TypeError) as e:
260		print(f"Error processing answers: {str(e)}")
261
262		# Try to use the answer in a string operation
263		if processed_results["answer"]:
264		context = "Direct answer: " + processed_results["answer"]
265		print(f"Successfully created context: {context}")
266		print("SUCCESS: Avoided sequence item error")
267		else:
268		print("FAILURE: Answer not processed correctly")
269		except Exception as e:
270		print(f"ERROR: Failed to process results: {str(e)}")
271
272		if __name__ == "__main__":
273		test_dict_answer_handling()
274		test_extract_search_context()
275		test_sequence_item_error()

ollamatools-project/test_function_execution.py +0 -154

…	…	@@ -1,154 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Simple test script for function execution in the TransparentOllamaAgent.
4
5		This script tests the agent's ability to:
6		1. Detect when a response contains a function call
7		2. Parse and execute the function call
8		3. Return a proper response
9
10		Usage:
11		python test_function_execution.py
12		"""
13
14		import os
15		import sys
16		import json
17		from unittest.mock import MagicMock
18
19		# Function to test JSON parsing and function execution
20		def test_function_execution():
21		"""Test the function execution logic."""
22		# Sample function call in JSON format
23		function_call_json = json.dumps({
24		"function": {
25		"name": "opensearch_query",
26		"arguments": json.dumps({"query": "recipe for chilli sauce"})
27		}
28		})
29
30		# Sample search results
31		sample_search_results = {
32		"query": "recipe for chilli sauce",
33		"results": [
34		{
35		"title": "Homemade Chili Sauce Recipe",
36		"url": "https://example.com/chili-sauce",
37		"content": "A delicious homemade chili sauce recipe with tomatoes and spices.",
38		"engine": "google"
39		},
40		{
41		"title": "Spicy Chili Sauce - Quick and Easy",
42		"url": "https://example.com/spicy-chili",
43		"content": "Make this quick and easy spicy chili sauce in just 15 minutes.",
44		"engine": "bing"
45		}
46		],
47		"answer": "Chili sauce typically contains chili peppers, vinegar, sugar, and spices."
48		}
49
50		# Create a mock execute_search function
51		execute_search_mock = MagicMock(return_value=sample_search_results)
52
53		# Test the function execution logic
54		print("Testing function execution logic...")
55
56		# Check if the response looks like a JSON function call
57		response = function_call_json
58		if response.strip().startswith('{') and ('function' in response or 'name' in response):
59		print("Detected potential function call in response")
60		try:
61		# Parse the JSON response
62		func_data = json.loads(response)
63
64		# Extract function name and args
65		function_name = None
66		args = {}
67
68		# Handle different JSON formats
69		if 'function' in func_data:
70		function_name = func_data['function'].get('name')
71		args_str = func_data['function'].get('arguments', '{}')
72		if isinstance(args_str, str):
73		args = json.loads(args_str)
74		else:
75		args = args_str
76		elif 'name' in func_data:
77		function_name = func_data.get('name')
78		args = func_data.get('arguments', {})
79
80		if function_name:
81		print(f"Executing function: {function_name} with args: {args}")
82
83		# Execute the function
84		if function_name == 'opensearch_query' or function_name == 'search_web':
85		results = execute_search_mock(args.get('query', "recipe for chilli sauce"))
86		print(f"Search executed with query: {args.get('query', 'recipe for chilli sauce')}")
87		print(f"Results: {len(results.get('results', []))} items found")
88
89		# Extract context from results
90		context_parts = []
91
92		# Add direct answer if available
93		if results.get("answer"):
94		context_parts.append(f"Direct answer: {results['answer']}")
95
96		# Add snippets from results
97		if results.get("results"):
98		context_parts.append("Information from search results:")
99		for i, result in enumerate(results["results"][:3]): # Limit to 3 results
100		title = result.get("title", "Untitled")
101		content = result.get("content", "No content")
102		url = result.get("url", "")
103		context_parts.append(f"- {title}: {content} (Source: {url})")
104
105		# Join everything with newlines
106		context = "\n".join(context_parts)
107
108		# Create a response with the search results
109		final_response = f"Here's what I found about '{args.get('query', 'recipe for chilli sauce')}':\n\n{context}"
110		print("\nFinal response:")
111		print(final_response)
112
113		# Verify that the response doesn't contain raw JSON
114		if '"function":' not in final_response:
115		print("\nSUCCESS: Response doesn't contain raw JSON function call")
116		else:
117		print("\nFAILURE: Response still contains raw JSON function call")
118		else:
119		print(f"Function '{function_name}' not found")
120		else:
121		print("No function name found in the response")
122		except json.JSONDecodeError as e:
123		print(f"Response looked like JSON but couldn't be parsed: {e}")
124		except Exception as e:
125		print(f"Error executing function from response: {e}")
126		else:
127		print("Response doesn't look like a function call")
128
129		# Test malformed JSON handling
130		def test_malformed_json():
131		"""Test handling of malformed JSON."""
132		# Malformed JSON that looks like a function call
133		malformed_json = "{function: {name: opensearch_query, arguments: {query: 'recipe'}}}"
134
135		print("\nTesting malformed JSON handling...")
136
137		# Check if the response looks like a JSON function call
138		if malformed_json.strip().startswith('{') and ('function' in malformed_json or 'name' in malformed_json):
139		print("Detected potential function call in response")
140		try:
141		# Parse the JSON response
142		func_data = json.loads(malformed_json)
143		print("JSON parsed successfully (unexpected)")
144		except json.JSONDecodeError as e:
145		print(f"JSON parsing failed as expected: {e}")
146		print("Original response returned (correct behavior)")
147		except Exception as e:
148		print(f"Unexpected error: {e}")
149		else:
150		print("Response doesn't look like a function call (unexpected)")
151
152		if __name__ == "__main__":
153		test_function_execution()
154		test_malformed_json()

ollamatools-project/test_opensearch_error_handling.py +0 -233

…	…	@@ -1,233 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Test script for OpenSearchNG integration error handling.
4
5		This script tests the improved error handling for OpenSearchNG integration,
6		particularly for the error "sequence item 0: expected str instance, dict found".
7
8		Usage:
9		python test_opensearch_error_handling.py
10		"""
11
12		import os
13		import sys
14		import json
15		from unittest.mock import MagicMock, patch
16
17		# Add the parent directory to the Python path to import ollamatools
18		sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
19
20		# Import directly from the module files using importlib
21		import importlib.util
22
23		# Load the search module
24		search_path = os.path.join(os.path.dirname(__file__), 'ollamatools', 'agent', 'transparent', 'search.py')
25		spec = importlib.util.spec_from_file_location("search_module", search_path)
26		search_module = importlib.util.module_from_spec(spec)
27		spec.loader.exec_module(search_module)
28		execute_search = search_module.execute_search
29		extract_search_context = search_module.extract_search_context
30
31		# Load the opensearch_web_tools module
32		opensearch_path = os.path.join(os.path.dirname(__file__), 'ollamatools', 'opensearch_web_tools.py')
33		spec = importlib.util.spec_from_file_location("opensearch_module", opensearch_path)
34		opensearch_module = importlib.util.module_from_spec(spec)
35		spec.loader.exec_module(opensearch_module)
36		process_opensearch_results = opensearch_module.process_opensearch_results
37
38		def test_dict_answer_handling():
39		"""Test handling of dict answers in search results."""
40		print("Testing dict answer handling...")
41
42		# Create a sample result with a dict answer
43		result_data = {
44		"results": [
45		{
46		"title": "Test Result",
47		"url": "https://example.com",
48		"content": "This is a test result",
49		"engine": "test"
50		}
51		],
52		"answers": [
53		{
54		"content": "This is a test answer",
55		"engine": "test"
56		}
57		],
58		"suggestions": [
59		{
60		"value": "test suggestion",
61		"engine": "test"
62		}
63		],
64		"infoboxes": [
65		{
66		"title": "Test Infobox",
67		"content": "This is a test infobox",
68		"engine": "test"
69		}
70		]
71		}
72
73		# Process the results
74		try:
75		processed_results = process_opensearch_results(result_data, "test query")
76		print("Successfully processed results with dict answer")
77
78		# Check if the answer was extracted correctly
79		if processed_results.get("answer") == "This is a test answer":
80		print("SUCCESS: Answer extracted correctly from dict")
81		else:
82		print(f"FAILURE: Answer not extracted correctly. Got: {processed_results.get('answer')}")
83
84		# Check if the suggestion was extracted correctly
85		if processed_results.get("suggestion") == "test suggestion":
86		print("SUCCESS: Suggestion extracted correctly from dict")
87		else:
88		print(f"FAILURE: Suggestion not extracted correctly. Got: {processed_results.get('suggestion')}")
89
90		# Check if the infobox was extracted correctly
91		if isinstance(processed_results.get("infobox"), dict):
92		print("SUCCESS: Infobox extracted correctly as dict")
93		else:
94		print(f"FAILURE: Infobox not extracted correctly. Got: {type(processed_results.get('infobox'))}")
95		except Exception as e:
96		print(f"ERROR: Failed to process results: {str(e)}")
97
98		def test_extract_search_context():
99		"""Test extraction of context from search results with dict answer."""
100		print("\nTesting extract_search_context with dict answer...")
101
102		# Create a sample search result with a dict answer
103		search_results = {
104		"query": "test query",
105		"results": [
106		{
107		"title": "Test Result",
108		"url": "https://example.com",
109		"content": "This is a test result",
110		"engine": "test"
111		}
112		],
113		"answer": {
114		"content": "This is a test answer",
115		"engine": "test"
116		}
117		}
118
119		# Extract context
120		try:
121		context = extract_search_context(search_results)
122		print("Successfully extracted context from search results with dict answer")
123
124		# Check if the answer was included in the context
125		if "This is a test answer" in context:
126		print("SUCCESS: Answer included in context")
127		else:
128		print(f"FAILURE: Answer not included in context. Context: {context}")
129		except Exception as e:
130		print(f"ERROR: Failed to extract context: {str(e)}")
131
132		@patch('ollamatools.opensearch_web_tools.opensearch_query')
133		def test_execute_search_with_dict_answer(mock_opensearch_query):
134		"""Test execute_search with dict answer."""
135		print("\nTesting execute_search with dict answer...")
136
137		# Create a sample search result with a dict answer
138		search_results = {
139		"query": "test query",
140		"results": [
141		{
142		"title": "Test Result",
143		"url": "https://example.com",
144		"content": "This is a test result",
145		"engine": "test"
146		}
147		],
148		"answer": {
149		"content": "This is a test answer",
150		"engine": "test"
151		}
152		}
153
154		# Mock the opensearch_query function to return our sample results
155		mock_opensearch_query.return_value = search_results
156
157		# Create a mock permission manager
158		permission_manager = MagicMock()
159
160		# Create a mock execution history
161		execution_history = []
162
163		# Execute the search
164		try:
165		results = execute_search(
166		query="test query",
167		opensearch_config={},
168		permission_manager=permission_manager,
169		execution_history=execution_history
170		)
171		print("Successfully executed search with dict answer")
172
173		# Check if the answer was preserved in the results
174		if isinstance(results.get("answer"), dict) and results["answer"].get("content") == "This is a test answer":
175		print("SUCCESS: Answer preserved in results")
176		else:
177		print(f"FAILURE: Answer not preserved in results. Got: {results.get('answer')}")
178		except Exception as e:
179		print(f"ERROR: Failed to execute search: {str(e)}")
180
181		@patch('ollamatools.opensearch_web_tools.opensearch_query')
182		def test_execute_search_with_connection_error(mock_opensearch_query):
183		"""Test execute_search with connection error."""
184		print("\nTesting execute_search with connection error...")
185
186		# Mock the opensearch_query function to raise a connection error
187		from ollamatools.errors import WebError, ERROR_CODES
188		mock_opensearch_query.side_effect = WebError(
189		"Could not connect to OpenSearchNG server",
190		url="http://localhost:8081",
191		code=ERROR_CODES["URL_CONNECTION_ERROR"]
192		)
193
194		# Create a mock permission manager
195		permission_manager = MagicMock()
196
197		# Create a mock execution history
198		execution_history = []
199
200		# Mock the search_web function to return sample results
201		with patch('ollamatools.tools.web_tools.search_web') as mock_search_web:
202		mock_search_web.return_value = [
203		{
204		"title": "Fallback Result",
205		"url": "https://example.com",
206		"snippet": "This is a fallback result",
207		"engine": "fallback"
208		}
209		]
210
211		# Execute the search
212		try:
213		results = execute_search(
214		query="test query",
215		opensearch_config={},
216		permission_manager=permission_manager,
217		execution_history=execution_history
218		)
219		print("Successfully executed search with fallback")
220
221		# Check if fallback results were used
222		if results.get("engines") == ["fallback"]:
223		print("SUCCESS: Fallback search used")
224		else:
225		print(f"FAILURE: Fallback search not used. Got: {results.get('engines')}")
226		except Exception as e:
227		print(f"ERROR: Failed to execute search: {str(e)}")
228
229		if __name__ == "__main__":
230		test_dict_answer_handling()
231		test_extract_search_context()
232		test_execute_search_with_dict_answer()
233		test_execute_search_with_connection_error()

ollamatools-project/test_refactored_code.sh +0 -133

…	…	@@ -1,133 +0,0 @@
1		#!/bin/bash
2
3		# Script to run tests for both the original and refactored codebases
4		# This helps ensure that the refactored code maintains the same functionality
5
6		# Define the parent directory and project directories
7		PARENT_DIR=".."
8		ORIGINAL_DIR="$PARENT_DIR/ollamatools-project"
9		NEW_DIR="$PARENT_DIR/ollamatools-new"
10
11		# Function to display usage information
12		usage() {
13		echo "Usage: $0 [options] <test_file>"
14		echo ""
15		echo "Arguments:"
16		echo " <test_file> Path to the test file to run (relative to the tests directory)"
17		echo ""
18		echo "Options:"
19		echo " -o, --original Run tests only on the original codebase"
20		echo " -n, --new Run tests only on the new codebase"
21		echo " -h, --help Display this help message"
22		echo ""
23		echo "If no options are provided, tests will be run on both codebases."
24		echo ""
25		echo "Example:"
26		echo " $0 test_agent.py # Run test_agent.py on both codebases"
27		echo " $0 --original test_agent.py # Run test_agent.py only on the original codebase"
28		echo " $0 --new test_agent.py # Run test_agent.py only on the new codebase"
29		exit 1
30		}
31
32		# Parse command line arguments
33		RUN_ORIGINAL=true
34		RUN_NEW=true
35		TEST_FILE=""
36
37		while [[ $# -gt 0 ]]; do
38		case $1 in
39		-o|--original)
40		RUN_ORIGINAL=true
41		RUN_NEW=false
42		shift
43		;;
44		-n|--new)
45		RUN_ORIGINAL=false
46		RUN_NEW=true
47		shift
48		;;
49		-h|--help)
50		usage
51		;;
52		*)
53		TEST_FILE="$1"
54		shift
55		;;
56		esac
57		done
58
59		# Check if a test file was provided
60		if [ -z "$TEST_FILE" ]; then
61		echo "Error: No test file provided."
62		usage
63		fi
64
65		# Function to run tests
66		run_tests() {
67		local dir="$1"
68		local test_file="$2"
69		local name="$3"
70
71		echo "Running tests on $name codebase..."
72		echo "Directory: $dir"
73		echo "Test file: $test_file"
74		echo ""
75
76		# Check if the test file exists
77		if [ ! -f "$dir/tests/$test_file" ]; then
78		echo "Error: Test file $dir/tests/$test_file does not exist."
79		return 1
80		fi
81
82		# Run the tests
83		cd "$dir" && python -m pytest "tests/$test_file" -v
84		local result=$?
85
86		if [ $result -eq 0 ]; then
87		echo "Tests passed on $name codebase."
88		else
89		echo "Tests failed on $name codebase."
90		fi
91
92		echo ""
93		return $result
94		}
95
96		# Run tests on the original codebase
97		ORIGINAL_RESULT=0
98		if [ "$RUN_ORIGINAL" = true ]; then
99		run_tests "$ORIGINAL_DIR" "$TEST_FILE" "original"
100		ORIGINAL_RESULT=$?
101		fi
102
103		# Run tests on the new codebase
104		NEW_RESULT=0
105		if [ "$RUN_NEW" = true ]; then
106		run_tests "$NEW_DIR" "$TEST_FILE" "new"
107		NEW_RESULT=$?
108		fi
109
110		# Print summary
111		echo "Test Summary:"
112		if [ "$RUN_ORIGINAL" = true ]; then
113		if [ $ORIGINAL_RESULT -eq 0 ]; then
114		echo " Original codebase: PASSED"
115		else
116		echo " Original codebase: FAILED"
117		fi
118		fi
119
120		if [ "$RUN_NEW" = true ]; then
121		if [ $NEW_RESULT -eq 0 ]; then
122		echo " New codebase: PASSED"
123		else
124		echo " New codebase: FAILED"
125		fi
126		fi
127
128		# Exit with an error if any tests failed
129		if [ $ORIGINAL_RESULT -ne 0 ] || [ $NEW_RESULT -ne 0 ]; then
130		exit 1
131		fi
132
133		exit 0

ollamatools-project/tests/__init__.py +0 -4

…	…	@@ -1,4 +0,0 @@
1		"""
2		Test package for OllamaTools.
3		"""
4		# Intentionally left empty

ollamatools-project/tests/simple_test_web_tools.py +0 -115

…	…	@@ -1,115 +0,0 @@
1		"""
2		Simple script to manually test the web tools functionality.
3		"""
4
5		import sys
6		import os
7		from unittest.mock import MagicMock
8
9		# Ensure the project directory is in the Python path
10		project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
11		if project_root not in sys.path:
12		sys.path.insert(0, project_root)
13
14		# Import the web tools
15		from ollamatools.tools.web_tools import fetch_url, get_weather, search_web
16
17		def test_fetch_url():
18		"""Test the fetch_url function."""
19		print("\nTesting fetch_url...")
20
21		# Test a simple GET request
22		try:
23		result = fetch_url("https://httpbin.org/get")
24		print(f"Status: {result['status']}")
25		print(f"Content type: {result['headers'].get('Content-Type')}")
26		print("Test passed!")
27		except Exception as e:
28		print(f"Test failed: {e}")
29
30		def test_search_web():
31		"""Test the search_web function."""
32		print("\nTesting search_web...")
33
34		# Test web search with fallback method (no API key)
35		try:
36		results = search_web("Python programming language")
37		print(f"Found {len(results)} results")
38		if results:
39		print(f"First result title: {results[0]['title']}")
40		print("Test passed!")
41		except Exception as e:
42		print(f"Test failed: {e}")
43
44		def test_get_weather_mock():
45		"""Test the get_weather function with mocked permission manager."""
46		print("\nTesting get_weather permission handling...")
47
48		# Create a mock permission manager
49		mock_permission_manager = MagicMock()
50
51		# Test 1: Permission granted
52		mock_permission_manager.request_permission.return_value = True
53
54		try:
55		# This should still fail because we don't have a valid API key,
56		# but it should get past the permission check
57		try:
58		get_weather(
59		"London",
60		api_key="fake_api_key",
61		permission_manager=mock_permission_manager
62		)
63		except ValueError as e:
64		if "API key" in str(e):
65		print("Permission check passed, failed with API key error as expected")
66		else:
67		raise
68
69		# Test that permission was requested
70		if mock_permission_manager.request_permission.called:
71		print("Permission request called correctly")
72		else:
73		print("Permission request was not called")
74		except Exception as e:
75		print(f"Test failed: {e}")
76
77		# Test 2: Permission denied
78		mock_permission_manager.request_permission.return_value = False
79		mock_permission_manager.request_permission.reset_mock()
80
81		try:
82		try:
83		get_weather(
84		"London",
85		api_key="fake_api_key",
86		permission_manager=mock_permission_manager
87		)
88		print("Test failed: Expected PermissionError but no exception was raised")
89		except PermissionError:
90		print("Permission denied test passed: PermissionError was raised")
91		except Exception as e:
92		print(f"Test failed: Expected PermissionError but got {type(e).__name__}: {e}")
93
94		# Test that permission was requested
95		if mock_permission_manager.request_permission.called:
96		print("Permission request called correctly")
97		else:
98		print("Permission request was not called")
99		except Exception as e:
100		print(f"Test failed: {e}")
101
102
103		def main():
104		"""Run all tests."""
105		print("Manual testing of web tools")
106		print("==========================")
107
108		test_fetch_url()
109		test_search_web()
110		test_get_weather_mock()
111
112		print("\nAll tests completed.")
113
114		if __name__ == "__main__":
115		main()

ollamatools-project/tests/test_agent.py +0 -540

…	…	@@ -1,540 +0,0 @@
1		"""
2		Tests for the refactored OllamaAgent class.
3
4		This module contains unit tests for the OllamaAgent implementation,
5		including initialization, function registration, function detection and execution,
6		error handling, and integration with the permission system.
7		"""
8
9		import json
10		import pytest
11		from unittest.mock import patch, MagicMock, call, PropertyMock
12		import inspect
13
14		from ollamatools.agent import OllamaAgent
15		from ollamatools.config import Config
16		from ollamatools.security import PermissionManager
17
18
19		class TestOllamaAgent:
20		"""Test suite for OllamaAgent class."""
21
22		@patch('ollama.Client')
23		def test_initialization(self, mock_client_class):
24		"""Test initialization of OllamaAgent with default parameters."""
25		# Setup mock
26		mock_client = MagicMock()
27		mock_client_class.return_value = mock_client
28
29		# Create agent
30		agent = OllamaAgent()
31
32		# Check default values
33		assert agent.config.model == "llama3:latest"
34		assert agent.config.host == "http://localhost:11434"
35		assert agent.config.permission_prompts is True
36		assert agent.config.sudo_always_prompt is True
37
38		# Check if client was initialized
39		mock_client_class.assert_called_once_with(host="http://localhost:11434")
40
41		# Check if permission manager was initialized
42		assert isinstance(agent.permission_manager, PermissionManager)
43		assert agent.permission_manager.prompt_enabled is True
44		assert agent.permission_manager.sudo_always_prompt is True
45
46		# Verify that registered_functions is initialized
47		assert hasattr(agent, 'registered_functions')
48		assert isinstance(agent.registered_functions, dict)
49		assert len(agent.registered_functions) > 0
50
51		@patch('ollama.Client')
52		def test_initialization_with_custom_params(self, mock_client_class):
53		"""Test initialization of OllamaAgent with custom parameters."""
54		# Setup mock
55		mock_client = MagicMock()
56		mock_client_class.return_value = mock_client
57
58		config = Config(
59		model="mistral:latest",
60		host="http://192.168.1.100:11434",
61		permission_prompts=False,
62		sudo_always_prompt=False
63		)
64
65		# Create agent
66		agent = OllamaAgent(config=config)
67
68		# Check custom values
69		assert agent.config.model == "mistral:latest"
70		assert agent.config.host == "http://192.168.1.100:11434"
71		assert agent.config.permission_prompts is False
72		assert agent.config.sudo_always_prompt is False
73
74		# Check if client was initialized with custom host
75		mock_client_class.assert_called_once_with(host="http://192.168.1.100:11434")
76
77		# Check if permission manager was initialized with custom parameters
78		assert isinstance(agent.permission_manager, PermissionManager)
79		assert agent.permission_manager.prompt_enabled is False
80		assert agent.permission_manager.sudo_always_prompt is False
81
82		@patch('ollama.Client')
83		def test_generate_tool_definitions(self, mock_client_class):
84		"""Test generation of tool definitions."""
85		# Setup mock
86		mock_client = MagicMock()
87		mock_client_class.return_value = mock_client
88
89		# Create agent
90		agent = OllamaAgent()
91
92		# Get tool definitions
93		tools = agent.generate_tool_definitions()
94
95		# Verify tool definitions
96		assert isinstance(tools, list)
97		assert len(tools) > 0
98
99		# Check format of first tool
100		first_tool = tools[0]
101		assert 'type' in first_tool
102		assert first_tool['type'] == 'function'
103		assert 'function' in first_tool
104		assert 'name' in first_tool['function']
105		assert 'description' in first_tool['function']
106		assert 'parameters' in first_tool['function']
107		assert 'properties' in first_tool['function']['parameters']
108		assert 'required' in first_tool['function']['parameters']
109
110		@patch('ollama.Client')
111		def test_chat_without_function_calls(self, mock_client_class):
112		"""Test basic chat functionality without function calls."""
113		# Setup mock client
114		mock_client = MagicMock()
115		mock_client_class.return_value = mock_client
116
117		# Create a mock response object with both dict and attribute access
118		mock_response = MagicMock()
119		mock_message = MagicMock()
120		type(mock_message).content = PropertyMock(return_value="This is a test response.")
121		type(mock_message).tool_calls = PropertyMock(return_value=None)
122		type(mock_response).message = PropertyMock(return_value=mock_message)
123
124		# Configure the mock client to return our mock response
125		mock_client.chat.return_value = mock_response
126
127		# Create agent
128		agent = OllamaAgent()
129
130		# Test chat
131		response = agent.chat("Hello, world!")
132
133		# Check if client.chat was called
134		mock_client.chat.assert_called_once()
135
136		# Check response
137		assert response == "This is a test response."
138
139		@patch('ollama.Client')
140		def test_chat_with_system_prompt(self, mock_client_class):
141		"""Test chat with system prompt."""
142		# Setup mock client
143		mock_client = MagicMock()
144		mock_client_class.return_value = mock_client
145
146		# Create a mock response object
147		mock_response = MagicMock()
148		mock_message = MagicMock()
149		type(mock_message).content = PropertyMock(return_value="This is a test response.")
150		type(mock_message).tool_calls = PropertyMock(return_value=None)
151		type(mock_response).message = PropertyMock(return_value=mock_message)
152
153		# Configure the mock client to return our mock response
154		mock_client.chat.return_value = mock_response
155
156		# Create agent
157		agent = OllamaAgent()
158
159		# Test chat with system prompt
160		system_prompt = "You are a helpful assistant."
161		response = agent.chat("Hello, world!", system_prompt=system_prompt)
162
163		# Check if arguments were correct
164		args, kwargs = mock_client.chat.call_args
165		assert kwargs["model"] == "llama3:latest"
166		assert kwargs["messages"] == [{"role": "user", "content": "Hello, world!"}]
167		assert kwargs["options"]["system"] == system_prompt
168
169		# Check response
170		assert response == "This is a test response."
171
172		@patch('ollama.Client')
173		def test_chat_with_context(self, mock_client_class):
174		"""Test chat with conversation context."""
175		# Setup mock client
176		mock_client = MagicMock()
177		mock_client_class.return_value = mock_client
178
179		# Create a mock response object
180		mock_response = MagicMock()
181		mock_message = MagicMock()
182		type(mock_message).content = PropertyMock(return_value="This is a test response.")
183		type(mock_message).tool_calls = PropertyMock(return_value=None)
184		type(mock_response).message = PropertyMock(return_value=mock_message)
185
186		# Configure the mock client to return our mock response
187		mock_client.chat.return_value = mock_response
188
189		# Create agent
190		agent = OllamaAgent()
191
192		# Create conversation context
193		context = [
194		{"role": "user", "content": "Hi there"},
195		{"role": "assistant", "content": "Hello! How can I help you?"}
196		]
197
198		# Test chat with context
199		response = agent.chat("Tell me a joke", context=context)
200
201		# Check if arguments were correct
202		args, kwargs = mock_client.chat.call_args
203		assert kwargs["model"] == "llama3:latest"
204		assert kwargs["messages"] == [
205		{"role": "user", "content": "Hi there"},
206		{"role": "assistant", "content": "Hello! How can I help you?"},
207		{"role": "user", "content": "Tell me a joke"}
208		]
209
210		# Check response
211		assert response == "This is a test response."
212
213		@patch('ollama.Client')
214		def test_chat_with_function_call(self, mock_client_class):
215		"""Test chat with function call detection and execution."""
216		# Setup mock client
217		mock_client = MagicMock()
218		mock_client_class.return_value = mock_client
219
220		# First response with tool call
221		mock_first_response = MagicMock()
222		mock_first_message = MagicMock()
223		mock_tool_call = MagicMock()
224		mock_function = MagicMock()
225
226		type(mock_function).name = PropertyMock(return_value="get_weather")
227		type(mock_function).arguments = PropertyMock(return_value={"location": "New York"})
228		type(mock_tool_call).function = PropertyMock(return_value=mock_function)
229
230		type(mock_first_message).content = PropertyMock(return_value="Let me check the weather for you.")
231		type(mock_first_message).tool_calls = PropertyMock(return_value=[mock_tool_call])
232		type(mock_first_response).message = PropertyMock(return_value=mock_first_message)
233
234		# Final response after function execution
235		mock_final_response = MagicMock()
236		mock_final_message = MagicMock()
237		type(mock_final_message).content = PropertyMock(return_value="The weather in New York is sunny with a temperature of (75°F).")
238		type(mock_final_message).tool_calls = PropertyMock(return_value=None)
239		type(mock_final_response).message = PropertyMock(return_value=mock_final_message)
240
241		# Configure the mock client to return our mock responses in sequence
242		mock_client.chat.side_effect = [mock_first_response, mock_final_response]
243
244		# Create agent with mocked functions
245		agent = OllamaAgent()
246
247		# Mock get_weather function
248		mock_get_weather = MagicMock(return_value={
249		"condition": "sunny",
250		"temperature": 75,
251		"unit": "F"
252		})
253		agent.registered_functions["get_weather"] = mock_get_weather
254
255		# Mock permission manager
256		agent.permission_manager.request_permission = MagicMock(return_value=True)
257
258		# Test chat with function call
259		response = agent.chat("What's the weather in New York?")
260
261		# Check if function was called with the expected arguments
262		mock_get_weather.assert_called_once_with(location="New York")
263
264		# Check if chat was called twice
265		assert mock_client.chat.call_count == 2
266
267		# Check final response
268		assert response == "The weather in New York is sunny with a temperature of (75°F)."
269
270		@patch('ollama.Client')
271		def test_chat_with_multiple_function_calls(self, mock_client_class):
272		"""Test chat with multiple function calls."""
273		# Setup mock client
274		mock_client = MagicMock()
275		mock_client_class.return_value = mock_client
276
277		# Mock response with multiple tool calls
278		mock_first_response = MagicMock()
279		mock_first_message = MagicMock()
280
281		# First tool call (get_weather)
282		mock_tool_call1 = MagicMock()
283		mock_function1 = MagicMock()
284		type(mock_function1).name = PropertyMock(return_value="get_weather")
285		type(mock_function1).arguments = PropertyMock(return_value={"location": "New York"})
286		type(mock_tool_call1).function = PropertyMock(return_value=mock_function1)
287
288		# Second tool call (search_web)
289		mock_tool_call2 = MagicMock()
290		mock_function2 = MagicMock()
291		type(mock_function2).name = PropertyMock(return_value="search_web")
292		type(mock_function2).arguments = PropertyMock(return_value={"query": "New York attractions"})
293		type(mock_tool_call2).function = PropertyMock(return_value=mock_function2)
294
295		type(mock_first_message).content = PropertyMock(return_value="Let me check both the weather and search for information.")
296		type(mock_first_message).tool_calls = PropertyMock(return_value=[mock_tool_call1, mock_tool_call2])
297		type(mock_first_response).message = PropertyMock(return_value=mock_first_message)
298
299		# Final response after function execution
300		mock_final_response = MagicMock()
301		mock_final_message = MagicMock()
302		type(mock_final_message).content = PropertyMock(return_value="The weather in New York is sunny. Some popular attractions include Central Park and the Statue of Liberty.")
303		type(mock_final_message).tool_calls = PropertyMock(return_value=None)
304		type(mock_final_response).message = PropertyMock(return_value=mock_final_message)
305
306		# Configure the mock client to return our mock responses in sequence
307		mock_client.chat.side_effect = [mock_first_response, mock_final_response]
308
309		# Create agent with mocked functions
310		agent = OllamaAgent()
311
312		# Mock functions
313		mock_get_weather = MagicMock(return_value={
314		"condition": "sunny",
315		"temperature": 75,
316		"unit": "F"
317		})
318		mock_search_web = MagicMock(return_value=[
319		{"title": "Top 10 Attractions in NYC", "url": "https://example.com/nyc"}
320		])
321		agent.registered_functions["get_weather"] = mock_get_weather
322		agent.registered_functions["search_web"] = mock_search_web
323
324		# Mock permission manager
325		agent.permission_manager.request_permission = MagicMock(return_value=True)
326
327		# Test chat with multiple function calls
328		response = agent.chat("What's the weather in New York and what attractions should I visit?")
329
330		# Check if functions were called with the expected arguments
331		mock_get_weather.assert_called_once_with(location="New York")
332		mock_search_web.assert_called_once_with(query="New York attractions")
333
334		# Check if chat was called twice
335		assert mock_client.chat.call_count == 2
336
337		# Check final response
338		assert response == "The weather in New York is sunny. Some popular attractions include Central Park and the Statue of Liberty."
339
340		@patch('ollama.Client')
341		def test_chat_with_permission_denied(self, mock_client_class):
342		"""Test chat with function call but permission denied."""
343		# Setup mock client
344		mock_client = MagicMock()
345		mock_client_class.return_value = mock_client
346
347		# Mock response with tool call
348		mock_first_response = MagicMock()
349		mock_first_message = MagicMock()
350		mock_tool_call = MagicMock()
351		mock_function = MagicMock()
352
353		type(mock_function).name = PropertyMock(return_value="execute_command")
354		type(mock_function).arguments = PropertyMock(return_value={"command": "rm -rf /"})
355		type(mock_tool_call).function = PropertyMock(return_value=mock_function)
356
357		type(mock_first_message).content = PropertyMock(return_value="I'll delete those files for you.")
358		type(mock_first_message).tool_calls = PropertyMock(return_value=[mock_tool_call])
359		type(mock_first_response).message = PropertyMock(return_value=mock_first_message)
360
361		# Final response after permission denial
362		mock_final_response = MagicMock()
363		mock_final_message = MagicMock()
364		type(mock_final_message).content = PropertyMock(return_value="I'm sorry, I don't have permission to execute that command.")
365		type(mock_final_message).tool_calls = PropertyMock(return_value=None)
366		type(mock_final_response).message = PropertyMock(return_value=mock_final_message)
367
368		# Configure the mock client to return our mock responses in sequence
369		mock_client.chat.side_effect = [mock_first_response, mock_final_response]
370
371		# Create agent
372		agent = OllamaAgent()
373
374		# Mock execute_command function (should never be called)
375		mock_execute_command = MagicMock()
376		agent.registered_functions["execute_command"] = mock_execute_command
377
378		# Mock permission manager to DENY permission
379		agent.permission_manager.request_permission = MagicMock(return_value=False)
380
381		# Test chat with function call but permission denied
382		response = agent.chat("Delete all my files")
383
384		# Function should not be called due to permission denial
385		mock_execute_command.assert_not_called()
386
387		# Check final response
388		assert response == "I'm sorry, I don't have permission to execute that command."
389
390		@patch('ollama.Client')
391		def test_chat_error_handling(self, mock_client_class):
392		"""Test error handling during chat."""
393		# Setup mock client to raise an exception
394		mock_client = MagicMock()
395		mock_client.chat.side_effect = Exception("Test error")
396		mock_client_class.return_value = mock_client
397
398		# Create agent
399		agent = OllamaAgent()
400
401		# Test chat error handling
402		response = agent.chat("Hello, world!")
403
404		# Check response contains error message
405		assert "Error communicating with Ollama" in response
406		assert "Test error" in response
407
408		@patch('ollama.Client')
409		def test_sudo_detection_and_permission(self, mock_client_class):
410		"""Test sudo detection and special permission handling."""
411		# Setup mock client
412		mock_client = MagicMock()
413		mock_client_class.return_value = mock_client
414
415		# Mock response with sudo command
416		mock_first_response = MagicMock()
417		mock_first_message = MagicMock()
418		mock_tool_call = MagicMock()
419		mock_function = MagicMock()
420
421		type(mock_function).name = PropertyMock(return_value="execute_command")
422		type(mock_function).arguments = PropertyMock(return_value={"command": "sudo apt-get update"})
423		type(mock_tool_call).function = PropertyMock(return_value=mock_function)
424
425		type(mock_first_message).content = PropertyMock(return_value="I'll update your packages.")
426		type(mock_first_message).tool_calls = PropertyMock(return_value=[mock_tool_call])
427		type(mock_first_response).message = PropertyMock(return_value=mock_first_message)
428
429		# Final response after function execution
430		mock_final_response = MagicMock()
431		mock_final_message = MagicMock()
432		type(mock_final_message).content = PropertyMock(return_value="I've updated your packages.")
433		type(mock_final_message).tool_calls = PropertyMock(return_value=None)
434		type(mock_final_response).message = PropertyMock(return_value=mock_final_message)
435
436		# Configure the mock client to return our mock responses in sequence
437		mock_client.chat.side_effect = [mock_first_response, mock_final_response]
438
439		# Create agent
440		agent = OllamaAgent()
441
442		# Mock execute_command function
443		mock_execute_command = MagicMock(return_value={"stdout": "Updated packages", "stderr": "", "returncode": 0})
444		agent.registered_functions["execute_command"] = mock_execute_command
445
446		# Mock sudo detection
447		agent.permission_manager.detect_sudo = MagicMock(return_value=True)
448
449		# Mock permission manager to grant permission
450		agent.permission_manager.request_permission = MagicMock(return_value=True)
451
452		# Test chat with sudo command
453		response = agent.chat("Update my packages")
454
455		# Check if sudo was detected
456		agent.permission_manager.detect_sudo.assert_called_once()
457
458		# Check if permission was requested with is_sudo=True
459		agent.permission_manager.request_permission.assert_called_once_with(
460		function_name="execute_command",
461		args={"command": "sudo apt-get update"},
462		is_sudo=True
463		)
464
465		# Check if function was called
466		mock_execute_command.assert_called_once_with(command="sudo apt-get update")
467
468		# Check final response
469		assert response == "I've updated your packages."
470
471		@patch('ollama.Client')
472		def test_tool_definition_generation(self, mock_client_class):
473		"""Test generation of tool definitions."""
474		# Setup mock client
475		mock_client = MagicMock()
476		mock_client_class.return_value = mock_client
477
478		# Create agent
479		agent = OllamaAgent()
480
481		# Define a test function
482		def test_function(required_param: str, optional_param: int = 42) -> str:
483		"""
484		Test function description.
485
486		Args:
487		required_param: Description of required parameter
488		optional_param: Description of optional parameter
489
490		Returns:
491		str: Return value description
492		"""
493		return f"{required_param} {optional_param}"
494
495		# Generate tool definition
496		tool_def = agent._generate_tool_definition(
497		"test_function",
498		test_function,
499		inspect.getdoc(test_function)
500		)
501
502		# Check tool definition structure
503		assert tool_def["type"] == "function"
504		assert tool_def["function"]["name"] == "test_function"
505		assert "Test function description" in tool_def["function"]["description"]
506		assert tool_def["function"]["parameters"]["type"] == "object"
507		assert "required_param" in tool_def["function"]["parameters"]["properties"]
508		assert "optional_param" in tool_def["function"]["parameters"]["properties"]
509		assert tool_def["function"]["parameters"]["properties"]["required_param"]["type"] == "string"
510		assert tool_def["function"]["parameters"]["properties"]["optional_param"]["type"] == "integer"
511		assert "required_param" in tool_def["function"]["parameters"]["required"]
512		assert "optional_param" not in tool_def["function"]["parameters"]["required"]
513
514		@patch('ollama.Client')
515		def test_dictionary_access_fallback(self, mock_client_class):
516		"""Test fallback to dictionary access for Ollama response."""
517		# Setup mock client
518		mock_client = MagicMock()
519		mock_client_class.return_value = mock_client
520
521		# Create a traditional dictionary response (older Ollama client)
522		dict_response = {
523		"message": {
524		"role": "assistant",
525		"content": "This is a dictionary response.",
526		"tool_calls": []
527		}
528		}
529
530		# Configure mock client to return dictionary response
531		mock_client.chat.return_value = dict_response
532
533		# Create agent
534		agent = OllamaAgent()
535
536		# Test chat with dictionary response
537		response = agent.chat("Hello, world!")
538
539		# Check response
540		assert response == "This is a dictionary response."

ollamatools-project/tests/test_config.py +0 -149

…	…	@@ -1,149 +0,0 @@
1		"""
2		Tests for the configuration module.
3		"""
4
5		import json
6		import os
7		import tempfile
8		from unittest.mock import patch, mock_open
9
10		import pytest
11		from pydantic import ValidationError
12
13		from ollamatools.config import Config
14
15
16		def test_config_defaults():
17		"""Test that default configuration values are set correctly."""
18		config = Config()
19		assert config.model == "llama3.1"
20		assert config.host == "http://localhost:11434"
21		assert config.permission_prompts is True
22		assert config.sudo_always_prompt is True
23		assert config.timeout_seconds == 60
24		assert config.max_file_size is None
25
26
27		def test_config_custom_values():
28		"""Test that custom configuration values are set correctly."""
29		config = Config(
30		model="custom-model",
31		host="http://custom-host:11434",
32		permission_prompts=False,
33		sudo_always_prompt=False,
34		timeout_seconds=30,
35		max_file_size=1024
36		)
37		assert config.model == "custom-model"
38		assert config.host == "http://custom-host:11434"
39		assert config.permission_prompts is False
40		assert config.sudo_always_prompt is False
41		assert config.timeout_seconds == 30
42		assert config.max_file_size == 1024
43
44
45		def test_config_validation():
46		"""Test configuration validation."""
47		# Test invalid timeout
48		with pytest.raises(ValidationError):
49		Config(timeout_seconds=0)
50
51		with pytest.raises(ValidationError):
52		Config(timeout_seconds=-10)
53
54		# Test invalid max_file_size
55		with pytest.raises(ValidationError):
56		Config(max_file_size=0)
57
58		with pytest.raises(ValidationError):
59		Config(max_file_size=-1024)
60
61
62		def test_config_load_from_file():
63		"""Test loading configuration from a file."""
64		config_data = {
65		"model": "file-model",
66		"host": "http://file-host:11434",
67		"permission_prompts": False,
68		"sudo_always_prompt": False,
69		"timeout_seconds": 45,
70		"max_file_size": 2048
71		}
72
73		# Mock the open function to return a file-like object
74		with patch("builtins.open", mock_open(read_data=json.dumps(config_data))):
75		# Mock os.path.exists to return True
76		with patch("os.path.exists", return_value=True):
77		config = Config.load_from_file("fake_path.json")
78
79		assert config.model == "file-model"
80		assert config.host == "http://file-host:11434"
81		assert config.permission_prompts is False
82		assert config.sudo_always_prompt is False
83		assert config.timeout_seconds == 45
84		assert config.max_file_size == 2048
85
86
87		def test_config_load_from_file_not_found():
88		"""Test loading configuration from a non-existent file."""
89		with patch("os.path.exists", return_value=False):
90		with pytest.raises(FileNotFoundError):
91		Config.load_from_file("non_existent_file.json")
92
93
94		def test_config_load_from_file_invalid_json():
95		"""Test loading configuration from a file with invalid JSON."""
96		with patch("builtins.open", mock_open(read_data="invalid json")):
97		with patch("os.path.exists", return_value=True):
98		with pytest.raises(json.JSONDecodeError):
99		Config.load_from_file("invalid_json.json")
100
101
102		def test_config_save_to_file():
103		"""Test saving configuration to a file."""
104		config = Config(
105		model="save-model",
106		timeout_seconds=90
107		)
108
109		# Use a temporary file for testing
110		with tempfile.NamedTemporaryFile(delete=False) as temp_file:
111		temp_path = temp_file.name
112
113		try:
114		# Save configuration to the temporary file
115		config.save_to_file(temp_path)
116
117		# Read the saved configuration
118		with open(temp_path, 'r') as f:
119		saved_data = json.load(f)
120
121		# Verify saved data
122		assert saved_data["model"] == "save-model"
123		assert saved_data["timeout_seconds"] == 90
124		assert saved_data["permission_prompts"] is True # Default value
125		finally:
126		# Clean up
127		if os.path.exists(temp_path):
128		os.unlink(temp_path)
129
130
131		def test_config_to_dict():
132		"""Test converting configuration to a dictionary."""
133		config = Config(
134		model="dict-model",
135		timeout_seconds=120
136		)
137
138		config_dict = config.to_dict()
139
140		assert isinstance(config_dict, dict)
141		assert config_dict["model"] == "dict-model"
142		assert config_dict["timeout_seconds"] == 120
143		assert config_dict["permission_prompts"] is True # Default value
144
145
146		def test_default_config_path():
147		"""Test getting the default configuration path."""
148		expected_path = os.path.expanduser("~/.config/ollamatools/config.json")
149		assert Config.default_config_path() == expected_path

ollamatools-project/tests/test_doc_tools.py +0 -468

…	…	@@ -1,468 +0,0 @@
1		"""
2		Tests for the documentation tools.
3		"""
4
5		import pytest
6		from unittest.mock import MagicMock, patch
7		from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
8
9
10		# Tests for generate_markdown
11		def test_generate_markdown_with_markdown_format():
12		"""Test markdown generation with markdown format."""
13		content = "This is a test content.\n\nWith multiple paragraphs."
14		title = "Test Document"
15
16		result = generate_markdown(content, title, format_type='markdown')
17
18		assert result.startswith("# Test Document")
19		assert "This is a test content." in result
20		assert "With multiple paragraphs." in result
21
22
23		def test_generate_markdown_with_html_format():
24		"""Test markdown generation with HTML format."""
25		content = "This is a test content.\n\nWith multiple paragraphs."
26		title = "Test Document"
27
28		result = generate_markdown(content, title, format_type='html')
29
30		assert "<title>Test Document</title>" in result
31		assert "<h1>Test Document</h1>" in result
32		assert "This is a test content." in result
33		assert "With multiple paragraphs." in result
34		assert "<!DOCTYPE html>" in result
35
36
37		def test_generate_markdown_with_invalid_format():
38		"""Test markdown generation with an invalid format."""
39		content = "This is a test content."
40		title = "Test Document"
41
42		with pytest.raises(ValueError) as excinfo:
43		generate_markdown(content, title, format_type='invalid')
44
45		assert "Format type must be 'markdown' or 'html'" in str(excinfo.value)
46
47
48		def test_generate_markdown_with_empty_content():
49		"""Test markdown generation with empty content."""
50		content = ""
51		title = "Test Document"
52
53		with pytest.raises(ValueError) as excinfo:
54		generate_markdown(content, title)
55
56		assert "Content must be a non-empty string" in str(excinfo.value)
57
58
59		def test_generate_markdown_with_empty_title():
60		"""Test markdown generation with empty title."""
61		content = "This is a test content."
62		title = ""
63
64		with pytest.raises(ValueError) as excinfo:
65		generate_markdown(content, title)
66
67		assert "Title must be a non-empty string" in str(excinfo.value)
68
69
70		def test_generate_markdown_with_permission_granted():
71		"""Test markdown generation with permission granted."""
72		content = "This is a test content."
73		title = "Test Document"
74
75		# Create a mock permission manager
76		permission_manager = MagicMock()
77		permission_manager.request_permission.return_value = True
78
79		result = generate_markdown(content, title, permission_manager=permission_manager)
80
81		assert result.startswith("# Test Document")
82		permission_manager.request_permission.assert_called_once()
83
84
85		def test_generate_markdown_with_permission_denied():
86		"""Test markdown generation with permission denied."""
87		content = "This is a test content."
88		title = "Test Document"
89
90		# Create a mock permission manager
91		permission_manager = MagicMock()
92		permission_manager.request_permission.return_value = False
93
94		with pytest.raises(PermissionError) as excinfo:
95		generate_markdown(content, title, permission_manager=permission_manager)
96
97		assert "Permission denied" in str(excinfo.value)
98		permission_manager.request_permission.assert_called_once()
99
100
101		# Tests for extract_context
102		def test_extract_context_basic():
103		"""Test basic context extraction."""
104		query = "python programming"
105		documents = [
106		"Python is a programming language. It is widely used in data science.",
107		"JavaScript is a web programming language. It runs in browsers.",
108		"Data science uses statistical methods to analyze data."
109		]
110
111		result = extract_context(query, documents)
112
113		assert "Python is a programming language" in result
114		assert "JavaScript is a web programming language" in result
115		assert "Data science" not in result # No keyword match
116
117
118		def test_extract_context_with_max_length():
119		"""Test context extraction with max length limit."""
120		query = "programming language"
121		documents = [
122		"Python is a programming language. It is widely used in data science.",
123		"JavaScript is a web programming language. It runs in browsers.",
124		"Java is an object-oriented programming language. It is used for enterprise applications."
125		]
126
127		# Set max_length to fit only the first two sentences
128		max_len = len("Python is a programming language. JavaScript is a web programming language.") + 5
129		result = extract_context(query, documents, max_length=max_len)
130
131		assert "Python is a programming language" in result
132		assert "JavaScript is a web programming language" in result
133		assert "Java is an object-oriented programming language" not in result
134
135
136		def test_extract_context_empty_query():
137		"""Test context extraction with empty query."""
138		query = ""
139		documents = ["Python is a programming language."]
140
141		with pytest.raises(ValueError) as excinfo:
142		extract_context(query, documents)
143
144		assert "Query must be a non-empty string" in str(excinfo.value)
145
146
147		def test_extract_context_empty_documents():
148		"""Test context extraction with empty documents."""
149		query = "python"
150		documents = []
151
152		with pytest.raises(ValueError) as excinfo:
153		extract_context(query, documents)
154
155		assert "Documents must be a non-empty list of strings" in str(excinfo.value)
156
157
158		def test_extract_context_invalid_max_length():
159		"""Test context extraction with invalid max length."""
160		query = "python"
161		documents = ["Python is a programming language."]
162
163		with pytest.raises(ValueError) as excinfo:
164		extract_context(query, documents, max_length=0)
165
166		assert "Max length must be a positive integer" in str(excinfo.value)
167
168
169		def test_extract_context_with_permission_granted():
170		"""Test context extraction with permission granted."""
171		query = "python"
172		documents = ["Python is a programming language."]
173
174		# Create a mock permission manager
175		permission_manager = MagicMock()
176		permission_manager.request_permission.return_value = True
177
178		result = extract_context(query, documents, permission_manager=permission_manager)
179
180		assert "Python is a programming language" in result
181		permission_manager.request_permission.assert_called_once()
182
183
184		def test_extract_context_with_permission_denied():
185		"""Test context extraction with permission denied."""
186		query = "python"
187		documents = ["Python is a programming language."]
188
189		# Create a mock permission manager
190		permission_manager = MagicMock()
191		permission_manager.request_permission.return_value = False
192
193		with pytest.raises(PermissionError) as excinfo:
194		extract_context(query, documents, permission_manager=permission_manager)
195
196		assert "Permission denied" in str(excinfo.value)
197		permission_manager.request_permission.assert_called_once()
198
199
200		# Tests for summarize_text
201		def test_summarize_text_basic():
202		"""Test basic text summarization."""
203		text = """Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.
204		Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming.
205		Python is often described as a "batteries included" language due to its comprehensive standard library."""
206
207		result = summarize_text(text)
208
209		assert len(result) <= 500
210		assert "Python" in result
211		assert "programming" in result
212
213
214		def test_summarize_text_short_input():
215		"""Test summarization with text shorter than max_length."""
216		text = "Python is a programming language."
217
218		result = summarize_text(text)
219
220		assert result == text # Should return original text unchanged
221
222
223		def test_summarize_text_single_sentence():
224		"""Test summarization with a single sentence longer than max_length."""
225		text = "Python is a high-level, general-purpose programming language with design philosophy that emphasizes code readability and significant whitespace and a syntax that allows programmers to express concepts in fewer lines of code than might be possible in languages such as C++ or Java."
226		max_len = 50
227
228		result = summarize_text(text, max_length=max_len)
229
230		assert len(result) <= max_len + 3 # +3 for the "..."
231		assert result.endswith("...")
232
233
234		def test_summarize_text_empty_text():
235		"""Test summarization with empty text."""
236		text = ""
237
238		with pytest.raises(ValueError) as excinfo:
239		summarize_text(text)
240
241		assert "Text must be a non-empty string" in str(excinfo.value)
242
243
244		def test_summarize_text_invalid_max_length():
245		"""Test summarization with invalid max length."""
246		text = "Python is a programming language."
247
248		with pytest.raises(ValueError) as excinfo:
249		summarize_text(text, max_length=0)
250
251		assert "Max length must be a positive integer" in str(excinfo.value)
252
253
254		def test_summarize_text_with_permission_granted():
255		"""Test text summarization with permission granted."""
256		text = "Python is a programming language."
257
258		# Create a mock permission manager
259		permission_manager = MagicMock()
260		permission_manager.request_permission.return_value = True
261
262		result = summarize_text(text, permission_manager=permission_manager)
263
264		assert result == text # Short text should be returned as is
265		permission_manager.request_permission.assert_called_once()
266
267
268		def test_summarize_text_with_permission_denied():
269		"""Test text summarization with permission denied."""
270		text = "Python is a programming language."
271
272		# Create a mock permission manager
273		permission_manager = MagicMock()
274		permission_manager.request_permission.return_value = False
275
276		with pytest.raises(PermissionError) as excinfo:
277		summarize_text(text, permission_manager=permission_manager)
278
279		assert "Permission denied" in str(excinfo.value)
280		permission_manager.request_permission.assert_called_once()
281		"""
282		Tests for the documentation tools.
283		"""
284
285		import pytest
286		from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
287
288
289		# Tests for generate_markdown
290		def test_generate_markdown_with_markdown_format():
291		"""Test markdown generation with markdown format."""
292		content = "This is a test content.\n\nWith multiple paragraphs."
293		title = "Test Document"
294
295		result = generate_markdown(content, title, format_type='markdown')
296
297		assert result.startswith("# Test Document")
298		assert "This is a test content." in result
299		assert "With multiple paragraphs." in result
300
301
302		def test_generate_markdown_with_html_format():
303		"""Test markdown generation with HTML format."""
304		content = "This is a test content.\n\nWith multiple paragraphs."
305		title = "Test Document"
306
307		result = generate_markdown(content, title, format_type='html')
308
309		assert "<title>Test Document</title>" in result
310		assert "<h1>Test Document</h1>" in result
311		assert "This is a test content." in result
312		assert "With multiple paragraphs." in result
313		assert "<!DOCTYPE html>" in result
314
315
316		def test_generate_markdown_with_invalid_format():
317		"""Test markdown generation with an invalid format."""
318		content = "This is a test content."
319		title = "Test Document"
320
321		with pytest.raises(ValueError) as excinfo:
322		generate_markdown(content, title, format_type='invalid')
323
324		assert "Format type must be 'markdown' or 'html'" in str(excinfo.value)
325
326
327		def test_generate_markdown_with_empty_content():
328		"""Test markdown generation with empty content."""
329		content = ""
330		title = "Test Document"
331
332		with pytest.raises(ValueError) as excinfo:
333		generate_markdown(content, title)
334
335		assert "Content must be a non-empty string" in str(excinfo.value)
336
337
338		def test_generate_markdown_with_empty_title():
339		"""Test markdown generation with empty title."""
340		content = "This is a test content."
341		title = ""
342
343		with pytest.raises(ValueError) as excinfo:
344		generate_markdown(content, title)
345
346		assert "Title must be a non-empty string" in str(excinfo.value)
347
348
349		# Tests for extract_context
350		def test_extract_context_basic():
351		"""Test basic context extraction."""
352		query = "python programming"
353		documents = [
354		"Python is a programming language. It is widely used in data science.",
355		"JavaScript is a web programming language. It runs in browsers.",
356		"Data science uses statistical methods to analyze data."
357		]
358
359		result = extract_context(query, documents)
360
361		assert "Python is a programming language" in result
362		assert "JavaScript is a web programming language" in result
363		assert "Data science" not in result # No keyword match
364
365
366		def test_extract_context_with_max_length():
367		"""Test context extraction with max length limit."""
368		query = "programming language"
369		documents = [
370		"Python is a programming language. It is widely used in data science.",
371		"JavaScript is a web programming language. It runs in browsers.",
372		"Java is an object-oriented programming language. It is used for enterprise applications."
373		]
374
375		# Set max_length to fit only the first two sentences
376		max_len = len("Python is a programming language. JavaScript is a web programming language.") + 5
377		result = extract_context(query, documents, max_length=max_len)
378
379		assert "Python is a programming language" in result
380		assert "JavaScript is a web programming language" in result
381		assert "Java is an object-oriented programming language" not in result
382
383
384		def test_extract_context_empty_query():
385		"""Test context extraction with empty query."""
386		query = ""
387		documents = ["Python is a programming language."]
388
389		with pytest.raises(ValueError) as excinfo:
390		extract_context(query, documents)
391
392		assert "Query must be a non-empty string" in str(excinfo.value)
393
394
395		def test_extract_context_empty_documents():
396		"""Test context extraction with empty documents."""
397		query = "python"
398		documents = []
399
400		with pytest.raises(ValueError) as excinfo:
401		extract_context(query, documents)
402
403		assert "Documents must be a non-empty list of strings" in str(excinfo.value)
404
405
406		def test_extract_context_invalid_max_length():
407		"""Test context extraction with invalid max length."""
408		query = "python"
409		documents = ["Python is a programming language."]
410
411		with pytest.raises(ValueError) as excinfo:
412		extract_context(query, documents, max_length=0)
413
414		assert "Max length must be a positive integer" in str(excinfo.value)
415
416
417		# Tests for summarize_text
418		def test_summarize_text_basic():
419		"""Test basic text summarization."""
420		text = """Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.
421		Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming.
422		Python is often described as a "batteries included" language due to its comprehensive standard library."""
423
424		result = summarize_text(text)
425
426		assert len(result) <= 500
427		assert "Python" in result
428		assert "programming" in result
429
430
431		def test_summarize_text_short_input():
432		"""Test summarization with text shorter than max_length."""
433		text = "Python is a programming language."
434
435		result = summarize_text(text)
436
437		assert result == text # Should return original text unchanged
438
439
440		def test_summarize_text_single_sentence():
441		"""Test summarization with a single sentence longer than max_length."""
442		text = "Python is a high-level, general-purpose programming language with design philosophy that emphasizes code readability and significant whitespace and a syntax that allows programmers to express concepts in fewer lines of code than might be possible in languages such as C++ or Java."
443		max_len = 50
444
445		result = summarize_text(text, max_length=max_len)
446
447		assert len(result) <= max_len + 3 # +3 for the "..."
448		assert result.endswith("...")
449
450
451		def test_summarize_text_empty_text():
452		"""Test summarization with empty text."""
453		text = ""
454
455		with pytest.raises(ValueError) as excinfo:
456		summarize_text(text)
457
458		assert "Text must be a non-empty string" in str(excinfo.value)
459
460
461		def test_summarize_text_invalid_max_length():
462		"""Test summarization with invalid max length."""
463		text = "Python is a programming language."
464
465		with pytest.raises(ValueError) as excinfo:
466		summarize_text(text, max_length=0)
467
468		assert "Max length must be a positive integer" in str(excinfo.value)

ollamatools-project/tests/test_integration.py +0 -165

…	…	@@ -1,165 +0,0 @@
1		import os
2		import unittest
3		from unittest.mock import patch, MagicMock
4
5		import pytest
6
7		from ollamatools.config import Config
8		from ollamatools.security import PermissionManager
9		from ollamatools.agent import OllamaAgent
10		from ollamatools.tools.system_tools import execute_command, read_file, write_file
11		from ollamatools.tools.web_tools import fetch_url
12		from ollamatools.tools.doc_tools import generate_markdown
13
14
15		class TestIntegration(unittest.TestCase):
16		def setUp(self):
17		self.config = Config(
18		model="llama3.1",
19		host="http://localhost:11434",
20		permission_prompts=False, # Disable for testing
21		sudo_always_prompt=True,
22		timeout_seconds=10,
23		max_file_size=1024 * 1024
24		)
25		self.permission_manager = PermissionManager(
26		prompt_enabled=False, # Disable for testing
27		sudo_always_prompt=True
28		)
29
30		# Create a temp file for testing
31		self.test_file = "test_file.txt"
32		with open(self.test_file, "w") as f:
33		f.write("Test content")
34
35		def tearDown(self):
36		# Clean up the test file
37		if os.path.exists(self.test_file):
38		os.remove(self.test_file)
39
40		@patch("ollamatools.tools.system_tools.subprocess.run")
41		def test_system_tools_integration(self, mock_run):
42		# Mock subprocess.run
43		mock_process = MagicMock()
44		mock_process.stdout = "command output" # Changed to string instead of bytes
45		mock_process.stderr = ""
46		mock_process.returncode = 0
47		mock_run.return_value = mock_process
48
49		# Test execute_command with permission manager
50		result = execute_command("echo", ["hello"], permission_manager=self.permission_manager)
51
52		self.assertEqual(result["stdout"], "command output")
53		self.assertEqual(result["stderr"], "")
54		self.assertEqual(result["returncode"], 0)
55
56		# Test read_file with permission manager
57		content = read_file(self.test_file, permission_manager=self.permission_manager)
58		self.assertEqual(content, "Test content")
59
60		# Test write_file with permission manager
61		success = write_file(self.test_file, "New content", permission_manager=self.permission_manager)
62		self.assertTrue(success)
63
64		# Verify the file was written correctly
65		with open(self.test_file, "r") as f:
66		self.assertEqual(f.read(), "New content")
67
68		@patch("ollamatools.tools.web_tools.requests.request")
69		def test_web_tools_integration(self, mock_request):
70		# Mock requests.request
71		mock_response = MagicMock()
72		mock_response.status_code = 200
73		mock_response.headers = {"Content-Type": "text/html"}
74		mock_response.text = "<html><body>Test response</body></html>"
75		mock_request.return_value = mock_response
76
77		# Test fetch_url with permission manager
78		result = fetch_url(
79		"https://example.com",
80		permission_manager=self.permission_manager
81		)
82
83		self.assertEqual(result["status"], 200)
84		self.assertEqual(result["content"], "<html><body>Test response</body></html>")
85		self.assertEqual(result["headers"]["Content-Type"], "text/html")
86
87		def test_doc_tools_integration(self):
88		# Test generate_markdown with permission manager
89		result = generate_markdown(
90		"# Test\nThis is a test.",
91		"Test Document",
92		permission_manager=self.permission_manager
93		)
94
95		self.assertIn("# Test", result)
96		self.assertIn("This is a test.", result)
97
98		@patch("ollamatools.agent.ollama")
99		def test_agent_function_execution(self, mock_ollama):
100		# Override the chat method to return a simpler response for testing
101		original_chat = OllamaAgent.chat
102
103		def mock_chat_method(self, message, system_prompt=None, context=None):
104		# Mock function call detection
105		if "List files" in message:
106		# Execute the function ourselves since we're bypassing the normal flow
107		result = self.registered_functions["execute_command"](
108		command="ls",
109		args=["-la"],
110		permission_manager=self.permission_manager
111		)
112		# Return a combined response
113		return f"Here's the list of files:\n\n{result['stdout']}"
114		return original_chat(self, message, system_prompt, context)
115
116		try:
117		# Apply the patch
118		OllamaAgent.chat = mock_chat_method
119
120		# Create the agent
121		agent = OllamaAgent(
122		model="llama3.1",
123		permission_prompts=False
124		)
125
126		# Mock the execute_command function
127		agent.registered_functions["execute_command"] = MagicMock(return_value={
128		"stdout": "file1.txt\nfile2.txt",
129		"stderr": "",
130		"returncode": 0
131		})
132
133		# Call chat with a message that would trigger function execution
134		response = agent.chat("List files in the current directory")
135
136		# Verify the function was called
137		agent.registered_functions["execute_command"].assert_called_once()
138
139		# Verify the response includes the function result
140		self.assertIn("file1.txt", response)
141		self.assertIn("file2.txt", response)
142
143		finally:
144		# Restore the original method
145		OllamaAgent.chat = original_chat
146
147		def test_error_propagation(self):
148		# Test how errors propagate through the system
149		with patch("ollamatools.tools.system_tools.subprocess.run") as mock_run:
150		# Make subprocess.run raise an exception
151		mock_run.side_effect = Exception("Test exception")
152
153		# Execute command should catch and handle the exception
154		result = execute_command(
155		"test_command",
156		permission_manager=self.permission_manager
157		)
158
159		# Verify the error was properly handled
160		self.assertIn("error", result)
161		self.assertIn("Test exception", result["error"])
162
163
164		if __name__ == "__main__":
165		unittest.main()

ollamatools-project/tests/test_logger.py +0 -137

…	…	@@ -1,137 +0,0 @@
1		import os
2		import sys
3		import unittest
4		import tempfile
5		import logging
6		from io import StringIO
7
8		from ollamatools.utils.logger import (
9		setup_logger,
10		get_logger,
11		enable_debug_logging,
12		disable_logging,
13		get_ollamatools_logger
14		)
15
16
17		class TestLogger(unittest.TestCase):
18		def setUp(self):
19		# Reset the root logger
20		logging.root.handlers = []
21		# Reset the stored loggers
22		logging.Logger.manager.loggerDict.clear()
23
24		# Set up a stream for capturing log output
25		self.log_capture = StringIO()
26		self.stream_handler = logging.StreamHandler(self.log_capture)
27		self.stream_handler.setFormatter(logging.Formatter('%(levelname)s - %(message)s'))
28
29		# Set up a temporary file for log output
30		self.temp_file = tempfile.mktemp(suffix=".log")
31
32		def tearDown(self):
33		# Remove the temporary log file
34		if os.path.exists(self.temp_file):
35		os.remove(self.temp_file)
36
37		def test_setup_logger_default(self):
38		"""Test creating a logger with default settings."""
39		logger = setup_logger("test")
40
41		# Check the logger level
42		self.assertEqual(logger.level, logging.INFO)
43
44		# Check the number of handlers
45		self.assertEqual(len(logger.handlers), 1)
46
47		# Check the handler type
48		self.assertIsInstance(logger.handlers[0], logging.StreamHandler)
49
50		def test_setup_logger_custom_level(self):
51		"""Test creating a logger with a custom level."""
52		logger = setup_logger("test", level="DEBUG")
53
54		# Check the logger level
55		self.assertEqual(logger.level, logging.DEBUG)
56
57		def test_setup_logger_with_file(self):
58		"""Test creating a logger that writes to a file."""
59		logger = setup_logger("test", log_file=self.temp_file, console_output=False)
60
61		# Check the number of handlers
62		self.assertEqual(len(logger.handlers), 1)
63
64		# Check the handler type
65		self.assertIsInstance(logger.handlers[0], logging.FileHandler)
66
67		# Write a log message
68		logger.info("File test message")
69
70		# Check that the message was written to the file
71		with open(self.temp_file, "r") as f:
72		content = f.read()
73
74		self.assertIn("File test message", content)
75
76		def test_get_logger(self):
77		"""Test getting a logger."""
78		# Get a logger
79		logger1 = get_logger("test_get")
80
81		# Check the logger level
82		self.assertEqual(logger1.level, logging.INFO)
83
84		# Get the same logger again
85		logger2 = get_logger("test_get")
86
87		# Check that it's the same logger instance
88		self.assertIs(logger1, logger2)
89
90		def test_enable_debug_logging(self):
91		"""Test enabling debug logging."""
92		# Create a logger
93		logger = setup_logger("test_debug")
94
95		# Enable debug logging
96		enable_debug_logging("test_debug")
97
98		# Check the logger level
99		self.assertEqual(logger.level, logging.DEBUG)
100
101		def test_disable_logging(self):
102		"""Test disabling logging."""
103		# Create a logger
104		logger = setup_logger("test_disable")
105
106		# Disable logging
107		disable_logging("test_disable")
108
109		# Check the logger level
110		self.assertEqual(logger.level, logging.CRITICAL)
111
112		def test_get_ollamatools_logger(self):
113		"""Test getting the main OllamaTools logger."""
114		# Get the OllamaTools logger
115		logger = get_ollamatools_logger()
116
117		# Check the logger name
118		self.assertEqual(logger.name, "ollamatools")
119
120		def test_log_to_custom_handler(self):
121		"""Test logging to a custom handler."""
122		# Create a logger
123		logger = setup_logger("test_custom")
124
125		# Add our custom handler
126		logger.addHandler(self.stream_handler)
127
128		# Write a log message
129		logger.info("Custom handler test message")
130
131		# Check that the message was captured
132		log_output = self.log_capture.getvalue()
133		self.assertIn("INFO - Custom handler test message", log_output)
134
135
136		if __name__ == "__main__":
137		unittest.main()

ollamatools-project/tests/test_matrix_tools.py +0 -272

…	…	@@ -1,272 +0,0 @@
1		"""
2		Tests for the Matrix integration tools.
3
4		These tests verify that the Matrix integration tools work correctly.
5		They use mocking to avoid actual network connections.
6		"""
7
8		import os
9		import sys
10		import pytest
11		import asyncio
12		from unittest.mock import MagicMock, patch, AsyncMock
13
14		# Add the parent directory to the path so we can import ollamatools
15		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
16
17		from ollamatools.tools.matrix_tools import (
18		MatrixClient,
19		MatrixConfig,
20		connect_to_matrix,
21		join_matrix_room,
22		send_matrix_message,
23		load_matrix_config,
24		save_matrix_config,
25		create_default_matrix_config
26		)
27
28
29		class TestMatrixConfig:
30		"""Tests for the MatrixConfig class."""
31
32		def test_create_default_config(self):
33		"""Test creating a default configuration."""
34		config = create_default_matrix_config()
35
36		assert 'MATRIX' in config
37		assert 'OLLAMA' in config
38		assert 'HOMESERVER' in config['MATRIX']
39		assert 'USER' in config['MATRIX']
40		assert 'PASSWORD' in config['MATRIX']
41		assert 'DEVICE_ID' in config['MATRIX']
42		assert 'ROOMS' in config['MATRIX']
43		assert 'HOST' in config['OLLAMA']
44		assert 'MODEL' in config['OLLAMA']
45		assert 'PROMPT' in config['OLLAMA']
46
47		def test_save_and_load_config(self, tmp_path):
48		"""Test saving and loading a configuration."""
49		config_path = tmp_path / "test_config.yaml"
50		config = create_default_matrix_config()
51
52		# Save the configuration
53		save_matrix_config(config, str(config_path))
54
55		# Check that the file exists
56		assert config_path.exists()
57
58		# Load the configuration
59		loaded_config = load_matrix_config(str(config_path))
60
61		# Check that the loaded configuration matches the original
62		assert loaded_config == config
63
64
65		@pytest.mark.asyncio
66		class TestMatrixClient:
67		"""Tests for the MatrixClient class."""
68
69		@patch('ollamatools.tools.matrix_tools.AsyncClient')
70		async def test_connect(self, mock_async_client):
71		"""Test connecting to a Matrix homeserver."""
72		# Set up the mock
73		mock_client = AsyncMock()
74		mock_async_client.return_value = mock_client
75		mock_client.login.return_value = MagicMock(spec=[''])
76
77		# Create a client
78		client = MatrixClient(
79		homeserver="https://matrix.example.org",
80		user_id="@user:example.org",
81		password="password",
82		device_id="device_id"
83		)
84
85		# Connect
86		success = await client.connect()
87
88		# Check that the login method was called
89		mock_client.login.assert_called_once_with(
90		password="password"
91		)
92
93		# Check that the connection was successful
94		assert success is True
95
96		@patch('ollamatools.tools.matrix_tools.AsyncClient')
97		async def test_join_room(self, mock_async_client):
98		"""Test joining a Matrix room."""
99		# Set up the mock
100		mock_client = AsyncMock()
101		mock_async_client.return_value = mock_client
102		mock_client.join.return_value = MagicMock(spec=[''])
103
104		# Create a client
105		client = MatrixClient(
106		homeserver="https://matrix.example.org",
107		user_id="@user:example.org",
108		password="password",
109		device_id="device_id"
110		)
111
112		# Join a room
113		success = await client.join_room("#room:example.org")
114
115		# Check that the join method was called
116		mock_client.join.assert_called_once_with("#room:example.org")
117
118		# Check that joining was successful
119		assert success is True
120
121		# Check that the room was added to the list of joined rooms
122		assert "#room:example.org" in client.rooms
123
124		@patch('ollamatools.tools.matrix_tools.AsyncClient')
125		async def test_send_message(self, mock_async_client):
126		"""Test sending a message to a Matrix room."""
127		# Set up the mock
128		mock_client = AsyncMock()
129		mock_async_client.return_value = mock_client
130		mock_client.room_send.return_value = MagicMock(spec=[''])
131
132		# Create a client
133		client = MatrixClient(
134		homeserver="https://matrix.example.org",
135		user_id="@user:example.org",
136		password="password",
137		device_id="device_id"
138		)
139
140		# Send a message
141		success = await client.send_message("#room:example.org", "Hello, world!")
142
143		# Check that the room_send method was called
144		mock_client.room_send.assert_called_once_with(
145		room_id="#room:example.org",
146		message_type="m.room.message",
147		content={
148		"msgtype": "m.text",
149		"body": "Hello, world!"
150		}
151		)
152
153		# Check that sending was successful
154		assert success is True
155
156		@patch('ollamatools.tools.matrix_tools.OllamaAsync')
157		async def test_process_with_ollama(self, mock_ollama_async):
158		"""Test processing a message with Ollama."""
159		# Set up the mock
160		mock_ollama = AsyncMock()
161		mock_ollama_async.return_value = mock_ollama
162		mock_ollama.chat.return_value = {
163		"message": {
164		"content": "Hello, I'm Ollama!"
165		}
166		}
167
168		# Create a client
169		client = MatrixClient(
170		homeserver="https://matrix.example.org",
171		user_id="@user:example.org",
172		password="password",
173		device_id="device_id",
174		ollama_host="localhost:11434",
175		ollama_model="llama3:latest",
176		ollama_prompt="You are a helpful assistant."
177		)
178
179		# Process a message
180		response = await client.process_with_ollama("Hello, Ollama!")
181
182		# Check that the chat method was called
183		mock_ollama.chat.assert_called_once_with(
184		model="llama3:latest",
185		messages=[
186		{
187		"role": "system",
188		"content": "You are a helpful assistant."
189		},
190		{
191		"role": "user",
192		"content": "Hello, Ollama!"
193		}
194		]
195		)
196
197		# Check that the response is correct
198		assert response == "Hello, I'm Ollama!"
199
200
201		@pytest.mark.asyncio
202		class TestMatrixToolsFunctions:
203		"""Tests for the Matrix tools functions."""
204
205		@patch('ollamatools.tools.matrix_tools.MatrixClient.connect')
206		async def test_connect_to_matrix(self, mock_connect):
207		"""Test the connect_to_matrix function."""
208		# Set up the mock
209		mock_connect.return_value = True
210
211		# Connect to Matrix
212		with patch('ollamatools.tools.matrix_tools.asyncio.run', side_effect=lambda x: asyncio.create_task(x)):
213		client = await connect_to_matrix(
214		homeserver="https://matrix.example.org",
215		user_id="@user:example.org",
216		password="password",
217		device_id="device_id"
218		)
219
220		# Check that the connect method was called
221		mock_connect.assert_called_once()
222
223		# Check that a client was returned
224		assert client is not None
225
226		@patch('ollamatools.tools.matrix_tools.MatrixClient.join_room')
227		async def test_join_matrix_room(self, mock_join_room):
228		"""Test the join_matrix_room function."""
229		# Set up the mock
230		mock_join_room.return_value = True
231
232		# Create a client
233		client = MatrixClient(
234		homeserver="https://matrix.example.org",
235		user_id="@user:example.org",
236		password="password",
237		device_id="device_id"
238		)
239
240		# Join a room
241		with patch('ollamatools.tools.matrix_tools.asyncio.run', side_effect=lambda x: asyncio.create_task(x)):
242		success = await join_matrix_room(client, "#room:example.org")
243
244		# Check that the join_room method was called
245		mock_join_room.assert_called_once_with("#room:example.org")
246
247		# Check that joining was successful
248		assert success is True
249
250		@patch('ollamatools.tools.matrix_tools.MatrixClient.send_message')
251		async def test_send_matrix_message(self, mock_send_message):
252		"""Test the send_matrix_message function."""
253		# Set up the mock
254		mock_send_message.return_value = True
255
256		# Create a client
257		client = MatrixClient(
258		homeserver="https://matrix.example.org",
259		user_id="@user:example.org",
260		password="password",
261		device_id="device_id"
262		)
263
264		# Send a message
265		with patch('ollamatools.tools.matrix_tools.asyncio.run', side_effect=lambda x: asyncio.create_task(x)):
266		success = await send_matrix_message(client, "#room:example.org", "Hello, world!")
267
268		# Check that the send_message method was called
269		mock_send_message.assert_called_once_with("#room:example.org", "Hello, world!")
270
271		# Check that sending was successful
272		assert success is True

ollamatools-project/tests/test_natural_language_search.py +0 -175

…	…	@@ -1,175 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Test script for natural language search understanding and function execution.
4
5		This script tests the ModularOllamaAgent's ability to:
6		1. Detect when a natural language query requires search
7		2. Properly execute function calls returned by the LLM
8		3. Handle search results correctly
9
10		Usage:
11		python -m tests.test_natural_language_search
12		"""
13
14		import os
15		import sys
16		import json
17		import unittest
18		from unittest.mock import patch, MagicMock
19
20		# Add the parent directory to the Python path to import ollamatools
21		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
22
23		# Import the modular agent
24		from ollamatools.agent import ModularOllamaAgent
25		from ollamatools.agent.modular.search import detect_factual_intent, extract_search_context, execute_search
26
27		class TestNaturalLanguageSearch(unittest.TestCase):
28		"""Test cases for natural language search understanding."""
29
30		def setUp(self):
31		"""Set up test environment."""
32		# Create a mock agent with mocked dependencies
33		self.agent = ModularOllamaAgent(
34		model="llama3.1:latest",
35		permission_prompts=False
36		)
37
38		# Mock the underlying OllamaAgent to avoid actual API calls
39		self.agent.agent = MagicMock()
40
41		# Sample search results for testing
42		self.sample_search_results = {
43		"query": "recipe for chilli sauce",
44		"results": [
45		{
46		"title": "Homemade Chili Sauce Recipe",
47		"url": "https://example.com/chili-sauce",
48		"content": "A delicious homemade chili sauce recipe with tomatoes and spices.",
49		"engine": "google"
50		},
51		{
52		"title": "Spicy Chili Sauce - Quick and Easy",
53		"url": "https://example.com/spicy-chili",
54		"content": "Make this quick and easy spicy chili sauce in just 15 minutes.",
55		"engine": "bing"
56		}
57		],
58		"answer": "Chili sauce typically contains chili peppers, vinegar, sugar, and spices."
59		}
60
61		def test_detect_factual_intent(self):
62		"""Test the detection of factual intent in user messages."""
63		# Messages that should be detected as needing search
64		search_messages = [
65		"What is the recipe for chilli sauce?",
66		"How do I make homemade pasta?",
67		"Tell me about the history of the Roman Empire",
68		"What's the weather like in New York today?",
69		"Who is the current president of France?"
70		]
71
72		# Messages that should not trigger search
73		non_search_messages = [
74		"Hello, how are you?",
75		"Thank you for your help",
76		"Can you help me with this code?",
77		"What can you do?",
78		"Please continue"
79		]
80
81		# Test messages that should trigger search
82		for message in search_messages:
83		with self.subTest(message=message):
84		self.assertTrue(
85		detect_factual_intent(message),
86		f"Failed to detect search intent in: {message}"
87		)
88
89		# Test messages that should not trigger search
90		for message in non_search_messages:
91		with self.subTest(message=message):
92		self.assertFalse(
93		detect_factual_intent(message),
94		f"Incorrectly detected search intent in: {message}"
95		)
96
97		def test_extract_search_context(self):
98		"""Test extraction of context from search results."""
99		context = extract_search_context(self.sample_search_results)
100
101		# Verify that the context contains the direct answer
102		self.assertIn("Chili sauce typically contains", context)
103
104		# Verify that the context contains information from search results
105		self.assertIn("Homemade Chili Sauce Recipe", context)
106		self.assertIn("Spicy Chili Sauce - Quick and Easy", context)
107
108		@patch('ollamatools.agent.modular.search.execute_search')
109		def test_natural_language_search_trigger(self, mock_execute_search):
110		"""Test that natural language queries trigger search appropriately."""
111		# Mock the execute_search function to return sample results
112		mock_execute_search.return_value = self.sample_search_results
113
114		# Mock the agent.chat method to return a normal response
115		self.agent.agent.chat.return_value = "Here's information about chili sauce recipes."
116
117		# Test with a message that should trigger search
118		response = self.agent.chat("What's a good recipe for chilli sauce?")
119
120		# Verify that execute_search was called
121		mock_execute_search.assert_called_once()
122
123		# Verify that the search query was passed correctly
124		self.assertEqual(
125		mock_execute_search.call_args[0][0],
126		"What's a good recipe for chilli sauce?"
127		)
128
129		@patch('ollamatools.agent.modular.agent.json.loads')
130		def test_function_call_detection(self, mock_json_loads):
131		"""Test detection and execution of function calls in LLM responses."""
132		# Mock the agent.chat method to return a function call
133		function_call_json = {
134		"function": {
135		"name": "opensearch_query",
136		"arguments": json.dumps({"query": "recipe for chilli sauce"})
137		}
138		}
139
140		# Set up the mock to return our function call data
141		mock_json_loads.return_value = function_call_json
142
143		# Mock the agent's chat method to return a JSON string
144		self.agent.agent.chat.return_value = json.dumps(function_call_json)
145
146		# Mock execute_search to return sample results
147		self.agent.execute_search = MagicMock(return_value=self.sample_search_results)
148
149		# Test with a message
150		response = self.agent.chat("Can you give me a recipe for chilli sauce?")
151
152		# Verify that execute_search was called with the right query
153		self.agent.execute_search.assert_called_once_with("recipe for chilli sauce")
154
155		# Verify that the response contains search results and not raw JSON
156		self.assertNotIn('"function":', response)
157		self.assertIn("Here's what I found about", response)
158
159		@patch('ollamatools.agent.modular.agent.json.loads')
160		def test_malformed_function_call_handling(self, mock_json_loads):
161		"""Test handling of malformed function calls."""
162		# Mock json.loads to raise JSONDecodeError
163		mock_json_loads.side_effect = json.JSONDecodeError("Expecting property name", "{", 1)
164
165		# Mock the agent's chat method to return something that looks like JSON but isn't valid
166		self.agent.agent.chat.return_value = "{function: {name: opensearch_query, arguments: {query: 'recipe'}}}"
167
168		# Test with a message
169		response = self.agent.chat("Can you give me a recipe for chilli sauce?")
170
171		# Verify that the response is still returned even if JSON parsing fails
172		self.assertEqual(response, "{function: {name: opensearch_query, arguments: {query: 'recipe'}}}")
173
174		if __name__ == "__main__":
175		unittest.main()

ollamatools-project/tests/test_ollama_integration.py +0 -85

…	…	@@ -1,85 +0,0 @@
1		#!/usr/bin/env python3
2		"""
3		Integration test script for the refactored OllamaAgent with Ollama's API.
4
5		This script tests the integration between the refactored OllamaAgent
6		and Ollama's function calling API. It verifies that the agent can
7		properly register tools with Ollama and execute function calls.
8
9		Usage:
10		python test_ollama_integration.py
11		"""
12
13		import os
14		import sys
15		import json
16		import logging
17		from typing import Dict, Any, List, Optional
18
19		# Add the parent directory to the path
20		parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
21		if parent_dir not in sys.path:
22		sys.path.insert(0, parent_dir)
23
24		from ollamatools import OllamaAgent, Config
25		from ollamatools.security import PermissionManager
26
27		# Configure logging
28		logging.basicConfig(level=logging.INFO,
29		format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
30		logger = logging.getLogger(__name__)
31
32		def main():
33		"""Run the integration test."""
34		# Test with permission prompts disabled for automation
35		config = Config(
36		model="llama3:latest", # Use your available model
37		host="http://localhost:11434",
38		permission_prompts=False,
39		sudo_always_prompt=False
40		)
41
42		try:
43		# 1. Test agent initialization
44		logger.info("Testing agent initialization...")
45		agent = OllamaAgent(config=config)
46		logger.info("Agent initialized successfully")
47
48		# 2. Test basic chat without function calls
49		logger.info("Testing basic chat without function calls...")
50		response = agent.chat("Hello, what's your name?")
51		logger.info(f"Response: {response}")
52
53		# 3. Test chat that should trigger system commands
54		logger.info("Testing chat that should trigger system commands...")
55		response = agent.chat("What files are in the current directory?")
56		logger.info(f"Response: {response}")
57
58		# 4. Test chat that should trigger web requests
59		logger.info("Testing chat that should trigger web requests...")
60		response = agent.chat("What's the weather like in New York?")
61		logger.info(f"Response: {response}")
62
63		# 5. Test chat that should trigger documentation tools
64		logger.info("Testing chat that should trigger documentation tools...")
65		response = agent.chat("Create a markdown document about Python programming.")
66		logger.info(f"Response: {response}")
67
68		# 6. Test chat with conversation context
69		logger.info("Testing chat with conversation context...")
70		context = [
71		{"role": "user", "content": "I want to learn about file operations."},
72		{"role": "assistant", "content": "I'd be happy to help you learn about file operations in various programming languages. What specific aspect are you interested in?"}
73		]
74		response = agent.chat("Can you write a Python function to count lines in a file?", context=context)
75		logger.info(f"Response: {response}")
76
77		logger.info("All tests completed successfully!")
78		return 0
79
80		except Exception as e:
81		logger.error(f"Error during integration test: {str(e)}")
82		return 1
83
84		if __name__ == "__main__":
85		sys.exit(main())

ollamatools-project/tests/test_performance.py +0 -138

…	…	@@ -1,138 +0,0 @@
1		import os
2		import time
3		import unittest
4		import tempfile
5		import random
6		import string
7		from unittest.mock import patch, MagicMock
8
9		import pytest
10
11		from ollamatools.config import Config
12		from ollamatools.security import PermissionManager
13		from ollamatools.agent import OllamaAgent
14		from ollamatools.tools.system_tools import execute_command, read_file, write_file, find_files
15
16
17		class TestPerformance(unittest.TestCase):
18		def setUp(self):
19		self.permission_manager = PermissionManager(
20		prompt_enabled=False # Disable for testing
21		)
22
23		# Create temp directory for file operations
24		self.temp_dir = tempfile.mkdtemp()
25
26		# Create a number of test files
27		self.num_files = 10 # Reduced for faster testing
28		self.test_files = []
29
30		for i in range(self.num_files):
31		filename = os.path.join(self.temp_dir, f"test_file_{i}.txt")
32		content = ''.join(random.choices(string.ascii_letters + string.digits, k=1000))
33		with open(filename, "w") as f:
34		f.write(content)
35		self.test_files.append(filename)
36
37		def tearDown(self):
38		# Clean up the test files
39		for filename in self.test_files:
40		if os.path.exists(filename):
41		os.remove(filename)
42
43		# Remove the temp directory
44		if os.path.exists(self.temp_dir):
45		os.rmdir(self.temp_dir)
46
47		def test_file_operations_performance(self):
48		"""Test the performance of file operations."""
49		# Measure time to read all test files
50		start_time = time.time()
51
52		for filename in self.test_files:
53		content = read_file(
54		path=filename,
55		permission_manager=self.permission_manager
56		)
57		self.assertIsNotNone(content)
58
59		read_duration = time.time() - start_time
60		print(f"Read {self.num_files} files in {read_duration:.2f} seconds")
61
62		# Measure time to write all test files
63		start_time = time.time()
64
65		for i, filename in enumerate(self.test_files):
66		content = f"New content for file {i}"
67		success = write_file(
68		path=filename,
69		content=content,
70		permission_manager=self.permission_manager
71		)
72		self.assertTrue(success)
73
74		write_duration = time.time() - start_time
75		print(f"Wrote {self.num_files} files in {write_duration:.2f} seconds")
76
77		# Measure time to find all test files
78		start_time = time.time()
79
80		found_files = find_files(
81		pattern="*.txt",
82		directory=self.temp_dir,
83		recursive=True,
84		permission_manager=self.permission_manager
85		)
86
87		find_duration = time.time() - start_time
88		print(f"Found {len(found_files)} files in {find_duration:.2f} seconds")
89
90		# Assert all files were found
91		self.assertEqual(len(found_files), self.num_files)
92
93		# Assert reasonable performance
94		# These thresholds are arbitrary and may need adjustment
95		self.assertLess(read_duration, 2.0, f"Reading {self.num_files} files took too long")
96		self.assertLess(write_duration, 2.0, f"Writing {self.num_files} files took too long")
97		self.assertLess(find_duration, 1.0, f"Finding {self.num_files} files took too long")
98
99		@patch("ollamatools.tools.system_tools.subprocess.run")
100		def test_command_execution_performance(self, mock_run):
101		"""Test the performance of command execution."""
102		# Mock subprocess.run to return quickly
103		mock_process = MagicMock()
104		mock_process.stdout = "command output" # Using string instead of bytes
105		mock_process.stderr = ""
106		mock_process.returncode = 0
107		mock_run.return_value = mock_process
108
109		# Execute a command multiple times and measure performance
110		num_executions = 10 # Reduced for faster testing
111		start_time = time.time()
112
113		for i in range(num_executions):
114		result = execute_command(
115		command="echo",
116		args=[f"test {i}"],
117		permission_manager=self.permission_manager
118		)
119		self.assertEqual(result["returncode"], 0)
120
121		duration = time.time() - start_time
122		print(f"Executed {num_executions} commands in {duration:.2f} seconds")
123
124		# Assert reasonable performance
125		# This threshold is arbitrary and may need adjustment
126		self.assertLess(duration, 2.0, f"Executing {num_executions} commands took too long")
127
128		# Test the throughput
129		throughput = num_executions / duration
130		print(f"Command execution throughput: {throughput:.2f} commands/second")
131
132		# Assert reasonable throughput
133		# This threshold is arbitrary and may need adjustment
134		self.assertGreater(throughput, 5.0, f"Command execution throughput is too low")
135
136
137		if __name__ == "__main__":
138		unittest.main()

ollamatools-project/tests/test_plugin_system.py +0 -287

…	…	@@ -1,287 +0,0 @@
1		"""
2		Tests for the OllamaTools plugin system.
3
4		This module contains tests for the core plugin system infrastructure.
5		"""
6
7		import os
8		import sys
9		import unittest
10		import tempfile
11		from unittest.mock import MagicMock, patch
12
13		# Add the parent directory to the Python path
14		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
15
16		from ollamatools.plugins import OllamaToolsPlugin, SharedPlugin, MatrixPlugin, AgentPlugin
17		from ollamatools.plugins.registry import PluginRegistry
18		from ollamatools.plugins.errors import (
19		PluginError,
20		PluginNotFoundError,
21		PluginInitializationError,
22		PluginDependencyError
23		)
24		from ollamatools.config import PluginConfig, load_plugin_config, create_default_config
25
26
27		class TestPlugin(SharedPlugin):
28		"""Test plugin for unit tests."""
29
30		def __init__(self, config=None):
31		super().__init__(config)
32		self.initialized = False
33		self.shutdown_called = False
34
35		def initialize(self, context):
36		self.initialized = True
37		self.context = context
38
39		def shutdown(self):
40		self.shutdown_called = True
41
42		@property
43		def version(self):
44		return "1.0.0-test"
45
46		def get_capabilities(self):
47		return ["test_capability"]
48
49
50		class DependentPlugin(SharedPlugin):
51		"""Plugin that depends on TestPlugin."""
52
53		@property
54		def dependencies(self):
55		return ["TestPlugin"]
56
57
58		class FailingPlugin(SharedPlugin):
59		"""Plugin that fails during initialization."""
60
61		def initialize(self, context):
62		raise RuntimeError("Initialization failure")
63
64
65		class TestPluginSystem(unittest.TestCase):
66		"""Test cases for the plugin system."""
67
68		def setUp(self):
69		"""Set up test fixtures."""
70		self.registry = PluginRegistry()
71		self.context = {"test_key": "test_value"}
72
73		def test_plugin_registration(self):
74		"""Test plugin registration."""
75		self.registry.register_plugin(TestPlugin)
76		self.assertIn("TestPlugin", self.registry.plugins)
77
78		plugin_info = self.registry.plugins["TestPlugin"]
79		self.assertEqual(plugin_info["class"], TestPlugin)
80		self.assertIsNone(plugin_info["config"])
81
82		def test_plugin_initialization(self):
83		"""Test plugin initialization."""
84		self.registry.register_plugin(TestPlugin)
85		plugin = self.registry.initialize_plugin("TestPlugin", self.context)
86
87		self.assertIsInstance(plugin, TestPlugin)
88		self.assertTrue(plugin.initialized)
89		self.assertEqual(plugin.context, self.context)
90		self.assertIn("TestPlugin", self.registry.initialized_plugins)
91
92		def test_plugin_dependency(self):
93		"""Test plugin dependency resolution."""
94		self.registry.register_plugin(TestPlugin)
95		self.registry.register_plugin(DependentPlugin)
96
97		# Initialize the dependent plugin
98		plugin = self.registry.initialize_plugin("DependentPlugin", self.context)
99
100		# Both plugins should be initialized
101		self.assertIn("TestPlugin", self.registry.initialized_plugins)
102		self.assertIn("DependentPlugin", self.registry.initialized_plugins)
103
104		def test_plugin_not_found(self):
105		"""Test handling of non-existent plugins."""
106		with self.assertRaises(PluginNotFoundError):
107		self.registry.initialize_plugin("NonExistentPlugin", self.context)
108
109		def test_plugin_initialization_error(self):
110		"""Test handling of plugin initialization errors."""
111		self.registry.register_plugin(FailingPlugin)
112
113		with self.assertRaises(PluginInitializationError):
114		self.registry.initialize_plugin("FailingPlugin", self.context)
115
116		def test_plugin_dependency_error(self):
117		"""Test handling of missing dependencies."""
118		self.registry.register_plugin(DependentPlugin)
119
120		with self.assertRaises(PluginDependencyError):
121		self.registry.initialize_plugin("DependentPlugin", self.context)
122
123		def test_get_plugin(self):
124		"""Test retrieving an initialized plugin."""
125		self.registry.register_plugin(TestPlugin)
126		self.registry.initialize_plugin("TestPlugin", self.context)
127
128		plugin = self.registry.get_plugin("TestPlugin")
129		self.assertIsInstance(plugin, TestPlugin)
130
131		# Non-existent plugin should return None
132		self.assertIsNone(self.registry.get_plugin("NonExistentPlugin"))
133
134		def test_get_plugins_by_type(self):
135		"""Test retrieving plugins by type."""
136		class AnotherTestPlugin(SharedPlugin):
137		pass
138
139		class TestMatrixPlugin(MatrixPlugin):
140		pass
141
142		self.registry.register_plugin(TestPlugin)
143		self.registry.register_plugin(AnotherTestPlugin)
144		self.registry.register_plugin(TestMatrixPlugin)
145
146		self.registry.initialize_all(self.context)
147
148		shared_plugins = self.registry.get_plugins_by_type(SharedPlugin)
149		self.assertEqual(len(shared_plugins), 2)
150		self.assertIn("TestPlugin", shared_plugins)
151		self.assertIn("AnotherTestPlugin", shared_plugins)
152
153		matrix_plugins = self.registry.get_plugins_by_type(MatrixPlugin)
154		self.assertEqual(len(matrix_plugins), 1)
155		self.assertIn("TestMatrixPlugin", matrix_plugins)
156
157		def test_shutdown(self):
158		"""Test plugin shutdown."""
159		self.registry.register_plugin(TestPlugin)
160		plugin = self.registry.initialize_plugin("TestPlugin", self.context)
161
162		self.registry.shutdown()
163
164		self.assertTrue(plugin.shutdown_called)
165		self.assertEqual(len(self.registry.initialized_plugins), 0)
166
167
168		class TestPluginConfig(unittest.TestCase):
169		"""Test cases for the plugin configuration system."""
170
171		def test_config_loading_from_dict(self):
172		"""Test loading configuration from a dictionary."""
173		config_dict = {"key1": "value1", "key2": 42}
174		config = PluginConfig(config_dict)
175
176		self.assertEqual(config.get("key1"), "value1")
177		self.assertEqual(config.get("key2"), 42)
178		self.assertIsNone(config.get("non_existent_key"))
179		self.assertEqual(config.get("non_existent_key", "default"), "default")
180
181		def test_config_validation(self):
182		"""Test configuration validation."""
183		schema = {
184		"required_key": {
185		"type": str,
186		"required": True
187		},
188		"optional_key": {
189		"type": int,
190		"required": False
191		},
192		"enum_key": {
193		"type": str,
194		"enum": ["option1", "option2"]
195		}
196		}
197
198		# Valid configuration
199		valid_config = {
200		"required_key": "value",
201		"optional_key": 42,
202		"enum_key": "option1"
203		}
204
205		config = PluginConfig(valid_config, schema)
206		self.assertTrue(config.validate())
207
208		# Missing required key
209		invalid_config = {
210		"optional_key": 42
211		}
212
213		with self.assertRaises(PluginError):
214		PluginConfig(invalid_config, schema)
215
216		# Invalid type
217		invalid_config = {
218		"required_key": "value",
219		"optional_key": "not_an_int"
220		}
221
222		with self.assertRaises(PluginError):
223		PluginConfig(invalid_config, schema)
224
225		# Invalid enum value
226		invalid_config = {
227		"required_key": "value",
228		"enum_key": "invalid_option"
229		}
230
231		with self.assertRaises(PluginError):
232		PluginConfig(invalid_config, schema)
233
234		def test_config_save_and_load(self):
235		"""Test saving and loading configuration to/from a file."""
236		config_dict = {
237		"string_key": "value",
238		"int_key": 42,
239		"bool_key": True,
240		"list_key": [1, 2, 3],
241		"dict_key": {"nested": "value"}
242		}
243
244		config = PluginConfig(config_dict)
245
246		# Save to a temporary file
247		with tempfile.NamedTemporaryFile(suffix=".yaml", delete=False) as temp_file:
248		temp_path = temp_file.name
249
250		try:
251		config.save(temp_path)
252
253		# Load the saved configuration
254		loaded_config = PluginConfig(temp_path)
255
256		# Check that the loaded configuration matches the original
257		self.assertEqual(loaded_config.get("string_key"), "value")
258		self.assertEqual(loaded_config.get("int_key"), 42)
259		self.assertEqual(loaded_config.get("bool_key"), True)
260		self.assertEqual(loaded_config.get("list_key"), [1, 2, 3])
261		self.assertEqual(loaded_config.get("dict_key"), {"nested": "value"})
262		finally:
263		# Clean up
264		if os.path.exists(temp_path):
265		os.unlink(temp_path)
266
267		def test_default_config(self):
268		"""Test creating default configuration."""
269		# Test shared plugin config
270		shared_config = create_default_config("TestPlugin", "shared")
271		self.assertTrue(shared_config["enabled"])
272		self.assertEqual(shared_config["name"], "TestPlugin")
273
274		# Test matrix plugin config
275		matrix_config = create_default_config("TestPlugin", "matrix")
276		self.assertTrue(matrix_config["enabled"])
277		self.assertTrue(matrix_config["commands_enabled"])
278		self.assertEqual(matrix_config["command_prefix"], "/")
279
280		# Test agent plugin config
281		agent_config = create_default_config("TestPlugin", "agent")
282		self.assertTrue(agent_config["enabled"])
283		self.assertTrue(agent_config["functions_enabled"])
284
285
286		if __name__ == "__main__":
287		unittest.main()

ollamatools-project/tests/test_security.py +0 -11

…	…	@@ -1,11 +0,0 @@
1		"""
2		Tests for the permission management system.
3		"""
4		import pytest
5		import sys
6		import os
7
8		# Add the parent directory to the path so we can import the package
9		sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
10
11		from ollamatools.security import PermissionManager

ollamatools-project/tests/test_system_tools.py +0 -260

…	…	@@ -1,260 +0,0 @@
1		"""
2		System tools for OllamaTools.
3		This module provides functions for executing system commands and working with files.
4		"""
5
6		import os
7		import glob
8		import subprocess
9		import logging
10		import re
11		from typing import List, Optional, Dict, Any, Union
12
13		# Set up logging
14		logger = logging.getLogger(__name__)
15
16		def _sanitize_command(command: str) -> str:
17		"""
18		Sanitize a command string to prevent command injection.
19
20		Args:
21		command: Command string to sanitize
22
23		Returns:
24		str: Sanitized command string
25		"""
26		# Basic sanitization to prevent shell metacharacters
27		# This is a simple version - a complete implementation would be more complex
28		sanitized = str(command).strip()
29
30		# Remove shell metacharacters
31		dangerous_chars = r'[;&|`$()"\'\\]'
32		sanitized = re.sub(dangerous_chars, '', sanitized)
33
34		return sanitized
35
36		def execute_command(
37		command: str,
38		args: Optional[List[str]] = None,
39		timeout: int = 60,
40		permission_manager = None
41		) -> Dict[str, Any]:
42		"""
43		Execute a shell command with arguments.
44
45		Args:
46		command: Command to execute
47		args: Command arguments
48		timeout: Command timeout in seconds
49		permission_manager: Permission manager instance
50
51		Returns:
52		dict: Result with stdout, stderr, and returncode
53		"""
54		# Default args to empty list if None
55		if args is None:
56		args = []
57
58		# Sanitize the command and arguments
59		command = _sanitize_command(command)
60		args = [_sanitize_command(arg) for arg in args]
61
62		# Check for sudo
63		is_sudo = False
64		if command == "sudo" or command.startswith("sudo "):
65		is_sudo = True
66		logger.warning(f"Sudo command detected: {command} {' '.join(args)}")
67
68		# Check permission if permission manager is provided
69		if permission_manager:
70		function_args = {"command": command, "args": args, "timeout": timeout}
71		permission_granted = permission_manager.request_permission(
72		function_name="execute_command",
73		args=function_args,
74		is_sudo=is_sudo
75		)
76
77		if not permission_granted:
78		logger.warning(f"Permission denied for command: {command} {' '.join(args)}")
79		return {"error": "Permission denied"}
80
81		# Construct the full command
82		full_command = [command] + args
83		logger.info(f"Executing command: {' '.join(full_command)}")
84
85		try:
86		# Execute the command
87		result = subprocess.run(
88		full_command,
89		capture_output=True,
90		check=False,
91		text=True,
92		timeout=timeout
93		)
94
95		# Return the result
96		return {
97		"stdout": result.stdout,
98		"stderr": result.stderr,
99		"returncode": result.returncode
100		}
101		except subprocess.TimeoutExpired:
102		logger.error(f"Command timed out after {timeout} seconds: {' '.join(full_command)}")
103		return {
104		"error": f"Command timed out after {timeout} seconds"
105		}
106		except Exception as e:
107		logger.error(f"Error executing command: {str(e)}")
108		return {
109		"error": str(e)
110		}
111
112		def read_file(
113		path: str,
114		max_size: Optional[int] = None,
115		permission_manager = None
116		) -> str:
117		"""
118		Read file contents.
119
120		Args:
121		path: File path
122		max_size: Maximum file size to read in bytes
123		permission_manager: Permission manager instance
124
125		Returns:
126		str: File contents
127		"""
128		# Sanitize the path
129		path = os.path.abspath(path)
130
131		# Check permission if permission manager is provided
132		if permission_manager:
133		function_args = {"path": path, "max_size": max_size}
134		permission_granted = permission_manager.request_permission(
135		function_name="read_file",
136		args=function_args
137		)
138
139		if not permission_granted:
140		logger.warning(f"Permission denied for reading file: {path}")
141		return "Permission denied"
142
143		try:
144		# Check if the file exists
145		if not os.path.isfile(path):
146		logger.error(f"File not found: {path}")
147		return f"File not found: {path}"
148
149		# Check the file size if max_size is specified
150		if max_size is not None:
151		file_size = os.path.getsize(path)
152		if file_size > max_size:
153		logger.error(f"File size ({file_size} bytes) exceeds maximum size ({max_size} bytes): {path}")
154		return f"File size ({file_size} bytes) exceeds maximum size ({max_size} bytes)"
155
156		# Read the file
157		with open(path, "r") as f:
158		content = f.read()
159
160		return content
161		except Exception as e:
162		logger.error(f"Error reading file: {str(e)}")
163		return f"Error reading file: {str(e)}"
164
165		def write_file(
166		path: str,
167		content: str,
168		mode: str = 'w',
169		permission_manager = None
170		) -> bool:
171		"""
172		Write content to a file.
173
174		Args:
175		path: File path
176		content: Content to write
177		mode: Write mode ('w' for write, 'a' for append)
178		permission_manager: Permission manager instance
179
180		Returns:
181		bool: True if successful
182		"""
183		# Sanitize the path and mode
184		path = os.path.abspath(path)
185		if mode not in ['w', 'a']:
186		logger.error(f"Invalid mode: {mode}")
187		return False
188
189		# Check permission if permission manager is provided
190		if permission_manager:
191		function_args = {"path": path, "content": content, "mode": mode}
192		permission_granted = permission_manager.request_permission(
193		function_name="write_file",
194		args=function_args
195		)
196
197		if not permission_granted:
198		logger.warning(f"Permission denied for writing to file: {path}")
199		return False
200
201		try:
202		# Create the directory if it doesn't exist
203		os.makedirs(os.path.dirname(path), exist_ok=True)
204
205		# Write to the file
206		with open(path, mode) as f:
207		f.write(content)
208
209		return True
210		except Exception as e:
211		logger.error(f"Error writing to file: {str(e)}")
212		return False
213
214		def find_files(
215		pattern: str,
216		directory: str = '.',
217		recursive: bool = True,
218		permission_manager = None
219		) -> List[str]:
220		"""
221		Find files matching a pattern.
222
223		Args:
224		pattern: File pattern
225		directory: Search directory
226		recursive: Whether to search recursively
227		permission_manager: Permission manager instance
228
229		Returns:
230		List[str]: Matching file paths
231		"""
232		# Sanitize the directory
233		directory = os.path.abspath(directory)
234
235		# Check permission if permission manager is provided
236		if permission_manager:
237		function_args = {"pattern": pattern, "directory": directory, "recursive": recursive}
238		permission_granted = permission_manager.request_permission(
239		function_name="find_files",
240		args=function_args
241		)
242
243		if not permission_granted:
244		logger.warning(f"Permission denied for finding files in: {directory}")
245		return []
246
247		try:
248		# Construct the search pattern
249		search_pattern = os.path.join(directory, '**', pattern) if recursive else os.path.join(directory, pattern)
250
251		# Find files
252		files = glob.glob(search_pattern, recursive=recursive)
253
254		# Convert to absolute paths
255		files = [os.path.abspath(file) for file in files]
256
257		return files
258		except Exception as e:
259		logger.error(f"Error finding files: {str(e)}")
260		return []

ollamatools-project/tests/test_web_tools.py +0 -127

…	…	@@ -1,127 +0,0 @@
1		"""
2		Tests for web tools module.
3		"""
4
5		import pytest
6		from unittest.mock import MagicMock, patch
7		import requests
8		from ollamatools.tools.web_tools import fetch_url
9
10		# Test successful GET request
11		@patch('requests.request')
12		def test_successful_get_request(mock_request):
13		# Setup mock response
14		mock_response = MagicMock()
15		mock_response.status_code = 200
16		mock_response.headers = {'Content-Type': 'application/json'}
17		mock_response.json.return_value = {'key': 'value'}
18		mock_response.url = 'https://example.com'
19		mock_request.return_value = mock_response
20
21		# Call function
22		result = fetch_url('https://example.com')
23
24		# Check result
25		assert result['status'] == 200
26		assert result['content'] == {'key': 'value'}
27		assert 'headers' in result
28		assert result['url'] == 'https://example.com'
29
30		# Check mock was called correctly
31		mock_request.assert_called_once_with(
32		method='GET',
33		url='https://example.com',
34		params=None,
35		headers=None,
36		json=None,
37		timeout=30
38		)
39
40		# Test successful POST request with data
41		@patch('requests.request')
42		def test_post_request_with_data(mock_request):
43		# Setup mock response
44		mock_response = MagicMock()
45		mock_response.status_code = 201
46		mock_response.headers = {'Content-Type': 'application/json'}
47		mock_response.json.return_value = {'id': '123'}
48		mock_response.url = 'https://example.com/api'
49		mock_request.return_value = mock_response
50
51		# Test data
52		data = {'name': 'test', 'value': 42}
53
54		# Call function
55		result = fetch_url(
56		'https://example.com/api',
57		method='POST',
58		data=data
59		)
60
61		# Check result
62		assert result['status'] == 201
63		assert result['content'] == {'id': '123'}
64
65		# Check mock was called correctly
66		mock_request.assert_called_once_with(
67		method='POST',
68		url='https://example.com/api',
69		params=None,
70		headers=None,
71		json=data,
72		timeout=30
73		)
74
75		# Test request with invalid URL
76		def test_invalid_url():
77		with pytest.raises(ValueError):
78		fetch_url('')
79
80		with pytest.raises(ValueError):
81		fetch_url('not-a-url')
82
83		# Test request with invalid method
84		def test_invalid_method():
85		with pytest.raises(ValueError):
86		fetch_url('https://example.com', method='INVALID')
87
88		# Test request timeout
89		@patch('requests.request')
90		def test_request_timeout(mock_request):
91		mock_request.side_effect = requests.exceptions.Timeout("Request timed out")
92
93		with pytest.raises(requests.exceptions.Timeout):
94		fetch_url('https://example.com', timeout=5)
95
96		# Test permission denied
97		def test_permission_denied():
98		# Setup mock permission manager
99		mock_permission_manager = MagicMock()
100		mock_permission_manager.request_permission.return_value = False
101
102		with pytest.raises(PermissionError):
103		fetch_url(
104		'https://example.com',
105		permission_manager=mock_permission_manager
106		)
107
108		# Check permission was requested
109		mock_permission_manager.request_permission.assert_called_once()
110
111		# Test handling non-JSON content
112		@patch('requests.request')
113		def test_non_json_content(mock_request):
114		# Setup mock response
115		mock_response = MagicMock()
116		mock_response.status_code = 200
117		mock_response.headers = {'Content-Type': 'text/html'}
118		mock_response.text = '<html><body>Hello World</body></html>'
119		mock_response.url = 'https://example.com'
120		mock_request.return_value = mock_response
121
122		# Call function
123		result = fetch_url('https://example.com')
124
125		# Check result
126		assert result['status'] == 200
127		assert result['content'] == '<html><body>Hello World</body></html>'

ollamatools-project/tree-struct +0 -1

…	…	@@ -1 +0,0 @@
1		tree -L 3 # Limit depth to 3 levels

ollamatools-project/update_mapping_status.sh +0 -72

…	…	@@ -1,72 +0,0 @@
1		#!/bin/bash
2
3		# Script to update the status of a file in the file mapping
4		# This helps track progress of the refactoring process
5
6		# Define the file mapping
7		MAPPING_FILE="file_mapping.md"
8
9		# Function to display usage information
10		usage() {
11		echo "Usage: $0 <original_file> <status> [notes]"
12		echo ""
13		echo "Arguments:"
14		echo " <original_file> Path to the original file in the codebase (relative to the project root)"
15		echo " <status> New status (e.g., 'Completed', 'In Progress', 'Pending')"
16		echo " [notes] Optional notes to add or replace existing notes"
17		echo ""
18		echo "Example:"
19		echo " $0 ollamatools/agent.py Completed 'Refactored with updated imports'"
20		echo ""
21		echo "This will update the status of ollamatools/agent.py to 'Completed' and add the notes."
22		exit 1
23		}
24
25		# Check if the correct number of arguments is provided
26		if [ $# -lt 2 ]; then
27		usage
28		fi
29
30		ORIGINAL_FILE="$1"
31		NEW_STATUS="$2"
32		NEW_NOTES="${3:-}"
33
34		# Check if the file mapping exists
35		if [ ! -f "$MAPPING_FILE" ]; then
36		echo "Error: File mapping $MAPPING_FILE does not exist."
37		echo "Run ./generate_file_mapping.sh first to create it."
38		exit 1
39		fi
40
41		# Escape special characters in the original file path for use in sed
42		ESCAPED_FILE=$(echo "$ORIGINAL_FILE" | sed 's/\//\\\//g')
43
44		# Check if the file exists in the mapping
45		if ! grep -q "\`$ORIGINAL_FILE\`" "$MAPPING_FILE"; then
46		echo "Error: File $ORIGINAL_FILE not found in the mapping."
47		echo "Available files:"
48		grep -o "\`[^']*\`" "$MAPPING_FILE" | grep -v "New File" | sort | uniq
49		exit 1
50		fi
51
52		# If notes are provided, update both status and notes
53		if [ -n "$NEW_NOTES" ]; then
54		# Escape special characters in the notes for use in sed
55		ESCAPED_NOTES=$(echo "$NEW_NOTES" | sed 's/\//\\\//g')
56
57		# Update the status and notes
58		sed -i "s/| \`$ESCAPED_FILE\` | \`[^']*\` | [^|]* | [^|]* |/| \`$ESCAPED_FILE\` | \`&\` | $NEW_STATUS | $ESCAPED_NOTES |/" "$MAPPING_FILE"
59		else
60		# Update only the status
61		sed -i "s/| \`$ESCAPED_FILE\` | \`[^']*\` | [^|]* |/| \`$ESCAPED_FILE\` | \`&\` | $NEW_STATUS |/" "$MAPPING_FILE"
62		fi
63
64		echo "Updated status of $ORIGINAL_FILE to '$NEW_STATUS'."
65		if [ -n "$NEW_NOTES" ]; then
66		echo "Updated notes to: $NEW_NOTES"
67		fi
68
69		# Display the updated entry
70		echo ""
71		echo "Updated entry:"
72		grep "\`$ORIGINAL_FILE\`" "$MAPPING_FILE"

ollamatools-project/url_validator.py +0 -321

…	…	@@ -1,321 +0,0 @@
1		"""
2		URL validation module for OllamaTools.
3
4		This module provides functions for validating and sanitizing URLs
5		to prevent security vulnerabilities like SSRF attacks and URL injection.
6		"""
7
8		import re
9		import ipaddress
10		from urllib.parse import urlparse, parse_qs, urlencode, urlunparse, unquote
11		import logging
12		from typing import Dict, Any, List, Set, Optional, Tuple, Union
13
14		# Set up logging
15		logger = logging.getLogger(__name__)
16
17		# Default allowed schemes
18		DEFAULT_ALLOWED_SCHEMES = {'http', 'https'}
19
20		# Default blocklist of domains
21		DEFAULT_DOMAIN_BLOCKLIST = {
22		'localhost', '127.0.0.1', '0.0.0.0', '::1', # Localhost variations
23		'169.254.169.254', # AWS metadata endpoint
24		'169.254.170.2', # AWS ECS metadata endpoint
25		'169.254.170.23', # AWS Lambda metadata endpoint
26		'10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16', # Private IP ranges
27		'fc00::/7', # Unique local addresses (IPv6)
28		'fd00::/8', # Unique local addresses (IPv6)
29		'fe80::/10', # Link-local addresses (IPv6)
30		}
31
32		# Default allowlist of domains (overrides blocklist)
33		DEFAULT_DOMAIN_ALLOWLIST = set()
34
35		# Suspicious URL patterns
36		SUSPICIOUS_URL_PATTERNS = [
37		r'(file|ftp|gopher|data|telnet|ldap|ssh)://', # Non-HTTP protocols
38		r'@', # URL with credentials
39		r'\.\.', # Directory traversal
40		r'<|>|\{|\}|\[|\]|\|', # Special characters
41		]
42
43		class URLValidationError(Exception):
44		"""Exception raised for URL validation errors."""
45		pass
46
47
48		def normalize_url(url: str) -> str:
49		"""
50		Normalize a URL to prevent bypass techniques.
51
52		Args:
53		url: URL to normalize
54
55		Returns:
56		str: Normalized URL
57		"""
58		# Handle empty URLs
59		if not url:
60		raise URLValidationError("URL cannot be empty")
61
62		# Ensure URL is a string
63		url = str(url).strip()
64
65		# Remove control characters and whitespace
66		url = re.sub(r'[\x00-\x20\x7F]', '', url)
67
68		try:
69		# Parse the URL
70		parsed = urlparse(url)
71
72		# Check if scheme and netloc exist
73		if not parsed.scheme or not parsed.netloc:
74		raise URLValidationError(f"Invalid URL: {url} - missing scheme or host")
75
76		# Decode any URL-encoded components that might be hiding malicious content
77		path = unquote(parsed.path)
78
79		# Normalize path to remove traversal sequences
80		path = normalize_path(path)
81
82		# Parse and rebuild query parameters to normalize
83		query_params = parse_qs(parsed.query, keep_blank_values=True)
84		normalized_query = urlencode(query_params, doseq=True)
85
86		# Rebuild the normalized URL
87		normalized_url = urlunparse((
88		parsed.scheme.lower(), # Normalize scheme to lowercase
89		parsed.netloc.lower(), # Normalize domain to lowercase
90		path,
91		parsed.params,
92		normalized_query,
93		'' # Remove fragment as it's client-side only
94		))
95
96		return normalized_url
97
98		except Exception as e:
99		raise URLValidationError(f"URL normalization error: {str(e)}")
100
101
102		def normalize_path(path: str) -> str:
103		"""
104		Normalize a URL path to remove traversal sequences and other issues.
105
106		Args:
107		path: URL path to normalize
108
109		Returns:
110		str: Normalized path
111		"""
112		# Remove consecutive slashes
113		path = re.sub(r'/+', '/', path)
114
115		# Remove trailing slash for consistency unless the path is just "/"
116		if path != '/' and path.endswith('/'):
117		path = path[:-1]
118
119		# Process path components to resolve traversal sequences
120		# This is complex, so we'll do it manually
121		components = path.split('/')
122		normalized_components = []
123
124		for component in components:
125		if component == '' or component == '.':
126		# Skip empty components and current directory references
127		continue
128		elif component == '..':
129		# Go up one directory level
130		if normalized_components:
131		normalized_components.pop()
132		else:
133		normalized_components.append(component)
134
135		# Rebuild the path
136		normalized_path = '/' + '/'.join(normalized_components)
137
138		# Ensure path starts with "/"
139		if not normalized_path.startswith('/'):
140		normalized_path = '/' + normalized_path
141
142		return normalized_path
143
144
145		def is_ip_address_in_blocklist(ip_str: str, blocklist: Set[str]) -> bool:
146		"""
147		Check if an IP address is in the blocklist.
148
149		Args:
150		ip_str: IP address string
151		blocklist: Set of blocked IP addresses or CIDR ranges
152
153		Returns:
154		bool: True if the IP is blocked, False otherwise
155		"""
156		try:
157		# Parse the IP address
158		ip = ipaddress.ip_address(ip_str)
159
160		# Check if the IP is directly in the blocklist
161		if ip_str in blocklist:
162		return True
163
164		# Check if the IP is in any of the blocklisted CIDR ranges
165		for block in blocklist:
166		if '/' in block: # It's a CIDR block
167		try:
168		network = ipaddress.ip_network(block, strict=False)
169		if ip in network:
170		return True
171		except ValueError:
172		continue
173
174		return False
175		except ValueError:
176		# Not a valid IP address
177		return False
178
179
180		def validate_url(
181		url: str,
182		allowed_schemes: Optional[Set[str]] = None,
183		domain_allowlist: Optional[Set[str]] = None,
184		domain_blocklist: Optional[Set[str]] = None,
185		) -> str:
186		"""
187		Validate a URL against security rules.
188
189		Args:
190		url: URL to validate
191		allowed_schemes: Optional set of allowed URL schemes (defaults to http, https)
192		domain_allowlist: Optional set of allowed domains
193		domain_blocklist: Optional set of blocked domains
194
195		Returns:
196		str: Validated and normalized URL
197
198		Raises:
199		URLValidationError: If the URL does not pass validation
200		"""
201		# Use default values if not provided
202		if allowed_schemes is None:
203		allowed_schemes = DEFAULT_ALLOWED_SCHEMES
204
205		if domain_blocklist is None:
206		domain_blocklist = DEFAULT_DOMAIN_BLOCKLIST
207
208		if domain_allowlist is None:
209		domain_allowlist = DEFAULT_DOMAIN_ALLOWLIST
210
211		# Normalize the URL
212		try:
213		normalized_url = normalize_url(url)
214		except URLValidationError as e:
215		raise URLValidationError(f"URL normalization failed: {str(e)}")
216
217		# Parse the normalized URL
218		parsed = urlparse(normalized_url)
219
220		# Check scheme
221		if parsed.scheme not in allowed_schemes:
222		raise URLValidationError(
223		f"URL scheme '{parsed.scheme}' not allowed. Allowed schemes: {allowed_schemes}"
224		)
225
226		# Extract hostname
227		hostname = parsed.netloc.split(':')[0].lower()
228
229		# Check allowlist first (override blocklist)
230		if domain_allowlist and hostname in domain_allowlist:
231		return normalized_url
232
233		# Check hostname against blocklist
234		if hostname in domain_blocklist:
235		raise URLValidationError(f"Domain '{hostname}' is blocked")
236
237		# Check for IP address
238		try:
239		ip = ipaddress.ip_address(hostname)
240
241		# Check if IP is in blocklist
242		if is_ip_address_in_blocklist(str(ip), domain_blocklist):
243		raise URLValidationError(f"IP address '{ip}' is blocked")
244
245		# Check for localhost/internal IP
246		if ip.is_loopback or ip.is_private or ip.is_reserved or ip.is_link_local:
247		raise URLValidationError(f"IP address '{ip}' is internal/reserved")
248		except ValueError:
249		# Not an IP address, continue with domain checks
250		pass
251
252		# Check for suspicious patterns
253		for pattern in SUSPICIOUS_URL_PATTERNS:
254		if re.search(pattern, normalized_url):
255		raise URLValidationError(f"URL contains suspicious pattern: {pattern}")
256
257		# Check port if specified
258		if ':' in parsed.netloc:
259		try:
260		port = int(parsed.netloc.split(':')[1])
261		if port < 1 or port > 65535:
262		raise URLValidationError(f"Invalid port number: {port}")
263		except (IndexError, ValueError):
264		raise URLValidationError(f"Invalid port specification in URL: {parsed.netloc}")
265
266		return normalized_url
267
268
269		def configure_domain_lists(
270		additional_allowlist: Optional[Set[str]] = None,
271		additional_blocklist: Optional[Set[str]] = None
272		) -> Tuple[Set[str], Set[str]]:
273		"""
274		Configure domain allowlist and blocklist by adding custom entries.
275
276		Args:
277		additional_allowlist: Additional domains to allow
278		additional_blocklist: Additional domains to block
279
280		Returns:
281		tuple: (configured_allowlist, configured_blocklist)
282		"""
283		# Start with default lists
284		allowlist = DEFAULT_DOMAIN_ALLOWLIST.copy()
285		blocklist = DEFAULT_DOMAIN_BLOCKLIST.copy()
286
287		# Add additional entries
288		if additional_allowlist:
289		allowlist.update(additional_allowlist)
290
291		if additional_blocklist:
292		blocklist.update(additional_blocklist)
293
294		return allowlist, blocklist
295
296
297		def create_url_validator(
298		allowed_schemes: Optional[Set[str]] = None,
299		domain_allowlist: Optional[Set[str]] = None,
300		domain_blocklist: Optional[Set[str]] = None,
301		) -> callable:
302		"""
303		Create a URL validator function with the specified configuration.
304
305		Args:
306		allowed_schemes: Set of allowed URL schemes
307		domain_allowlist: Set of allowed domains
308		domain_blocklist: Set of blocked domains
309
310		Returns:
311		callable: URL validator function
312		"""
313		def validate(url: str) -> str:
314		return validate_url(
315		url,
316		allowed_schemes=allowed_schemes,
317		domain_allowlist=domain_allowlist,
318		domain_blocklist=domain_blocklist
319		)
320
321		return validate