OverviewFilesCommitsIssuesDocsReleasesPeers

ollama_semantic_function_calling commits / c3be0c6

Add simplification plan and prompt templates for OllamaTools refactoring
c3be0c6db293354a1b7afb5df1d76a4ffc9483e2
by fleeky.ashatron <fleeky@7rnx.net> · 2025-04-19T01:27:15.000Z

Changed files (3) +814 -0

ollamatools-project/SIMPLIFICATION_PLAN.md +394 -0
……@@ -0,0 +1,394 @@
1# OllamaTools Simplification Plan
2
3This document outlines a comprehensive plan for simplifying and modularizing the OllamaTools project. It serves as a guide for both human developers and AI assistants working on the codebase.
4
5## Current State Analysis
6
7The OllamaTools project has evolved through several phases, resulting in a complex architecture with multiple overlapping components:
8
91. **Core Agent Functionality**: Basic Ollama agent with semantic function calling
102. **Modular Agent**: A more structured approach with separate modules for search, commands, issues, etc.
113. **Extended Agent**: Builds on the modular agent with conversation history and analytics
124. **Matrix Integration**: Connects to Matrix chat rooms with basic and enhanced clients
135. **Plugin System**: A unified architecture for extending functionality
14
15This evolution has led to redundancy, complexity, and maintenance challenges.
16
17## Simplification Goals
18
191. **Reduce Redundancy**: Eliminate duplicate code and consolidate similar functionality
202. **Improve Modularity**: Create a clear, plugin-based architecture
213. **Simplify Interfaces**: Provide consistent, well-documented interfaces
224. **Enhance Maintainability**: Make the codebase easier to understand and modify
235. **Preserve Functionality**: Maintain all current features and capabilities
246. **Streamline Testing**: Improve testability through better separation of concerns
25
26## Target Architecture
27
28```
29ollamatools/
30├── core/ # Core functionality
31├── plugins/ # Plugin system and implementations
32│ ├── standard/ # Standard plugins
33│ ├── tools/ # Tool plugins
34│ └── integrations/ # Integration plugins
35├── agents/ # Agent implementations
36├── config/ # Configuration system
37└── cli/ # Command-line interface
38```
39
40## Detailed Component Breakdown
41
42### 1. Core Layer
43
44The Core Layer provides essential functionality that all other components depend on.
45
46**Files:**
47```
48ollamatools/core/
49├── __init__.py # Core module initialization
50├── agent.py # Base agent implementation
51├── config.py # Configuration management
52├── errors.py # Error definitions and handling
53├── security.py # Security utilities and permission system
54└── utils/
55 ├── __init__.py
56 ├── logger.py # Logging utilities
57 ├── validators.py # Input validation (combines command/path/url validators)
58 └── formatting.py # Text formatting utilities
59```
60
61**Key Responsibilities:**
62- Provide base agent functionality
63- Handle configuration management
64- Define error types and handling
65- Implement security features
66- Provide utility functions
67
68### 2. Plugin System
69
70The Plugin System enables extending functionality through plugins.
71
72**Files:**
73```
74ollamatools/plugins/
75├── __init__.py # Plugin system initialization
76├── base.py # Base plugin interfaces
77├── registry.py # Plugin discovery and management
78└── errors.py # Plugin-specific error handling
79```
80
81**Key Responsibilities:**
82- Define plugin interfaces
83- Manage plugin lifecycle
84- Handle plugin discovery and registration
85- Resolve plugin dependencies
86
87### 3. Standard Plugins
88
89Standard Plugins implement common functionality used by different agents.
90
91**Files:**
92```
93ollamatools/plugins/standard/
94├── __init__.py
95├── search.py # Search functionality
96├── commands.py # Command management
97├── conversation/
98│ ├── __init__.py
99│ ├── history.py # Conversation history tracking
100│ ├── context.py # Conversation context management
101│ └── analysis.py # Topic analysis and metrics
102├── feedback.py # Feedback collection
103├── issues.py # Issue tracking
104└── function_calling.py # Semantic function calling
105```
106
107**Key Responsibilities:**
108- Implement search functionality
109- Manage commands
110- Track conversation history and context
111- Collect and analyze feedback
112- Track issues
113- Handle semantic function calling
114
115### 4. Tool Plugins
116
117Tool Plugins provide specific tools for agents to use.
118
119**Files:**
120```
121ollamatools/plugins/tools/
122├── __init__.py
123├── system.py # System tools (file operations, commands)
124├── web.py # Web tools (HTTP requests, web searches)
125└── doc.py # Documentation tools
126```
127
128**Key Responsibilities:**
129- Execute system commands
130- Perform file operations
131- Make web requests
132- Generate documentation
133
134### 5. Integration Plugins
135
136Integration Plugins connect OllamaTools to external systems.
137
138**Files:**
139```
140ollamatools/plugins/integrations/
141├── __init__.py
142├── matrix/
143│ ├── __init__.py
144│ ├── client.py # Matrix client core
145│ ├── commands.py # Matrix-specific commands
146│ ├── config.py # Matrix configuration
147│ ├── search.py # Matrix search integration
148│ ├── history.py # Matrix history tracking
149│ └── rate_limit.py # Rate limiting for Matrix
150└── opensearch/
151 ├── __init__.py
152 ├── client.py # OpenSearch client
153 └── tools.py # OpenSearch web tools
154```
155
156**Key Responsibilities:**
157- Connect to Matrix chat rooms
158- Process Matrix messages
159- Integrate with OpenSearch
160- Handle rate limiting
161
162### 6. Agent Implementations
163
164Agent Implementations provide ready-to-use agents with different feature sets.
165
166**Files:**
167```
168ollamatools/agents/
169├── __init__.py # Agent module initialization
170├── basic.py # Basic agent implementation
171├── modular.py # Modular agent implementation
172└── extended.py # Extended agent implementation
173```
174
175**Key Responsibilities:**
176- Implement different agent types
177- Compose functionality from plugins
178- Provide user-friendly interfaces
179
180### 7. Configuration System
181
182The Configuration System handles loading, validating, and providing access to configuration.
183
184**Files:**
185```
186ollamatools/config/
187├── __init__.py # Configuration module initialization
188├── schemas.py # Configuration schemas
189├── loader.py # Configuration loading/validation
190└── defaults.py # Default configurations
191```
192
193**Key Responsibilities:**
194- Define configuration schemas
195- Load and validate configuration
196- Provide default configurations
197- Handle configuration merging
198
199### 8. Command-Line Interface
200
201The Command-Line Interface provides a way to interact with OllamaTools from the command line.
202
203**Files:**
204```
205ollamatools/cli/
206├── __init__.py # CLI module initialization
207└── main.py # Command-line interface
208```
209
210**Key Responsibilities:**
211- Parse command-line arguments
212- Execute commands
213- Provide interactive mode
214- Handle configuration
215
216## Implementation Phases
217
218### Phase 1: Core Infrastructure (Weeks 1-2)
219
220**Goals:**
221- Implement the core layer
222- Create the plugin system
223- Set up the configuration system
224
225**Tasks:**
2261. Create the directory structure
2272. Implement the base agent class
2283. Implement the plugin system
2294. Implement the configuration system
2305. Write tests for core components
231
232### Phase 2: Standard Plugins (Weeks 3-4)
233
234**Goals:**
235- Implement standard plugins
236- Extract common functionality from existing code
237- Ensure backward compatibility
238
239**Tasks:**
2401. Implement search plugin
2412. Implement commands plugin
2423. Implement conversation plugins
2434. Implement feedback plugin
2445. Implement issues plugin
2456. Implement function calling plugin
2467. Write tests for standard plugins
247
248### Phase 3: Tool Plugins (Weeks 5-6)
249
250**Goals:**
251- Implement tool plugins
252- Ensure security and permission handling
253- Provide comprehensive documentation
254
255**Tasks:**
2561. Implement system tools plugin
2572. Implement web tools plugin
2583. Implement documentation tools plugin
2594. Write tests for tool plugins
2605. Document tool usage and security considerations
261
262### Phase 4: Integration Plugins (Weeks 7-8)
263
264**Goals:**
265- Implement integration plugins
266- Ensure proper error handling
267- Provide configuration examples
268
269**Tasks:**
2701. Implement Matrix integration plugins
2712. Implement OpenSearch integration plugins
2723. Write tests for integration plugins
2734. Create example configurations
2745. Document integration setup and usage
275
276### Phase 5: Agent Implementations (Weeks 9-10)
277
278**Goals:**
279- Implement agent classes
280- Ensure feature parity with existing agents
281- Provide migration guides
282
283**Tasks:**
2841. Implement basic agent
2852. Implement modular agent
2863. Implement extended agent
2874. Write tests for agent implementations
2885. Create migration guides for existing users
289
290### Phase 6: Documentation and Examples (Weeks 11-12)
291
292**Goals:**
293- Provide comprehensive documentation
294- Create example applications
295- Ensure smooth migration path
296
297**Tasks:**
2981. Update README and documentation
2992. Create example applications
3003. Write migration guides
3014. Create tutorials and guides
3025. Update API documentation
303
304## Migration Strategy
305
306### For Users
307
3081. **Basic Agent Users:**
309 - Update imports to use new package structure
310 - Minimal configuration changes required
311
3122. **Modular Agent Users:**
313 - Update imports to use new package structure
314 - Adapt to new plugin system
315 - Follow migration guide for configuration changes
316
3173. **Extended Agent Users:**
318 - Update imports to use new package structure
319 - Adapt to new plugin system
320 - Follow migration guide for configuration changes
321
3224. **Matrix Integration Users:**
323 - Update imports to use new package structure
324 - Adapt to new integration plugins
325 - Follow migration guide for configuration changes
326
327### For Developers
328
3291. **Plugin Developers:**
330 - Learn new plugin system
331 - Update plugins to use new interfaces
332 - Follow plugin development guide
333
3342. **Agent Developers:**
335 - Learn new agent architecture
336 - Update agents to use plugin system
337 - Follow agent development guide
338
3393. **Integration Developers:**
340 - Learn new integration architecture
341 - Update integrations to use plugin system
342 - Follow integration development guide
343
344## Testing Strategy
345
3461. **Unit Tests:**
347 - Test each component in isolation
348 - Use mock objects for dependencies
349 - Ensure high code coverage
350
3512. **Integration Tests:**
352 - Test interactions between components
353 - Verify plugin loading and initialization
354 - Test configuration loading and validation
355
3563. **End-to-End Tests:**
357 - Test complete workflows
358 - Verify backward compatibility
359 - Test with real external systems (Matrix, OpenSearch)
360
3614. **Performance Tests:**
362 - Measure performance impact
363 - Identify bottlenecks
364 - Optimize critical paths
365
366## Documentation Strategy
367
3681. **API Documentation:**
369 - Document all public interfaces
370 - Provide examples for common use cases
371 - Include type hints and docstrings
372
3732. **User Guides:**
374 - Create guides for different user types
375 - Provide step-by-step instructions
376 - Include configuration examples
377
3783. **Developer Guides:**
379 - Create guides for plugin development
380 - Document architecture and design decisions
381 - Provide best practices
382
3834. **Migration Guides:**
384 - Create guides for migrating from old to new architecture
385 - Provide code examples for common migration scenarios
386 - Include troubleshooting information
387
388## Conclusion
389
390This simplification plan provides a comprehensive roadmap for transforming the OllamaTools project into a more modular, maintainable, and extensible system. By following this plan, we can reduce redundancy, improve code quality, and enhance the developer and user experience while preserving all existing functionality.
391
392The plan is designed to be implemented incrementally, allowing for continuous testing and validation throughout the process. Each phase builds on the previous one, gradually transforming the codebase into the target architecture.
393
394By the end of this process, OllamaTools will be a more cohesive, well-structured project that is easier to understand, maintain, and extend.
ollamatools-project/SIMPLIFICATION_PROMPT_TEMPLATE.md +335 -0
……@@ -0,0 +1,335 @@
1# OllamaTools Simplification Prompt Template
2
3This document provides a template for prompts that can be used by both humans and AI assistants when working on the OllamaTools simplification project. It's designed to be used alongside the more detailed `SIMPLIFICATION_PLAN.md`.
4
5## General Prompt Structure
6
7When working on the OllamaTools simplification project, use the following prompt structure to ensure consistent and effective implementation:
8
9```
10# Task: [Brief description of the specific task]
11
12## Context
13- Current component: [Specify which component you're working on]
14- Related files: [List relevant files]
15- Dependencies: [List dependencies]
16- Reference: [Link to relevant section in SIMPLIFICATION_PLAN.md]
17
18## Objective
19[Clear statement of what needs to be accomplished]
20
21## Requirements
22- [Requirement 1]
23- [Requirement 2]
24- ...
25
26## Constraints
27- [Constraint 1]
28- [Constraint 2]
29- ...
30
31## Expected Output
32[Description of the expected result]
33
34## Additional Notes
35[Any additional information that might be helpful]
36```
37
38## Phase-Specific Prompt Templates
39
40### Phase 1: Core Infrastructure
41
42```
43# Task: Implement Core Component
44
45## Context
46- Current component: Core Layer - [Specific component]
47- Related files: [List relevant files]
48- Dependencies: [List dependencies]
49- Reference: SIMPLIFICATION_PLAN.md - Core Layer
50
51## Objective
52Implement the [specific component] of the Core Layer according to the simplification plan.
53
54## Requirements
55- Follow the simplified architecture design
56- Ensure backward compatibility where possible
57- Implement clear interfaces
58- Include comprehensive docstrings
59- Write unit tests
60
61## Constraints
62- Minimize external dependencies
63- Keep the core layer focused and minimal
64- Ensure security best practices
65
66## Expected Output
67- Implemented [specific component] with tests
68- Documentation for public interfaces
69- Example usage
70
71## Additional Notes
72This component will be used by multiple other components, so clarity and stability are critical.
73```
74
75### Phase 2: Standard Plugins
76
77```
78# Task: Implement Standard Plugin
79
80## Context
81- Current component: Standard Plugins - [Specific plugin]
82- Related files: [List relevant files]
83- Dependencies: [List dependencies]
84- Reference: SIMPLIFICATION_PLAN.md - Standard Plugins
85
86## Objective
87Implement the [specific plugin] according to the simplification plan.
88
89## Requirements
90- Implement the plugin interface
91- Extract functionality from existing code
92- Ensure backward compatibility
93- Include comprehensive docstrings
94- Write unit tests
95
96## Constraints
97- Keep plugins focused on a single responsibility
98- Minimize dependencies between plugins
99- Follow security best practices
100
101## Expected Output
102- Implemented plugin with tests
103- Documentation for public interfaces
104- Example usage
105
106## Additional Notes
107This plugin will be used by multiple agent implementations, so clarity and stability are important.
108```
109
110### Phase 3: Tool Plugins
111
112```
113# Task: Implement Tool Plugin
114
115## Context
116- Current component: Tool Plugins - [Specific tool]
117- Related files: [List relevant files]
118- Dependencies: [List dependencies]
119- Reference: SIMPLIFICATION_PLAN.md - Tool Plugins
120
121## Objective
122Implement the [specific tool] plugin according to the simplification plan.
123
124## Requirements
125- Implement the tool plugin interface
126- Extract functionality from existing code
127- Ensure proper security and permission handling
128- Include comprehensive docstrings
129- Write unit tests
130
131## Constraints
132- Keep tools focused on a specific operation
133- Implement proper validation and error handling
134- Follow security best practices
135
136## Expected Output
137- Implemented tool plugin with tests
138- Documentation for public interfaces
139- Example usage
140- Security considerations
141
142## Additional Notes
143Tools interact with external systems, so security and error handling are critical.
144```
145
146### Phase 4: Integration Plugins
147
148```
149# Task: Implement Integration Plugin
150
151## Context
152- Current component: Integration Plugins - [Specific integration]
153- Related files: [List relevant files]
154- Dependencies: [List dependencies]
155- Reference: SIMPLIFICATION_PLAN.md - Integration Plugins
156
157## Objective
158Implement the [specific integration] plugin according to the simplification plan.
159
160## Requirements
161- Implement the integration plugin interface
162- Extract functionality from existing code
163- Ensure proper error handling
164- Include comprehensive docstrings
165- Write unit tests
166
167## Constraints
168- Keep integrations modular and self-contained
169- Implement proper validation and error handling
170- Follow security best practices
171
172## Expected Output
173- Implemented integration plugin with tests
174- Documentation for public interfaces
175- Example usage
176- Configuration examples
177
178## Additional Notes
179Integrations connect to external systems, so error handling and configuration are critical.
180```
181
182### Phase 5: Agent Implementations
183
184```
185# Task: Implement Agent
186
187## Context
188- Current component: Agent Implementations - [Specific agent]
189- Related files: [List relevant files]
190- Dependencies: [List dependencies]
191- Reference: SIMPLIFICATION_PLAN.md - Agent Implementations
192
193## Objective
194Implement the [specific agent] according to the simplification plan.
195
196## Requirements
197- Implement the agent using the plugin system
198- Ensure feature parity with existing implementation
199- Provide a clean, user-friendly interface
200- Include comprehensive docstrings
201- Write unit tests
202
203## Constraints
204- Use plugins for functionality rather than implementing directly
205- Maintain backward compatibility where possible
206- Follow security best practices
207
208## Expected Output
209- Implemented agent with tests
210- Documentation for public interfaces
211- Example usage
212- Migration guide for existing users
213
214## Additional Notes
215Agents are the main entry point for users, so usability and documentation are critical.
216```
217
218### Phase 6: Documentation and Examples
219
220```
221# Task: Create Documentation/Examples
222
223## Context
224- Current component: Documentation - [Specific documentation]
225- Related files: [List relevant files]
226- Dependencies: [List dependencies]
227- Reference: SIMPLIFICATION_PLAN.md - Documentation Strategy
228
229## Objective
230Create [specific documentation/example] according to the simplification plan.
231
232## Requirements
233- Clear, concise explanations
234- Code examples for common use cases
235- Step-by-step instructions where appropriate
236- Consistent formatting and style
237
238## Constraints
239- Focus on user needs
240- Keep examples simple and focused
241- Follow documentation best practices
242
243## Expected Output
244- Completed documentation/example
245- Any necessary supporting files
246
247## Additional Notes
248Documentation is critical for user adoption and developer onboarding.
249```
250
251## Example Prompts
252
253### Example 1: Implementing the Base Agent Class
254
255```
256# Task: Implement Base Agent Class
257
258## Context
259- Current component: Core Layer - Agent
260- Related files: ollamatools/core/agent.py
261- Dependencies: core/config.py, core/errors.py, plugins/base.py
262- Reference: SIMPLIFICATION_PLAN.md - Core Layer
263
264## Objective
265Implement the base Agent class that will serve as the foundation for all agent implementations.
266
267## Requirements
268- Implement plugin management (enabling/disabling plugins)
269- Implement basic chat processing
270- Implement error handling
271- Implement lifecycle management (startup/shutdown)
272- Include comprehensive docstrings
273- Write unit tests
274
275## Constraints
276- Keep the base agent minimal and focused
277- Avoid dependencies on specific plugins
278- Ensure security best practices
279
280## Expected Output
281- Implemented base Agent class with tests
282- Documentation for public interfaces
283- Example usage
284
285## Additional Notes
286The base Agent class is the foundation of the entire system, so clarity, stability, and extensibility are critical.
287```
288
289### Example 2: Implementing the Search Plugin
290
291```
292# Task: Implement Search Plugin
293
294## Context
295- Current component: Standard Plugins - Search
296- Related files: ollamatools/plugins/standard/search.py
297- Dependencies: plugins/base.py, core/errors.py, integrations/opensearch/
298- Reference: SIMPLIFICATION_PLAN.md - Standard Plugins
299
300## Objective
301Implement the Search plugin that provides search functionality for agents.
302
303## Requirements
304- Implement the plugin interface
305- Extract functionality from existing search code
306- Implement intent detection for search queries
307- Implement search result processing
308- Include comprehensive docstrings
309- Write unit tests
310
311## Constraints
312- Keep the plugin focused on search functionality
313- Minimize dependencies on other plugins
314- Follow security best practices
315
316## Expected Output
317- Implemented Search plugin with tests
318- Documentation for public interfaces
319- Example usage
320
321## Additional Notes
322The Search plugin is used by multiple agent implementations and integrations, so clarity and stability are important.
323```
324
325## Using This Template
326
327When working on the OllamaTools simplification project:
328
3291. **For Humans**: Use this template to create specific task descriptions for yourself or other developers. Fill in the details based on the component you're working on.
330
3312. **For AI Assistants**: When asking an AI assistant to help with implementation, use this template to provide clear context and requirements. The more specific you are, the better the assistance will be.
332
3333. **For Both**: Always refer to the detailed `SIMPLIFICATION_PLAN.md` for comprehensive information about the architecture and implementation strategy.
334
335By using consistent prompts, we can ensure that all contributors (human and AI) have a clear understanding of the tasks and requirements, leading to more efficient and effective implementation of the simplification plan.
ollamatools-project/SIMPLIFY_README.md +85 -0
……@@ -0,0 +1,85 @@
1# OllamaTools Simplification Project
2
3This branch (`simplify`) contains the work-in-progress for simplifying and modularizing the OllamaTools project. The goal is to create a more maintainable, extensible, and user-friendly codebase while preserving all existing functionality.
4
5## Overview
6
7The OllamaTools project has evolved through several phases, resulting in a complex architecture with multiple overlapping components. This simplification project aims to:
8
91. **Reduce Redundancy**: Eliminate duplicate code and consolidate similar functionality
102. **Improve Modularity**: Create a clear, plugin-based architecture
113. **Simplify Interfaces**: Provide consistent, well-documented interfaces
124. **Enhance Maintainability**: Make the codebase easier to understand and modify
135. **Preserve Functionality**: Maintain all current features and capabilities
146. **Streamline Testing**: Improve testability through better separation of concerns
15
16## Key Documents
17
18This branch includes the following key documents:
19
201. **[SIMPLIFICATION_PLAN.md](./SIMPLIFICATION_PLAN.md)**: A comprehensive plan detailing the target architecture, component breakdown, implementation phases, migration strategy, testing strategy, and documentation strategy.
21
222. **[SIMPLIFICATION_PROMPT_TEMPLATE.md](./SIMPLIFICATION_PROMPT_TEMPLATE.md)**: A template for creating prompts that can be used by both humans and AI assistants when working on specific tasks within the simplification project.
23
24## Target Architecture
25
26The simplified architecture organizes the codebase into clear, focused components:
27
28```
29ollamatools/
30├── core/ # Core functionality
31├── plugins/ # Plugin system and implementations
32│ ├── standard/ # Standard plugins
33│ ├── tools/ # Tool plugins
34│ └── integrations/ # Integration plugins
35├── agents/ # Agent implementations
36├── config/ # Configuration system
37└── cli/ # Command-line interface
38```
39
40## Implementation Phases
41
42The simplification project is divided into six phases:
43
441. **Core Infrastructure** (Weeks 1-2): Implement the core layer, plugin system, and configuration system
452. **Standard Plugins** (Weeks 3-4): Implement standard plugins and extract common functionality
463. **Tool Plugins** (Weeks 5-6): Implement tool plugins with proper security and permission handling
474. **Integration Plugins** (Weeks 7-8): Implement integration plugins with proper error handling
485. **Agent Implementations** (Weeks 9-10): Implement agent classes with feature parity
496. **Documentation and Examples** (Weeks 11-12): Provide comprehensive documentation and examples
50
51## How to Contribute
52
53If you're contributing to the simplification project:
54
551. **Read the Plan**: Start by reading the [SIMPLIFICATION_PLAN.md](./SIMPLIFICATION_PLAN.md) document to understand the overall architecture and strategy.
56
572. **Use the Prompt Template**: When working on a specific task, use the template in [SIMPLIFICATION_PROMPT_TEMPLATE.md](./SIMPLIFICATION_PROMPT_TEMPLATE.md) to create a clear task description.
58
593. **Follow the Architecture**: Ensure your implementation follows the target architecture and adheres to the design principles.
60
614. **Write Tests**: Include comprehensive tests for all new code.
62
635. **Document Your Work**: Add docstrings and update documentation as needed.
64
656. **Submit Focused PRs**: Keep pull requests focused on specific components or tasks.
66
67## Working with AI Assistants
68
69When working with AI assistants on this project:
70
711. **Provide Clear Context**: Use the prompt template to give the AI assistant clear context about the task.
72
732. **Reference the Plan**: Point the AI to specific sections of the simplification plan for detailed guidance.
74
753. **Review Carefully**: Always review AI-generated code for adherence to the architecture and best practices.
76
774. **Iterate**: Be prepared to iterate with the AI to refine the implementation.
78
79## Current Status
80
81This branch is currently in the initial planning phase. Implementation will begin with the Core Infrastructure phase.
82
83## Questions and Feedback
84
85If you have questions or feedback about the simplification project, please open an issue with the tag `simplification` or contact the project maintainers.