OverviewFilesCommitsIssuesDocsReleasesPeers

ollama_semantic_function_calling commits / cf2717e

Add refactoring scripts and documentation for OllamaTools simplification
cf2717e9281ee98e2f7b81c4e91fa1fadb0e7e2b
by fleeky.ashatron <fleeky@7rnx.net> · 2025-04-19T01:59:36.000Z

Changed files (11) +1480 -0

ollamatools-project/README_REFACTORING.md +79 -0
……@@ -0,0 +1,79 @@
1# OllamaTools Refactoring Guide
2
3This guide explains the refactoring approach for the OllamaTools project, which involves creating a parallel directory structure for the new implementation while keeping the original codebase as a reference.
4
5## Refactoring Approach
6
7The refactoring approach follows these steps:
8
91. Create a new directory (`ollamatools-new`) at the same level as the current project
102. Set up the new directory structure according to the simplification plan
113. Implement components incrementally following the phased approach in the plan
124. Use the original codebase as reference while building the new structure
135. Run tests against both codebases to ensure functional equivalence
146. Once verified, switch references to the new structure
157. Rename directories (old to `ollamatools-legacy`, new to `ollamatools`)
16
17This approach has several advantages:
18- Allows for a clean implementation of the new architecture
19- Minimizes confusion between old and new code
20- Enables incremental testing and verification
21- Provides a clear separation between legacy and new code
22- Makes it easier to follow the phased implementation plan
23- Allows for easier rollback if issues arise
24
25## Setup Script
26
27The `setup_refactored_structure.sh` script automates the creation of the new directory structure. It:
28
291. Creates a new directory (`ollamatools-new`) at the parent level
302. Copies essential files from the current project (README, setup.py, etc.)
313. Creates the main package directory and all subdirectories according to the simplification plan
324. Creates empty files for all modules defined in the plan
335. Creates a REFACTORING.md file with information about the refactoring process
34
35### Usage
36
37To use the script:
38
39```bash
40# Make the script executable
41chmod +x setup_refactored_structure.sh
42
43# Run the script
44./setup_refactored_structure.sh
45```
46
47## Implementation Process
48
49After setting up the directory structure, the implementation should follow the phased approach outlined in the simplification plan:
50
511. **Core Infrastructure** (core module)
522. **Plugin System** (plugins base)
533. **Standard Plugins** (plugins/standard)
544. **Tool Plugins** (plugins/tools)
555. **Integration Plugins** (plugins/integrations)
566. **Agent Implementations** (agents)
577. **Configuration System** (config)
588. **Command-Line Interface** (cli)
59
60For each component:
611. Refer to the original codebase to understand the current functionality
622. Implement the component according to the new architecture
633. Write tests to verify the implementation
644. Document the component
65
66## Final Migration
67
68Once the refactored implementation is complete and tested:
69
701. Create a backup of the original codebase
712. Rename the original directory to `ollamatools-legacy`
723. Rename the new directory to `ollamatools`
734. Update any references to the project path
74
75## Additional Resources
76
77- See `SIMPLIFICATION_PLAN.md` for the overall refactoring plan
78- See `ai_simplification_guides/` for detailed implementation guides
79- See `REFACTORING.md` in the new project directory for more information
ollamatools-project/REFACTORING_SCRIPTS.md +171 -0
……@@ -0,0 +1,171 @@
1# OllamaTools Refactoring Scripts
2
3This document provides a summary of all the scripts created to help with the refactoring process for the OllamaTools project.
4
5## Git Scripts
6
7### `git_refactoring_commands.sh`
8
9Provides Git commands for different stages of the refactoring process.
10
11**Usage:**
12```bash
13./git_refactoring_commands.sh
14```
15
16**What it does:**
17- Provides Git commands for initial setup, refactoring process, testing, and finalization
18- Includes a Git commands cheat sheet for quick reference
19- Guides you through the Git workflow for the refactoring process
20
21## Setup Scripts
22
23### `setup_refactored_structure.sh`
24
25Creates the new directory structure for the refactored project according to the simplification plan.
26
27**Usage:**
28```bash
29./setup_refactored_structure.sh
30```
31
32**What it does:**
33- Creates a new directory (`ollamatools-new`) at the parent level
34- Sets up the directory structure according to the simplification plan
35- Creates empty files for all modules defined in the plan
36- Copies essential files from the current project (README, setup.py, etc.)
37- Creates a REFACTORING.md file with information about the refactoring process
38
39## File Mapping Scripts
40
41### `generate_file_mapping.sh`
42
43Generates a mapping between the old and new file structures to guide the refactoring process.
44
45**Usage:**
46```bash
47./generate_file_mapping.sh
48```
49
50**What it does:**
51- Creates a file_mapping.md file with a mapping between the original and new file structures
52- Each entry includes the original file path, new file path, status, and notes
53- The mapping serves as a guide for the refactoring process
54
55### `update_mapping_status.sh`
56
57Updates the status of a file in the file mapping as it is refactored.
58
59**Usage:**
60```bash
61./update_mapping_status.sh <original_file> <status> [notes]
62```
63
64**Example:**
65```bash
66./update_mapping_status.sh ollamatools/agent.py Completed "Refactored with updated imports"
67```
68
69**What it does:**
70- Updates the status of a file in the file mapping
71- Optionally updates the notes for the file
72- Helps track progress of the refactoring process
73
74## File Refactoring Scripts
75
76### `copy_refactor_file.sh`
77
78Copies a file from the original codebase to the new structure and opens it in the editor for refactoring.
79
80**Usage:**
81```bash
82./copy_refactor_file.sh <source_file> <destination_file>
83```
84
85**Example:**
86```bash
87./copy_refactor_file.sh ollamatools/agent.py ollamatools/core/agent.py
88```
89
90**What it does:**
91- Copies a file from the original codebase to the new structure
92- Opens the file in the editor for refactoring
93- Creates any necessary directories for the destination file
94
95## Testing Scripts
96
97### `test_refactored_code.sh`
98
99Runs tests for both the original and refactored codebases to ensure they have the same functionality.
100
101**Usage:**
102```bash
103./test_refactored_code.sh [options] <test_file>
104```
105
106**Options:**
107- `-o, --original`: Run tests only on the original codebase
108- `-n, --new`: Run tests only on the new codebase
109- `-h, --help`: Display help message
110
111**Example:**
112```bash
113./test_refactored_code.sh test_agent.py # Run test_agent.py on both codebases
114./test_refactored_code.sh --original test_agent.py # Run test_agent.py only on the original codebase
115./test_refactored_code.sh --new test_agent.py # Run test_agent.py only on the new codebase
116```
117
118**What it does:**
119- Runs tests for both the original and refactored codebases
120- Compares the results to ensure they have the same functionality
121- Helps verify that the refactored code maintains the same behavior
122
123## Finalization Scripts
124
125### `finalize_refactoring.sh`
126
127Finalizes the refactoring process by renaming directories and creating backups.
128
129**Usage:**
130```bash
131./finalize_refactoring.sh
132```
133
134**What it does:**
135- Creates a backup of the original codebase
136- Renames the original directory to `ollamatools-legacy`
137- Renames the new directory to `ollamatools-project`
138- Provides a confirmation prompt before proceeding
139
140## Guide Scripts
141
142### `refactoring_guide.sh`
143
144Provides a step-by-step guide for the refactoring process.
145
146**Usage:**
147```bash
148./refactoring_guide.sh
149```
150
151**What it does:**
152- Outlines the steps to follow for the refactoring process
153- Provides commands to run at each step
154- Includes notes and warnings for each step
155- Guides you through the entire refactoring process from setup to finalization
156
157## Workflow
158
159The typical workflow for using these scripts is:
160
1611. Run `./git_refactoring_commands.sh` to understand the Git workflow for the refactoring process
1622. Run `./setup_refactored_structure.sh` to create the new directory structure
1633. Run `./generate_file_mapping.sh` to create the file mapping
1644. Follow the steps in `./refactoring_guide.sh` to refactor the codebase
1655. Use `./copy_refactor_file.sh` to copy and refactor files
1666. Use `./update_mapping_status.sh` to track progress
1677. Use `./test_refactored_code.sh` to test the refactored code
1688. Run `./finalize_refactoring.sh` to finalize the refactoring process
1699. Use Git commands from `./git_refactoring_commands.sh` to commit and push your changes
170
171This workflow ensures a systematic and organized approach to the refactoring process, with proper tracking, testing, and version control at each step.
ollamatools-project/copy_refactor_file.sh +81 -0
……@@ -0,0 +1,81 @@
1#!/bin/bash
2
3# Script to help copy files from the original codebase to the new structure
4# This script copies a file from the original codebase to the new structure
5# and opens it in the editor for refactoring
6
7# Define the parent directory and project directories
8PARENT_DIR=".."
9ORIGINAL_DIR="$PARENT_DIR/ollamatools-project"
10NEW_DIR="$PARENT_DIR/ollamatools-new"
11
12# Function to display usage information
13usage() {
14 echo "Usage: $0 <source_file> <destination_file>"
15 echo ""
16 echo "Arguments:"
17 echo " <source_file> Path to the source file in the original codebase (relative to $ORIGINAL_DIR)"
18 echo " <destination_file> Path to the destination file in the new codebase (relative to $NEW_DIR)"
19 echo ""
20 echo "Example:"
21 echo " $0 ollamatools/agent.py ollamatools/core/agent.py"
22 echo ""
23 echo "This will copy ollamatools/agent.py from the original codebase to ollamatools/core/agent.py in the new codebase"
24 echo "and open it in the editor for refactoring."
25 exit 1
26}
27
28# Check if the correct number of arguments is provided
29if [ $# -ne 2 ]; then
30 usage
31fi
32
33SOURCE_FILE="$1"
34DEST_FILE="$2"
35
36# Check if the source file exists
37if [ ! -f "$ORIGINAL_DIR/$SOURCE_FILE" ]; then
38 echo "Error: Source file $ORIGINAL_DIR/$SOURCE_FILE does not exist."
39 exit 1
40fi
41
42# Check if the destination directory exists, create it if it doesn't
43DEST_DIR=$(dirname "$NEW_DIR/$DEST_FILE")
44if [ ! -d "$DEST_DIR" ]; then
45 echo "Creating destination directory: $DEST_DIR"
46 mkdir -p "$DEST_DIR"
47fi
48
49# Copy the file
50echo "Copying $ORIGINAL_DIR/$SOURCE_FILE to $NEW_DIR/$DEST_FILE"
51cp "$ORIGINAL_DIR/$SOURCE_FILE" "$NEW_DIR/$DEST_FILE"
52
53# Check if the copy was successful
54if [ $? -ne 0 ]; then
55 echo "Error: Failed to copy file."
56 exit 1
57fi
58
59# Open the file in the editor
60if command -v code &> /dev/null; then
61 echo "Opening file in VS Code..."
62 code "$NEW_DIR/$DEST_FILE"
63elif command -v vim &> /dev/null; then
64 echo "Opening file in Vim..."
65 vim "$NEW_DIR/$DEST_FILE"
66elif command -v nano &> /dev/null; then
67 echo "Opening file in Nano..."
68 nano "$NEW_DIR/$DEST_FILE"
69else
70 echo "No suitable editor found. File has been copied but not opened."
71fi
72
73echo ""
74echo "File copied successfully!"
75echo "You can now refactor the file according to the new architecture."
76echo ""
77echo "Remember to:"
78echo "1. Update imports to use the new package structure"
79echo "2. Adapt the code to the new architecture"
80echo "3. Remove any redundant or deprecated functionality"
81echo "4. Add proper documentation"
ollamatools-project/file_mapping.md +41 -0
……@@ -0,0 +1,41 @@
1# OllamaTools File Mapping
2
3This document provides a mapping between the original codebase and the new refactored structure.
4Use this as a guide for the refactoring process to ensure all functionality is migrated correctly.
5
6## File Mapping
7
8| Original File | New File | Status | Notes |
9|---------------|----------|--------|-------|
10| `ollamatools/agent.py` | `ollamatools/core/agent.py` | Pending | Base agent implementation |
11| `ollamatools/config.py` | `ollamatools/core/config.py` | Pending | Configuration management |
12| `ollamatools/errors.py` | `ollamatools/core/errors.py` | Pending | Error definitions and handling |
13| `ollamatools/security.py` | `ollamatools/core/security.py` | Pending | Security utilities and permission system |
14| `ollamatools/logger.py` | `ollamatools/core/utils/logger.py` | Pending | Logging utilities |
15| `ollamatools/command_validator.py` | `ollamatools/core/utils/validators.py` | Pending | Command validation (to be merged with other validators) |
16| `ollamatools/path_validator.py` | `ollamatools/core/utils/validators.py` | Pending | Path validation (to be merged with other validators) |
17| `ollamatools/url_validator.py` | `ollamatools/core/utils/validators.py` | Pending | URL validation (to be merged with other validators) |
18| `ollamatools/plugins/__init__.py` | `ollamatools/plugins/__init__.py` | Pending | Plugin system initialization |
19| `ollamatools/plugins/registry.py` | `ollamatools/plugins/registry.py` | Pending | Plugin discovery and management |
20| `ollamatools/plugins/errors.py` | `ollamatools/plugins/errors.py` | Pending | Plugin-specific error handling |
21| `ollamatools/agent/modular/search.py` | `ollamatools/plugins/standard/search.py` | Pending | Search functionality |
22| `ollamatools/agent/modular/commands.py` | `ollamatools/plugins/standard/commands.py` | Pending | Command management |
23| `ollamatools/agent/modular/feedback.py` | `ollamatools/plugins/standard/feedback.py` | Pending | Feedback collection |
24| `ollamatools/agent/modular/issues.py` | `ollamatools/plugins/standard/issues.py` | Pending | Issue tracking |
25| `ollamatools/plugins/conversation_history.py` | `ollamatools/plugins/standard/conversation/history.py` | Pending | Conversation history tracking |
26| `ollamatools/plugins/conversation_context.py` | `ollamatools/plugins/standard/conversation/context.py` | Pending | Conversation context management |
27| `ollamatools/tools/system_tools.py` | `ollamatools/plugins/tools/system.py` | Pending | System tools (file operations, commands) |
28| `ollamatools/tools/web_tools.py` | `ollamatools/plugins/tools/web.py` | Pending | Web tools (HTTP requests, web searches) |
29| `ollamatools/tools/doc_tools.py` | `ollamatools/plugins/tools/doc.py` | Pending | Documentation tools |
30| `ollamatools/opensearch_web_tools.py` | `ollamatools/plugins/integrations/opensearch/tools.py` | Pending | OpenSearch web tools |
31| `ollamatools/tools/matrix_tools.py` | `ollamatools/plugins/integrations/matrix/client.py` | Pending | Matrix client core |
32| `ollamatools/tools/enhanced_matrix_core.py` | `ollamatools/plugins/integrations/matrix/client.py` | Pending | Enhanced Matrix client core |
33| `ollamatools/tools/enhanced_matrix_history.py` | `ollamatools/plugins/integrations/matrix/history.py` | Pending | Matrix history tracking |
34| `ollamatools/tools/enhanced_matrix_search.py` | `ollamatools/plugins/integrations/matrix/search.py` | Pending | Matrix search integration |
35| `ollamatools/tools/matrix_rate_limit_patch.py` | `ollamatools/plugins/integrations/matrix/rate_limit.py` | Pending | Rate limiting for Matrix |
36| `examples/basic_agent.py` | `ollamatools/agents/basic.py` | Pending | Basic agent implementation |
37| `ollamatools/agent/modular/agent.py` | `ollamatools/agents/modular.py` | Pending | Modular agent implementation |
38| `ollamatools/agent/extended/agent.py` | `ollamatools/agents/extended.py` | Pending | Extended agent implementation |
39| `ollamatools/config/schemas.py` | `ollamatools/config/schemas.py` | Pending | Configuration schemas |
40| `ollamatools/config/core_config.py` | `ollamatools/config/loader.py` | Pending | Configuration loading/validation |
41| `ollamatools/config/plugin_config.py` | `ollamatools/config/defaults.py` | Pending | Default configurations |
ollamatools-project/finalize_refactoring.sh +74 -0
……@@ -0,0 +1,74 @@
1#!/bin/bash
2
3# Script to finalize the refactoring process by:
4# 1. Creating a backup of the original codebase
5# 2. Renaming the original directory to ollamatools-legacy
6# 3. Renaming the new directory to ollamatools
7
8# Define the parent directory and project directories
9PARENT_DIR=".."
10ORIGINAL_DIR="$PARENT_DIR/ollamatools-project"
11NEW_DIR="$PARENT_DIR/ollamatools-new"
12LEGACY_DIR="$PARENT_DIR/ollamatools-legacy"
13BACKUP_DIR="$PARENT_DIR/ollamatools-backup-$(date +%Y%m%d%H%M%S)"
14
15# Check if the directories exist
16if [ ! -d "$ORIGINAL_DIR" ]; then
17 echo "Error: Original directory $ORIGINAL_DIR does not exist."
18 exit 1
19fi
20
21if [ ! -d "$NEW_DIR" ]; then
22 echo "Error: New directory $NEW_DIR does not exist."
23 exit 1
24fi
25
26# Confirm with the user
27echo "This script will finalize the refactoring process by:"
28echo "1. Creating a backup of the original codebase at $BACKUP_DIR"
29echo "2. Renaming $ORIGINAL_DIR to $LEGACY_DIR"
30echo "3. Renaming $NEW_DIR to $ORIGINAL_DIR"
31echo ""
32read -p "Are you sure you want to proceed? (y/n): " -n 1 -r
33echo ""
34
35if [[ ! $REPLY =~ ^[Yy]$ ]]; then
36 echo "Operation cancelled."
37 exit 0
38fi
39
40# Create a backup of the original codebase
41echo "Creating backup of the original codebase..."
42cp -r "$ORIGINAL_DIR" "$BACKUP_DIR"
43if [ $? -ne 0 ]; then
44 echo "Error: Failed to create backup. Aborting."
45 exit 1
46fi
47echo "Backup created at $BACKUP_DIR"
48
49# Rename the original directory to ollamatools-legacy
50echo "Renaming original directory to ollamatools-legacy..."
51mv "$ORIGINAL_DIR" "$LEGACY_DIR"
52if [ $? -ne 0 ]; then
53 echo "Error: Failed to rename original directory. Aborting."
54 exit 1
55fi
56echo "Original directory renamed to ollamatools-legacy"
57
58# Rename the new directory to ollamatools
59echo "Renaming new directory to ollamatools-project..."
60mv "$NEW_DIR" "$ORIGINAL_DIR"
61if [ $? -ne 0 ]; then
62 echo "Error: Failed to rename new directory. Restoring original directory."
63 mv "$LEGACY_DIR" "$ORIGINAL_DIR"
64 exit 1
65fi
66echo "New directory renamed to ollamatools-project"
67
68echo ""
69echo "Refactoring finalization complete!"
70echo "- Original codebase backup: $BACKUP_DIR"
71echo "- Legacy codebase: $LEGACY_DIR"
72echo "- New codebase: $ORIGINAL_DIR"
73echo ""
74echo "Note: You may need to update any references to the project paths in your environment."
ollamatools-project/generate_file_mapping.sh +98 -0
……@@ -0,0 +1,98 @@
1#!/bin/bash
2
3# Script to generate a mapping between the old and new file structures
4# This helps guide the refactoring process by showing which files from the original codebase
5# should be moved to which locations in the new structure
6
7# Define the output file
8OUTPUT_FILE="file_mapping.md"
9
10# Create the output file with a header
11cat > "$OUTPUT_FILE" << 'EOF'
12# OllamaTools File Mapping
13
14This document provides a mapping between the original codebase and the new refactored structure.
15Use this as a guide for the refactoring process to ensure all functionality is migrated correctly.
16
17## File Mapping
18
19| Original File | New File | Status | Notes |
20|---------------|----------|--------|-------|
21EOF
22
23# Function to add a mapping entry
24add_mapping() {
25 local original_file="$1"
26 local new_file="$2"
27 local status="$3"
28 local notes="$4"
29
30 echo "| \`$original_file\` | \`$new_file\` | $status | $notes |" >> "$OUTPUT_FILE"
31}
32
33# Core module mappings
34echo "Adding Core module mappings..."
35add_mapping "ollamatools/agent.py" "ollamatools/core/agent.py" "Pending" "Base agent implementation"
36add_mapping "ollamatools/config.py" "ollamatools/core/config.py" "Pending" "Configuration management"
37add_mapping "ollamatools/errors.py" "ollamatools/core/errors.py" "Pending" "Error definitions and handling"
38add_mapping "ollamatools/security.py" "ollamatools/core/security.py" "Pending" "Security utilities and permission system"
39add_mapping "ollamatools/logger.py" "ollamatools/core/utils/logger.py" "Pending" "Logging utilities"
40add_mapping "ollamatools/command_validator.py" "ollamatools/core/utils/validators.py" "Pending" "Command validation (to be merged with other validators)"
41add_mapping "ollamatools/path_validator.py" "ollamatools/core/utils/validators.py" "Pending" "Path validation (to be merged with other validators)"
42add_mapping "ollamatools/url_validator.py" "ollamatools/core/utils/validators.py" "Pending" "URL validation (to be merged with other validators)"
43
44# Plugin system mappings
45echo "Adding Plugin system mappings..."
46add_mapping "ollamatools/plugins/__init__.py" "ollamatools/plugins/__init__.py" "Pending" "Plugin system initialization"
47add_mapping "ollamatools/plugins/registry.py" "ollamatools/plugins/registry.py" "Pending" "Plugin discovery and management"
48add_mapping "ollamatools/plugins/errors.py" "ollamatools/plugins/errors.py" "Pending" "Plugin-specific error handling"
49
50# Standard plugins mappings
51echo "Adding Standard plugins mappings..."
52add_mapping "ollamatools/agent/modular/search.py" "ollamatools/plugins/standard/search.py" "Pending" "Search functionality"
53add_mapping "ollamatools/agent/modular/commands.py" "ollamatools/plugins/standard/commands.py" "Pending" "Command management"
54add_mapping "ollamatools/agent/modular/feedback.py" "ollamatools/plugins/standard/feedback.py" "Pending" "Feedback collection"
55add_mapping "ollamatools/agent/modular/issues.py" "ollamatools/plugins/standard/issues.py" "Pending" "Issue tracking"
56add_mapping "ollamatools/plugins/conversation_history.py" "ollamatools/plugins/standard/conversation/history.py" "Pending" "Conversation history tracking"
57add_mapping "ollamatools/plugins/conversation_context.py" "ollamatools/plugins/standard/conversation/context.py" "Pending" "Conversation context management"
58
59# Tool plugins mappings
60echo "Adding Tool plugins mappings..."
61add_mapping "ollamatools/tools/system_tools.py" "ollamatools/plugins/tools/system.py" "Pending" "System tools (file operations, commands)"
62add_mapping "ollamatools/tools/web_tools.py" "ollamatools/plugins/tools/web.py" "Pending" "Web tools (HTTP requests, web searches)"
63add_mapping "ollamatools/tools/doc_tools.py" "ollamatools/plugins/tools/doc.py" "Pending" "Documentation tools"
64add_mapping "ollamatools/opensearch_web_tools.py" "ollamatools/plugins/integrations/opensearch/tools.py" "Pending" "OpenSearch web tools"
65
66# Integration plugins mappings
67echo "Adding Integration plugins mappings..."
68add_mapping "ollamatools/tools/matrix_tools.py" "ollamatools/plugins/integrations/matrix/client.py" "Pending" "Matrix client core"
69add_mapping "ollamatools/tools/enhanced_matrix_core.py" "ollamatools/plugins/integrations/matrix/client.py" "Pending" "Enhanced Matrix client core"
70add_mapping "ollamatools/tools/enhanced_matrix_history.py" "ollamatools/plugins/integrations/matrix/history.py" "Pending" "Matrix history tracking"
71add_mapping "ollamatools/tools/enhanced_matrix_search.py" "ollamatools/plugins/integrations/matrix/search.py" "Pending" "Matrix search integration"
72add_mapping "ollamatools/tools/matrix_rate_limit_patch.py" "ollamatools/plugins/integrations/matrix/rate_limit.py" "Pending" "Rate limiting for Matrix"
73
74# Agent implementations mappings
75echo "Adding Agent implementations mappings..."
76add_mapping "examples/basic_agent.py" "ollamatools/agents/basic.py" "Pending" "Basic agent implementation"
77add_mapping "ollamatools/agent/modular/agent.py" "ollamatools/agents/modular.py" "Pending" "Modular agent implementation"
78add_mapping "ollamatools/agent/extended/agent.py" "ollamatools/agents/extended.py" "Pending" "Extended agent implementation"
79
80# Configuration system mappings
81echo "Adding Configuration system mappings..."
82add_mapping "ollamatools/config/schemas.py" "ollamatools/config/schemas.py" "Pending" "Configuration schemas"
83add_mapping "ollamatools/config/core_config.py" "ollamatools/config/loader.py" "Pending" "Configuration loading/validation"
84add_mapping "ollamatools/config/plugin_config.py" "ollamatools/config/defaults.py" "Pending" "Default configurations"
85
86echo ""
87echo "File mapping generated: $OUTPUT_FILE"
88echo ""
89echo "This mapping provides a guide for the refactoring process."
90echo "For each file in the original codebase, it shows where the functionality should be moved in the new structure."
91echo ""
92echo "To use this mapping with the copy_refactor_file.sh script:"
93echo " ./copy_refactor_file.sh <original_file> <new_file>"
94echo ""
95echo "For example:"
96echo " ./copy_refactor_file.sh ollamatools/agent.py ollamatools/core/agent.py"
97echo ""
98echo "After refactoring each file, update the Status column in the mapping file to track progress."
ollamatools-project/git_refactoring_commands.sh +185 -0
……@@ -0,0 +1,185 @@
1#!/bin/bash
2
3# Script with Git commands to help with the refactoring process
4# This script provides Git commands for different stages of the refactoring process
5
6# Define colors for better readability
7RED='\033[0;31m'
8GREEN='\033[0;32m'
9YELLOW='\033[0;33m'
10BLUE='\033[0;34m'
11PURPLE='\033[0;35m'
12CYAN='\033[0;36m'
13NC='\033[0m' # No Color
14
15# Function to print section headers
16print_header() {
17 echo -e "\n${BLUE}========== $1 ==========${NC}\n"
18}
19
20# Function to print steps
21print_step() {
22 echo -e "${GREEN}Step $1:${NC} $2"
23}
24
25# Function to print commands
26print_command() {
27 echo -e " ${YELLOW}$1${NC}"
28}
29
30# Function to print notes
31print_note() {
32 echo -e " ${PURPLE}Note:${NC} $1"
33}
34
35# Function to print warnings
36print_warning() {
37 echo -e " ${RED}Warning:${NC} $1"
38}
39
40# Clear the screen
41clear
42
43# Print the title
44echo -e "${CYAN}"
45echo "======================================================"
46echo " OllamaTools Git Refactoring Commands"
47echo "======================================================"
48echo -e "${NC}"
49echo "This script provides Git commands for different stages of the refactoring process."
50echo "Follow these commands to ensure proper version control during the refactoring."
51echo ""
52echo -e "${YELLOW}Note: This is a guide only. You will need to run the commands manually.${NC}"
53echo ""
54
55# Phase 1: Initial Setup
56print_header "Phase 1: Initial Setup"
57
58print_step "1" "Verify you're on the correct branch"
59print_command "git status"
60print_note "Ensure you're on the 'simplify' branch"
61
62print_step "2" "Create a new branch for the refactoring scripts (if needed)"
63print_command "git checkout -b refactoring-scripts"
64print_note "This creates a new branch for the refactoring scripts"
65print_warning "Skip this step if you want to keep working on the 'simplify' branch"
66
67print_step "3" "Add the refactoring scripts to Git"
68print_command "git add setup_refactored_structure.sh generate_file_mapping.sh update_mapping_status.sh copy_refactor_file.sh test_refactored_code.sh finalize_refactoring.sh refactoring_guide.sh git_refactoring_commands.sh"
69print_note "This adds all the refactoring scripts to Git"
70
71print_step "4" "Add the documentation files to Git"
72print_command "git add README_REFACTORING.md REFACTORING_SCRIPTS.md file_mapping.md"
73print_note "This adds all the documentation files to Git"
74
75print_step "5" "Commit the refactoring scripts and documentation"
76print_command "git commit -m 'Add refactoring scripts and documentation'"
77print_note "This commits all the refactoring scripts and documentation"
78
79# Phase 2: Refactoring Process
80print_header "Phase 2: Refactoring Process"
81
82print_step "1" "Create a new branch for the refactored code"
83print_command "git checkout -b refactored-code"
84print_note "This creates a new branch for the refactored code"
85print_warning "This should be done after running setup_refactored_structure.sh"
86
87print_step "2" "Add the new directory to .gitignore (optional)"
88print_command "echo '../ollamatools-new/' >> .gitignore"
89print_command "git add .gitignore"
90print_command "git commit -m 'Add ollamatools-new to .gitignore'"
91print_note "This prevents the new directory from being tracked by Git until it's ready"
92print_warning "Skip this step if you want to track the new directory with Git"
93
94print_step "3" "Commit changes after each phase of refactoring"
95print_command "git add ."
96print_command "git commit -m 'Complete Phase X: [Description]'"
97print_note "Replace X with the phase number and [Description] with a brief description"
98print_note "Commit after each major phase to maintain a clean history"
99
100# Phase 3: Testing
101print_header "Phase 3: Testing"
102
103print_step "1" "Create a branch for testing the refactored code"
104print_command "git checkout -b refactored-testing"
105print_note "This creates a new branch for testing the refactored code"
106print_warning "This should be done after completing the refactoring"
107
108print_step "2" "Commit test results and fixes"
109print_command "git add ."
110print_command "git commit -m 'Test results and fixes for [Component]'"
111print_note "Replace [Component] with the component being tested"
112print_note "Commit after testing each major component"
113
114# Phase 4: Finalization
115print_header "Phase 4: Finalization"
116
117print_step "1" "Merge the refactored code back to the main branch"
118print_command "git checkout simplify"
119print_command "git merge refactored-testing"
120print_note "This merges the refactored code back to the main branch"
121print_warning "Resolve any merge conflicts that may arise"
122
123print_step "2" "Tag the release"
124print_command "git tag -a v2.0.0 -m 'Refactored version 2.0.0'"
125print_note "This tags the release with a version number"
126print_note "Replace v2.0.0 with the appropriate version number"
127
128print_step "3" "Push the changes to the remote repository"
129print_command "git push origin simplify"
130print_command "git push origin v2.0.0"
131print_note "This pushes the changes and tag to the remote repository"
132
133# Current Status
134print_header "Current Status"
135
136print_step "1" "Add and commit the current refactoring scripts"
137print_command "git add setup_refactored_structure.sh generate_file_mapping.sh update_mapping_status.sh copy_refactor_file.sh test_refactored_code.sh finalize_refactoring.sh refactoring_guide.sh git_refactoring_commands.sh README_REFACTORING.md REFACTORING_SCRIPTS.md file_mapping.md"
138print_command "git commit -m 'Add refactoring scripts and documentation'"
139print_note "This adds and commits all the current refactoring scripts and documentation"
140
141print_step "2" "Push the changes to the remote repository"
142print_command "git push origin simplify"
143print_note "This pushes the changes to the remote repository"
144
145print_header "Git Commands Cheat Sheet"
146
147echo -e "${GREEN}Branching:${NC}"
148echo -e " ${YELLOW}git branch${NC} - List all branches"
149echo -e " ${YELLOW}git checkout <branch>${NC} - Switch to branch"
150echo -e " ${YELLOW}git checkout -b <new-branch>${NC} - Create and switch to new branch"
151
152echo -e "${GREEN}Status and Diff:${NC}"
153echo -e " ${YELLOW}git status${NC} - Show status of working directory"
154echo -e " ${YELLOW}git diff${NC} - Show changes in working directory"
155echo -e " ${YELLOW}git diff --staged${NC} - Show changes in staging area"
156
157echo -e "${GREEN}Staging and Committing:${NC}"
158echo -e " ${YELLOW}git add <file>${NC} - Add file to staging area"
159echo -e " ${YELLOW}git add .${NC} - Add all changes to staging area"
160echo -e " ${YELLOW}git commit -m 'message'${NC} - Commit staged changes"
161
162echo -e "${GREEN}History:${NC}"
163echo -e " ${YELLOW}git log${NC} - Show commit history"
164echo -e " ${YELLOW}git log --oneline${NC} - Show commit history (one line per commit)"
165echo -e " ${YELLOW}git blame <file>${NC} - Show who changed what in a file"
166
167echo -e "${GREEN}Remote Operations:${NC}"
168echo -e " ${YELLOW}git push origin <branch>${NC} - Push changes to remote repository"
169echo -e " ${YELLOW}git pull origin <branch>${NC} - Pull changes from remote repository"
170echo -e " ${YELLOW}git fetch${NC} - Fetch changes from remote repository"
171
172echo -e "${GREEN}Merging and Rebasing:${NC}"
173echo -e " ${YELLOW}git merge <branch>${NC} - Merge branch into current branch"
174echo -e " ${YELLOW}git rebase <branch>${NC} - Rebase current branch onto branch"
175echo -e " ${YELLOW}git cherry-pick <commit>${NC} - Apply changes from a specific commit"
176
177echo -e "${GREEN}Stashing:${NC}"
178echo -e " ${YELLOW}git stash${NC} - Stash changes in working directory"
179echo -e " ${YELLOW}git stash pop${NC} - Apply and remove the latest stash"
180echo -e " ${YELLOW}git stash list${NC} - List all stashes"
181
182echo -e "${GREEN}Tagging:${NC}"
183echo -e " ${YELLOW}git tag${NC} - List all tags"
184echo -e " ${YELLOW}git tag -a <tag> -m 'message'${NC} - Create annotated tag"
185echo -e " ${YELLOW}git push origin <tag>${NC} - Push tag to remote repository"
ollamatools-project/refactoring_guide.sh +314 -0
……@@ -0,0 +1,314 @@
1#!/bin/bash
2
3# Master script to guide the refactoring process
4# This script outlines the steps to follow for the refactoring process
5# and provides commands to run at each step
6
7# Define colors for better readability
8RED='\033[0;31m'
9GREEN='\033[0;32m'
10YELLOW='\033[0;33m'
11BLUE='\033[0;34m'
12PURPLE='\033[0;35m'
13CYAN='\033[0;36m'
14NC='\033[0m' # No Color
15
16# Function to print section headers
17print_header() {
18 echo -e "\n${BLUE}========== $1 ==========${NC}\n"
19}
20
21# Function to print steps
22print_step() {
23 echo -e "${GREEN}Step $1:${NC} $2"
24}
25
26# Function to print commands
27print_command() {
28 echo -e " ${YELLOW}$1${NC}"
29}
30
31# Function to print notes
32print_note() {
33 echo -e " ${PURPLE}Note:${NC} $1"
34}
35
36# Function to print warnings
37print_warning() {
38 echo -e " ${RED}Warning:${NC} $1"
39}
40
41# Clear the screen
42clear
43
44# Print the title
45echo -e "${CYAN}"
46echo "======================================================"
47echo " OllamaTools Refactoring Guide"
48echo "======================================================"
49echo -e "${NC}"
50echo "This script will guide you through the refactoring process for the OllamaTools project."
51echo "Follow the steps in order to ensure a smooth refactoring process."
52echo ""
53echo -e "${YELLOW}Note: This is a guide only. You will need to run the commands manually.${NC}"
54echo ""
55
56# Phase 1: Setup
57print_header "Phase 1: Setup"
58
59print_step "1" "Create the new directory structure"
60print_command "./setup_refactored_structure.sh"
61print_note "This will create a new directory structure at ../ollamatools-new"
62
63print_step "2" "Generate the file mapping"
64print_command "./generate_file_mapping.sh"
65print_note "This will create a file_mapping.md file with a mapping between the original and new file structures"
66
67# Phase 2: Core Infrastructure
68print_header "Phase 2: Core Infrastructure"
69
70print_step "1" "Implement the Error Handling System"
71print_command "./copy_refactor_file.sh ollamatools/errors.py ollamatools/core/errors.py"
72print_note "Refactor the error handling system according to the simplification plan"
73print_command "./update_mapping_status.sh ollamatools/errors.py 'In Progress' 'Refactoring error handling system'"
74
75print_step "2" "Implement the Utility Functions"
76print_command "./copy_refactor_file.sh ollamatools/logger.py ollamatools/core/utils/logger.py"
77print_note "Refactor the logger according to the simplification plan"
78print_command "./update_mapping_status.sh ollamatools/logger.py 'In Progress' 'Refactoring logger'"
79
80print_command "./copy_refactor_file.sh ollamatools/command_validator.py ollamatools/core/utils/validators.py"
81print_note "Merge the command validator into the validators module"
82print_command "./update_mapping_status.sh ollamatools/command_validator.py 'In Progress' 'Merging into validators module'"
83
84print_command "./copy_refactor_file.sh ollamatools/path_validator.py ollamatools/core/utils/validators.py"
85print_note "Merge the path validator into the validators module"
86print_command "./update_mapping_status.sh ollamatools/path_validator.py 'In Progress' 'Merging into validators module'"
87
88print_command "./copy_refactor_file.sh ollamatools/url_validator.py ollamatools/core/utils/validators.py"
89print_note "Merge the URL validator into the validators module"
90print_command "./update_mapping_status.sh ollamatools/url_validator.py 'In Progress' 'Merging into validators module'"
91
92print_step "3" "Implement the Configuration Management"
93print_command "./copy_refactor_file.sh ollamatools/config.py ollamatools/core/config.py"
94print_note "Refactor the configuration management according to the simplification plan"
95print_command "./update_mapping_status.sh ollamatools/config.py 'In Progress' 'Refactoring configuration management'"
96
97print_step "4" "Implement the Security System"
98print_command "./copy_refactor_file.sh ollamatools/security.py ollamatools/core/security.py"
99print_note "Refactor the security system according to the simplification plan"
100print_command "./update_mapping_status.sh ollamatools/security.py 'In Progress' 'Refactoring security system'"
101
102print_step "5" "Implement the Base Agent"
103print_command "./copy_refactor_file.sh ollamatools/agent.py ollamatools/core/agent.py"
104print_note "Refactor the base agent according to the simplification plan"
105print_command "./update_mapping_status.sh ollamatools/agent.py 'In Progress' 'Refactoring base agent'"
106
107print_step "6" "Test the Core Infrastructure"
108print_command "./test_refactored_code.sh test_agent.py"
109print_note "Run tests for the agent to ensure it works correctly"
110print_command "./test_refactored_code.sh test_config.py"
111print_note "Run tests for the configuration to ensure it works correctly"
112print_command "./test_refactored_code.sh test_security.py"
113print_note "Run tests for the security system to ensure it works correctly"
114
115# Phase 3: Plugin System
116print_header "Phase 3: Plugin System"
117
118print_step "1" "Implement the Plugin Base"
119print_command "./copy_refactor_file.sh ollamatools/plugins/base.py ollamatools/plugins/base.py"
120print_note "Create the plugin base interfaces"
121print_command "./update_mapping_status.sh ollamatools/plugins/base.py 'In Progress' 'Creating plugin base interfaces'"
122
123print_step "2" "Implement the Plugin Registry"
124print_command "./copy_refactor_file.sh ollamatools/plugins/registry.py ollamatools/plugins/registry.py"
125print_note "Refactor the plugin registry according to the simplification plan"
126print_command "./update_mapping_status.sh ollamatools/plugins/registry.py 'In Progress' 'Refactoring plugin registry'"
127
128print_step "3" "Implement the Plugin Error Handling"
129print_command "./copy_refactor_file.sh ollamatools/plugins/errors.py ollamatools/plugins/errors.py"
130print_note "Refactor the plugin error handling according to the simplification plan"
131print_command "./update_mapping_status.sh ollamatools/plugins/errors.py 'In Progress' 'Refactoring plugin error handling'"
132
133print_step "4" "Test the Plugin System"
134print_command "./test_refactored_code.sh test_plugin_system.py"
135print_note "Run tests for the plugin system to ensure it works correctly"
136
137# Phase 4: Standard Plugins
138print_header "Phase 4: Standard Plugins"
139
140print_step "1" "Implement the Search Plugin"
141print_command "./copy_refactor_file.sh ollamatools/agent/modular/search.py ollamatools/plugins/standard/search.py"
142print_note "Refactor the search plugin according to the simplification plan"
143print_command "./update_mapping_status.sh ollamatools/agent/modular/search.py 'In Progress' 'Refactoring search plugin'"
144
145print_step "2" "Implement the Commands Plugin"
146print_command "./copy_refactor_file.sh ollamatools/agent/modular/commands.py ollamatools/plugins/standard/commands.py"
147print_note "Refactor the commands plugin according to the simplification plan"
148print_command "./update_mapping_status.sh ollamatools/agent/modular/commands.py 'In Progress' 'Refactoring commands plugin'"
149
150print_step "3" "Implement the Conversation Plugins"
151print_command "./copy_refactor_file.sh ollamatools/plugins/conversation_history.py ollamatools/plugins/standard/conversation/history.py"
152print_note "Refactor the conversation history plugin according to the simplification plan"
153print_command "./update_mapping_status.sh ollamatools/plugins/conversation_history.py 'In Progress' 'Refactoring conversation history plugin'"
154
155print_command "./copy_refactor_file.sh ollamatools/plugins/conversation_context.py ollamatools/plugins/standard/conversation/context.py"
156print_note "Refactor the conversation context plugin according to the simplification plan"
157print_command "./update_mapping_status.sh ollamatools/plugins/conversation_context.py 'In Progress' 'Refactoring conversation context plugin'"
158
159print_step "4" "Implement the Feedback Plugin"
160print_command "./copy_refactor_file.sh ollamatools/agent/modular/feedback.py ollamatools/plugins/standard/feedback.py"
161print_note "Refactor the feedback plugin according to the simplification plan"
162print_command "./update_mapping_status.sh ollamatools/agent/modular/feedback.py 'In Progress' 'Refactoring feedback plugin'"
163
164print_step "5" "Implement the Issues Plugin"
165print_command "./copy_refactor_file.sh ollamatools/agent/modular/issues.py ollamatools/plugins/standard/issues.py"
166print_note "Refactor the issues plugin according to the simplification plan"
167print_command "./update_mapping_status.sh ollamatools/agent/modular/issues.py 'In Progress' 'Refactoring issues plugin'"
168
169print_step "6" "Test the Standard Plugins"
170print_command "./test_refactored_code.sh test_search_plugin.py"
171print_note "Run tests for the search plugin to ensure it works correctly"
172print_command "./test_refactored_code.sh test_conversation_history_plugin.py"
173print_note "Run tests for the conversation history plugin to ensure it works correctly"
174print_command "./test_refactored_code.sh test_conversation_context_plugin.py"
175print_note "Run tests for the conversation context plugin to ensure it works correctly"
176
177# Phase 5: Tool Plugins
178print_header "Phase 5: Tool Plugins"
179
180print_step "1" "Implement the System Tools Plugin"
181print_command "./copy_refactor_file.sh ollamatools/tools/system_tools.py ollamatools/plugins/tools/system.py"
182print_note "Refactor the system tools plugin according to the simplification plan"
183print_command "./update_mapping_status.sh ollamatools/tools/system_tools.py 'In Progress' 'Refactoring system tools plugin'"
184
185print_step "2" "Implement the Web Tools Plugin"
186print_command "./copy_refactor_file.sh ollamatools/tools/web_tools.py ollamatools/plugins/tools/web.py"
187print_note "Refactor the web tools plugin according to the simplification plan"
188print_command "./update_mapping_status.sh ollamatools/tools/web_tools.py 'In Progress' 'Refactoring web tools plugin'"
189
190print_step "3" "Implement the Documentation Tools Plugin"
191print_command "./copy_refactor_file.sh ollamatools/tools/doc_tools.py ollamatools/plugins/tools/doc.py"
192print_note "Refactor the documentation tools plugin according to the simplification plan"
193print_command "./update_mapping_status.sh ollamatools/tools/doc_tools.py 'In Progress' 'Refactoring documentation tools plugin'"
194
195print_step "4" "Test the Tool Plugins"
196print_command "./test_refactored_code.sh test_system_tools.py"
197print_note "Run tests for the system tools plugin to ensure it works correctly"
198print_command "./test_refactored_code.sh test_web_tools.py"
199print_note "Run tests for the web tools plugin to ensure it works correctly"
200print_command "./test_refactored_code.sh test_doc_tools.py"
201print_note "Run tests for the documentation tools plugin to ensure it works correctly"
202
203# Phase 6: Integration Plugins
204print_header "Phase 6: Integration Plugins"
205
206print_step "1" "Implement the Matrix Integration"
207print_command "./copy_refactor_file.sh ollamatools/tools/matrix_tools.py ollamatools/plugins/integrations/matrix/client.py"
208print_note "Refactor the Matrix client according to the simplification plan"
209print_command "./update_mapping_status.sh ollamatools/tools/matrix_tools.py 'In Progress' 'Refactoring Matrix client'"
210
211print_command "./copy_refactor_file.sh ollamatools/tools/enhanced_matrix_core.py ollamatools/plugins/integrations/matrix/client.py"
212print_note "Merge the enhanced Matrix client into the Matrix client module"
213print_command "./update_mapping_status.sh ollamatools/tools/enhanced_matrix_core.py 'In Progress' 'Merging into Matrix client module'"
214
215print_command "./copy_refactor_file.sh ollamatools/tools/enhanced_matrix_history.py ollamatools/plugins/integrations/matrix/history.py"
216print_note "Refactor the Matrix history tracking according to the simplification plan"
217print_command "./update_mapping_status.sh ollamatools/tools/enhanced_matrix_history.py 'In Progress' 'Refactoring Matrix history tracking'"
218
219print_command "./copy_refactor_file.sh ollamatools/tools/enhanced_matrix_search.py ollamatools/plugins/integrations/matrix/search.py"
220print_note "Refactor the Matrix search integration according to the simplification plan"
221print_command "./update_mapping_status.sh ollamatools/tools/enhanced_matrix_search.py 'In Progress' 'Refactoring Matrix search integration'"
222
223print_command "./copy_refactor_file.sh ollamatools/tools/matrix_rate_limit_patch.py ollamatools/plugins/integrations/matrix/rate_limit.py"
224print_note "Refactor the Matrix rate limiting according to the simplification plan"
225print_command "./update_mapping_status.sh ollamatools/tools/matrix_rate_limit_patch.py 'In Progress' 'Refactoring Matrix rate limiting'"
226
227print_step "2" "Implement the OpenSearch Integration"
228print_command "./copy_refactor_file.sh ollamatools/opensearch_web_tools.py ollamatools/plugins/integrations/opensearch/tools.py"
229print_note "Refactor the OpenSearch web tools according to the simplification plan"
230print_command "./update_mapping_status.sh ollamatools/opensearch_web_tools.py 'In Progress' 'Refactoring OpenSearch web tools'"
231
232print_step "3" "Test the Integration Plugins"
233print_command "./test_refactored_code.sh test_matrix_tools.py"
234print_note "Run tests for the Matrix integration to ensure it works correctly"
235print_command "./test_refactored_code.sh test_opensearch_error_handling.py"
236print_note "Run tests for the OpenSearch integration to ensure it works correctly"
237
238# Phase 7: Agent Implementations
239print_header "Phase 7: Agent Implementations"
240
241print_step "1" "Implement the Basic Agent"
242print_command "./copy_refactor_file.sh examples/basic_agent.py ollamatools/agents/basic.py"
243print_note "Refactor the basic agent according to the simplification plan"
244print_command "./update_mapping_status.sh examples/basic_agent.py 'In Progress' 'Refactoring basic agent'"
245
246print_step "2" "Implement the Modular Agent"
247print_command "./copy_refactor_file.sh ollamatools/agent/modular/agent.py ollamatools/agents/modular.py"
248print_note "Refactor the modular agent according to the simplification plan"
249print_command "./update_mapping_status.sh ollamatools/agent/modular/agent.py 'In Progress' 'Refactoring modular agent'"
250
251print_step "3" "Implement the Extended Agent"
252print_command "./copy_refactor_file.sh ollamatools/agent/extended/agent.py ollamatools/agents/extended.py"
253print_note "Refactor the extended agent according to the simplification plan"
254print_command "./update_mapping_status.sh ollamatools/agent/extended/agent.py 'In Progress' 'Refactoring extended agent'"
255
256print_step "4" "Test the Agent Implementations"
257print_command "./test_refactored_code.sh test_agent.py"
258print_note "Run tests for the agents to ensure they work correctly"
259print_command "./test_refactored_code.sh test_plugin_enabled_agent.py"
260print_note "Run tests for the plugin-enabled agents to ensure they work correctly"
261
262# Phase 8: Configuration System
263print_header "Phase 8: Configuration System"
264
265print_step "1" "Implement the Configuration Schemas"
266print_command "./copy_refactor_file.sh ollamatools/config/schemas.py ollamatools/config/schemas.py"
267print_note "Refactor the configuration schemas according to the simplification plan"
268print_command "./update_mapping_status.sh ollamatools/config/schemas.py 'In Progress' 'Refactoring configuration schemas'"
269
270print_step "2" "Implement the Configuration Loader"
271print_command "./copy_refactor_file.sh ollamatools/config/core_config.py ollamatools/config/loader.py"
272print_note "Refactor the configuration loader according to the simplification plan"
273print_command "./update_mapping_status.sh ollamatools/config/core_config.py 'In Progress' 'Refactoring configuration loader'"
274
275print_step "3" "Implement the Default Configurations"
276print_command "./copy_refactor_file.sh ollamatools/config/plugin_config.py ollamatools/config/defaults.py"
277print_note "Refactor the default configurations according to the simplification plan"
278print_command "./update_mapping_status.sh ollamatools/config/plugin_config.py 'In Progress' 'Refactoring default configurations'"
279
280print_step "4" "Test the Configuration System"
281print_command "./test_refactored_code.sh test_config.py"
282print_note "Run tests for the configuration system to ensure it works correctly"
283
284# Phase 9: Final Testing
285print_header "Phase 9: Final Testing"
286
287print_step "1" "Run all tests"
288print_command "cd ../ollamatools-new && python -m pytest"
289print_note "Run all tests to ensure everything works correctly"
290
291print_step "2" "Run integration tests"
292print_command "./test_refactored_code.sh test_integration.py"
293print_note "Run integration tests to ensure all components work together correctly"
294
295print_step "3" "Run performance tests"
296print_command "./test_refactored_code.sh test_performance.py"
297print_note "Run performance tests to ensure the refactored code performs well"
298
299# Phase 10: Finalization
300print_header "Phase 10: Finalization"
301
302print_step "1" "Finalize the refactoring"
303print_command "./finalize_refactoring.sh"
304print_note "This will create a backup of the original codebase, rename the original directory to ollamatools-legacy, and rename the new directory to ollamatools-project"
305print_warning "Make sure all tests pass before running this command!"
306
307print_header "Refactoring Complete!"
308echo -e "${GREEN}Congratulations! You have successfully refactored the OllamaTools project.${NC}"
309echo ""
310echo "The refactored codebase is now available at ../ollamatools-project"
311echo "The original codebase is available at ../ollamatools-legacy"
312echo "A backup of the original codebase is available at ../ollamatools-backup-<timestamp>"
313echo ""
314echo -e "${YELLOW}Note: You may need to update any references to the project paths in your environment.${NC}"
ollamatools-project/setup_refactored_structure.sh +232 -0
……@@ -0,0 +1,232 @@
1#!/bin/bash
2
3# Script to set up the directory structure for the refactored OllamaTools project
4# This creates a parallel directory structure alongside the current one
5
6# Define the parent directory and new project directory
7PARENT_DIR=".."
8NEW_PROJECT_DIR="$PARENT_DIR/ollamatools-new"
9
10# Create the new project directory
11echo "Creating new project directory: $NEW_PROJECT_DIR"
12mkdir -p "$NEW_PROJECT_DIR"
13
14# Copy essential files from the current project
15echo "Copying essential files from current project..."
16cp ../ollamatools-project/README.md "$NEW_PROJECT_DIR/"
17cp ../ollamatools-project/setup.py "$NEW_PROJECT_DIR/"
18cp ../ollamatools-project/requirements.txt "$NEW_PROJECT_DIR/"
19cp ../ollamatools-project/CHANGELOG.md "$NEW_PROJECT_DIR/"
20cp ../ollamatools-project/LICENSE "$NEW_PROJECT_DIR/" 2>/dev/null || echo "LICENSE file not found, skipping"
21cp ../ollamatools-project/.gitignore "$NEW_PROJECT_DIR/" 2>/dev/null || echo ".gitignore file not found, skipping"
22
23# Create the main package directory
24mkdir -p "$NEW_PROJECT_DIR/ollamatools"
25touch "$NEW_PROJECT_DIR/ollamatools/__init__.py"
26touch "$NEW_PROJECT_DIR/ollamatools/version.py"
27
28# Create the directory structure according to the simplification plan
29echo "Creating directory structure according to the simplification plan..."
30
31# Core module
32mkdir -p "$NEW_PROJECT_DIR/ollamatools/core/utils"
33touch "$NEW_PROJECT_DIR/ollamatools/core/__init__.py"
34touch "$NEW_PROJECT_DIR/ollamatools/core/agent.py"
35touch "$NEW_PROJECT_DIR/ollamatools/core/config.py"
36touch "$NEW_PROJECT_DIR/ollamatools/core/errors.py"
37touch "$NEW_PROJECT_DIR/ollamatools/core/security.py"
38touch "$NEW_PROJECT_DIR/ollamatools/core/utils/__init__.py"
39touch "$NEW_PROJECT_DIR/ollamatools/core/utils/logger.py"
40touch "$NEW_PROJECT_DIR/ollamatools/core/utils/validators.py"
41touch "$NEW_PROJECT_DIR/ollamatools/core/utils/formatting.py"
42
43# Plugin system
44mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins"
45touch "$NEW_PROJECT_DIR/ollamatools/plugins/__init__.py"
46touch "$NEW_PROJECT_DIR/ollamatools/plugins/base.py"
47touch "$NEW_PROJECT_DIR/ollamatools/plugins/registry.py"
48touch "$NEW_PROJECT_DIR/ollamatools/plugins/errors.py"
49
50# Standard plugins
51mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation"
52touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/__init__.py"
53touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/search.py"
54touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/commands.py"
55touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation/__init__.py"
56touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation/history.py"
57touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation/context.py"
58touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/conversation/analysis.py"
59touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/feedback.py"
60touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/issues.py"
61touch "$NEW_PROJECT_DIR/ollamatools/plugins/standard/function_calling.py"
62
63# Tool plugins
64mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins/tools"
65touch "$NEW_PROJECT_DIR/ollamatools/plugins/tools/__init__.py"
66touch "$NEW_PROJECT_DIR/ollamatools/plugins/tools/system.py"
67touch "$NEW_PROJECT_DIR/ollamatools/plugins/tools/web.py"
68touch "$NEW_PROJECT_DIR/ollamatools/plugins/tools/doc.py"
69
70# Integration plugins
71mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix"
72mkdir -p "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/opensearch"
73touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/__init__.py"
74touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/__init__.py"
75touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/client.py"
76touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/commands.py"
77touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/config.py"
78touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/search.py"
79touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/history.py"
80touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/matrix/rate_limit.py"
81touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/opensearch/__init__.py"
82touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/opensearch/client.py"
83touch "$NEW_PROJECT_DIR/ollamatools/plugins/integrations/opensearch/tools.py"
84
85# Agent implementations
86mkdir -p "$NEW_PROJECT_DIR/ollamatools/agents"
87touch "$NEW_PROJECT_DIR/ollamatools/agents/__init__.py"
88touch "$NEW_PROJECT_DIR/ollamatools/agents/basic.py"
89touch "$NEW_PROJECT_DIR/ollamatools/agents/modular.py"
90touch "$NEW_PROJECT_DIR/ollamatools/agents/extended.py"
91
92# Configuration system
93mkdir -p "$NEW_PROJECT_DIR/ollamatools/config"
94touch "$NEW_PROJECT_DIR/ollamatools/config/__init__.py"
95touch "$NEW_PROJECT_DIR/ollamatools/config/schemas.py"
96touch "$NEW_PROJECT_DIR/ollamatools/config/loader.py"
97touch "$NEW_PROJECT_DIR/ollamatools/config/defaults.py"
98
99# Command-line interface
100mkdir -p "$NEW_PROJECT_DIR/ollamatools/cli"
101touch "$NEW_PROJECT_DIR/ollamatools/cli/__init__.py"
102touch "$NEW_PROJECT_DIR/ollamatools/cli/main.py"
103
104# Tests directory
105mkdir -p "$NEW_PROJECT_DIR/tests"
106touch "$NEW_PROJECT_DIR/tests/__init__.py"
107
108# Examples directory
109mkdir -p "$NEW_PROJECT_DIR/examples"
110touch "$NEW_PROJECT_DIR/examples/README.md"
111
112# Docs directory
113mkdir -p "$NEW_PROJECT_DIR/docs"
114touch "$NEW_PROJECT_DIR/docs/README.md"
115
116# Create a README file in the new project directory explaining the refactoring
117cat > "$NEW_PROJECT_DIR/REFACTORING.md" << 'EOF'
118# OllamaTools Refactoring
119
120This directory contains the refactored version of the OllamaTools project, following the simplification plan outlined in the original project.
121
122## Directory Structure
123
124The refactored project follows this directory structure:
125
126```
127ollamatools/
128├── core/ # Core functionality
129│ ├── __init__.py
130│ ├── agent.py # Base agent implementation
131│ ├── config.py # Configuration management
132│ ├── errors.py # Error definitions and handling
133│ ├── security.py # Security utilities and permission system
134│ └── utils/
135│ ├── __init__.py
136│ ├── logger.py # Logging utilities
137│ ├── validators.py # Input validation (combines command/path/url validators)
138│ └── formatting.py # Text formatting utilities
139│
140├── plugins/ # Plugin system and implementations
141│ ├── __init__.py
142│ ├── base.py # Base plugin interfaces
143│ ├── registry.py # Plugin discovery and management
144│ ├── errors.py # Plugin-specific error handling
145│ ├── standard/ # Standard plugins
146│ │ ├── __init__.py
147│ │ ├── search.py # Search functionality
148│ │ ├── commands.py # Command management
149│ │ ├── conversation/
150│ │ │ ├── __init__.py
151│ │ │ ├── history.py # Conversation history tracking
152│ │ │ ├── context.py # Conversation context management
153│ │ │ └── analysis.py # Topic analysis and metrics
154│ │ ├── feedback.py # Feedback collection
155│ │ ├── issues.py # Issue tracking
156│ │ └── function_calling.py # Semantic function calling
157│ │
158│ ├── tools/ # Tool plugins
159│ │ ├── __init__.py
160│ │ ├── system.py # System tools (file operations, commands)
161│ │ ├── web.py # Web tools (HTTP requests, web searches)
162│ │ └── doc.py # Documentation tools
163│ │
164│ └── integrations/ # Integration plugins
165│ ├── __init__.py
166│ ├── matrix/
167│ │ ├── __init__.py
168│ │ ├── client.py # Matrix client core
169│ │ ├── commands.py # Matrix-specific commands
170│ │ ├── config.py # Matrix configuration
171│ │ ├── search.py # Matrix search integration
172│ │ ├── history.py # Matrix history tracking
173│ │ └── rate_limit.py # Rate limiting for Matrix
174│ └── opensearch/
175│ ├── __init__.py
176│ ├── client.py # OpenSearch client
177│ └── tools.py # OpenSearch web tools
178│
179├── agents/ # Agent implementations
180│ ├── __init__.py
181│ ├── basic.py # Basic agent implementation
182│ ├── modular.py # Modular agent implementation
183│ └── extended.py # Extended agent implementation
184│
185├── config/ # Configuration system
186│ ├── __init__.py
187│ ├── schemas.py # Configuration schemas
188│ ├── loader.py # Configuration loading/validation
189│ └── defaults.py # Default configurations
190│
191├── cli/ # Command-line interface
192│ ├── __init__.py
193│ └── main.py # Command-line interface
194│
195├── __init__.py # Package initialization
196└── version.py # Version information
197```
198
199## Implementation Strategy
200
201The implementation follows the phased approach outlined in the simplification plan:
202
2031. Core Infrastructure
2042. Plugin System
2053. Standard Plugins
2064. Tool Plugins
2075. Integration Plugins
2086. Agent Implementations
2097. Configuration System
2108. Command-Line Interface
211
212## Using the Original Codebase as Reference
213
214The original codebase (`../ollamatools-project/`) should be used as a reference when implementing the refactored version. This allows for:
215
216- Understanding the current functionality
217- Ensuring feature parity
218- Reusing code where appropriate
219- Verifying correctness of the refactored implementation
220
221## Testing
222
223Tests should be written for each component as it is implemented. The tests should verify that the refactored implementation maintains the same functionality as the original.
224
225## Migration
226
227Once the refactored implementation is complete and tested, the original codebase can be archived and the refactored version can be moved to the main project directory.
228EOF
229
230echo "Directory structure setup complete!"
231echo "New project directory: $NEW_PROJECT_DIR"
232echo "You can now begin implementing the refactored project using the original codebase as a reference."
ollamatools-project/test_refactored_code.sh +133 -0
……@@ -0,0 +1,133 @@
1#!/bin/bash
2
3# Script to run tests for both the original and refactored codebases
4# This helps ensure that the refactored code maintains the same functionality
5
6# Define the parent directory and project directories
7PARENT_DIR=".."
8ORIGINAL_DIR="$PARENT_DIR/ollamatools-project"
9NEW_DIR="$PARENT_DIR/ollamatools-new"
10
11# Function to display usage information
12usage() {
13 echo "Usage: $0 [options] <test_file>"
14 echo ""
15 echo "Arguments:"
16 echo " <test_file> Path to the test file to run (relative to the tests directory)"
17 echo ""
18 echo "Options:"
19 echo " -o, --original Run tests only on the original codebase"
20 echo " -n, --new Run tests only on the new codebase"
21 echo " -h, --help Display this help message"
22 echo ""
23 echo "If no options are provided, tests will be run on both codebases."
24 echo ""
25 echo "Example:"
26 echo " $0 test_agent.py # Run test_agent.py on both codebases"
27 echo " $0 --original test_agent.py # Run test_agent.py only on the original codebase"
28 echo " $0 --new test_agent.py # Run test_agent.py only on the new codebase"
29 exit 1
30}
31
32# Parse command line arguments
33RUN_ORIGINAL=true
34RUN_NEW=true
35TEST_FILE=""
36
37while [[ $# -gt 0 ]]; do
38 case $1 in
39 -o|--original)
40 RUN_ORIGINAL=true
41 RUN_NEW=false
42 shift
43 ;;
44 -n|--new)
45 RUN_ORIGINAL=false
46 RUN_NEW=true
47 shift
48 ;;
49 -h|--help)
50 usage
51 ;;
52 *)
53 TEST_FILE="$1"
54 shift
55 ;;
56 esac
57done
58
59# Check if a test file was provided
60if [ -z "$TEST_FILE" ]; then
61 echo "Error: No test file provided."
62 usage
63fi
64
65# Function to run tests
66run_tests() {
67 local dir="$1"
68 local test_file="$2"
69 local name="$3"
70
71 echo "Running tests on $name codebase..."
72 echo "Directory: $dir"
73 echo "Test file: $test_file"
74 echo ""
75
76 # Check if the test file exists
77 if [ ! -f "$dir/tests/$test_file" ]; then
78 echo "Error: Test file $dir/tests/$test_file does not exist."
79 return 1
80 fi
81
82 # Run the tests
83 cd "$dir" && python -m pytest "tests/$test_file" -v
84 local result=$?
85
86 if [ $result -eq 0 ]; then
87 echo "Tests passed on $name codebase."
88 else
89 echo "Tests failed on $name codebase."
90 fi
91
92 echo ""
93 return $result
94}
95
96# Run tests on the original codebase
97ORIGINAL_RESULT=0
98if [ "$RUN_ORIGINAL" = true ]; then
99 run_tests "$ORIGINAL_DIR" "$TEST_FILE" "original"
100 ORIGINAL_RESULT=$?
101fi
102
103# Run tests on the new codebase
104NEW_RESULT=0
105if [ "$RUN_NEW" = true ]; then
106 run_tests "$NEW_DIR" "$TEST_FILE" "new"
107 NEW_RESULT=$?
108fi
109
110# Print summary
111echo "Test Summary:"
112if [ "$RUN_ORIGINAL" = true ]; then
113 if [ $ORIGINAL_RESULT -eq 0 ]; then
114 echo " Original codebase: PASSED"
115 else
116 echo " Original codebase: FAILED"
117 fi
118fi
119
120if [ "$RUN_NEW" = true ]; then
121 if [ $NEW_RESULT -eq 0 ]; then
122 echo " New codebase: PASSED"
123 else
124 echo " New codebase: FAILED"
125 fi
126fi
127
128# Exit with an error if any tests failed
129if [ $ORIGINAL_RESULT -ne 0 ] || [ $NEW_RESULT -ne 0 ]; then
130 exit 1
131fi
132
133exit 0
ollamatools-project/update_mapping_status.sh +72 -0
……@@ -0,0 +1,72 @@
1#!/bin/bash
2
3# Script to update the status of a file in the file mapping
4# This helps track progress of the refactoring process
5
6# Define the file mapping
7MAPPING_FILE="file_mapping.md"
8
9# Function to display usage information
10usage() {
11 echo "Usage: $0 <original_file> <status> [notes]"
12 echo ""
13 echo "Arguments:"
14 echo " <original_file> Path to the original file in the codebase (relative to the project root)"
15 echo " <status> New status (e.g., 'Completed', 'In Progress', 'Pending')"
16 echo " [notes] Optional notes to add or replace existing notes"
17 echo ""
18 echo "Example:"
19 echo " $0 ollamatools/agent.py Completed 'Refactored with updated imports'"
20 echo ""
21 echo "This will update the status of ollamatools/agent.py to 'Completed' and add the notes."
22 exit 1
23}
24
25# Check if the correct number of arguments is provided
26if [ $# -lt 2 ]; then
27 usage
28fi
29
30ORIGINAL_FILE="$1"
31NEW_STATUS="$2"
32NEW_NOTES="${3:-}"
33
34# Check if the file mapping exists
35if [ ! -f "$MAPPING_FILE" ]; then
36 echo "Error: File mapping $MAPPING_FILE does not exist."
37 echo "Run ./generate_file_mapping.sh first to create it."
38 exit 1
39fi
40
41# Escape special characters in the original file path for use in sed
42ESCAPED_FILE=$(echo "$ORIGINAL_FILE" | sed 's/\//\\\//g')
43
44# Check if the file exists in the mapping
45if ! grep -q "\`$ORIGINAL_FILE\`" "$MAPPING_FILE"; then
46 echo "Error: File $ORIGINAL_FILE not found in the mapping."
47 echo "Available files:"
48 grep -o "\`[^']*\`" "$MAPPING_FILE" | grep -v "New File" | sort | uniq
49 exit 1
50fi
51
52# If notes are provided, update both status and notes
53if [ -n "$NEW_NOTES" ]; then
54 # Escape special characters in the notes for use in sed
55 ESCAPED_NOTES=$(echo "$NEW_NOTES" | sed 's/\//\\\//g')
56
57 # Update the status and notes
58 sed -i "s/| \`$ESCAPED_FILE\` | \`[^']*\` | [^|]* | [^|]* |/| \`$ESCAPED_FILE\` | \`&\` | $NEW_STATUS | $ESCAPED_NOTES |/" "$MAPPING_FILE"
59else
60 # Update only the status
61 sed -i "s/| \`$ESCAPED_FILE\` | \`[^']*\` | [^|]* |/| \`$ESCAPED_FILE\` | \`&\` | $NEW_STATUS |/" "$MAPPING_FILE"
62fi
63
64echo "Updated status of $ORIGINAL_FILE to '$NEW_STATUS'."
65if [ -n "$NEW_NOTES" ]; then
66 echo "Updated notes to: $NEW_NOTES"
67fi
68
69# Display the updated entry
70echo ""
71echo "Updated entry:"
72grep "\`$ORIGINAL_FILE\`" "$MAPPING_FILE"