OverviewFilesCommitsIssuesDocsReleasesPeers

ollama_semantic_function_calling commits / a1fbfdf

implementing phase 5
a1fbfdf6252dec8252a175910a9aaf539bb1c5ba
by fleeky.ashatron <fleeky@7rnx.net> · 2025-04-10T16:50:05.000Z

Changed files (9) +1105 -23

ollamatools-project/example_doc.html +27 -0
……@@ -0,0 +1,27 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <title>OllamaTools Documentation Example</title>
5 <meta charset="UTF-8">
6 <meta name="viewport" content="width=device-width, initial-scale=1.0">
7</head>
8<body>
9 <h1>OllamaTools Documentation Example</h1>
10 <h2>Introduction</h2>
11<p>This is a sample document generated using OllamaTools' documentation utilities.</p>
12<h2>Features</h2>
13<ul>
14<li>Markdown formatting</li>
15<li>HTML conversion</li>
16<li>Simple and easy to use</li>
17</ul>
18<h2>Usage</h2>
19<p>```python
20from ollamatools.tools.doc_tools import generate_markdown</p>
21<h1>Generate markdown</h1>
22<p>doc = generate_markdown(content, title, format_type='markdown')</p>
23<h1>Generate HTML</h1>
24<p>html = generate_markdown(content, title, format_type='html')
25```</p>
26</body>
27</html>
ollamatools-project/example_doc.md +24 -0
……@@ -0,0 +1,24 @@
1# OllamaTools Documentation Example
2
3
4## Introduction
5
6This is a sample document generated using OllamaTools' documentation utilities.
7
8## Features
9
10- Markdown formatting
11- HTML conversion
12- Simple and easy to use
13
14## Usage
15
16```python
17from ollamatools.tools.doc_tools import generate_markdown
18
19# Generate markdown
20doc = generate_markdown(content, title, format_type='markdown')
21
22# Generate HTML
23html = generate_markdown(content, title, format_type='html')
24```
ollamatools-project/examples/documentation_tools.py +111 -0
……@@ -0,0 +1,111 @@
1#!/usr/bin/env python3
2"""
3Example script demonstrating the use of documentation tools.
4
5This script shows how to use the documentation tools to generate
6formatted documentation in both Markdown and HTML formats, as well
7as extracting context and summarizing text.
8"""
9
10from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
11
12
13def main():
14 """Run the documentation tools example."""
15 print("=== OllamaTools Documentation Tools Example ===\n")
16
17 # Part 1: Document Generation
18 print("\n--- Document Generation Example ---\n")
19
20 # Sample content for documentation
21 content = """
22## Introduction
23
24This is a sample document generated using OllamaTools' documentation utilities.
25
26## Features
27
28- Markdown formatting
29- HTML conversion
30- Simple and easy to use
31
32## Usage
33
34```python
35from ollamatools.tools.doc_tools import generate_markdown
36
37# Generate markdown
38doc = generate_markdown(content, title, format_type='markdown')
39
40# Generate HTML
41html = generate_markdown(content, title, format_type='html')
42```
43"""
44
45 # Generate markdown document
46 title = "OllamaTools Documentation Example"
47 markdown_doc = generate_markdown(content, title, format_type='markdown')
48 print("Markdown Document:")
49 print("-----------------")
50 print(markdown_doc[:200] + "...\n")
51
52 # Generate HTML document
53 html_doc = generate_markdown(content, title, format_type='html')
54 print("HTML Document (first 200 chars):")
55 print("--------------------------------")
56 print(html_doc[:200] + "...\n")
57
58 # Save the documents to files
59 with open("example_doc.md", "w") as f:
60 f.write(markdown_doc)
61 print("Saved markdown document to 'example_doc.md'")
62
63 with open("example_doc.html", "w") as f:
64 f.write(html_doc)
65 print("Saved HTML document to 'example_doc.html'\n")
66
67 # Part 2: Context Extraction
68 print("\n--- Context Extraction Example ---\n")
69
70 # Sample documents
71 documents = [
72 "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.",
73 "JavaScript is a programming language that is one of the core technologies of the World Wide Web, alongside HTML and CSS.",
74 "Rust is a multi-paradigm, general-purpose programming language that emphasizes performance, type safety, and concurrency.",
75 "OllamaTools is a Python library that extends Ollama with semantic function calling capabilities."
76 ]
77
78 # Extract context based on a query
79 query = "programming language features"
80 context = extract_context(query, documents)
81
82 print(f"Query: '{query}'")
83 print("Extracted Context:")
84 print("-----------------")
85 print(context)
86 print()
87
88 # Part 3: Text Summarization
89 print("\n--- Text Summarization Example ---\n")
90
91 # Sample text to summarize
92 long_text = """
93 Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically-typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming. It is often described as a "batteries included" language due to its comprehensive standard library.
94
95 Guido van Rossum began working on Python in the late 1980s, as a successor to the ABC programming language, and first released it in 1991 as Python 0.9.0. Python 2.0 was released in 2000 and introduced new features, such as list comprehensions and a garbage collection system using reference counting. Python 3.0 was released in 2008 and was a major revision of the language that is not completely backward-compatible. Python 2 was discontinued with version 2.7.18 in 2020.
96
97 Python consistently ranks as one of the most popular programming languages. Python is used in many application domains, such as web development, artificial intelligence, machine learning, data science, scientific computing, computer vision, computational statistics, computational biology, computer-aided design, computational economics, computational physics, SEO, automation, and information security.
98 """
99
100 # Summarize the text
101 summary = summarize_text(long_text, max_length=200)
102
103 print("Original Text Length:", len(long_text))
104 print("Summary Length:", len(summary))
105 print("Summary:")
106 print("--------")
107 print(summary)
108
109
110if __name__ == "__main__":
111 main()
ollamatools-project/examples/secure_doc_tools.py +161 -0
……@@ -0,0 +1,161 @@
1#!/usr/bin/env python3
2"""
3Example script demonstrating secure documentation tools with permission system integration.
4
5This script shows how to use the documentation tools with the permission system
6to ensure security when processing potentially sensitive documents.
7"""
8
9from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
10from ollamatools.security import PermissionManager
11
12
13def main():
14 """Run the secure documentation tools example."""
15 print("=== OllamaTools Secure Documentation Tools Example ===\n")
16
17 # Initialize the permission manager
18 permission_manager = PermissionManager(
19 prompt_enabled=True,
20 sudo_always_prompt=True
21 )
22
23 print("Permission manager initialized. All documentation operations will require permission.\n")
24
25 # Part 1: Secure Document Generation
26 print("\n--- Secure Document Generation Example ---\n")
27
28 # Sample content for documentation
29 content = """
30## Secure Document Generation
31
32This document demonstrates the integration of OllamaTools' documentation
33utilities with the permission system for enhanced security.
34
35## Security Features
36
37- Permission prompts before document generation
38- Secure handling of potentially sensitive content
39- Integration with OllamaTools permission system
40
41## Usage with Permission System
42
43```python
44from ollamatools.tools.doc_tools import generate_markdown
45from ollamatools.security import PermissionManager
46
47# Initialize permission manager
48permission_manager = PermissionManager()
49
50# Generate markdown with permission checks
51doc = generate_markdown(content, title,
52 format_type='markdown',
53 permission_manager=permission_manager)
54```
55"""
56
57 # Generate markdown document with permission check
58 print("Attempting to generate markdown document...\n")
59 print("You will be prompted for permission to generate the document.")
60 print("(Choose 'y' to allow, 'n' to deny)\n")
61
62 try:
63 title = "Secure Documentation Example"
64 markdown_doc = generate_markdown(
65 content,
66 title,
67 format_type='markdown',
68 permission_manager=permission_manager
69 )
70 print("\nPermission granted! Document generated successfully.\n")
71 print("First 200 characters of the document:")
72 print("-" * 50)
73 print(markdown_doc[:200] + "...")
74
75 # Save the document if permission was granted
76 with open("secure_doc.md", "w") as f:
77 f.write(markdown_doc)
78 print("\nSaved markdown document to 'secure_doc.md'")
79
80 except PermissionError as e:
81 print(f"\nDocument generation failed: {e}")
82
83 # Part 2: Secure Context Extraction
84 print("\n\n--- Secure Context Extraction Example ---\n")
85
86 # Sample documents (potentially sensitive information)
87 documents = [
88 "The system password is 'secure123'. It should be changed every 30 days.",
89 "Python is a programming language that is widely used in data science.",
90 "The admin credentials are stored in the config file at /etc/app/config.json.",
91 "JavaScript is a programming language used for web development."
92 ]
93
94 query = "programming language"
95
96 print(f"Attempting to extract context about '{query}' from documents...\n")
97 print("You will be prompted for permission to extract context from documents.")
98 print("(Choose 'y' to allow, 'n' to deny)\n")
99
100 try:
101 context = extract_context(
102 query,
103 documents,
104 max_length=500,
105 permission_manager=permission_manager
106 )
107 print("\nPermission granted! Context extracted successfully.\n")
108 print("Extracted Context:")
109 print("-" * 50)
110 print(context)
111
112 except PermissionError as e:
113 print(f"\nContext extraction failed: {e}")
114
115 # Part 3: Secure Text Summarization
116 print("\n\n--- Secure Text Summarization Example ---\n")
117
118 # Sample text to summarize (potentially sensitive information)
119 long_text = """
120 Project XYZ Security Analysis Report - CONFIDENTIAL
121
122 The security audit of Project XYZ revealed several critical vulnerabilities that need immediate attention.
123 The most severe issue is the unencrypted storage of user credentials in the database. This violates our security policy and industry standards.
124 We also found that the API endpoints are not properly protected against SQL injection attacks. This could allow malicious users to access sensitive data.
125 The authentication mechanism uses outdated hashing algorithms (MD5) which are considered insecure by modern standards.
126 The session management system doesn't implement proper timeout controls, allowing sessions to remain active indefinitely.
127 All API requests are transmitted over HTTP rather than HTTPS, exposing data to potential man-in-the-middle attacks.
128
129 We recommend implementing the following changes immediately:
130 1. Encrypt all stored credentials using industry-standard algorithms.
131 2. Implement parameterized queries for all database operations.
132 3. Update the password hashing to use bcrypt or Argon2.
133 4. Implement a session timeout of 30 minutes for inactive users.
134 5. Enforce HTTPS for all API communications.
135
136 These issues must be addressed before the product can be deployed to production.
137 """
138
139 print("Attempting to summarize confidential text...\n")
140 print("You will be prompted for permission to summarize the text.")
141 print("(Choose 'y' to allow, 'n' to deny)\n")
142
143 try:
144 summary = summarize_text(
145 long_text,
146 max_length=150,
147 permission_manager=permission_manager
148 )
149 print("\nPermission granted! Text summarized successfully.\n")
150 print("Original Text Length:", len(long_text))
151 print("Summary Length:", len(summary))
152 print("Summary:")
153 print("-" * 50)
154 print(summary)
155
156 except PermissionError as e:
157 print(f"\nText summarization failed: {e}")
158
159
160if __name__ == "__main__":
161 main()
ollamatools-project/ollamatools/tools/__init__.py +27 -15
……@@ -1,28 +1,40 @@
11"""
2Tools package for OllamaTools.
2OllamaTools function libraries.
33
4This package provides various tools for the OllamaTools library, including:
5- System tools: Execute commands and file operations
6- Web tools: Make web requests and process responses
7- Documentation tools: Generate and process documentation
4This package contains various tool modules for use with OllamaTools:
5- system_tools: Execute Linux commands and file operations
6- web_tools: Make web requests and process responses
7- doc_tools: Generate and process documentation
88"""
99
10# Import and expose system tools
11from .system_tools import execute_command, read_file, write_file, find_files
10from ollamatools.tools.system_tools import (
11 execute_command,
12 read_file,
13 write_file,
14 find_files
15)
1216
13# Import and expose web tools
14from .web_tools import fetch_url, get_weather, search_web
17from ollamatools.tools.web_tools import (
18 fetch_url,
19 get_weather,
20 search_web
21)
22
23from ollamatools.tools.doc_tools import (
24 generate_markdown,
25 extract_context,
26 summarize_text
27)
1528
16# Export all tools
1729__all__ = [
18 # System tools
19 'execute_command',
20 'read_file',
30 'execute_command',
31 'read_file',
2132 'write_file',
2233 'find_files',
23
24 # Web tools
2534 'fetch_url',
2635 'get_weather',
2736 'search_web',
37 'generate_markdown',
38 'extract_context',
39 'summarize_text'
2840]
ollamatools-project/ollamatools/tools/doc_tools.py +258 -8
……@@ -1,28 +1,278 @@
11"""
2Documentation tools for OllamaTools.
2Documentation Tools for OllamaTools.
33
4This module contains functions for creating and processing documentation.
4This module provides utilities for generating and processing documentation.
5It includes tools for creating formatted documentation, extracting relevant
6context from documents, and summarizing text.
7
8All functions support integration with the permission system for security.
59"""
610
11import markdown
12import re
13from collections import Counter
14from typing import Optional, Dict, Any, List
15
716
817def generate_markdown(
918 content: str,
1019 title: str,
11 format_type: str = 'markdown'
20 format_type: str = 'markdown',
21 permission_manager = None
1222) -> str:
1323 """
1424 Generate formatted documentation.
1525
1626 Args:
17 content: Document content
27 content: Document content to format
1828 title: Document title
1929 format_type: Output format ('markdown' or 'html')
30 permission_manager: Optional permission manager for security checks
2031
2132 Returns:
2233 str: Formatted documentation
34
35 Raises:
36 ValueError: If format_type is not supported
37 PermissionError: If permission is denied
2338 """
24 # Placeholder implementation
39 # Permission check
40 if permission_manager is not None:
41 args = {
42 'content_preview': content[:50] + ('...' if len(content) > 50 else ''),
43 'title': title,
44 'format_type': format_type
45 }
46 if not permission_manager.request_permission('generate_markdown', args):
47 raise PermissionError("Permission denied to generate markdown document")
48
49 # Input validation
50 if not content or not isinstance(content, str):
51 raise ValueError("Content must be a non-empty string")
52
53 if not title or not isinstance(title, str):
54 raise ValueError("Title must be a non-empty string")
55
56 if format_type not in ['markdown', 'html']:
57 raise ValueError("Format type must be 'markdown' or 'html'")
58
59 # Create the formatted document
2560 if format_type == 'markdown':
26 return f"# {title}\n\n{content}"
27 else:
28 return f"<h1>{title}</h1><p>{content}</p>"
61 # For markdown, we'll create a simple markdown document with a title
62 formatted_doc = f"# {title}\n\n{content}"
63 return formatted_doc
64 else: # html
65 # For HTML, we'll convert the markdown to HTML
66 html_content = markdown.markdown(content)
67 formatted_doc = f"""<!DOCTYPE html>
68<html>
69<head>
70 <title>{title}</title>
71 <meta charset="UTF-8">
72 <meta name="viewport" content="width=device-width, initial-scale=1.0">
73</head>
74<body>
75 <h1>{title}</h1>
76 {html_content}
77</body>
78</html>"""
79 return formatted_doc
80
81
82def extract_context(
83 query: str,
84 documents: List[str],
85 max_length: int = 1000,
86 permission_manager = None
87) -> str:
88 """
89 Extract relevant context from documents based on a query.
90
91 This function extracts passages from documents that are most relevant to the query
92 based on keyword matching. It prioritizes sentences containing query keywords.
93
94 Args:
95 query: Search query to find relevant contexts
96 documents: List of document contents
97 max_length: Maximum total length of extracted context
98 permission_manager: Optional permission manager for security checks
99
100 Returns:
101 str: Extracted context
102
103 Raises:
104 ValueError: If inputs are invalid
105 PermissionError: If permission is denied
106 """
107 # Permission check
108 if permission_manager is not None:
109 args = {
110 'query': query,
111 'documents_count': len(documents) if documents else 0,
112 'max_length': max_length
113 }
114 if not permission_manager.request_permission('extract_context', args):
115 raise PermissionError("Permission denied to extract context from documents")
116
117 # Input validation
118 if not query or not isinstance(query, str):
119 raise ValueError("Query must be a non-empty string")
120
121 if not documents or not isinstance(documents, list):
122 raise ValueError("Documents must be a non-empty list of strings")
123
124 if not all(isinstance(doc, str) for doc in documents):
125 raise ValueError("All documents must be strings")
126
127 if not max_length or not isinstance(max_length, int) or max_length <= 0:
128 raise ValueError("Max length must be a positive integer")
129
130 # Extract keywords from the query
131 # Normalize by removing common words and punctuation
132 query = query.lower()
133 query_words = re.findall(r'\b\w+\b', query)
134
135 # Remove common stop words
136 stop_words = {'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are', 'of', 'to', 'in',
137 'for', 'with', 'by', 'at', 'on', 'from'}
138 keywords = [word for word in query_words if word not in stop_words]
139
140 if not keywords:
141 # Use the original words if all were stop words
142 keywords = query_words
143
144 # Find sentences in documents containing keywords
145 relevant_sentences = []
146 for doc in documents:
147 # Split documents into sentences (simple heuristic)
148 sentences = re.split(r'(?<=[.!?])\s+', doc)
149
150 for sentence in sentences:
151 # Check for keyword presence
152 sentence_lower = sentence.lower()
153 sentence_words = set(re.findall(r'\b\w+\b', sentence_lower))
154
155 # Calculate relevance score as the number of keywords found
156 keywords_found = sum(1 for keyword in keywords if keyword in sentence_words)
157
158 if keywords_found > 0:
159 relevant_sentences.append((sentence, keywords_found))
160
161 # Sort sentences by relevance (number of keywords found)
162 relevant_sentences.sort(key=lambda x: x[1], reverse=True)
163
164 # Build context up to max_length
165 context = []
166 current_length = 0
167
168 for sentence, _ in relevant_sentences:
169 if current_length + len(sentence) + 1 <= max_length: # +1 for space
170 context.append(sentence)
171 current_length += len(sentence) + 1
172 else:
173 # If we can't fit the next sentence, stop
174 break
175
176 # If no relevant sentences found, return empty string
177 if not context:
178 return ""
179
180 return " ".join(context)
181
182
183def summarize_text(
184 text: str,
185 max_length: int = 500,
186 permission_manager = None
187) -> str:
188 """
189 Generate a summary of text using extractive summarization.
190
191 This function extracts the most important sentences from the text
192 based on word frequency analysis.
193
194 Args:
195 text: Text to summarize
196 max_length: Maximum summary length
197 permission_manager: Optional permission manager for security checks
198
199 Returns:
200 str: Text summary
201
202 Raises:
203 ValueError: If inputs are invalid
204 PermissionError: If permission is denied
205 """
206 # Permission check
207 if permission_manager is not None:
208 args = {
209 'text_preview': text[:50] + ('...' if len(text) > 50 else ''),
210 'text_length': len(text),
211 'max_length': max_length
212 }
213 if not permission_manager.request_permission('summarize_text', args):
214 raise PermissionError("Permission denied to summarize text")
215
216 # Input validation
217 if not text or not isinstance(text, str):
218 raise ValueError("Text must be a non-empty string")
219
220 if not max_length or not isinstance(max_length, int) or max_length <= 0:
221 raise ValueError("Max length must be a positive integer")
222
223 # If text is already shorter than max_length, return it as is
224 if len(text) <= max_length:
225 return text
226
227 # Split text into sentences
228 sentences = re.split(r'(?<=[.!?])\s+', text)
229
230 # Handle case with only one sentence
231 if len(sentences) <= 1:
232 return text[:max_length] + "..." if len(text) > max_length else text
233
234 # Extract all words, lowercase them, and remove stop words
235 stop_words = {'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are', 'of', 'to', 'in',
236 'for', 'with', 'by', 'at', 'on', 'from'}
237
238 words = []
239 for sentence in sentences:
240 sentence_words = re.findall(r'\b\w+\b', sentence.lower())
241 words.extend([word for word in sentence_words if word not in stop_words])
242
243 # Count word frequencies
244 word_freq = Counter(words)
245
246 # Score sentences based on word frequencies
247 sentence_scores = []
248 for i, sentence in enumerate(sentences):
249 sentence_words = re.findall(r'\b\w+\b', sentence.lower())
250 score = sum(word_freq[word] for word in sentence_words if word not in stop_words)
251 # Normalize by sentence length to avoid bias towards longer sentences
252 if len(sentence_words) > 0:
253 score = score / len(sentence_words)
254 sentence_scores.append((i, sentence, score))
255
256 # Sort sentences by score
257 sentence_scores.sort(key=lambda x: x[2], reverse=True)
258
259 # Select top sentences and restore original order
260 top_sentences = sentence_scores[:len(sentences)//3 + 1] # Take ~1/3 of sentences
261 top_sentences.sort(key=lambda x: x[0]) # Sort by original position
262
263 # Build summary up to max_length
264 summary = []
265 current_length = 0
266
267 for _, sentence, _ in top_sentences:
268 if current_length + len(sentence) + 1 <= max_length: # +1 for space
269 summary.append(sentence)
270 current_length += len(sentence) + 1
271 else:
272 break
273
274 # If no sentences fit, take the first sentence and truncate
275 if not summary:
276 return sentences[0][:max_length] + "..."
277
278 return " ".join(summary)
ollamatools-project/secure_doc.md +28 -0
……@@ -0,0 +1,28 @@
1# Secure Documentation Example
2
3
4## Secure Document Generation
5
6This document demonstrates the integration of OllamaTools' documentation
7utilities with the permission system for enhanced security.
8
9## Security Features
10
11- Permission prompts before document generation
12- Secure handling of potentially sensitive content
13- Integration with OllamaTools permission system
14
15## Usage with Permission System
16
17```python
18from ollamatools.tools.doc_tools import generate_markdown
19from ollamatools.security import PermissionManager
20
21# Initialize permission manager
22permission_manager = PermissionManager()
23
24# Generate markdown with permission checks
25doc = generate_markdown(content, title,
26 format_type='markdown',
27 permission_manager=permission_manager)
28```
ollamatools-project/tests/test_doc_tools.py +468 -0
……@@ -0,0 +1,468 @@
1"""
2Tests for the documentation tools.
3"""
4
5import pytest
6from unittest.mock import MagicMock, patch
7from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
8
9
10# Tests for generate_markdown
11def test_generate_markdown_with_markdown_format():
12 """Test markdown generation with markdown format."""
13 content = "This is a test content.\n\nWith multiple paragraphs."
14 title = "Test Document"
15
16 result = generate_markdown(content, title, format_type='markdown')
17
18 assert result.startswith("# Test Document")
19 assert "This is a test content." in result
20 assert "With multiple paragraphs." in result
21
22
23def test_generate_markdown_with_html_format():
24 """Test markdown generation with HTML format."""
25 content = "This is a test content.\n\nWith multiple paragraphs."
26 title = "Test Document"
27
28 result = generate_markdown(content, title, format_type='html')
29
30 assert "<title>Test Document</title>" in result
31 assert "<h1>Test Document</h1>" in result
32 assert "This is a test content." in result
33 assert "With multiple paragraphs." in result
34 assert "<!DOCTYPE html>" in result
35
36
37def test_generate_markdown_with_invalid_format():
38 """Test markdown generation with an invalid format."""
39 content = "This is a test content."
40 title = "Test Document"
41
42 with pytest.raises(ValueError) as excinfo:
43 generate_markdown(content, title, format_type='invalid')
44
45 assert "Format type must be 'markdown' or 'html'" in str(excinfo.value)
46
47
48def test_generate_markdown_with_empty_content():
49 """Test markdown generation with empty content."""
50 content = ""
51 title = "Test Document"
52
53 with pytest.raises(ValueError) as excinfo:
54 generate_markdown(content, title)
55
56 assert "Content must be a non-empty string" in str(excinfo.value)
57
58
59def test_generate_markdown_with_empty_title():
60 """Test markdown generation with empty title."""
61 content = "This is a test content."
62 title = ""
63
64 with pytest.raises(ValueError) as excinfo:
65 generate_markdown(content, title)
66
67 assert "Title must be a non-empty string" in str(excinfo.value)
68
69
70def test_generate_markdown_with_permission_granted():
71 """Test markdown generation with permission granted."""
72 content = "This is a test content."
73 title = "Test Document"
74
75 # Create a mock permission manager
76 permission_manager = MagicMock()
77 permission_manager.request_permission.return_value = True
78
79 result = generate_markdown(content, title, permission_manager=permission_manager)
80
81 assert result.startswith("# Test Document")
82 permission_manager.request_permission.assert_called_once()
83
84
85def test_generate_markdown_with_permission_denied():
86 """Test markdown generation with permission denied."""
87 content = "This is a test content."
88 title = "Test Document"
89
90 # Create a mock permission manager
91 permission_manager = MagicMock()
92 permission_manager.request_permission.return_value = False
93
94 with pytest.raises(PermissionError) as excinfo:
95 generate_markdown(content, title, permission_manager=permission_manager)
96
97 assert "Permission denied" in str(excinfo.value)
98 permission_manager.request_permission.assert_called_once()
99
100
101# Tests for extract_context
102def test_extract_context_basic():
103 """Test basic context extraction."""
104 query = "python programming"
105 documents = [
106 "Python is a programming language. It is widely used in data science.",
107 "JavaScript is a web programming language. It runs in browsers.",
108 "Data science uses statistical methods to analyze data."
109 ]
110
111 result = extract_context(query, documents)
112
113 assert "Python is a programming language" in result
114 assert "JavaScript is a web programming language" in result
115 assert "Data science" not in result # No keyword match
116
117
118def test_extract_context_with_max_length():
119 """Test context extraction with max length limit."""
120 query = "programming language"
121 documents = [
122 "Python is a programming language. It is widely used in data science.",
123 "JavaScript is a web programming language. It runs in browsers.",
124 "Java is an object-oriented programming language. It is used for enterprise applications."
125 ]
126
127 # Set max_length to fit only the first two sentences
128 max_len = len("Python is a programming language. JavaScript is a web programming language.") + 5
129 result = extract_context(query, documents, max_length=max_len)
130
131 assert "Python is a programming language" in result
132 assert "JavaScript is a web programming language" in result
133 assert "Java is an object-oriented programming language" not in result
134
135
136def test_extract_context_empty_query():
137 """Test context extraction with empty query."""
138 query = ""
139 documents = ["Python is a programming language."]
140
141 with pytest.raises(ValueError) as excinfo:
142 extract_context(query, documents)
143
144 assert "Query must be a non-empty string" in str(excinfo.value)
145
146
147def test_extract_context_empty_documents():
148 """Test context extraction with empty documents."""
149 query = "python"
150 documents = []
151
152 with pytest.raises(ValueError) as excinfo:
153 extract_context(query, documents)
154
155 assert "Documents must be a non-empty list of strings" in str(excinfo.value)
156
157
158def test_extract_context_invalid_max_length():
159 """Test context extraction with invalid max length."""
160 query = "python"
161 documents = ["Python is a programming language."]
162
163 with pytest.raises(ValueError) as excinfo:
164 extract_context(query, documents, max_length=0)
165
166 assert "Max length must be a positive integer" in str(excinfo.value)
167
168
169def test_extract_context_with_permission_granted():
170 """Test context extraction with permission granted."""
171 query = "python"
172 documents = ["Python is a programming language."]
173
174 # Create a mock permission manager
175 permission_manager = MagicMock()
176 permission_manager.request_permission.return_value = True
177
178 result = extract_context(query, documents, permission_manager=permission_manager)
179
180 assert "Python is a programming language" in result
181 permission_manager.request_permission.assert_called_once()
182
183
184def test_extract_context_with_permission_denied():
185 """Test context extraction with permission denied."""
186 query = "python"
187 documents = ["Python is a programming language."]
188
189 # Create a mock permission manager
190 permission_manager = MagicMock()
191 permission_manager.request_permission.return_value = False
192
193 with pytest.raises(PermissionError) as excinfo:
194 extract_context(query, documents, permission_manager=permission_manager)
195
196 assert "Permission denied" in str(excinfo.value)
197 permission_manager.request_permission.assert_called_once()
198
199
200# Tests for summarize_text
201def test_summarize_text_basic():
202 """Test basic text summarization."""
203 text = """Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.
204 Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming.
205 Python is often described as a "batteries included" language due to its comprehensive standard library."""
206
207 result = summarize_text(text)
208
209 assert len(result) <= 500
210 assert "Python" in result
211 assert "programming" in result
212
213
214def test_summarize_text_short_input():
215 """Test summarization with text shorter than max_length."""
216 text = "Python is a programming language."
217
218 result = summarize_text(text)
219
220 assert result == text # Should return original text unchanged
221
222
223def test_summarize_text_single_sentence():
224 """Test summarization with a single sentence longer than max_length."""
225 text = "Python is a high-level, general-purpose programming language with design philosophy that emphasizes code readability and significant whitespace and a syntax that allows programmers to express concepts in fewer lines of code than might be possible in languages such as C++ or Java."
226 max_len = 50
227
228 result = summarize_text(text, max_length=max_len)
229
230 assert len(result) <= max_len + 3 # +3 for the "..."
231 assert result.endswith("...")
232
233
234def test_summarize_text_empty_text():
235 """Test summarization with empty text."""
236 text = ""
237
238 with pytest.raises(ValueError) as excinfo:
239 summarize_text(text)
240
241 assert "Text must be a non-empty string" in str(excinfo.value)
242
243
244def test_summarize_text_invalid_max_length():
245 """Test summarization with invalid max length."""
246 text = "Python is a programming language."
247
248 with pytest.raises(ValueError) as excinfo:
249 summarize_text(text, max_length=0)
250
251 assert "Max length must be a positive integer" in str(excinfo.value)
252
253
254def test_summarize_text_with_permission_granted():
255 """Test text summarization with permission granted."""
256 text = "Python is a programming language."
257
258 # Create a mock permission manager
259 permission_manager = MagicMock()
260 permission_manager.request_permission.return_value = True
261
262 result = summarize_text(text, permission_manager=permission_manager)
263
264 assert result == text # Short text should be returned as is
265 permission_manager.request_permission.assert_called_once()
266
267
268def test_summarize_text_with_permission_denied():
269 """Test text summarization with permission denied."""
270 text = "Python is a programming language."
271
272 # Create a mock permission manager
273 permission_manager = MagicMock()
274 permission_manager.request_permission.return_value = False
275
276 with pytest.raises(PermissionError) as excinfo:
277 summarize_text(text, permission_manager=permission_manager)
278
279 assert "Permission denied" in str(excinfo.value)
280 permission_manager.request_permission.assert_called_once()
281"""
282Tests for the documentation tools.
283"""
284
285import pytest
286from ollamatools.tools.doc_tools import generate_markdown, extract_context, summarize_text
287
288
289# Tests for generate_markdown
290def test_generate_markdown_with_markdown_format():
291 """Test markdown generation with markdown format."""
292 content = "This is a test content.\n\nWith multiple paragraphs."
293 title = "Test Document"
294
295 result = generate_markdown(content, title, format_type='markdown')
296
297 assert result.startswith("# Test Document")
298 assert "This is a test content." in result
299 assert "With multiple paragraphs." in result
300
301
302def test_generate_markdown_with_html_format():
303 """Test markdown generation with HTML format."""
304 content = "This is a test content.\n\nWith multiple paragraphs."
305 title = "Test Document"
306
307 result = generate_markdown(content, title, format_type='html')
308
309 assert "<title>Test Document</title>" in result
310 assert "<h1>Test Document</h1>" in result
311 assert "This is a test content." in result
312 assert "With multiple paragraphs." in result
313 assert "<!DOCTYPE html>" in result
314
315
316def test_generate_markdown_with_invalid_format():
317 """Test markdown generation with an invalid format."""
318 content = "This is a test content."
319 title = "Test Document"
320
321 with pytest.raises(ValueError) as excinfo:
322 generate_markdown(content, title, format_type='invalid')
323
324 assert "Format type must be 'markdown' or 'html'" in str(excinfo.value)
325
326
327def test_generate_markdown_with_empty_content():
328 """Test markdown generation with empty content."""
329 content = ""
330 title = "Test Document"
331
332 with pytest.raises(ValueError) as excinfo:
333 generate_markdown(content, title)
334
335 assert "Content must be a non-empty string" in str(excinfo.value)
336
337
338def test_generate_markdown_with_empty_title():
339 """Test markdown generation with empty title."""
340 content = "This is a test content."
341 title = ""
342
343 with pytest.raises(ValueError) as excinfo:
344 generate_markdown(content, title)
345
346 assert "Title must be a non-empty string" in str(excinfo.value)
347
348
349# Tests for extract_context
350def test_extract_context_basic():
351 """Test basic context extraction."""
352 query = "python programming"
353 documents = [
354 "Python is a programming language. It is widely used in data science.",
355 "JavaScript is a web programming language. It runs in browsers.",
356 "Data science uses statistical methods to analyze data."
357 ]
358
359 result = extract_context(query, documents)
360
361 assert "Python is a programming language" in result
362 assert "JavaScript is a web programming language" in result
363 assert "Data science" not in result # No keyword match
364
365
366def test_extract_context_with_max_length():
367 """Test context extraction with max length limit."""
368 query = "programming language"
369 documents = [
370 "Python is a programming language. It is widely used in data science.",
371 "JavaScript is a web programming language. It runs in browsers.",
372 "Java is an object-oriented programming language. It is used for enterprise applications."
373 ]
374
375 # Set max_length to fit only the first two sentences
376 max_len = len("Python is a programming language. JavaScript is a web programming language.") + 5
377 result = extract_context(query, documents, max_length=max_len)
378
379 assert "Python is a programming language" in result
380 assert "JavaScript is a web programming language" in result
381 assert "Java is an object-oriented programming language" not in result
382
383
384def test_extract_context_empty_query():
385 """Test context extraction with empty query."""
386 query = ""
387 documents = ["Python is a programming language."]
388
389 with pytest.raises(ValueError) as excinfo:
390 extract_context(query, documents)
391
392 assert "Query must be a non-empty string" in str(excinfo.value)
393
394
395def test_extract_context_empty_documents():
396 """Test context extraction with empty documents."""
397 query = "python"
398 documents = []
399
400 with pytest.raises(ValueError) as excinfo:
401 extract_context(query, documents)
402
403 assert "Documents must be a non-empty list of strings" in str(excinfo.value)
404
405
406def test_extract_context_invalid_max_length():
407 """Test context extraction with invalid max length."""
408 query = "python"
409 documents = ["Python is a programming language."]
410
411 with pytest.raises(ValueError) as excinfo:
412 extract_context(query, documents, max_length=0)
413
414 assert "Max length must be a positive integer" in str(excinfo.value)
415
416
417# Tests for summarize_text
418def test_summarize_text_basic():
419 """Test basic text summarization."""
420 text = """Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.
421 Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming.
422 Python is often described as a "batteries included" language due to its comprehensive standard library."""
423
424 result = summarize_text(text)
425
426 assert len(result) <= 500
427 assert "Python" in result
428 assert "programming" in result
429
430
431def test_summarize_text_short_input():
432 """Test summarization with text shorter than max_length."""
433 text = "Python is a programming language."
434
435 result = summarize_text(text)
436
437 assert result == text # Should return original text unchanged
438
439
440def test_summarize_text_single_sentence():
441 """Test summarization with a single sentence longer than max_length."""
442 text = "Python is a high-level, general-purpose programming language with design philosophy that emphasizes code readability and significant whitespace and a syntax that allows programmers to express concepts in fewer lines of code than might be possible in languages such as C++ or Java."
443 max_len = 50
444
445 result = summarize_text(text, max_length=max_len)
446
447 assert len(result) <= max_len + 3 # +3 for the "..."
448 assert result.endswith("...")
449
450
451def test_summarize_text_empty_text():
452 """Test summarization with empty text."""
453 text = ""
454
455 with pytest.raises(ValueError) as excinfo:
456 summarize_text(text)
457
458 assert "Text must be a non-empty string" in str(excinfo.value)
459
460
461def test_summarize_text_invalid_max_length():
462 """Test summarization with invalid max length."""
463 text = "Python is a programming language."
464
465 with pytest.raises(ValueError) as excinfo:
466 summarize_text(text, max_length=0)
467
468 assert "Max length must be a positive integer" in str(excinfo.value)
ollamatools-project/tree-struct +1 -0
……@@ -0,0 +1 @@
1tree -L 3 # Limit depth to 3 levels