Core LLM integration infrastructure to allow pgAdmin to connect to AI providers. #9641
* Core infrastructure for LLM integration. * Add support for a number of different AI generated reports on security, performance, and schema design on servers, databases, and schemas, as appropriate. * Add a Natural Language AI assistant to the Query Tool. * Add an AI Insights panel to the EXPLAIN tool in the Query Tool, to analyse and report on issues in query plans.
This commit is contained in:
242
docs/en_US/ai_tools.rst
Normal file
242
docs/en_US/ai_tools.rst
Normal file
@@ -0,0 +1,242 @@
|
||||
.. _ai_tools:
|
||||
|
||||
*******************
|
||||
`AI Reports`:index:
|
||||
*******************
|
||||
|
||||
**AI Reports** is a feature that provides AI-powered database analysis and insights
|
||||
using Large Language Models (LLMs). Use the *Tools → AI Reports* menu to access
|
||||
the various AI-powered reports.
|
||||
|
||||
The AI Reports feature allows you to:
|
||||
|
||||
* Generate security reports to identify potential security vulnerabilities and configuration issues.
|
||||
|
||||
* Create performance reports with optimization recommendations for queries and configurations.
|
||||
|
||||
* Perform design reviews to analyze database schema structure and suggest improvements.
|
||||
|
||||
**Prerequisites:**
|
||||
|
||||
Before using AI Reports, you must:
|
||||
|
||||
1. Ensure AI features are enabled in the server configuration (set ``LLM_ENABLED`` to ``True`` in ``config.py``).
|
||||
|
||||
2. Configure an LLM provider in :ref:`Preferences → AI <preferences>`.
|
||||
|
||||
**Note:**
|
||||
|
||||
* AI Reports using cloud providers (Anthropic, OpenAI) require an active internet connection.
|
||||
Local providers (Ollama, Docker Model Runner) do not require internet access.
|
||||
|
||||
* API usage may incur costs depending on your LLM provider's pricing model.
|
||||
Local providers (Ollama, Docker Model Runner) are free to use.
|
||||
|
||||
* The quality and accuracy of reports depend on the LLM provider and model configured.
|
||||
|
||||
|
||||
Configuring AI Reports
|
||||
**********************
|
||||
|
||||
To configure AI Reports, navigate to *File → Preferences → AI* (or click the *Settings*
|
||||
button and select *AI*).
|
||||
|
||||
.. image:: images/preferences_ai.png
|
||||
:alt: AI preferences
|
||||
:align: center
|
||||
|
||||
Select your preferred LLM provider from the dropdown:
|
||||
|
||||
**Anthropic**
|
||||
Use Claude models from Anthropic. Requires an Anthropic API key.
|
||||
|
||||
* **API Key File**: Path to a file containing your Anthropic API key (obtain from https://console.anthropic.com/).
|
||||
* **Model**: Select from available Claude models (e.g., claude-sonnet-4-20250514).
|
||||
|
||||
**OpenAI**
|
||||
Use GPT models from OpenAI. Requires an OpenAI API key.
|
||||
|
||||
* **API Key File**: Path to a file containing your OpenAI API key (obtain from https://platform.openai.com/).
|
||||
* **Model**: Select from available GPT models (e.g., gpt-4).
|
||||
|
||||
**Ollama**
|
||||
Use locally-hosted open-source models via Ollama. Requires a running Ollama instance.
|
||||
|
||||
* **API URL**: The URL of your Ollama server (default: http://localhost:11434).
|
||||
* **Model**: Enter the name of the Ollama model to use (e.g., llama2, mistral).
|
||||
|
||||
**Docker Model Runner**
|
||||
Use models running in Docker Desktop's built-in model runner (available in Docker Desktop 4.40+).
|
||||
No API key is required.
|
||||
|
||||
* **API URL**: The URL of the Docker Model Runner API (default: http://localhost:12434).
|
||||
* **Model**: Select from available models or enter a custom model name.
|
||||
|
||||
After configuring your provider, click *Save* to apply the changes.
|
||||
|
||||
|
||||
Security Reports
|
||||
****************
|
||||
|
||||
Security Reports analyze your PostgreSQL server, database, or schema for potential
|
||||
security vulnerabilities and configuration issues.
|
||||
|
||||
To generate a security report:
|
||||
|
||||
1. In the *Browser* tree, select a server, database, or schema.
|
||||
|
||||
2. Choose *Tools → AI Reports → Security* from the menu, or right-click the
|
||||
object and select *Security* from the context menu.
|
||||
|
||||
3. The report will be generated and displayed in a new tab.
|
||||
|
||||
.. image:: images/ai_security_report.png
|
||||
:alt: AI security report
|
||||
:align: center
|
||||
|
||||
**Security Report Scope:**
|
||||
|
||||
* **Server Level**: Analyzes server configuration, authentication settings, roles, and permissions.
|
||||
|
||||
* **Database Level**: Reviews database-specific security settings, roles with database access, and object permissions.
|
||||
|
||||
* **Schema Level**: Examines schema permissions, object ownership, and access controls.
|
||||
|
||||
Each report includes:
|
||||
|
||||
* **Security Findings**: Identified vulnerabilities or security concerns.
|
||||
|
||||
* **Risk Assessment**: Severity levels for each finding (Critical, High, Medium, Low).
|
||||
|
||||
* **Recommendations**: Specific actions to remediate security issues.
|
||||
|
||||
* **Best Practices**: General security recommendations for PostgreSQL.
|
||||
|
||||
|
||||
Performance Reports
|
||||
*******************
|
||||
|
||||
Performance Reports analyze query performance, configuration settings, and provide
|
||||
optimization recommendations.
|
||||
|
||||
To generate a performance report:
|
||||
|
||||
1. In the *Browser* tree, select a server or database.
|
||||
|
||||
2. Choose *Tools → AI Reports → Performance* from the menu, or right-click the
|
||||
object and select *Performance* from the context menu.
|
||||
|
||||
3. The report will be generated and displayed in a new tab.
|
||||
|
||||
**Performance Report Scope:**
|
||||
|
||||
* **Server Level**: Analyzes server configuration parameters, resource utilization, and overall server performance metrics.
|
||||
|
||||
* **Database Level**: Reviews database-specific configuration, query performance, index usage, and table statistics.
|
||||
|
||||
Each report includes:
|
||||
|
||||
* **Performance Metrics**: Key performance indicators and statistics.
|
||||
|
||||
* **Configuration Analysis**: Review of relevant configuration parameters.
|
||||
|
||||
* **Query Optimization**: Recommendations for improving slow queries.
|
||||
|
||||
* **Index Recommendations**: Suggestions for adding, removing, or modifying indexes.
|
||||
|
||||
* **Capacity Planning**: Resource utilization trends and recommendations.
|
||||
|
||||
|
||||
Design Review Reports
|
||||
*********************
|
||||
|
||||
Design Review Reports analyze your database schema structure and suggest
|
||||
improvements for normalization, naming conventions, and best practices.
|
||||
|
||||
To generate a design review report:
|
||||
|
||||
1. In the *Browser* tree, select a database or schema.
|
||||
|
||||
2. Choose *Tools → AI Reports → Design* from the menu, or right-click the
|
||||
object and select *Design* from the context menu.
|
||||
|
||||
3. The report will be generated and displayed in a new tab.
|
||||
|
||||
**Design Review Scope:**
|
||||
|
||||
* **Database Level**: Reviews overall database structure, schema organization, and cross-schema dependencies.
|
||||
|
||||
* **Schema Level**: Analyzes tables, views, functions, and other objects within the schema.
|
||||
|
||||
Each report includes:
|
||||
|
||||
* **Schema Structure Analysis**: Review of table structures, relationships, and constraints.
|
||||
|
||||
* **Normalization Review**: Recommendations for database normalization (1NF, 2NF, 3NF, etc.).
|
||||
|
||||
* **Naming Conventions**: Suggestions for consistent naming patterns.
|
||||
|
||||
* **Data Type Usage**: Review of data type choices and recommendations.
|
||||
|
||||
* **Index Design**: Analysis of indexing strategy.
|
||||
|
||||
* **Best Practices**: General PostgreSQL schema design recommendations.
|
||||
|
||||
|
||||
Working with Reports
|
||||
********************
|
||||
|
||||
All AI reports are displayed in a dedicated panel with the following features:
|
||||
|
||||
**Report Display**
|
||||
Reports are formatted as Markdown and rendered with syntax highlighting for SQL code.
|
||||
|
||||
**Toolbar Actions**
|
||||
|
||||
* **Stop** - Cancel the current report generation. This is useful if the report
|
||||
is taking too long or if you want to change parameters.
|
||||
|
||||
* **Regenerate** - Generate a new report for the same object. Useful when you
|
||||
want to get a fresh analysis or if data has changed.
|
||||
|
||||
* **Download** - Download the report as a Markdown (.md) file. The filename
|
||||
includes the report type, object name, and date for easy identification.
|
||||
|
||||
**Multiple Reports**
|
||||
You can generate and view multiple reports simultaneously. Each report opens in
|
||||
a new tab, allowing you to compare reports across different servers, databases,
|
||||
or schemas.
|
||||
|
||||
**Report Management**
|
||||
Each report tab can be closed individually by clicking the *X* in the tab.
|
||||
Panel titles show the object name and report type for easy identification.
|
||||
|
||||
**Copying Content**
|
||||
You can select and copy text from reports to use in documentation or share with
|
||||
your team.
|
||||
|
||||
|
||||
Troubleshooting
|
||||
***************
|
||||
|
||||
**"AI features are disabled in the server configuration"**
|
||||
The administrator has disabled AI features on the server. Contact your
|
||||
pgAdmin administrator to enable the ``LLM_ENABLED`` configuration option.
|
||||
|
||||
**"Please configure an LLM provider in Preferences"**
|
||||
You need to configure an LLM provider before using AI Reports. See *Configuring AI Reports* above.
|
||||
|
||||
**"Please connect to the server/database first"**
|
||||
You must establish a connection to the server or database before generating reports.
|
||||
|
||||
**API Connection Errors**
|
||||
* Verify your API key is correct (for Anthropic and OpenAI).
|
||||
* Check your internet connection (for cloud providers).
|
||||
* For Ollama, ensure the Ollama server is running and accessible.
|
||||
* For Docker Model Runner, ensure Docker Desktop 4.40+ is running with the model runner enabled.
|
||||
* Check that your firewall allows connections to the LLM provider's API.
|
||||
|
||||
**Report Generation Fails**
|
||||
* Check the pgAdmin logs for detailed error messages.
|
||||
* Verify the database connection is still active.
|
||||
* Ensure the selected model is available for your account/subscription.
|
||||
@@ -17,3 +17,4 @@ PL/SQL code.
|
||||
schema_diff
|
||||
erd_tool
|
||||
psql_tool
|
||||
ai_tools
|
||||
|
||||
BIN
docs/en_US/images/ai_security_report.png
Normal file
BIN
docs/en_US/images/ai_security_report.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 237 KiB |
BIN
docs/en_US/images/preferences_ai.png
Normal file
BIN
docs/en_US/images/preferences_ai.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 136 KiB |
BIN
docs/en_US/images/query_ai_assistant.png
Normal file
BIN
docs/en_US/images/query_ai_assistant.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 158 KiB |
BIN
docs/en_US/images/query_explain_ai_insights.png
Normal file
BIN
docs/en_US/images/query_explain_ai_insights.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 247 KiB |
@@ -156,6 +156,12 @@ Use the *Tools* menu to access the following options (in alphabetical order):
|
||||
+------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| *Search Objects...* | Click to open the :ref:`Search Objects... <search_objects>` and start searching any kind of objects in a database. |
|
||||
+------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| *AI Reports* | Click to access a submenu with AI-powered analysis options (requires :ref:`AI configuration <ai_tools>`): |
|
||||
| | |
|
||||
| | - *Security Report* - Generate an AI-powered security analysis for the selected server, database, or schema. |
|
||||
| | - *Performance Report* - Generate an AI-powered performance analysis for the selected server or database. |
|
||||
| | - *Design Report* - Generate an AI-powered design review for the selected database or schema. |
|
||||
+------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| *Add named restore point* | Click to open the :ref:`Add named restore point... <add_restore_point_dialog>` dialog to take a point-in-time snapshot of the current |
|
||||
| | server state. |
|
||||
+------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
@@ -27,6 +27,58 @@ The left pane of the *Preferences* tab displays a tree control; each node of
|
||||
the tree control provides access to options that are related to the node under
|
||||
which they are displayed.
|
||||
|
||||
The AI Node
|
||||
***********
|
||||
|
||||
Use preferences found in the *AI* node of the tree control to configure
|
||||
AI-powered features and LLM (Large Language Model) providers.
|
||||
|
||||
.. image:: images/preferences_ai.png
|
||||
:alt: Preferences AI section
|
||||
:align: center
|
||||
|
||||
**Note:** AI features must be enabled in the server configuration (``LLM_ENABLED = True``
|
||||
in ``config.py``) for these preferences to be available.
|
||||
|
||||
Use the fields on the *AI* panel to configure your LLM provider:
|
||||
|
||||
* Use the *Default Provider* drop-down to select your LLM provider. Options include:
|
||||
*Anthropic*, *OpenAI*, *Ollama*, or *Docker Model Runner*.
|
||||
|
||||
**Anthropic Settings:**
|
||||
|
||||
* Use the *API Key File* field to specify the path to a file containing your
|
||||
Anthropic API key.
|
||||
|
||||
* Use the *Model* field to select from the available Claude models. Click the
|
||||
refresh button to fetch the latest available models from Anthropic.
|
||||
|
||||
**OpenAI Settings:**
|
||||
|
||||
* Use the *API Key File* field to specify the path to a file containing your
|
||||
OpenAI API key.
|
||||
|
||||
* Use the *Model* field to select from the available GPT models. Click the
|
||||
refresh button to fetch the latest available models from OpenAI.
|
||||
|
||||
**Ollama Settings:**
|
||||
|
||||
* Use the *API URL* field to specify the Ollama server URL
|
||||
(default: ``http://localhost:11434``).
|
||||
|
||||
* Use the *Model* field to select from the available models or enter a custom
|
||||
model name (e.g., ``llama2``, ``mistral``). Click the refresh button to fetch
|
||||
the latest available models from your Ollama server.
|
||||
|
||||
**Docker Model Runner Settings:**
|
||||
|
||||
* Use the *API URL* field to specify the Docker Model Runner API URL
|
||||
(default: ``http://localhost:12434``). Available in Docker Desktop 4.40+.
|
||||
|
||||
* Use the *Model* field to select from the available models or enter a custom
|
||||
model name. Click the refresh button to fetch the latest available models
|
||||
from your Docker Model Runner.
|
||||
|
||||
The Browser Node
|
||||
****************
|
||||
|
||||
|
||||
@@ -32,8 +32,9 @@ The Query Tool features two panels:
|
||||
|
||||
* The upper panel displays the *SQL Editor*. You can use the panel to enter,
|
||||
edit, or execute a query or a script. It also shows the *History* tab which can be used
|
||||
to view the queries that have been executed in the session, and a *Scratch Pad*
|
||||
which can be used to hold text snippets during editing. If the Scratch Pad is
|
||||
to view the queries that have been executed in the session, a *Scratch Pad*
|
||||
which can be used to hold text snippets during editing, and an *AI Assistant*
|
||||
tab for generating SQL from natural language (when AI is configured). If the Scratch Pad is
|
||||
closed, it can be re-opened (or additional ones opened) by right-clicking in
|
||||
the SQL Editor and other panels and adding a new panel.
|
||||
* The lower panel displays the *Data Output* panel. The tabbed panel displays
|
||||
@@ -201,6 +202,49 @@ can be adjusted in ``config_local.py`` or ``config_system.py`` (see the
|
||||
`MAX_QUERY_HIST_STORED` value. See the :ref:`Deployment <deployment>` section
|
||||
for more information.
|
||||
|
||||
AI Assistant Panel
|
||||
******************
|
||||
|
||||
The *AI Assistant* tab provides a chat-style interface for generating SQL queries
|
||||
from natural language descriptions. This feature requires an AI provider to be
|
||||
configured in *Preferences > AI*. For configuration details, see the
|
||||
:ref:`preferences` documentation.
|
||||
|
||||
.. image:: images/query_ai_assistant.png
|
||||
:alt: Query tool AI Assistant panel
|
||||
:align: center
|
||||
|
||||
To use the AI Assistant:
|
||||
|
||||
1. Click on the *AI Assistant* tab in the upper panel, or use the *AI Assistant*
|
||||
toolbar button.
|
||||
2. Type a description of the SQL query you need in natural language.
|
||||
3. Press Enter or click the send button to submit your request.
|
||||
4. The AI will analyze your database schema and generate appropriate SQL.
|
||||
|
||||
The AI Assistant displays conversations with your messages and AI responses. When
|
||||
the AI generates SQL, it appears in a syntax-highlighted code block with action
|
||||
buttons:
|
||||
|
||||
* **Insert** - Insert the SQL at the current cursor position in the SQL Editor.
|
||||
* **Replace** - Replace all content in the SQL Editor with the generated SQL.
|
||||
* **Copy** - Copy the SQL to the clipboard.
|
||||
|
||||
The AI Assistant maintains conversation context, allowing you to refine queries
|
||||
iteratively. For example, you can ask for a query and then follow up with
|
||||
"also add a filter for active users" to modify the previous result.
|
||||
|
||||
**Tips for effective use:**
|
||||
|
||||
* Be specific about table and column names if you know them.
|
||||
* Describe the desired output format (e.g., "show count by category").
|
||||
* For complex queries, break down requirements step by step.
|
||||
* Use the *Clear* button to start a fresh conversation.
|
||||
|
||||
**Note:** The AI Assistant uses database schema inspection tools to understand
|
||||
your database structure. It supports SELECT, INSERT, UPDATE, DELETE, and DDL
|
||||
statements. All generated queries should be reviewed before execution.
|
||||
|
||||
The Data Output Panel
|
||||
*********************
|
||||
|
||||
@@ -335,6 +379,44 @@ If planner mis-estimated number of rows (actual vs planned) by
|
||||
:alt: Query tool explain plan statistics
|
||||
:align: center
|
||||
|
||||
* AI Insights
|
||||
|
||||
The *AI Insights* tab provides AI-powered analysis of query execution plans,
|
||||
identifying performance bottlenecks and suggesting optimizations. This tab is
|
||||
only available when an AI provider is configured in *Preferences > AI*.
|
||||
|
||||
.. image:: images/query_explain_ai_insights.png
|
||||
:alt: Query tool explain plan AI insights
|
||||
:align: center
|
||||
|
||||
When you switch to the AI Insights tab, the AI analyzes the execution plan and
|
||||
provides:
|
||||
|
||||
**Performance Bottlenecks** - Issues identified in the query plan, such as:
|
||||
|
||||
* Sequential scans on large tables that could benefit from indexes
|
||||
* Significant differences between estimated and actual row counts
|
||||
* Expensive sort or hash operations
|
||||
* Nested loops with high iteration counts
|
||||
|
||||
**Recommendations** - Concrete suggestions to improve query performance:
|
||||
|
||||
* Index creation statements with appropriate columns
|
||||
* ANALYZE commands to update table statistics
|
||||
* Configuration parameter adjustments
|
||||
* Query restructuring suggestions
|
||||
|
||||
Each recommendation that includes SQL (such as CREATE INDEX statements) has
|
||||
action buttons to *Copy* the SQL to the clipboard or *Insert* it into the
|
||||
Query Editor.
|
||||
|
||||
Click the *Regenerate* button to request a fresh analysis of the current plan.
|
||||
|
||||
**Note:** AI analysis is generated on-demand when you first click the AI Insights
|
||||
tab or when a new explain plan is generated while the tab is active. The analysis
|
||||
provides guidance but all suggested changes should be carefully evaluated before
|
||||
applying to production databases.
|
||||
|
||||
Messages Panel
|
||||
**************
|
||||
|
||||
|
||||
@@ -970,6 +970,68 @@ MAX_SERVER_TAGS_ALLOWED = 5
|
||||
|
||||
ON_DEMAND_LOG_COUNT = 10000
|
||||
|
||||
##########################################################################
|
||||
# AI/LLM Settings
|
||||
##########################################################################
|
||||
|
||||
# Master switch to enable/disable LLM features entirely.
|
||||
# When False, all AI/LLM features are disabled and cannot be enabled
|
||||
# by users through preferences. When True, users can configure their
|
||||
# preferred LLM provider in preferences.
|
||||
LLM_ENABLED = True
|
||||
|
||||
# Default LLM Provider
|
||||
# Specifies which LLM provider to use by default when LLM_ENABLED is True.
|
||||
# Users can override this in their preferences.
|
||||
# Valid values: 'anthropic', 'openai', 'ollama', 'docker', or '' (disabled)
|
||||
DEFAULT_LLM_PROVIDER = ''
|
||||
|
||||
# Anthropic Configuration
|
||||
# Path to a file containing the Anthropic API key. The file should contain
|
||||
# only the API key with no additional whitespace or formatting.
|
||||
# Default: ~/.anthropic-api-key
|
||||
ANTHROPIC_API_KEY_FILE = '~/.anthropic-api-key'
|
||||
|
||||
# The Anthropic model to use for AI features.
|
||||
# Examples: claude-sonnet-4-20250514, claude-3-5-haiku-20241022
|
||||
ANTHROPIC_API_MODEL = ''
|
||||
|
||||
# OpenAI Configuration
|
||||
# Path to a file containing the OpenAI API key. The file should contain
|
||||
# only the API key with no additional whitespace or formatting.
|
||||
# Default: ~/.openai-api-key
|
||||
OPENAI_API_KEY_FILE = '~/.openai-api-key'
|
||||
|
||||
# The OpenAI model to use for AI features.
|
||||
# Examples: gpt-4o, gpt-4o-mini, gpt-4-turbo
|
||||
OPENAI_API_MODEL = ''
|
||||
|
||||
# Ollama Configuration
|
||||
# URL for the Ollama API endpoint. Leave empty to disable Ollama.
|
||||
# Typical value: http://localhost:11434
|
||||
OLLAMA_API_URL = ''
|
||||
|
||||
# The Ollama model to use for AI features.
|
||||
# Examples: llama3.2, codellama, mistral
|
||||
OLLAMA_API_MODEL = ''
|
||||
|
||||
# Docker Model Runner Configuration
|
||||
# Docker Desktop 4.40+ includes a built-in model runner with an
|
||||
# OpenAI-compatible API. No API key is required.
|
||||
# URL for the Docker Model Runner API endpoint. Leave empty to disable.
|
||||
# Typical value: http://localhost:12434
|
||||
DOCKER_API_URL = ''
|
||||
|
||||
# The Docker Model Runner model to use for AI features.
|
||||
# Examples: ai/qwen3-coder, ai/llama3.2
|
||||
DOCKER_API_MODEL = ''
|
||||
|
||||
# Maximum Tool Iterations
|
||||
# The maximum number of tool call iterations allowed during an AI conversation.
|
||||
# This prevents runaway conversations that could consume excessive resources.
|
||||
# Users can override this in their preferences.
|
||||
MAX_LLM_TOOL_ITERATIONS = 20
|
||||
|
||||
#############################################################################
|
||||
# Patch the default config with custom config and other manipulations
|
||||
#############################################################################
|
||||
|
||||
@@ -52,7 +52,7 @@ module.exports = {
|
||||
],
|
||||
'testEnvironment': 'jsdom',
|
||||
'transformIgnorePatterns': [
|
||||
'[/\\\\]node_modules[/\\\\](?!react-dnd|dnd-core|@react-dnd|react-resize-detector|react-data-grid).+\\.(js|jsx|mjs|cjs|ts|tsx)$',
|
||||
'[/\\\\]node_modules[/\\\\](?!react-dnd|dnd-core|@react-dnd|react-resize-detector|react-data-grid|marked).+\\.(js|jsx|mjs|cjs|ts|tsx)$',
|
||||
'^.+\\.module\\.(css|sass|scss)$'
|
||||
]
|
||||
};
|
||||
|
||||
58
web/migrations/versions/add_tools_ai_perm_.py
Normal file
58
web/migrations/versions/add_tools_ai_perm_.py
Normal file
@@ -0,0 +1,58 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Add tools_ai permission to existing roles
|
||||
|
||||
Revision ID: add_tools_ai_perm
|
||||
Revises: 018e16dad6aa
|
||||
Create Date: 2025-12-01
|
||||
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision = 'add_tools_ai_perm'
|
||||
down_revision = '018e16dad6aa'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade():
|
||||
# Get metadata from current connection
|
||||
meta = sa.MetaData()
|
||||
meta.reflect(op.get_bind(), only=('role',))
|
||||
role_table = sa.Table('role', meta)
|
||||
|
||||
# Get all roles with permissions
|
||||
conn = op.get_bind()
|
||||
result = conn.execute(
|
||||
sa.select(role_table.c.id, role_table.c.permissions)
|
||||
.where(role_table.c.permissions.isnot(None))
|
||||
)
|
||||
|
||||
# Add tools_ai permission to each role that has permissions
|
||||
for row in result:
|
||||
role_id = row[0]
|
||||
permissions = row[1]
|
||||
if permissions:
|
||||
perms_list = permissions.split(',')
|
||||
if 'tools_ai' not in perms_list:
|
||||
perms_list.append('tools_ai')
|
||||
new_permissions = ','.join(perms_list)
|
||||
conn.execute(
|
||||
role_table.update()
|
||||
.where(role_table.c.id == role_id)
|
||||
.values(permissions=new_permissions)
|
||||
)
|
||||
|
||||
|
||||
def downgrade():
|
||||
# pgAdmin only upgrades, downgrade not implemented.
|
||||
pass
|
||||
@@ -109,6 +109,7 @@
|
||||
"convert-units": "^2.3.4",
|
||||
"date-fns": "^4.1.0",
|
||||
"diff-arrays-of-objects": "^1.1.8",
|
||||
"dompurify": "^3.2.4",
|
||||
"hotkeys-js": "^4.0.0",
|
||||
"html-to-image": "1.11.11",
|
||||
"immutability-helper": "^3.0.0",
|
||||
@@ -117,6 +118,7 @@
|
||||
"json-bignumber": "^1.0.1",
|
||||
"leaflet": "^1.9.4",
|
||||
"lodash": "4.*",
|
||||
"marked": "^17.0.1",
|
||||
"moment": "^2.29.4",
|
||||
"moment-timezone": "^0.6.0",
|
||||
"notificar": "^1.0.1",
|
||||
|
||||
@@ -44,7 +44,8 @@ export const BROWSER_PANELS = {
|
||||
USER_MANAGEMENT: 'id-user-management',
|
||||
IMPORT_EXPORT_SERVERS: 'id-import-export-servers',
|
||||
WELCOME_QUERY_TOOL: 'id-welcome-querytool',
|
||||
WELCOME_PSQL_TOOL: 'id-welcome-psql'
|
||||
WELCOME_PSQL_TOOL: 'id-welcome-psql',
|
||||
AI_REPORT_PREFIX: 'id-ai-report'
|
||||
};
|
||||
|
||||
|
||||
@@ -139,6 +140,7 @@ export const AllPermissionTypes = {
|
||||
TOOLS_MAINTENANCE: 'tools_maintenance',
|
||||
TOOLS_SCHEMA_DIFF: 'tools_schema_diff',
|
||||
TOOLS_GRANT_WIZARD: 'tools_grant_wizard',
|
||||
TOOLS_AI: 'tools_ai',
|
||||
STORAGE_ADD_FOLDER: 'storage_add_folder',
|
||||
STORAGE_REMOVE_FOLDER: 'storage_remove_folder'
|
||||
};
|
||||
|
||||
90
web/pgadmin/llm/README.md
Normal file
90
web/pgadmin/llm/README.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# pgAdmin LLM Integration
|
||||
|
||||
This module provides AI/LLM functionality for pgAdmin, including database security analysis, performance reports, and design reviews powered by large language models.
|
||||
|
||||
## Features
|
||||
|
||||
- **Security Reports**: Analyze database configurations for security issues
|
||||
- **Performance Reports**: Get optimization recommendations for databases
|
||||
- **Design Reviews**: Review schema design and structure
|
||||
- **Streaming Reports**: Real-time report generation with progress updates via Server-Sent Events (SSE)
|
||||
|
||||
## Supported LLM Providers
|
||||
|
||||
- **Anthropic Claude** (recommended)
|
||||
- **OpenAI GPT**
|
||||
- **Ollama** (local models)
|
||||
|
||||
## Configuration
|
||||
|
||||
Configure LLM providers in `config.py`:
|
||||
|
||||
- `DEFAULT_LLM_PROVIDER`: Set to 'anthropic', 'openai', or 'ollama'
|
||||
- `ANTHROPIC_API_KEY_FILE`: Path to file containing Anthropic API key
|
||||
- `OPENAI_API_KEY_FILE`: Path to file containing OpenAI API key
|
||||
- `OLLAMA_API_URL`: URL for Ollama server (e.g., 'http://localhost:11434')
|
||||
|
||||
If API keys are not found, the LLM features will be gracefully disabled.
|
||||
|
||||
## Testing
|
||||
|
||||
### Python Tests
|
||||
|
||||
The Python test suite uses pgAdmin's existing test framework based on `BaseTestGenerator` with the scenarios pattern.
|
||||
|
||||
Run all LLM tests:
|
||||
```bash
|
||||
cd web/regression
|
||||
python3 runtests.py --pkg llm
|
||||
```
|
||||
|
||||
Run specific test modules:
|
||||
```bash
|
||||
python3 runtests.py --pkg llm --modules test_llm_status
|
||||
python3 runtests.py --pkg llm --modules test_report_endpoints
|
||||
```
|
||||
|
||||
### JavaScript Tests
|
||||
|
||||
The JavaScript test suite uses Jest with React Testing Library.
|
||||
|
||||
Run all JavaScript tests (including LLM tests):
|
||||
```bash
|
||||
cd web
|
||||
yarn run test:js
|
||||
```
|
||||
|
||||
Run only LLM JavaScript tests:
|
||||
```bash
|
||||
cd web
|
||||
yarn run test:js-once -- llm
|
||||
```
|
||||
|
||||
### Test Coverage
|
||||
|
||||
The tests use mocking to avoid requiring actual LLM API credentials. All external dependencies (utility functions, report generators) are mocked, allowing the tests to run in CI/CD environments without any API keys configured.
|
||||
|
||||
Test files:
|
||||
- `tests/test_llm_status.py` - Tests LLM client initialization and status endpoint
|
||||
- `tests/test_report_endpoints.py` - Tests report generation endpoints at server, database, and schema levels
|
||||
- `regression/javascript/llm/AIReport.spec.js` - Tests React component for report display
|
||||
|
||||
## Architecture
|
||||
|
||||
- `client.py` - LLM client abstraction layer supporting multiple providers
|
||||
- `reports/` - Report generation system
|
||||
- `generator.py` - Main report generation logic
|
||||
- `security.py` - Security analysis prompts and logic
|
||||
- `performance.py` - Performance analysis prompts and logic
|
||||
- `design.py` - Design review prompts and logic
|
||||
- `views.py` - Flask endpoints for reports and chat
|
||||
- `static/js/AIReport.jsx` - React component for displaying reports with dark mode support
|
||||
|
||||
## Usage
|
||||
|
||||
Access AI reports through the pgAdmin browser tree:
|
||||
1. Right-click on a server, database, or schema
|
||||
2. Select "AI Analysis" submenu
|
||||
3. Choose report type (Security, Performance, or Design)
|
||||
4. View streaming report generation with progress updates
|
||||
5. Download reports as markdown files
|
||||
1853
web/pgadmin/llm/__init__.py
Normal file
1853
web/pgadmin/llm/__init__.py
Normal file
File diff suppressed because it is too large
Load Diff
190
web/pgadmin/llm/chat.py
Normal file
190
web/pgadmin/llm/chat.py
Normal file
@@ -0,0 +1,190 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""LLM chat functionality with database tool integration.
|
||||
|
||||
This module provides high-level functions for running LLM conversations
|
||||
that can use database tools to query and inspect PostgreSQL databases.
|
||||
"""
|
||||
|
||||
import json
|
||||
from typing import Optional
|
||||
|
||||
from pgadmin.llm.client import get_llm_client, is_llm_available
|
||||
from pgadmin.llm.models import Message, StopReason
|
||||
from pgadmin.llm.tools import DATABASE_TOOLS, execute_tool, DatabaseToolError
|
||||
from pgadmin.llm.utils import get_max_tool_iterations
|
||||
|
||||
|
||||
# Default system prompt for database assistant
|
||||
DEFAULT_SYSTEM_PROMPT = (
|
||||
"You are a PostgreSQL database assistant integrated into pgAdmin 4. "
|
||||
"You have access to tools that allow you to query the database and "
|
||||
"inspect its schema.\n\n"
|
||||
"When helping users:\n"
|
||||
"1. First understand the database structure using get_database_schema "
|
||||
"or get_table_info\n"
|
||||
"2. Write efficient SQL queries to answer questions about the data\n"
|
||||
"3. Explain your findings clearly and concisely\n"
|
||||
"4. If a query might return many rows, consider using LIMIT or "
|
||||
"aggregations\n\n"
|
||||
"Important:\n"
|
||||
"- All queries run in READ ONLY mode - you cannot modify data\n"
|
||||
"- Results are limited to 1000 rows\n"
|
||||
"- Always validate your understanding of the schema before writing "
|
||||
"complex queries"
|
||||
)
|
||||
|
||||
|
||||
def chat_with_database(
|
||||
user_message: str,
|
||||
sid: int,
|
||||
did: int,
|
||||
conversation_history: Optional[list[Message]] = None,
|
||||
system_prompt: Optional[str] = None,
|
||||
max_tool_iterations: Optional[int] = None,
|
||||
provider: Optional[str] = None,
|
||||
model: Optional[str] = None
|
||||
) -> tuple[str, list[Message]]:
|
||||
"""
|
||||
Run an LLM chat conversation with database tool access.
|
||||
|
||||
This function handles the full conversation loop, executing any
|
||||
tool calls the LLM makes and continuing until a final response
|
||||
is generated.
|
||||
|
||||
Args:
|
||||
user_message: The user's message/question
|
||||
sid: Server ID for database connection
|
||||
did: Database ID for database connection
|
||||
conversation_history: Optional list of previous messages
|
||||
system_prompt: Optional custom system prompt (uses default if None)
|
||||
max_tool_iterations: Maximum number of tool call
|
||||
rounds. Uses preference setting if None.
|
||||
provider: Optional LLM provider override
|
||||
model: Optional model override
|
||||
|
||||
Returns:
|
||||
Tuple of (final_response_text, updated_conversation_history)
|
||||
|
||||
Raises:
|
||||
LLMClientError: If the LLM request fails
|
||||
RuntimeError: If LLM is not available or max iterations exceeded
|
||||
"""
|
||||
if not is_llm_available():
|
||||
raise RuntimeError("LLM is not configured. Please configure an LLM "
|
||||
"provider in Preferences > AI.")
|
||||
|
||||
client = get_llm_client(provider=provider, model=model)
|
||||
if not client:
|
||||
raise RuntimeError("Failed to create LLM client")
|
||||
|
||||
# Initialize conversation history
|
||||
messages = list(conversation_history) if conversation_history else []
|
||||
messages.append(Message.user(user_message))
|
||||
|
||||
# Use default system prompt if none provided
|
||||
if system_prompt is None:
|
||||
system_prompt = DEFAULT_SYSTEM_PROMPT
|
||||
|
||||
# Get max iterations from preferences if not specified
|
||||
if max_tool_iterations is None:
|
||||
max_tool_iterations = get_max_tool_iterations()
|
||||
|
||||
iteration = 0
|
||||
while iteration < max_tool_iterations:
|
||||
iteration += 1
|
||||
|
||||
# Call the LLM
|
||||
response = client.chat(
|
||||
messages=messages,
|
||||
tools=DATABASE_TOOLS,
|
||||
system_prompt=system_prompt
|
||||
)
|
||||
|
||||
# Add assistant response to history
|
||||
messages.append(response.to_message())
|
||||
|
||||
# Check if we're done
|
||||
if response.stop_reason != StopReason.TOOL_USE:
|
||||
return response.content, messages
|
||||
|
||||
# Execute tool calls
|
||||
tool_results = []
|
||||
for tool_call in response.tool_calls:
|
||||
try:
|
||||
result = execute_tool(
|
||||
tool_name=tool_call.name,
|
||||
arguments=tool_call.arguments,
|
||||
sid=sid,
|
||||
did=did
|
||||
)
|
||||
tool_results.append(Message.tool_result(
|
||||
tool_call_id=tool_call.id,
|
||||
content=json.dumps(result, default=str),
|
||||
is_error=False
|
||||
))
|
||||
except (DatabaseToolError, ValueError) as e:
|
||||
tool_results.append(Message.tool_result(
|
||||
tool_call_id=tool_call.id,
|
||||
content=json.dumps({"error": str(e)}),
|
||||
is_error=True
|
||||
))
|
||||
except Exception as e:
|
||||
tool_results.append(Message.tool_result(
|
||||
tool_call_id=tool_call.id,
|
||||
content=json.dumps({
|
||||
"error": f"Unexpected error: {str(e)}"
|
||||
}),
|
||||
is_error=True
|
||||
))
|
||||
|
||||
# Add tool results to history
|
||||
messages.extend(tool_results)
|
||||
|
||||
raise RuntimeError(
|
||||
f"Exceeded maximum tool iterations ({max_tool_iterations})"
|
||||
)
|
||||
|
||||
|
||||
def single_query(
|
||||
question: str,
|
||||
sid: int,
|
||||
did: int,
|
||||
provider: Optional[str] = None,
|
||||
model: Optional[str] = None
|
||||
) -> str:
|
||||
"""
|
||||
Ask a single question about the database.
|
||||
|
||||
This is a convenience function for one-shot questions without
|
||||
maintaining conversation history.
|
||||
|
||||
Args:
|
||||
question: The question to ask
|
||||
sid: Server ID
|
||||
did: Database ID
|
||||
provider: Optional LLM provider override
|
||||
model: Optional model override
|
||||
|
||||
Returns:
|
||||
The LLM's response text
|
||||
|
||||
Raises:
|
||||
LLMClientError: If the LLM request fails
|
||||
RuntimeError: If LLM is not available
|
||||
"""
|
||||
response, _ = chat_with_database(
|
||||
user_message=question,
|
||||
sid=sid,
|
||||
did=did,
|
||||
provider=provider,
|
||||
model=model
|
||||
)
|
||||
return response
|
||||
204
web/pgadmin/llm/client.py
Normal file
204
web/pgadmin/llm/client.py
Normal file
@@ -0,0 +1,204 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Base LLM client interface and factory."""
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Optional
|
||||
|
||||
from pgadmin.llm.models import (
|
||||
Message, Tool, LLMResponse, LLMError
|
||||
)
|
||||
|
||||
|
||||
class LLMClient(ABC):
|
||||
"""
|
||||
Abstract base class for LLM clients.
|
||||
|
||||
All LLM provider implementations should inherit from this class
|
||||
and implement the required methods.
|
||||
"""
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def provider_name(self) -> str:
|
||||
"""Return the name of the LLM provider."""
|
||||
pass
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def model_name(self) -> str:
|
||||
"""Return the name of the model being used."""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def is_available(self) -> bool:
|
||||
"""
|
||||
Check if the LLM client is properly configured and available.
|
||||
|
||||
Returns:
|
||||
True if the client can be used, False otherwise.
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
tools: Optional[list[Tool]] = None,
|
||||
system_prompt: Optional[str] = None,
|
||||
max_tokens: int = 4096,
|
||||
temperature: float = 0.0,
|
||||
**kwargs
|
||||
) -> LLMResponse:
|
||||
"""
|
||||
Send a chat request to the LLM.
|
||||
|
||||
Args:
|
||||
messages: List of conversation messages.
|
||||
tools: Optional list of tools the LLM can use.
|
||||
system_prompt: Optional system prompt to set context.
|
||||
max_tokens: Maximum tokens in the response.
|
||||
temperature: Sampling temperature (0.0 = deterministic).
|
||||
**kwargs: Additional provider-specific parameters.
|
||||
|
||||
Returns:
|
||||
LLMResponse containing the model's response.
|
||||
|
||||
Raises:
|
||||
LLMError: If the request fails.
|
||||
"""
|
||||
pass
|
||||
|
||||
def validate_connection(self) -> tuple[bool, Optional[str]]:
|
||||
"""
|
||||
Validate the connection to the LLM provider.
|
||||
|
||||
Returns:
|
||||
Tuple of (success, error_message).
|
||||
If success is True, error_message is None.
|
||||
"""
|
||||
try:
|
||||
# Try a minimal request to validate the connection
|
||||
self.chat(
|
||||
messages=[Message.user("Hello")],
|
||||
max_tokens=10
|
||||
)
|
||||
return True, None
|
||||
except LLMError as e:
|
||||
return False, str(e)
|
||||
except Exception as e:
|
||||
return False, f"Connection failed: {str(e)}"
|
||||
|
||||
|
||||
class LLMClientError(Exception):
|
||||
"""Exception raised for LLM client errors."""
|
||||
|
||||
def __init__(self, error: LLMError):
|
||||
self.error = error
|
||||
super().__init__(str(error))
|
||||
|
||||
|
||||
def get_llm_client(
|
||||
provider: Optional[str] = None,
|
||||
model: Optional[str] = None
|
||||
) -> Optional[LLMClient]:
|
||||
"""
|
||||
Get an LLM client instance for the specified or default provider.
|
||||
|
||||
Args:
|
||||
provider: Optional provider name ('anthropic', 'openai', 'ollama',
|
||||
'docker'). If not specified, uses the configured default
|
||||
provider.
|
||||
model: Optional model name to use. If not specified, uses the
|
||||
configured default model for the provider.
|
||||
|
||||
Returns:
|
||||
An LLMClient instance, or None if no provider is configured.
|
||||
|
||||
Raises:
|
||||
ValueError: If an invalid provider is specified.
|
||||
LLMClientError: If the client cannot be initialized.
|
||||
"""
|
||||
from pgadmin.llm.utils import (
|
||||
get_default_provider,
|
||||
get_anthropic_api_key, get_anthropic_model,
|
||||
get_openai_api_key, get_openai_model,
|
||||
get_ollama_api_url, get_ollama_model,
|
||||
get_docker_api_url, get_docker_model
|
||||
)
|
||||
|
||||
# Determine which provider to use
|
||||
if provider is None:
|
||||
provider = get_default_provider()
|
||||
if provider is None:
|
||||
return None
|
||||
|
||||
provider = provider.lower()
|
||||
|
||||
if provider == 'anthropic':
|
||||
from pgadmin.llm.providers.anthropic import AnthropicClient
|
||||
api_key = get_anthropic_api_key()
|
||||
if not api_key:
|
||||
raise LLMClientError(LLMError(
|
||||
message="Anthropic API key not configured",
|
||||
provider="anthropic"
|
||||
))
|
||||
model_name = model or get_anthropic_model()
|
||||
return AnthropicClient(api_key=api_key, model=model_name)
|
||||
|
||||
elif provider == 'openai':
|
||||
from pgadmin.llm.providers.openai import OpenAIClient
|
||||
api_key = get_openai_api_key()
|
||||
if not api_key:
|
||||
raise LLMClientError(LLMError(
|
||||
message="OpenAI API key not configured",
|
||||
provider="openai"
|
||||
))
|
||||
model_name = model or get_openai_model()
|
||||
return OpenAIClient(api_key=api_key, model=model_name)
|
||||
|
||||
elif provider == 'ollama':
|
||||
from pgadmin.llm.providers.ollama import OllamaClient
|
||||
api_url = get_ollama_api_url()
|
||||
if not api_url:
|
||||
raise LLMClientError(LLMError(
|
||||
message="Ollama API URL not configured",
|
||||
provider="ollama"
|
||||
))
|
||||
model_name = model or get_ollama_model()
|
||||
return OllamaClient(api_url=api_url, model=model_name)
|
||||
|
||||
elif provider == 'docker':
|
||||
from pgadmin.llm.providers.docker import DockerClient
|
||||
api_url = get_docker_api_url()
|
||||
if not api_url:
|
||||
raise LLMClientError(LLMError(
|
||||
message="Docker Model Runner API URL not configured",
|
||||
provider="docker"
|
||||
))
|
||||
model_name = model or get_docker_model()
|
||||
return DockerClient(api_url=api_url, model=model_name)
|
||||
|
||||
else:
|
||||
raise ValueError(f"Unknown LLM provider: {provider}")
|
||||
|
||||
|
||||
def is_llm_available() -> bool:
|
||||
"""
|
||||
Check if an LLM client is available and properly configured.
|
||||
|
||||
Returns:
|
||||
True if an LLM client can be created, False otherwise.
|
||||
"""
|
||||
try:
|
||||
client = get_llm_client()
|
||||
return client is not None and client.is_available()
|
||||
except (LLMClientError, ValueError):
|
||||
return False
|
||||
201
web/pgadmin/llm/models.py
Normal file
201
web/pgadmin/llm/models.py
Normal file
@@ -0,0 +1,201 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Data models for LLM interactions."""
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Optional
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class Role(str, Enum):
|
||||
"""Message roles in a conversation."""
|
||||
SYSTEM = 'system'
|
||||
USER = 'user'
|
||||
ASSISTANT = 'assistant'
|
||||
TOOL = 'tool'
|
||||
|
||||
|
||||
class StopReason(str, Enum):
|
||||
"""Reasons why the LLM stopped generating."""
|
||||
END_TURN = 'end_turn'
|
||||
TOOL_USE = 'tool_use'
|
||||
MAX_TOKENS = 'max_tokens'
|
||||
STOP_SEQUENCE = 'stop_sequence'
|
||||
ERROR = 'error'
|
||||
UNKNOWN = 'unknown'
|
||||
|
||||
|
||||
@dataclass
|
||||
class ToolCall:
|
||||
"""Represents a tool call requested by the LLM."""
|
||||
id: str
|
||||
name: str
|
||||
arguments: dict[str, Any]
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary representation."""
|
||||
return {
|
||||
'id': self.id,
|
||||
'name': self.name,
|
||||
'arguments': self.arguments
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class ToolResult:
|
||||
"""Represents the result of a tool execution."""
|
||||
tool_call_id: str
|
||||
content: str
|
||||
is_error: bool = False
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary representation."""
|
||||
return {
|
||||
'tool_call_id': self.tool_call_id,
|
||||
'content': self.content,
|
||||
'is_error': self.is_error
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Message:
|
||||
"""Represents a message in a conversation."""
|
||||
role: Role
|
||||
content: str
|
||||
tool_calls: list[ToolCall] = field(default_factory=list)
|
||||
tool_results: list[ToolResult] = field(default_factory=list)
|
||||
name: Optional[str] = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary representation."""
|
||||
result = {
|
||||
'role': self.role.value,
|
||||
'content': self.content
|
||||
}
|
||||
if self.tool_calls:
|
||||
result['tool_calls'] = [tc.to_dict() for tc in self.tool_calls]
|
||||
if self.tool_results:
|
||||
result['tool_results'] = [tr.to_dict() for tr in self.tool_results]
|
||||
if self.name:
|
||||
result['name'] = self.name
|
||||
return result
|
||||
|
||||
@classmethod
|
||||
def system(cls, content: str) -> 'Message':
|
||||
"""Create a system message."""
|
||||
return cls(role=Role.SYSTEM, content=content)
|
||||
|
||||
@classmethod
|
||||
def user(cls, content: str) -> 'Message':
|
||||
"""Create a user message."""
|
||||
return cls(role=Role.USER, content=content)
|
||||
|
||||
@classmethod
|
||||
def assistant(cls, content: str,
|
||||
tool_calls: list[ToolCall] = None) -> 'Message':
|
||||
"""Create an assistant message."""
|
||||
return cls(
|
||||
role=Role.ASSISTANT,
|
||||
content=content,
|
||||
tool_calls=tool_calls or []
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def tool_result(cls, tool_call_id: str, content: str,
|
||||
is_error: bool = False) -> 'Message':
|
||||
"""Create a tool result message."""
|
||||
return cls(
|
||||
role=Role.TOOL,
|
||||
content='',
|
||||
tool_results=[ToolResult(
|
||||
tool_call_id=tool_call_id,
|
||||
content=content,
|
||||
is_error=is_error
|
||||
)]
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Tool:
|
||||
"""Represents a tool that can be called by the LLM."""
|
||||
name: str
|
||||
description: str
|
||||
parameters: dict[str, Any]
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary representation."""
|
||||
return {
|
||||
'name': self.name,
|
||||
'description': self.description,
|
||||
'parameters': self.parameters
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Usage:
|
||||
"""Token usage information."""
|
||||
input_tokens: int = 0
|
||||
output_tokens: int = 0
|
||||
total_tokens: int = 0
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary representation."""
|
||||
return {
|
||||
'input_tokens': self.input_tokens,
|
||||
'output_tokens': self.output_tokens,
|
||||
'total_tokens': self.total_tokens
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class LLMResponse:
|
||||
"""Represents a response from an LLM."""
|
||||
content: str
|
||||
tool_calls: list[ToolCall] = field(default_factory=list)
|
||||
stop_reason: StopReason = StopReason.END_TURN
|
||||
model: str = ''
|
||||
usage: Usage = field(default_factory=Usage)
|
||||
raw_response: Optional[Any] = None
|
||||
|
||||
@property
|
||||
def has_tool_calls(self) -> bool:
|
||||
"""Check if the response contains tool calls."""
|
||||
return len(self.tool_calls) > 0
|
||||
|
||||
def to_message(self) -> Message:
|
||||
"""Convert response to an assistant message."""
|
||||
return Message.assistant(
|
||||
content=self.content,
|
||||
tool_calls=self.tool_calls
|
||||
)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary representation."""
|
||||
return {
|
||||
'content': self.content,
|
||||
'tool_calls': [tc.to_dict() for tc in self.tool_calls],
|
||||
'stop_reason': self.stop_reason.value,
|
||||
'model': self.model,
|
||||
'usage': self.usage.to_dict()
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class LLMError:
|
||||
"""Represents an error from an LLM operation."""
|
||||
message: str
|
||||
code: Optional[str] = None
|
||||
provider: Optional[str] = None
|
||||
retryable: bool = False
|
||||
|
||||
def __str__(self) -> str:
|
||||
if self.code:
|
||||
return f"[{self.code}] {self.message}"
|
||||
return self.message
|
||||
15
web/pgadmin/llm/prompts/__init__.py
Normal file
15
web/pgadmin/llm/prompts/__init__.py
Normal file
@@ -0,0 +1,15 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""LLM prompt templates for various features."""
|
||||
|
||||
from pgadmin.llm.prompts.nlq import NLQ_SYSTEM_PROMPT
|
||||
from pgadmin.llm.prompts.explain import EXPLAIN_ANALYSIS_PROMPT
|
||||
|
||||
__all__ = ['NLQ_SYSTEM_PROMPT', 'EXPLAIN_ANALYSIS_PROMPT']
|
||||
91
web/pgadmin/llm/prompts/explain.py
Normal file
91
web/pgadmin/llm/prompts/explain.py
Normal file
@@ -0,0 +1,91 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""System prompt for EXPLAIN plan analysis."""
|
||||
|
||||
EXPLAIN_ANALYSIS_PROMPT = """You are a PostgreSQL performance
|
||||
expert integrated into pgAdmin 4.
|
||||
Your task is to analyze EXPLAIN plan output and provide
|
||||
actionable optimization recommendations.
|
||||
|
||||
## Input Format
|
||||
|
||||
You will receive:
|
||||
1. The EXPLAIN plan output in JSON format
|
||||
(from EXPLAIN (FORMAT JSON, ANALYZE, ...))
|
||||
2. The original SQL query that was analyzed
|
||||
|
||||
## Analysis Guidelines
|
||||
|
||||
1. **Identify Performance Bottlenecks**:
|
||||
- Sequential scans on large tables (consider if an index would help)
|
||||
- Nested loops with high row counts
|
||||
(may indicate missing indexes or poor join order)
|
||||
- Large row estimate variances (actual vs planned)
|
||||
suggesting stale statistics
|
||||
- Sort operations on large datasets without indexes
|
||||
- Hash joins spilling to disk (indicated by batch counts > 1)
|
||||
- High startup costs relative to total costs
|
||||
- Bitmap heap scans with many recheck conditions
|
||||
|
||||
2. **Severity Classification**:
|
||||
- "high": Major performance impact, should be addressed
|
||||
- "medium": Notable impact, worth investigating
|
||||
- "low": Minor optimization opportunity
|
||||
|
||||
3. **Provide Actionable Recommendations**:
|
||||
- Suggest specific CREATE INDEX statements when appropriate
|
||||
- Recommend ANALYZE for tables with row estimate issues
|
||||
- Suggest query rewrites when the structure is suboptimal
|
||||
- Recommend configuration changes (work_mem, etc.) when relevant
|
||||
- Include the exact SQL for any suggested changes
|
||||
|
||||
4. **Consider Context**:
|
||||
- Small tables may not benefit from indexes
|
||||
- Some sequential scans are optimal (e.g., selecting most rows)
|
||||
- ANALYZE timing may be relevant for row estimate issues
|
||||
- Partial indexes may be better than full indexes
|
||||
|
||||
## Response Format
|
||||
|
||||
IMPORTANT: Your response MUST be ONLY a valid JSON object
|
||||
with no additional text,
|
||||
no markdown formatting, and no code blocks. Return exactly this format:
|
||||
|
||||
{
|
||||
"bottlenecks": [
|
||||
{
|
||||
"severity": "high|medium|low",
|
||||
"node": "Node description from plan",
|
||||
"issue": "Brief description of the problem",
|
||||
"details": "Detailed explanation of why this is a problem and its impact"
|
||||
}
|
||||
],
|
||||
"recommendations": [
|
||||
{
|
||||
"priority": 1,
|
||||
"title": "Short title for the recommendation",
|
||||
"explanation": "Why this change will help",
|
||||
"sql": "Exact SQL to execute (if applicable, otherwise null)"
|
||||
}
|
||||
],
|
||||
"summary": "One paragraph summary of the overall
|
||||
plan performance and key takeaways"
|
||||
}
|
||||
|
||||
Rules:
|
||||
- Return ONLY the JSON object, nothing before or after it
|
||||
- Do NOT wrap the JSON in markdown code blocks (no ```)
|
||||
- Order bottlenecks by severity (high first)
|
||||
- Order recommendations by priority (1 = highest)
|
||||
- If the plan looks optimal, return empty bottlenecks
|
||||
array with a positive summary
|
||||
- Always include at least a summary, even for simple plans
|
||||
- The "sql" field should be null if no SQL action is applicable
|
||||
"""
|
||||
39
web/pgadmin/llm/prompts/nlq.py
Normal file
39
web/pgadmin/llm/prompts/nlq.py
Normal file
@@ -0,0 +1,39 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""System prompt for Natural Language to SQL translation."""
|
||||
|
||||
NLQ_SYSTEM_PROMPT = """You are a PostgreSQL SQL expert \
|
||||
integrated into pgAdmin 4.
|
||||
Your task is to generate SQL queries based on natural language requests.
|
||||
|
||||
You have access to database inspection tools:
|
||||
- get_database_schema: Get list of schemas, tables, and views in the database
|
||||
- get_table_info: Get detailed column, constraint, and \
|
||||
index information for a table
|
||||
- execute_sql_query: Run read-only queries to understand \
|
||||
data structure (SELECT only)
|
||||
|
||||
Guidelines:
|
||||
- Use get_database_schema to discover available tables before writing queries
|
||||
- For statistics queries, use pg_stat_user_tables or pg_statio_user_tables
|
||||
- For I/O statistics specifically, use pg_statio_user_tables
|
||||
- Support SELECT, INSERT, UPDATE, DELETE, and DDL statements
|
||||
- Use explicit column names instead of SELECT *
|
||||
- For UPDATE/DELETE, always include WHERE clauses
|
||||
|
||||
Your response MUST be a JSON object in this exact format:
|
||||
{"sql": "YOUR SQL QUERY HERE", "explanation": "Brief explanation"}
|
||||
|
||||
Rules:
|
||||
- Return ONLY the JSON object, nothing else
|
||||
- No markdown code blocks
|
||||
- If you need clarification, set "sql" to null and put \
|
||||
your question in "explanation"
|
||||
"""
|
||||
16
web/pgadmin/llm/providers/__init__.py
Normal file
16
web/pgadmin/llm/providers/__init__.py
Normal file
@@ -0,0 +1,16 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""LLM provider implementations."""
|
||||
|
||||
from pgadmin.llm.providers.anthropic import AnthropicClient
|
||||
from pgadmin.llm.providers.openai import OpenAIClient
|
||||
from pgadmin.llm.providers.ollama import OllamaClient
|
||||
|
||||
__all__ = ['AnthropicClient', 'OpenAIClient', 'OllamaClient']
|
||||
276
web/pgadmin/llm/providers/anthropic.py
Normal file
276
web/pgadmin/llm/providers/anthropic.py
Normal file
@@ -0,0 +1,276 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Anthropic Claude LLM client implementation."""
|
||||
|
||||
import json
|
||||
import ssl
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
from typing import Optional
|
||||
import uuid
|
||||
|
||||
# Try to use certifi for proper SSL certificate handling
|
||||
try:
|
||||
import certifi
|
||||
SSL_CONTEXT = ssl.create_default_context(cafile=certifi.where())
|
||||
except ImportError:
|
||||
SSL_CONTEXT = ssl.create_default_context()
|
||||
|
||||
# Enforce minimum TLS 1.2 to satisfy security requirements
|
||||
SSL_CONTEXT.minimum_version = ssl.TLSVersion.TLSv1_2
|
||||
|
||||
from pgadmin.llm.client import LLMClient, LLMClientError
|
||||
from pgadmin.llm.models import (
|
||||
Message, Tool, ToolCall, LLMResponse, LLMError,
|
||||
Role, StopReason, Usage
|
||||
)
|
||||
|
||||
|
||||
# Default model if none specified
|
||||
DEFAULT_MODEL = 'claude-sonnet-4-20250514'
|
||||
|
||||
# API configuration
|
||||
API_URL = 'https://api.anthropic.com/v1/messages'
|
||||
API_VERSION = '2023-06-01'
|
||||
|
||||
|
||||
class AnthropicClient(LLMClient):
|
||||
"""
|
||||
Anthropic Claude API client.
|
||||
|
||||
Implements the LLMClient interface for Anthropic's Claude models.
|
||||
"""
|
||||
|
||||
def __init__(self, api_key: str, model: Optional[str] = None):
|
||||
"""
|
||||
Initialize the Anthropic client.
|
||||
|
||||
Args:
|
||||
api_key: The Anthropic API key.
|
||||
model: Optional model name. Defaults to claude-sonnet-4-20250514.
|
||||
"""
|
||||
self._api_key = api_key
|
||||
self._model = model or DEFAULT_MODEL
|
||||
|
||||
@property
|
||||
def provider_name(self) -> str:
|
||||
return 'anthropic'
|
||||
|
||||
@property
|
||||
def model_name(self) -> str:
|
||||
return self._model
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Check if the client is properly configured."""
|
||||
return bool(self._api_key)
|
||||
|
||||
def chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
tools: Optional[list[Tool]] = None,
|
||||
system_prompt: Optional[str] = None,
|
||||
max_tokens: int = 4096,
|
||||
temperature: float = 0.0,
|
||||
**kwargs
|
||||
) -> LLMResponse:
|
||||
"""
|
||||
Send a chat request to Claude.
|
||||
|
||||
Args:
|
||||
messages: List of conversation messages.
|
||||
tools: Optional list of tools Claude can use.
|
||||
system_prompt: Optional system prompt.
|
||||
max_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature.
|
||||
**kwargs: Additional parameters.
|
||||
|
||||
Returns:
|
||||
LLMResponse containing Claude's response.
|
||||
|
||||
Raises:
|
||||
LLMClientError: If the request fails.
|
||||
"""
|
||||
# Build the request payload
|
||||
payload = {
|
||||
'model': self._model,
|
||||
'max_tokens': max_tokens,
|
||||
'messages': self._convert_messages(messages)
|
||||
}
|
||||
|
||||
if system_prompt:
|
||||
payload['system'] = system_prompt
|
||||
|
||||
if temperature > 0:
|
||||
payload['temperature'] = temperature
|
||||
|
||||
if tools:
|
||||
payload['tools'] = self._convert_tools(tools)
|
||||
|
||||
# Make the API request
|
||||
try:
|
||||
response_data = self._make_request(payload)
|
||||
return self._parse_response(response_data)
|
||||
except LLMClientError:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Request failed: {str(e)}",
|
||||
provider=self.provider_name
|
||||
))
|
||||
|
||||
def _convert_messages(self, messages: list[Message]) -> list[dict]:
|
||||
"""Convert Message objects to Anthropic API format."""
|
||||
result = []
|
||||
|
||||
for msg in messages:
|
||||
if msg.role == Role.SYSTEM:
|
||||
# System messages are handled separately in Anthropic API
|
||||
continue
|
||||
|
||||
if msg.role == Role.USER:
|
||||
result.append({
|
||||
'role': 'user',
|
||||
'content': msg.content
|
||||
})
|
||||
|
||||
elif msg.role == Role.ASSISTANT:
|
||||
content = []
|
||||
if msg.content:
|
||||
content.append({'type': 'text', 'text': msg.content})
|
||||
|
||||
# Add tool use blocks
|
||||
for tc in msg.tool_calls:
|
||||
content.append({
|
||||
'type': 'tool_use',
|
||||
'id': tc.id,
|
||||
'name': tc.name,
|
||||
'input': tc.arguments
|
||||
})
|
||||
|
||||
result.append({
|
||||
'role': 'assistant',
|
||||
'content': content if content else msg.content
|
||||
})
|
||||
|
||||
elif msg.role == Role.TOOL:
|
||||
# Tool results in Anthropic are sent as user messages
|
||||
content = []
|
||||
for tr in msg.tool_results:
|
||||
content.append({
|
||||
'type': 'tool_result',
|
||||
'tool_use_id': tr.tool_call_id,
|
||||
'content': tr.content,
|
||||
'is_error': tr.is_error
|
||||
})
|
||||
result.append({
|
||||
'role': 'user',
|
||||
'content': content
|
||||
})
|
||||
|
||||
return result
|
||||
|
||||
def _convert_tools(self, tools: list[Tool]) -> list[dict]:
|
||||
"""Convert Tool objects to Anthropic API format."""
|
||||
return [
|
||||
{
|
||||
'name': tool.name,
|
||||
'description': tool.description,
|
||||
'input_schema': tool.parameters
|
||||
}
|
||||
for tool in tools
|
||||
]
|
||||
|
||||
def _make_request(self, payload: dict) -> dict:
|
||||
"""Make an HTTP request to the Anthropic API."""
|
||||
headers = {
|
||||
'Content-Type': 'application/json',
|
||||
'x-api-key': self._api_key,
|
||||
'anthropic-version': API_VERSION
|
||||
}
|
||||
|
||||
request = urllib.request.Request(
|
||||
API_URL,
|
||||
data=json.dumps(payload).encode('utf-8'),
|
||||
headers=headers,
|
||||
method='POST'
|
||||
)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(
|
||||
request, timeout=120, context=SSL_CONTEXT
|
||||
) as response:
|
||||
return json.loads(response.read().decode('utf-8'))
|
||||
except urllib.error.HTTPError as e:
|
||||
error_body = e.read().decode('utf-8')
|
||||
try:
|
||||
error_data = json.loads(error_body)
|
||||
error_msg = error_data.get('error', {}).get('message', str(e))
|
||||
except json.JSONDecodeError:
|
||||
error_msg = error_body or str(e)
|
||||
|
||||
raise LLMClientError(LLMError(
|
||||
message=error_msg,
|
||||
code=str(e.code),
|
||||
provider=self.provider_name,
|
||||
retryable=e.code in (429, 500, 502, 503, 504)
|
||||
))
|
||||
except urllib.error.URLError as e:
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Connection error: {e.reason}",
|
||||
provider=self.provider_name,
|
||||
retryable=True
|
||||
))
|
||||
|
||||
def _parse_response(self, data: dict) -> LLMResponse:
|
||||
"""Parse the Anthropic API response into an LLMResponse."""
|
||||
content_parts = []
|
||||
tool_calls = []
|
||||
|
||||
for block in data.get('content', []):
|
||||
if block.get('type') == 'text':
|
||||
content_parts.append(block.get('text', ''))
|
||||
elif block.get('type') == 'tool_use':
|
||||
tool_calls.append(ToolCall(
|
||||
id=block.get('id', str(uuid.uuid4())),
|
||||
name=block.get('name', ''),
|
||||
arguments=block.get('input', {})
|
||||
))
|
||||
|
||||
# Map Anthropic stop reasons to our enum
|
||||
stop_reason_map = {
|
||||
'end_turn': StopReason.END_TURN,
|
||||
'tool_use': StopReason.TOOL_USE,
|
||||
'max_tokens': StopReason.MAX_TOKENS,
|
||||
'stop_sequence': StopReason.STOP_SEQUENCE
|
||||
}
|
||||
stop_reason = stop_reason_map.get(
|
||||
data.get('stop_reason', ''),
|
||||
StopReason.UNKNOWN
|
||||
)
|
||||
|
||||
# Parse usage information
|
||||
usage_data = data.get('usage', {})
|
||||
usage = Usage(
|
||||
input_tokens=usage_data.get('input_tokens', 0),
|
||||
output_tokens=usage_data.get('output_tokens', 0),
|
||||
total_tokens=(
|
||||
usage_data.get('input_tokens', 0) +
|
||||
usage_data.get('output_tokens', 0)
|
||||
)
|
||||
)
|
||||
|
||||
return LLMResponse(
|
||||
content='\n'.join(content_parts),
|
||||
tool_calls=tool_calls,
|
||||
stop_reason=stop_reason,
|
||||
model=data.get('model', self._model),
|
||||
usage=usage,
|
||||
raw_response=data
|
||||
)
|
||||
359
web/pgadmin/llm/providers/docker.py
Normal file
359
web/pgadmin/llm/providers/docker.py
Normal file
@@ -0,0 +1,359 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Docker Model Runner LLM client implementation.
|
||||
|
||||
Docker Desktop 4.40+ includes a built-in model runner that provides an
|
||||
OpenAI-compatible API at http://localhost:12434. No API key is required.
|
||||
"""
|
||||
|
||||
import json
|
||||
import socket
|
||||
import ssl
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
from typing import Optional
|
||||
import uuid
|
||||
|
||||
# Try to use certifi for proper SSL certificate handling
|
||||
try:
|
||||
import certifi
|
||||
SSL_CONTEXT = ssl.create_default_context(cafile=certifi.where())
|
||||
except ImportError:
|
||||
SSL_CONTEXT = ssl.create_default_context()
|
||||
|
||||
# Enforce minimum TLS 1.2 to satisfy security requirements
|
||||
SSL_CONTEXT.minimum_version = ssl.TLSVersion.TLSv1_2
|
||||
|
||||
from pgadmin.llm.client import LLMClient, LLMClientError
|
||||
from pgadmin.llm.models import (
|
||||
Message, Tool, ToolCall, LLMResponse, LLMError,
|
||||
Role, StopReason, Usage
|
||||
)
|
||||
|
||||
|
||||
# Default configuration
|
||||
DEFAULT_API_URL = 'http://localhost:12434'
|
||||
DEFAULT_MODEL = 'ai/qwen3-coder'
|
||||
|
||||
|
||||
class DockerClient(LLMClient):
|
||||
"""
|
||||
Docker Model Runner API client.
|
||||
|
||||
Implements the LLMClient interface for Docker's built-in model runner,
|
||||
which provides an OpenAI-compatible API.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, api_url: Optional[str] = None, model: Optional[str] = None
|
||||
):
|
||||
"""
|
||||
Initialize the Docker Model Runner client.
|
||||
|
||||
Args:
|
||||
api_url: The Docker Model Runner API URL
|
||||
(default: http://localhost:12434).
|
||||
model: Optional model name. Defaults to ai/qwen3-coder.
|
||||
"""
|
||||
self._api_url = (api_url or DEFAULT_API_URL).rstrip('/')
|
||||
self._model = model or DEFAULT_MODEL
|
||||
|
||||
@property
|
||||
def provider_name(self) -> str:
|
||||
return 'docker'
|
||||
|
||||
@property
|
||||
def model_name(self) -> str:
|
||||
return self._model
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Check if the client is properly configured."""
|
||||
return bool(self._api_url)
|
||||
|
||||
def chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
tools: Optional[list[Tool]] = None,
|
||||
system_prompt: Optional[str] = None,
|
||||
max_tokens: int = 4096,
|
||||
temperature: float = 0.0,
|
||||
**kwargs
|
||||
) -> LLMResponse:
|
||||
"""
|
||||
Send a chat request to Docker Model Runner.
|
||||
|
||||
Args:
|
||||
messages: List of conversation messages.
|
||||
tools: Optional list of tools the model can use.
|
||||
system_prompt: Optional system prompt.
|
||||
max_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature.
|
||||
**kwargs: Additional parameters.
|
||||
|
||||
Returns:
|
||||
LLMResponse containing the model's response.
|
||||
|
||||
Raises:
|
||||
LLMClientError: If the request fails.
|
||||
"""
|
||||
# Build the request payload
|
||||
converted_messages = self._convert_messages(messages)
|
||||
|
||||
# Add system prompt at the beginning if provided
|
||||
if system_prompt:
|
||||
converted_messages.insert(0, {
|
||||
'role': 'system',
|
||||
'content': system_prompt
|
||||
})
|
||||
|
||||
payload = {
|
||||
'model': self._model,
|
||||
'messages': converted_messages,
|
||||
'max_completion_tokens': max_tokens,
|
||||
'temperature': temperature
|
||||
}
|
||||
|
||||
if tools:
|
||||
payload['tools'] = self._convert_tools(tools)
|
||||
payload['tool_choice'] = 'auto'
|
||||
|
||||
# Make the API request
|
||||
try:
|
||||
response_data = self._make_request(payload)
|
||||
return self._parse_response(response_data)
|
||||
except LLMClientError:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Request failed: {str(e)}",
|
||||
provider=self.provider_name
|
||||
))
|
||||
|
||||
def _convert_messages(self, messages: list[Message]) -> list[dict]:
|
||||
"""Convert Message objects to OpenAI API format."""
|
||||
result = []
|
||||
|
||||
for msg in messages:
|
||||
if msg.role == Role.SYSTEM:
|
||||
result.append({
|
||||
'role': 'system',
|
||||
'content': msg.content
|
||||
})
|
||||
|
||||
elif msg.role == Role.USER:
|
||||
result.append({
|
||||
'role': 'user',
|
||||
'content': msg.content
|
||||
})
|
||||
|
||||
elif msg.role == Role.ASSISTANT:
|
||||
message = {
|
||||
'role': 'assistant',
|
||||
'content': msg.content or None
|
||||
}
|
||||
|
||||
# Add tool calls if present
|
||||
if msg.tool_calls:
|
||||
message['tool_calls'] = [
|
||||
{
|
||||
'id': tc.id,
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': tc.name,
|
||||
'arguments': json.dumps(tc.arguments)
|
||||
}
|
||||
}
|
||||
for tc in msg.tool_calls
|
||||
]
|
||||
|
||||
result.append(message)
|
||||
|
||||
elif msg.role == Role.TOOL:
|
||||
# Each tool result is a separate message in OpenAI format
|
||||
for tr in msg.tool_results:
|
||||
result.append({
|
||||
'role': 'tool',
|
||||
'tool_call_id': tr.tool_call_id,
|
||||
'content': tr.content
|
||||
})
|
||||
|
||||
return result
|
||||
|
||||
def _convert_tools(self, tools: list[Tool]) -> list[dict]:
|
||||
"""Convert Tool objects to OpenAI API format."""
|
||||
return [
|
||||
{
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': tool.name,
|
||||
'description': tool.description,
|
||||
'parameters': tool.parameters
|
||||
}
|
||||
}
|
||||
for tool in tools
|
||||
]
|
||||
|
||||
def _make_request(self, payload: dict) -> dict:
|
||||
"""Make an HTTP request to the Docker Model Runner API."""
|
||||
headers = {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
|
||||
# Docker Model Runner uses /engines/v1 path for OpenAI-compatible API
|
||||
url = f'{self._api_url}/engines/v1/chat/completions'
|
||||
|
||||
request = urllib.request.Request(
|
||||
url,
|
||||
data=json.dumps(payload).encode('utf-8'),
|
||||
headers=headers,
|
||||
method='POST'
|
||||
)
|
||||
|
||||
try:
|
||||
# Use longer timeout for local models which can be slower
|
||||
with urllib.request.urlopen(
|
||||
request, timeout=300, context=SSL_CONTEXT
|
||||
) as response:
|
||||
return json.loads(response.read().decode('utf-8'))
|
||||
except urllib.error.HTTPError as e:
|
||||
error_body = e.read().decode('utf-8')
|
||||
try:
|
||||
error_data = json.loads(error_body)
|
||||
error_msg = error_data.get('error', {}).get('message', str(e))
|
||||
except json.JSONDecodeError:
|
||||
error_msg = error_body or str(e)
|
||||
|
||||
raise LLMClientError(LLMError(
|
||||
message=error_msg,
|
||||
code=str(e.code),
|
||||
provider=self.provider_name,
|
||||
retryable=e.code in (429, 500, 502, 503, 504)
|
||||
))
|
||||
except urllib.error.URLError as e:
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Connection error: {e.reason}. "
|
||||
f"Is Docker Model Runner running at {self._api_url}?",
|
||||
provider=self.provider_name,
|
||||
retryable=True
|
||||
))
|
||||
except socket.timeout:
|
||||
raise LLMClientError(LLMError(
|
||||
message="Request timed out. Local models can be slow - "
|
||||
"try a smaller model or wait for the response.",
|
||||
code='timeout',
|
||||
provider=self.provider_name,
|
||||
retryable=True
|
||||
))
|
||||
|
||||
def _parse_response(self, data: dict) -> LLMResponse:
|
||||
"""Parse the API response into an LLMResponse."""
|
||||
# Check for API-level errors in the response
|
||||
if 'error' in data:
|
||||
error_info = data['error']
|
||||
raise LLMClientError(LLMError(
|
||||
message=error_info.get('message', 'Unknown API error'),
|
||||
code=error_info.get('code', 'unknown'),
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
|
||||
choices = data.get('choices', [])
|
||||
if not choices:
|
||||
raise LLMClientError(LLMError(
|
||||
message='No response choices returned from API',
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
|
||||
choice = choices[0]
|
||||
message = choice.get('message', {})
|
||||
|
||||
# Check for refusal (content moderation)
|
||||
if message.get('refusal'):
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Request refused: {message.get('refusal')}",
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
|
||||
content = message.get('content', '') or ''
|
||||
tool_calls = []
|
||||
|
||||
# Parse tool calls if present
|
||||
for tc in message.get('tool_calls', []):
|
||||
if tc.get('type') == 'function':
|
||||
func = tc.get('function', {})
|
||||
try:
|
||||
arguments = json.loads(func.get('arguments', '{}'))
|
||||
except json.JSONDecodeError:
|
||||
arguments = {}
|
||||
|
||||
tool_calls.append(ToolCall(
|
||||
id=tc.get('id', str(uuid.uuid4())),
|
||||
name=func.get('name', ''),
|
||||
arguments=arguments
|
||||
))
|
||||
|
||||
# Map finish reasons to our enum
|
||||
finish_reason = choice.get('finish_reason', '')
|
||||
stop_reason_map = {
|
||||
'stop': StopReason.END_TURN,
|
||||
'tool_calls': StopReason.TOOL_USE,
|
||||
'length': StopReason.MAX_TOKENS,
|
||||
'content_filter': StopReason.STOP_SEQUENCE
|
||||
}
|
||||
stop_reason = stop_reason_map.get(finish_reason, StopReason.UNKNOWN)
|
||||
|
||||
# Parse usage information
|
||||
usage_data = data.get('usage', {})
|
||||
usage = Usage(
|
||||
input_tokens=usage_data.get('prompt_tokens', 0),
|
||||
output_tokens=usage_data.get('completion_tokens', 0),
|
||||
total_tokens=usage_data.get('total_tokens', 0)
|
||||
)
|
||||
|
||||
# Check for problematic responses
|
||||
if not content and not tool_calls:
|
||||
if stop_reason == StopReason.MAX_TOKENS:
|
||||
input_tokens = usage.input_tokens
|
||||
raise LLMClientError(LLMError(
|
||||
message=(
|
||||
f'Response truncated due to token limit '
|
||||
f'(input: {input_tokens} tokens). '
|
||||
f'The request is too large for model '
|
||||
f'{self._model}. '
|
||||
f'Try using a model with a larger context '
|
||||
f'window, or analyze a smaller scope.'
|
||||
),
|
||||
code='max_tokens',
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
elif finish_reason and finish_reason not in (
|
||||
'stop', 'tool_calls'
|
||||
):
|
||||
raise LLMClientError(LLMError(
|
||||
message=(
|
||||
f'Empty response with finish reason: '
|
||||
f'{finish_reason}'
|
||||
),
|
||||
code=finish_reason,
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
|
||||
return LLMResponse(
|
||||
content=content,
|
||||
tool_calls=tool_calls,
|
||||
stop_reason=stop_reason,
|
||||
model=data.get('model', self._model),
|
||||
usage=usage,
|
||||
raw_response=data
|
||||
)
|
||||
290
web/pgadmin/llm/providers/ollama.py
Normal file
290
web/pgadmin/llm/providers/ollama.py
Normal file
@@ -0,0 +1,290 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Ollama LLM client implementation."""
|
||||
|
||||
import json
|
||||
import re
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
from typing import Optional
|
||||
import uuid
|
||||
|
||||
from pgadmin.llm.client import LLMClient, LLMClientError
|
||||
from pgadmin.llm.models import (
|
||||
Message, Tool, ToolCall, LLMResponse, LLMError,
|
||||
Role, StopReason, Usage
|
||||
)
|
||||
|
||||
|
||||
# Default configuration
|
||||
DEFAULT_API_URL = 'http://localhost:11434'
|
||||
DEFAULT_MODEL = 'llama3.2'
|
||||
|
||||
|
||||
class OllamaClient(LLMClient):
|
||||
"""
|
||||
Ollama API client.
|
||||
|
||||
Implements the LLMClient interface for locally-hosted Ollama models.
|
||||
Uses the Ollama chat API with tool support.
|
||||
"""
|
||||
|
||||
def __init__(self, api_url: str, model: Optional[str] = None):
|
||||
"""
|
||||
Initialize the Ollama client.
|
||||
|
||||
Args:
|
||||
api_url: The Ollama API base URL (e.g., http://localhost:11434).
|
||||
model: Optional model name. Defaults to llama3.2.
|
||||
"""
|
||||
self._api_url = api_url.rstrip('/')
|
||||
self._model = model or DEFAULT_MODEL
|
||||
|
||||
@property
|
||||
def provider_name(self) -> str:
|
||||
return 'ollama'
|
||||
|
||||
@property
|
||||
def model_name(self) -> str:
|
||||
return self._model
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Check if Ollama is running and the model is available."""
|
||||
if not self._api_url:
|
||||
return False
|
||||
|
||||
try:
|
||||
# Check if Ollama is running
|
||||
req = urllib.request.Request(f'{self._api_url}/api/tags')
|
||||
with urllib.request.urlopen(req, timeout=5) as response:
|
||||
data = json.loads(response.read().decode('utf-8'))
|
||||
# Check if our model is available
|
||||
models = [m.get('name', '') for m in data.get('models', [])]
|
||||
# Model names might include tags like ':latest'
|
||||
return any(
|
||||
self._model == m or self._model == m.split(':')[0]
|
||||
for m in models
|
||||
)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
tools: Optional[list[Tool]] = None,
|
||||
system_prompt: Optional[str] = None,
|
||||
max_tokens: int = 4096,
|
||||
temperature: float = 0.0,
|
||||
**kwargs
|
||||
) -> LLMResponse:
|
||||
"""
|
||||
Send a chat request to Ollama.
|
||||
|
||||
Args:
|
||||
messages: List of conversation messages.
|
||||
tools: Optional list of tools the model can use.
|
||||
system_prompt: Optional system prompt.
|
||||
max_tokens: Maximum tokens in response (num_predict in Ollama).
|
||||
temperature: Sampling temperature.
|
||||
**kwargs: Additional parameters.
|
||||
|
||||
Returns:
|
||||
LLMResponse containing the model's response.
|
||||
|
||||
Raises:
|
||||
LLMClientError: If the request fails.
|
||||
"""
|
||||
# Build the request payload
|
||||
converted_messages = self._convert_messages(messages)
|
||||
|
||||
# Add system prompt at the beginning if provided
|
||||
if system_prompt:
|
||||
converted_messages.insert(0, {
|
||||
'role': 'system',
|
||||
'content': system_prompt
|
||||
})
|
||||
|
||||
payload = {
|
||||
'model': self._model,
|
||||
'messages': converted_messages,
|
||||
'stream': False,
|
||||
'options': {
|
||||
'num_predict': max_tokens,
|
||||
'temperature': temperature
|
||||
}
|
||||
}
|
||||
|
||||
if tools:
|
||||
payload['tools'] = self._convert_tools(tools)
|
||||
|
||||
# Make the API request
|
||||
try:
|
||||
response_data = self._make_request(payload)
|
||||
return self._parse_response(response_data)
|
||||
except LLMClientError:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Request failed: {str(e)}",
|
||||
provider=self.provider_name
|
||||
))
|
||||
|
||||
def _convert_messages(self, messages: list[Message]) -> list[dict]:
|
||||
"""Convert Message objects to Ollama API format."""
|
||||
result = []
|
||||
|
||||
for msg in messages:
|
||||
if msg.role == Role.SYSTEM:
|
||||
result.append({
|
||||
'role': 'system',
|
||||
'content': msg.content
|
||||
})
|
||||
|
||||
elif msg.role == Role.USER:
|
||||
result.append({
|
||||
'role': 'user',
|
||||
'content': msg.content
|
||||
})
|
||||
|
||||
elif msg.role == Role.ASSISTANT:
|
||||
message = {
|
||||
'role': 'assistant',
|
||||
'content': msg.content or ''
|
||||
}
|
||||
|
||||
# Add tool calls if present
|
||||
if msg.tool_calls:
|
||||
message['tool_calls'] = [
|
||||
{
|
||||
'function': {
|
||||
'name': tc.name,
|
||||
'arguments': tc.arguments
|
||||
}
|
||||
}
|
||||
for tc in msg.tool_calls
|
||||
]
|
||||
|
||||
result.append(message)
|
||||
|
||||
elif msg.role == Role.TOOL:
|
||||
# Tool results in Ollama
|
||||
for tr in msg.tool_results:
|
||||
result.append({
|
||||
'role': 'tool',
|
||||
'content': tr.content
|
||||
})
|
||||
|
||||
return result
|
||||
|
||||
def _convert_tools(self, tools: list[Tool]) -> list[dict]:
|
||||
"""Convert Tool objects to Ollama API format."""
|
||||
return [
|
||||
{
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': tool.name,
|
||||
'description': tool.description,
|
||||
'parameters': tool.parameters
|
||||
}
|
||||
}
|
||||
for tool in tools
|
||||
]
|
||||
|
||||
def _make_request(self, payload: dict) -> dict:
|
||||
"""Make an HTTP request to the Ollama API."""
|
||||
url = f'{self._api_url}/api/chat'
|
||||
|
||||
request = urllib.request.Request(
|
||||
url,
|
||||
data=json.dumps(payload).encode('utf-8'),
|
||||
headers={'Content-Type': 'application/json'},
|
||||
method='POST'
|
||||
)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=300) as response:
|
||||
return json.loads(response.read().decode('utf-8'))
|
||||
except urllib.error.HTTPError as e:
|
||||
error_body = e.read().decode('utf-8')
|
||||
try:
|
||||
error_data = json.loads(error_body)
|
||||
error_msg = error_data.get('error', str(e))
|
||||
except json.JSONDecodeError:
|
||||
error_msg = error_body or str(e)
|
||||
|
||||
raise LLMClientError(LLMError(
|
||||
message=error_msg,
|
||||
code=str(e.code),
|
||||
provider=self.provider_name,
|
||||
retryable=e.code in (500, 502, 503, 504)
|
||||
))
|
||||
except urllib.error.URLError as e:
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Cannot connect to Ollama: {e.reason}",
|
||||
provider=self.provider_name,
|
||||
retryable=True
|
||||
))
|
||||
|
||||
def _parse_response(self, data: dict) -> LLMResponse:
|
||||
"""Parse the Ollama API response into an LLMResponse."""
|
||||
import re
|
||||
|
||||
message = data.get('message', {})
|
||||
content = message.get('content', '')
|
||||
tool_calls = []
|
||||
|
||||
# Parse tool calls if present (native Ollama format)
|
||||
for tc in message.get('tool_calls', []):
|
||||
func = tc.get('function', {})
|
||||
arguments = func.get('arguments', {})
|
||||
|
||||
# Arguments might be a string that needs parsing
|
||||
if isinstance(arguments, str):
|
||||
try:
|
||||
arguments = json.loads(arguments)
|
||||
except json.JSONDecodeError:
|
||||
arguments = {}
|
||||
|
||||
tool_calls.append(ToolCall(
|
||||
id=str(uuid.uuid4()), # Ollama doesn't provide IDs
|
||||
name=func.get('name', ''),
|
||||
arguments=arguments
|
||||
))
|
||||
|
||||
# Determine stop reason
|
||||
done_reason = data.get('done_reason', '')
|
||||
if tool_calls:
|
||||
stop_reason = StopReason.TOOL_USE
|
||||
elif done_reason == 'stop':
|
||||
stop_reason = StopReason.END_TURN
|
||||
elif done_reason == 'length':
|
||||
stop_reason = StopReason.MAX_TOKENS
|
||||
else:
|
||||
stop_reason = StopReason.UNKNOWN
|
||||
|
||||
# Parse usage information
|
||||
# Ollama provides eval_count (output) and prompt_eval_count (input)
|
||||
usage = Usage(
|
||||
input_tokens=data.get('prompt_eval_count', 0),
|
||||
output_tokens=data.get('eval_count', 0),
|
||||
total_tokens=(
|
||||
data.get('prompt_eval_count', 0) +
|
||||
data.get('eval_count', 0)
|
||||
)
|
||||
)
|
||||
|
||||
return LLMResponse(
|
||||
content=content,
|
||||
tool_calls=tool_calls,
|
||||
stop_reason=stop_reason,
|
||||
model=data.get('model', self._model),
|
||||
usage=usage,
|
||||
raw_response=data
|
||||
)
|
||||
345
web/pgadmin/llm/providers/openai.py
Normal file
345
web/pgadmin/llm/providers/openai.py
Normal file
@@ -0,0 +1,345 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""OpenAI GPT LLM client implementation."""
|
||||
|
||||
import json
|
||||
import socket
|
||||
import ssl
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
from typing import Optional
|
||||
import uuid
|
||||
|
||||
# Try to use certifi for proper SSL certificate handling
|
||||
try:
|
||||
import certifi
|
||||
SSL_CONTEXT = ssl.create_default_context(cafile=certifi.where())
|
||||
except ImportError:
|
||||
SSL_CONTEXT = ssl.create_default_context()
|
||||
|
||||
# Enforce minimum TLS 1.2 to satisfy security requirements
|
||||
SSL_CONTEXT.minimum_version = ssl.TLSVersion.TLSv1_2
|
||||
|
||||
from pgadmin.llm.client import LLMClient, LLMClientError
|
||||
from pgadmin.llm.models import (
|
||||
Message, Tool, ToolCall, LLMResponse, LLMError,
|
||||
Role, StopReason, Usage
|
||||
)
|
||||
|
||||
|
||||
# Default model if none specified
|
||||
DEFAULT_MODEL = 'gpt-4o'
|
||||
|
||||
# API configuration
|
||||
API_URL = 'https://api.openai.com/v1/chat/completions'
|
||||
|
||||
|
||||
class OpenAIClient(LLMClient):
|
||||
"""
|
||||
OpenAI GPT API client.
|
||||
|
||||
Implements the LLMClient interface for OpenAI's GPT models.
|
||||
"""
|
||||
|
||||
def __init__(self, api_key: str, model: Optional[str] = None):
|
||||
"""
|
||||
Initialize the OpenAI client.
|
||||
|
||||
Args:
|
||||
api_key: The OpenAI API key.
|
||||
model: Optional model name. Defaults to gpt-4o.
|
||||
"""
|
||||
self._api_key = api_key
|
||||
self._model = model or DEFAULT_MODEL
|
||||
|
||||
@property
|
||||
def provider_name(self) -> str:
|
||||
return 'openai'
|
||||
|
||||
@property
|
||||
def model_name(self) -> str:
|
||||
return self._model
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Check if the client is properly configured."""
|
||||
return bool(self._api_key)
|
||||
|
||||
def chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
tools: Optional[list[Tool]] = None,
|
||||
system_prompt: Optional[str] = None,
|
||||
max_tokens: int = 4096,
|
||||
temperature: float = 0.0,
|
||||
**kwargs
|
||||
) -> LLMResponse:
|
||||
"""
|
||||
Send a chat request to OpenAI.
|
||||
|
||||
Args:
|
||||
messages: List of conversation messages.
|
||||
tools: Optional list of tools the model can use.
|
||||
system_prompt: Optional system prompt.
|
||||
max_tokens: Maximum tokens in response.
|
||||
temperature: Sampling temperature.
|
||||
**kwargs: Additional parameters.
|
||||
|
||||
Returns:
|
||||
LLMResponse containing the model's response.
|
||||
|
||||
Raises:
|
||||
LLMClientError: If the request fails.
|
||||
"""
|
||||
# Build the request payload
|
||||
converted_messages = self._convert_messages(messages)
|
||||
|
||||
# Add system prompt at the beginning if provided
|
||||
if system_prompt:
|
||||
converted_messages.insert(0, {
|
||||
'role': 'system',
|
||||
'content': system_prompt
|
||||
})
|
||||
|
||||
payload = {
|
||||
'model': self._model,
|
||||
'messages': converted_messages,
|
||||
'max_completion_tokens': max_tokens,
|
||||
'temperature': temperature
|
||||
}
|
||||
|
||||
if tools:
|
||||
payload['tools'] = self._convert_tools(tools)
|
||||
payload['tool_choice'] = 'auto'
|
||||
|
||||
# Make the API request
|
||||
try:
|
||||
response_data = self._make_request(payload)
|
||||
return self._parse_response(response_data)
|
||||
except LLMClientError:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Request failed: {str(e)}",
|
||||
provider=self.provider_name
|
||||
))
|
||||
|
||||
def _convert_messages(self, messages: list[Message]) -> list[dict]:
|
||||
"""Convert Message objects to OpenAI API format."""
|
||||
result = []
|
||||
|
||||
for msg in messages:
|
||||
if msg.role == Role.SYSTEM:
|
||||
result.append({
|
||||
'role': 'system',
|
||||
'content': msg.content
|
||||
})
|
||||
|
||||
elif msg.role == Role.USER:
|
||||
result.append({
|
||||
'role': 'user',
|
||||
'content': msg.content
|
||||
})
|
||||
|
||||
elif msg.role == Role.ASSISTANT:
|
||||
message = {
|
||||
'role': 'assistant',
|
||||
'content': msg.content or None
|
||||
}
|
||||
|
||||
# Add tool calls if present
|
||||
if msg.tool_calls:
|
||||
message['tool_calls'] = [
|
||||
{
|
||||
'id': tc.id,
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': tc.name,
|
||||
'arguments': json.dumps(tc.arguments)
|
||||
}
|
||||
}
|
||||
for tc in msg.tool_calls
|
||||
]
|
||||
|
||||
result.append(message)
|
||||
|
||||
elif msg.role == Role.TOOL:
|
||||
# Each tool result is a separate message in OpenAI
|
||||
for tr in msg.tool_results:
|
||||
result.append({
|
||||
'role': 'tool',
|
||||
'tool_call_id': tr.tool_call_id,
|
||||
'content': tr.content
|
||||
})
|
||||
|
||||
return result
|
||||
|
||||
def _convert_tools(self, tools: list[Tool]) -> list[dict]:
|
||||
"""Convert Tool objects to OpenAI API format."""
|
||||
return [
|
||||
{
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': tool.name,
|
||||
'description': tool.description,
|
||||
'parameters': tool.parameters
|
||||
}
|
||||
}
|
||||
for tool in tools
|
||||
]
|
||||
|
||||
def _make_request(self, payload: dict) -> dict:
|
||||
"""Make an HTTP request to the OpenAI API."""
|
||||
headers = {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': f'Bearer {self._api_key}'
|
||||
}
|
||||
|
||||
request = urllib.request.Request(
|
||||
API_URL,
|
||||
data=json.dumps(payload).encode('utf-8'),
|
||||
headers=headers,
|
||||
method='POST'
|
||||
)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(
|
||||
request, timeout=120, context=SSL_CONTEXT
|
||||
) as response:
|
||||
return json.loads(response.read().decode('utf-8'))
|
||||
except urllib.error.HTTPError as e:
|
||||
error_body = e.read().decode('utf-8')
|
||||
try:
|
||||
error_data = json.loads(error_body)
|
||||
error_msg = error_data.get('error', {}).get('message', str(e))
|
||||
except json.JSONDecodeError:
|
||||
error_msg = error_body or str(e)
|
||||
|
||||
raise LLMClientError(LLMError(
|
||||
message=error_msg,
|
||||
code=str(e.code),
|
||||
provider=self.provider_name,
|
||||
retryable=e.code in (429, 500, 502, 503, 504)
|
||||
))
|
||||
except urllib.error.URLError as e:
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Connection error: {e.reason}",
|
||||
provider=self.provider_name,
|
||||
retryable=True
|
||||
))
|
||||
except socket.timeout:
|
||||
raise LLMClientError(LLMError(
|
||||
message="Request timed out. The request may be too large "
|
||||
"or the server is slow to respond.",
|
||||
code='timeout',
|
||||
provider=self.provider_name,
|
||||
retryable=True
|
||||
))
|
||||
|
||||
def _parse_response(self, data: dict) -> LLMResponse:
|
||||
"""Parse the OpenAI API response into an LLMResponse."""
|
||||
# Check for API-level errors in the response
|
||||
if 'error' in data:
|
||||
error_info = data['error']
|
||||
raise LLMClientError(LLMError(
|
||||
message=error_info.get('message', 'Unknown API error'),
|
||||
code=error_info.get('code', 'unknown'),
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
|
||||
choices = data.get('choices', [])
|
||||
if not choices:
|
||||
raise LLMClientError(LLMError(
|
||||
message='No response choices returned from API',
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
|
||||
choice = choices[0]
|
||||
message = choice.get('message', {})
|
||||
|
||||
# Check for refusal (content moderation)
|
||||
if message.get('refusal'):
|
||||
raise LLMClientError(LLMError(
|
||||
message=f"Request refused: {message.get('refusal')}",
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
|
||||
content = message.get('content', '') or ''
|
||||
tool_calls = []
|
||||
|
||||
# Parse tool calls if present
|
||||
for tc in message.get('tool_calls', []):
|
||||
if tc.get('type') == 'function':
|
||||
func = tc.get('function', {})
|
||||
try:
|
||||
arguments = json.loads(func.get('arguments', '{}'))
|
||||
except json.JSONDecodeError:
|
||||
arguments = {}
|
||||
|
||||
tool_calls.append(ToolCall(
|
||||
id=tc.get('id', str(uuid.uuid4())),
|
||||
name=func.get('name', ''),
|
||||
arguments=arguments
|
||||
))
|
||||
|
||||
# Map OpenAI finish reasons to our enum
|
||||
finish_reason = choice.get('finish_reason', '')
|
||||
stop_reason_map = {
|
||||
'stop': StopReason.END_TURN,
|
||||
'tool_calls': StopReason.TOOL_USE,
|
||||
'length': StopReason.MAX_TOKENS,
|
||||
'content_filter': StopReason.STOP_SEQUENCE
|
||||
}
|
||||
stop_reason = stop_reason_map.get(finish_reason, StopReason.UNKNOWN)
|
||||
|
||||
# Parse usage information
|
||||
usage_data = data.get('usage', {})
|
||||
usage = Usage(
|
||||
input_tokens=usage_data.get('prompt_tokens', 0),
|
||||
output_tokens=usage_data.get('completion_tokens', 0),
|
||||
total_tokens=usage_data.get('total_tokens', 0)
|
||||
)
|
||||
|
||||
# Check for problematic responses
|
||||
if not content and not tool_calls:
|
||||
if stop_reason == StopReason.MAX_TOKENS:
|
||||
input_tokens = usage.input_tokens
|
||||
raise LLMClientError(LLMError(
|
||||
message=f'Response truncated due to token limit '
|
||||
f'(input: {input_tokens} tokens). '
|
||||
f'The request is too large for model '
|
||||
f'{self._model}. '
|
||||
f'Try using a model with a larger context '
|
||||
f'window, or analyze a smaller scope (e.g., a '
|
||||
f'specific schema instead of the entire '
|
||||
f'database).',
|
||||
code='max_tokens',
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
elif finish_reason and finish_reason not in ('stop', 'tool_calls'):
|
||||
raise LLMClientError(LLMError(
|
||||
message=(f'Empty response with finish reason: '
|
||||
f'{finish_reason}'),
|
||||
code=finish_reason,
|
||||
provider=self.provider_name,
|
||||
retryable=False
|
||||
))
|
||||
|
||||
return LLMResponse(
|
||||
content=content,
|
||||
tool_calls=tool_calls,
|
||||
stop_reason=stop_reason,
|
||||
model=data.get('model', self._model),
|
||||
usage=usage,
|
||||
raw_response=data
|
||||
)
|
||||
37
web/pgadmin/llm/reports/__init__.py
Normal file
37
web/pgadmin/llm/reports/__init__.py
Normal file
@@ -0,0 +1,37 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Multi-stage LLM report generation pipeline.
|
||||
|
||||
This module provides a staged approach to generating reports that works
|
||||
within token limits of various LLM models by breaking analysis into
|
||||
sections that are summarized independently and then synthesized.
|
||||
"""
|
||||
|
||||
from pgadmin.llm.reports.pipeline import ReportPipeline
|
||||
from pgadmin.llm.reports.models import Section, SectionResult, Severity
|
||||
from pgadmin.llm.reports.sections import (
|
||||
SECURITY_SECTIONS, PERFORMANCE_SECTIONS, DESIGN_SECTIONS,
|
||||
get_sections_for_report, get_sections_for_scope
|
||||
)
|
||||
from pgadmin.llm.reports.queries import get_query, execute_query
|
||||
|
||||
__all__ = [
|
||||
'ReportPipeline',
|
||||
'Section',
|
||||
'SectionResult',
|
||||
'Severity',
|
||||
'SECURITY_SECTIONS',
|
||||
'PERFORMANCE_SECTIONS',
|
||||
'DESIGN_SECTIONS',
|
||||
'get_sections_for_report',
|
||||
'get_sections_for_scope',
|
||||
'get_query',
|
||||
'execute_query',
|
||||
]
|
||||
291
web/pgadmin/llm/reports/generator.py
Normal file
291
web/pgadmin/llm/reports/generator.py
Normal file
@@ -0,0 +1,291 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""High-level report generation functions using the pipeline."""
|
||||
|
||||
import json
|
||||
from typing import Generator, Optional, Any
|
||||
|
||||
from flask import Response, stream_with_context
|
||||
from flask_babel import gettext
|
||||
|
||||
from pgadmin.llm.client import get_llm_client, LLMClient
|
||||
from pgadmin.llm.reports.pipeline import ReportPipeline
|
||||
from pgadmin.llm.reports.sections import get_sections_for_scope
|
||||
from pgadmin.llm.reports.queries import QUERIES
|
||||
|
||||
|
||||
def create_query_executor(conn) -> callable:
|
||||
"""Create a query executor function for the pipeline.
|
||||
|
||||
Args:
|
||||
conn: Database connection object.
|
||||
|
||||
Returns:
|
||||
A callable that executes queries by ID.
|
||||
"""
|
||||
def executor(query_id: str, context: dict) -> dict[str, Any]:
|
||||
"""Execute a query by ID.
|
||||
|
||||
Args:
|
||||
query_id: The query identifier from QUERIES registry.
|
||||
context: Execution context (may contain schema_id for filtering).
|
||||
|
||||
Returns:
|
||||
Dictionary with query results.
|
||||
"""
|
||||
query_def = QUERIES.get(query_id)
|
||||
if not query_def:
|
||||
return {'error': f'Unknown query: {query_id}', 'rows': []}
|
||||
|
||||
sql = query_def['sql']
|
||||
|
||||
# Check if query requires an extension
|
||||
required_ext = query_def.get('requires_extension')
|
||||
if required_ext:
|
||||
check_sql = """
|
||||
SELECT EXISTS (
|
||||
SELECT 1 FROM pg_extension WHERE extname = %s
|
||||
) as available
|
||||
"""
|
||||
status, result = conn.execute_dict(check_sql, [required_ext])
|
||||
if not (status and result and
|
||||
result.get('rows', [{}])[0].get('available', False)):
|
||||
return {
|
||||
'note': f"Extension '{required_ext}' not installed",
|
||||
'rows': []
|
||||
}
|
||||
|
||||
# Handle schema-scoped queries
|
||||
schema_id = context.get('schema_id')
|
||||
if schema_id and '%s' in sql:
|
||||
status, result = conn.execute_dict(sql, [schema_id])
|
||||
else:
|
||||
status, result = conn.execute_dict(sql)
|
||||
|
||||
if status and result:
|
||||
return {'rows': result.get('rows', [])}
|
||||
else:
|
||||
return {'error': 'Query failed', 'rows': []}
|
||||
|
||||
return executor
|
||||
|
||||
|
||||
def generate_report_streaming(
|
||||
report_type: str,
|
||||
scope: str,
|
||||
conn,
|
||||
manager,
|
||||
context: dict,
|
||||
client: Optional[LLMClient] = None
|
||||
) -> Generator[str, None, None]:
|
||||
"""Generate a report with streaming progress updates.
|
||||
|
||||
Yields Server-Sent Events (SSE) formatted strings.
|
||||
|
||||
Args:
|
||||
report_type: One of 'security', 'performance', 'design'.
|
||||
scope: One of 'server', 'database', 'schema'.
|
||||
conn: Database connection.
|
||||
manager: Connection manager.
|
||||
context: Report context dict with keys like:
|
||||
- server_version
|
||||
- database_name
|
||||
- schema_name
|
||||
- schema_id (for schema-scoped reports)
|
||||
client: Optional LLM client (will create one if not provided).
|
||||
|
||||
Yields:
|
||||
SSE-formatted event strings.
|
||||
"""
|
||||
# Get or create LLM client
|
||||
if client is None:
|
||||
client = get_llm_client()
|
||||
if not client:
|
||||
yield _sse_event({
|
||||
'type': 'error',
|
||||
'message': gettext('Failed to initialize LLM client.')
|
||||
})
|
||||
return
|
||||
|
||||
# Get sections for this report type and scope
|
||||
sections = get_sections_for_scope(report_type, scope)
|
||||
if not sections:
|
||||
yield _sse_event({
|
||||
'type': 'error',
|
||||
'message': gettext('No sections available for this report type.')
|
||||
})
|
||||
return
|
||||
|
||||
# Add server version to context
|
||||
context['server_version'] = manager.ver
|
||||
|
||||
# Create the pipeline
|
||||
query_executor = create_query_executor(conn)
|
||||
pipeline = ReportPipeline(
|
||||
report_type=report_type,
|
||||
sections=sections,
|
||||
client=client,
|
||||
query_executor=query_executor
|
||||
)
|
||||
|
||||
# Execute pipeline and stream events
|
||||
try:
|
||||
for event in pipeline.execute_with_progress(context):
|
||||
if event.get('type') == 'complete':
|
||||
# Add disclaimer to final report
|
||||
report = event.get('report', '')
|
||||
disclaimer = gettext(
|
||||
'> **Note:** This report was generated by '
|
||||
'%(provider)s / %(model)s. '
|
||||
'AI systems can make mistakes. Please verify all findings '
|
||||
'and recommendations before taking action.\n\n'
|
||||
) % {
|
||||
'provider': client.provider_name,
|
||||
'model': client.model_name
|
||||
}
|
||||
event['report'] = disclaimer + report
|
||||
|
||||
yield _sse_event(event)
|
||||
|
||||
except Exception as e:
|
||||
yield _sse_event({
|
||||
'type': 'error',
|
||||
'message': gettext('Failed to generate report: ') + str(e)
|
||||
})
|
||||
|
||||
|
||||
def generate_report_sync(
|
||||
report_type: str,
|
||||
scope: str,
|
||||
conn,
|
||||
manager,
|
||||
context: dict,
|
||||
client: Optional[LLMClient] = None
|
||||
) -> tuple[bool, str]:
|
||||
"""Generate a report synchronously (non-streaming).
|
||||
|
||||
Args:
|
||||
report_type: One of 'security', 'performance', 'design'.
|
||||
scope: One of 'server', 'database', 'schema'.
|
||||
conn: Database connection.
|
||||
manager: Connection manager.
|
||||
context: Report context dict.
|
||||
client: Optional LLM client.
|
||||
|
||||
Returns:
|
||||
Tuple of (success, report_or_error_message).
|
||||
"""
|
||||
# Get or create LLM client
|
||||
if client is None:
|
||||
client = get_llm_client()
|
||||
if not client:
|
||||
return False, gettext('Failed to initialize LLM client.')
|
||||
|
||||
# Get sections for this report type and scope
|
||||
sections = get_sections_for_scope(report_type, scope)
|
||||
if not sections:
|
||||
return False, gettext('No sections available for this report type.')
|
||||
|
||||
# Add server version to context
|
||||
context['server_version'] = manager.ver
|
||||
|
||||
# Create and execute the pipeline
|
||||
query_executor = create_query_executor(conn)
|
||||
pipeline = ReportPipeline(
|
||||
report_type=report_type,
|
||||
sections=sections,
|
||||
client=client,
|
||||
query_executor=query_executor
|
||||
)
|
||||
|
||||
try:
|
||||
report = pipeline.execute(context)
|
||||
|
||||
# Add disclaimer
|
||||
disclaimer = gettext(
|
||||
'> **Note:** This report was generated by '
|
||||
'%(provider)s / %(model)s. '
|
||||
'AI systems can make mistakes. Please verify all findings '
|
||||
'and recommendations before taking action.\n\n'
|
||||
) % {
|
||||
'provider': client.provider_name,
|
||||
'model': client.model_name
|
||||
}
|
||||
|
||||
return True, disclaimer + report
|
||||
|
||||
except Exception as e:
|
||||
return False, gettext('Failed to generate report: ') + str(e)
|
||||
|
||||
|
||||
def _sse_event(data: dict) -> bytes:
|
||||
"""Format data as an SSE event.
|
||||
|
||||
Args:
|
||||
data: Event data dictionary.
|
||||
|
||||
Returns:
|
||||
SSE-formatted bytes with padding to help flush buffers.
|
||||
"""
|
||||
# Add padding comment to help flush buffers in some WSGI servers
|
||||
# Some servers buffer until a certain amount of data is received
|
||||
json_data = json.dumps(data)
|
||||
# Minimum 2KB total to help flush various buffer sizes
|
||||
padding_needed = max(0, 2048 - len(json_data) - 20)
|
||||
padding = f": {'.' * padding_needed}\n" if padding_needed > 0 else ""
|
||||
return f"{padding}data: {json_data}\n\n".encode('utf-8')
|
||||
|
||||
|
||||
def _wrap_generator_with_keepalive(generator: Generator) -> Generator:
|
||||
"""Wrap a generator to add SSE keepalive and initial flush.
|
||||
|
||||
Args:
|
||||
generator: Original event generator.
|
||||
|
||||
Yields:
|
||||
SSE events (as bytes) with initial connection event.
|
||||
"""
|
||||
# Send initial comment to establish connection and flush headers
|
||||
# The retry directive tells browser to reconnect after 3s if disconnected
|
||||
yield b": SSE stream connected\nretry: 3000\n\n"
|
||||
|
||||
# Yield all events from the original generator
|
||||
for event in generator:
|
||||
yield event
|
||||
|
||||
|
||||
def create_sse_response(generator: Generator) -> Response:
|
||||
"""Create a Flask Response for SSE streaming.
|
||||
|
||||
Args:
|
||||
generator: Generator that yields SSE event strings.
|
||||
|
||||
Returns:
|
||||
Flask Response configured for SSE.
|
||||
"""
|
||||
# Wrap generator with keepalive/flush helper
|
||||
wrapped = _wrap_generator_with_keepalive(generator)
|
||||
|
||||
# stream_with_context maintains Flask's request context throughout
|
||||
# the generator's lifecycle, which is required for streaming responses
|
||||
response = Response(
|
||||
stream_with_context(wrapped),
|
||||
mimetype='text/event-stream',
|
||||
headers={
|
||||
'Cache-Control': 'no-cache, no-store, must-revalidate',
|
||||
'Pragma': 'no-cache',
|
||||
'Expires': '0',
|
||||
'Connection': 'keep-alive',
|
||||
'X-Accel-Buffering': 'no', # Disable nginx buffering
|
||||
}
|
||||
)
|
||||
# Disable Werkzeug's response buffering - critical for SSE to work
|
||||
response.direct_passthrough = True
|
||||
return response
|
||||
79
web/pgadmin/llm/reports/models.py
Normal file
79
web/pgadmin/llm/reports/models.py
Normal file
@@ -0,0 +1,79 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Data models for the report generation pipeline."""
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Optional
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class Severity(str, Enum):
|
||||
"""Severity levels for report findings."""
|
||||
CRITICAL = 'critical'
|
||||
WARNING = 'warning'
|
||||
ADVISORY = 'advisory'
|
||||
GOOD = 'good'
|
||||
INFO = 'info'
|
||||
|
||||
|
||||
@dataclass
|
||||
class Section:
|
||||
"""Definition of a report section.
|
||||
|
||||
Attributes:
|
||||
id: Unique identifier for the section.
|
||||
name: Human-readable name for display.
|
||||
description: What this section analyzes.
|
||||
queries: List of query identifiers to run for this section.
|
||||
scope: What scope this section applies to
|
||||
('server', 'database', 'schema').
|
||||
"""
|
||||
id: str
|
||||
name: str
|
||||
description: str
|
||||
queries: list[str]
|
||||
scope: list[str] = field(
|
||||
default_factory=lambda: ['server', 'database', 'schema']
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class SectionResult:
|
||||
"""Result from analyzing a report section.
|
||||
|
||||
Attributes:
|
||||
section_id: The section that was analyzed.
|
||||
section_name: Human-readable section name.
|
||||
data: Raw data gathered from SQL queries.
|
||||
summary: LLM-generated summary of the section.
|
||||
severity: Overall severity of findings in this section.
|
||||
error: Error message if analysis failed.
|
||||
"""
|
||||
section_id: str
|
||||
section_name: str
|
||||
data: dict[str, Any] = field(default_factory=dict)
|
||||
summary: str = ''
|
||||
severity: Severity = Severity.INFO
|
||||
error: Optional[str] = None
|
||||
|
||||
@property
|
||||
def has_error(self) -> bool:
|
||||
"""Check if this section had an error."""
|
||||
return self.error is not None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary representation."""
|
||||
return {
|
||||
'section_id': self.section_id,
|
||||
'section_name': self.section_name,
|
||||
'summary': self.summary,
|
||||
'severity': self.severity.value,
|
||||
'error': self.error
|
||||
}
|
||||
464
web/pgadmin/llm/reports/pipeline.py
Normal file
464
web/pgadmin/llm/reports/pipeline.py
Normal file
@@ -0,0 +1,464 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Core report generation pipeline implementation."""
|
||||
|
||||
import json
|
||||
import time
|
||||
from typing import Generator, Callable, Any
|
||||
|
||||
from pgadmin.llm.client import LLMClient, LLMClientError
|
||||
from pgadmin.llm.models import Message
|
||||
from pgadmin.llm.reports.models import (
|
||||
Section, SectionResult, Severity
|
||||
)
|
||||
from pgadmin.llm.reports.prompts import (
|
||||
PLANNING_SYSTEM_PROMPT, get_planning_user_prompt,
|
||||
SECTION_ANALYSIS_SYSTEM_PROMPT, get_section_analysis_prompt,
|
||||
SYNTHESIS_SYSTEM_PROMPT, get_synthesis_prompt
|
||||
)
|
||||
|
||||
|
||||
class ReportPipelineError(Exception):
|
||||
"""Error during report pipeline execution."""
|
||||
pass
|
||||
|
||||
|
||||
class ReportPipeline:
|
||||
"""Multi-stage report generation pipeline.
|
||||
|
||||
This pipeline breaks report generation into 4 stages:
|
||||
1. Planning - LLM selects which sections to analyze
|
||||
2. Data Gathering - Run SQL queries for each section
|
||||
3. Section Analysis - LLM summarizes each section independently
|
||||
4. Synthesis - LLM merges section summaries into final report
|
||||
|
||||
This approach keeps each LLM call within token limits while
|
||||
producing comprehensive, well-structured reports.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
report_type: str,
|
||||
sections: list[Section],
|
||||
client: LLMClient,
|
||||
query_executor: Callable[[str, dict], dict],
|
||||
max_retries: int = 3,
|
||||
retry_base_delay: float = 5.0
|
||||
):
|
||||
"""Initialize the pipeline.
|
||||
|
||||
Args:
|
||||
report_type: Type of report ('security', 'performance', 'design').
|
||||
sections: List of available Section definitions.
|
||||
client: LLM client for making API calls.
|
||||
query_executor: Function to execute queries given query_id and
|
||||
context.
|
||||
max_retries: Maximum retry attempts for rate-limited calls.
|
||||
retry_base_delay: Base delay in seconds for exponential backoff.
|
||||
"""
|
||||
self.report_type = report_type
|
||||
self.sections = {s.id: s for s in sections}
|
||||
self.client = client
|
||||
self.query_executor = query_executor
|
||||
self.max_retries = max_retries
|
||||
self.retry_base_delay = retry_base_delay
|
||||
|
||||
def execute(self, context: dict) -> str:
|
||||
"""Execute the pipeline and return the final report.
|
||||
|
||||
Args:
|
||||
context: Dictionary with database context (server_version,
|
||||
database_name, schema_name, etc.)
|
||||
|
||||
Returns:
|
||||
Final report as markdown string.
|
||||
|
||||
Raises:
|
||||
ReportPipelineError: If pipeline fails.
|
||||
"""
|
||||
# Consume the generator to get final result
|
||||
result = None
|
||||
for event in self.execute_with_progress(context):
|
||||
if event.get('type') == 'complete':
|
||||
result = event.get('report', '')
|
||||
elif event.get('type') == 'error':
|
||||
raise ReportPipelineError(
|
||||
event.get('message', 'Unknown error')
|
||||
)
|
||||
return result or ''
|
||||
|
||||
def execute_with_progress(
|
||||
self,
|
||||
context: dict
|
||||
) -> Generator[dict, None, None]:
|
||||
"""Execute the pipeline with progress updates.
|
||||
|
||||
Yields SSE-compatible event dictionaries throughout execution.
|
||||
|
||||
Args:
|
||||
context: Dictionary with database context.
|
||||
|
||||
Yields:
|
||||
Event dictionaries with type, stage, message, etc.
|
||||
"""
|
||||
try:
|
||||
# Stage 1: Planning
|
||||
yield {'type': 'stage', 'stage': 'planning',
|
||||
'message': 'Planning analysis sections...'}
|
||||
|
||||
selected_section_ids = self._planning_stage(context)
|
||||
|
||||
if not selected_section_ids:
|
||||
# Fallback to all sections if planning returns empty
|
||||
selected_section_ids = list(self.sections.keys())
|
||||
|
||||
total_sections = len(selected_section_ids)
|
||||
|
||||
# Stage 2: Data Gathering
|
||||
yield {'type': 'stage', 'stage': 'gathering',
|
||||
'message': 'Gathering data...'}
|
||||
|
||||
section_data = {}
|
||||
for i, section_id in enumerate(selected_section_ids):
|
||||
section = self.sections.get(section_id)
|
||||
if not section:
|
||||
continue
|
||||
|
||||
yield {'type': 'progress', 'stage': 'gathering',
|
||||
'section': section.name,
|
||||
'message': f'Gathering {section.name} data...',
|
||||
'completed': i, 'total': total_sections}
|
||||
|
||||
section_data[section_id] = self._gather_section_data(
|
||||
section, context
|
||||
)
|
||||
|
||||
# Stage 3: Section Analysis
|
||||
yield {'type': 'stage', 'stage': 'analyzing',
|
||||
'message': 'Analyzing sections...'}
|
||||
|
||||
section_results = []
|
||||
for i, section_id in enumerate(selected_section_ids):
|
||||
section = self.sections.get(section_id)
|
||||
if not section or section_id not in section_data:
|
||||
continue
|
||||
|
||||
yield {'type': 'progress', 'stage': 'analyzing',
|
||||
'section': section.name,
|
||||
'message': f'Analyzing {section.name}...',
|
||||
'completed': i, 'total': total_sections}
|
||||
|
||||
# Call LLM with retry for rate limits
|
||||
for retry_event in self._analyze_section_with_retry(
|
||||
section, section_data[section_id], context
|
||||
):
|
||||
if retry_event.get('type') == 'retry':
|
||||
yield retry_event
|
||||
elif retry_event.get('type') == 'result':
|
||||
section_results.append(retry_event['result'])
|
||||
|
||||
# Stage 4: Synthesis
|
||||
yield {'type': 'stage', 'stage': 'synthesizing',
|
||||
'message': 'Creating final report...'}
|
||||
|
||||
final_report = ''
|
||||
for retry_event in self._synthesize_with_retry(
|
||||
section_results, context
|
||||
):
|
||||
if retry_event.get('type') == 'retry':
|
||||
yield retry_event
|
||||
elif retry_event.get('type') == 'result':
|
||||
final_report = retry_event['result']
|
||||
|
||||
yield {'type': 'complete', 'report': final_report}
|
||||
|
||||
except ReportPipelineError:
|
||||
raise
|
||||
except Exception as e:
|
||||
yield {'type': 'error', 'message': str(e)}
|
||||
|
||||
def _planning_stage(self, context: dict) -> list[str]:
|
||||
"""Run the planning stage to select relevant sections.
|
||||
|
||||
Args:
|
||||
context: Database context.
|
||||
|
||||
Returns:
|
||||
List of section IDs to analyze.
|
||||
"""
|
||||
# Filter sections by scope
|
||||
scope = 'server'
|
||||
if context.get('schema_name'):
|
||||
scope = 'schema'
|
||||
elif context.get('database_name'):
|
||||
scope = 'database'
|
||||
|
||||
available_sections = [
|
||||
{'id': s.id, 'name': s.name, 'description': s.description}
|
||||
for s in self.sections.values()
|
||||
if scope in s.scope
|
||||
]
|
||||
|
||||
if not available_sections:
|
||||
return []
|
||||
|
||||
# Ask LLM to select sections
|
||||
user_prompt = get_planning_user_prompt(
|
||||
self.report_type, available_sections, context
|
||||
)
|
||||
|
||||
try:
|
||||
response = self._call_llm_with_retry(
|
||||
messages=[Message.user(user_prompt)],
|
||||
system_prompt=PLANNING_SYSTEM_PROMPT,
|
||||
max_tokens=500,
|
||||
temperature=0.0
|
||||
)
|
||||
|
||||
# Parse JSON response
|
||||
content = response.content.strip()
|
||||
# Handle markdown code blocks
|
||||
if content.startswith('```'):
|
||||
content = content.split('\n', 1)[1]
|
||||
content = content.rsplit('```', 1)[0]
|
||||
|
||||
selected_ids = json.loads(content)
|
||||
|
||||
# Validate section IDs
|
||||
valid_ids = [
|
||||
sid for sid in selected_ids
|
||||
if sid in self.sections
|
||||
]
|
||||
|
||||
return (valid_ids if valid_ids else
|
||||
[s['id'] for s in available_sections])
|
||||
|
||||
except (json.JSONDecodeError, LLMClientError):
|
||||
# Fallback to all available sections
|
||||
return [s['id'] for s in available_sections]
|
||||
|
||||
def _gather_section_data(
|
||||
self,
|
||||
section: Section,
|
||||
context: dict
|
||||
) -> dict[str, Any]:
|
||||
"""Gather data for a section by executing its queries.
|
||||
|
||||
Args:
|
||||
section: Section definition with query IDs.
|
||||
context: Database context.
|
||||
|
||||
Returns:
|
||||
Dictionary mapping query_id to query results.
|
||||
"""
|
||||
data = {}
|
||||
for query_id in section.queries:
|
||||
try:
|
||||
result = self.query_executor(query_id, context)
|
||||
data[query_id] = result
|
||||
except Exception as e:
|
||||
data[query_id] = {'error': str(e)}
|
||||
return data
|
||||
|
||||
def _analyze_section_with_retry(
|
||||
self,
|
||||
section: Section,
|
||||
data: dict,
|
||||
context: dict
|
||||
) -> Generator[dict, None, None]:
|
||||
"""Analyze a section with retry logic.
|
||||
|
||||
Args:
|
||||
section: Section to analyze.
|
||||
data: Query results for this section.
|
||||
context: Database context.
|
||||
|
||||
Yields:
|
||||
Retry events and final result event.
|
||||
"""
|
||||
user_prompt = get_section_analysis_prompt(
|
||||
section.name, section.description, data, context
|
||||
)
|
||||
|
||||
for attempt in range(self.max_retries):
|
||||
try:
|
||||
response = self.client.chat(
|
||||
messages=[Message.user(user_prompt)],
|
||||
system_prompt=SECTION_ANALYSIS_SYSTEM_PROMPT,
|
||||
max_tokens=1500,
|
||||
temperature=0.3
|
||||
)
|
||||
|
||||
# Determine severity from content
|
||||
severity = self._extract_severity(response.content)
|
||||
|
||||
result = SectionResult(
|
||||
section_id=section.id,
|
||||
section_name=section.name,
|
||||
data=data,
|
||||
summary=response.content,
|
||||
severity=severity
|
||||
)
|
||||
|
||||
yield {'type': 'result', 'result': result}
|
||||
return
|
||||
|
||||
except LLMClientError as e:
|
||||
if e.error.retryable and attempt < self.max_retries - 1:
|
||||
wait_time = int(self.retry_base_delay * (2 ** attempt))
|
||||
yield {
|
||||
'type': 'retry',
|
||||
'reason': 'rate_limit',
|
||||
'message': (f'Rate limited, retrying in '
|
||||
f'{wait_time}s...'),
|
||||
'wait_seconds': wait_time
|
||||
}
|
||||
time.sleep(wait_time)
|
||||
else:
|
||||
# Return error result
|
||||
result = SectionResult(
|
||||
section_id=section.id,
|
||||
section_name=section.name,
|
||||
data=data,
|
||||
error=str(e)
|
||||
)
|
||||
yield {'type': 'result', 'result': result}
|
||||
return
|
||||
|
||||
def _synthesize_with_retry(
|
||||
self,
|
||||
section_results: list[SectionResult],
|
||||
context: dict
|
||||
) -> Generator[dict, None, None]:
|
||||
"""Synthesize final report with retry logic.
|
||||
|
||||
Args:
|
||||
section_results: Results from section analysis.
|
||||
context: Database context.
|
||||
|
||||
Yields:
|
||||
Retry events and final result event.
|
||||
"""
|
||||
# Filter out failed sections
|
||||
successful_results = [
|
||||
{
|
||||
'section_id': r.section_id,
|
||||
'section_name': r.section_name,
|
||||
'summary': r.summary,
|
||||
'severity': r.severity.value
|
||||
}
|
||||
for r in section_results
|
||||
if not r.has_error and r.summary
|
||||
]
|
||||
|
||||
if not successful_results:
|
||||
yield {
|
||||
'type': 'result',
|
||||
'result': '**Error**: No sections were successfully analyzed.'
|
||||
}
|
||||
return
|
||||
|
||||
user_prompt = get_synthesis_prompt(
|
||||
self.report_type, successful_results, context
|
||||
)
|
||||
|
||||
for attempt in range(self.max_retries):
|
||||
try:
|
||||
response = self.client.chat(
|
||||
messages=[Message.user(user_prompt)],
|
||||
system_prompt=SYNTHESIS_SYSTEM_PROMPT,
|
||||
max_tokens=4096,
|
||||
temperature=0.3
|
||||
)
|
||||
|
||||
yield {'type': 'result', 'result': response.content}
|
||||
return
|
||||
|
||||
except LLMClientError as e:
|
||||
if e.error.retryable and attempt < self.max_retries - 1:
|
||||
wait_time = int(self.retry_base_delay * (2 ** attempt))
|
||||
yield {
|
||||
'type': 'retry',
|
||||
'reason': 'rate_limit',
|
||||
'message': (f'Rate limited, retrying in '
|
||||
f'{wait_time}s...'),
|
||||
'wait_seconds': wait_time
|
||||
}
|
||||
time.sleep(wait_time)
|
||||
else:
|
||||
# Return partial report with section summaries
|
||||
partial = (
|
||||
"**Note**: Synthesis failed. Section summaries:\n\n"
|
||||
)
|
||||
for r in successful_results:
|
||||
partial += (
|
||||
f"## {r['section_name']}\n\n{r['summary']}\n\n"
|
||||
)
|
||||
yield {'type': 'result', 'result': partial}
|
||||
return
|
||||
|
||||
def _call_llm_with_retry(
|
||||
self,
|
||||
messages: list[Message],
|
||||
system_prompt: str,
|
||||
max_tokens: int = 4096,
|
||||
temperature: float = 0.3
|
||||
):
|
||||
"""Call LLM with exponential backoff retry.
|
||||
|
||||
Args:
|
||||
messages: Messages to send.
|
||||
system_prompt: System prompt.
|
||||
max_tokens: Maximum response tokens.
|
||||
temperature: Sampling temperature.
|
||||
|
||||
Returns:
|
||||
LLMResponse from the client.
|
||||
|
||||
Raises:
|
||||
LLMClientError: If all retries fail.
|
||||
"""
|
||||
for attempt in range(self.max_retries):
|
||||
try:
|
||||
return self.client.chat(
|
||||
messages=messages,
|
||||
system_prompt=system_prompt,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature
|
||||
)
|
||||
except LLMClientError as e:
|
||||
if e.error.retryable and attempt < self.max_retries - 1:
|
||||
wait_time = self.retry_base_delay * (2 ** attempt)
|
||||
time.sleep(wait_time)
|
||||
else:
|
||||
raise
|
||||
|
||||
def _extract_severity(self, content: str) -> Severity:
|
||||
"""Extract overall severity from section analysis content.
|
||||
|
||||
Args:
|
||||
content: LLM response content.
|
||||
|
||||
Returns:
|
||||
Extracted Severity level.
|
||||
"""
|
||||
content_lower = content.lower()
|
||||
|
||||
# Look for status line
|
||||
if '**status**: critical' in content_lower or '🔴' in content:
|
||||
return Severity.CRITICAL
|
||||
elif '**status**: warning' in content_lower or '🟠' in content:
|
||||
return Severity.WARNING
|
||||
elif '**status**: advisory' in content_lower or '🟡' in content:
|
||||
return Severity.ADVISORY
|
||||
elif '**status**: good' in content_lower or '🟢' in content:
|
||||
return Severity.GOOD
|
||||
|
||||
return Severity.INFO
|
||||
250
web/pgadmin/llm/reports/prompts.py
Normal file
250
web/pgadmin/llm/reports/prompts.py
Normal file
@@ -0,0 +1,250 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Prompt templates for report generation pipeline stages."""
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Planning Stage Prompts
|
||||
# =============================================================================
|
||||
|
||||
PLANNING_SYSTEM_PROMPT = """You are a PostgreSQL expert helping to plan a
|
||||
database analysis report.
|
||||
|
||||
Your task is to select which analysis sections are most relevant for the
|
||||
given report type and database context.
|
||||
|
||||
Return ONLY a JSON array of section IDs to analyze, ordered by priority.
|
||||
Only include sections that are relevant given the database characteristics.
|
||||
Do not include any explanation, just the JSON array."""
|
||||
|
||||
|
||||
def get_planning_user_prompt(
|
||||
report_type: str,
|
||||
sections: list[dict],
|
||||
context: dict
|
||||
) -> str:
|
||||
"""Build the planning stage user prompt.
|
||||
|
||||
Args:
|
||||
report_type: Type of report ('security', 'performance', 'design').
|
||||
sections: List of available sections with id, name, description.
|
||||
context: Database context (version, size, table count, etc.).
|
||||
|
||||
Returns:
|
||||
Formatted user prompt for planning.
|
||||
"""
|
||||
sections_list = '\n'.join([
|
||||
f"- {s['id']}: {s['name']} - {s['description']}"
|
||||
for s in sections
|
||||
])
|
||||
|
||||
return f"""Select the most relevant sections for a {report_type} report.
|
||||
|
||||
Available sections:
|
||||
{sections_list}
|
||||
|
||||
Database context:
|
||||
- Server version: {context.get('server_version', 'Unknown')}
|
||||
- Database name: {context.get('database_name', 'N/A')}
|
||||
- Schema name: {context.get('schema_name', 'N/A')}
|
||||
- Table count: {context.get('table_count', 'Unknown')}
|
||||
- Has pg_stat_statements: {context.get('has_stat_statements', False)}
|
||||
|
||||
Return a JSON array of section IDs to analyze, e.g.:
|
||||
["section1", "section2", "section3"]"""
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Section Analysis Prompts
|
||||
# =============================================================================
|
||||
|
||||
SECTION_ANALYSIS_SYSTEM_PROMPT = """You are a PostgreSQL expert analyzing
|
||||
database configuration.
|
||||
|
||||
Analyze the provided data and generate a concise summary (max 300 words).
|
||||
|
||||
Your response MUST follow this exact format:
|
||||
### [Section Name]
|
||||
|
||||
**Status**: [One of: Good, Advisory, Warning, Critical]
|
||||
|
||||
**Findings**:
|
||||
- [Finding 1]
|
||||
- [Finding 2]
|
||||
- [etc.]
|
||||
|
||||
**Recommendations**:
|
||||
- [Recommendation 1 with specific action]
|
||||
- [Recommendation 2 with specific action]
|
||||
- [etc.]
|
||||
|
||||
Use these severity indicators in findings:
|
||||
- 🔴 for Critical issues
|
||||
- 🟠 for Warning issues
|
||||
- 🟡 for Advisory items
|
||||
- 🟢 for Good/positive findings
|
||||
|
||||
Be specific and actionable. Include SQL commands where relevant."""
|
||||
|
||||
|
||||
def get_section_analysis_prompt(
|
||||
section_name: str,
|
||||
section_description: str,
|
||||
data: dict,
|
||||
context: dict
|
||||
) -> str:
|
||||
"""Build the section analysis user prompt.
|
||||
|
||||
Args:
|
||||
section_name: Name of the section being analyzed.
|
||||
section_description: Description of what this section covers.
|
||||
data: Query results for this section.
|
||||
context: Database context.
|
||||
|
||||
Returns:
|
||||
Formatted user prompt for section analysis.
|
||||
"""
|
||||
import json
|
||||
|
||||
data_json = json.dumps(data, indent=2, default=str)
|
||||
|
||||
return f"""Analyze the following {section_name} data for a PostgreSQL
|
||||
{context.get('server_version', '')} server.
|
||||
|
||||
Section focus: {section_description}
|
||||
|
||||
Database: {context.get('database_name', 'N/A')}
|
||||
Schema: {context.get('schema_name', 'all schemas')}
|
||||
|
||||
Data:
|
||||
```json
|
||||
{data_json}
|
||||
```
|
||||
|
||||
Provide your analysis following the required format."""
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Synthesis Prompts
|
||||
# =============================================================================
|
||||
|
||||
SYNTHESIS_SYSTEM_PROMPT = """You are a PostgreSQL expert creating a
|
||||
comprehensive report.
|
||||
|
||||
Combine the section summaries into a cohesive, well-organized report.
|
||||
|
||||
Your report MUST:
|
||||
1. Start with an **Executive Summary** (3-5 sentences overview)
|
||||
2. Include a **Critical Issues** section (aggregate all critical/warning
|
||||
findings)
|
||||
3. Include each section's detailed analysis (use the section content as-is,
|
||||
don't add duplicate headers)
|
||||
4. End with **Prioritized Recommendations** (numbered list, most important
|
||||
first)
|
||||
|
||||
IMPORTANT:
|
||||
- Do NOT include a report title at the very beginning - start directly with
|
||||
Executive Summary
|
||||
- Each section already has its own ### header - do NOT add extra headers
|
||||
around them
|
||||
- Simply organize and flow the sections together naturally
|
||||
|
||||
Use severity indicators consistently:
|
||||
- 🔴 Critical - Immediate action required
|
||||
- 🟠 Warning - Should be addressed soon
|
||||
- 🟡 Advisory - Consider improving
|
||||
- 🟢 Good - No issues found
|
||||
|
||||
Be professional and actionable. Include SQL commands for recommendations
|
||||
where helpful."""
|
||||
|
||||
|
||||
def get_synthesis_prompt(
|
||||
report_type: str,
|
||||
section_summaries: list[dict],
|
||||
context: dict
|
||||
) -> str:
|
||||
"""Build the synthesis stage user prompt.
|
||||
|
||||
Args:
|
||||
report_type: Type of report being generated.
|
||||
section_summaries: List of section results with summaries.
|
||||
context: Database context.
|
||||
|
||||
Returns:
|
||||
Formatted user prompt for synthesis.
|
||||
"""
|
||||
# Don't add extra headers - the section summaries already include them
|
||||
summaries_text = '\n\n---\n\n'.join([
|
||||
s['summary']
|
||||
for s in section_summaries
|
||||
if s.get('summary') and not s.get('error')
|
||||
])
|
||||
|
||||
report_type_display = {
|
||||
'security': 'Security',
|
||||
'performance': 'Performance',
|
||||
'design': 'Design Review'
|
||||
}.get(report_type, report_type.title())
|
||||
|
||||
scope_info = context.get('database_name', 'server')
|
||||
if context.get('schema_name'):
|
||||
scope_info = f"{context['schema_name']} schema in {scope_info}"
|
||||
|
||||
return f"""Create a comprehensive {report_type_display} Report for
|
||||
{scope_info}.
|
||||
|
||||
Server: PostgreSQL {context.get('server_version', 'Unknown')}
|
||||
|
||||
Section Summaries:
|
||||
|
||||
{summaries_text}
|
||||
|
||||
---
|
||||
|
||||
Combine these into a final report following the required format.
|
||||
Start with Executive Summary (do not add a title before it)."""
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Report Type Specific Guidance
|
||||
# =============================================================================
|
||||
|
||||
SECURITY_GUIDANCE = """
|
||||
Focus areas for security analysis:
|
||||
- Authentication configuration and password policies
|
||||
- Role privileges and permission escalation risks
|
||||
- Network exposure and connection security
|
||||
- Encryption settings (SSL/TLS, password hashing)
|
||||
- Row-level security and object permissions
|
||||
- Security definer functions
|
||||
- Audit logging configuration
|
||||
"""
|
||||
|
||||
PERFORMANCE_GUIDANCE = """
|
||||
Focus areas for performance analysis:
|
||||
- Memory configuration (shared_buffers, work_mem, effective_cache_size)
|
||||
- Checkpoint and WAL settings
|
||||
- Autovacuum effectiveness
|
||||
- Query planner configuration
|
||||
- Index utilization and missing indexes
|
||||
- Cache hit ratios
|
||||
- Connection management
|
||||
"""
|
||||
|
||||
DESIGN_GUIDANCE = """
|
||||
Focus areas for design analysis:
|
||||
- Table structure and normalization
|
||||
- Primary key and foreign key design
|
||||
- Index strategy and coverage
|
||||
- Constraint completeness
|
||||
- Data type appropriateness
|
||||
- Naming conventions
|
||||
"""
|
||||
1001
web/pgadmin/llm/reports/queries.py
Normal file
1001
web/pgadmin/llm/reports/queries.py
Normal file
File diff suppressed because it is too large
Load Diff
387
web/pgadmin/llm/reports/sections.py
Normal file
387
web/pgadmin/llm/reports/sections.py
Normal file
@@ -0,0 +1,387 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Section definitions for report generation pipeline.
|
||||
|
||||
Each report type has a set of sections that can be analyzed independently.
|
||||
Sections are mapped to SQL queries and have descriptions for LLM guidance.
|
||||
"""
|
||||
|
||||
from pgadmin.llm.reports.models import Section
|
||||
|
||||
# =============================================================================
|
||||
# SECURITY REPORT SECTIONS
|
||||
# =============================================================================
|
||||
|
||||
SECURITY_SECTIONS = [
|
||||
Section(
|
||||
id='authentication',
|
||||
name='Authentication Configuration',
|
||||
description=(
|
||||
'Password policies, SSL/TLS settings, authentication methods, '
|
||||
'and connection security settings.'
|
||||
),
|
||||
queries=['security_settings', 'hba_rules'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='access_control',
|
||||
name='Access Control & Roles',
|
||||
description=(
|
||||
'Superuser accounts, privileged roles, login roles, '
|
||||
'and role privilege assignments.'
|
||||
),
|
||||
queries=['superusers', 'privileged_roles', 'roles_no_expiry'],
|
||||
scope=['server', 'database']
|
||||
),
|
||||
Section(
|
||||
id='network_security',
|
||||
name='Network Security',
|
||||
description=(
|
||||
'Network exposure settings, listen addresses, connection limits, '
|
||||
'and pg_hba.conf rules.'
|
||||
),
|
||||
queries=['security_settings', 'hba_rules'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='encryption',
|
||||
name='Encryption & SSL',
|
||||
description=(
|
||||
'SSL/TLS configuration, password encryption method, '
|
||||
'and data-at-rest encryption settings.'
|
||||
),
|
||||
queries=['security_settings'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='object_permissions',
|
||||
name='Object Permissions',
|
||||
description=(
|
||||
'Schema, table, and function access control lists (ACLs), '
|
||||
'default privileges, and ownership.'
|
||||
),
|
||||
queries=['database_settings', 'schema_acls', 'table_acls',
|
||||
'default_privileges'],
|
||||
scope=['database']
|
||||
),
|
||||
Section(
|
||||
id='rls_policies',
|
||||
name='Row-Level Security',
|
||||
description=(
|
||||
'Row-level security policies, RLS-enabled tables, '
|
||||
'and policy coverage analysis.'
|
||||
),
|
||||
queries=['rls_enabled_tables', 'rls_policies'],
|
||||
scope=['database', 'schema']
|
||||
),
|
||||
Section(
|
||||
id='security_definer',
|
||||
name='Security Definer Functions',
|
||||
description=(
|
||||
'Functions running with elevated privileges (SECURITY DEFINER), '
|
||||
'their ownership, and permissions.'
|
||||
),
|
||||
queries=['security_definer_functions'],
|
||||
scope=['database', 'schema']
|
||||
),
|
||||
Section(
|
||||
id='audit_logging',
|
||||
name='Audit & Logging',
|
||||
description=(
|
||||
'Connection logging, statement logging, error logging, '
|
||||
'and audit trail configuration.'
|
||||
),
|
||||
queries=['logging_settings'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='extensions',
|
||||
name='Extensions',
|
||||
description=(
|
||||
'Installed extensions and their security implications.'
|
||||
),
|
||||
queries=['extensions'],
|
||||
scope=['server', 'database']
|
||||
),
|
||||
]
|
||||
|
||||
# =============================================================================
|
||||
# PERFORMANCE REPORT SECTIONS
|
||||
# =============================================================================
|
||||
|
||||
PERFORMANCE_SECTIONS = [
|
||||
Section(
|
||||
id='memory_config',
|
||||
name='Memory Configuration',
|
||||
description=(
|
||||
'shared_buffers, work_mem, effective_cache_size, '
|
||||
'maintenance_work_mem, and other memory settings.'
|
||||
),
|
||||
queries=['memory_settings'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='checkpoint_wal',
|
||||
name='Checkpoint & WAL',
|
||||
description=(
|
||||
'Checkpoint settings, WAL configuration, background writer stats, '
|
||||
'and write-ahead log tuning.'
|
||||
),
|
||||
queries=['checkpoint_settings', 'wal_settings', 'bgwriter_stats'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='autovacuum',
|
||||
name='Autovacuum Configuration',
|
||||
description=(
|
||||
'Autovacuum settings, tables needing vacuum, '
|
||||
'dead tuple accumulation, and maintenance status.'
|
||||
),
|
||||
queries=['autovacuum_settings', 'tables_needing_vacuum'],
|
||||
scope=['server', 'database']
|
||||
),
|
||||
Section(
|
||||
id='query_planner',
|
||||
name='Query Planner Settings',
|
||||
description=(
|
||||
'Cost parameters, statistics targets, JIT compilation, '
|
||||
'and planner optimization settings.'
|
||||
),
|
||||
queries=['planner_settings'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='parallelism',
|
||||
name='Parallelism & Workers',
|
||||
description=(
|
||||
'Parallel query configuration, worker processes, '
|
||||
'and parallel maintenance settings.'
|
||||
),
|
||||
queries=['parallel_settings'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='connection_pooling',
|
||||
name='Connection Management',
|
||||
description=(
|
||||
'Max connections, reserved connections, timeouts, '
|
||||
'and current connection status.'
|
||||
),
|
||||
queries=['connection_settings', 'active_connections'],
|
||||
scope=['server']
|
||||
),
|
||||
Section(
|
||||
id='cache_efficiency',
|
||||
name='Cache Efficiency',
|
||||
description=(
|
||||
'Buffer cache hit ratios, database-level cache stats, '
|
||||
'and table-level I/O patterns.'
|
||||
),
|
||||
queries=['database_stats', 'table_cache_stats'],
|
||||
scope=['server', 'database']
|
||||
),
|
||||
Section(
|
||||
id='index_usage',
|
||||
name='Index Analysis',
|
||||
description=(
|
||||
'Index utilization, unused indexes, tables needing indexes, '
|
||||
'and index size analysis.'
|
||||
),
|
||||
queries=['table_stats', 'unused_indexes', 'tables_needing_indexes',
|
||||
'table_sizes'],
|
||||
scope=['database']
|
||||
),
|
||||
Section(
|
||||
id='slow_queries',
|
||||
name='Query Performance',
|
||||
description=(
|
||||
'Slowest queries, most frequent queries, '
|
||||
'and query execution statistics (requires pg_stat_statements).'
|
||||
),
|
||||
queries=['stat_statements_check', 'top_queries_by_time',
|
||||
'top_queries_by_calls'],
|
||||
scope=['server', 'database']
|
||||
),
|
||||
Section(
|
||||
id='replication',
|
||||
name='Replication Status',
|
||||
description=(
|
||||
'Replication lag, standby status, and WAL sender statistics.'
|
||||
),
|
||||
queries=['replication_status'],
|
||||
scope=['server']
|
||||
),
|
||||
]
|
||||
|
||||
# =============================================================================
|
||||
# DESIGN REPORT SECTIONS
|
||||
# =============================================================================
|
||||
|
||||
DESIGN_SECTIONS = [
|
||||
Section(
|
||||
id='table_structure',
|
||||
name='Table Structure',
|
||||
description=(
|
||||
'Table definitions, column counts, sizes, ownership, '
|
||||
'and documentation coverage.'
|
||||
),
|
||||
queries=['tables_overview', 'columns_info'],
|
||||
scope=['database', 'schema']
|
||||
),
|
||||
Section(
|
||||
id='primary_keys',
|
||||
name='Primary Key Analysis',
|
||||
description=(
|
||||
'Primary key design, tables without primary keys, '
|
||||
'and key column choices.'
|
||||
),
|
||||
queries=['primary_keys', 'tables_without_pk'],
|
||||
scope=['database', 'schema']
|
||||
),
|
||||
Section(
|
||||
id='foreign_keys',
|
||||
name='Referential Integrity',
|
||||
description=(
|
||||
'Foreign key relationships, orphan references, '
|
||||
'and relationship coverage.'
|
||||
),
|
||||
queries=['foreign_keys'],
|
||||
scope=['database', 'schema']
|
||||
),
|
||||
Section(
|
||||
id='indexes',
|
||||
name='Index Strategy',
|
||||
description=(
|
||||
'Index definitions, duplicate indexes, index types, '
|
||||
'and coverage analysis.'
|
||||
),
|
||||
queries=['indexes_info', 'duplicate_indexes'],
|
||||
scope=['database', 'schema']
|
||||
),
|
||||
Section(
|
||||
id='constraints',
|
||||
name='Constraints',
|
||||
description=(
|
||||
'Check constraints, unique constraints, '
|
||||
'and data validation coverage.'
|
||||
),
|
||||
queries=['check_constraints', 'unique_constraints'],
|
||||
scope=['database', 'schema']
|
||||
),
|
||||
Section(
|
||||
id='normalization',
|
||||
name='Normalization Analysis',
|
||||
description=(
|
||||
'Repeated column patterns, potential denormalization issues, '
|
||||
'and data redundancy.'
|
||||
),
|
||||
queries=['repeated_column_names'],
|
||||
scope=['database']
|
||||
),
|
||||
Section(
|
||||
id='naming_conventions',
|
||||
name='Naming Conventions',
|
||||
description=(
|
||||
'Table and column naming patterns, consistency analysis, '
|
||||
'and naming standard compliance.'
|
||||
),
|
||||
queries=['object_names'],
|
||||
scope=['database', 'schema']
|
||||
),
|
||||
Section(
|
||||
id='data_types',
|
||||
name='Data Type Review',
|
||||
description=(
|
||||
'Data type usage patterns, type consistency, '
|
||||
'and type appropriateness.'
|
||||
),
|
||||
queries=['column_types'],
|
||||
scope=['database']
|
||||
),
|
||||
]
|
||||
|
||||
# =============================================================================
|
||||
# SECTION LOOKUPS
|
||||
# =============================================================================
|
||||
|
||||
# Convert lists to dictionaries for quick lookup
|
||||
SECURITY_SECTIONS_DICT = {s.id: s for s in SECURITY_SECTIONS}
|
||||
PERFORMANCE_SECTIONS_DICT = {s.id: s for s in PERFORMANCE_SECTIONS}
|
||||
DESIGN_SECTIONS_DICT = {s.id: s for s in DESIGN_SECTIONS}
|
||||
|
||||
# Combined lookup by report type
|
||||
SECTIONS_BY_TYPE = {
|
||||
'security': SECURITY_SECTIONS,
|
||||
'performance': PERFORMANCE_SECTIONS,
|
||||
'design': DESIGN_SECTIONS,
|
||||
}
|
||||
|
||||
SECTIONS_DICT_BY_TYPE = {
|
||||
'security': SECURITY_SECTIONS_DICT,
|
||||
'performance': PERFORMANCE_SECTIONS_DICT,
|
||||
'design': DESIGN_SECTIONS_DICT,
|
||||
}
|
||||
|
||||
|
||||
def get_sections_for_report(report_type: str) -> list[Section]:
|
||||
"""Get all sections for a report type.
|
||||
|
||||
Args:
|
||||
report_type: One of 'security', 'performance', 'design'.
|
||||
|
||||
Returns:
|
||||
List of Section objects.
|
||||
|
||||
Raises:
|
||||
ValueError: If report_type is invalid.
|
||||
"""
|
||||
sections = SECTIONS_BY_TYPE.get(report_type)
|
||||
if sections is None:
|
||||
raise ValueError(f"Invalid report type: {report_type}")
|
||||
return sections
|
||||
|
||||
|
||||
def get_sections_for_scope(
|
||||
report_type: str,
|
||||
scope: str
|
||||
) -> list[Section]:
|
||||
"""Get sections applicable to a specific scope.
|
||||
|
||||
Args:
|
||||
report_type: One of 'security', 'performance', 'design'.
|
||||
scope: One of 'server', 'database', 'schema'.
|
||||
|
||||
Returns:
|
||||
List of Section objects applicable to the scope.
|
||||
"""
|
||||
all_sections = get_sections_for_report(report_type)
|
||||
return [s for s in all_sections if scope in s.scope]
|
||||
|
||||
|
||||
def get_section(report_type: str, section_id: str) -> Section:
|
||||
"""Get a specific section by ID.
|
||||
|
||||
Args:
|
||||
report_type: One of 'security', 'performance', 'design'.
|
||||
section_id: The section identifier.
|
||||
|
||||
Returns:
|
||||
Section object.
|
||||
|
||||
Raises:
|
||||
ValueError: If section not found.
|
||||
"""
|
||||
sections_dict = SECTIONS_DICT_BY_TYPE.get(report_type, {})
|
||||
section = sections_dict.get(section_id)
|
||||
if section is None:
|
||||
raise ValueError(
|
||||
f"Section '{section_id}' not found in {report_type} report"
|
||||
)
|
||||
return section
|
||||
767
web/pgadmin/llm/static/js/AIReport.jsx
Normal file
767
web/pgadmin/llm/static/js/AIReport.jsx
Normal file
@@ -0,0 +1,767 @@
|
||||
/////////////////////////////////////////////////////////////
|
||||
//
|
||||
// pgAdmin 4 - PostgreSQL Tools
|
||||
//
|
||||
// Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
// This software is released under the PostgreSQL Licence
|
||||
//
|
||||
//////////////////////////////////////////////////////////////
|
||||
|
||||
import { useState, useEffect, useRef, useCallback } from 'react';
|
||||
import { Box, Paper, Typography, LinearProgress } from '@mui/material';
|
||||
import { styled } from '@mui/material/styles';
|
||||
import DownloadIcon from '@mui/icons-material/Download';
|
||||
import RefreshIcon from '@mui/icons-material/Refresh';
|
||||
import StopIcon from '@mui/icons-material/Stop';
|
||||
import InfoOutlinedIcon from '@mui/icons-material/InfoOutlined';
|
||||
import PropTypes from 'prop-types';
|
||||
import { marked } from 'marked';
|
||||
import DOMPurify from 'dompurify';
|
||||
|
||||
import gettext from 'sources/gettext';
|
||||
import url_for from 'sources/url_for';
|
||||
import getApiInstance from '../../../static/js/api_instance';
|
||||
import Loader from '../../../static/js/components/Loader';
|
||||
import { PrimaryButton, DefaultButton } from '../../../static/js/components/Buttons';
|
||||
import { usePgAdmin } from '../../../static/js/PgAdminProvider';
|
||||
|
||||
// Helper to get the internal key for desktop mode authentication
|
||||
// The key is passed as a URL parameter when pgAdmin launches in desktop mode
|
||||
function getInternalKey() {
|
||||
// Try to get from current URL's query params
|
||||
const urlParams = new URLSearchParams(window.location.search);
|
||||
const key = urlParams.get('key');
|
||||
if (key) return key;
|
||||
|
||||
// Try to get from cookie (if not HTTPOnly)
|
||||
const cookieValue = `; ${document.cookie}`;
|
||||
const parts = cookieValue.split('; PGADMIN_INT_KEY=');
|
||||
if (parts.length === 2) return parts.pop().split(';').shift();
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// Configure marked for security and rendering
|
||||
marked.setOptions({
|
||||
gfm: true, // GitHub Flavored Markdown
|
||||
breaks: true, // Convert \n to <br>
|
||||
});
|
||||
|
||||
|
||||
const StyledBox = styled(Box)(({ theme }) => ({
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
height: '100%',
|
||||
background: theme.palette.grey[400],
|
||||
'& .AIReport-header': {
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
justifyContent: 'flex-end',
|
||||
padding: theme.spacing(1, 2),
|
||||
borderBottom: `1px solid ${theme.palette.divider}`,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
},
|
||||
'& .AIReport-actions': {
|
||||
display: 'flex',
|
||||
gap: theme.spacing(1),
|
||||
},
|
||||
'& .AIReport-content': {
|
||||
flex: 1,
|
||||
overflow: 'auto',
|
||||
padding: theme.spacing(3),
|
||||
position: 'relative',
|
||||
display: 'flex',
|
||||
justifyContent: 'center',
|
||||
},
|
||||
'& .AIReport-paper': {
|
||||
width: '100%',
|
||||
maxWidth: '900px',
|
||||
minHeight: 'fit-content',
|
||||
},
|
||||
'& .AIReport-markdown': {
|
||||
...theme.mixins.panelBorder.all,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
color: theme.palette.text.primary,
|
||||
fontFamily: theme.typography.fontFamily,
|
||||
fontSize: '0.9rem',
|
||||
lineHeight: 1.6,
|
||||
padding: theme.spacing(4),
|
||||
boxShadow: theme.shadows[2],
|
||||
userSelect: 'text',
|
||||
cursor: 'text',
|
||||
// Ensure all elements inherit the text color for dark mode support
|
||||
'& *': {
|
||||
color: 'inherit',
|
||||
},
|
||||
'& a': {
|
||||
color: theme.palette.primary.main,
|
||||
},
|
||||
'& h1': {
|
||||
fontSize: '1.5rem',
|
||||
fontWeight: 600,
|
||||
marginTop: theme.spacing(2),
|
||||
marginBottom: theme.spacing(1),
|
||||
borderBottom: `1px solid ${theme.palette.divider}`,
|
||||
paddingBottom: theme.spacing(0.5),
|
||||
color: theme.palette.text.primary,
|
||||
},
|
||||
'& h1:first-of-type': {
|
||||
marginTop: 0,
|
||||
},
|
||||
'& h2': {
|
||||
fontSize: '1.25rem',
|
||||
fontWeight: 600,
|
||||
marginTop: theme.spacing(2),
|
||||
marginBottom: theme.spacing(1),
|
||||
color: theme.palette.text.primary,
|
||||
},
|
||||
'& h3': {
|
||||
fontSize: '1.1rem',
|
||||
fontWeight: 600,
|
||||
marginTop: theme.spacing(1.5),
|
||||
marginBottom: theme.spacing(0.5),
|
||||
color: theme.palette.text.primary,
|
||||
},
|
||||
'& p': {
|
||||
marginTop: 0,
|
||||
marginBottom: theme.spacing(1.5),
|
||||
color: theme.palette.text.primary,
|
||||
},
|
||||
'& ul, & ol': {
|
||||
marginTop: 0,
|
||||
marginBottom: theme.spacing(1.5),
|
||||
paddingLeft: theme.spacing(3),
|
||||
color: theme.palette.text.primary,
|
||||
},
|
||||
'& ul ul, & ol ol, & ul ol, & ol ul': {
|
||||
marginBottom: 0,
|
||||
},
|
||||
'& li': {
|
||||
marginBottom: theme.spacing(0.5),
|
||||
color: theme.palette.text.primary,
|
||||
'& > p': {
|
||||
marginBottom: theme.spacing(0.5),
|
||||
},
|
||||
},
|
||||
'& li > ul, & li > ol': {
|
||||
marginTop: theme.spacing(0.5),
|
||||
},
|
||||
// Task list checkboxes (GitHub style)
|
||||
'& input[type="checkbox"]': {
|
||||
marginRight: theme.spacing(0.5),
|
||||
},
|
||||
'& code': {
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
padding: '2px 6px',
|
||||
borderRadius: '3px',
|
||||
fontFamily: 'monospace',
|
||||
fontSize: '0.85em',
|
||||
},
|
||||
'& pre': {
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
padding: theme.spacing(1.5),
|
||||
borderRadius: '4px',
|
||||
overflow: 'auto',
|
||||
'& code': {
|
||||
backgroundColor: 'transparent',
|
||||
padding: 0,
|
||||
},
|
||||
},
|
||||
'& blockquote': {
|
||||
borderLeft: `4px solid ${theme.palette.primary.main}`,
|
||||
margin: theme.spacing(1.5, 0),
|
||||
padding: theme.spacing(1, 2),
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
'& p:last-child': {
|
||||
marginBottom: 0,
|
||||
},
|
||||
},
|
||||
'& table': {
|
||||
borderCollapse: 'collapse',
|
||||
width: '100%',
|
||||
marginBottom: theme.spacing(1.5),
|
||||
display: 'block',
|
||||
overflowX: 'auto',
|
||||
},
|
||||
'& thead': {
|
||||
display: 'table',
|
||||
width: '100%',
|
||||
tableLayout: 'fixed',
|
||||
},
|
||||
'& tbody': {
|
||||
display: 'table',
|
||||
width: '100%',
|
||||
tableLayout: 'fixed',
|
||||
},
|
||||
'& tr': {
|
||||
borderBottom: `1px solid ${theme.palette.divider}`,
|
||||
},
|
||||
'& th, & td': {
|
||||
border: `1px solid ${theme.palette.divider}`,
|
||||
padding: theme.spacing(1, 1.5),
|
||||
textAlign: 'left',
|
||||
verticalAlign: 'top',
|
||||
color: theme.palette.text.primary,
|
||||
},
|
||||
'& th': {
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
fontWeight: 600,
|
||||
color: theme.palette.text.primary,
|
||||
},
|
||||
'& tbody tr:hover': {
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
},
|
||||
'& hr': {
|
||||
border: 'none',
|
||||
borderTop: `1px solid ${theme.palette.divider}`,
|
||||
margin: theme.spacing(2, 0),
|
||||
},
|
||||
'& strong': {
|
||||
fontWeight: 600,
|
||||
},
|
||||
'& em': {
|
||||
fontStyle: 'italic',
|
||||
},
|
||||
},
|
||||
'& .AIReport-error': {
|
||||
...theme.mixins.panelBorder.all,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
color: theme.palette.error.main,
|
||||
padding: theme.spacing(4),
|
||||
textAlign: 'center',
|
||||
width: '100%',
|
||||
maxWidth: '900px',
|
||||
boxShadow: theme.shadows[2],
|
||||
userSelect: 'text',
|
||||
cursor: 'text',
|
||||
},
|
||||
'& .AIReport-placeholder': {
|
||||
...theme.mixins.panelBorder.all,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
color: theme.palette.text.secondary,
|
||||
padding: theme.spacing(4),
|
||||
textAlign: 'center',
|
||||
width: '100%',
|
||||
maxWidth: '900px',
|
||||
boxShadow: theme.shadows[2],
|
||||
},
|
||||
'& .AIReport-progress': {
|
||||
...theme.mixins.panelBorder.all,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
padding: theme.spacing(4),
|
||||
width: '100%',
|
||||
maxWidth: '900px',
|
||||
boxShadow: theme.shadows[2],
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
alignItems: 'center',
|
||||
gap: theme.spacing(2),
|
||||
},
|
||||
'& .AIReport-progress-bar': {
|
||||
width: '100%',
|
||||
maxWidth: '400px',
|
||||
},
|
||||
}));
|
||||
|
||||
// Report category configurations
|
||||
const REPORT_CONFIGS = {
|
||||
security: {
|
||||
endpoints: {
|
||||
server: 'llm.security_report',
|
||||
database: 'llm.database_security_report',
|
||||
schema: 'llm.schema_security_report',
|
||||
},
|
||||
streamEndpoints: {
|
||||
server: 'llm.security_report_stream',
|
||||
database: 'llm.database_security_report_stream',
|
||||
schema: 'llm.schema_security_report_stream',
|
||||
},
|
||||
titles: {
|
||||
server: () => gettext('Server Security Report'),
|
||||
database: () => gettext('Database Security Report'),
|
||||
schema: () => gettext('Schema Security Report'),
|
||||
},
|
||||
loadingMessage: () => gettext('Generating security report'),
|
||||
filePrefix: 'security-report',
|
||||
},
|
||||
performance: {
|
||||
endpoints: {
|
||||
server: 'llm.performance_report',
|
||||
database: 'llm.database_performance_report',
|
||||
},
|
||||
streamEndpoints: {
|
||||
server: 'llm.performance_report_stream',
|
||||
database: 'llm.database_performance_report_stream',
|
||||
},
|
||||
titles: {
|
||||
server: () => gettext('Server Performance Report'),
|
||||
database: () => gettext('Database Performance Report'),
|
||||
},
|
||||
loadingMessage: () => gettext('Generating performance report'),
|
||||
filePrefix: 'performance-report',
|
||||
},
|
||||
design: {
|
||||
endpoints: {
|
||||
database: 'llm.database_design_report',
|
||||
schema: 'llm.schema_design_report',
|
||||
},
|
||||
streamEndpoints: {
|
||||
database: 'llm.database_design_report_stream',
|
||||
schema: 'llm.schema_design_report_stream',
|
||||
},
|
||||
titles: {
|
||||
database: () => gettext('Database Design Review'),
|
||||
schema: () => gettext('Schema Design Review'),
|
||||
},
|
||||
loadingMessage: () => gettext('Generating design review'),
|
||||
filePrefix: 'design-review',
|
||||
},
|
||||
};
|
||||
|
||||
// Stage display names
|
||||
const STAGE_NAMES = {
|
||||
planning: () => gettext('Planning Analysis'),
|
||||
gathering: () => gettext('Gathering Data'),
|
||||
analyzing: () => gettext('Analyzing Sections'),
|
||||
synthesizing: () => gettext('Creating Report'),
|
||||
};
|
||||
|
||||
|
||||
export default function AIReport({
|
||||
sid, did, scid, reportCategory = 'security', reportType = 'server',
|
||||
serverName, databaseName, schemaName,
|
||||
onClose: _onClose
|
||||
}) {
|
||||
const [report, setReport] = useState('');
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
const [progress, setProgress] = useState(null);
|
||||
const [stopped, setStopped] = useState(false);
|
||||
const pgAdmin = usePgAdmin();
|
||||
const eventSourceRef = useRef(null);
|
||||
const stoppedRef = useRef(false);
|
||||
|
||||
// Get text colors from the body element to match pgAdmin's theme
|
||||
// The MUI theme may not be synced with pgAdmin's theme in docker tabs
|
||||
const [textColors, setTextColors] = useState({
|
||||
primary: 'inherit',
|
||||
secondary: 'inherit',
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
const updateColors = () => {
|
||||
const bodyStyles = window.getComputedStyle(document.body);
|
||||
const primaryColor = bodyStyles.color;
|
||||
|
||||
// For secondary color, create a semi-transparent version of the primary
|
||||
// by parsing the RGB values and adding opacity
|
||||
const rgbMatch = primaryColor.match(/rgb\((\d+),\s*(\d+),\s*(\d+)\)/);
|
||||
let secondaryColor = primaryColor;
|
||||
if (rgbMatch) {
|
||||
const [, r, g, b] = rgbMatch;
|
||||
secondaryColor = `rgba(${r}, ${g}, ${b}, 0.7)`;
|
||||
}
|
||||
|
||||
setTextColors({
|
||||
primary: primaryColor,
|
||||
secondary: secondaryColor,
|
||||
});
|
||||
};
|
||||
|
||||
updateColors();
|
||||
|
||||
// Check periodically in case theme changes
|
||||
const interval = setInterval(updateColors, 1000);
|
||||
return () => clearInterval(interval);
|
||||
}, []);
|
||||
|
||||
const api = getApiInstance();
|
||||
const config = REPORT_CONFIGS[reportCategory];
|
||||
|
||||
// Build the API URL based on report category and type
|
||||
const getReportUrl = useCallback((useStream = false) => {
|
||||
const endpoints = useStream ? config.streamEndpoints : config.endpoints;
|
||||
const endpoint = endpoints?.[reportType];
|
||||
if (!endpoint) {
|
||||
console.error(`No endpoint for ${reportCategory}/${reportType}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
if (reportType === 'schema') {
|
||||
return url_for(endpoint, { sid, did, scid });
|
||||
} else if (reportType === 'database') {
|
||||
return url_for(endpoint, { sid, did });
|
||||
} else {
|
||||
return url_for(endpoint, { sid });
|
||||
}
|
||||
}, [config, reportType, reportCategory, sid, did, scid]);
|
||||
|
||||
// Close any existing EventSource connection
|
||||
const closeEventSource = useCallback(() => {
|
||||
if (eventSourceRef.current) {
|
||||
eventSourceRef.current.close();
|
||||
eventSourceRef.current = null;
|
||||
}
|
||||
}, []);
|
||||
|
||||
// Stop the current report generation
|
||||
const stopReport = useCallback(() => {
|
||||
stoppedRef.current = true;
|
||||
closeEventSource();
|
||||
setLoading(false);
|
||||
setProgress(null);
|
||||
setStopped(true);
|
||||
setError(null);
|
||||
}, [closeEventSource]);
|
||||
|
||||
// Fallback to non-streaming API call
|
||||
const generateReportFallback = useCallback(() => {
|
||||
const url = getReportUrl(false);
|
||||
if (!url) {
|
||||
setError(gettext('Invalid report configuration.'));
|
||||
return;
|
||||
}
|
||||
|
||||
stoppedRef.current = false;
|
||||
setStopped(false);
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
setReport('');
|
||||
setProgress(null);
|
||||
|
||||
api.get(url)
|
||||
.then((res) => {
|
||||
if (res.data && res.data.success) {
|
||||
setReport(res.data.data?.report || '');
|
||||
} else {
|
||||
setError(res.data?.errormsg || gettext('Failed to generate report.'));
|
||||
}
|
||||
})
|
||||
.catch((err) => {
|
||||
let errMsg = gettext('Failed to generate report.');
|
||||
if (err.response?.data?.errormsg) {
|
||||
errMsg = err.response.data.errormsg;
|
||||
} else if (err.message) {
|
||||
errMsg = err.message;
|
||||
}
|
||||
setError(errMsg);
|
||||
pgAdmin.Browser.notifier.error(errMsg);
|
||||
})
|
||||
.finally(() => {
|
||||
setLoading(false);
|
||||
});
|
||||
}, [getReportUrl, api, pgAdmin]);
|
||||
|
||||
// Generate report using SSE streaming
|
||||
const generateReportStream = useCallback(() => {
|
||||
let url = getReportUrl(true);
|
||||
if (!url) {
|
||||
setError(gettext('Invalid report configuration.'));
|
||||
return;
|
||||
}
|
||||
|
||||
// In desktop mode, add the internal key to the URL for authentication
|
||||
const internalKey = getInternalKey();
|
||||
if (internalKey) {
|
||||
const separator = url.includes('?') ? '&' : '?';
|
||||
url = `${url}${separator}key=${encodeURIComponent(internalKey)}`;
|
||||
}
|
||||
|
||||
closeEventSource();
|
||||
stoppedRef.current = false;
|
||||
setStopped(false);
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
setReport('');
|
||||
setProgress({ stage: 'planning', message: gettext('Starting...') });
|
||||
|
||||
const eventSource = new EventSource(url, { withCredentials: true });
|
||||
eventSourceRef.current = eventSource;
|
||||
|
||||
eventSource.onmessage = (event) => {
|
||||
try {
|
||||
const data = JSON.parse(event.data);
|
||||
|
||||
if (data.type === 'stage') {
|
||||
setProgress({
|
||||
stage: data.stage,
|
||||
message: data.message,
|
||||
completed: 0,
|
||||
total: 0,
|
||||
});
|
||||
} else if (data.type === 'progress') {
|
||||
setProgress((prev) => ({
|
||||
...prev,
|
||||
stage: data.stage,
|
||||
message: data.message,
|
||||
section: data.section,
|
||||
completed: data.completed || 0,
|
||||
total: data.total || 0,
|
||||
}));
|
||||
} else if (data.type === 'retry') {
|
||||
setProgress((prev) => ({
|
||||
...prev,
|
||||
message: data.message,
|
||||
retrying: true,
|
||||
}));
|
||||
} else if (data.type === 'complete') {
|
||||
setReport(data.report || '');
|
||||
setLoading(false);
|
||||
setProgress(null);
|
||||
closeEventSource();
|
||||
} else if (data.type === 'error') {
|
||||
setError(data.message || gettext('Failed to generate report.'));
|
||||
setLoading(false);
|
||||
setProgress(null);
|
||||
closeEventSource();
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('Error parsing SSE event:', e);
|
||||
}
|
||||
};
|
||||
|
||||
// Track error count to detect persistent failures (like 401)
|
||||
let errorCount = 0;
|
||||
|
||||
eventSource.onerror = () => {
|
||||
errorCount++;
|
||||
|
||||
// If we get multiple errors quickly (like 401 retries), fall back immediately
|
||||
if (errorCount >= 2) {
|
||||
console.warn('SSE connection failed repeatedly, falling back to non-streaming');
|
||||
closeEventSource();
|
||||
generateReportFallback();
|
||||
return;
|
||||
}
|
||||
|
||||
// If the connection is closed, fall back
|
||||
if (eventSource.readyState === EventSource.CLOSED) {
|
||||
closeEventSource();
|
||||
generateReportFallback();
|
||||
}
|
||||
};
|
||||
}, [getReportUrl, closeEventSource, generateReportFallback]);
|
||||
|
||||
// Main generate function - tries streaming first
|
||||
const generateReport = useCallback(() => {
|
||||
// Check if streaming endpoints are available
|
||||
const streamUrl = getReportUrl(true);
|
||||
if (streamUrl) {
|
||||
generateReportStream();
|
||||
} else {
|
||||
generateReportFallback();
|
||||
}
|
||||
}, [getReportUrl, generateReportStream, generateReportFallback]);
|
||||
|
||||
useEffect(() => {
|
||||
// Generate report on mount
|
||||
generateReport();
|
||||
|
||||
// Cleanup on unmount
|
||||
return () => {
|
||||
closeEventSource();
|
||||
};
|
||||
}, [sid, did, scid, reportCategory, reportType]);
|
||||
|
||||
// Build markdown header for the report
|
||||
const getReportHeader = () => {
|
||||
const titleFn = config.titles[reportType];
|
||||
let title = titleFn ? titleFn() : gettext('Report');
|
||||
let subtitle;
|
||||
|
||||
if (reportType === 'schema') {
|
||||
title += ': ' + schemaName;
|
||||
subtitle = `${schemaName} ${gettext('in')} ${databaseName} ${gettext('on')} ${serverName}`;
|
||||
} else if (reportType === 'database') {
|
||||
title += ': ' + databaseName;
|
||||
subtitle = `${databaseName} ${gettext('on')} ${serverName}`;
|
||||
} else {
|
||||
title += ': ' + serverName;
|
||||
subtitle = serverName;
|
||||
}
|
||||
|
||||
const date = new Date().toLocaleDateString(undefined, {
|
||||
year: 'numeric',
|
||||
month: 'long',
|
||||
day: 'numeric'
|
||||
});
|
||||
|
||||
return `# ${title}\n\n*${subtitle} • ${date}*\n\n---\n\n`;
|
||||
};
|
||||
|
||||
// Build filename for download based on report type
|
||||
const getDownloadFilename = () => {
|
||||
const date = new Date().toISOString().slice(0, 10);
|
||||
const sanitize = (str) => str ? str.replace(/[^a-z0-9]/gi, '_') : '';
|
||||
const prefix = config.filePrefix;
|
||||
|
||||
if (reportType === 'schema') {
|
||||
return `${prefix}-${sanitize(schemaName)}-${sanitize(databaseName)}-${sanitize(serverName)}-${date}.md`;
|
||||
} else if (reportType === 'database') {
|
||||
return `${prefix}-${sanitize(databaseName)}-${sanitize(serverName)}-${date}.md`;
|
||||
} else {
|
||||
return `${prefix}-${sanitize(serverName)}-${date}.md`;
|
||||
}
|
||||
};
|
||||
|
||||
const handleDownload = () => {
|
||||
if (!report) return;
|
||||
|
||||
const blob = new Blob([getReportHeader() + report], { type: 'text/markdown' });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = getDownloadFilename();
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
URL.revokeObjectURL(url);
|
||||
};
|
||||
|
||||
const reportHtml = report
|
||||
? DOMPurify.sanitize(marked.parse(getReportHeader() + report))
|
||||
: '';
|
||||
|
||||
return (
|
||||
<StyledBox>
|
||||
<Box className="AIReport-header">
|
||||
<Box className="AIReport-actions">
|
||||
<DefaultButton
|
||||
onClick={stopReport}
|
||||
disabled={!loading}
|
||||
startIcon={<StopIcon />}
|
||||
>
|
||||
{gettext('Stop')}
|
||||
</DefaultButton>
|
||||
<DefaultButton
|
||||
onClick={generateReport}
|
||||
disabled={loading}
|
||||
startIcon={<RefreshIcon />}
|
||||
>
|
||||
{gettext('Regenerate')}
|
||||
</DefaultButton>
|
||||
<PrimaryButton
|
||||
onClick={handleDownload}
|
||||
disabled={!report || loading}
|
||||
startIcon={<DownloadIcon />}
|
||||
>
|
||||
{gettext('Download')}
|
||||
</PrimaryButton>
|
||||
</Box>
|
||||
</Box>
|
||||
|
||||
<Box className="AIReport-content">
|
||||
{/* Progress display during streaming */}
|
||||
{loading && progress && (
|
||||
<Box className="AIReport-progress">
|
||||
<Typography
|
||||
style={{
|
||||
fontSize: '1rem',
|
||||
fontWeight: 600,
|
||||
marginBottom: '8px',
|
||||
color: textColors.primary,
|
||||
}}>
|
||||
{STAGE_NAMES[progress.stage]?.() || progress.stage}
|
||||
</Typography>
|
||||
<Typography
|
||||
style={{
|
||||
fontSize: '0.875rem',
|
||||
color: textColors.secondary,
|
||||
}}>
|
||||
{progress.message}
|
||||
</Typography>
|
||||
{progress.total > 0 && (
|
||||
<Box className="AIReport-progress-bar">
|
||||
<LinearProgress
|
||||
variant="determinate"
|
||||
value={(progress.completed / progress.total) * 100}
|
||||
/>
|
||||
<Typography
|
||||
variant="caption"
|
||||
style={{
|
||||
marginTop: '4px',
|
||||
display: 'block',
|
||||
textAlign: 'center',
|
||||
color: textColors.secondary
|
||||
}}>
|
||||
{progress.completed} / {progress.total}
|
||||
</Typography>
|
||||
</Box>
|
||||
)}
|
||||
{!progress.total && (
|
||||
<Box className="AIReport-progress-bar">
|
||||
<LinearProgress variant="indeterminate" />
|
||||
</Box>
|
||||
)}
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{/* Fallback loader when not using streaming */}
|
||||
{loading && !progress && (
|
||||
<Loader message={config.loadingMessage()} autoEllipsis />
|
||||
)}
|
||||
|
||||
{error && !loading && (
|
||||
<Box className="AIReport-error">
|
||||
<Typography variant="body1">{error}</Typography>
|
||||
<DefaultButton onClick={generateReport} style={{ marginTop: '16px' }}>
|
||||
{gettext('Retry')}
|
||||
</DefaultButton>
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{stopped && !loading && !error && (
|
||||
<Box
|
||||
className="AIReport-placeholder"
|
||||
sx={{ display: 'flex', flexDirection: 'column', alignItems: 'center' }}
|
||||
>
|
||||
<InfoOutlinedIcon sx={{ fontSize: 48, color: 'text.secondary', mb: 2 }} />
|
||||
<Typography variant="body1">
|
||||
{gettext('Report generation was cancelled.')}
|
||||
</Typography>
|
||||
<Typography variant="body2" color="textSecondary" sx={{ mt: 1 }}>
|
||||
{gettext('Click Regenerate to start a new report.')}
|
||||
</Typography>
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{!report && !loading && !error && !stopped && (
|
||||
<Box className="AIReport-placeholder">
|
||||
<Typography variant="body1">
|
||||
{gettext('Generating report...')}
|
||||
</Typography>
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{report && !loading && (
|
||||
<Box className="AIReport-paper">
|
||||
<Paper
|
||||
elevation={0}
|
||||
className="AIReport-markdown"
|
||||
sx={(theme) => ({
|
||||
color: `${theme.palette.text.primary} !important`,
|
||||
'& *': {
|
||||
color: 'inherit !important'
|
||||
}
|
||||
})}
|
||||
>
|
||||
<div dangerouslySetInnerHTML={{ __html: reportHtml }} />
|
||||
</Paper>
|
||||
</Box>
|
||||
)}
|
||||
</Box>
|
||||
</StyledBox>
|
||||
);
|
||||
}
|
||||
|
||||
AIReport.propTypes = {
|
||||
sid: PropTypes.oneOfType([PropTypes.string, PropTypes.number]).isRequired,
|
||||
did: PropTypes.oneOfType([PropTypes.string, PropTypes.number]),
|
||||
scid: PropTypes.oneOfType([PropTypes.string, PropTypes.number]),
|
||||
reportCategory: PropTypes.oneOf(['security', 'performance', 'design']),
|
||||
reportType: PropTypes.oneOf(['server', 'database', 'schema']),
|
||||
serverName: PropTypes.string.isRequired,
|
||||
databaseName: PropTypes.string,
|
||||
schemaName: PropTypes.string,
|
||||
onClose: PropTypes.func,
|
||||
};
|
||||
386
web/pgadmin/llm/static/js/SecurityReport.jsx
Normal file
386
web/pgadmin/llm/static/js/SecurityReport.jsx
Normal file
@@ -0,0 +1,386 @@
|
||||
/////////////////////////////////////////////////////////////
|
||||
//
|
||||
// pgAdmin 4 - PostgreSQL Tools
|
||||
//
|
||||
// Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
// This software is released under the PostgreSQL Licence
|
||||
//
|
||||
//////////////////////////////////////////////////////////////
|
||||
|
||||
import { useState, useEffect } from 'react';
|
||||
import { Box, Paper, Typography } from '@mui/material';
|
||||
import { styled } from '@mui/material/styles';
|
||||
import DownloadIcon from '@mui/icons-material/Download';
|
||||
import RefreshIcon from '@mui/icons-material/Refresh';
|
||||
import PropTypes from 'prop-types';
|
||||
import { marked } from 'marked';
|
||||
import DOMPurify from 'dompurify';
|
||||
|
||||
import gettext from 'sources/gettext';
|
||||
import url_for from 'sources/url_for';
|
||||
import getApiInstance from '../../../static/js/api_instance';
|
||||
import Loader from '../../../static/js/components/Loader';
|
||||
import { PrimaryButton, DefaultButton } from '../../../static/js/components/Buttons';
|
||||
import { usePgAdmin } from '../../../static/js/PgAdminProvider';
|
||||
|
||||
// Configure marked for security and rendering
|
||||
marked.setOptions({
|
||||
gfm: true, // GitHub Flavored Markdown
|
||||
breaks: true, // Convert \n to <br>
|
||||
});
|
||||
|
||||
|
||||
const StyledBox = styled(Box)(({ theme }) => ({
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
height: '100%',
|
||||
background: theme.palette.grey[400],
|
||||
'& .SecurityReport-header': {
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
justifyContent: 'flex-end',
|
||||
padding: theme.spacing(1, 2),
|
||||
borderBottom: `1px solid ${theme.palette.divider}`,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
},
|
||||
'& .SecurityReport-actions': {
|
||||
display: 'flex',
|
||||
gap: theme.spacing(1),
|
||||
},
|
||||
'& .SecurityReport-content': {
|
||||
flex: 1,
|
||||
overflow: 'auto',
|
||||
padding: theme.spacing(3),
|
||||
position: 'relative',
|
||||
display: 'flex',
|
||||
justifyContent: 'center',
|
||||
},
|
||||
'& .SecurityReport-paper': {
|
||||
width: '100%',
|
||||
maxWidth: '900px',
|
||||
minHeight: 'fit-content',
|
||||
},
|
||||
'& .SecurityReport-markdown': {
|
||||
...theme.mixins.panelBorder.all,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
fontFamily: theme.typography.fontFamily,
|
||||
fontSize: '0.9rem',
|
||||
lineHeight: 1.6,
|
||||
padding: theme.spacing(4),
|
||||
boxShadow: theme.shadows[2],
|
||||
userSelect: 'text',
|
||||
cursor: 'text',
|
||||
'& h1': {
|
||||
fontSize: '1.5rem',
|
||||
fontWeight: 600,
|
||||
marginTop: theme.spacing(2),
|
||||
marginBottom: theme.spacing(1),
|
||||
borderBottom: `1px solid ${theme.palette.divider}`,
|
||||
paddingBottom: theme.spacing(0.5),
|
||||
},
|
||||
'& h2': {
|
||||
fontSize: '1.25rem',
|
||||
fontWeight: 600,
|
||||
marginTop: theme.spacing(2),
|
||||
marginBottom: theme.spacing(1),
|
||||
},
|
||||
'& h3': {
|
||||
fontSize: '1.1rem',
|
||||
fontWeight: 600,
|
||||
marginTop: theme.spacing(1.5),
|
||||
marginBottom: theme.spacing(0.5),
|
||||
},
|
||||
'& p': {
|
||||
marginTop: 0,
|
||||
marginBottom: theme.spacing(1.5),
|
||||
},
|
||||
'& ul, & ol': {
|
||||
marginTop: 0,
|
||||
marginBottom: theme.spacing(1.5),
|
||||
paddingLeft: theme.spacing(3),
|
||||
},
|
||||
'& ul ul, & ol ol, & ul ol, & ol ul': {
|
||||
marginBottom: 0,
|
||||
},
|
||||
'& li': {
|
||||
marginBottom: theme.spacing(0.5),
|
||||
'& > p': {
|
||||
marginBottom: theme.spacing(0.5),
|
||||
},
|
||||
},
|
||||
'& li > ul, & li > ol': {
|
||||
marginTop: theme.spacing(0.5),
|
||||
},
|
||||
// Task list checkboxes (GitHub style)
|
||||
'& input[type="checkbox"]': {
|
||||
marginRight: theme.spacing(0.5),
|
||||
},
|
||||
'& code': {
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
padding: '2px 6px',
|
||||
borderRadius: '3px',
|
||||
fontFamily: 'monospace',
|
||||
fontSize: '0.85em',
|
||||
},
|
||||
'& pre': {
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
padding: theme.spacing(1.5),
|
||||
borderRadius: '4px',
|
||||
overflow: 'auto',
|
||||
'& code': {
|
||||
backgroundColor: 'transparent',
|
||||
padding: 0,
|
||||
},
|
||||
},
|
||||
'& blockquote': {
|
||||
borderLeft: `4px solid ${theme.palette.primary.main}`,
|
||||
margin: theme.spacing(1.5, 0),
|
||||
padding: theme.spacing(1, 2),
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
'& p:last-child': {
|
||||
marginBottom: 0,
|
||||
},
|
||||
},
|
||||
'& table': {
|
||||
borderCollapse: 'collapse',
|
||||
width: '100%',
|
||||
marginBottom: theme.spacing(1.5),
|
||||
display: 'block',
|
||||
overflowX: 'auto',
|
||||
},
|
||||
'& thead': {
|
||||
display: 'table',
|
||||
width: '100%',
|
||||
tableLayout: 'fixed',
|
||||
},
|
||||
'& tbody': {
|
||||
display: 'table',
|
||||
width: '100%',
|
||||
tableLayout: 'fixed',
|
||||
},
|
||||
'& tr': {
|
||||
borderBottom: `1px solid ${theme.palette.divider}`,
|
||||
},
|
||||
'& th, & td': {
|
||||
border: `1px solid ${theme.palette.divider}`,
|
||||
padding: theme.spacing(1, 1.5),
|
||||
textAlign: 'left',
|
||||
verticalAlign: 'top',
|
||||
},
|
||||
'& th': {
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
fontWeight: 600,
|
||||
},
|
||||
'& tbody tr:hover': {
|
||||
backgroundColor: theme.palette.action.hover,
|
||||
},
|
||||
'& hr': {
|
||||
border: 'none',
|
||||
borderTop: `1px solid ${theme.palette.divider}`,
|
||||
margin: theme.spacing(2, 0),
|
||||
},
|
||||
'& strong': {
|
||||
fontWeight: 600,
|
||||
},
|
||||
'& em': {
|
||||
fontStyle: 'italic',
|
||||
},
|
||||
},
|
||||
'& .SecurityReport-error': {
|
||||
...theme.mixins.panelBorder.all,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
color: theme.palette.error.main,
|
||||
padding: theme.spacing(4),
|
||||
textAlign: 'center',
|
||||
width: '100%',
|
||||
maxWidth: '900px',
|
||||
boxShadow: theme.shadows[2],
|
||||
},
|
||||
'& .SecurityReport-placeholder': {
|
||||
...theme.mixins.panelBorder.all,
|
||||
backgroundColor: theme.palette.background.default,
|
||||
color: theme.palette.text.secondary,
|
||||
padding: theme.spacing(4),
|
||||
textAlign: 'center',
|
||||
width: '100%',
|
||||
maxWidth: '900px',
|
||||
boxShadow: theme.shadows[2],
|
||||
},
|
||||
}));
|
||||
|
||||
|
||||
export default function SecurityReport({
|
||||
sid, did, scid, reportType = 'server',
|
||||
serverName, databaseName, schemaName,
|
||||
onClose: _onClose
|
||||
}) {
|
||||
const [report, setReport] = useState('');
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
const pgAdmin = usePgAdmin();
|
||||
|
||||
const api = getApiInstance();
|
||||
|
||||
// Build the API URL based on report type
|
||||
const getReportUrl = () => {
|
||||
if (reportType === 'schema') {
|
||||
return url_for('llm.schema_security_report', { sid, did, scid });
|
||||
} else if (reportType === 'database') {
|
||||
return url_for('llm.database_security_report', { sid, did });
|
||||
} else {
|
||||
return url_for('llm.security_report', { sid });
|
||||
}
|
||||
};
|
||||
|
||||
const generateReport = () => {
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
setReport('');
|
||||
|
||||
api.get(getReportUrl())
|
||||
.then((res) => {
|
||||
if (res.data && res.data.success) {
|
||||
setReport(res.data.data?.report || '');
|
||||
} else {
|
||||
setError(res.data?.errormsg || gettext('Failed to generate security report.'));
|
||||
}
|
||||
})
|
||||
.catch((err) => {
|
||||
let errMsg = gettext('Failed to generate security report.');
|
||||
if (err.response?.data?.errormsg) {
|
||||
errMsg = err.response.data.errormsg;
|
||||
} else if (err.message) {
|
||||
errMsg = err.message;
|
||||
}
|
||||
setError(errMsg);
|
||||
pgAdmin.Browser.notifier.error(errMsg);
|
||||
})
|
||||
.finally(() => {
|
||||
setLoading(false);
|
||||
});
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
// Generate report on mount
|
||||
generateReport();
|
||||
}, [sid, did, scid, reportType]);
|
||||
|
||||
// Build markdown header for the report
|
||||
const getReportHeader = () => {
|
||||
let title, subtitle;
|
||||
|
||||
if (reportType === 'schema') {
|
||||
title = gettext('Schema Security Report') + ': ' + schemaName;
|
||||
subtitle = `${schemaName} ${gettext('in')} ${databaseName} ${gettext('on')} ${serverName}`;
|
||||
} else if (reportType === 'database') {
|
||||
title = gettext('Database Security Report') + ': ' + databaseName;
|
||||
subtitle = `${databaseName} ${gettext('on')} ${serverName}`;
|
||||
} else {
|
||||
title = gettext('Server Security Report') + ': ' + serverName;
|
||||
subtitle = serverName;
|
||||
}
|
||||
|
||||
const date = new Date().toLocaleDateString(undefined, {
|
||||
year: 'numeric',
|
||||
month: 'long',
|
||||
day: 'numeric'
|
||||
});
|
||||
|
||||
return `# ${title}\n\n*${subtitle} • ${date}*\n\n---\n\n`;
|
||||
};
|
||||
|
||||
// Build filename for download based on report type
|
||||
const getDownloadFilename = () => {
|
||||
const date = new Date().toISOString().slice(0, 10);
|
||||
const sanitize = (str) => str ? str.replace(/[^a-z0-9]/gi, '_') : '';
|
||||
|
||||
if (reportType === 'schema') {
|
||||
return `security-report-${sanitize(schemaName)}-${sanitize(databaseName)}-${sanitize(serverName)}-${date}.md`;
|
||||
} else if (reportType === 'database') {
|
||||
return `security-report-${sanitize(databaseName)}-${sanitize(serverName)}-${date}.md`;
|
||||
} else {
|
||||
return `security-report-${sanitize(serverName)}-${date}.md`;
|
||||
}
|
||||
};
|
||||
|
||||
const handleDownload = () => {
|
||||
if (!report) return;
|
||||
|
||||
const blob = new Blob([getReportHeader() + report], { type: 'text/markdown' });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = getDownloadFilename();
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
URL.revokeObjectURL(url);
|
||||
};
|
||||
|
||||
const reportHtml = report
|
||||
? DOMPurify.sanitize(marked.parse(getReportHeader() + report))
|
||||
: '';
|
||||
|
||||
return (
|
||||
<StyledBox>
|
||||
<Box className="SecurityReport-header">
|
||||
<Box className="SecurityReport-actions">
|
||||
<DefaultButton
|
||||
onClick={generateReport}
|
||||
disabled={loading}
|
||||
startIcon={<RefreshIcon />}
|
||||
>
|
||||
{gettext('Regenerate')}
|
||||
</DefaultButton>
|
||||
<PrimaryButton
|
||||
onClick={handleDownload}
|
||||
disabled={!report || loading}
|
||||
startIcon={<DownloadIcon />}
|
||||
>
|
||||
{gettext('Download')}
|
||||
</PrimaryButton>
|
||||
</Box>
|
||||
</Box>
|
||||
|
||||
<Box className="SecurityReport-content">
|
||||
<Loader message={loading ? gettext('Generating security report') : null} autoEllipsis />
|
||||
|
||||
{error && !loading && (
|
||||
<Box className="SecurityReport-error">
|
||||
<Typography variant="body1">{error}</Typography>
|
||||
<DefaultButton onClick={generateReport} style={{ marginTop: '16px' }}>
|
||||
{gettext('Retry')}
|
||||
</DefaultButton>
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{!report && !loading && !error && (
|
||||
<Box className="SecurityReport-placeholder">
|
||||
<Typography variant="body1">
|
||||
{gettext('Click "Generate" to create a security report for this server.')}
|
||||
</Typography>
|
||||
</Box>
|
||||
)}
|
||||
|
||||
{report && !loading && (
|
||||
<Box className="SecurityReport-paper">
|
||||
<Paper elevation={0} className="SecurityReport-markdown">
|
||||
<div dangerouslySetInnerHTML={{ __html: reportHtml }} />
|
||||
</Paper>
|
||||
</Box>
|
||||
)}
|
||||
</Box>
|
||||
</StyledBox>
|
||||
);
|
||||
}
|
||||
|
||||
SecurityReport.propTypes = {
|
||||
sid: PropTypes.oneOfType([PropTypes.string, PropTypes.number]).isRequired,
|
||||
did: PropTypes.oneOfType([PropTypes.string, PropTypes.number]),
|
||||
scid: PropTypes.oneOfType([PropTypes.string, PropTypes.number]),
|
||||
reportType: PropTypes.oneOf(['server', 'database', 'schema']),
|
||||
serverName: PropTypes.string.isRequired,
|
||||
databaseName: PropTypes.string,
|
||||
schemaName: PropTypes.string,
|
||||
onClose: PropTypes.func,
|
||||
};
|
||||
492
web/pgadmin/llm/static/js/ai_tools.js
Normal file
492
web/pgadmin/llm/static/js/ai_tools.js
Normal file
@@ -0,0 +1,492 @@
|
||||
/////////////////////////////////////////////////////////////
|
||||
//
|
||||
// pgAdmin 4 - PostgreSQL Tools
|
||||
//
|
||||
// Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
// This software is released under the PostgreSQL Licence
|
||||
//
|
||||
//////////////////////////////////////////////////////////////
|
||||
|
||||
import AIReport from './AIReport';
|
||||
import { AllPermissionTypes, BROWSER_PANELS } from '../../../browser/static/js/constants';
|
||||
import getApiInstance from '../../../static/js/api_instance';
|
||||
import url_for from 'sources/url_for';
|
||||
|
||||
// AI Reports Module
|
||||
define([
|
||||
'sources/gettext', 'pgadmin.browser',
|
||||
], function(
|
||||
gettext, pgBrowser
|
||||
) {
|
||||
|
||||
// if module is already initialized, refer to that.
|
||||
if (pgBrowser.AITools) {
|
||||
return pgBrowser.AITools;
|
||||
}
|
||||
|
||||
// Create an Object AITools of pgBrowser class
|
||||
pgBrowser.AITools = {
|
||||
llmEnabled: false,
|
||||
llmSystemEnabled: false,
|
||||
llmStatusChecked: false,
|
||||
|
||||
init: function() {
|
||||
if (this.initialized)
|
||||
return;
|
||||
|
||||
this.initialized = true;
|
||||
|
||||
// Check LLM status
|
||||
this.checkLLMStatus();
|
||||
|
||||
// Register AI Reports menu category
|
||||
pgBrowser.add_menu_category({
|
||||
name: 'ai_tools',
|
||||
label: gettext('AI Reports'),
|
||||
priority: 100,
|
||||
});
|
||||
|
||||
// Define the menus
|
||||
let menus = [];
|
||||
|
||||
// =====================================================================
|
||||
// Security Reports - Server, Database, Schema
|
||||
// =====================================================================
|
||||
menus.push({
|
||||
name: 'ai_security_report',
|
||||
module: this,
|
||||
applies: ['tools'],
|
||||
callback: 'show_security_report',
|
||||
category: 'ai_tools',
|
||||
priority: 1,
|
||||
label: gettext('Security'),
|
||||
icon: 'fa fa-shield-alt',
|
||||
enable: this.security_report_enabled.bind(this),
|
||||
data: {
|
||||
data_disabled: gettext('Please select a server, database, or schema.'),
|
||||
},
|
||||
permission: AllPermissionTypes.TOOLS_AI,
|
||||
});
|
||||
|
||||
// Context menus for security reports
|
||||
for (let node_val of ['server', 'database', 'schema']) {
|
||||
menus.push({
|
||||
name: 'ai_security_report_context_' + node_val,
|
||||
node: node_val,
|
||||
module: this,
|
||||
applies: ['context'],
|
||||
callback: 'show_security_report',
|
||||
category: 'ai_tools',
|
||||
priority: 100,
|
||||
label: gettext('Security'),
|
||||
icon: 'fa fa-shield-alt',
|
||||
enable: this.security_report_enabled.bind(this),
|
||||
permission: AllPermissionTypes.TOOLS_AI,
|
||||
});
|
||||
}
|
||||
|
||||
// =====================================================================
|
||||
// Performance Reports - Server, Database
|
||||
// =====================================================================
|
||||
menus.push({
|
||||
name: 'ai_performance_report',
|
||||
module: this,
|
||||
applies: ['tools'],
|
||||
callback: 'show_performance_report',
|
||||
category: 'ai_tools',
|
||||
priority: 2,
|
||||
label: gettext('Performance'),
|
||||
icon: 'fa fa-tachometer-alt',
|
||||
enable: this.performance_report_enabled.bind(this),
|
||||
data: {
|
||||
data_disabled: gettext('Please select a server or database.'),
|
||||
},
|
||||
permission: AllPermissionTypes.TOOLS_AI,
|
||||
});
|
||||
|
||||
// Context menus for performance reports (server and database only)
|
||||
for (let node_val of ['server', 'database']) {
|
||||
menus.push({
|
||||
name: 'ai_performance_report_context_' + node_val,
|
||||
node: node_val,
|
||||
module: this,
|
||||
applies: ['context'],
|
||||
callback: 'show_performance_report',
|
||||
category: 'ai_tools',
|
||||
priority: 101,
|
||||
label: gettext('Performance'),
|
||||
icon: 'fa fa-tachometer-alt',
|
||||
enable: this.performance_report_enabled.bind(this),
|
||||
permission: AllPermissionTypes.TOOLS_AI,
|
||||
});
|
||||
}
|
||||
|
||||
// =====================================================================
|
||||
// Design Review Reports - Database, Schema
|
||||
// =====================================================================
|
||||
menus.push({
|
||||
name: 'ai_design_report',
|
||||
module: this,
|
||||
applies: ['tools'],
|
||||
callback: 'show_design_report',
|
||||
category: 'ai_tools',
|
||||
priority: 3,
|
||||
label: gettext('Design'),
|
||||
icon: 'fa fa-drafting-compass',
|
||||
enable: this.design_report_enabled.bind(this),
|
||||
data: {
|
||||
data_disabled: gettext('Please select a database or schema.'),
|
||||
},
|
||||
permission: AllPermissionTypes.TOOLS_AI,
|
||||
});
|
||||
|
||||
// Context menus for design review (database and schema only)
|
||||
for (let node_val of ['database', 'schema']) {
|
||||
menus.push({
|
||||
name: 'ai_design_report_context_' + node_val,
|
||||
node: node_val,
|
||||
module: this,
|
||||
applies: ['context'],
|
||||
callback: 'show_design_report',
|
||||
category: 'ai_tools',
|
||||
priority: 102,
|
||||
label: gettext('Design'),
|
||||
icon: 'fa fa-drafting-compass',
|
||||
enable: this.design_report_enabled.bind(this),
|
||||
permission: AllPermissionTypes.TOOLS_AI,
|
||||
});
|
||||
}
|
||||
|
||||
pgBrowser.add_menus(menus);
|
||||
|
||||
return this;
|
||||
},
|
||||
|
||||
// Check if LLM is configured
|
||||
checkLLMStatus: function() {
|
||||
const api = getApiInstance();
|
||||
api.get(url_for('llm.status'))
|
||||
.then((res) => {
|
||||
if (res?.data?.success) {
|
||||
this.llmEnabled = res.data.data?.enabled || false;
|
||||
this.llmSystemEnabled = res.data.data?.system_enabled || false;
|
||||
}
|
||||
this.llmStatusChecked = true;
|
||||
})
|
||||
.catch(() => {
|
||||
this.llmEnabled = false;
|
||||
this.llmSystemEnabled = false;
|
||||
this.llmStatusChecked = true;
|
||||
});
|
||||
},
|
||||
|
||||
// Get the node type from tree item
|
||||
getNodeType: function(item) {
|
||||
let tree = pgBrowser.tree;
|
||||
let nodeData = tree.itemData(item);
|
||||
|
||||
if (!nodeData) return null;
|
||||
return nodeData._type;
|
||||
},
|
||||
|
||||
// Common LLM enablement check
|
||||
checkLLMEnabled: function(data) {
|
||||
if (!this.llmStatusChecked) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext(
|
||||
'Checking AI configuration...'
|
||||
);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!this.llmSystemEnabled) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext(
|
||||
'AI features are disabled in the '
|
||||
+ 'server configuration.'
|
||||
);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!this.llmEnabled) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext(
|
||||
'Please configure an LLM provider in '
|
||||
+ 'Preferences > AI to enable this feature.'
|
||||
);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
},
|
||||
|
||||
// =====================================================================
|
||||
// Security Report Functions
|
||||
// =====================================================================
|
||||
|
||||
security_report_enabled: function(node, item, data) {
|
||||
if (!this.checkLLMEnabled(data)) return false;
|
||||
|
||||
if (!node || !item) return false;
|
||||
|
||||
let tree = pgBrowser.tree;
|
||||
let info = tree.getTreeNodeHierarchy(item);
|
||||
|
||||
if (!info || !info.server) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please select a server, database, or schema.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!info.server.connected) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please connect to the server first.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
let nodeType = this.getNodeType(item);
|
||||
if (!['server', 'database', 'schema'].includes(nodeType)) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please select a server, database, or schema.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (nodeType === 'database' || nodeType === 'schema') {
|
||||
if (!info.database || !info.database.connected) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please connect to the database first.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
},
|
||||
|
||||
show_security_report: function() {
|
||||
this._showReport('security', ['server', 'database', 'schema']);
|
||||
},
|
||||
|
||||
// =====================================================================
|
||||
// Performance Report Functions
|
||||
// =====================================================================
|
||||
|
||||
performance_report_enabled: function(node, item, data) {
|
||||
if (!this.checkLLMEnabled(data)) return false;
|
||||
|
||||
if (!node || !item) return false;
|
||||
|
||||
let tree = pgBrowser.tree;
|
||||
let info = tree.getTreeNodeHierarchy(item);
|
||||
|
||||
if (!info || !info.server) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please select a server or database.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!info.server.connected) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please connect to the server first.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
let nodeType = this.getNodeType(item);
|
||||
if (!['server', 'database'].includes(nodeType)) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please select a server or database.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (nodeType === 'database') {
|
||||
if (!info.database || !info.database.connected) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please connect to the database first.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
},
|
||||
|
||||
show_performance_report: function() {
|
||||
this._showReport('performance', ['server', 'database']);
|
||||
},
|
||||
|
||||
// =====================================================================
|
||||
// Design Review Functions
|
||||
// =====================================================================
|
||||
|
||||
design_report_enabled: function(node, item, data) {
|
||||
if (!this.checkLLMEnabled(data)) return false;
|
||||
|
||||
if (!node || !item) return false;
|
||||
|
||||
let tree = pgBrowser.tree;
|
||||
let info = tree.getTreeNodeHierarchy(item);
|
||||
|
||||
if (!info || !info.server) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please select a database or schema.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!info.server.connected) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please connect to the server first.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
let nodeType = this.getNodeType(item);
|
||||
if (!['database', 'schema'].includes(nodeType)) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please select a database or schema.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!info.database || !info.database.connected) {
|
||||
if (data) {
|
||||
data.data_disabled = gettext('Please connect to the database first.');
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
},
|
||||
|
||||
show_design_report: function() {
|
||||
this._showReport('design', ['database', 'schema']);
|
||||
},
|
||||
|
||||
// =====================================================================
|
||||
// Common Report Display Function
|
||||
// =====================================================================
|
||||
|
||||
_showReport: function(reportCategory, validNodeTypes) {
|
||||
let t = pgBrowser.tree,
|
||||
i = t.selected(),
|
||||
info = pgBrowser.tree.getTreeNodeHierarchy(i);
|
||||
|
||||
if (!info || !info.server) {
|
||||
pgBrowser.report_error(
|
||||
gettext('Report'),
|
||||
gettext('Please select a valid node.')
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let nodeType = this.getNodeType(i);
|
||||
if (!validNodeTypes.includes(nodeType)) {
|
||||
pgBrowser.report_error(
|
||||
gettext('Report'),
|
||||
gettext('Please select a valid node for this report type.')
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let sid = info.server._id;
|
||||
let did = info.database ? info.database._id : null;
|
||||
let scid = info.schema ? info.schema._id : null;
|
||||
|
||||
// Determine report type based on node
|
||||
let reportType = nodeType;
|
||||
|
||||
// Build panel title and ID with timestamp for uniqueness
|
||||
let panelTitle = this._buildPanelTitle(reportCategory, reportType, info);
|
||||
let panelIdSuffix = this._buildPanelIdSuffix(reportCategory, reportType, sid, did, scid);
|
||||
const timestamp = Date.now();
|
||||
const panelId = `${BROWSER_PANELS.AI_REPORT_PREFIX}-${panelIdSuffix}-${timestamp}`;
|
||||
|
||||
// Get docker handler and open as tab in main panel area
|
||||
let handler = pgBrowser.getDockerHandler?.(
|
||||
BROWSER_PANELS.AI_REPORT_PREFIX,
|
||||
pgBrowser.docker.default_workspace
|
||||
);
|
||||
if (!handler) {
|
||||
pgBrowser.report_error(
|
||||
gettext('Report'),
|
||||
gettext('Unable to open the report panel.')
|
||||
);
|
||||
return;
|
||||
}
|
||||
handler.focus();
|
||||
handler.docker.openTab({
|
||||
id: panelId,
|
||||
title: panelTitle,
|
||||
icon: 'fa fa-file-alt',
|
||||
content: (
|
||||
<AIReport
|
||||
sid={sid}
|
||||
did={did}
|
||||
scid={scid}
|
||||
reportCategory={reportCategory}
|
||||
reportType={reportType}
|
||||
serverName={info.server.label}
|
||||
databaseName={info.database?.label}
|
||||
schemaName={info.schema?.label}
|
||||
onClose={() => { handler.docker.close(panelId); }}
|
||||
/>
|
||||
),
|
||||
closable: true,
|
||||
cache: false,
|
||||
group: 'playground'
|
||||
}, BROWSER_PANELS.MAIN, 'middle', true);
|
||||
},
|
||||
|
||||
_buildPanelTitle: function(reportCategory, reportType, info) {
|
||||
let categoryLabel;
|
||||
switch (reportCategory) {
|
||||
case 'security':
|
||||
categoryLabel = gettext('Security Report');
|
||||
break;
|
||||
case 'performance':
|
||||
categoryLabel = gettext('Performance Report');
|
||||
break;
|
||||
case 'design':
|
||||
categoryLabel = gettext('Design Review');
|
||||
break;
|
||||
default:
|
||||
categoryLabel = gettext('Report');
|
||||
}
|
||||
|
||||
if (reportType === 'server') {
|
||||
return info.server.label + ' ' + categoryLabel;
|
||||
} else if (reportType === 'database') {
|
||||
return info.database.label + ' ' + gettext('on') + ' ' +
|
||||
info.server.label + ' ' + categoryLabel;
|
||||
} else if (reportType === 'schema') {
|
||||
return info.schema.label + ' ' + gettext('in') + ' ' +
|
||||
info.database.label + ' ' + gettext('on') + ' ' +
|
||||
info.server.label + ' ' + categoryLabel;
|
||||
}
|
||||
return categoryLabel;
|
||||
},
|
||||
|
||||
_buildPanelIdSuffix: function(reportCategory, reportType, sid, did, scid) {
|
||||
let base = `${reportCategory}_${reportType}`;
|
||||
if (reportType === 'server') {
|
||||
return `${base}_${sid}`;
|
||||
} else if (reportType === 'database') {
|
||||
return `${base}_${sid}_${did}`;
|
||||
} else if (reportType === 'schema') {
|
||||
return `${base}_${sid}_${did}_${scid}`;
|
||||
}
|
||||
return base;
|
||||
},
|
||||
};
|
||||
|
||||
return pgBrowser.AITools;
|
||||
});
|
||||
187
web/pgadmin/llm/tests/README.md
Normal file
187
web/pgadmin/llm/tests/README.md
Normal file
@@ -0,0 +1,187 @@
|
||||
# LLM Module Tests
|
||||
|
||||
This directory contains comprehensive tests for the pgAdmin LLM/AI functionality.
|
||||
|
||||
## Test Files
|
||||
|
||||
### Python Tests
|
||||
|
||||
#### `test_client.py` - LLM Client Tests
|
||||
Tests the core LLM client functionality including:
|
||||
- Provider initialization (Anthropic, OpenAI, Ollama)
|
||||
- API key loading from files and environment variables
|
||||
- Graceful handling of missing API keys
|
||||
- User preference overrides
|
||||
- Provider selection logic
|
||||
- Whitespace handling in API keys
|
||||
|
||||
**Key Features:**
|
||||
- Tests pass even without API keys configured
|
||||
- Mocks external API calls
|
||||
- Tests all three provider types
|
||||
|
||||
#### `test_reports.py` - Report Generation Tests
|
||||
Tests report generation functionality including:
|
||||
- Security, performance, and design report types
|
||||
- Server, database, and schema level reports
|
||||
- Report request validation
|
||||
- Progress callback functionality
|
||||
- Error handling during generation
|
||||
- Markdown formatting
|
||||
|
||||
**Key Features:**
|
||||
- Tests data collection from PostgreSQL
|
||||
- Validates report structure
|
||||
- Tests streaming progress updates
|
||||
|
||||
#### `test_chat.py` - Chat Session Tests
|
||||
Tests interactive chat functionality including:
|
||||
- Chat session initialization
|
||||
- Message history management
|
||||
- Context passing (database, SQL queries)
|
||||
- Streaming responses
|
||||
- Token counting for context management
|
||||
- Maximum history limits
|
||||
- Error handling
|
||||
|
||||
**Key Features:**
|
||||
- Tests conversation flow
|
||||
- Validates context integration
|
||||
- Tests memory management
|
||||
|
||||
#### `test_views.py` - API Endpoint Tests
|
||||
Tests Flask endpoints including:
|
||||
- `/llm/status` - LLM availability check
|
||||
- `/llm/reports/security/*` - Security report endpoints
|
||||
- `/llm/reports/performance/*` - Performance report endpoints
|
||||
- `/llm/reports/design/*` - Design review endpoints
|
||||
- `/llm/chat` - Chat endpoint
|
||||
- Streaming endpoints with SSE
|
||||
|
||||
**Key Features:**
|
||||
- Tests authentication and permissions
|
||||
- Tests API error responses
|
||||
- Tests SSE streaming format
|
||||
|
||||
### JavaScript Tests
|
||||
|
||||
#### `AIReport.spec.js` - AIReport Component Tests
|
||||
Tests the React component for AI report display including:
|
||||
- Component rendering in light and dark modes
|
||||
- Theme detection from body styles
|
||||
- Progress display during generation
|
||||
- Error handling
|
||||
- Markdown rendering
|
||||
- Download functionality
|
||||
- SSE event handling
|
||||
- Support for all report categories and types
|
||||
|
||||
**Key Features:**
|
||||
- Tests with React Testing Library
|
||||
- Mocks EventSource for SSE
|
||||
- Tests theme transitions
|
||||
- Validates accessibility
|
||||
|
||||
## Running the Tests
|
||||
|
||||
### Python Tests
|
||||
|
||||
From the `web` directory:
|
||||
|
||||
```bash
|
||||
# Run all LLM tests
|
||||
python -m pytest pgadmin/llm/tests/
|
||||
|
||||
# Run specific test file
|
||||
python -m pytest pgadmin/llm/tests/test_client.py
|
||||
|
||||
# Run specific test case
|
||||
python -m pytest pgadmin/llm/tests/test_client.py::LLMClientTestCase::test_anthropic_provider_with_api_key
|
||||
|
||||
# Run with coverage
|
||||
python -m pytest --cov=pgadmin/llm pgadmin/llm/tests/
|
||||
```
|
||||
|
||||
### JavaScript Tests
|
||||
|
||||
From the `web` directory:
|
||||
|
||||
```bash
|
||||
# Run all JavaScript tests
|
||||
yarn run test:karma
|
||||
|
||||
# Run specific test file
|
||||
yarn run test:karma -- --file regression/javascript/llm/AIReport.spec.js
|
||||
```
|
||||
|
||||
## Test Coverage
|
||||
|
||||
### What's Tested
|
||||
|
||||
✅ LLM client initialization with all providers
|
||||
✅ API key loading from files and environment
|
||||
✅ Graceful handling of missing API keys
|
||||
✅ User preference overrides
|
||||
✅ Report generation for all categories (security, performance, design)
|
||||
✅ Report generation for all levels (server, database, schema)
|
||||
✅ Chat session management and history
|
||||
✅ Streaming progress updates via SSE
|
||||
✅ API endpoint authentication and authorization
|
||||
✅ React component rendering in both themes
|
||||
✅ Dark mode text color detection
|
||||
✅ Error handling throughout the stack
|
||||
|
||||
### What's Mocked
|
||||
|
||||
- External LLM API calls (Anthropic, OpenAI, Ollama)
|
||||
- PostgreSQL database connections
|
||||
- File system access for API keys
|
||||
- EventSource for SSE streaming
|
||||
- Theme detection (window.getComputedStyle)
|
||||
|
||||
## Environment Variables for Testing
|
||||
|
||||
These environment variables can be set for integration testing with real APIs:
|
||||
|
||||
```bash
|
||||
# For Anthropic
|
||||
export ANTHROPIC_API_KEY="your-api-key"
|
||||
|
||||
# For OpenAI
|
||||
export OPENAI_API_KEY="your-api-key"
|
||||
|
||||
# For Ollama
|
||||
export OLLAMA_API_URL="http://localhost:11434"
|
||||
```
|
||||
|
||||
**Note:** Tests are designed to pass without these variables set. They will mock API responses when keys are not available.
|
||||
|
||||
## Test Philosophy
|
||||
|
||||
1. **Graceful Degradation**: All tests pass even without API keys configured
|
||||
2. **Mocking by Default**: External APIs are mocked to avoid dependencies
|
||||
3. **Comprehensive Coverage**: Tests cover happy paths, error cases, and edge cases
|
||||
4. **Documentation**: Tests serve as documentation for expected behavior
|
||||
5. **Integration Ready**: Tests can be run with real APIs when keys are provided
|
||||
|
||||
## Adding New Tests
|
||||
|
||||
When adding new functionality to the LLM module:
|
||||
|
||||
1. Add unit tests to the appropriate test file
|
||||
2. Mock external dependencies
|
||||
3. Test both success and failure cases
|
||||
4. Test with and without API keys/configuration
|
||||
5. Update this README with new test coverage
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Import errors**: Make sure you're running tests from the `web` directory
|
||||
|
||||
**API key warnings**: These are expected - tests should pass without API keys
|
||||
|
||||
**Theme mocking errors**: Ensure `fake_theme.js` is available in regression/javascript/
|
||||
|
||||
**EventSource not found**: This is mocked in JavaScript tests, ensure mocks are properly set up
|
||||
8
web/pgadmin/llm/tests/__init__.py
Normal file
8
web/pgadmin/llm/tests/__init__.py
Normal file
@@ -0,0 +1,8 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
82
web/pgadmin/llm/tests/test_llm_status.py
Normal file
82
web/pgadmin/llm/tests/test_llm_status.py
Normal file
@@ -0,0 +1,82 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
import json
|
||||
from unittest.mock import patch, MagicMock, mock_open
|
||||
from pgadmin.utils.route import BaseTestGenerator
|
||||
from regression.python_test_utils import test_utils as utils
|
||||
|
||||
|
||||
class LLMStatusTestCase(BaseTestGenerator):
|
||||
"""Test cases for LLM status endpoint"""
|
||||
|
||||
scenarios = [
|
||||
('LLM Status - Disabled', dict(
|
||||
url='/llm/status',
|
||||
provider_enabled=False,
|
||||
expected_enabled=False
|
||||
)),
|
||||
('LLM Status - Anthropic Enabled', dict(
|
||||
url='/llm/status',
|
||||
provider_enabled=True,
|
||||
expected_enabled=True,
|
||||
provider_name='anthropic'
|
||||
)),
|
||||
('LLM Status - OpenAI Enabled', dict(
|
||||
url='/llm/status',
|
||||
provider_enabled=True,
|
||||
expected_enabled=True,
|
||||
provider_name='openai'
|
||||
)),
|
||||
('LLM Status - Ollama Enabled', dict(
|
||||
url='/llm/status',
|
||||
provider_enabled=True,
|
||||
expected_enabled=True,
|
||||
provider_name='ollama'
|
||||
)),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
# Required by BaseTestGenerator but no setup needed for these tests
|
||||
pass
|
||||
|
||||
def runTest(self):
|
||||
"""Test LLM status endpoint returns correct availability status"""
|
||||
provider_value = self.provider_name if (
|
||||
self.provider_enabled and hasattr(self, 'provider_name')
|
||||
) else None
|
||||
|
||||
with patch(
|
||||
'pgadmin.llm.utils.is_llm_enabled'
|
||||
) as mock_enabled, patch(
|
||||
'pgadmin.llm.utils.is_llm_enabled_system'
|
||||
) as mock_system, patch(
|
||||
'pgadmin.llm.utils.get_default_provider'
|
||||
) as mock_provider, patch(
|
||||
'pgadmin.authenticate.mfa.utils.mfa_required',
|
||||
lambda f: f
|
||||
):
|
||||
|
||||
mock_enabled.return_value = self.expected_enabled
|
||||
mock_system.return_value = self.provider_enabled
|
||||
mock_provider.return_value = provider_value
|
||||
|
||||
response = self.tester.get(
|
||||
self.url,
|
||||
content_type='application/json',
|
||||
follow_redirects=True
|
||||
)
|
||||
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertTrue(data['success'])
|
||||
self.assertEqual(data['data']['enabled'], self.expected_enabled)
|
||||
|
||||
if self.expected_enabled and hasattr(self, 'provider_name'):
|
||||
self.assertEqual(data['data']['provider'], self.provider_name)
|
||||
249
web/pgadmin/llm/tests/test_report_endpoints.py
Normal file
249
web/pgadmin/llm/tests/test_report_endpoints.py
Normal file
@@ -0,0 +1,249 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
import json
|
||||
from unittest.mock import patch, MagicMock
|
||||
from pgadmin.utils.route import BaseTestGenerator
|
||||
from regression.python_test_utils import test_utils as utils
|
||||
|
||||
|
||||
class SecurityReportServerTestCase(BaseTestGenerator):
|
||||
"""Test cases for security report generation at server level"""
|
||||
|
||||
scenarios = [
|
||||
('Security Report - LLM Disabled', dict(
|
||||
llm_enabled=False
|
||||
)),
|
||||
('Security Report - LLM Enabled', dict(
|
||||
llm_enabled=True
|
||||
)),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
self.server_id = 1
|
||||
|
||||
def runTest(self):
|
||||
"""Test security report endpoint at server level"""
|
||||
with patch('pgadmin.llm.utils.is_llm_enabled') as mock_enabled, \
|
||||
patch('pgadmin.llm.reports.generator.generate_report_sync'
|
||||
) as mock_generate, \
|
||||
patch('pgadmin.utils.driver.get_driver') as mock_get_driver:
|
||||
|
||||
# Mock database connection
|
||||
mock_conn = MagicMock()
|
||||
mock_conn.connected.return_value = True
|
||||
|
||||
mock_manager = MagicMock()
|
||||
mock_manager.connection.return_value = mock_conn
|
||||
|
||||
mock_driver = MagicMock()
|
||||
mock_driver.connection_manager.return_value = mock_manager
|
||||
mock_get_driver.return_value = mock_driver
|
||||
|
||||
mock_enabled.return_value = self.llm_enabled
|
||||
|
||||
if self.llm_enabled:
|
||||
mock_generate.return_value = (
|
||||
True, "# Security Report\n\nNo issues found."
|
||||
)
|
||||
|
||||
url = '/llm/security-report/' + str(self.server_id)
|
||||
response = self.tester.get(url, content_type='application/json')
|
||||
|
||||
# All responses return 200, check success field in JSON
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
|
||||
if self.llm_enabled:
|
||||
self.assertTrue(data['success'])
|
||||
self.assertIn('report', data['data'])
|
||||
else:
|
||||
self.assertFalse(data['success'])
|
||||
self.assertIn('errormsg', data)
|
||||
|
||||
|
||||
class PerformanceReportDatabaseTestCase(BaseTestGenerator):
|
||||
"""Test cases for performance report generation at database level"""
|
||||
|
||||
scenarios = [
|
||||
('Performance Report - Database Level', dict(
|
||||
llm_enabled=True
|
||||
)),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
self.server_id = 1
|
||||
self.db_id = 2
|
||||
|
||||
def runTest(self):
|
||||
"""Test performance report endpoint at database level"""
|
||||
with patch('pgadmin.llm.utils.is_llm_enabled') as mock_enabled, \
|
||||
patch('pgadmin.llm.reports.generator.generate_report_sync'
|
||||
) as mock_generate, \
|
||||
patch('pgadmin.utils.driver.get_driver') as mock_get_driver:
|
||||
|
||||
# Mock database connection
|
||||
mock_conn = MagicMock()
|
||||
mock_conn.connected.return_value = True
|
||||
mock_conn.db = 'testdb'
|
||||
|
||||
mock_manager = MagicMock()
|
||||
mock_manager.connection.return_value = mock_conn
|
||||
|
||||
mock_driver = MagicMock()
|
||||
mock_driver.connection_manager.return_value = mock_manager
|
||||
mock_get_driver.return_value = mock_driver
|
||||
|
||||
mock_enabled.return_value = self.llm_enabled
|
||||
mock_generate.return_value = (
|
||||
True, "# Performance Report\n\nOptimization suggestions..."
|
||||
)
|
||||
|
||||
url = ('/llm/database-performance-report/' +
|
||||
str(self.server_id) + '/' + str(self.db_id))
|
||||
response = self.tester.get(url, content_type='application/json')
|
||||
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertTrue(data['success'])
|
||||
|
||||
|
||||
class DesignReportSchemaTestCase(BaseTestGenerator):
|
||||
"""Test cases for design review report generation at schema level"""
|
||||
|
||||
scenarios = [
|
||||
('Design Report - Schema Level', dict(
|
||||
llm_enabled=True
|
||||
)),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
self.server_id = 1
|
||||
self.db_id = 2
|
||||
self.schema_id = 3
|
||||
|
||||
def runTest(self):
|
||||
"""Test design review report endpoint at schema level"""
|
||||
with patch('pgadmin.llm.utils.is_llm_enabled') as mock_enabled, \
|
||||
patch('pgadmin.llm.reports.generator.generate_report_sync'
|
||||
) as mock_generate, \
|
||||
patch('pgadmin.utils.driver.get_driver') as mock_get_driver:
|
||||
|
||||
# Mock connection to return schema name
|
||||
mock_conn = MagicMock()
|
||||
mock_conn.connected.return_value = True
|
||||
mock_conn.db = 'testdb'
|
||||
mock_conn.execute_dict.return_value = (
|
||||
True, {'rows': [{'nspname': 'public'}]}
|
||||
)
|
||||
|
||||
mock_manager = MagicMock()
|
||||
mock_manager.connection.return_value = mock_conn
|
||||
|
||||
mock_driver = MagicMock()
|
||||
mock_driver.connection_manager.return_value = mock_manager
|
||||
mock_get_driver.return_value = mock_driver
|
||||
|
||||
mock_enabled.return_value = self.llm_enabled
|
||||
mock_generate.return_value = (
|
||||
True, "# Design Review\n\nSchema structure looks good..."
|
||||
)
|
||||
|
||||
url = ('/llm/schema-design-report/' + str(self.server_id) +
|
||||
'/' + str(self.db_id) + '/' + str(self.schema_id))
|
||||
response = self.tester.get(url, content_type='application/json')
|
||||
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertTrue(data['success'])
|
||||
|
||||
|
||||
class StreamingReportTestCase(BaseTestGenerator):
|
||||
"""Test cases for streaming report endpoints with SSE"""
|
||||
|
||||
scenarios = [
|
||||
('Streaming Security Report - Server', dict()),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
self.server_id = 1
|
||||
|
||||
def runTest(self):
|
||||
"""Test streaming report endpoint uses SSE format"""
|
||||
with patch('pgadmin.llm.utils.is_llm_enabled') as mock_enabled, \
|
||||
patch('pgadmin.llm.reports.generator.generate_report_streaming'
|
||||
) as mock_streaming, \
|
||||
patch('pgadmin.utils.driver.get_driver') as mock_get_driver:
|
||||
|
||||
# Mock connection
|
||||
mock_conn = MagicMock()
|
||||
mock_conn.connected.return_value = True
|
||||
|
||||
mock_manager = MagicMock()
|
||||
mock_manager.connection.return_value = mock_conn
|
||||
|
||||
mock_driver = MagicMock()
|
||||
mock_driver.connection_manager.return_value = mock_manager
|
||||
mock_get_driver.return_value = mock_driver
|
||||
|
||||
mock_enabled.return_value = True
|
||||
mock_streaming.return_value = iter([]) # Empty generator
|
||||
|
||||
url = '/llm/security-report/' + str(self.server_id) + '/stream'
|
||||
response = self.tester.get(url)
|
||||
|
||||
# SSE endpoints should return 200 and have text/event-stream
|
||||
# content type
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.assertIn('text/event-stream', response.content_type)
|
||||
|
||||
|
||||
class ReportErrorHandlingTestCase(BaseTestGenerator):
|
||||
"""Test cases for report error handling"""
|
||||
|
||||
scenarios = [
|
||||
('Report with API Error', dict(
|
||||
simulate_error=True
|
||||
)),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
self.server_id = 1
|
||||
|
||||
def runTest(self):
|
||||
"""Test report endpoint handles LLM API errors gracefully"""
|
||||
with patch('pgadmin.llm.utils.is_llm_enabled') as mock_enabled, \
|
||||
patch('pgadmin.llm.reports.generator.generate_report_sync'
|
||||
) as mock_generate, \
|
||||
patch('pgadmin.utils.driver.get_driver') as mock_get_driver:
|
||||
|
||||
# Mock database connection
|
||||
mock_conn = MagicMock()
|
||||
mock_conn.connected.return_value = True
|
||||
|
||||
mock_manager = MagicMock()
|
||||
mock_manager.connection.return_value = mock_conn
|
||||
|
||||
mock_driver = MagicMock()
|
||||
mock_driver.connection_manager.return_value = mock_manager
|
||||
mock_get_driver.return_value = mock_driver
|
||||
|
||||
mock_enabled.return_value = True
|
||||
|
||||
if self.simulate_error:
|
||||
mock_generate.side_effect = Exception("API connection failed")
|
||||
|
||||
url = '/llm/security-report/' + str(self.server_id)
|
||||
response = self.tester.get(url, content_type='application/json')
|
||||
|
||||
# Should return 200 with error in JSON, not crash
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertFalse(data['success'])
|
||||
self.assertIn('errormsg', data)
|
||||
30
web/pgadmin/llm/tools/__init__.py
Normal file
30
web/pgadmin/llm/tools/__init__.py
Normal file
@@ -0,0 +1,30 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""LLM tools for interacting with PostgreSQL databases."""
|
||||
|
||||
from pgadmin.llm.tools.database import (
|
||||
execute_readonly_query,
|
||||
get_database_schema,
|
||||
get_table_columns,
|
||||
get_table_info,
|
||||
execute_tool,
|
||||
DatabaseToolError,
|
||||
DATABASE_TOOLS
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
'execute_readonly_query',
|
||||
'get_database_schema',
|
||||
'get_table_columns',
|
||||
'get_table_info',
|
||||
'execute_tool',
|
||||
'DatabaseToolError',
|
||||
'DATABASE_TOOLS'
|
||||
]
|
||||
813
web/pgadmin/llm/tools/database.py
Normal file
813
web/pgadmin/llm/tools/database.py
Normal file
@@ -0,0 +1,813 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Database tools for LLM interactions.
|
||||
|
||||
These tools allow the LLM to query PostgreSQL databases in a safe,
|
||||
read-only manner. All queries are executed within read-only transactions
|
||||
to prevent any data modification.
|
||||
|
||||
Uses pgAdmin's SQL template infrastructure for version-aware queries.
|
||||
"""
|
||||
|
||||
import secrets
|
||||
from typing import Optional
|
||||
|
||||
from flask import render_template
|
||||
|
||||
from pgadmin.utils.driver import get_driver
|
||||
from pgadmin.utils.compile_template_name import compile_template_path
|
||||
from pgadmin.llm.models import Tool
|
||||
import config
|
||||
|
||||
|
||||
# Template paths for SQL queries (used with compile_template_path)
|
||||
SCHEMAS_TEMPLATE_PATH = 'schemas/pg'
|
||||
TABLES_TEMPLATE_PATH = 'tables/sql'
|
||||
COLUMNS_TEMPLATE_PATH = 'columns/sql'
|
||||
INDEXES_TEMPLATE_PATH = 'indexes/sql'
|
||||
|
||||
|
||||
# Application name prefix for LLM connections
|
||||
LLM_APP_NAME_PREFIX = 'pgAdmin 4 - LLM'
|
||||
|
||||
|
||||
class DatabaseToolError(Exception):
|
||||
"""Exception raised when a database tool operation fails."""
|
||||
|
||||
def __init__(self, message: str, code: Optional[str] = None):
|
||||
self.message = message
|
||||
self.code = code
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
def _get_connection(sid: int, did: int, conn_id: str):
|
||||
"""
|
||||
Get a database connection for the specified server and database.
|
||||
|
||||
Args:
|
||||
sid: Server ID
|
||||
did: Database ID (OID)
|
||||
conn_id: Unique connection identifier
|
||||
|
||||
Returns:
|
||||
Tuple of (manager, connection) objects
|
||||
|
||||
Raises:
|
||||
DatabaseToolError: If connection fails
|
||||
"""
|
||||
try:
|
||||
driver = get_driver(config.PG_DEFAULT_DRIVER)
|
||||
manager = driver.connection_manager(sid)
|
||||
|
||||
# Get connection - this will create one if it doesn't exist
|
||||
conn = manager.connection(
|
||||
did=did,
|
||||
conn_id=conn_id,
|
||||
auto_reconnect=False, # Don't auto-reconnect for LLM queries
|
||||
use_binary_placeholder=True,
|
||||
array_to_string=True
|
||||
)
|
||||
|
||||
return manager, conn
|
||||
|
||||
except Exception as e:
|
||||
raise DatabaseToolError(
|
||||
f"Failed to get connection: {str(e)}",
|
||||
code="CONNECTION_ERROR"
|
||||
)
|
||||
|
||||
|
||||
def _connect_readonly(_manager, conn, conn_id: str) -> tuple[bool, str | None]:
|
||||
"""
|
||||
Establish a read-only connection.
|
||||
|
||||
Sets the application_name to identify this as an LLM connection
|
||||
and ensures the connection is in read-only mode.
|
||||
|
||||
Args:
|
||||
_manager: The server manager (unused, kept for API consistency)
|
||||
conn: The connection object
|
||||
conn_id: Connection identifier
|
||||
|
||||
Returns:
|
||||
Tuple of (success, error_message)
|
||||
"""
|
||||
try:
|
||||
# Connect if not already connected
|
||||
if not conn.connected():
|
||||
status, msg = conn.connect()
|
||||
if not status:
|
||||
return False, msg
|
||||
|
||||
# Set application name via SQL - this is thread-safe and doesn't
|
||||
# require environment variables. The name will be visible in
|
||||
# pg_stat_activity to identify LLM connections.
|
||||
app_name = f'{LLM_APP_NAME_PREFIX} - {conn_id}'
|
||||
# Escape single quotes in the app name for safety
|
||||
app_name_escaped = app_name.replace("'", "''")
|
||||
status, _ = conn.execute_void(
|
||||
f"SET application_name = '{app_name_escaped}'"
|
||||
)
|
||||
if not status:
|
||||
# Non-fatal - connection still works without custom app name
|
||||
pass
|
||||
|
||||
return True, None
|
||||
|
||||
except Exception as e:
|
||||
return False, str(e)
|
||||
|
||||
|
||||
def _execute_readonly_query(conn, query: str) -> dict:
|
||||
"""
|
||||
Execute a query in a read-only transaction.
|
||||
|
||||
The query is wrapped in a read-only transaction to ensure
|
||||
no data modifications can occur.
|
||||
|
||||
Args:
|
||||
conn: Database connection
|
||||
query: SQL query to execute
|
||||
|
||||
Returns:
|
||||
Dictionary with 'columns' and 'rows' keys
|
||||
|
||||
Raises:
|
||||
DatabaseToolError: If query execution fails
|
||||
"""
|
||||
# Set the transaction to read-only, execute, then rollback.
|
||||
# This ensures even if the query tries to modify data, it will fail.
|
||||
try:
|
||||
# First, set the transaction to read-only mode
|
||||
status, result = conn.execute_void(
|
||||
"BEGIN TRANSACTION READ ONLY"
|
||||
)
|
||||
if not status:
|
||||
raise DatabaseToolError(
|
||||
f"Failed to start read-only transaction: {result}",
|
||||
code="TRANSACTION_ERROR"
|
||||
)
|
||||
|
||||
try:
|
||||
# Execute the actual query
|
||||
status, result = conn.execute_2darray(query)
|
||||
|
||||
if not status:
|
||||
raise DatabaseToolError(
|
||||
f"Query execution failed: {result}",
|
||||
code="QUERY_ERROR"
|
||||
)
|
||||
|
||||
# Format the result
|
||||
columns = []
|
||||
rows = []
|
||||
|
||||
if result and 'columns' in result:
|
||||
columns = [col['name'] for col in result['columns']]
|
||||
|
||||
if result and 'rows' in result:
|
||||
rows = result['rows']
|
||||
|
||||
return {
|
||||
'columns': columns,
|
||||
'rows': rows,
|
||||
'row_count': len(rows)
|
||||
}
|
||||
|
||||
finally:
|
||||
# Always rollback - we're read-only anyway
|
||||
conn.execute_void("ROLLBACK")
|
||||
|
||||
except DatabaseToolError:
|
||||
raise
|
||||
except Exception as e:
|
||||
# Attempt rollback on any error
|
||||
try:
|
||||
conn.execute_void("ROLLBACK")
|
||||
except Exception:
|
||||
pass
|
||||
raise DatabaseToolError(
|
||||
f"Query execution error: {str(e)}",
|
||||
code="EXECUTION_ERROR"
|
||||
)
|
||||
|
||||
|
||||
def execute_readonly_query(
|
||||
sid: int,
|
||||
did: int,
|
||||
query: str,
|
||||
max_rows: int = 1000
|
||||
) -> dict:
|
||||
"""
|
||||
Execute a read-only SQL query against a PostgreSQL database.
|
||||
|
||||
This function:
|
||||
1. Opens a new connection with LLM-specific application_name
|
||||
2. Starts a READ ONLY transaction
|
||||
3. Executes the query
|
||||
4. Returns results (limited to max_rows)
|
||||
5. Rolls back and closes the connection
|
||||
|
||||
Args:
|
||||
sid: Server ID from the Object Explorer
|
||||
did: Database ID (OID) from the Object Explorer
|
||||
query: SQL query to execute (should be SELECT or read-only)
|
||||
max_rows: Maximum number of rows to return (default 1000)
|
||||
|
||||
Returns:
|
||||
Dictionary containing:
|
||||
- columns: List of column names
|
||||
- rows: List of row data (as lists)
|
||||
- row_count: Number of rows returned
|
||||
- truncated: True if results were limited
|
||||
|
||||
Raises:
|
||||
DatabaseToolError: If the query fails or connection
|
||||
cannot be established
|
||||
"""
|
||||
# Generate unique connection ID for this LLM query
|
||||
conn_id = f"llm_{secrets.choice(range(1, 9999999))}"
|
||||
|
||||
manager = None
|
||||
conn = None
|
||||
|
||||
try:
|
||||
# Get connection manager and connection object
|
||||
manager, conn = _get_connection(sid, did, conn_id)
|
||||
|
||||
# Connect with read-only settings
|
||||
status, error = _connect_readonly(manager, conn, conn_id)
|
||||
if not status:
|
||||
raise DatabaseToolError(
|
||||
f"Connection failed: {error}",
|
||||
code="CONNECTION_ERROR"
|
||||
)
|
||||
|
||||
# Add LIMIT if not already present and query looks like SELECT
|
||||
query_upper = query.strip().upper()
|
||||
if query_upper.startswith('SELECT') and 'LIMIT' not in query_upper:
|
||||
query = f"({query}) AS llm_subquery LIMIT {max_rows + 1}"
|
||||
query = f"SELECT * FROM {query}"
|
||||
|
||||
# Execute the query
|
||||
result = _execute_readonly_query(conn, query)
|
||||
|
||||
# Check if we need to truncate
|
||||
if len(result['rows']) > max_rows:
|
||||
result['rows'] = result['rows'][:max_rows]
|
||||
result['truncated'] = True
|
||||
result['row_count'] = max_rows
|
||||
else:
|
||||
result['truncated'] = False
|
||||
|
||||
return result
|
||||
|
||||
finally:
|
||||
# Always release the connection
|
||||
if manager and conn_id:
|
||||
try:
|
||||
manager.release(conn_id=conn_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def get_database_schema(sid: int, did: int) -> dict:
|
||||
"""
|
||||
Get the schema information for a database.
|
||||
|
||||
Uses pgAdmin's SQL templates for version-aware schema listing.
|
||||
|
||||
Args:
|
||||
sid: Server ID
|
||||
did: Database ID
|
||||
|
||||
Returns:
|
||||
Dictionary containing schema information organized by schema name
|
||||
"""
|
||||
conn_id = f"llm_{secrets.choice(range(1, 9999999))}"
|
||||
manager = None
|
||||
|
||||
try:
|
||||
manager, conn = _get_connection(sid, did, conn_id)
|
||||
status, error = _connect_readonly(manager, conn, conn_id)
|
||||
if not status:
|
||||
raise DatabaseToolError(f"Connection failed: {error}",
|
||||
code="CONNECTION_ERROR")
|
||||
|
||||
# Get server version for template selection
|
||||
sversion = manager.sversion or 0
|
||||
|
||||
# Build template path with version - the versioned loader will
|
||||
# find the appropriate directory (e.g., 15_plus, 14_plus, default)
|
||||
schema_template_path = compile_template_path(
|
||||
SCHEMAS_TEMPLATE_PATH, sversion
|
||||
)
|
||||
|
||||
# Get list of schemas using the template
|
||||
schema_sql = render_template(
|
||||
"/".join([schema_template_path, 'sql', 'nodes.sql']),
|
||||
show_sysobj=False,
|
||||
scid=None,
|
||||
schema_restrictions=None
|
||||
)
|
||||
|
||||
# Execute in read-only mode
|
||||
status, _ = conn.execute_void("BEGIN TRANSACTION READ ONLY")
|
||||
if not status:
|
||||
raise DatabaseToolError("Failed to start transaction",
|
||||
code="TRANSACTION_ERROR")
|
||||
|
||||
try:
|
||||
status, schema_res = conn.execute_dict(schema_sql)
|
||||
if not status:
|
||||
raise DatabaseToolError(f"Schema query failed: {schema_res}",
|
||||
code="QUERY_ERROR")
|
||||
|
||||
schemas = {}
|
||||
table_template_path = compile_template_path(
|
||||
TABLES_TEMPLATE_PATH, sversion
|
||||
)
|
||||
|
||||
for schema_row in schema_res.get('rows', []):
|
||||
schema_name = schema_row['name']
|
||||
schema_oid = schema_row['oid']
|
||||
|
||||
# Get tables for this schema using the template
|
||||
tables_sql = render_template(
|
||||
"/".join([table_template_path, 'nodes.sql']),
|
||||
scid=schema_oid,
|
||||
tid=None,
|
||||
schema_diff=False
|
||||
)
|
||||
|
||||
status, tables_res = conn.execute_dict(tables_sql)
|
||||
tables = []
|
||||
if status and tables_res:
|
||||
for row in tables_res.get('rows', []):
|
||||
tables.append({
|
||||
'name': row.get('name'),
|
||||
'oid': row.get('oid'),
|
||||
'description': row.get('description')
|
||||
})
|
||||
|
||||
# Get views (relkind v=view, m=materialized view)
|
||||
views_sql = f"""
|
||||
SELECT c.oid,
|
||||
c.relname AS name,
|
||||
pg_catalog.obj_description(
|
||||
c.oid, 'pg_class'
|
||||
) AS description
|
||||
FROM pg_catalog.pg_class c
|
||||
WHERE c.relkind IN ('v', 'm')
|
||||
AND c.relnamespace = {schema_oid}::oid
|
||||
ORDER BY c.relname
|
||||
"""
|
||||
status, views_res = conn.execute_dict(views_sql)
|
||||
views = []
|
||||
if status and views_res:
|
||||
for row in views_res.get('rows', []):
|
||||
views.append({
|
||||
'name': row.get('name'),
|
||||
'oid': row.get('oid'),
|
||||
'description': row.get('description')
|
||||
})
|
||||
|
||||
schemas[schema_name] = {
|
||||
'oid': schema_oid,
|
||||
'tables': tables,
|
||||
'views': views,
|
||||
'description': schema_row.get('description')
|
||||
}
|
||||
|
||||
return {'schemas': schemas}
|
||||
|
||||
finally:
|
||||
conn.execute_void("ROLLBACK")
|
||||
|
||||
finally:
|
||||
if manager and conn_id:
|
||||
try:
|
||||
manager.release(conn_id=conn_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def get_table_columns(
|
||||
sid: int,
|
||||
did: int,
|
||||
schema_name: str,
|
||||
table_name: str
|
||||
) -> dict:
|
||||
"""
|
||||
Get column information for a specific table.
|
||||
|
||||
Uses pgAdmin's SQL templates for version-aware column listing.
|
||||
|
||||
Args:
|
||||
sid: Server ID
|
||||
did: Database ID
|
||||
schema_name: Schema name
|
||||
table_name: Table name
|
||||
|
||||
Returns:
|
||||
Dictionary containing column information
|
||||
"""
|
||||
conn_id = f"llm_{secrets.choice(range(1, 9999999))}"
|
||||
manager = None
|
||||
|
||||
try:
|
||||
manager, conn = _get_connection(sid, did, conn_id)
|
||||
status, error = _connect_readonly(manager, conn, conn_id)
|
||||
if not status:
|
||||
raise DatabaseToolError(f"Connection failed: {error}",
|
||||
code="CONNECTION_ERROR")
|
||||
|
||||
sversion = manager.sversion or 0
|
||||
driver = get_driver(config.PG_DEFAULT_DRIVER)
|
||||
|
||||
# Use qtLiteral for safe SQL escaping
|
||||
schema_lit = driver.qtLiteral(schema_name, conn)
|
||||
table_lit = driver.qtLiteral(table_name, conn)
|
||||
|
||||
# Get table OID first
|
||||
oid_sql = f"""
|
||||
SELECT c.oid
|
||||
FROM pg_catalog.pg_class c
|
||||
JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
|
||||
WHERE c.relname = {table_lit}
|
||||
AND n.nspname = {schema_lit}
|
||||
"""
|
||||
|
||||
status, _ = conn.execute_void("BEGIN TRANSACTION READ ONLY")
|
||||
if not status:
|
||||
raise DatabaseToolError("Failed to start transaction",
|
||||
code="TRANSACTION_ERROR")
|
||||
|
||||
try:
|
||||
status, oid_res = conn.execute_dict(oid_sql)
|
||||
if not status or not oid_res.get('rows'):
|
||||
raise DatabaseToolError(
|
||||
f"Table {schema_name}.{table_name} not found",
|
||||
code="NOT_FOUND"
|
||||
)
|
||||
|
||||
table_oid = oid_res['rows'][0]['oid']
|
||||
|
||||
# Use the columns template
|
||||
col_template_path = compile_template_path(
|
||||
COLUMNS_TEMPLATE_PATH, sversion
|
||||
)
|
||||
columns_sql = render_template(
|
||||
"/".join([col_template_path, 'nodes.sql']),
|
||||
tid=table_oid,
|
||||
clid=None,
|
||||
show_sys_objects=False,
|
||||
has_oids=False,
|
||||
conn=conn
|
||||
)
|
||||
|
||||
status, cols_res = conn.execute_dict(columns_sql)
|
||||
if not status:
|
||||
raise DatabaseToolError(f"Column query failed: {cols_res}",
|
||||
code="QUERY_ERROR")
|
||||
|
||||
columns = []
|
||||
for row in cols_res.get('rows', []):
|
||||
columns.append({
|
||||
'name': row.get('name'),
|
||||
'data_type': (
|
||||
row.get('displaytypname') or
|
||||
row.get('datatype')
|
||||
),
|
||||
'not_null': row.get('not_null', False),
|
||||
'has_default': row.get('has_default_val', False),
|
||||
'description': row.get('description')
|
||||
})
|
||||
|
||||
return {
|
||||
'schema': schema_name,
|
||||
'table': table_name,
|
||||
'oid': table_oid,
|
||||
'columns': columns
|
||||
}
|
||||
|
||||
finally:
|
||||
conn.execute_void("ROLLBACK")
|
||||
|
||||
finally:
|
||||
if manager and conn_id:
|
||||
try:
|
||||
manager.release(conn_id=conn_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def get_table_info(
|
||||
sid: int,
|
||||
did: int,
|
||||
schema_name: str,
|
||||
table_name: str
|
||||
) -> dict:
|
||||
"""
|
||||
Get detailed information about a table including columns,
|
||||
constraints, and indexes.
|
||||
|
||||
Uses pgAdmin's SQL templates for version-aware queries.
|
||||
|
||||
Args:
|
||||
sid: Server ID
|
||||
did: Database ID
|
||||
schema_name: Schema name
|
||||
table_name: Table name
|
||||
|
||||
Returns:
|
||||
Dictionary containing comprehensive table information
|
||||
"""
|
||||
conn_id = f"llm_{secrets.choice(range(1, 9999999))}"
|
||||
manager = None
|
||||
|
||||
try:
|
||||
manager, conn = _get_connection(sid, did, conn_id)
|
||||
status, error = _connect_readonly(manager, conn, conn_id)
|
||||
if not status:
|
||||
raise DatabaseToolError(f"Connection failed: {error}",
|
||||
code="CONNECTION_ERROR")
|
||||
|
||||
sversion = manager.sversion or 0
|
||||
driver = get_driver(config.PG_DEFAULT_DRIVER)
|
||||
|
||||
# Use qtLiteral for safe SQL escaping
|
||||
schema_lit = driver.qtLiteral(schema_name, conn)
|
||||
table_lit = driver.qtLiteral(table_name, conn)
|
||||
|
||||
# Get table OID first
|
||||
oid_sql = f"""
|
||||
SELECT c.oid, n.oid as schema_oid
|
||||
FROM pg_catalog.pg_class c
|
||||
JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
|
||||
WHERE c.relname = {table_lit}
|
||||
AND n.nspname = {schema_lit}
|
||||
"""
|
||||
|
||||
status, _ = conn.execute_void("BEGIN TRANSACTION READ ONLY")
|
||||
if not status:
|
||||
raise DatabaseToolError("Failed to start transaction",
|
||||
code="TRANSACTION_ERROR")
|
||||
|
||||
try:
|
||||
status, oid_res = conn.execute_dict(oid_sql)
|
||||
if not status or not oid_res.get('rows'):
|
||||
raise DatabaseToolError(
|
||||
f"Table {schema_name}.{table_name} not found",
|
||||
code="NOT_FOUND"
|
||||
)
|
||||
|
||||
table_oid = oid_res['rows'][0]['oid']
|
||||
|
||||
# Get columns using template
|
||||
col_template_path = compile_template_path(
|
||||
COLUMNS_TEMPLATE_PATH, sversion
|
||||
)
|
||||
columns_sql = render_template(
|
||||
"/".join([col_template_path, 'nodes.sql']),
|
||||
tid=table_oid,
|
||||
clid=None,
|
||||
show_sys_objects=False,
|
||||
has_oids=False,
|
||||
conn=conn
|
||||
)
|
||||
|
||||
status, cols_res = conn.execute_dict(columns_sql)
|
||||
columns = []
|
||||
if status and cols_res:
|
||||
for row in cols_res.get('rows', []):
|
||||
columns.append({
|
||||
'name': row.get('name'),
|
||||
'data_type': (
|
||||
row.get('displaytypname') or
|
||||
row.get('datatype')
|
||||
),
|
||||
'not_null': row.get('not_null', False),
|
||||
'has_default': row.get('has_default_val', False),
|
||||
'description': row.get('description')
|
||||
})
|
||||
|
||||
# Get constraints (using table OID for safety)
|
||||
constraints_sql = f"""
|
||||
SELECT
|
||||
con.conname AS name,
|
||||
CASE con.contype
|
||||
WHEN 'p' THEN 'PRIMARY KEY'
|
||||
WHEN 'u' THEN 'UNIQUE'
|
||||
WHEN 'f' THEN 'FOREIGN KEY'
|
||||
WHEN 'c' THEN 'CHECK'
|
||||
WHEN 'x' THEN 'EXCLUSION'
|
||||
END AS type,
|
||||
pg_catalog.pg_get_constraintdef(
|
||||
con.oid, true
|
||||
) AS definition
|
||||
FROM pg_catalog.pg_constraint con
|
||||
WHERE con.conrelid = {table_oid}::oid
|
||||
ORDER BY con.contype, con.conname
|
||||
"""
|
||||
|
||||
status, cons_res = conn.execute_dict(constraints_sql)
|
||||
constraints = []
|
||||
if status and cons_res:
|
||||
for row in cons_res.get('rows', []):
|
||||
constraints.append({
|
||||
'name': row.get('name'),
|
||||
'type': row.get('type'),
|
||||
'definition': row.get('definition')
|
||||
})
|
||||
|
||||
# Get indexes using template
|
||||
idx_template_path = compile_template_path(
|
||||
INDEXES_TEMPLATE_PATH, sversion
|
||||
)
|
||||
indexes_sql = render_template(
|
||||
"/".join([idx_template_path, 'nodes.sql']),
|
||||
tid=table_oid,
|
||||
idx=None
|
||||
)
|
||||
|
||||
status, idx_res = conn.execute_dict(indexes_sql)
|
||||
indexes = []
|
||||
if status and idx_res:
|
||||
for row in idx_res.get('rows', []):
|
||||
indexes.append({
|
||||
'name': row.get('name'),
|
||||
'oid': row.get('oid')
|
||||
})
|
||||
|
||||
return {
|
||||
'schema': schema_name,
|
||||
'table': table_name,
|
||||
'oid': table_oid,
|
||||
'columns': columns,
|
||||
'constraints': constraints,
|
||||
'indexes': indexes
|
||||
}
|
||||
|
||||
finally:
|
||||
conn.execute_void("ROLLBACK")
|
||||
|
||||
finally:
|
||||
if manager and conn_id:
|
||||
try:
|
||||
manager.release(conn_id=conn_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def execute_tool(
|
||||
tool_name: str,
|
||||
arguments: dict,
|
||||
sid: int,
|
||||
did: int
|
||||
) -> dict:
|
||||
"""
|
||||
Execute a database tool by name.
|
||||
|
||||
This is the dispatcher function that maps tool calls from the LLM
|
||||
to the actual function implementations.
|
||||
|
||||
Args:
|
||||
tool_name: Name of the tool to execute
|
||||
arguments: Tool arguments from the LLM
|
||||
sid: Server ID
|
||||
did: Database ID
|
||||
|
||||
Returns:
|
||||
Dictionary containing the tool result
|
||||
|
||||
Raises:
|
||||
DatabaseToolError: If the tool execution fails
|
||||
ValueError: If the tool name is not recognized
|
||||
"""
|
||||
if tool_name == "execute_sql_query":
|
||||
query = arguments.get("query")
|
||||
if not query:
|
||||
raise DatabaseToolError(
|
||||
"Missing required argument: query",
|
||||
code="INVALID_ARGUMENTS"
|
||||
)
|
||||
return execute_readonly_query(sid, did, query)
|
||||
|
||||
elif tool_name == "get_database_schema":
|
||||
return get_database_schema(sid, did)
|
||||
|
||||
elif tool_name == "get_table_columns":
|
||||
schema_name = arguments.get("schema_name")
|
||||
table_name = arguments.get("table_name")
|
||||
if not schema_name or not table_name:
|
||||
raise DatabaseToolError(
|
||||
"Missing required arguments: schema_name and table_name",
|
||||
code="INVALID_ARGUMENTS"
|
||||
)
|
||||
return get_table_columns(sid, did, schema_name, table_name)
|
||||
|
||||
elif tool_name == "get_table_info":
|
||||
schema_name = arguments.get("schema_name")
|
||||
table_name = arguments.get("table_name")
|
||||
if not schema_name or not table_name:
|
||||
raise DatabaseToolError(
|
||||
"Missing required arguments: schema_name and table_name",
|
||||
code="INVALID_ARGUMENTS"
|
||||
)
|
||||
return get_table_info(sid, did, schema_name, table_name)
|
||||
|
||||
else:
|
||||
raise ValueError(f"Unknown tool: {tool_name}")
|
||||
|
||||
|
||||
# Tool definitions for LLM use
|
||||
DATABASE_TOOLS = [
|
||||
Tool(
|
||||
name="execute_sql_query",
|
||||
description=(
|
||||
"Execute a read-only SQL query against the PostgreSQL database. "
|
||||
"The query runs in a READ ONLY transaction so no data can be "
|
||||
"modified. Use this to retrieve data, check table contents, "
|
||||
"or run analytical queries. Results are limited to 1000 rows."
|
||||
),
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"The SQL query to execute. Should "
|
||||
"be a SELECT query or other "
|
||||
"read-only statement. DML "
|
||||
"statements will fail."
|
||||
)
|
||||
}
|
||||
},
|
||||
"required": ["query"]
|
||||
}
|
||||
),
|
||||
Tool(
|
||||
name="get_database_schema",
|
||||
description=(
|
||||
"Get a list of all schemas, tables, and views "
|
||||
"in the database. Use this to understand the "
|
||||
"database structure before writing queries."
|
||||
),
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {},
|
||||
"required": []
|
||||
}
|
||||
),
|
||||
Tool(
|
||||
name="get_table_columns",
|
||||
description=(
|
||||
"Get detailed column information for a specific table, including "
|
||||
"data types, nullability, defaults, and primary key status."
|
||||
),
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"schema_name": {
|
||||
"type": "string",
|
||||
"description": "The schema name (e.g., 'public')"
|
||||
},
|
||||
"table_name": {
|
||||
"type": "string",
|
||||
"description": "The table name"
|
||||
}
|
||||
},
|
||||
"required": ["schema_name", "table_name"]
|
||||
}
|
||||
),
|
||||
Tool(
|
||||
name="get_table_info",
|
||||
description=(
|
||||
"Get comprehensive information about a table including columns, "
|
||||
"constraints (primary keys, foreign keys, check constraints), "
|
||||
"and indexes."
|
||||
),
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"schema_name": {
|
||||
"type": "string",
|
||||
"description": "The schema name (e.g., 'public')"
|
||||
},
|
||||
"table_name": {
|
||||
"type": "string",
|
||||
"description": "The table name"
|
||||
}
|
||||
},
|
||||
"required": ["schema_name", "table_name"]
|
||||
}
|
||||
)
|
||||
]
|
||||
357
web/pgadmin/llm/utils.py
Normal file
357
web/pgadmin/llm/utils.py
Normal file
@@ -0,0 +1,357 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Utility functions for LLM configuration access."""
|
||||
|
||||
import os
|
||||
from pgadmin.utils.preferences import Preferences
|
||||
import config
|
||||
|
||||
|
||||
def _expand_path(path):
|
||||
"""Expand user home directory in path."""
|
||||
if path:
|
||||
return os.path.expanduser(path)
|
||||
return path
|
||||
|
||||
|
||||
def _read_api_key_from_file(file_path):
|
||||
"""
|
||||
Read an API key from a file.
|
||||
|
||||
Args:
|
||||
file_path: Path to the file containing the API key.
|
||||
|
||||
Returns:
|
||||
The API key string, or None if the file doesn't exist or is empty.
|
||||
"""
|
||||
if not file_path:
|
||||
return None
|
||||
|
||||
expanded_path = _expand_path(file_path)
|
||||
|
||||
if not os.path.isfile(expanded_path):
|
||||
return None
|
||||
|
||||
try:
|
||||
with open(expanded_path, 'r') as f:
|
||||
key = f.read().strip()
|
||||
return key if key else None
|
||||
except (IOError, OSError):
|
||||
return None
|
||||
|
||||
|
||||
# Public alias for use by refresh endpoints
|
||||
read_api_key_file = _read_api_key_from_file
|
||||
|
||||
|
||||
def _get_preference_value(name):
|
||||
"""
|
||||
Get a preference value, returning None if empty or not set.
|
||||
|
||||
Args:
|
||||
name: The preference name (e.g., 'anthropic_api_key_file')
|
||||
|
||||
Returns:
|
||||
The preference value or None if empty/not set.
|
||||
"""
|
||||
try:
|
||||
pref_module = Preferences.module('ai')
|
||||
if pref_module:
|
||||
pref = pref_module.preference(name)
|
||||
if pref:
|
||||
value = pref.get()
|
||||
if value and str(value).strip():
|
||||
return str(value).strip()
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def get_anthropic_api_key():
|
||||
"""
|
||||
Get the Anthropic API key.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The API key string, or None if not configured or file doesn't exist.
|
||||
"""
|
||||
# Check user preference first
|
||||
pref_file = _get_preference_value('anthropic_api_key_file')
|
||||
if pref_file:
|
||||
key = _read_api_key_from_file(pref_file)
|
||||
if key:
|
||||
return key
|
||||
|
||||
# Fall back to system configuration
|
||||
return _read_api_key_from_file(config.ANTHROPIC_API_KEY_FILE)
|
||||
|
||||
|
||||
def get_anthropic_model():
|
||||
"""
|
||||
Get the Anthropic model to use.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The model name string, or empty string if not configured.
|
||||
"""
|
||||
# Check user preference first
|
||||
pref_model = _get_preference_value('anthropic_api_model')
|
||||
if pref_model:
|
||||
return pref_model
|
||||
|
||||
# Fall back to system configuration
|
||||
return config.ANTHROPIC_API_MODEL or ''
|
||||
|
||||
|
||||
def get_openai_api_key():
|
||||
"""
|
||||
Get the OpenAI API key.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The API key string, or None if not configured or file doesn't exist.
|
||||
"""
|
||||
# Check user preference first
|
||||
pref_file = _get_preference_value('openai_api_key_file')
|
||||
if pref_file:
|
||||
key = _read_api_key_from_file(pref_file)
|
||||
if key:
|
||||
return key
|
||||
|
||||
# Fall back to system configuration
|
||||
return _read_api_key_from_file(config.OPENAI_API_KEY_FILE)
|
||||
|
||||
|
||||
def get_openai_model():
|
||||
"""
|
||||
Get the OpenAI model to use.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The model name string, or empty string if not configured.
|
||||
"""
|
||||
# Check user preference first
|
||||
pref_model = _get_preference_value('openai_api_model')
|
||||
if pref_model:
|
||||
return pref_model
|
||||
|
||||
# Fall back to system configuration
|
||||
return config.OPENAI_API_MODEL or ''
|
||||
|
||||
|
||||
def get_ollama_api_url():
|
||||
"""
|
||||
Get the Ollama API URL.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The URL string, or empty string if not configured.
|
||||
"""
|
||||
# Check user preference first
|
||||
pref_url = _get_preference_value('ollama_api_url')
|
||||
if pref_url:
|
||||
return pref_url
|
||||
|
||||
# Fall back to system configuration
|
||||
return config.OLLAMA_API_URL or ''
|
||||
|
||||
|
||||
def get_ollama_model():
|
||||
"""
|
||||
Get the Ollama model to use.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The model name string, or empty string if not configured.
|
||||
"""
|
||||
# Check user preference first
|
||||
pref_model = _get_preference_value('ollama_api_model')
|
||||
if pref_model:
|
||||
return pref_model
|
||||
|
||||
# Fall back to system configuration
|
||||
return config.OLLAMA_API_MODEL or ''
|
||||
|
||||
|
||||
def get_docker_api_url():
|
||||
"""
|
||||
Get the Docker Model Runner API URL.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The URL string, or empty string if not configured.
|
||||
"""
|
||||
# Check user preference first
|
||||
pref_url = _get_preference_value('docker_api_url')
|
||||
if pref_url:
|
||||
return pref_url
|
||||
|
||||
# Fall back to system configuration
|
||||
return config.DOCKER_API_URL or ''
|
||||
|
||||
|
||||
def get_docker_model():
|
||||
"""
|
||||
Get the Docker Model Runner model to use.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The model name string, or empty string if not configured.
|
||||
"""
|
||||
# Check user preference first
|
||||
pref_model = _get_preference_value('docker_api_model')
|
||||
if pref_model:
|
||||
return pref_model
|
||||
|
||||
# Fall back to system configuration
|
||||
return config.DOCKER_API_MODEL or ''
|
||||
|
||||
|
||||
def get_default_provider():
|
||||
"""
|
||||
Get the default LLM provider.
|
||||
|
||||
First checks if LLM is enabled at the system level (config.LLM_ENABLED).
|
||||
If enabled, reads from user preferences (which default to system config).
|
||||
Returns None if disabled at system level or user preference is empty.
|
||||
|
||||
Returns:
|
||||
The provider name ('anthropic', 'openai', 'ollama', 'docker')
|
||||
or None if disabled.
|
||||
"""
|
||||
# Check master switch first - cannot be overridden by user
|
||||
if not getattr(config, 'LLM_ENABLED', False):
|
||||
return None
|
||||
|
||||
# Valid provider values
|
||||
valid_providers = {'anthropic', 'openai', 'ollama', 'docker'}
|
||||
|
||||
# Get preference value (includes config default if not set by user)
|
||||
try:
|
||||
pref_module = Preferences.module('ai')
|
||||
if pref_module:
|
||||
pref = pref_module.preference('default_provider')
|
||||
if pref:
|
||||
value = pref.get()
|
||||
# Check if it's a valid provider
|
||||
if value and str(value).strip() in valid_providers:
|
||||
return str(value).strip()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# No valid provider configured
|
||||
return None
|
||||
|
||||
|
||||
def is_llm_enabled_system():
|
||||
"""
|
||||
Check if LLM features are enabled at the system level.
|
||||
|
||||
This checks the config.LLM_ENABLED setting which cannot be
|
||||
overridden by user preferences.
|
||||
|
||||
Returns:
|
||||
True if LLM is enabled in system config, False otherwise.
|
||||
"""
|
||||
return getattr(config, 'LLM_ENABLED', False)
|
||||
|
||||
|
||||
def is_llm_enabled():
|
||||
"""
|
||||
Check if LLM features are enabled for the current user.
|
||||
|
||||
This checks both the system-level config (LLM_ENABLED) and
|
||||
whether a valid provider is configured in user preferences.
|
||||
|
||||
Returns:
|
||||
True if LLM is enabled and a provider is configured, False otherwise.
|
||||
"""
|
||||
return get_default_provider() is not None
|
||||
|
||||
|
||||
def get_max_tool_iterations():
|
||||
"""
|
||||
Get the maximum number of tool iterations for AI conversations.
|
||||
|
||||
Checks user preferences first, then falls back to system configuration.
|
||||
|
||||
Returns:
|
||||
The maximum tool iterations (default 20).
|
||||
"""
|
||||
try:
|
||||
pref_module = Preferences.module('ai')
|
||||
if pref_module:
|
||||
pref = pref_module.preference('max_tool_iterations')
|
||||
if pref:
|
||||
value = pref.get()
|
||||
if value is not None:
|
||||
return int(value)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Fall back to system configuration
|
||||
return getattr(config, 'MAX_LLM_TOOL_ITERATIONS', 20)
|
||||
|
||||
|
||||
def get_llm_config():
|
||||
"""
|
||||
Get complete LLM configuration for all providers.
|
||||
|
||||
Returns:
|
||||
A dictionary containing configuration for all providers:
|
||||
{
|
||||
'default_provider': str or None,
|
||||
'enabled': bool,
|
||||
'anthropic': {
|
||||
'api_key': str or None,
|
||||
'model': str
|
||||
},
|
||||
'openai': {
|
||||
'api_key': str or None,
|
||||
'model': str
|
||||
},
|
||||
'ollama': {
|
||||
'api_url': str,
|
||||
'model': str
|
||||
},
|
||||
'docker': {
|
||||
'api_url': str,
|
||||
'model': str
|
||||
}
|
||||
}
|
||||
"""
|
||||
return {
|
||||
'default_provider': get_default_provider(),
|
||||
'enabled': is_llm_enabled(),
|
||||
'anthropic': {
|
||||
'api_key': get_anthropic_api_key(),
|
||||
'model': get_anthropic_model()
|
||||
},
|
||||
'openai': {
|
||||
'api_key': get_openai_api_key(),
|
||||
'model': get_openai_model()
|
||||
},
|
||||
'ollama': {
|
||||
'api_url': get_ollama_api_url(),
|
||||
'model': get_ollama_model()
|
||||
},
|
||||
'docker': {
|
||||
'api_url': get_docker_api_url(),
|
||||
'model': get_docker_model()
|
||||
}
|
||||
}
|
||||
@@ -18,7 +18,17 @@ import { getBrowser } from '../../../../static/js/utils';
|
||||
import SaveSharpIcon from '@mui/icons-material/SaveSharp';
|
||||
import CloseIcon from '@mui/icons-material/CloseRounded';
|
||||
import HTMLReactParser from 'html-react-parser/lib/index';
|
||||
import getApiInstance from '../../../../static/js/api_instance';
|
||||
|
||||
// Cache for dynamically loaded options to avoid repeated API calls
|
||||
const optionsCache = {};
|
||||
|
||||
// Clear cache for a specific endpoint (used when refresh button updates options)
|
||||
export function clearOptionsCache(endpoint) {
|
||||
if (endpoint) {
|
||||
delete optionsCache[endpoint];
|
||||
}
|
||||
}
|
||||
|
||||
export async function reloadPgAdmin() {
|
||||
const { name: browser } = getBrowser();
|
||||
@@ -95,11 +105,90 @@ export function prepareSubnodeData(node, subNode, nodeData, preferencesStore) {
|
||||
fieldValues[element.id] = element.value;
|
||||
|
||||
if (element.name === 'theme') {
|
||||
// Theme has special handling - process before dynamic options
|
||||
element.type = 'theme';
|
||||
element.options.forEach((opt) => {
|
||||
opt.selected = opt.value === element.value;
|
||||
opt.preview_src = opt.preview_src && url_for('static', { filename: opt.preview_src });
|
||||
});
|
||||
} else if (element.controlProps.optionsRefreshUrl) {
|
||||
// Use select-refresh type when refresh URL is provided
|
||||
element.type = 'select-refresh';
|
||||
|
||||
// Build refreshDeps by looking up IDs for the named dependencies
|
||||
const refreshDepNames = element.controlProps.refreshDepNames || {};
|
||||
const refreshDeps = {};
|
||||
for (const [paramName, prefName] of Object.entries(refreshDepNames)) {
|
||||
// Find the preference with this name in the same subNode
|
||||
const depPref = subNode.preferences.find((p) => p.name === prefName);
|
||||
if (depPref) {
|
||||
refreshDeps[paramName] = depPref.id;
|
||||
}
|
||||
}
|
||||
element.controlProps.refreshDeps = refreshDeps;
|
||||
|
||||
// Also set up initial options loading via optionsUrl
|
||||
if (element.controlProps.optionsUrl) {
|
||||
const optionsEndpoint = element.controlProps.optionsUrl;
|
||||
const staticOptions = element.options || [];
|
||||
element.options = () => {
|
||||
// Check cache first to avoid repeated API calls
|
||||
if (optionsCache[optionsEndpoint]) {
|
||||
return Promise.resolve([...optionsCache[optionsEndpoint], ...staticOptions]);
|
||||
}
|
||||
return new Promise((resolve) => {
|
||||
const api = getApiInstance();
|
||||
const optionsUrl = url_for(optionsEndpoint);
|
||||
api.get(optionsUrl)
|
||||
.then((res) => {
|
||||
if (res.data?.data?.models) {
|
||||
const dynamicOptions = res.data.data.models;
|
||||
// Cache the results
|
||||
optionsCache[optionsEndpoint] = dynamicOptions;
|
||||
resolve([...dynamicOptions, ...staticOptions]);
|
||||
} else {
|
||||
resolve(staticOptions);
|
||||
}
|
||||
})
|
||||
.catch(() => {
|
||||
resolve(staticOptions);
|
||||
});
|
||||
});
|
||||
};
|
||||
}
|
||||
} else if (element.controlProps.optionsUrl) {
|
||||
// Support dynamic options loading via optionsUrl (endpoint name)
|
||||
const optionsEndpoint = element.controlProps.optionsUrl;
|
||||
const staticOptions = element.options || [];
|
||||
// Replace options with a function that fetches from the URL
|
||||
element.options = () => {
|
||||
// Check cache first to avoid repeated API calls
|
||||
if (optionsCache[optionsEndpoint]) {
|
||||
return Promise.resolve([...optionsCache[optionsEndpoint], ...staticOptions]);
|
||||
}
|
||||
return new Promise((resolve) => {
|
||||
const api = getApiInstance();
|
||||
// Use url_for to resolve the endpoint to a proper URL
|
||||
const optionsUrl = url_for(optionsEndpoint);
|
||||
api.get(optionsUrl)
|
||||
.then((res) => {
|
||||
if (res.data?.data?.models) {
|
||||
// Dynamic models loaded successfully
|
||||
const dynamicOptions = res.data.data.models;
|
||||
// Cache the results
|
||||
optionsCache[optionsEndpoint] = dynamicOptions;
|
||||
resolve([...dynamicOptions, ...staticOptions]);
|
||||
} else {
|
||||
// No models in response, use static options
|
||||
resolve(staticOptions);
|
||||
}
|
||||
})
|
||||
.catch(() => {
|
||||
// On error, fall back to static options
|
||||
resolve(staticOptions);
|
||||
});
|
||||
});
|
||||
};
|
||||
}
|
||||
} else if (type === 'keyboardShortcut') {
|
||||
element.type = 'keyboardShortcut';
|
||||
|
||||
1073
web/pgadmin/static/js/Explain/AIInsights.jsx
Normal file
1073
web/pgadmin/static/js/Explain/AIInsights.jsx
Normal file
File diff suppressed because it is too large
Load Diff
@@ -8,14 +8,17 @@
|
||||
//////////////////////////////////////////////////////////////
|
||||
import { Box, Tab, Tabs } from '@mui/material';
|
||||
import { styled } from '@mui/material/styles';
|
||||
import React from 'react';
|
||||
import React, { useState, useEffect } from 'react';
|
||||
import _ from 'lodash';
|
||||
import Graphical from './Graphical';
|
||||
import TabPanel from '../components/TabPanel';
|
||||
import gettext from 'sources/gettext';
|
||||
import url_for from 'sources/url_for';
|
||||
import getApiInstance from '../api_instance';
|
||||
import ImageMapper from './ImageMapper';
|
||||
import Analysis from './Analysis';
|
||||
import ExplainStatistics from './ExplainStatistics';
|
||||
import AIInsights from './AIInsights';
|
||||
import PropTypes from 'prop-types';
|
||||
import EmptyPanelMessage from '../components/EmptyPanelMessage';
|
||||
|
||||
@@ -505,11 +508,31 @@ function parsePlanData(data, ctx) {
|
||||
return retPlan;
|
||||
}
|
||||
|
||||
export default function Explain({plans=[],
|
||||
emptyMessage=gettext('Use the Explain/Explain Analyze button to generate the plan for a query. Alternatively, you can also execute "EXPLAIN (FORMAT JSON) [QUERY]".')
|
||||
export default function Explain({
|
||||
plans=[],
|
||||
emptyMessage=gettext('Use the Explain/Explain Analyze button to generate the plan for a query. Alternatively, you can also execute "EXPLAIN (FORMAT JSON) [QUERY]".'),
|
||||
llmEnabled: llmEnabledProp=false,
|
||||
sql='',
|
||||
transId=null,
|
||||
onInsertSQL=null,
|
||||
}) {
|
||||
|
||||
const [tabValue, setTabValue] = React.useState(0);
|
||||
const [tabValue, setTabValue] = useState(0);
|
||||
const [llmEnabled, setLlmEnabled] = useState(llmEnabledProp);
|
||||
|
||||
// Fetch LLM status independently to handle timing issues
|
||||
useEffect(() => {
|
||||
const api = getApiInstance();
|
||||
api.get(url_for('llm.status'))
|
||||
.then((res) => {
|
||||
if (res.data?.success && res.data?.data?.enabled) {
|
||||
setLlmEnabled(true);
|
||||
}
|
||||
})
|
||||
.catch(() => {
|
||||
// LLM not available - this is fine
|
||||
});
|
||||
}, []);
|
||||
|
||||
let ctx = React.useRef({});
|
||||
let planData = React.useMemo(()=>{
|
||||
@@ -549,9 +572,10 @@ export default function Explain({plans=[],
|
||||
scrollButtons="auto"
|
||||
action={(ref)=>ref?.updateIndicator()}
|
||||
>
|
||||
<Tab label="Graphical" />
|
||||
<Tab label="Analysis" />
|
||||
<Tab label="Statistics" />
|
||||
<Tab label={gettext('Graphical')} />
|
||||
<Tab label={gettext('Analysis')} />
|
||||
<Tab label={gettext('Statistics')} />
|
||||
{llmEnabled && <Tab label={gettext('AI Insights')} />}
|
||||
</Tabs>
|
||||
</Box>
|
||||
<TabPanel value={tabValue} index={0} classNameRoot='Explain-tabPanel'>
|
||||
@@ -563,6 +587,17 @@ export default function Explain({plans=[],
|
||||
<TabPanel value={tabValue} index={2} classNameRoot='Explain-tabPanel'>
|
||||
<ExplainStatistics explainTable={ctx.current.explainTable} />
|
||||
</TabPanel>
|
||||
{llmEnabled && (
|
||||
<TabPanel value={tabValue} index={3} classNameRoot='Explain-tabPanel'>
|
||||
<AIInsights
|
||||
plans={plans}
|
||||
sql={sql}
|
||||
transId={transId}
|
||||
isActive={tabValue === 3}
|
||||
onInsertSQL={onInsertSQL}
|
||||
/>
|
||||
</TabPanel>
|
||||
)}
|
||||
</StyledBox>
|
||||
);
|
||||
}
|
||||
@@ -570,4 +605,8 @@ export default function Explain({plans=[],
|
||||
Explain.propTypes = {
|
||||
plans: PropTypes.array.isRequired,
|
||||
emptyMessage: PropTypes.string,
|
||||
llmEnabled: PropTypes.bool,
|
||||
sql: PropTypes.string,
|
||||
transId: PropTypes.oneOfType([PropTypes.string, PropTypes.number]),
|
||||
onInsertSQL: PropTypes.func,
|
||||
};
|
||||
|
||||
@@ -918,6 +918,8 @@ InputSelectNonSearch.propTypes = {
|
||||
|
||||
export function InputSelect({ref, cid, helpid, onChange, options, readonly = false, value, controlProps = {}, optionsLoaded, optionsReloadBasis, disabled, onError, ...props}) {
|
||||
const [[finalOptions, isLoading], setFinalOptions] = useState([[], true]);
|
||||
// Force options to reload on component remount (each mount gets a new ID)
|
||||
const [mountId] = useState(() => Math.random());
|
||||
const theme = useTheme();
|
||||
|
||||
useWindowSize();
|
||||
@@ -954,12 +956,12 @@ export function InputSelect({ref, cid, helpid, onChange, options, readonly = fal
|
||||
}
|
||||
})
|
||||
.catch((err)=>{
|
||||
let error_msg = err.response.data.errormsg;
|
||||
let error_msg = err?.response?.data?.errormsg || err?.message || 'Unknown error';
|
||||
onError?.(error_msg);
|
||||
setFinalOptions([[], false]);
|
||||
});
|
||||
return () => umounted = true;
|
||||
}, [optionsReloadBasis]);
|
||||
}, [optionsReloadBasis, mountId]);
|
||||
|
||||
/* Apply filter if any */
|
||||
const filteredOptions = (controlProps.filter?.(finalOptions)) || finalOptions;
|
||||
|
||||
@@ -7,48 +7,149 @@
|
||||
//
|
||||
//////////////////////////////////////////////////////////////
|
||||
|
||||
import { useState } from 'react';
|
||||
import { Box} from '@mui/material';
|
||||
import {InputSelect, FormInput} from './FormComponents';
|
||||
import { useState, useContext, useCallback } from 'react';
|
||||
import { Box } from '@mui/material';
|
||||
import { styled } from '@mui/material/styles';
|
||||
import { InputSelect, FormInput } from './FormComponents';
|
||||
import PropTypes from 'prop-types';
|
||||
import CustomPropTypes from '../custom_prop_types';
|
||||
import RefreshIcon from '@mui/icons-material/Refresh';
|
||||
import { PgIconButton } from './Buttons';
|
||||
import getApiInstance from '../api_instance';
|
||||
import url_for from 'sources/url_for';
|
||||
import gettext from 'sources/gettext';
|
||||
import { SchemaStateContext } from '../SchemaView/SchemaState';
|
||||
import { usePgAdmin } from '../PgAdminProvider';
|
||||
import { clearOptionsCache } from '../../../preferences/static/js/components/PreferencesHelper';
|
||||
|
||||
function ChildContent({cid, helpid, onRefreshClick, label, ...props}) {
|
||||
return <Box display="flex" >
|
||||
<Box flexGrow="1">
|
||||
<InputSelect {...props} cid={cid} helpid={helpid} />
|
||||
</Box>
|
||||
<Box>
|
||||
<PgIconButton onClick={onRefreshClick} icon={<RefreshIcon />} title={label||''}/>
|
||||
</Box>
|
||||
</Box>;
|
||||
const StyledBox = styled(Box)(() => ({
|
||||
display: 'flex',
|
||||
alignItems: 'flex-start',
|
||||
'& .SelectRefresh-selectContainer': {
|
||||
flexGrow: 1,
|
||||
},
|
||||
'& .SelectRefresh-buttonContainer': {
|
||||
marginLeft: '4px',
|
||||
'& button': {
|
||||
height: '30px',
|
||||
width: '30px',
|
||||
},
|
||||
},
|
||||
}));
|
||||
|
||||
function ChildContent({ cid, helpid, onRefreshClick, isRefreshing, ...props }) {
|
||||
return (
|
||||
<StyledBox>
|
||||
<Box className="SelectRefresh-selectContainer">
|
||||
<InputSelect {...props} cid={cid} helpid={helpid} />
|
||||
</Box>
|
||||
<Box className="SelectRefresh-buttonContainer">
|
||||
<PgIconButton
|
||||
onClick={onRefreshClick}
|
||||
icon={<RefreshIcon />}
|
||||
title={gettext('Refresh models')}
|
||||
disabled={isRefreshing}
|
||||
/>
|
||||
</Box>
|
||||
</StyledBox>
|
||||
);
|
||||
}
|
||||
|
||||
ChildContent.propTypes = {
|
||||
cid: PropTypes.string,
|
||||
helpid: PropTypes.string,
|
||||
onRefreshClick: PropTypes.func,
|
||||
label: PropTypes.string,
|
||||
isRefreshing: PropTypes.bool,
|
||||
};
|
||||
export function SelectRefresh({ required, className, label, helpMessage, testcid, controlProps, ...props }){
|
||||
const [options, setOptions] = useState([]);
|
||||
const [optionsReloadBasis, setOptionsReloadBasis] = useState(false);
|
||||
const {getOptionsOnRefresh, ...selectControlProps} = controlProps;
|
||||
|
||||
const onRefreshClick = ()=>{
|
||||
getOptionsOnRefresh?.()
|
||||
.then((res)=>{
|
||||
setOptions(res);
|
||||
setOptionsReloadBasis((prevVal)=>!prevVal);
|
||||
});
|
||||
};
|
||||
export function SelectRefresh({ required, className, label, helpMessage, testcid, controlProps, ...props }) {
|
||||
const [optionsState, setOptionsState] = useState({ options: [], reloadBasis: 0 });
|
||||
const [isRefreshing, setIsRefreshing] = useState(false);
|
||||
const schemaState = useContext(SchemaStateContext);
|
||||
const pgAdmin = usePgAdmin();
|
||||
const {
|
||||
getOptionsOnRefresh,
|
||||
optionsRefreshUrl,
|
||||
optionsUrl,
|
||||
refreshDeps,
|
||||
...selectControlProps
|
||||
} = controlProps;
|
||||
|
||||
const onRefreshClick = useCallback(() => {
|
||||
// If we have an optionsRefreshUrl, make a POST request with dependent field values
|
||||
if (optionsRefreshUrl && refreshDeps && schemaState) {
|
||||
setIsRefreshing(true);
|
||||
|
||||
// Build the request body from dependent field values
|
||||
const requestBody = {};
|
||||
for (const [paramName, fieldId] of Object.entries(refreshDeps)) {
|
||||
// Find the field value from schema state
|
||||
// fieldId is the preference ID, we need to look it up in state
|
||||
const fieldValue = schemaState.data?.[fieldId];
|
||||
// Only include non-empty values
|
||||
if (fieldValue !== undefined && fieldValue !== null && fieldValue !== '') {
|
||||
requestBody[paramName] = fieldValue;
|
||||
}
|
||||
}
|
||||
|
||||
const api = getApiInstance();
|
||||
const refreshUrl = url_for(optionsRefreshUrl);
|
||||
|
||||
api.post(refreshUrl, requestBody)
|
||||
.then((res) => {
|
||||
if (res.data?.data?.error) {
|
||||
// Server returned an error message - clear options and show error
|
||||
setOptionsState((prev) => ({ options: [], reloadBasis: prev.reloadBasis + 1 }));
|
||||
pgAdmin.Browser.notifier.error(res.data.data.error);
|
||||
} else if (res.data?.data?.models) {
|
||||
const models = res.data.data.models;
|
||||
// Clear the cache so next time preferences opens, it uses the refreshed data
|
||||
if (optionsUrl) {
|
||||
clearOptionsCache(optionsUrl);
|
||||
}
|
||||
setOptionsState((prev) => ({ options: models, reloadBasis: prev.reloadBasis + 1 }));
|
||||
} else {
|
||||
// No models returned - clear the list
|
||||
setOptionsState((prev) => ({ options: [], reloadBasis: prev.reloadBasis + 1 }));
|
||||
}
|
||||
})
|
||||
.catch((err) => {
|
||||
// Network or other error - clear options and show error
|
||||
setOptionsState((prev) => ({ options: [], reloadBasis: prev.reloadBasis + 1 }));
|
||||
const errMsg = err.response?.data?.errormsg || err.message || gettext('Failed to refresh models');
|
||||
pgAdmin.Browser.notifier.error(errMsg);
|
||||
})
|
||||
.finally(() => {
|
||||
setIsRefreshing(false);
|
||||
});
|
||||
} else if (getOptionsOnRefresh) {
|
||||
// Fall back to the original getOptionsOnRefresh callback
|
||||
setIsRefreshing(true);
|
||||
getOptionsOnRefresh()
|
||||
.then((res) => {
|
||||
setOptionsState((prev) => ({ options: res, reloadBasis: prev.reloadBasis + 1 }));
|
||||
})
|
||||
.catch((err) => {
|
||||
setOptionsState((prev) => ({ options: [], reloadBasis: prev.reloadBasis + 1 }));
|
||||
const errMsg = err.message || gettext('Failed to refresh options');
|
||||
pgAdmin.Browser.notifier.error(errMsg);
|
||||
})
|
||||
.finally(() => {
|
||||
setIsRefreshing(false);
|
||||
});
|
||||
}
|
||||
}, [optionsRefreshUrl, optionsUrl, refreshDeps, schemaState, getOptionsOnRefresh, pgAdmin]);
|
||||
|
||||
return (
|
||||
<FormInput required={required} label={label} className={className} helpMessage={helpMessage} testcid={testcid}>
|
||||
<ChildContent options={options} optionsReloadBasis={optionsReloadBasis}
|
||||
onRefreshClick={onRefreshClick} controlProps={selectControlProps} label={label} {...props} />
|
||||
<ChildContent
|
||||
options={optionsState.options}
|
||||
optionsReloadBasis={optionsState.reloadBasis}
|
||||
onRefreshClick={onRefreshClick}
|
||||
controlProps={selectControlProps}
|
||||
isRefreshing={isRefreshing}
|
||||
{...props}
|
||||
/>
|
||||
</FormInput>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -222,12 +222,79 @@ export class LayoutDocker {
|
||||
loadLayout(savedLayout) {
|
||||
try {
|
||||
this.layoutObj.loadLayout(JSON.parse(savedLayout));
|
||||
this.addMissingDefaultPanels();
|
||||
} catch {
|
||||
/* Fallback to default */
|
||||
this.layoutObj.loadLayout(this.defaultLayout);
|
||||
}
|
||||
}
|
||||
|
||||
addMissingDefaultPanels() {
|
||||
// Flatten both layouts to get all tabs
|
||||
const flattenLayout = (box, arr) => {
|
||||
box.children.forEach((child) => {
|
||||
if (child.children) {
|
||||
flattenLayout(child, arr);
|
||||
} else {
|
||||
arr.push(...(child.tabs ?? []));
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
const flatDefault = [];
|
||||
const flatCurrent = [];
|
||||
flattenLayout(this.defaultLayout.dockbox, flatDefault);
|
||||
flattenLayout(this.layoutObj.getLayout().dockbox, flatCurrent);
|
||||
|
||||
// Find tabs in default but not in saved layout
|
||||
const missingTabs = _.differenceBy(flatDefault, flatCurrent, 'id');
|
||||
|
||||
// Only add non-closable tabs (closable tabs may have been intentionally removed)
|
||||
const missingNonClosableTabs = missingTabs.filter(tab => !tab.internal?.closable);
|
||||
|
||||
// Add each missing tab next to a sibling from its original panel group
|
||||
missingNonClosableTabs.forEach((tab) => {
|
||||
const siblingId = this.findSiblingTab(tab.id, flatDefault, flatCurrent);
|
||||
if (siblingId) {
|
||||
this.openTab({
|
||||
id: tab.id,
|
||||
content: tab.content,
|
||||
...tab.internal
|
||||
}, siblingId, 'middle');
|
||||
} else if (this.resetToTabPanel) {
|
||||
// Fallback: add to the reset panel location
|
||||
this.openTab({
|
||||
id: tab.id,
|
||||
content: tab.content,
|
||||
...tab.internal
|
||||
}, this.resetToTabPanel, 'middle');
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
findSiblingTab(tabId, flatDefault, flatCurrent) {
|
||||
// Find which panel group this tab belongs to in the default layout
|
||||
const findPanelTabs = (box, targetId) => {
|
||||
for (const child of box.children) {
|
||||
if (child.children) {
|
||||
const result = findPanelTabs(child, targetId);
|
||||
if (result) return result;
|
||||
} else if (child.tabs) {
|
||||
const hasTarget = child.tabs.some(t => t.id === targetId);
|
||||
if (hasTarget) return child.tabs.map(t => t.id);
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
const siblingIds = findPanelTabs(this.defaultLayout.dockbox, tabId);
|
||||
if (!siblingIds) return null;
|
||||
|
||||
// Find a sibling that exists in current layout
|
||||
const currentIds = flatCurrent.map(t => t.id);
|
||||
return siblingIds.find(id => id !== tabId && currentIds.includes(id));
|
||||
}
|
||||
|
||||
saveLayout(l) {
|
||||
let api = getApiInstance();
|
||||
if(!this.layoutId || !this.layoutObj) {
|
||||
|
||||
@@ -3,6 +3,7 @@ from .authenticate import blueprint as AuthenticateModule
|
||||
from .browser import blueprint as BrowserModule
|
||||
from .dashboard import blueprint as DashboardModule
|
||||
from .help import blueprint as HelpModule
|
||||
from .llm import blueprint as LLMModule
|
||||
from .misc import blueprint as MiscModule
|
||||
from .preferences import blueprint as PreferencesModule
|
||||
from .redirects import blueprint as RedirectModule
|
||||
@@ -17,6 +18,7 @@ def get_submodules():
|
||||
BrowserModule,
|
||||
DashboardModule,
|
||||
HelpModule,
|
||||
LLMModule,
|
||||
MiscModule,
|
||||
PreferencesModule,
|
||||
RedirectModule,
|
||||
|
||||
@@ -48,6 +48,7 @@ from pgadmin.utils.exception import ConnectionLost, SSHTunnelConnectionLost, \
|
||||
CryptKeyMissing, ObjectGone
|
||||
from pgadmin.browser.utils import underscore_escape
|
||||
from pgadmin.utils.menu import MenuItem
|
||||
from pgadmin.utils.csrf import pgCSRFProtect
|
||||
from pgadmin.utils.sqlautocomplete.autocomplete import SQLAutoComplete
|
||||
from pgadmin.tools.sqleditor.utils.query_tool_preferences import \
|
||||
register_query_tool_preferences
|
||||
@@ -144,6 +145,8 @@ class SqlEditorModule(PgAdminModule):
|
||||
'sqleditor.get_new_connection_role',
|
||||
'sqleditor.connect_server',
|
||||
'sqleditor.server_cursor',
|
||||
'sqleditor.nlq_chat_stream',
|
||||
'sqleditor.explain_analyze_stream',
|
||||
]
|
||||
|
||||
def on_logout(self):
|
||||
@@ -2736,3 +2739,372 @@ def user_macros(json_resp=True):
|
||||
This method is used to fetch all user macros.
|
||||
"""
|
||||
return get_user_macros()
|
||||
|
||||
# =============================================================================
|
||||
# Natural Language Query (NLQ) to SQL
|
||||
# =============================================================================
|
||||
|
||||
|
||||
@blueprint.route(
|
||||
'/nlq/chat/<int:trans_id>/stream',
|
||||
methods=["POST"],
|
||||
endpoint='nlq_chat_stream'
|
||||
)
|
||||
@pgCSRFProtect.exempt
|
||||
@pga_login_required
|
||||
def nlq_chat_stream(trans_id):
|
||||
"""
|
||||
Stream NLQ chat response via Server-Sent Events (SSE).
|
||||
|
||||
This endpoint accepts a natural language query and streams back
|
||||
the generated SQL query along with progress updates.
|
||||
|
||||
Args:
|
||||
trans_id: Transaction ID for the current Query Tool session
|
||||
|
||||
Request Body (JSON):
|
||||
message: The natural language query from the user
|
||||
conversation_id: Optional ID to continue a conversation
|
||||
history: Optional list of previous messages for context
|
||||
|
||||
Returns:
|
||||
SSE stream with events:
|
||||
- {type: "thinking", message: "..."} - Progress updates
|
||||
- {type: "sql", sql: "...", explanation: "..."} - Generated SQL
|
||||
- {type: "complete", sql: "...", explanation: "...",
|
||||
conversation_id: "..."} - Final response
|
||||
- {type: "error", message: "..."} - Error message
|
||||
"""
|
||||
from flask import stream_with_context
|
||||
from pgadmin.llm.utils import is_llm_enabled
|
||||
from pgadmin.llm.chat import chat_with_database
|
||||
from pgadmin.llm.prompts.nlq import NLQ_SYSTEM_PROMPT
|
||||
|
||||
# Check if LLM is configured
|
||||
if not is_llm_enabled():
|
||||
return make_json_response(
|
||||
success=0,
|
||||
errormsg=gettext(
|
||||
'AI features are not configured. Please configure an LLM '
|
||||
'provider in Preferences > AI.'
|
||||
)
|
||||
)
|
||||
|
||||
# Get session data for this transaction
|
||||
status, error_msg, conn, trans_obj, session_obj = \
|
||||
check_transaction_status(trans_id)
|
||||
|
||||
if not status:
|
||||
return make_json_response(
|
||||
success=0,
|
||||
errormsg=error_msg or ERROR_MSG_TRANS_ID_NOT_FOUND
|
||||
)
|
||||
|
||||
if not conn or not trans_obj:
|
||||
return make_json_response(
|
||||
success=0,
|
||||
errormsg=gettext('Database connection not available.')
|
||||
)
|
||||
|
||||
# Parse request data
|
||||
data = request.get_json(silent=True) or {}
|
||||
user_message = data.get('message', '').strip()
|
||||
conversation_id = data.get('conversation_id')
|
||||
|
||||
if not user_message:
|
||||
return make_json_response(
|
||||
success=0,
|
||||
errormsg=gettext('Please provide a message.')
|
||||
)
|
||||
|
||||
def generate():
|
||||
"""Generator for SSE events."""
|
||||
import secrets as py_secrets
|
||||
|
||||
try:
|
||||
# Send thinking status
|
||||
yield _nlq_sse_event({
|
||||
'type': 'thinking',
|
||||
'message': gettext('Analyzing your request...')
|
||||
})
|
||||
|
||||
# Call the LLM with database tools
|
||||
response_text, _ = chat_with_database(
|
||||
user_message=user_message,
|
||||
sid=trans_obj.sid,
|
||||
did=trans_obj.did,
|
||||
system_prompt=NLQ_SYSTEM_PROMPT
|
||||
)
|
||||
|
||||
# Try to parse the response as JSON
|
||||
sql = None
|
||||
explanation = ''
|
||||
|
||||
# First, try to extract JSON from markdown code blocks
|
||||
json_text = response_text.strip()
|
||||
|
||||
# Look for ```json ... ``` blocks
|
||||
json_match = re.search(
|
||||
r'```json\s*\n?(.*?)\n?```',
|
||||
json_text,
|
||||
re.DOTALL
|
||||
)
|
||||
if json_match:
|
||||
json_text = json_match.group(1).strip()
|
||||
else:
|
||||
# Also try to find a plain JSON object in the response
|
||||
# Look for {"sql": ... } pattern anywhere in the text
|
||||
sql_pattern = (
|
||||
r'\{["\']?sql["\']?\s*:\s*'
|
||||
r'(?:null|"[^"]*"|\'[^\']*\').*?\}'
|
||||
)
|
||||
plain_json_match = re.search(sql_pattern, json_text, re.DOTALL)
|
||||
if plain_json_match:
|
||||
json_text = plain_json_match.group(0)
|
||||
|
||||
try:
|
||||
result = json.loads(json_text)
|
||||
sql = result.get('sql')
|
||||
explanation = result.get('explanation', '')
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
# If not valid JSON, try to extract SQL from the response
|
||||
# Look for SQL code blocks first
|
||||
sql_match = re.search(
|
||||
r'```sql\s*\n?(.*?)\n?```',
|
||||
response_text,
|
||||
re.DOTALL
|
||||
)
|
||||
if sql_match:
|
||||
sql = sql_match.group(1).strip()
|
||||
else:
|
||||
# Check for malformed tool call text patterns
|
||||
# Some models output tool calls as text instead of
|
||||
# proper tool use blocks
|
||||
tool_call_match = re.search(
|
||||
r'<function=execute_sql_query>\s*'
|
||||
r'<parameter=query>\s*(.*?)\s*</parameter>',
|
||||
response_text,
|
||||
re.DOTALL
|
||||
)
|
||||
if tool_call_match:
|
||||
sql = tool_call_match.group(1).strip()
|
||||
explanation = gettext(
|
||||
'Generated SQL query from your request.'
|
||||
)
|
||||
else:
|
||||
# No parseable JSON or SQL block found
|
||||
# Treat the response as an explanation/error message
|
||||
explanation = response_text.strip()
|
||||
# Don't set sql - leave it as None
|
||||
|
||||
# Generate a conversation ID if not provided
|
||||
if not conversation_id:
|
||||
new_conversation_id = py_secrets.token_hex(8)
|
||||
else:
|
||||
new_conversation_id = conversation_id
|
||||
|
||||
# Send the final result
|
||||
yield _nlq_sse_event({
|
||||
'type': 'complete',
|
||||
'sql': sql,
|
||||
'explanation': explanation,
|
||||
'conversation_id': new_conversation_id
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
current_app.logger.error(f'NLQ chat error: {str(e)}')
|
||||
yield _nlq_sse_event({
|
||||
'type': 'error',
|
||||
'message': str(e)
|
||||
})
|
||||
|
||||
# Create SSE response
|
||||
response = Response(
|
||||
stream_with_context(generate()),
|
||||
mimetype='text/event-stream',
|
||||
headers={
|
||||
'Cache-Control': 'no-cache, no-store, must-revalidate',
|
||||
'Pragma': 'no-cache',
|
||||
'Expires': '0',
|
||||
'Connection': 'keep-alive',
|
||||
'X-Accel-Buffering': 'no',
|
||||
}
|
||||
)
|
||||
response.direct_passthrough = True
|
||||
return response
|
||||
|
||||
|
||||
def _nlq_sse_event(data: dict) -> bytes:
|
||||
"""Format data as an SSE event with padding for buffer flushing.
|
||||
|
||||
Args:
|
||||
data: Event data dictionary.
|
||||
|
||||
Returns:
|
||||
SSE-formatted bytes.
|
||||
"""
|
||||
json_data = json.dumps(data)
|
||||
# Add padding to help flush buffers in WSGI servers
|
||||
padding_needed = max(0, 2048 - len(json_data) - 20)
|
||||
padding = f": {'.' * padding_needed}\n" if padding_needed > 0 else ""
|
||||
return f"{padding}data: {json_data}\n\n".encode('utf-8')
|
||||
|
||||
|
||||
@blueprint.route(
|
||||
'/explain/analyze/<int:trans_id>/stream',
|
||||
methods=["POST"],
|
||||
endpoint='explain_analyze_stream'
|
||||
)
|
||||
@pgCSRFProtect.exempt
|
||||
@pga_login_required
|
||||
def explain_analyze_stream(trans_id):
|
||||
"""
|
||||
Stream AI analysis of an EXPLAIN plan via Server-Sent Events (SSE).
|
||||
|
||||
This endpoint accepts an EXPLAIN plan JSON and the original SQL query,
|
||||
then streams back AI-generated performance analysis and recommendations.
|
||||
|
||||
Args:
|
||||
trans_id: Transaction ID for the current Query Tool session
|
||||
|
||||
Request Body (JSON):
|
||||
plan: The EXPLAIN plan output (JSON format from PostgreSQL)
|
||||
sql: The original SQL query that was explained
|
||||
|
||||
Returns:
|
||||
SSE stream with events:
|
||||
- {type: "thinking", message: "..."} - Progress updates
|
||||
- {type: "analysis", bottlenecks: [...], recommendations: [...],
|
||||
summary: "..."} - Analysis results
|
||||
- {type: "complete", ...} - Final response with full analysis
|
||||
- {type: "error", message: "..."} - Error message
|
||||
"""
|
||||
from flask import stream_with_context
|
||||
from pgadmin.llm.utils import is_llm_enabled
|
||||
from pgadmin.llm.client import get_llm_client
|
||||
from pgadmin.llm.models import Message
|
||||
from pgadmin.llm.prompts.explain import EXPLAIN_ANALYSIS_PROMPT
|
||||
|
||||
# Check if LLM is configured
|
||||
if not is_llm_enabled():
|
||||
return make_json_response(
|
||||
success=0,
|
||||
errormsg=gettext(
|
||||
'AI features are not configured. Please configure an LLM '
|
||||
'provider in Preferences > AI.'
|
||||
)
|
||||
)
|
||||
|
||||
# Verify transaction exists (for authentication context)
|
||||
status, error_msg, conn, trans_obj, session_obj = \
|
||||
check_transaction_status(trans_id)
|
||||
|
||||
if not status:
|
||||
return make_json_response(
|
||||
success=0,
|
||||
errormsg=error_msg or ERROR_MSG_TRANS_ID_NOT_FOUND
|
||||
)
|
||||
|
||||
# Parse request data
|
||||
data = request.get_json(silent=True) or {}
|
||||
plan_data = data.get('plan')
|
||||
sql_query = data.get('sql', '')
|
||||
|
||||
if not plan_data:
|
||||
return make_json_response(
|
||||
success=0,
|
||||
errormsg=gettext('Please provide an EXPLAIN plan to analyze.')
|
||||
)
|
||||
|
||||
def generate():
|
||||
"""Generator for SSE events."""
|
||||
try:
|
||||
# Send thinking status
|
||||
yield _nlq_sse_event({
|
||||
'type': 'thinking',
|
||||
'message': gettext('Analyzing query plan...')
|
||||
})
|
||||
|
||||
# Format the plan for the LLM
|
||||
plan_json = json.dumps(plan_data, indent=2) if isinstance(
|
||||
plan_data, (dict, list)
|
||||
) else str(plan_data)
|
||||
|
||||
# Build the user message with plan and SQL
|
||||
user_message = f"""Please analyze this PostgreSQL EXPLAIN plan:
|
||||
|
||||
```json
|
||||
{plan_json}
|
||||
```
|
||||
|
||||
Original SQL query:
|
||||
```sql
|
||||
{sql_query}
|
||||
```
|
||||
|
||||
Provide your analysis identifying performance bottlenecks and \
|
||||
optimization recommendations."""
|
||||
|
||||
# Call the LLM
|
||||
client = get_llm_client()
|
||||
response = client.chat(
|
||||
messages=[Message.user(user_message)],
|
||||
system_prompt=EXPLAIN_ANALYSIS_PROMPT
|
||||
)
|
||||
response_text = response.content
|
||||
|
||||
# Parse the response
|
||||
bottlenecks = []
|
||||
recommendations = []
|
||||
summary = ''
|
||||
|
||||
# Try to extract JSON from the response
|
||||
json_text = response_text.strip()
|
||||
|
||||
# Look for ```json ... ``` blocks
|
||||
json_match = re.search(
|
||||
r'```json\s*\n?(.*?)\n?```',
|
||||
json_text,
|
||||
re.DOTALL
|
||||
)
|
||||
if json_match:
|
||||
json_text = json_match.group(1).strip()
|
||||
|
||||
try:
|
||||
result = json.loads(json_text)
|
||||
bottlenecks = result.get('bottlenecks', [])
|
||||
recommendations = result.get('recommendations', [])
|
||||
summary = result.get('summary', '')
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
# If parsing fails, use the raw response as summary
|
||||
summary = response_text.strip()
|
||||
|
||||
# Send the final result
|
||||
yield _nlq_sse_event({
|
||||
'type': 'complete',
|
||||
'bottlenecks': bottlenecks,
|
||||
'recommendations': recommendations,
|
||||
'summary': summary
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
current_app.logger.error(f'Explain analysis error: {str(e)}')
|
||||
yield _nlq_sse_event({
|
||||
'type': 'error',
|
||||
'message': str(e)
|
||||
})
|
||||
|
||||
# Create SSE response
|
||||
response = Response(
|
||||
stream_with_context(generate()),
|
||||
mimetype='text/event-stream',
|
||||
headers={
|
||||
'Cache-Control': 'no-cache, no-store, must-revalidate',
|
||||
'Pragma': 'no-cache',
|
||||
'Expires': '0',
|
||||
'Connection': 'keep-alive',
|
||||
'X-Accel-Buffering': 'no',
|
||||
}
|
||||
)
|
||||
response.direct_passthrough = True
|
||||
return response
|
||||
|
||||
@@ -29,6 +29,7 @@ import { Notifications } from './sections/Notifications';
|
||||
import MacrosDialog from './dialogs/MacrosDialog';
|
||||
import FilterDialog from './dialogs/FilterDialog';
|
||||
import { QueryHistory } from './sections/QueryHistory';
|
||||
import { NLQChatPanel } from './sections/NLQChatPanel';
|
||||
import * as showQueryTool from '../show_query_tool';
|
||||
import * as commonUtils from 'sources/utils';
|
||||
import * as Kerberos from 'pgadmin.authenticate.kerberos';
|
||||
@@ -232,6 +233,7 @@ export default function QueryToolComponent({params, pgWindow, pgAdmin, selectedN
|
||||
tabs: [
|
||||
LayoutDocker.getPanel({id: PANELS.QUERY, title: gettext('Query'), content: <Query onTextSelect={(text) => setSelectedText(text)} setQtStatePartial={setQtStatePartial}/>}),
|
||||
LayoutDocker.getPanel({id: PANELS.HISTORY, title: gettext('Query History'), content: <QueryHistory />}),
|
||||
LayoutDocker.getPanel({id: PANELS.AI_ASSISTANT, title: gettext('AI Assistant'), content: <NLQChatPanel />}),
|
||||
],
|
||||
},
|
||||
{
|
||||
|
||||
@@ -72,6 +72,8 @@ export const QUERY_TOOL_EVENTS = {
|
||||
EDITOR_TOGGLE_CASE: 'EDITOR_TOGGLE_CASE',
|
||||
COPY_TO_EDITOR: 'COPY_TO_EDITOR',
|
||||
|
||||
NLQ_INSERT_SQL: 'NLQ_INSERT_SQL',
|
||||
|
||||
WARN_SAVE_DATA_CLOSE: 'WARN_SAVE_DATA_CLOSE',
|
||||
WARN_SAVE_TEXT_CLOSE: 'WARN_SAVE_TEXT_CLOSE',
|
||||
WARN_TXN_CLOSE: 'WARN_TXN_CLOSE',
|
||||
@@ -115,6 +117,7 @@ export const PANELS = {
|
||||
NOTIFICATIONS: 'id-notifications',
|
||||
HISTORY: 'id-history',
|
||||
GRAPH_VISUALISER: 'id-graph-visualiser',
|
||||
AI_ASSISTANT: 'id-ai-assistant',
|
||||
};
|
||||
|
||||
export const MAX_QUERY_LENGTH = 1000000;
|
||||
|
||||
@@ -0,0 +1,788 @@
|
||||
/////////////////////////////////////////////////////////////
|
||||
//
|
||||
// pgAdmin 4 - PostgreSQL Tools
|
||||
//
|
||||
// Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
// This software is released under the PostgreSQL Licence
|
||||
//
|
||||
//////////////////////////////////////////////////////////////
|
||||
import { useState, useContext, useRef, useEffect, useCallback } from 'react';
|
||||
import { styled } from '@mui/material/styles';
|
||||
import {
|
||||
Box,
|
||||
TextField,
|
||||
IconButton,
|
||||
Paper,
|
||||
Typography,
|
||||
Tooltip,
|
||||
} from '@mui/material';
|
||||
import SendIcon from '@mui/icons-material/Send';
|
||||
import StopIcon from '@mui/icons-material/Stop';
|
||||
import ContentCopyIcon from '@mui/icons-material/ContentCopy';
|
||||
import AddIcon from '@mui/icons-material/Add';
|
||||
import ClearAllIcon from '@mui/icons-material/ClearAll';
|
||||
import AutoFixHighIcon from '@mui/icons-material/AutoFixHigh';
|
||||
import { format as formatSQL } from 'sql-formatter';
|
||||
import gettext from 'sources/gettext';
|
||||
import url_for from 'sources/url_for';
|
||||
import getApiInstance from '../../../../../../static/js/api_instance';
|
||||
import usePreferences from '../../../../../../preferences/static/js/store';
|
||||
import {
|
||||
QueryToolContext,
|
||||
QueryToolEventsContext,
|
||||
} from '../QueryToolComponent';
|
||||
import { PANELS, QUERY_TOOL_EVENTS } from '../QueryToolConstants';
|
||||
import CodeMirror from '../../../../../../static/js/components/ReactCodeMirror';
|
||||
import { PgIconButton, DefaultButton } from '../../../../../../static/js/components/Buttons';
|
||||
import EmptyPanelMessage from '../../../../../../static/js/components/EmptyPanelMessage';
|
||||
import Loader from 'sources/components/Loader';
|
||||
|
||||
// Styled components
|
||||
const ChatContainer = styled('div')(({ theme }) => ({
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
height: '100%',
|
||||
width: '100%',
|
||||
overflow: 'hidden',
|
||||
backgroundColor: theme.palette.background.default,
|
||||
}));
|
||||
|
||||
const HeaderBar = styled('div')(({ theme }) => ({
|
||||
flex: '0 0 auto',
|
||||
display: 'flex',
|
||||
justifyContent: 'space-between',
|
||||
alignItems: 'center',
|
||||
padding: theme.spacing(0.5, 1),
|
||||
backgroundColor: theme.otherVars.editorToolbarBg,
|
||||
borderBottom: `1px solid ${theme.otherVars.borderColor}`,
|
||||
}));
|
||||
|
||||
const MessagesArea = styled('div')(({ theme }) => ({
|
||||
flex: '1 1 0',
|
||||
minHeight: 0,
|
||||
overflow: 'auto',
|
||||
padding: theme.spacing(1),
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
gap: theme.spacing(1),
|
||||
}));
|
||||
|
||||
const MessageBubble = styled(Paper)(({ theme, isuser }) => ({
|
||||
padding: theme.spacing(1, 1.5),
|
||||
maxWidth: '90%',
|
||||
alignSelf: isuser === 'true' ? 'flex-end' : 'flex-start',
|
||||
backgroundColor:
|
||||
isuser === 'true'
|
||||
? theme.palette.primary.main
|
||||
: theme.palette.background.paper,
|
||||
color:
|
||||
isuser === 'true'
|
||||
? theme.palette.primary.contrastText
|
||||
: theme.palette.text.primary,
|
||||
borderRadius: theme.spacing(1.5),
|
||||
wordWrap: 'break-word',
|
||||
overflowWrap: 'break-word',
|
||||
...(isuser !== 'true' && {
|
||||
border: `1px solid ${theme.otherVars.borderColor}`,
|
||||
}),
|
||||
}));
|
||||
|
||||
const SQLPreviewBox = styled(Box)(({ theme }) => ({
|
||||
marginTop: theme.spacing(1),
|
||||
'& .sql-preview-header': {
|
||||
display: 'flex',
|
||||
justifyContent: 'space-between',
|
||||
alignItems: 'center',
|
||||
marginBottom: theme.spacing(0.5),
|
||||
},
|
||||
'& .sql-preview-actions': {
|
||||
display: 'flex',
|
||||
gap: theme.spacing(0.5),
|
||||
},
|
||||
'& .sql-preview-editor': {
|
||||
border: `1px solid ${theme.otherVars.borderColor}`,
|
||||
borderRadius: theme.spacing(0.5),
|
||||
overflow: 'auto',
|
||||
'& .cm-editor': {
|
||||
minHeight: '60px',
|
||||
maxHeight: '250px',
|
||||
},
|
||||
'& .cm-scroller': {
|
||||
overflow: 'auto',
|
||||
},
|
||||
},
|
||||
}));
|
||||
|
||||
const InputArea = styled('div')(({ theme }) => ({
|
||||
flex: '0 0 auto',
|
||||
padding: theme.spacing(1),
|
||||
borderTop: `1px solid ${theme.otherVars.borderColor}`,
|
||||
backgroundColor: theme.otherVars.editorToolbarBg,
|
||||
display: 'flex',
|
||||
gap: theme.spacing(1),
|
||||
alignItems: 'flex-end',
|
||||
}));
|
||||
|
||||
const ThinkingIndicator = styled(Box)(({ theme }) => ({
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
gap: theme.spacing(1),
|
||||
color: theme.palette.text.secondary,
|
||||
}));
|
||||
|
||||
// Message types
|
||||
const MESSAGE_TYPES = {
|
||||
USER: 'user',
|
||||
ASSISTANT: 'assistant',
|
||||
SQL: 'sql',
|
||||
THINKING: 'thinking',
|
||||
ERROR: 'error',
|
||||
};
|
||||
|
||||
// Elephant/PostgreSQL-themed processing messages
|
||||
const THINKING_MESSAGES = [
|
||||
'Consulting the elephant...',
|
||||
'Traversing the B-tree...',
|
||||
'Vacuuming the catalog...',
|
||||
'Analyzing table statistics...',
|
||||
'Joining the herds...',
|
||||
'Indexing the savanna...',
|
||||
'Querying the watering hole...',
|
||||
'Optimizing the plan...',
|
||||
'Warming up the cache...',
|
||||
'Gathering the tuples...',
|
||||
'Scanning the relations...',
|
||||
'Checking constraints...',
|
||||
'Rolling back the peanuts...',
|
||||
'Committing to memory...',
|
||||
'Trumpeting the results...',
|
||||
];
|
||||
|
||||
// Helper function to get a random thinking message
|
||||
function getRandomThinkingMessage() {
|
||||
return THINKING_MESSAGES[Math.floor(Math.random() * THINKING_MESSAGES.length)];
|
||||
}
|
||||
|
||||
// Single chat message component
|
||||
function ChatMessage({ message, onInsertSQL, onReplaceSQL, textColors, cmKey }) {
|
||||
if (message.type === MESSAGE_TYPES.USER) {
|
||||
return (
|
||||
<MessageBubble isuser="true">
|
||||
<Typography variant="body2">{message.content}</Typography>
|
||||
</MessageBubble>
|
||||
);
|
||||
}
|
||||
|
||||
if (message.type === MESSAGE_TYPES.SQL) {
|
||||
return (
|
||||
<MessageBubble isuser="false">
|
||||
{message.explanation && (
|
||||
<Typography variant="body2" gutterBottom>
|
||||
{message.explanation}
|
||||
</Typography>
|
||||
)}
|
||||
<SQLPreviewBox>
|
||||
<Box className="sql-preview-header">
|
||||
<Typography variant="caption" style={{ color: textColors.secondary }}>
|
||||
{gettext('Generated SQL')}
|
||||
</Typography>
|
||||
<Box className="sql-preview-actions">
|
||||
<Tooltip title={gettext('Insert at cursor')}>
|
||||
<IconButton
|
||||
size="small"
|
||||
onClick={() => onInsertSQL(message.sql)}
|
||||
>
|
||||
<AddIcon fontSize="small" />
|
||||
</IconButton>
|
||||
</Tooltip>
|
||||
<Tooltip title={gettext('Replace query')}>
|
||||
<IconButton
|
||||
size="small"
|
||||
onClick={() => onReplaceSQL(message.sql)}
|
||||
>
|
||||
<AutoFixHighIcon fontSize="small" />
|
||||
</IconButton>
|
||||
</Tooltip>
|
||||
<Tooltip title={gettext('Copy to clipboard')}>
|
||||
<IconButton
|
||||
size="small"
|
||||
onClick={() => navigator.clipboard.writeText(message.sql)}
|
||||
>
|
||||
<ContentCopyIcon fontSize="small" />
|
||||
</IconButton>
|
||||
</Tooltip>
|
||||
</Box>
|
||||
</Box>
|
||||
<Box className="sql-preview-editor">
|
||||
<CodeMirror
|
||||
key={`sql-preview-${cmKey}`}
|
||||
value={message.sql}
|
||||
readonly={true}
|
||||
options={{
|
||||
lineNumbers: true,
|
||||
foldGutter: false,
|
||||
mode: 'text/x-pgsql',
|
||||
}}
|
||||
/>
|
||||
</Box>
|
||||
</SQLPreviewBox>
|
||||
</MessageBubble>
|
||||
);
|
||||
}
|
||||
|
||||
if (message.type === MESSAGE_TYPES.THINKING) {
|
||||
return (
|
||||
<MessageBubble isuser="false">
|
||||
<ThinkingIndicator>
|
||||
<Loader
|
||||
message=""
|
||||
style={{ position: 'relative', height: 20, width: 20 }}
|
||||
/>
|
||||
<Typography variant="body2" style={{ color: textColors.secondary }}>
|
||||
{message.content}
|
||||
</Typography>
|
||||
</ThinkingIndicator>
|
||||
</MessageBubble>
|
||||
);
|
||||
}
|
||||
|
||||
if (message.type === MESSAGE_TYPES.ERROR) {
|
||||
return (
|
||||
<MessageBubble
|
||||
isuser="false"
|
||||
sx={{ backgroundColor: 'error.light', borderColor: 'error.main' }}
|
||||
>
|
||||
<Typography variant="body2" color="error.dark">
|
||||
{message.content}
|
||||
</Typography>
|
||||
</MessageBubble>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<MessageBubble isuser="false">
|
||||
<Typography variant="body2">{message.content}</Typography>
|
||||
</MessageBubble>
|
||||
);
|
||||
}
|
||||
|
||||
// Main NLQ Chat Panel
|
||||
export function NLQChatPanel() {
|
||||
const [messages, setMessages] = useState([]);
|
||||
const [inputValue, setInputValue] = useState('');
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
const [conversationId, setConversationId] = useState(null);
|
||||
const [thinkingMessageId, setThinkingMessageId] = useState(null);
|
||||
const [llmInfo, setLlmInfo] = useState({ provider: null, model: null });
|
||||
|
||||
// History navigation state
|
||||
const [queryHistory, setQueryHistory] = useState([]);
|
||||
const [historyIndex, setHistoryIndex] = useState(-1);
|
||||
const [savedInput, setSavedInput] = useState('');
|
||||
|
||||
// Get text colors from the body element to match pgAdmin's theme
|
||||
// The MUI theme may not be synced with pgAdmin's theme in docker tabs
|
||||
const [textColors, setTextColors] = useState({
|
||||
primary: 'inherit',
|
||||
secondary: 'inherit',
|
||||
});
|
||||
|
||||
const messagesEndRef = useRef(null);
|
||||
const abortControllerRef = useRef(null);
|
||||
const readerRef = useRef(null);
|
||||
const stoppedRef = useRef(false);
|
||||
const eventBus = useContext(QueryToolEventsContext);
|
||||
const queryToolCtx = useContext(QueryToolContext);
|
||||
const editorPrefs = usePreferences().getPreferencesForModule('editor');
|
||||
|
||||
// Format SQL using pgAdmin's editor preferences
|
||||
const formatSqlWithPrefs = useCallback((sql) => {
|
||||
if (!sql) return sql;
|
||||
try {
|
||||
const formatPrefs = {
|
||||
language: 'postgresql',
|
||||
keywordCase: editorPrefs.keyword_case === 'capitalize' ? 'preserve' : editorPrefs.keyword_case,
|
||||
identifierCase: editorPrefs.identifier_case === 'capitalize' ? 'preserve' : editorPrefs.identifier_case,
|
||||
dataTypeCase: editorPrefs.data_type_case,
|
||||
functionCase: editorPrefs.function_case,
|
||||
logicalOperatorNewline: editorPrefs.logical_operator_new_line,
|
||||
expressionWidth: editorPrefs.expression_width,
|
||||
linesBetweenQueries: editorPrefs.lines_between_queries,
|
||||
tabWidth: editorPrefs.tab_size,
|
||||
useTabs: !editorPrefs.use_spaces,
|
||||
denseOperators: !editorPrefs.spaces_around_operators,
|
||||
newlineBeforeSemicolon: editorPrefs.new_line_before_semicolon
|
||||
};
|
||||
return formatSQL(sql, formatPrefs);
|
||||
} catch {
|
||||
// If formatting fails, return original SQL
|
||||
return sql;
|
||||
}
|
||||
}, [editorPrefs]);
|
||||
|
||||
// Update text colors from body styles for theme compatibility
|
||||
useEffect(() => {
|
||||
const updateColors = () => {
|
||||
const bodyStyles = window.getComputedStyle(document.body);
|
||||
const primaryColor = bodyStyles.color;
|
||||
|
||||
// For secondary color, create a semi-transparent version of the primary
|
||||
// Use higher opacity (0.85) to ensure readability in light mode
|
||||
const rgbMatch = primaryColor.match(/rgb\((\d+),\s*(\d+),\s*(\d+)\)/);
|
||||
let secondaryColor = primaryColor;
|
||||
if (rgbMatch) {
|
||||
const [, r, g, b] = rgbMatch;
|
||||
secondaryColor = `rgba(${r}, ${g}, ${b}, 0.85)`;
|
||||
}
|
||||
|
||||
setTextColors({
|
||||
primary: primaryColor,
|
||||
secondary: secondaryColor,
|
||||
});
|
||||
};
|
||||
|
||||
updateColors();
|
||||
}, []);
|
||||
|
||||
// Fetch LLM info on mount
|
||||
useEffect(() => {
|
||||
const api = getApiInstance();
|
||||
api.get(url_for('llm.status'))
|
||||
.then((res) => {
|
||||
if (res.data?.success && res.data?.data) {
|
||||
setLlmInfo({
|
||||
provider: res.data.data.provider,
|
||||
model: res.data.data.model
|
||||
});
|
||||
}
|
||||
})
|
||||
.catch(() => {
|
||||
// Ignore errors fetching LLM status
|
||||
});
|
||||
}, []);
|
||||
|
||||
// Auto-scroll to bottom on new messages
|
||||
useEffect(() => {
|
||||
messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
|
||||
}, [messages]);
|
||||
|
||||
// Force CodeMirror re-render when panel becomes visible (fixes tab switching issue)
|
||||
const [cmKey, setCmKey] = useState(0);
|
||||
useEffect(() => {
|
||||
const unregister = eventBus.registerListener(QUERY_TOOL_EVENTS.FOCUS_PANEL, (panelId) => {
|
||||
if (panelId === PANELS.AI_ASSISTANT) {
|
||||
// Increment key to force CodeMirror re-render
|
||||
setCmKey((prev) => prev + 1);
|
||||
}
|
||||
});
|
||||
return () => unregister?.();
|
||||
}, [eventBus]);
|
||||
|
||||
// Cycle through thinking messages while loading
|
||||
useEffect(() => {
|
||||
if (!isLoading || !thinkingMessageId) return;
|
||||
|
||||
const interval = setInterval(() => {
|
||||
const newMessage = getRandomThinkingMessage();
|
||||
setMessages((prev) =>
|
||||
prev.map((m) =>
|
||||
m.id === thinkingMessageId ? { ...m, content: newMessage } : m
|
||||
)
|
||||
);
|
||||
}, 2000); // Change message every 2 seconds
|
||||
|
||||
return () => clearInterval(interval);
|
||||
}, [isLoading, thinkingMessageId]);
|
||||
|
||||
const handleInsertSQL = (sql) => {
|
||||
eventBus.fireEvent(QUERY_TOOL_EVENTS.NLQ_INSERT_SQL, sql);
|
||||
eventBus.fireEvent(QUERY_TOOL_EVENTS.FOCUS_PANEL, PANELS.QUERY);
|
||||
};
|
||||
|
||||
const handleReplaceSQL = (sql) => {
|
||||
eventBus.fireEvent(QUERY_TOOL_EVENTS.EDITOR_SET_SQL, sql);
|
||||
eventBus.fireEvent(QUERY_TOOL_EVENTS.FOCUS_PANEL, PANELS.QUERY);
|
||||
};
|
||||
|
||||
const handleClearConversation = () => {
|
||||
setMessages([]);
|
||||
setConversationId(null);
|
||||
};
|
||||
|
||||
// Stop the current request
|
||||
const handleStop = useCallback(() => {
|
||||
// Mark as stopped so the read loop knows to show stopped message
|
||||
stoppedRef.current = true;
|
||||
// Cancel the active reader first (this actually stops the streaming)
|
||||
if (readerRef.current) {
|
||||
readerRef.current.cancel();
|
||||
readerRef.current = null;
|
||||
}
|
||||
// Then abort the fetch controller
|
||||
if (abortControllerRef.current) {
|
||||
abortControllerRef.current.abort();
|
||||
abortControllerRef.current = null;
|
||||
}
|
||||
}, []);
|
||||
|
||||
// Fetch current LLM provider/model info
|
||||
const fetchLlmInfo = useCallback(async () => {
|
||||
try {
|
||||
const api = getApiInstance();
|
||||
const res = await api.get(url_for('llm.status'));
|
||||
if (res.data?.success && res.data?.data) {
|
||||
setLlmInfo({
|
||||
provider: res.data.data.provider,
|
||||
model: res.data.data.model
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// Ignore errors fetching LLM status
|
||||
}
|
||||
}, []);
|
||||
|
||||
const handleSubmit = async () => {
|
||||
if (!inputValue.trim() || isLoading) return;
|
||||
|
||||
// Reset stopped flag
|
||||
stoppedRef.current = false;
|
||||
|
||||
// Fetch latest LLM provider/model info before submitting
|
||||
fetchLlmInfo();
|
||||
|
||||
const userMessage = inputValue.trim();
|
||||
setInputValue('');
|
||||
|
||||
// Add to query history (avoid duplicates of the last entry)
|
||||
setQueryHistory((prev) => {
|
||||
if (prev.length === 0 || prev[prev.length - 1] !== userMessage) {
|
||||
return [...prev, userMessage];
|
||||
}
|
||||
return prev;
|
||||
});
|
||||
setHistoryIndex(-1);
|
||||
setSavedInput('');
|
||||
|
||||
// Add user message
|
||||
setMessages((prev) => [
|
||||
...prev,
|
||||
{
|
||||
type: MESSAGE_TYPES.USER,
|
||||
content: userMessage,
|
||||
},
|
||||
]);
|
||||
|
||||
// Add thinking indicator with random elephant-themed message
|
||||
const thinkingId = Date.now();
|
||||
setThinkingMessageId(thinkingId);
|
||||
setMessages((prev) => [
|
||||
...prev,
|
||||
{
|
||||
type: MESSAGE_TYPES.THINKING,
|
||||
content: getRandomThinkingMessage(),
|
||||
id: thinkingId,
|
||||
},
|
||||
]);
|
||||
|
||||
setIsLoading(true);
|
||||
|
||||
// Create abort controller with 5 minute timeout
|
||||
const controller = new AbortController();
|
||||
abortControllerRef.current = controller;
|
||||
const timeoutId = setTimeout(() => controller.abort(), 5 * 60 * 1000);
|
||||
|
||||
try {
|
||||
const response = await fetch(
|
||||
url_for('sqleditor.nlq_chat_stream', {
|
||||
trans_id: queryToolCtx.params.trans_id,
|
||||
}),
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
[window.pgAdmin.csrf_token_header]: window.pgAdmin.csrf_token,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
message: userMessage,
|
||||
conversation_id: conversationId,
|
||||
}),
|
||||
signal: controller.signal,
|
||||
}
|
||||
);
|
||||
|
||||
clearTimeout(timeoutId);
|
||||
abortControllerRef.current = null;
|
||||
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}));
|
||||
throw new Error(errorData.errormsg || `HTTP error! status: ${response.status}`);
|
||||
}
|
||||
|
||||
const reader = response.body.getReader();
|
||||
readerRef.current = reader;
|
||||
const decoder = new TextDecoder();
|
||||
let buffer = '';
|
||||
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
|
||||
buffer += decoder.decode(value, { stream: true });
|
||||
const lines = buffer.split('\n');
|
||||
buffer = lines.pop() || '';
|
||||
|
||||
for (const line of lines) {
|
||||
if (line.startsWith('data: ')) {
|
||||
try {
|
||||
const data = JSON.parse(line.slice(6));
|
||||
handleSSEEvent(data, thinkingId);
|
||||
} catch {
|
||||
// Skip malformed JSON
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
readerRef.current = null;
|
||||
|
||||
// Check if user manually stopped
|
||||
if (stoppedRef.current) {
|
||||
setMessages((prev) => [
|
||||
...prev.filter((m) => m.id !== thinkingId),
|
||||
{
|
||||
type: MESSAGE_TYPES.ASSISTANT,
|
||||
content: gettext('Generation stopped.'),
|
||||
},
|
||||
]);
|
||||
}
|
||||
} catch (error) {
|
||||
clearTimeout(timeoutId);
|
||||
abortControllerRef.current = null;
|
||||
readerRef.current = null;
|
||||
// Show appropriate message based on error type
|
||||
if (error.name === 'AbortError') {
|
||||
// Check if this was a user-initiated stop or a timeout
|
||||
if (stoppedRef.current) {
|
||||
// User manually stopped
|
||||
setMessages((prev) => [
|
||||
...prev.filter((m) => m.id !== thinkingId),
|
||||
{
|
||||
type: MESSAGE_TYPES.ASSISTANT,
|
||||
content: gettext('Generation stopped.'),
|
||||
},
|
||||
]);
|
||||
} else {
|
||||
// Timeout occurred
|
||||
setMessages((prev) => [
|
||||
...prev.filter((m) => m.id !== thinkingId),
|
||||
{
|
||||
type: MESSAGE_TYPES.ERROR,
|
||||
content: gettext('Request timed out. The query may be too complex. Please try a simpler request.'),
|
||||
},
|
||||
]);
|
||||
}
|
||||
} else {
|
||||
setMessages((prev) => [
|
||||
...prev.filter((m) => m.id !== thinkingId),
|
||||
{
|
||||
type: MESSAGE_TYPES.ERROR,
|
||||
content: gettext('Failed to generate SQL: ') + error.message,
|
||||
},
|
||||
]);
|
||||
}
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
setThinkingMessageId(null);
|
||||
}
|
||||
};
|
||||
|
||||
const handleSSEEvent = (event, thinkingId) => {
|
||||
switch (event.type) {
|
||||
case 'thinking':
|
||||
setMessages((prev) =>
|
||||
prev.map((m) =>
|
||||
m.id === thinkingId ? { ...m, content: event.message } : m
|
||||
)
|
||||
);
|
||||
break;
|
||||
|
||||
case 'sql':
|
||||
case 'complete':
|
||||
// If sql is null/empty, show as regular assistant message (e.g., clarification questions)
|
||||
if (!event.sql) {
|
||||
setMessages((prev) => [
|
||||
...prev.filter((m) => m.id !== thinkingId),
|
||||
{
|
||||
type: MESSAGE_TYPES.ASSISTANT,
|
||||
content: event.explanation || gettext('I need more information to generate the SQL.'),
|
||||
},
|
||||
]);
|
||||
} else {
|
||||
setMessages((prev) => [
|
||||
...prev.filter((m) => m.id !== thinkingId),
|
||||
{
|
||||
type: MESSAGE_TYPES.SQL,
|
||||
sql: formatSqlWithPrefs(event.sql),
|
||||
explanation: event.explanation,
|
||||
},
|
||||
]);
|
||||
}
|
||||
if (event.conversation_id) {
|
||||
setConversationId(event.conversation_id);
|
||||
}
|
||||
break;
|
||||
|
||||
case 'error':
|
||||
setMessages((prev) => [
|
||||
...prev.filter((m) => m.id !== thinkingId),
|
||||
{
|
||||
type: MESSAGE_TYPES.ERROR,
|
||||
content: event.message,
|
||||
},
|
||||
]);
|
||||
break;
|
||||
}
|
||||
};
|
||||
|
||||
const handleKeyDown = (e) => {
|
||||
if (e.key === 'Enter' && !e.shiftKey) {
|
||||
e.preventDefault();
|
||||
handleSubmit();
|
||||
} else if (e.key === 'ArrowUp' && queryHistory.length > 0) {
|
||||
e.preventDefault();
|
||||
if (historyIndex === -1) {
|
||||
// Starting to navigate history, save current input
|
||||
setSavedInput(inputValue);
|
||||
const newIndex = queryHistory.length - 1;
|
||||
setHistoryIndex(newIndex);
|
||||
setInputValue(queryHistory[newIndex]);
|
||||
} else if (historyIndex > 0) {
|
||||
// Move further back in history
|
||||
const newIndex = historyIndex - 1;
|
||||
setHistoryIndex(newIndex);
|
||||
setInputValue(queryHistory[newIndex]);
|
||||
}
|
||||
} else if (e.key === 'ArrowDown' && historyIndex !== -1) {
|
||||
e.preventDefault();
|
||||
if (historyIndex < queryHistory.length - 1) {
|
||||
// Move forward in history
|
||||
const newIndex = historyIndex + 1;
|
||||
setHistoryIndex(newIndex);
|
||||
setInputValue(queryHistory[newIndex]);
|
||||
} else {
|
||||
// At the end of history, restore saved input
|
||||
setHistoryIndex(-1);
|
||||
setInputValue(savedInput);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// Don't render if not a query tool (e.g., View Data mode)
|
||||
if (!queryToolCtx?.params?.is_query_tool) {
|
||||
return (
|
||||
<EmptyPanelMessage
|
||||
text={gettext('AI Assistant is only available in Query Tool mode.')}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<ChatContainer>
|
||||
<HeaderBar>
|
||||
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
|
||||
<Typography variant="subtitle2">
|
||||
{gettext('AI Assistant')}
|
||||
</Typography>
|
||||
{llmInfo.provider && (
|
||||
<Typography variant="caption" sx={{ color: textColors.secondary }}>
|
||||
({llmInfo.provider}{llmInfo.model ? ` / ${llmInfo.model}` : ''})
|
||||
</Typography>
|
||||
)}
|
||||
</Box>
|
||||
<DefaultButton
|
||||
onClick={handleClearConversation}
|
||||
startIcon={<ClearAllIcon />}
|
||||
>
|
||||
{gettext('Clear')}
|
||||
</DefaultButton>
|
||||
</HeaderBar>
|
||||
|
||||
<MessagesArea>
|
||||
{messages.length === 0 ? (
|
||||
<Box
|
||||
display="flex"
|
||||
alignItems="center"
|
||||
justifyContent="center"
|
||||
height="100%"
|
||||
textAlign="center"
|
||||
p={2}
|
||||
>
|
||||
<Typography variant="body2" style={{ color: textColors.secondary }}>
|
||||
{gettext(
|
||||
'Describe what SQL you need and I\'ll generate it for you. ' +
|
||||
'I can help with SELECT, INSERT, UPDATE, DELETE, and DDL statements.'
|
||||
)}
|
||||
</Typography>
|
||||
</Box>
|
||||
) : (
|
||||
messages.map((msg, idx) => (
|
||||
<ChatMessage
|
||||
key={idx}
|
||||
message={msg}
|
||||
onInsertSQL={handleInsertSQL}
|
||||
onReplaceSQL={handleReplaceSQL}
|
||||
textColors={textColors}
|
||||
cmKey={cmKey}
|
||||
/>
|
||||
))
|
||||
)}
|
||||
<div ref={messagesEndRef} />
|
||||
</MessagesArea>
|
||||
|
||||
<InputArea>
|
||||
<TextField
|
||||
fullWidth
|
||||
multiline
|
||||
minRows={1}
|
||||
maxRows={4}
|
||||
placeholder={gettext('Describe the SQL you need...')}
|
||||
value={inputValue}
|
||||
onChange={(e) => setInputValue(e.target.value)}
|
||||
onKeyDown={handleKeyDown}
|
||||
disabled={isLoading}
|
||||
sx={{
|
||||
flex: 1,
|
||||
minWidth: 0,
|
||||
'& .MuiOutlinedInput-root': {
|
||||
backgroundColor: 'background.paper',
|
||||
alignItems: 'flex-start',
|
||||
padding: '4px 8px',
|
||||
},
|
||||
'& .MuiOutlinedInput-root.Mui-disabled': {
|
||||
backgroundColor: 'transparent',
|
||||
},
|
||||
'& .MuiOutlinedInput-notchedOutline': {
|
||||
borderColor: 'divider',
|
||||
},
|
||||
'& .MuiInputBase-input': {
|
||||
padding: '4px 0',
|
||||
fontSize: '0.875rem',
|
||||
},
|
||||
'& .MuiOutlinedInput-input::placeholder': {
|
||||
color: textColors.secondary,
|
||||
opacity: 1,
|
||||
},
|
||||
}}
|
||||
/>
|
||||
<PgIconButton
|
||||
title={isLoading ? gettext('Stop') : gettext('Send')}
|
||||
icon={isLoading ? <StopIcon /> : <SendIcon />}
|
||||
onClick={isLoading ? handleStop : handleSubmit}
|
||||
disabled={!isLoading && !inputValue.trim()}
|
||||
/>
|
||||
</InputArea>
|
||||
</ChatContainer>
|
||||
);
|
||||
}
|
||||
|
||||
export default NLQChatPanel;
|
||||
@@ -224,6 +224,13 @@ export default function Query({onTextSelect, setQtStatePartial}) {
|
||||
}, 250);
|
||||
});
|
||||
|
||||
eventBus.registerListener(QUERY_TOOL_EVENTS.NLQ_INSERT_SQL, (sql)=>{
|
||||
// Insert SQL at current cursor position
|
||||
const cursorPos = editor.current?.getCursor() || {line: 0, ch: 0};
|
||||
editor.current?.replaceRange(sql, cursorPos);
|
||||
editor.current?.focus();
|
||||
});
|
||||
|
||||
eventBus.registerListener(QUERY_TOOL_EVENTS.EDITOR_SET_SQL, (value, focus=true)=>{
|
||||
focus && editor.current?.focus();
|
||||
editor.current?.setValue(value, !queryToolCtx.params.is_query_tool);
|
||||
|
||||
@@ -834,6 +834,7 @@ export function ResultSet() {
|
||||
const layoutDocker = useContext(LayoutDockerContext);
|
||||
const [loaderText, setLoaderText] = useState('');
|
||||
const [dataOutputQuery,setDataOutputQuery] = useState('');
|
||||
const [llmEnabled, setLlmEnabled] = useState(false);
|
||||
const [queryData, setQueryData] = useState(null);
|
||||
const [rows, setRows] = useState([]);
|
||||
const [columns, setColumns] = useState([]);
|
||||
@@ -925,7 +926,15 @@ export function ResultSet() {
|
||||
layoutDocker.openTab({
|
||||
id: PANELS.EXPLAIN,
|
||||
title: gettext('Explain'),
|
||||
content: <Explain plans={planJson} />,
|
||||
content: <Explain
|
||||
plans={planJson}
|
||||
llmEnabled={llmEnabled}
|
||||
sql={dataOutputQuery}
|
||||
transId={queryToolCtx.params.trans_id}
|
||||
onInsertSQL={(sql) => {
|
||||
eventBus.fireEvent(QUERY_TOOL_EVENTS.EDITOR_SET_SQL, sql, true);
|
||||
}}
|
||||
/>,
|
||||
closable: true,
|
||||
}, PANELS.MESSAGES, 'after-tab', true);
|
||||
},
|
||||
@@ -988,6 +997,19 @@ export function ResultSet() {
|
||||
}
|
||||
};
|
||||
|
||||
// Fetch LLM status on mount
|
||||
useEffect(()=>{
|
||||
api.get(url_for('llm.status'))
|
||||
.then((res)=>{
|
||||
if(res.data?.success && res.data?.data?.enabled) {
|
||||
setLlmEnabled(true);
|
||||
}
|
||||
})
|
||||
.catch(()=>{
|
||||
// LLM not available - this is fine
|
||||
});
|
||||
}, []);
|
||||
|
||||
useEffect(()=>{
|
||||
eventBus.registerListener(QUERY_TOOL_EVENTS.TRIGGER_STOP_EXECUTION, async ()=>{
|
||||
try {
|
||||
|
||||
203
web/pgadmin/tools/sqleditor/tests/test_explain_analyze_ai.py
Normal file
203
web/pgadmin/tools/sqleditor/tests/test_explain_analyze_ai.py
Normal file
@@ -0,0 +1,203 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Tests for the AI-powered EXPLAIN plan analysis endpoint."""
|
||||
|
||||
import json
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
from pgadmin.utils.route import BaseTestGenerator
|
||||
|
||||
|
||||
class ExplainAnalyzeAITestCase(BaseTestGenerator):
|
||||
"""Test cases for EXPLAIN plan AI analysis streaming endpoint"""
|
||||
|
||||
scenarios = [
|
||||
('Explain AI - LLM Disabled', dict(
|
||||
llm_enabled=False,
|
||||
expected_error=True,
|
||||
error_contains='AI features are not configured'
|
||||
)),
|
||||
('Explain AI - Invalid Transaction', dict(
|
||||
llm_enabled=True,
|
||||
valid_transaction=False,
|
||||
expected_error=True,
|
||||
error_contains='Transaction ID'
|
||||
)),
|
||||
('Explain AI - Empty Plan', dict(
|
||||
llm_enabled=True,
|
||||
valid_transaction=True,
|
||||
plan=None,
|
||||
expected_error=True,
|
||||
error_contains='provide an EXPLAIN plan'
|
||||
)),
|
||||
('Explain AI - Success', dict(
|
||||
llm_enabled=True,
|
||||
valid_transaction=True,
|
||||
plan=[{
|
||||
'Plan': {
|
||||
'Node Type': 'Seq Scan',
|
||||
'Relation Name': 'users',
|
||||
'Total Cost': 100.0,
|
||||
'Plan Rows': 1000
|
||||
}
|
||||
}],
|
||||
sql='SELECT * FROM users',
|
||||
expected_error=False,
|
||||
mock_response=json.dumps({
|
||||
'bottlenecks': [{
|
||||
'severity': 'high',
|
||||
'node': 'Seq Scan on users',
|
||||
'issue': 'Sequential scan on large table',
|
||||
'details': 'Consider adding an index'
|
||||
}],
|
||||
'recommendations': [{
|
||||
'priority': 1,
|
||||
'title': 'Add index',
|
||||
'explanation': 'Will improve query performance',
|
||||
'sql': 'CREATE INDEX idx_users ON users (id);'
|
||||
}],
|
||||
'summary': 'Query could benefit from indexing.'
|
||||
})
|
||||
)),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
pass
|
||||
|
||||
def runTest(self):
|
||||
"""Test EXPLAIN analysis endpoint"""
|
||||
trans_id = 12345
|
||||
|
||||
# Build the mock chain
|
||||
patches = []
|
||||
|
||||
# Mock LLM availability (patch where it's imported from)
|
||||
mock_llm_enabled = patch(
|
||||
'pgadmin.llm.utils.is_llm_enabled',
|
||||
return_value=self.llm_enabled
|
||||
)
|
||||
patches.append(mock_llm_enabled)
|
||||
|
||||
# Mock check_transaction_status
|
||||
if hasattr(self, 'valid_transaction') and self.valid_transaction:
|
||||
mock_trans_obj = MagicMock()
|
||||
mock_trans_obj.sid = 1
|
||||
mock_trans_obj.did = 1
|
||||
|
||||
mock_conn = MagicMock()
|
||||
mock_conn.connected.return_value = True
|
||||
|
||||
mock_session = {'sid': 1, 'did': 1}
|
||||
|
||||
mock_check_trans = patch(
|
||||
'pgadmin.tools.sqleditor.check_transaction_status',
|
||||
return_value=(
|
||||
True, None, mock_conn, mock_trans_obj, mock_session
|
||||
)
|
||||
)
|
||||
else:
|
||||
mock_check_trans = patch(
|
||||
'pgadmin.tools.sqleditor.check_transaction_status',
|
||||
return_value=(
|
||||
False, 'Transaction ID not found', None, None, None
|
||||
)
|
||||
)
|
||||
patches.append(mock_check_trans)
|
||||
|
||||
# Mock get_llm_client (the endpoint uses client.chat())
|
||||
if hasattr(self, 'mock_response'):
|
||||
mock_response_obj = MagicMock()
|
||||
mock_response_obj.content = self.mock_response
|
||||
mock_client = MagicMock()
|
||||
mock_client.chat.return_value = mock_response_obj
|
||||
mock_get_client = patch(
|
||||
'pgadmin.llm.client.get_llm_client',
|
||||
return_value=mock_client
|
||||
)
|
||||
patches.append(mock_get_client)
|
||||
|
||||
# Mock CSRF protection
|
||||
mock_csrf = patch(
|
||||
'pgadmin.authenticate.mfa.utils.mfa_required',
|
||||
lambda f: f
|
||||
)
|
||||
patches.append(mock_csrf)
|
||||
|
||||
# Start all patches
|
||||
for p in patches:
|
||||
p.start()
|
||||
|
||||
try:
|
||||
# Build request data
|
||||
request_data = {}
|
||||
if hasattr(self, 'plan'):
|
||||
request_data['plan'] = self.plan
|
||||
if hasattr(self, 'sql'):
|
||||
request_data['sql'] = self.sql
|
||||
|
||||
# Make request
|
||||
response = self.tester.post(
|
||||
f'/sqleditor/explain/analyze/{trans_id}/stream',
|
||||
data=json.dumps(request_data),
|
||||
content_type='application/json',
|
||||
follow_redirects=True
|
||||
)
|
||||
|
||||
if self.expected_error:
|
||||
# For error cases, we expect JSON response
|
||||
if response.status_code == 200 and \
|
||||
response.content_type == 'application/json':
|
||||
data = json.loads(response.data)
|
||||
self.assertFalse(data.get('success', True))
|
||||
if hasattr(self, 'error_contains'):
|
||||
self.assertIn(
|
||||
self.error_contains,
|
||||
data.get('errormsg', '')
|
||||
)
|
||||
else:
|
||||
# For success, we expect SSE stream
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.assertIn('text/event-stream', response.content_type)
|
||||
|
||||
finally:
|
||||
# Stop all patches
|
||||
for p in patches:
|
||||
p.stop()
|
||||
|
||||
def tearDown(self):
|
||||
pass
|
||||
|
||||
|
||||
class ExplainPromptTestCase(BaseTestGenerator):
|
||||
"""Test cases for EXPLAIN analysis system prompt"""
|
||||
|
||||
scenarios = [
|
||||
('Explain Prompt - Import', dict()),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
pass
|
||||
|
||||
def runTest(self):
|
||||
"""Test EXPLAIN analysis system prompt can be imported"""
|
||||
from pgadmin.llm.prompts.explain import EXPLAIN_ANALYSIS_PROMPT
|
||||
|
||||
# Verify prompt is a non-empty string
|
||||
self.assertIsInstance(EXPLAIN_ANALYSIS_PROMPT, str)
|
||||
self.assertGreater(len(EXPLAIN_ANALYSIS_PROMPT), 100)
|
||||
|
||||
# Verify key content is present
|
||||
self.assertIn('PostgreSQL', EXPLAIN_ANALYSIS_PROMPT)
|
||||
self.assertIn('EXPLAIN', EXPLAIN_ANALYSIS_PROMPT)
|
||||
self.assertIn('bottlenecks', EXPLAIN_ANALYSIS_PROMPT)
|
||||
self.assertIn('recommendations', EXPLAIN_ANALYSIS_PROMPT)
|
||||
|
||||
def tearDown(self):
|
||||
pass
|
||||
173
web/pgadmin/tools/sqleditor/tests/test_nlq_chat.py
Normal file
173
web/pgadmin/tools/sqleditor/tests/test_nlq_chat.py
Normal file
@@ -0,0 +1,173 @@
|
||||
##########################################################################
|
||||
#
|
||||
# pgAdmin 4 - PostgreSQL Tools
|
||||
#
|
||||
# Copyright (C) 2013 - 2026, The pgAdmin Development Team
|
||||
# This software is released under the PostgreSQL Licence
|
||||
#
|
||||
##########################################################################
|
||||
|
||||
"""Tests for the NLQ (Natural Language Query) chat endpoint."""
|
||||
|
||||
import json
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
from pgadmin.utils.route import BaseTestGenerator
|
||||
|
||||
|
||||
class NLQChatTestCase(BaseTestGenerator):
|
||||
"""Test cases for NLQ chat streaming endpoint"""
|
||||
|
||||
scenarios = [
|
||||
('NLQ Chat - LLM Disabled', dict(
|
||||
llm_enabled=False,
|
||||
expected_error=True,
|
||||
error_contains='AI features are not configured'
|
||||
)),
|
||||
('NLQ Chat - Invalid Transaction', dict(
|
||||
llm_enabled=True,
|
||||
valid_transaction=False,
|
||||
expected_error=True,
|
||||
error_contains='Transaction ID'
|
||||
)),
|
||||
('NLQ Chat - Empty Message', dict(
|
||||
llm_enabled=True,
|
||||
valid_transaction=True,
|
||||
message='',
|
||||
expected_error=True,
|
||||
error_contains='provide a message'
|
||||
)),
|
||||
('NLQ Chat - Success', dict(
|
||||
llm_enabled=True,
|
||||
valid_transaction=True,
|
||||
message='Find all users',
|
||||
expected_error=False,
|
||||
mock_response=(
|
||||
'{"sql": "SELECT * FROM users;", '
|
||||
'"explanation": "Gets all users"}'
|
||||
)
|
||||
)),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
pass
|
||||
|
||||
def runTest(self):
|
||||
"""Test NLQ chat endpoint"""
|
||||
trans_id = 12345
|
||||
|
||||
# Build the mock chain
|
||||
patches = []
|
||||
|
||||
# Mock LLM availability (patch where it's imported from)
|
||||
mock_llm_enabled = patch(
|
||||
'pgadmin.llm.utils.is_llm_enabled',
|
||||
return_value=self.llm_enabled
|
||||
)
|
||||
patches.append(mock_llm_enabled)
|
||||
|
||||
# Mock check_transaction_status
|
||||
if hasattr(self, 'valid_transaction') and self.valid_transaction:
|
||||
mock_trans_obj = MagicMock()
|
||||
mock_trans_obj.sid = 1
|
||||
mock_trans_obj.did = 1
|
||||
|
||||
mock_conn = MagicMock()
|
||||
mock_conn.connected.return_value = True
|
||||
|
||||
mock_session = {'sid': 1, 'did': 1}
|
||||
|
||||
mock_check_trans = patch(
|
||||
'pgadmin.tools.sqleditor.check_transaction_status',
|
||||
return_value=(
|
||||
True, None, mock_conn, mock_trans_obj, mock_session
|
||||
)
|
||||
)
|
||||
else:
|
||||
mock_check_trans = patch(
|
||||
'pgadmin.tools.sqleditor.check_transaction_status',
|
||||
return_value=(
|
||||
False, 'Transaction ID not found', None, None, None
|
||||
)
|
||||
)
|
||||
patches.append(mock_check_trans)
|
||||
|
||||
# Mock chat_with_database
|
||||
if hasattr(self, 'mock_response'):
|
||||
mock_chat = patch(
|
||||
'pgadmin.llm.chat.chat_with_database',
|
||||
return_value=(self.mock_response, [])
|
||||
)
|
||||
patches.append(mock_chat)
|
||||
|
||||
# Mock CSRF protection
|
||||
mock_csrf = patch(
|
||||
'pgadmin.authenticate.mfa.utils.mfa_required',
|
||||
lambda f: f
|
||||
)
|
||||
patches.append(mock_csrf)
|
||||
|
||||
# Start all patches
|
||||
for p in patches:
|
||||
p.start()
|
||||
|
||||
try:
|
||||
# Make request
|
||||
message = getattr(self, 'message', 'test query')
|
||||
response = self.tester.post(
|
||||
f'/sqleditor/nlq/chat/{trans_id}/stream',
|
||||
data=json.dumps({'message': message}),
|
||||
content_type='application/json',
|
||||
follow_redirects=True
|
||||
)
|
||||
|
||||
if self.expected_error:
|
||||
# For error cases, we expect JSON response
|
||||
if response.status_code == 200 and \
|
||||
response.content_type == 'application/json':
|
||||
data = json.loads(response.data)
|
||||
self.assertFalse(data.get('success', True))
|
||||
if hasattr(self, 'error_contains'):
|
||||
self.assertIn(
|
||||
self.error_contains,
|
||||
data.get('errormsg', '')
|
||||
)
|
||||
else:
|
||||
# For success, we expect SSE stream
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.assertIn('text/event-stream', response.content_type)
|
||||
|
||||
finally:
|
||||
# Stop all patches
|
||||
for p in patches:
|
||||
p.stop()
|
||||
|
||||
def tearDown(self):
|
||||
pass
|
||||
|
||||
|
||||
class NLQSystemPromptTestCase(BaseTestGenerator):
|
||||
"""Test cases for NLQ system prompt"""
|
||||
|
||||
scenarios = [
|
||||
('NLQ Prompt - Import', dict()),
|
||||
]
|
||||
|
||||
def setUp(self):
|
||||
pass
|
||||
|
||||
def runTest(self):
|
||||
"""Test NLQ system prompt can be imported"""
|
||||
from pgadmin.llm.prompts.nlq import NLQ_SYSTEM_PROMPT
|
||||
|
||||
# Verify prompt is a non-empty string
|
||||
self.assertIsInstance(NLQ_SYSTEM_PROMPT, str)
|
||||
self.assertGreater(len(NLQ_SYSTEM_PROMPT), 100)
|
||||
|
||||
# Verify key content is present
|
||||
self.assertIn('PostgreSQL', NLQ_SYSTEM_PROMPT)
|
||||
self.assertIn('SQL', NLQ_SYSTEM_PROMPT)
|
||||
self.assertIn('get_database_schema', NLQ_SYSTEM_PROMPT)
|
||||
|
||||
def tearDown(self):
|
||||
pass
|
||||
@@ -24,6 +24,7 @@ class AllPermissionTypes:
|
||||
tools_maintenance = 'tools_maintenance'
|
||||
tools_schema_diff = 'tools_schema_diff'
|
||||
tools_grant_wizard = 'tools_grant_wizard'
|
||||
tools_ai = 'tools_ai'
|
||||
storage_add_folder = 'storage_add_folder'
|
||||
storage_remove_folder = 'storage_remove_folder'
|
||||
change_password = 'change_password'
|
||||
@@ -110,6 +111,11 @@ class PgAdminPermissions:
|
||||
AllPermissionTypes.tools_erd_tool,
|
||||
gettext("ERD Tool")
|
||||
)
|
||||
self.add_permission(
|
||||
AllPermissionCategories.tools,
|
||||
AllPermissionTypes.tools_ai,
|
||||
gettext("AI Reports")
|
||||
)
|
||||
self.add_permission(
|
||||
AllPermissionCategories.storage_manager,
|
||||
AllPermissionTypes.storage_add_folder,
|
||||
|
||||
@@ -32,6 +32,7 @@ PREF_LABEL_REFRESH_RATES = gettext('Refresh rates')
|
||||
PREF_LABEL_GRAPH_VISUALISER = gettext('Graph Visualiser')
|
||||
PREF_LABEL_USER_INTERFACE = gettext('User Interface')
|
||||
PREF_LABEL_FILE_DOWNLOADS = gettext('File Downloads')
|
||||
PREF_LABEL_AI = gettext('AI')
|
||||
|
||||
PGADMIN_STRING_SEPARATOR = '_$PGADMIN$_'
|
||||
PGADMIN_NODE = 'pgadmin.node.%s'
|
||||
|
||||
@@ -94,10 +94,10 @@ class KeyboardShortcutFeatureTest(BaseFeatureTest):
|
||||
self.page.click_tab("Preferences")
|
||||
|
||||
# Wait till the preference dialogue box is displayed by checking the
|
||||
# visibility of Show System Object label
|
||||
# visibility of the preference tree
|
||||
wait.until(EC.presence_of_element_located(
|
||||
(By.XPATH,
|
||||
PreferencesLocaltors.show_system_objects_pref_label_xpath))
|
||||
PreferencesLocaltors.tree_container_xpath))
|
||||
)
|
||||
|
||||
keyboard_node = self.page.find_by_xpath(
|
||||
|
||||
@@ -258,10 +258,10 @@ class PGUtilitiesBackupFeatureTest(BaseFeatureTest):
|
||||
self.page.click_tab("Preferences")
|
||||
|
||||
# Wait till the preference dialogue box is displayed by checking the
|
||||
# visibility of Show System Object label
|
||||
# visibility of the preference tree
|
||||
wait.until(EC.presence_of_element_located(
|
||||
(By.XPATH, PreferencesLocaltors.
|
||||
show_system_objects_pref_label_xpath))
|
||||
(By.XPATH,
|
||||
PreferencesLocaltors.tree_container_xpath))
|
||||
)
|
||||
|
||||
binary_path = self.page.find_by_xpath(
|
||||
|
||||
@@ -105,10 +105,10 @@ class CopySQLFeatureTest(BaseFeatureTest):
|
||||
self.page.click_tab("Preferences")
|
||||
|
||||
# Wait till the preference dialogue box is displayed by checking the
|
||||
# visibility of Show System Object label
|
||||
# visibility of the preference tree
|
||||
wait.until(EC.presence_of_element_located(
|
||||
(By.XPATH,
|
||||
PreferencesLocaltors.show_system_objects_pref_label_xpath))
|
||||
PreferencesLocaltors.tree_container_xpath))
|
||||
)
|
||||
|
||||
option_node = self.page.find_by_xpath(
|
||||
|
||||
@@ -116,6 +116,8 @@ class PreferencesLocaltors:
|
||||
show_system_objects_pref_label_xpath = \
|
||||
"//label[contains(text(), 'Show system objects?')]"
|
||||
|
||||
tree_container_xpath = "//*[@id='treeContainer']"
|
||||
|
||||
specified_preference_tree_node_xpath = \
|
||||
("//*[@id='treeContainer']//div[contains(@class,'PgTree-nodeLabel')]"
|
||||
"[text()='{0}']")
|
||||
|
||||
220
web/regression/javascript/Explain/AIInsights.spec.js
Normal file
220
web/regression/javascript/Explain/AIInsights.spec.js
Normal file
@@ -0,0 +1,220 @@
|
||||
/////////////////////////////////////////////////////////////
|
||||
//
|
||||
// pgAdmin 4 - PostgreSQL Tools
|
||||
//
|
||||
// Copyright (C) 2013 - 2025, The pgAdmin Development Team
|
||||
// This software is released under the PostgreSQL Licence
|
||||
//
|
||||
//////////////////////////////////////////////////////////////
|
||||
|
||||
import { render, screen, waitFor } from '@testing-library/react';
|
||||
import '@testing-library/jest-dom';
|
||||
import { withTheme } from '../fake_theme';
|
||||
import AIInsights from '../../../pgadmin/static/js/Explain/AIInsights';
|
||||
|
||||
// Mock url_for
|
||||
jest.mock('sources/url_for', () => ({
|
||||
__esModule: true,
|
||||
default: jest.fn((endpoint) => `/mock/${endpoint}`),
|
||||
}));
|
||||
|
||||
// Mock gettext
|
||||
jest.mock('sources/gettext', () => ({
|
||||
__esModule: true,
|
||||
default: jest.fn((str) => str),
|
||||
}));
|
||||
|
||||
// Mock the Loader component
|
||||
jest.mock('../../../pgadmin/static/js/components/Loader', () => ({
|
||||
__esModule: true,
|
||||
default: () => <div data-testid="loader">Loading...</div>,
|
||||
}));
|
||||
|
||||
// Mock EmptyPanelMessage
|
||||
jest.mock('../../../pgadmin/static/js/components/EmptyPanelMessage', () => ({
|
||||
__esModule: true,
|
||||
default: ({ text }) => <div data-testid="empty-message">{text}</div>,
|
||||
}));
|
||||
|
||||
describe('AIInsights Component', () => {
|
||||
let ThemedAIInsights;
|
||||
|
||||
const mockPlans = [{
|
||||
Plan: {
|
||||
'Node Type': 'Seq Scan',
|
||||
'Relation Name': 'users',
|
||||
'Total Cost': 100.0,
|
||||
'Plan Rows': 1000,
|
||||
},
|
||||
}];
|
||||
|
||||
beforeAll(() => {
|
||||
ThemedAIInsights = withTheme(AIInsights);
|
||||
|
||||
// Mock fetch for SSE
|
||||
global.fetch = jest.fn();
|
||||
|
||||
// Mock window.getComputedStyle
|
||||
window.getComputedStyle = jest.fn().mockReturnValue({
|
||||
color: 'rgb(0, 0, 0)',
|
||||
});
|
||||
|
||||
// Mock clipboard API
|
||||
Object.assign(navigator, {
|
||||
clipboard: {
|
||||
writeText: jest.fn(),
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
jest.clearAllMocks();
|
||||
});
|
||||
|
||||
it('should show empty message when no plans provided', () => {
|
||||
render(<ThemedAIInsights plans={null} isActive={true} />);
|
||||
expect(screen.getByTestId('empty-message')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should show idle state with analyze button when plans provided but not active', () => {
|
||||
render(
|
||||
<ThemedAIInsights
|
||||
plans={mockPlans}
|
||||
sql="SELECT * FROM users"
|
||||
transId={12345}
|
||||
isActive={false}
|
||||
/>
|
||||
);
|
||||
// Component should be in idle state when not active
|
||||
expect(screen.getByText('Analyze')).toBeInTheDocument();
|
||||
expect(screen.getByText(/Click Analyze to get AI-powered insights/i)).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should start analysis when tab becomes active', async () => {
|
||||
const mockReader = {
|
||||
read: jest.fn()
|
||||
.mockResolvedValueOnce({
|
||||
done: false,
|
||||
value: new TextEncoder().encode('data: {"type":"thinking","message":"Analyzing..."}\n\n'),
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
done: false,
|
||||
value: new TextEncoder().encode('data: {"type":"complete","bottlenecks":[],"recommendations":[],"summary":"Plan looks good"}\n\n'),
|
||||
})
|
||||
.mockResolvedValueOnce({ done: true }),
|
||||
};
|
||||
|
||||
global.fetch.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
body: {
|
||||
getReader: () => mockReader,
|
||||
},
|
||||
});
|
||||
|
||||
const { rerender } = render(
|
||||
<ThemedAIInsights
|
||||
plans={mockPlans}
|
||||
sql="SELECT * FROM users"
|
||||
transId={12345}
|
||||
isActive={false}
|
||||
/>
|
||||
);
|
||||
|
||||
// Rerender with isActive=true to trigger analysis
|
||||
rerender(
|
||||
<ThemedAIInsights
|
||||
plans={mockPlans}
|
||||
sql="SELECT * FROM users"
|
||||
transId={12345}
|
||||
isActive={true}
|
||||
/>
|
||||
);
|
||||
|
||||
// Wait for the analysis to complete
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Plan looks good')).toBeInTheDocument();
|
||||
}, { timeout: 3000 });
|
||||
});
|
||||
|
||||
it('should display bottlenecks when present', async () => {
|
||||
const mockReader = {
|
||||
read: jest.fn()
|
||||
.mockResolvedValueOnce({
|
||||
done: false,
|
||||
value: new TextEncoder().encode('data: {"type":"complete","bottlenecks":[{"severity":"high","node":"Seq Scan on users","issue":"Sequential scan","details":"Consider index"}],"recommendations":[],"summary":"Found issues"}\n\n'),
|
||||
})
|
||||
.mockResolvedValueOnce({ done: true }),
|
||||
};
|
||||
|
||||
global.fetch.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
body: {
|
||||
getReader: () => mockReader,
|
||||
},
|
||||
});
|
||||
|
||||
render(
|
||||
<ThemedAIInsights
|
||||
plans={mockPlans}
|
||||
sql="SELECT * FROM users"
|
||||
transId={12345}
|
||||
isActive={true}
|
||||
/>
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Performance Bottlenecks')).toBeInTheDocument();
|
||||
expect(screen.getByText('Seq Scan on users')).toBeInTheDocument();
|
||||
}, { timeout: 3000 });
|
||||
});
|
||||
|
||||
it('should display recommendations with SQL when present', async () => {
|
||||
const mockReader = {
|
||||
read: jest.fn()
|
||||
.mockResolvedValueOnce({
|
||||
done: false,
|
||||
value: new TextEncoder().encode('data: {"type":"complete","bottlenecks":[],"recommendations":[{"priority":1,"title":"Create index on users","explanation":"Will help performance","sql":"CREATE INDEX idx ON users(id);"}],"summary":"Consider adding an index"}\n\n'),
|
||||
})
|
||||
.mockResolvedValueOnce({ done: true }),
|
||||
};
|
||||
|
||||
global.fetch.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
body: {
|
||||
getReader: () => mockReader,
|
||||
},
|
||||
});
|
||||
|
||||
render(
|
||||
<ThemedAIInsights
|
||||
plans={mockPlans}
|
||||
sql="SELECT * FROM users"
|
||||
transId={12345}
|
||||
isActive={true}
|
||||
/>
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Recommendations')).toBeInTheDocument();
|
||||
expect(screen.getByText('Create index on users')).toBeInTheDocument();
|
||||
expect(screen.getByText('CREATE INDEX idx ON users(id);')).toBeInTheDocument();
|
||||
}, { timeout: 3000 });
|
||||
});
|
||||
|
||||
it('should show error state on failure', async () => {
|
||||
global.fetch.mockRejectedValueOnce(new Error('Network error'));
|
||||
|
||||
render(
|
||||
<ThemedAIInsights
|
||||
plans={mockPlans}
|
||||
sql="SELECT * FROM users"
|
||||
transId={12345}
|
||||
isActive={true}
|
||||
/>
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Network error')).toBeInTheDocument();
|
||||
}, { timeout: 3000 });
|
||||
});
|
||||
});
|
||||
297
web/regression/javascript/llm/AIReport.spec.js
Normal file
297
web/regression/javascript/llm/AIReport.spec.js
Normal file
@@ -0,0 +1,297 @@
|
||||
/////////////////////////////////////////////////////////////
|
||||
//
|
||||
// pgAdmin 4 - PostgreSQL Tools
|
||||
//
|
||||
// Copyright (C) 2013 - 2025, The pgAdmin Development Team
|
||||
// This software is released under the PostgreSQL Licence
|
||||
//
|
||||
//////////////////////////////////////////////////////////////
|
||||
|
||||
import { render, screen, waitFor, fireEvent } from '@testing-library/react';
|
||||
import '@testing-library/jest-dom';
|
||||
import { withTheme } from '../fake_theme';
|
||||
import AIReport from '../../../pgadmin/llm/static/js/AIReport.jsx';
|
||||
|
||||
describe('AIReport Component', () => {
|
||||
let ThemedAIReport;
|
||||
|
||||
beforeAll(() => {
|
||||
ThemedAIReport = withTheme(AIReport);
|
||||
|
||||
// Mock window.getComputedStyle for dark mode detection
|
||||
window.getComputedStyle = jest.fn().mockReturnValue({
|
||||
color: 'rgb(212, 212, 212)',
|
||||
backgroundColor: 'rgb(30, 30, 30)'
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
jest.clearAllMocks();
|
||||
});
|
||||
|
||||
it('should render without crashing', () => {
|
||||
const { container } = render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
expect(container).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should show regenerate and download buttons', () => {
|
||||
render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
expect(screen.getByText('Regenerate')).toBeInTheDocument();
|
||||
expect(screen.getByText('Download')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should disable download button when no report exists', () => {
|
||||
render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
const downloadButton = screen.getByText('Download').closest('button');
|
||||
expect(downloadButton).toBeDisabled();
|
||||
});
|
||||
|
||||
it('should detect dark mode from body styles', async () => {
|
||||
render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
// Wait for dark mode detection to run
|
||||
await waitFor(() => {
|
||||
// The component should apply light colors in dark mode
|
||||
// This would be verified by checking computed styles
|
||||
}, { timeout: 1500 });
|
||||
});
|
||||
|
||||
it('should handle light mode correctly', async () => {
|
||||
// Mock light mode
|
||||
window.getComputedStyle = jest.fn().mockReturnValue({
|
||||
color: 'rgb(0, 0, 0)',
|
||||
backgroundColor: 'rgb(255, 255, 255)'
|
||||
});
|
||||
|
||||
render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
// Component should apply dark colors in light mode
|
||||
}, { timeout: 1500 });
|
||||
});
|
||||
|
||||
it('should handle report generation error gracefully', async () => {
|
||||
// Mock fetch to return error
|
||||
global.fetch = jest.fn().mockRejectedValue(new Error('API Error'));
|
||||
|
||||
render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
const regenerateButton = screen.getByText('Regenerate');
|
||||
fireEvent.click(regenerateButton);
|
||||
|
||||
await waitFor(() => {
|
||||
// Should show error message
|
||||
// expect(screen.getByText(/error/i)).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
it('should display progress during report generation', async () => {
|
||||
// Mock SSE EventSource
|
||||
const mockEventSource = {
|
||||
addEventListener: jest.fn(),
|
||||
close: jest.fn(),
|
||||
onerror: null
|
||||
};
|
||||
|
||||
global.EventSource = jest.fn(() => mockEventSource);
|
||||
|
||||
render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
const regenerateButton = screen.getByText('Regenerate');
|
||||
fireEvent.click(regenerateButton);
|
||||
|
||||
// Simulate SSE progress event
|
||||
const onMessage = mockEventSource.addEventListener.mock.calls.find(
|
||||
call => call[0] === 'message'
|
||||
)?.[1];
|
||||
|
||||
if (onMessage) {
|
||||
onMessage({
|
||||
data: JSON.stringify({
|
||||
type: 'progress',
|
||||
stage: 'analyzing',
|
||||
message: 'Analyzing database structure...',
|
||||
completed: 1,
|
||||
total: 5
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
await waitFor(() => {
|
||||
// Progress should be visible
|
||||
// expect(screen.getByText(/analyzing/i)).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
it('should support all report categories', () => {
|
||||
const categories = ['security', 'performance', 'design'];
|
||||
|
||||
categories.forEach(category => {
|
||||
const { unmount } = render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory={category}
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
expect(screen.getByText('Regenerate')).toBeInTheDocument();
|
||||
unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it('should support all report types', () => {
|
||||
const types = [
|
||||
{ type: 'server', props: { sid: 1, serverName: 'Test' } },
|
||||
{ type: 'database', props: { sid: 1, did: 5, serverName: 'Test', databaseName: 'TestDB' } },
|
||||
{ type: 'schema', props: { sid: 1, did: 5, scid: 10, serverName: 'Test', databaseName: 'TestDB', schemaName: 'public' } }
|
||||
];
|
||||
|
||||
types.forEach(({ type, props }) => {
|
||||
const { unmount } = render(
|
||||
<ThemedAIReport
|
||||
reportCategory="security"
|
||||
reportType={type}
|
||||
{...props}
|
||||
/>
|
||||
);
|
||||
|
||||
expect(screen.getByText('Regenerate')).toBeInTheDocument();
|
||||
unmount();
|
||||
});
|
||||
});
|
||||
|
||||
it('should render markdown content correctly', () => {
|
||||
render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
// Would need to simulate report completion and verify markdown rendering
|
||||
});
|
||||
|
||||
it('should handle download functionality', () => {
|
||||
// Mock URL.createObjectURL
|
||||
global.URL.createObjectURL = jest.fn(() => 'blob:mock-url');
|
||||
global.URL.revokeObjectURL = jest.fn();
|
||||
|
||||
// Mock document.createElement for download link
|
||||
const mockLink = {
|
||||
click: jest.fn(),
|
||||
setAttribute: jest.fn()
|
||||
};
|
||||
const createElementSpy = jest.spyOn(document, 'createElement').mockReturnValue(mockLink);
|
||||
const appendChildSpy = jest.spyOn(document.body, 'appendChild').mockImplementation(() => {});
|
||||
const removeChildSpy = jest.spyOn(document.body, 'removeChild').mockImplementation(() => {});
|
||||
|
||||
// Test would simulate having a report and clicking download
|
||||
|
||||
// Restore document mocks
|
||||
createElementSpy.mockRestore();
|
||||
appendChildSpy.mockRestore();
|
||||
removeChildSpy.mockRestore();
|
||||
});
|
||||
|
||||
it('should close EventSource on component unmount', () => {
|
||||
const mockEventSource = {
|
||||
addEventListener: jest.fn(),
|
||||
close: jest.fn(),
|
||||
onerror: null
|
||||
};
|
||||
|
||||
global.EventSource = jest.fn(() => mockEventSource);
|
||||
|
||||
const { unmount } = render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
unmount();
|
||||
|
||||
// EventSource should be closed on unmount
|
||||
// Would verify mockEventSource.close was called
|
||||
});
|
||||
|
||||
it('should update text colors when theme changes', async () => {
|
||||
render(
|
||||
<ThemedAIReport
|
||||
sid={1}
|
||||
reportCategory="security"
|
||||
reportType="server"
|
||||
serverName="TestServer"
|
||||
/>
|
||||
);
|
||||
|
||||
// Change theme
|
||||
window.getComputedStyle = jest.fn().mockReturnValue({
|
||||
color: 'rgb(255, 255, 255)',
|
||||
backgroundColor: 'rgb(0, 0, 0)'
|
||||
});
|
||||
|
||||
// Wait for theme detection interval
|
||||
await waitFor(() => {
|
||||
// Colors should update
|
||||
}, { timeout: 1500 });
|
||||
});
|
||||
});
|
||||
181
web/regression/javascript/sqleditor/NLQChatPanel.spec.js
Normal file
181
web/regression/javascript/sqleditor/NLQChatPanel.spec.js
Normal file
@@ -0,0 +1,181 @@
|
||||
/////////////////////////////////////////////////////////////
|
||||
//
|
||||
// pgAdmin 4 - PostgreSQL Tools
|
||||
//
|
||||
// Copyright (C) 2013 - 2025, The pgAdmin Development Team
|
||||
// This software is released under the PostgreSQL Licence
|
||||
//
|
||||
//////////////////////////////////////////////////////////////
|
||||
|
||||
// Mock url_for
|
||||
jest.mock('sources/url_for', () => ({
|
||||
__esModule: true,
|
||||
default: jest.fn((endpoint) => `/mock/${endpoint}`),
|
||||
}));
|
||||
|
||||
// Mock preferences store
|
||||
jest.mock('../../../pgadmin/preferences/static/js/store', () => ({
|
||||
__esModule: true,
|
||||
default: jest.fn(() => ({
|
||||
getPreferencesForModule: jest.fn(() => ({})),
|
||||
})),
|
||||
}));
|
||||
|
||||
// Mock the QueryToolComponent to avoid importing all its dependencies
|
||||
jest.mock('../../../pgadmin/tools/sqleditor/static/js/components/QueryToolComponent.jsx', () => {
|
||||
const React = require('react');
|
||||
return {
|
||||
QueryToolContext: React.createContext(null),
|
||||
QueryToolEventsContext: React.createContext(null),
|
||||
};
|
||||
});
|
||||
|
||||
// Mock CodeMirror
|
||||
jest.mock('../../../pgadmin/static/js/components/ReactCodeMirror', () => ({
|
||||
__esModule: true,
|
||||
default: ({ value }) => <pre data-testid="codemirror">{value}</pre>,
|
||||
}));
|
||||
|
||||
// Mock EmptyPanelMessage
|
||||
jest.mock('../../../pgadmin/static/js/components/EmptyPanelMessage', () => ({
|
||||
__esModule: true,
|
||||
default: ({ text }) => <div data-testid="empty-message">{text}</div>,
|
||||
}));
|
||||
|
||||
// Mock Loader
|
||||
jest.mock('sources/components/Loader', () => ({
|
||||
__esModule: true,
|
||||
default: () => <div data-testid="loader">Loading...</div>,
|
||||
}));
|
||||
|
||||
import { render, screen, fireEvent } from '@testing-library/react';
|
||||
import '@testing-library/jest-dom';
|
||||
import { withTheme } from '../fake_theme';
|
||||
import { NLQChatPanel } from '../../../pgadmin/tools/sqleditor/static/js/components/sections/NLQChatPanel.jsx';
|
||||
import {
|
||||
QueryToolContext,
|
||||
QueryToolEventsContext,
|
||||
} from '../../../pgadmin/tools/sqleditor/static/js/components/QueryToolComponent.jsx';
|
||||
|
||||
// Mock the EventBus
|
||||
const createMockEventBus = () => ({
|
||||
fireEvent: jest.fn(),
|
||||
registerListener: jest.fn(),
|
||||
});
|
||||
|
||||
// Mock the QueryToolContext
|
||||
const createMockQueryToolCtx = (isQueryTool = true) => ({
|
||||
params: {
|
||||
trans_id: 12345,
|
||||
is_query_tool: isQueryTool,
|
||||
},
|
||||
api: {
|
||||
post: jest.fn(),
|
||||
get: jest.fn(),
|
||||
},
|
||||
});
|
||||
|
||||
// Helper to render with contexts
|
||||
const renderWithContexts = (component, { queryToolCtx, eventBus } = {}) => {
|
||||
const mockEventBus = eventBus || createMockEventBus();
|
||||
const mockQueryToolCtx = queryToolCtx || createMockQueryToolCtx();
|
||||
|
||||
return render(
|
||||
<QueryToolContext.Provider value={mockQueryToolCtx}>
|
||||
<QueryToolEventsContext.Provider value={mockEventBus}>
|
||||
{component}
|
||||
</QueryToolEventsContext.Provider>
|
||||
</QueryToolContext.Provider>
|
||||
);
|
||||
};
|
||||
|
||||
describe('NLQChatPanel Component', () => {
|
||||
let ThemedNLQChatPanel;
|
||||
|
||||
beforeAll(() => {
|
||||
ThemedNLQChatPanel = withTheme(NLQChatPanel);
|
||||
|
||||
// Mock fetch for SSE
|
||||
global.fetch = jest.fn();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
jest.clearAllMocks();
|
||||
});
|
||||
|
||||
it('should render without crashing', () => {
|
||||
const { container } = renderWithContexts(<ThemedNLQChatPanel />);
|
||||
expect(container).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should show AI Assistant header', () => {
|
||||
renderWithContexts(<ThemedNLQChatPanel />);
|
||||
expect(screen.getByText('AI Assistant')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should show empty state message when no messages', () => {
|
||||
renderWithContexts(<ThemedNLQChatPanel />);
|
||||
expect(
|
||||
screen.getByText(/Describe what SQL you need/i)
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should have input field for typing queries', () => {
|
||||
renderWithContexts(<ThemedNLQChatPanel />);
|
||||
const input = screen.getByPlaceholderText(/Describe the SQL you need/i);
|
||||
expect(input).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should have send button', () => {
|
||||
renderWithContexts(<ThemedNLQChatPanel />);
|
||||
const sendButton = screen.getByLabelText('Send');
|
||||
expect(sendButton).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should have clear conversation button', () => {
|
||||
renderWithContexts(<ThemedNLQChatPanel />);
|
||||
const clearButton = screen.getByText('Clear');
|
||||
expect(clearButton).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should disable send button when input is empty', () => {
|
||||
const { container } = renderWithContexts(<ThemedNLQChatPanel />);
|
||||
const sendButton = container.querySelector('button[data-label="Send"]');
|
||||
expect(sendButton).toBeDisabled();
|
||||
});
|
||||
|
||||
it('should enable send button when input has text', () => {
|
||||
const { container } = renderWithContexts(<ThemedNLQChatPanel />);
|
||||
const input = screen.getByPlaceholderText(/Describe the SQL you need/i);
|
||||
|
||||
fireEvent.change(input, { target: { value: 'Find all users' } });
|
||||
|
||||
const sendButton = container.querySelector('button[data-label="Send"]');
|
||||
expect(sendButton).not.toBeDisabled();
|
||||
});
|
||||
|
||||
it('should show message when not in query tool mode', () => {
|
||||
const mockQueryToolCtx = createMockQueryToolCtx(false);
|
||||
renderWithContexts(<ThemedNLQChatPanel />, {
|
||||
queryToolCtx: mockQueryToolCtx,
|
||||
});
|
||||
|
||||
expect(
|
||||
screen.getByText(/AI Assistant is only available in Query Tool mode/i)
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should clear input after typing and clicking clear', () => {
|
||||
renderWithContexts(<ThemedNLQChatPanel />);
|
||||
const input = screen.getByPlaceholderText(/Describe the SQL you need/i);
|
||||
|
||||
fireEvent.change(input, { target: { value: 'Find all users' } });
|
||||
expect(input.value).toBe('Find all users');
|
||||
|
||||
const clearButton = screen.getByText('Clear');
|
||||
fireEvent.click(clearButton);
|
||||
|
||||
// Input should still have text (clear only clears messages)
|
||||
expect(input.value).toBe('Find all users');
|
||||
});
|
||||
});
|
||||
@@ -259,6 +259,7 @@ module.exports = [{
|
||||
'pure|pgadmin.tools.psql',
|
||||
'pure|pgadmin.tools.sqleditor',
|
||||
'pure|pgadmin.misc.cloud',
|
||||
'pure|pgadmin.browser.ai_tools',
|
||||
],
|
||||
},
|
||||
},
|
||||
|
||||
@@ -156,6 +156,7 @@ let webpackShimConfig = {
|
||||
'pgadmin.tools.sqleditor': path.join(__dirname, './pgadmin/tools/sqleditor/static/js/'),
|
||||
'pgadmin.tools.user_management': path.join(__dirname, './pgadmin/tools/user_management/static/js/'),
|
||||
'pgadmin.user_management.current_user': '/user_management/current_user',
|
||||
'pgadmin.browser.ai_tools': path.join(__dirname, './pgadmin/llm/static/js/ai_tools'),
|
||||
},
|
||||
externals: [
|
||||
'pgadmin.user_management.current_user',
|
||||
|
||||
928
web/yarn.lock
928
web/yarn.lock
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user