¶ PBXware - AI Hub Setup Guide (Multi-Tenant)
This guide explains how to configure PBXware AI Hub features in the Multi-Tenant (MT) edition, including AI Providers, AI Voice Agents, MCP Servers, Live Transcription, Call Transcription, Text to Speech (TTS), and Voicemail Transcription. It also covers how MCP Servers can be associated with Voice Agents and how inbound calls can be routed to Voice Agents using DIDs.
Info: Some AI Hub features are currently in beta. In the PBXware GUI, these features are marked with a BETA label next to the feature name. Functionality, supported options, and behavior may change in future PBXware versions.
¶ AI Hub overview
PBXware AI Hub groups multiple AI-related features under one place. Depending on your needs, you can configure:
- AI Providers — vendor credentials/configurations used across AI features
- AI Voice Agents — inbound AI call handling (with DID routing)
- MCP Servers — MCP connections and external tools that can be shared across multiple Voice Agents within the same tenant
- Live Transcription — real-time streaming transcription with WebSocket callbacks
- Call Transcription — transcription of recorded calls
- Text to Speech (TTS) — generating sound files from text
- Voicemail Transcription — transcription of voicemail messages
¶ AI Providers
An AI Provider is the external service PBXware uses to enable AI Hub features across the platform. Depending on your configuration, AI Providers can be used for AI Voice Agents, Live Transcription, Call Transcription, Text to Speech (TTS), and Voicemail Transcription. Before using any AI Hub feature that depends on a provider, you must first create at least one AI Provider.
In the Multi-Tenant (MT) edition, AI Providers can be created either on the System level or directly on a Sub-Tenant level. The provider form is the same in both places, with one key difference explained below.
¶ Where AI Providers are managed
-
System
To open AI Providers, navigate to:
System -> Home -> AI Hub -> AI ProvidersThe System-level list provides oversight of all AI Providers in the environment — providers created on the System level and providers created on Sub-Tenants.
-
Sub-Tenant
To open AI Providers inside a Sub-Tenant, navigate to:
Select Sub-Tenant -> Home -> AI Hub -> AI ProvidersThe Sub-Tenant list shows only providers that are assigned to that Sub-Tenant or created within that Sub-Tenant.
¶ What changes in the Add AI Provider form
When you click Add AI Provider, you will fill in the fields below.
The only difference is that System level includes an extra "Tenant" field, while Sub-Tenant level does not.
-
Name
A friendly label for this configuration (for example:OpenAI - Production). -
Tenant (System level only)
Select the Sub-Tenant you want to assign this AI Provider to.
This field is not shown when creating the provider inside a Sub-Tenant, because it is automatically created for that Sub-Tenant. -
AI Provider
Select the provider. Available options may include:OpenAIGoogle SpeechGoogle Real-timeIBM (Deprecated)DeepgramAWS TranscribexAIHosted WhisperEleven Labs
These providers can be used in multiple PBXware areas, not only AI Voice Agents.
For example, a provider that supports transcription can also be used for transcription-related features where applicable.
Note: IBM Watson voicemail transcription is deprecated since PBXware 8.0.0 and should no longer be used for new configurations. We recommend migrating to another supported provider.
-
Scope
Select one or more scopes from the drop-down menu. Scopes define which capabilities PBXware can use through this provider. Available scopes:-
Text To Speech (TTS) — converts text into spoken voice
-
Speech To Text (STT) — converts spoken audio into text
-
Speech To Speech (STS) — converts spoken input into spoken AI-generated output
-
Real Time Speech To Text — converts spoken audio into text in real time (used by live/streaming transcription features where applicable)
-
Text Processing — processes text-based requests, such as generating summaries and other AI-generated text content
-
-
Credentials
Enter the credentials required by the selected provider.- For most providers, this is a single API Key field.
- For AWS Transcribe, these fields are Access Key ID and Secret Access Key.
-
Provider URL
The provider API endpoint specified by your selected provider (for example:https://your-provider-endpoint).
When you finish, click Save. The provider becomes available on the Sub-Tenant where it was created (or assigned).
¶ Voice Agents
¶ Feature overview: AI Voice Agents
AI Voice Agents allow PBXware to answer inbound calls with an AI-driven voice experience. With the right configuration, an AI Voice Agent can:
- Answer calls automatically with a consistent greeting and tone
- Ask clarifying questions and guide the caller through a conversation
- Transfer the caller to the correct destination when needed
- End calls politely when the request is completed
In PBXware, this feature is configured through:
-
AI Providers
The AI service connection used for speech, transcription, and/or real-time conversation. -
Voice Agents
The agent configuration, including provider selection, model/voice where applicable, prompt/behavior, call limits, fallback behavior, and optional MCP Server association. -
DID routing
Used to route inbound calls to the configured Voice Agent.
MCP note: MCP Servers are configured separately under AI Hub → MCP Servers.
After an MCP Server is configured, it can be selected in the Voice Agent configuration so its tools are available to that agent.
A Voice Agent is the AI call handler. You assign it an AI Provider, select a model and voice, define how it should behave through the Prompt, configure call handling options, and optionally select MCP Servers that were already configured under AI Hub → MCP Servers.
To open Voice Agents, navigate to:
Select Sub-Tenant -> Home -> AI Hub -> Voice Agents
Click Add Voice Agent to open the configuration form.
¶ Voice Agent settings
Supported AI Providers for AI Voice Agents: OpenAI, xAI, Google Real-time, and Eleven Labs.
The Voice Agent form changes depending on the selected provider — some fields are common, while others are provider-specific.
All Voice Agent configuration is done inside the General settings form. Depending on your selected AI provider, the available fields may differ.
¶ Common fields (all providers)
These fields are available for OpenAI, xAI, Google Real-time, and Eleven Labs Voice Agents:
-
Agent name
The name shown in PBXware (for example:Help Desk Agent). Choose a name that matches the role of the agent. -
Agent slug
A short, unique identifier used internally for agent-to-agent transfers.
When configuring an Agent transfer through the Tools section, select the destination agent from the drop-down menu. The slug can still be referenced by existing prompt-based transfer configurations. -
AI provider
Select the provider you created earlier from the drop-down list (OpenAI, xAI, Google Real-time, or Eleven Labs). -
Dial Number
This is the agent's local/internal number in PBXware. You can dial it internally, route calls to it, or transfer calls to it. -
Agent enabled
Enables/disables the agent. -
Limit call duration / Max call duration (min)
Restricts maximum call duration (when enabled). -
Fallback / Fallback timeout (sec) / Transfer call number
Defines what PBXware should do if the agent cannot continue (transfer, end call, or no fallback). -
Silence detection / Silence timeout (sec)
Controls automatic call termination when continuous silence is detected during the call.
When Silence detection is enabled, PBXware monitors the call and ends it if silence continues for the configured Silence timeout (sec) value.
¶ Provider-specific fields and configuration
This section highlights which fields are available per provider and what to configure.
¶ OpenAI Voice Agent configuration (PBXware-managed)
For OpenAI Voice Agents, PBXware provides full behavior configuration directly in the Voice Agent form.
-
Model
The model defines how the agent processes the conversation in real time. Your available options may include:-
gpt-realtime
A general-purpose real-time voice model suitable for most day-to-day call scenarios where you want stable, interactive conversations. -
gpt-4o-realtime-preview
A higher-capability real-time model designed for more complex conversations (for example, when callers ask multi-step questions or you need stronger reasoning and accuracy in responses). -
gpt-4o-mini-realtime-preview
A lightweight real-time model intended for simpler call flows and quick interactions, typically chosen when you want lower overhead for straightforward Q&A or short scripted-style conversations.
Models may differ in response quality, speed, and resource usage. Select the model that matches your expected call complexity and performance requirements.
-
-
Transcription model
Used to convert the caller's speech into text when required by the selected provider/model configuration. -
Voice
The voice used for the agent's spoken responses (for example:alloy). -
Introduction and Prompt
These fields define the call opening behavior and the agent rules/personality (details below). -
MCP Servers
Select one or more previously configured MCP Servers from the drop-down menu.
The selected MCP Servers and their available tools become accessible to the Voice Agent during calls.MCP Servers must be configured under AI Hub → MCP Servers before they can be selected here.
¶ xAI Voice Agent configuration (PBXware-managed + xAI extensions)
For xAI Voice Agents, PBXware provides the same core behavior settings as OpenAI, along with additional xAI-specific options.
-
Model
Select the xAI model available (for example:grok). -
Voice
Select the voice available for xAI (voice options depend on provider). -
Introduction and Prompt
These fields define the call opening behavior and the agent rules/personality (details below). -
MCP Servers
Select one or more previously configured MCP Servers from the drop-down menu.
The selected MCP Servers and their available tools become accessible to the Voice Agent during calls.MCP Servers must be configured under AI Hub → MCP Servers before they can be selected here.
¶ xAI-specific options (search + collections)
Depending on configuration, xAI providers can expose additional fields such as:
-
Web search (On/Off)
Enables provider-side web search capability for the agent (if supported by your xAI setup/policy). -
X Search Allowed Handles
A restriction list for which X (Twitter) handles the agent is allowed to search.
Use this to limit the agent to specific official accounts (for example your company handle) instead of unrestricted searching. -
Collection Ids
One or more IDs referencing xAI collections created on the xAI side.
Collections typically represent a curated knowledge base (documents you upload on the provider side) that the agent can use as grounded context. -
Max number of results from collections
Controls how many retrieved results the agent can use from the configured collection(s).
Increase if you want broader context, or keep lower for more focused answers.
Important: Collections and their documents are managed in the xAI provider environment.
PBXware only references them via Collection Ids so the agent can use that uploaded knowledge during calls.
¶ Google Real-time Voice Agent configuration (Gemini)
For Google Real-time Voice Agents, PBXware provides Gemini-based real-time voice configuration directly in the Voice Agent form.
In addition to the common Voice Agent fields, the following configuration options are available for Google Real-time Voice Agents:
-
Model
Select the Google real-time model used to process the conversation and generate the agent's responses.Available models include:
gemini-3.8-live (BETA)gemini-3.8-live-extended-thinking (BETA)
Available models may vary depending on the Google AI Provider configuration and the installed PBXware version.
-
Voice
Select the voice used for the agent's spoken responses.
For example,Zephyrprovides one of the available predefined Google voices. -
Thinking level
Defines how much reasoning the selected model may apply before generating a response.
Lower settings prioritize faster responses and reduced latency, while higher settings may provide better results for more complex requests but can increase response time. -
Google Search grounding
Enables the agent to use Google Search when generating responses.
This allows the model to ground answers using current information available through Google Search. -
VAD start sensitivity
Defines how readily Voice Activity Detection (VAD) recognizes that the caller has started speaking.
A higher sensitivity detects the beginning of speech more easily, while a lower sensitivity requires a stronger speech signal before starting the caller's turn. -
VAD end sensitivity
Defines how readily VAD determines that the caller has finished speaking.
A higher sensitivity ends the caller's turn more quickly after speech stops, while a lower sensitivity is less likely to end the caller's turn during short pauses. -
VAD prefix padding (ms)
Defines how long speech must be detected before the beginning of the caller's turn is confirmed.
Lower values allow shorter speech segments to be recognized more quickly but may increase the possibility of detecting background noise as speech. -
VAD silence duration (ms)
Defines how long silence must continue before the caller's current speaking turn is considered complete.
Higher values allow longer pauses without ending the caller's turn but may increase the delay before the agent responds.
Values that are too low may split a single sentence into multiple turns during natural pauses. -
Introduction
Defines the initial caller message sent to the model when the call begins.
Use this field to start the conversation in a predictable way. -
Prompt
Defines the agent's role, objectives, tone, behavior, restrictions, and tool usage rules.
See the Introduction vs Prompt section below for detailed guidance. -
MCP Servers
Select one or more previously configured MCP Servers from the drop-down menu.
Tools provided by the selected MCP Servers become available to the Voice Agent during calls.MCP Servers must first be configured under AI Hub → MCP Servers.
NOTE: The Google-specific VAD settings control when the beginning and end of each caller speaking turn are detected.
The separate PBXware Silence detection and Silence timeout (sec) options control when the entire call is ended because of prolonged silence.
¶ Eleven Labs Voice Agent configuration (provider-managed)
For Eleven Labs Voice Agents, the agent behavior is primarily configured in the Eleven Labs dashboard rather than in PBXware.
In PBXware, you configure:
- Agent ID
The identifier of the agent created in Eleven Labs.
After you create/configure the agent in the Eleven Labs dashboard, copy its Agent ID and paste it into this field.
Important limitation / expected behavior:
PBXware does not mirror the Eleven Labs agent configuration UI. Prompt/voice/knowledge/tools are configured on the Eleven Labs side.
PBXware is responsible for routing calls to the Eleven Labs agent using the Agent ID, and applying PBXware-side call controls (limits/fallback/transfer behavior).
¶ Model, transcription, and voice (provider-specific)
The model defines how the agent processes the conversation in real time.
The available fields in this section depend on the selected AI provider.
-
OpenAI
PBXware displays Model, Transcription model, and Voice fields.
See OpenAI Voice Agent configuration above for model details and recommendations. -
xAI
PBXware displays Model and Voice fields.
The model is selected from the available xAI options (for example:grok).
Transcription model is not shown for xAI Voice Agents. -
Google Real-time
PBXware displays Model, Voice, Thinking level, Google Search grounding, and Google-specific Voice Activity Detection settings.
Transcription model is not shown for Google Real-time Voice Agents because transcription is handled by the selected real-time model. -
Eleven Labs
PBXware does not display Model, Transcription model, or Voice fields.
These are configured in the Eleven Labs dashboard when creating the agent. PBXware connects to that agent using the Agent ID.
¶ Tools
Applies to: OpenAI, xAI, and Google Real-time Voice Agents. Built-in tool configuration is not available for Eleven Labs Voice Agents because their behavior and tools are managed through the Eleven Labs dashboard.
The Tools section allows built-in call control tools to be configured directly in the Voice Agent form. Click Add tool and select the required tool from the drop-down menu.
The following built-in tools can be configured:
-
Call transfer
Transfers the active call to the specified destination number.
Enter the Destination number and describe when the agent should transfer the call. For example, the call can be transferred when the caller asks to speak to support or requests a department that the Voice Agent does not handle.
-
End call
Allows the Voice Agent to end the active call.
Describe when the tool should be used. For example, the agent can end the call after confirming that the caller's request has been resolved and no further assistance is required.
-
Agent transfer
Transfers the call to another configured AI Voice Agent.
Select the Destination agent and describe when the call should be transferred to that agent.
Enable Include conversation transcript to provide the destination agent with the transcript of the current conversation.
Use Collected data to send to define information that should be passed to the destination agent. Click Add field and enter the field name and a description of the collected value.
Multiple tools can be added to the same Voice Agent. Each tool should have a clear description so the agent can determine when and how it should be used.
NOTE: Built-in tools can still be configured through the Prompt for backward compatibility. When tools are configured through the Tools section, avoid adding conflicting transfer destinations or tool instructions to the Prompt.
¶ Introduction vs Prompt (important to configure correctly)
¶ Introduction (what it really does)
The field is named Introduction, but functionally it acts like the first caller message sent to the AI model when the call starts.
Think of it as: "What is the first thing the caller 'says' to start the conversation?"
This is useful when you want the agent to begin the call in a predictable way (for example: to always introduce itself first).
Good examples:
I am the caller. Please introduce yourself and ask how you can help me.Hello.Hi, I need help.
Avoid placing role definitions here (for example: "You are the Help Desk.").
Role and behavior should be defined in the Prompt.
¶ Prompt (the agent's behavior and rules)
Applies to: OpenAI, xAI, and Google Real-time Voice Agents.
For Eleven Labs, configure the agent prompt/behavior in the Eleven Labs dashboard.
The Prompt is the most important configuration field. It defines:
- what the agent is (role)
- how it speaks (tone)
- what it should do (objective)
- what it must avoid (boundaries)
- how call actions should be handled (transfer/end call rules)
- how built-in and MCP-provided tools should be used
A well-written prompt makes the agent consistent and reliable. A vague prompt often results in unpredictable responses.
¶ Prompt-based built-in tool configuration
PBXware Voice Agents include built-in call control tools that are available without configuring an MCP Server. These tools can be configured through the Tools section or referenced directly in the Prompt. Prompt-based configuration remains supported for backward compatibility with existing Voice Agents.
universal_transfer_tool(number)— transfers the active call to the specified destination numberend_call_tool()— ends the callagent_transfer_tool(slug, context)— transfers the call to the specified AI agent and passes collected context to that agent
For prompt-based configuration, use the Prompt to define when the agent is allowed to transfer or end the call, which numbers are valid transfer destinations, and what the agent should confirm with the caller before performing an action.
¶ Default prompt structure (example)
PBXware includes a default prompt example similar to the one below. You can keep the structure and customize the content for your business needs:
# Prompt example
This is our prompt example. Change it however suits your needs.
# Role & Objective
You are a help desk agent. Your task is to answer the user's questions about company.
# Personality & Tone
## Personality
- Friendly, calm and approachable help desk agent.
## Tone
- Concise and confident.
## Length
- 2-3 sentences per turn.
# Context
- Company info page www.bicomsystems.com
- Caller ID of the current caller: %CALLER_ID%
- Current UTC time: %CURRENT_TIME%
- Current UTC date: %CURRENT_DATE%
- Current UTC date and time: %CURRENT_DATE_TIME%
# Tools
- Before any tool call, say one short line like "I'm checking that now." Then call the tool immediately.
## universal_transfer_tool(number)
- Use with number parameter "1074" when user wants to talk to support
- Use with number parameter "1067" when user wants to talk to sales
## agent_transfer_tool(slug, context)
- Use with slug parameter "data-agent" and context parameter in format key value pairs, for example: [name: collected_name, age: collected_age], after you collect the caller's name and age
¶ What matters most in a Prompt
-
Role & Objective
Define what the agent is responsible for (support, sales, receptionist, help desk, etc.).
This is where you describe who the agent is and what success looks like. -
Tone / Length
Use these sections to keep the experience consistent (for example: short answers, calm tone, professional wording). -
Context
Describe which information sources the agent should rely on (public website, internal KB, policies).
If the agent must not guess or invent, state that clearly (for example: "If you are not sure, ask a follow-up question.").
PBXware supports dynamic variables that can be used throughout the Prompt. The variables are automatically resolved at runtime, allowing the agent to receive current call-specific and time-based information where needed.
Available variables:%CALLER_ID%— the Caller ID of the current caller%CURRENT_TIME%— the current UTC time%CURRENT_DATE%— the current UTC date%CURRENT_DATE_TIME%— the current UTC date and time
-
Transfer and End Call behavior
PBXware supports call actions (such as transfer and ending the call). In your prompt, define:- when the agent is allowed to transfer
- what the agent must confirm before transferring (for example: reason for calling, department)
- where the agent should transfer (destination numbers)
- when the agent should end the call (confirm resolution and close politely)
Example tool names you may see in prompts:
universal_transfer_tool(number)— transfers the call to the specified numberend_call_tool()— ends the callagent_transfer_tool(slug, context)— transfers the call to the specified AI agent and passes collected context to that agent
Update any example numbers (like
1074and1067) to match your real destinations.
¶ Call limits and fallback behavior
These options define how PBXware should handle long calls or situations where the agent cannot continue.
-
Limit call duration
Enable to restrict maximum call duration. -
Max call duration (min)
Maximum duration in minutes (applies when limit is enabled). -
Fallback
What PBXware should do if the agent cannot continue:Transfer CallEnd CallNot Set
-
Fallback timeout (sec)
Time PBXware waits before applying fallback. -
Transfer call number
If fallback is Transfer Call, enter the destination number. -
Silence detection
Enables automatic call termination based on detected silence during the conversation. -
Silence timeout (sec)
Defines how long silence must continue before PBXware ends the call.
This value is used only when Silence detection is enabled.
When you finish configuring the agent, click Save.
¶ DID Routing (Assign an inbound DID to a Voice Agent)
In the Multi-Tenant (MT) edition, the DID is created on the System level and then becomes visible on the Sub-Tenant.
After the DID is available on the Sub-Tenant, open it and set its routing destination to a Voice Agent.
On the Edit DID screen, the following fields are available:
-
Name
A friendly label for the DID (for example:AI Agent DID).
This helps you identify the DID in lists and reports. -
Destination (most important)
Defines where inbound calls to this DID will be routed.
Open the drop-down menu and select AI Voice Agents to route calls to a Voice Agent. -
Value
This field depends on the selected Destination.
When Destination is set to AI Voice Agents, Value becomes a drop-down list of available Voice Agents.
Select the Voice Agent that should receive inbound calls to this DID. -
Service Plan (optional / depends on environment)
Displays the service plan assigned to the DID (if applicable). In many setups this may be blank or not used. -
Call Rating Extension (optional)
Used only if your deployment uses call rating / billing logic that requires an extension reference. If not used, it can remain empty.
When you finish, click Save to apply the DID routing changes.
¶ AI Voice Agent Call Log (CDR integration)
After configuring and using AI Voice Agents, PBXware also provides a dedicated AI Voice Agent Call Log view inside CDR for calls handled by an AI agent.
This allows administrators to inspect the AI side of the conversation in more detail, including:
- the call transcript
- raw event data generated during the interaction
- general call information related to the selected AI agent
- agent selection in cases where an agent-to-agent transfer occurred during the same call
This view is useful for troubleshooting, validating agent behavior, reviewing transfer flow, and downloading conversation-related data for further analysis.
To access it, navigate to:
Select Sub-Tenant -> Home -> Reports -> CDR
If you click on the number that belongs to the AI agent side of the call, PBXware opens the AI Voice Agent Call Log side panel.
¶ AI Voice Agent Call Log overview
The AI Voice Agent Call Log is opened as a side panel from the selected CDR entry.
At the top of the panel, PBXware shows the currently selected AI agent related to that call.
If the call included an agent-to-agent transfer, the top drop-down menu allows you to switch between the agents that participated in that call flow.
This makes it easier to review which part of the interaction belonged to which agent.
The side panel contains three tabs:
- Transcript
- Events
- Info
Each tab provides a different level of detail about the AI call.
¶ Transcript tab
The Transcript tab displays the conversation between the caller and the selected AI Voice Agent.
This view is intended for human-readable review of the call flow and is useful when validating:
- the greeting/introduction used by the agent
- the sequence of questions and answers
- whether the agent followed the configured prompt correctly
- whether the interaction sounds natural and contextually correct
If multiple agents were involved in the same call due to transfer, the transcript shown depends on the currently selected agent from the drop-down menu.
A Download transcript option is also available, allowing the transcript to be exported for offline review, QA validation, or troubleshooting documentation.
¶ Events tab
The Events tab displays the underlying event data related to the selected AI Voice Agent call.
This tab is more technical in nature and is primarily useful for:
- debugging
- verifying what was exchanged during the AI session
- reviewing session/event payload structure
- investigating provider-side or real-time interaction details
Each event is shown in its own block, identified by an Event ID, and the content is displayed in structured raw format.
Depending on the provider and call flow, this section may include technical session details such as:
- audio input format
- turn detection/session configuration
- provider-generated event payloads
- internal interaction metadata relevant to the AI session
A Download events option is available at the top of the tab, allowing the full event data to be exported for deeper technical analysis.
¶ Info tab
The Info tab provides a short summary of the selected AI agent call handling details.
This tab is intended as a quick overview and includes information such as:
- whether the call was transferred
- whether the call was transferred to another agent
- the name of the selected AI agent
- the AI provider used for that part of the call
This is useful for quickly confirming the final routing/handling outcome without reviewing the full transcript or raw events.
For example, in transferred call scenarios, this tab helps distinguish whether:
- the call remained with one agent
- the call was handed over to another AI agent
- the selected view belongs to the original or transferred agent
¶ Agent selection in transferred calls
If an agent-to-agent transfer occurs during the conversation, the AI Voice Agent Call Log top drop-down menu displays the agents involved in that call.
By selecting a different agent from the list, you can review that specific agent's:
- transcript
- events
- info summary
This is especially useful when validating multi-agent flows, because each agent may handle only one segment of the overall conversation.
¶ Practical use cases
The AI Voice Agent Call Log can be used for several common validation and troubleshooting scenarios:
- Reviewing whether the agent answered with the correct introduction and tone
- Confirming that the configured prompt behavior was followed during the call
- Checking whether an agent-to-agent transfer occurred and which agent handled each part
- Downloading transcript data for QA or documentation purposes
- Downloading event data for technical investigation and provider/session troubleshooting
- Verifying which provider and agent handled the call from the Info tab
This makes the AI Voice Agent Call Log a useful extension of standard CDR data for AI-driven call flows.
¶ MCP Servers
MCP (Model Context Protocol) is the bridge between your Voice Agent and external services.
It allows the agent to request additional information or run controlled actions by calling tools hosted outside PBXware.
In our setup, we used Dify to host the MCP endpoint. Dify is an AI workflow/orchestration platform that helps you build agent workflows and expose tool endpoints, including MCP-compatible endpoints, without writing everything from scratch.
You are not limited to Dify — you can use any other service that can host an MCP-compatible endpoint and provide a stable URL with authentication.
¶ Why MCP is used in PBXware Voice Agents
A Prompt can describe behavior, tone, and rules, but it cannot reliably provide dynamic data by itself. MCP is used when you want the Voice Agent to:
- Retrieve up-to-date information from a system you control, such as internal knowledge, rules, or structured data
- Keep tool logic outside PBXware so it can be updated without changing the PBXware configuration
- Enforce controlled access through authentication headers and a dedicated endpoint
- Provide a single place where MCP tools are managed and maintained
MCP Servers are configured centrally instead of separately for each Voice Agent. This prevents the same MCP Server configuration from being duplicated when it is used by multiple agents.
A single MCP Server connection can be associated with multiple Voice Agents, reducing unnecessary connections and simplifying configuration management.
¶ How the integration works (high-level)
A typical flow looks like this:
- An administrator configures the MCP Server under AI Hub → MCP Servers.
- PBXware connects to the external MCP Server and retrieves the available tools.
- The configured MCP Server is associated with one or more Voice Agents.
- A caller reaches PBXware and is routed to a Voice Agent.
- The Voice Agent determines when an MCP tool should be called based on the conversation, the tool description, and any additional instructions included in the Prompt.
- The MCP Server, for example one hosted in Dify, runs the tool logic and returns a response.
- The Voice Agent uses the returned result to continue the conversation.
Note: MCP tools do not need to be explicitly referenced in the Voice Agent Prompt.
The agent can determine when to call an MCP tool based on the tool name and description provided by the MCP Server. The Prompt can optionally include additional instructions or restrictions for tool usage.
¶ About Dify (what it is and why we used it)
Dify is a workflow platform designed to connect AI agents to external systems. In practice, it helps you:
- Build a structured flow for how the agent should retrieve information
- Connect to data sources, such as databases, APIs, or internal services
- Control access using tokens and authentication headers
- Test and update tool behavior without changing the PBXware configuration
We used Dify because it makes it easier to manage and evolve tool logic over time.
However, any MCP-capable tool host can be used if it meets your requirements, including secure access, a stable endpoint, and predictable output.
¶ Configure an MCP Server in PBXware
MCP Servers are configured from a dedicated page under the AI Hub.
To open the page, navigate to:
Select Sub-Tenant -> Home -> AI Hub -> MCP Servers
The MCP Servers page displays all configured MCP Servers.
The list contains the following information:
-
Name
The name used to identify the MCP Server configuration. -
URL
The endpoint used to connect to the external MCP Server. -
Status
Displays the current connection status between PBXware and the MCP Server.When the connection is active, the status is displayed as Connected with a green check icon.
-
Edit
Opens the MCP Server configuration page. -
Delete
Removes the MCP Server configuration.
To create a new configuration, click Add MCP Server.
¶ MCP Server settings
When adding or editing an MCP Server, the configuration page contains the following sections:
¶ General
-
Name
Enter a descriptive label for the MCP Server, for exampleMCP - Production. -
Type
Select the connection type supported by the MCP Server, for exampleStreamable HTTP. -
URL
Enter the MCP endpoint URL provided by the external tool-hosting service.
¶ Headers
The Headers section allows authentication or other required HTTP headers to be included when PBXware connects to the MCP Server.
-
Header Name
Enter the HTTP header name, for exampleAuthorization. -
Header Value
Enter the corresponding header value, for exampleBearer <token>.
Click Add item to configure additional headers when required.
Use authentication headers whenever the MCP endpoint requires controlled or secured access.
¶ Connection status and available tools
The Connection status section allows administrators to verify whether PBXware is successfully connected to the configured MCP Server.
When the connection is established, the status is displayed as Connected.
PBXware also displays the tools currently available through the MCP Server. The tools table contains:
-
ID
A unique identifier generated for the MCP tool. PBXware uses this identifier internally when providing the tool to the Voice Agent. This prevents naming conflicts when tools from different MCP Servers have the same name. -
Tool
The original name of the tool exposed by the MCP Server. This name is displayed to help administrators identify the tool. -
Description
The description received from the MCP Server. The Voice Agent uses this information to understand the purpose of the tool and determine when it should be called.
The tool description helps administrators verify what functionality is available and helps the Voice Agent determine when the tool should be called.
If the server is unavailable or the connection cannot be established, the connection status indicates that the MCP Server is not connected and the available tools cannot be retrieved.
When finished, click Save.
¶ Associate MCP Servers with a Voice Agent
To associate MCP Servers with a Voice Agent, navigate to:
Select Sub-Tenant -> Home -> AI Hub -> Voice Agents
Open Add Voice Agent or edit an existing Voice Agent.
In the Voice Agent configuration form, locate the MCP Servers field and select one or more configured MCP Servers from the drop-down menu.
The selected MCP Servers and their available tools become accessible to that Voice Agent.
The same MCP Server can be associated with multiple Voice Agents within the same tenant without duplicating its URL, connection type, authentication headers, or server connection.
Changes made to an MCP Server configuration apply to every Voice Agent associated with that server within the same tenant.
Removing an MCP Server from a Voice Agent only removes the association. It does not delete the MCP Server configuration from the MCP Servers page.
¶ Existing MCP Server configurations
During an upgrade, existing MCP Server configurations that were previously defined inside Voice Agents are migrated to the corresponding tenant's MCP Servers page.
Existing associations between Voice Agents and their MCP Servers are preserved during the migration.
After upgrading, administrators should review the migrated MCP Server configurations, verify their connection status, and confirm that the expected tools are displayed.
¶ Prompt guidance (optional)
MCP tools can be used by the Voice Agent even when they are not explicitly mentioned in the Prompt.
The Voice Agent uses the tool Description provided by the MCP Server to understand the tool's purpose and determine when it should be called. Therefore, each tool should have a clear and accurate description explaining what it does and when it should be used.
Additional Prompt instructions can optionally be added when you want to:
- restrict when a particular tool may be called
- require specific information to be collected before calling a tool
- define how the agent should handle failed or empty tool responses
- require confirmation from the caller before an action is performed
- provide business-specific rules that are not covered by the tool description
Note: PBXware built-in call-control tools, such as transfer tools, require instructions in the Prompt to define destination mappings and the conditions under which a particular number should be called.
¶ AI Voice Agent Setup Summary
You have now:
- Created an AI Provider with the required scopes and credentials
- Created and configured a Voice Agent, including Introduction, Prompt, provider settings, and call handling options
- Configured MCP Servers under AI Hub → MCP Servers, if external tools are needed
- Associated configured MCP Servers with the Voice Agent, if applicable
- Routed an inbound DID to the Voice Agent
Next step: test with real calls and confirm the full call flow, including greeting, responses, transfers, fallback behavior, and MCP tool usage if MCP Servers are associated with the agent.
¶ Live Transcription
Live Transcription allows PBXware to stream call audio in real time to a supported Speech-to-Text service and forward transcription results to your WebSocket callback URL. This is designed for scenarios where you want transcription events while the call is still in progress.
Important prerequisite: Ensure Stereo Recording and Recording are enabled.
Live Transcription requires recording and stereo audio to stream the call correctly.
¶ Where to configure
To open Live Transcription, navigate to:
Select Sub-Tenant -> Home -> AI Hub -> Live Transcription
The Live Transcription page contains two tabs:
-
Live Transcription
Used to view, add, edit, and delete Live Transcription configurations. -
Configuration
Used to control whether Live Transcription is enabled for internal calls.
¶ Live Transcription tab (what you see on entry)
The Live Transcription tab shows all saved configurations. Each row includes:
-
Name
A friendly label used to identify the configuration. -
AI Service
The transcription service type used by this configuration (for example:OpenAI,AWS Transcribe,Deepgram,Google Speech To Text). -
Websocket Url
The WebSocket destination that receives live transcription events. -
Status
Indicates the configuration health/validity (green check icon). -
Edit / Delete
- Edit (pencil icon) opens the configuration for changes.
- Delete (X icon) removes the configuration.
To create a new configuration, click Add Live Transcription Config.
¶ Configuration tab
The Configuration tab contains global Live Transcription settings.
The following option is available:
- Enable Live Transcription for internal calls (On/Off)
Enables or disables Live Transcription support for internal calls.
¶ Create a Live Transcription configuration
Click Add Live Transcription Config to open the configuration screen:
Live Transcription -> Add Live Transcription Config
This screen contains three sections:
¶ 1) Live Transcription Info
- Request Name
The display name of this configuration.
¶ 2) AI Transcription
-
AI Service
Select one of the supported services:OpenAIAWS TranscribeDeepgramGoogle Speech To Text
-
Provider (appears after selecting AI Service)
Select the AI Provider credentials you already configured under:
Select Sub-Tenant -> Home -> AI Hub -> AI ProvidersNote: The Provider list depends on which AI Service you choose, which provider credentials exist in your system, and whether the provider supports the required Real Time Speech to Text scope.
¶ 3) Websocket Callback
-
URL
WebSocket server URL that will receive live transcription events (example format:wss://your-provider-endpoint/realtime). -
Metadata
A JSON formatted field. Whatever you enter here is included with every WebSocket message PBXware sends.Supported placeholders you can include inside Metadata:
%CALLER_ID%— Caller ID%CHANNEL_ID%— ID of the Queue, ERG, or Extension where Live Transcription is enabled%CHANNEL_NAME%— Name of the Queue or ERG that was called%CALLEE_ID%— Extension that answered in the Queue/ERG%CALLEE_NAME%— Name of the extension that answered in the Queue/ERG
When finished, click Save.
¶ AI Service setup details (per provider)
Each AI Service selection exposes a different set of fields.
¶ OpenAI
-
Provider
Select the OpenAI AI Provider entry (credentials). -
Model
The transcription model used for Live Transcription (example shown:gpt-realtime-whisper,gpt-4o-transcribe,gpt-4o-mini-transcribe,whisper-1). -
URL
WebSocket endpoint used for the OpenAI real-time transcription connection. -
Language
Language used for transcription (example shown:English).
¶ AWS Transcribe
-
Provider
Select the AWS AI Provider entry (credentials). -
Content Identification Type (On/Off)
Enables identification features tied to selected PII Entity Type options. -
Content Redaction Type (On/Off)
Enables redaction based on selected PII Entity Type options. -
PII Entity Type
Select PII types for identification/redaction (depending on enabled option). -
Partial Results Stabilization (On/Off)
Controls how partial transcription results are handled. -
Partial Results Stability
Stability level used when stabilization is enabled (example shown:None). -
Regions
AWS region endpoint targeted by the transcription request (example shown:US East (N. Virginia)). -
Language Code
Transcription language (example shown:English (US)). -
Custom Language Model Name (optional)
Name of a custom AWS language model, if used. -
Vocabulary Filter Method / Vocabulary Filter Name / Vocabulary Name (optional)
Optional AWS vocabulary/vocabulary filter configuration. -
Sentiment Analysis (On/Off)
Enables sentiment analysis for AWS Transcribe live transcription. When enabled, completed transcription segments include a sentiment value in the response. Supported sentiment values arePOSITIVE,NEGATIVE,NEUTRAL, andMIXED.
¶ Deepgram
-
Provider
Select the Deepgram AI Provider entry (credentials). -
URL
Deepgram streaming endpoint (example shown:wss://api.deepgram.com/v1/listen). -
Dictation (On/Off)
Dictation automatically formats spoken commands for punctuation into their respective punctuation marks. -
Filler Words (On/Off)
Includes filler words like "uh" and "um" in transcription output. -
Interim Results (On/Off)
Sends ongoing transcription updates while the call is in progress. -
Keywords / Keyterm
Boosts recognition of important words or phrases, like names, product terms, or jargon. The model pays extra attention to these. -
Language
Language used for transcription. -
Deepgram Model
Model used for transcription (example shown:nova-3). -
Numerals (On/Off)
Converts written numbers into numeric format. -
Profanity filter (On/Off)
Filters profanity in output. -
Punctuate (On/Off)
Adds punctuation/capitalization. -
Redact
Indicates whether to redact sensitive information, replacing redacted content with entity tags. -
Smart Format (On/Off)
Smart Format improves readability by applying additional formatting. When enabled, punctuation and paragraph breaks will be applied as well as formatting of other entities, such as dates, times, and numbers.
¶ Google Speech To Text
-
Provider
Select the Google AI Provider entry (credentials). -
Language
Language used for transcription (example shown:English (United States)). -
Profanity filter (On/Off)
When enabled, profanity is censored. -
Automatic Punctuation (On/Off)
Adds punctuation to transcription output. -
Model
Model used for transcription (example shown:phone_call). -
Use Enhanced (On/Off)
Uses enhanced recognition models where applicable.
¶ Enabling Live Transcription on call flows
Live Transcription can be assigned to Extensions, Queues, and Enhanced Ring Groups (ERGs) by selecting the created Live Transcription configuration in the corresponding settings.
The Configuration tab on the main Live Transcription page also includes the Enable Live Transcription for internal calls option, which controls whether Live Transcription can be used for internal call scenarios.
The exact steps are documented in the Extension, Queue, and ERG sections of their respective manuals.
¶ Validate Live Transcription (what to check)
- Ensure Stereo Recording is enabled, and Call Recording is enabled on the selected Queue/ERG (prerequisites).
- Create and save a Live Transcription configuration (AI Service + Provider + WebSocket callback URL).
- Enable the configuration on a Queue or ERG (per the Queue/ERG documentation).
- Place a test call through that Queue/ERG.
- Verify your WebSocket callback server receives transcription events in real time.
If no events arrive:
- Confirm the callback URL uses
wss://and is reachable from PBXware. - Re-check the selected Provider credentials for the chosen AI Service.
- Confirm Stereo Recording is enabled and the call flow is using the correct configuration.
¶ Call Transcription
Call Transcription allows PBXware to generate a text transcript of recorded calls using a supported transcription provider. In the current implementation, PBXware supports two providers for Call Transcription: OpenAI and Hosted Whisper (the configuration fields are the same for both).
Important Note: Since Call Recording Transcriptions have been moved to the AI Hub, this is considered a breaking change. Users who had Call Recording Transcriptions configured before upgrading to PBXware 8.0 will need to reconfigure them. Additionally, the Use RAM disk option, located in server settings, must be set to Yes in order for Call Recording Transcriptions to function properly.
¶ Where to configure
To open Call Transcription, navigate to:
Select Sub-Tenant -> Home -> AI Hub -> Call Transcription
¶ Call Transcription settings
The Call Transcription page is divided into the following configuration groups:
- General
- Summary
- Options
¶ General
-
AI Provider
Select the provider that will be used for Call Transcription. -
Model
Select the transcription model available for the selected provider. -
Prompt
Optional text field used to influence transcription behavior, such as terminology, context, or expected wording that can improve recognition.
If not needed, leave it empty.
¶ Summary
The Summary section controls how AI-generated summaries (Smart Call Notes) are generated for completed call transcriptions.
-
Summary Generation
Select how call summaries are generated.Available options are:
- Disable - Summary generation is disabled.
- Automatic - PBXware automatically generates a summary after the transcription is completed.
- On-Demand - Users can manually generate summaries from the Call Transcription page in CDR reports.
-
Summary AI Provider
Select the AI provider that will be used for generating call summaries. -
Summary Model
The AI model used to generate call summaries. Select a supported model from the drop-down list or manually enter another model identifier supported by the selected AI provider.Examples of available models include:
gpt-4o,gpt-4o-mini,gpt-4-turbo,gpt-4,gpt-3.5-turbo,o1,o3-mini. -
Prompt
Instructions that tell the selected model how to process the completed call transcription and what kind of summary to generate.The prompt can define the required structure, tone, length, and information that should be included. For example:
Write a summary of the conversation.
IMPORTANT: The prompt should clearly instruct the model to generate a summary. If the instruction does not request a summary, the generated result may not be suitable for use as a Call Transcription Summary.
¶ Options
The following options control how call transcription can be used by users.
-
Allow Transcription in OSC (Yes/No)
Controls whether call transcription is available in OSC. -
Disable Transcription On-Demand (Yes/No)
Controls whether users can manually request transcriptions for recorded calls.- Yes - Manual transcription requests are disabled.
- No - Manual transcription requests are allowed.
-
Disable Automatic Transcription (Yes/No)
Controls whether PBXware automatically transcribes recorded calls.- Yes - Automatic transcription is disabled.
- No - Automatic transcription is enabled.
Click Save to apply changes.
¶ Viewing Call Transcription Summaries
When Call Transcription Summary is enabled, generated summaries can be viewed from the CDR page.
To access a summary:
Select Sub-Tenant -> Home -> Reports -> CDR
- Select the desired recorded call.
- Click Transcription.
- Open the Transcription Summary tab.
If Summary Generation is configured as On-Demand, click Request Summary to generate the summary.
If Summary Generation is configured as Automatic, the summary is generated automatically after the transcription has been completed and is displayed when available.
¶ Text to speech
Text to Speech (TTS) allows PBXware to generate spoken audio from text using a configured provider. Once enabled, PBXware exposes a Generate sound file option in the Sound Files area so you can create audio prompts directly from text.
Important Note: Since Text To Speech has been moved to the AI Hub, this is considered a breaking change. Users who had Text To Speech configured before upgrading to PBXware 8.0 will need to reconfigure it.
¶ Where to configure
To open Text to speech, navigate to:
Select Sub-Tenant -> Home -> AI Hub -> Text to speech
On the Text to speech page:
- Set Enable TTS to Yes
- In Provider, select your preferred AI Provider from the drop-down list
- Click Save to apply the configuration
¶ Where TTS is used (Generate sound files)
After enabling TTS, navigate to:
Select Sub-Tenant -> Home -> System -> Sound Files
A new toolbar option becomes available:
- Generate sound file (AI)
Use Generate sound file to create a new audio prompt from text using the selected TTS provider.
¶ Voicemail Transcription
Voicemail Transcription converts voicemail audio into text using the selected AI Provider.
Important Note: Since Voicemail Transcription has been moved to the AI Hub, this is considered a breaking change. Users who had Voicemail Transcription configured before upgrading to PBXware 8.0 will need to reconfigure it.
¶ Where to configure
To open Voicemail Transcription, navigate to:
Select Sub-Tenant -> Home -> AI Hub -> Voicemail Transcription
¶ General settings
The Voicemail Transcription page contains the following fields:
-
Enable Service
Enables or disables the Voicemail Transcription service. -
Transcribe by Default
Defines the default behavior for voicemail transcription. -
AI Provider
Select the provider that will be used for voicemail transcription. Supported providers are:Google SpeechOpenAIIBM (Deprecated)
The available fields below depend on the selected AI Provider.
¶ Google Speech configuration
When Google Speech is selected as the AI Provider, the following additional field is shown:
- Language
Select the language that will be used for voicemail transcription.
¶ IBM configuration (Deprecated)
When IBM is selected as the AI Provider, the following additional field is shown:
- Language
Select the language that will be used for voicemail transcription.
Deprecated: IBM Watson voicemail transcription is deprecated since PBXware 8.0.0.
We recommend migrating to another supported provider for new configurations.
¶ OpenAI configuration
When OpenAI is selected as the AI Provider, the following additional fields are shown:
-
Language
Select the language that will be used for voicemail transcription. -
Prompt
Defines additional instructions for how the voicemail transcription should be processed.
When finished, click Save.
¶ Enabling voicemail transcription on an Extension
After configuring the Voicemail Transcription service, voicemail transcription can be enabled on an Extension.
To do this, open the Extension and navigate to the Voicemail subsection:
Edit Extension -> Voicemail
Then set:
- Transcribe Content =
Yes
This enables voicemail transcription for that Extension.