23 KiB
Setting up baibot (optional)
baibot
🤖 baibot (pronounced bye-bot) is a Matrix bot developed by etke.cc that exposes the power of AI / Large Language Models to you. 🤖
It supports OpenAI's ChatGPT models, as many well as other ☁️ providers.
It's designed as a more private and ✨ featureful alternative to matrix-chatgpt-bot. See the baibot project and its documentation for more information.
Prerequisites
API access to one or more LLM ☁️ providers.
Adjusting the playbook configuration
There are a lot of configuration options (some required, some possibly required, some optional), so they're split into multiple sections below:
- Base configuration
- 👮♂️ Administrator configuration
- 👥 Initial users configuration
- 🤖 Configuring agents via Ansible
- 🤝 Configuring initial default handlers
Depending on your current vars.yml file and desired configuration, you may require more than just the base configuration.
Base configuration
Add the following configuration to your inventory/host_vars/matrix.DOMAIN/vars.yml file:
matrix_bot_baibot_enabled: true
# Uncomment and adjust this part if you'd like to use a username different than the default
# matrix_bot_baibot_config_user_mxid_localpart: baibot
# Generate a strong password here. Consider generating it with `pwgen -s 64 1`.
# If you'd like to change this password subsequently, see the details below.
matrix_bot_baibot_config_user_password: 'PASSWORD_FOR_THE_BOT'
# An optional passphrase to use for backing up and recovering the bot's encryption keys.
# You can use any string here. Consider generating it with `pwgen -s 64 1`.
#
# If set to null, the recovery module will not be used and losing your session/database
# will mean you lose access to old messages in encrypted room.
# It's highly recommended that you configure this to avoid losing access to encrypted messages.
#
# Changing this subsequently will also cause you to lose access to old messages in encrypted rooms.
# For details about changing this subsequently or resetting, see `defaults/main.yml` in the baibot role.
matrix_bot_baibot_config_user_encryption_recovery_passphrase: 'ANY_LONG_AND_SECURE_PASSPHRASE_STRING_HERE'
# An optional secret for encrypting the bot's session data (see `matrix_bot_baibot_data_path`).
# This must be 32-bytes (64 characters when HEX-encoded).
# Generate it with: `openssl rand -hex 32`
# Set to null or empty to avoid using encryption.
# Changing this subsequently requires that you also throw away all data (see `matrix_bot_baibot_data_path`)
matrix_bot_baibot_config_persistence_session_encryption_key: 'A_HEX_STRING_OF_64_CHARACTERS_HERE'
# An optional secret for encrypting bot configuration stored in Matrix's account data.
# This must be 32-bytes (64 characters when HEX-encoded).
# Generate it with: `openssl rand -hex 32`
# Set to null or empty to avoid using encryption.
# Changing this subsequently will make you lose your configuration.
matrix_bot_baibot_config_persistence_config_encryption_key: 'A_HEX_STRING_OF_64_CHARACTERS_HERE'
As mentioned above, this may not be enough. Continue with the configuration sections below.
👮♂️ Administrator configuration
This is an addition to the base configuration.
To specify who is considered a bot 👮♂️ Administrator, you either need to specify matrix_bot_baibot_config_access_admin_patterns or matrix_admin. The latter is a single variable which affects all bridges and bots.
If matrix_admin is already configured in your vars.yml configuration, you can skip this section.
If necessary, add the following configuration to your inventory/host_vars/matrix.DOMAIN/vars.yml file:
# Uncomment to add one or more admins to this bridge:
#
# matrix_bot_baibot_config_access_admin_patterns:
# - "@*:example.com"
# - "@admin:another.com"
#
# .. unless you've made yourself an admin of all bots/bridges like this:
#
# matrix_admin: '@yourAdminAccount:domain.com'
👥 Initial users configuration
By default, all users on your homeserver are considered allowed users. If that's OK, you can skip this section.
This is an addition to the base configuration.
To specify who is considered a bot 👥 User, you may:
- define an initial value for
matrix_bot_baibot_config_initial_global_config_user_patternsAnsible variable, as shown below - configure the list at runtime via the bot's
!bai access set-users SPACE_SEPARATED_PATTERNScommand
Configuring matrix_bot_baibot_config_initial_global_config_user_patterns is optional, but it can be useful to pre-configure the bot with a list of users who should have access to the bot's features.
Note: Once initially configured, the allowed users list cannot be managed via Ansible anymore. It can only be managed subsequently via bot commands.
If necessary, add the following configuration to your inventory/host_vars/matrix.DOMAIN/vars.yml file:
# Uncomment and adjust the bot users if necessary:
#
# Subsequent changes to `matrix_bot_baibot_config_initial_global_config_user_patterns` do not affect the bot's behavior.
# Once initially configured, the allowed users list is managed via bot commands, not via Ansible.
#
# matrix_bot_baibot_config_initial_global_config_user_patterns:
# - "@*:{{ matrix_bot_baibot_config_homeserver_server_name }}"
🤖 Configuring agents via Ansible
You are not required to define agents statically via Ansible. To get started quickly, you can skip this section and define agents at runtime via chat commands (following the bot's guidance).
Privileged users (like the 👮♂️ Administrator, but potentially others too - see the upstream 🔒 access documentation) can define agents dynamically at any time via chat commands.
The Ansible role includes preset variables for easily enabling some 🤖 agents on various ☁️ providers (e.g. OpenAI, etc).
Besides the presets, the Ansible role also includes support for configuring additional statically-defined agents via the matrix_bot_baibot_config_agents_static_definitions_custom Ansible variable.
Agents defined statically and those created dynamically (via chat) are named differently, so conflict cannot arise.
Depending on your propensity for GitOps, you may prefer to define agents statically via Ansible, or you may wish to do it dynamically via chat.
Before proceeding, we recommend reading the upstream documentation on How to choose a provider. In short, it's probably best to go with OpenAI.
Anthropic
You can statically-define a single 🤖 agent instance powered by the Anthropic provider with the help of the playbook's preset variables.
Here's an example addition to your vars.yml file:
matrix_bot_baibot_config_agents_static_definitions_anthropic_enabled: true
matrix_bot_baibot_config_agents_static_definitions_anthropic_config_api_key: "YOUR_API_KEY_HERE"
# If you'd like to use another text-generation agent, uncomment and adjust:
# matrix_bot_baibot_config_agents_static_definitions_anthropic_config_text_generation_model_id: claude-3-5-sonnet-20240620
# See `defaults/main.yml` in the baibot role for more configuration options.
If you'd like to use more than one model, take a look at the Configuring additional agents (without a preset) section below.
💡 You may also wish to use this new agent for 🤝 Configuring initial default handlers.
Groq
You can statically-define a single 🤖 agent instance powered by the Groq provider with the help of the playbook's preset variables.
Here's an example addition to your vars.yml file:
matrix_bot_baibot_config_agents_static_definitions_groq_enabled: true
matrix_bot_baibot_config_agents_static_definitions_groq_config_api_key: "YOUR_API_KEY_HERE"
# Specify the text-generation agent you'd like to use
matrix_bot_baibot_config_agents_static_definitions_groq_config_text_generation_model_id: "llama3-70b-8192"
# Uncomment and adjust if you're not happy with these speech-to-text defaults:
#
# matrix_bot_baibot_config_agents_static_definitions_groq_config_speech_to_text_enabled: true
# matrix_bot_baibot_config_agents_static_definitions_groq_config_speech_to_text_model_id: whisper-large-v3
# See `defaults/main.yml` in the baibot role for more configuration options.
Because this is a statically-defined agent, it will be given a static/ ID prefix and will be named static/groq.
If you'd like to use more than one model, take a look at the Configuring additional agents (without a preset) section below.
💡 You may also wish to use this new agent for 🤝 Configuring initial default handlers.
Mistral
You can statically-define a single 🤖 agent instance powered by the 🇫🇷 Mistral provider with the help of the playbook's preset variables.
Here's an example addition to your vars.yml file:
matrix_bot_baibot_config_agents_static_definitions_mistral_enabled: true
matrix_bot_baibot_config_agents_static_definitions_mistral_config_api_key: "YOUR_API_KEY_HERE"
# Uncomment and adjust if you're not happy with these defaults:
# matrix_bot_baibot_config_agents_static_definitions_mistral_config_text_generation_model_id: mistral-large-latest
# See `defaults/main.yml` in the baibot role for more configuration options.
Because this is a statically-defined agent, it will be given a static/ ID prefix and will be named static/mistral.
If you'd like to use more than one model, take a look at the Configuring additional agents (without a preset) section below.
💡 You may also wish to use this new agent for 🤝 Configuring initial default handlers.
OpenAI
You can statically-define a single 🤖 agent instance powered by the OpenAI provider with the help of the playbook's preset variables.
The OpenAI provider is only meant to be used with OpenAI's official API and compatibility with other services (which do not fully adhere to the OpenAI API spec completely) is limited. If you're targeting an OpenAI-compatible service, use the OpenAI Compatible provider instead.
Here's an example addition to your vars.yml file:
matrix_bot_baibot_config_agents_static_definitions_openai_enabled: true
matrix_bot_baibot_config_agents_static_definitions_openai_config_api_key: "YOUR_API_KEY_HERE"
# If you'd like to use another text-generation agent, uncomment and adjust:
# matrix_bot_baibot_config_agents_static_definitions_openai_config_text_generation_model_id: gpt-4o
# See `defaults/main.yml` in the baibot role for more configuration options.
Because this is a statically-defined agent, it will be given a static/ ID prefix and will be named static/openai.
If you'd like to use more than one model, take a look at the Configuring additional agents (without a preset) section below.
💡 You may also wish to use this new agent for 🤝 Configuring initial default handlers.
OpenAI Compatible
You can statically-define a single 🤖 agent instance powered by the OpenAI Compatible provider with the help of the playbook's preset variables.
This provider allows you to use OpenAI-compatible API services like OpenRouter, Together AI, etc.
Some of these popular services already have shortcut providers (see supported providers leading to this one behind the scenes - this make it easier to get started.
As of this moment, the playbook does not include presets for any of these services, so you'll need to Configuring additional agents (without a preset).
Configuring additional agents (without a preset)
The Ansible role may be lacking preset variables for some ☁️ provider, or you may wish to statically-define an agent on the same provider twice (or more) with different configuration.
It's possible to inject your own agent configuration using the matrix_bot_baibot_config_agents_static_definitions_custom Ansible variable.
You can also define providers at runtime, by chatting with the bot, so using Ansible is not a requirement.
Below is an an example demonstrating statically-defining agents via Ansible without using presets:
matrix_bot_baibot_config_agents_static_definitions_custom:
# This agent will use the GPT 3.5 model and will only support text-generation,
# even though the `openai` provider could support other features (e.g. image-generation).
- id: my-openai-gpt-3.5-turbo-agent
provider: openai
config:
base_url: https://api.openai.com/v1
api_key: "YOUR_API_KEY_HERE"
text_generation:
model_id: gpt-3.5-turbo-0125
prompt: You are a brief, but helpful bot.
temperature: 1.0
max_response_tokens: 4096
max_context_tokens: 16385
speech_to_text: null
text_to_speech: null
image_generation: null
# This agent uses the `openai` provider, but adjusts the base URL, so that it points to some Ollama instance
# (which supports an OpenAI-compatible API).
- id: my-ollama-agent
provider: openai
config:
base_url: http://ollama-service:1234/v1
api_key: ""
text_generation:
model_id: "llama3.1:8b"
prompt: "You are an assistant based on the Llama3.1:8b model. Be brief in your responses."
temperature: 1.0
max_response_tokens: 4096
max_context_tokens: 128000
speech_to_text: null
text_to_speech: null
image_generation: null
Because these are statically-defined agents, they will be given a static/ ID prefix and will be named static/my-openai-gpt-3.5-turbo-agent and static/my-ollama-agent, respectively.
💡 To figure out what to put in the config section, refer to the ☁️ provider page, which contains sample configuration YAML for each provider.
As with any 🤖 agent, defining them means they exist. To actually make use of them, they need to be configured as handlers globally or in a specific room - see Mixing & matching models.
💡 You may also wish to use these new agents for 🤝 Configuring initial default handlers.
🤝 Configuring initial default handlers
This section is only useful if you're 🤖 Configuring agents via Ansible, as it lets you put these agents to use as soon as the bot starts (by adjusting the bot's initial global configuration).
If you're not configuring agents via Ansible, you can skip this section.
This section is only useful the first time around. Once initially configured the global configuration cannot be managed Ansible, but only via bot commands.
baibot supports various purposes:
-
💬 text-generation: communicating with you via text
-
🦻 speech-to-text: turning your voice messages into text
-
🗣️ text-to-speech: turning bot or users text messages into voice messages
-
🖌️ image-generation: generating images based on instructions
-
❓ catch-all: special purposes, indicating use as a fallback (when no specific handler is configured)
Mixing & matching models is made possible by the bot's ability to have different 🤝 handlers configured for different purposes.
This configuration can be done as a global fallback, or per-room. Both of these 🛠️ configurations are managed at runtime (viat chat), but the global configuration can have some initial defaults configured via Ansible.
You can configure the initial values for these via Ansible, via the matrix_bot_baibot_config_initial_global_config_handler_* variables.
Example additional vars.yml configuration:
# NOTE: these are initial defaults for the bot's global configuration.
# As such, changing any of these values subsequently has no effect on the bot's behavior.
# Once initially configured, the global configuration is managed via bot commands, not via Ansible.
matrix_bot_baibot_config_initial_global_config_handler_catch_all: static/openai
# In this example, there's no need to define any of these below.
# Configuring the catch-all purpose handler is enough.
matrix_bot_baibot_config_initial_global_config_handler_text_generation: null
matrix_bot_baibot_config_initial_global_config_handler_text_to_speech: null
matrix_bot_baibot_config_initial_global_config_handler_speech_to_text: null
matrix_bot_baibot_config_initial_global_config_handler_image_generation: null
Note: these are initial defaults for the bot's global configuration. As such, changing any of these values subsequently has no effect on the bot's behavior. Once initially configured the global configuration cannot be managed Ansible, but only via bot commands.
Installing
After configuring the playbook, run the installation command again:
just run-tags install-all,ensure-matrix-users-created,start
Notes:
-
the
ensure-matrix-users-createdplaybook tag makes the playbook automatically create the bot's user account -
if you change the bot password (
matrix_bot_baibot_config_user_passwordin yourvars.ymlfile) subsequently, the bot user's credentials on the homeserver won't be updated automatically. If you'd like to change the bot user's password, use a tool like synapse-admin to change it, and then updatematrix_bot_baibot_config_user_passwordto let the bot know its new password
Usage
To use the bot, invite the @baibot:DOMAIN bot user into a room.
If you're an allowed bot 👥 user (see 👥 Initial users configuration), the bot will accept your invitation and join the room.
After joining, the bot will introduce itself and show information about the ✨ features that are enabled for it.
If you've 🤖 configured one or more agents via Ansible and have 🤝 configured initial default handlers, the bot will immediately be able to make use of these agents for this new room. Otherwise, you will need to configure agents and/or handlers via chat commands.
Send !bai help to the room at any time to see the bot's help menu for additional commands.
You can also refer to the upstream baibot project's documentation.
Debugging
As with all other services, you can find service logs in systemd-journald by running something like journalctl -fu matrix-bot-baibot
The default logging level for this service is info, but you can increase it to debug (or even trace) with the following additional configuration:
# Adjust the bot's own logging level.
matrix_bot_baibot_config_logging_level_baibot: debug
# Adjust the logging level for the mxlink bot library used by the bot.
matrix_bot_baibot_config_logging_level_mxlink: debug
# Adjust the logging level for other libraries used by the bot.
# Having this set to a value other than "warn" can be very noisy.
matrix_bot_baibot_config_logging_level_other_libs: debug
Alternatively, you can use a single variable to set the logging level for all of the above (bot + all libraries):
matrix_bot_baibot_config_logging: debug