Settings

The Settings is designed to offer you a centralized space to manage vital aspects of your account and configurations. This centralized space is accessible by clicking on your avatar located at the top right corner of the page.

The Settings consists of several tabs and settings each dedicated to specific functionalities:

Navigation:

To navigate through the Settings menus, follow these steps:

  1. Click on the Your Avatar icon located at the top right corner of the page to open the Sidebar menu.
  2. Select the desired tab by clicking on its name to navigate to that specific section.

Profile

In the Profile, you’re presented with options to personalize your account within ELITEA.

About me - fill in or update your personal details to ensure your ELITEA profile remains current. Note: Markdown is supported.

Intro-Settings_Profile

Monitoring

The Monitoring feature in ELITEA is designed to provide a comprehensive overview of the application's usage and performance. This feature is essential for administrators and users who want to gain insights into various aspects of the application, from user engagement to the effectiveness of configured artifacts like prompts, datasources, and agents. By leveraging the detailed charts and statistics available within the Monitoring feature, you can make informed decisions to optimize the performance and user experience of your ELITEA application.

This section will guide you through the various components of the Monitoring feature, including configuration options, key metrics, adoption and usage statistics, sentiment analysis, accuracy metrics, prompt topics, and topics summary. Each of these components offers valuable insights that can help you understand how the application is being used and how it can be improved.

Configuration Options

At the top of the Monitoring page, you have several options and settings to configure the charts and metrics you wish to monitor:

To apply any changes or selections, click the Apply button. Use the Refresh button to update the monitoring data based on the latest activities and configurations.

Key Metrics

Below the configuration options, you'll find an overview of key metrics that give you a snapshot of the current state of the system:

These metrics are accompanied by a date range selector, allowing you to filter the data for a specific period.

Intro-Settings_Monitoring(Part1)

Adoption and Usage

Below the key metrics, you'll find the Adoption and Usage section, which includes:

These charts provide insights into user engagement and the application's token economy.

Acceptance Rate

The Acceptance Rate section provides a comprehensive view of user interactions with the ELITEA application, focusing on how often users accept the generated outputs. This section includes visualizations that help you understand user satisfaction and the effectiveness of the system's responses.

Acceptance Rate Chart

The Acceptance Rate Chart displays the number of accepted and not accepted interactions for the selected period and filter options. This chart helps you assess how frequently users find the generated outputs useful enough to accept by copying, downloading, or saving them.

By analyzing the Acceptance Rate Chart, you can gain insights into user satisfaction levels and identify opportunities to enhance the effectiveness of the ELITEA application, ultimately improving the overall user experience.

Sentiments

The Sentiments section provides a visual representation of the emotional tone of both user inputs and the outputs generated by LLMs. Understanding sentiment is crucial for tailoring responses to better meet user needs and improve overall interaction quality.

Sentiment Analysis Overview

Sentiment analysis categorizes text into three primary emotional states:

ELITEA performs sentiment analysis on user inputs to gauge the user's emotional state. This capability is particularly important for providing high-quality customer service, as the LLM can adjust its response tone and content based on the user's emotions.

Visual Representation

The Sentiments section includes two pie charts that offer a clear visual representation of sentiment distribution:

Practical Applications

Understanding sentiment can significantly enhance the user experience in several ways:

Intro-Settings_Monitoring(Part2)

Accuracy

The Accuracy section provides detailed insights into the performance and reliability of the ELITEA application. This section includes various metrics and visualizations that help you understand how well the system is responding to user inputs and how effective your configured artifacts (prompts, datasources, agents, conversations) are.

Relevance

The Relevance metric is divided into two key lines:

Note: The maximum value for relevance is 6. The higher the score, the better the relevance, indicating a more accurate and contextually appropriate interaction.

Reliability

The Reliability Score answers the question of whether there is enough context to respond accurately to the user's questions or queries. This metric helps you gauge the confidence level of the LLM's responses. Note: The maximum reliability score is 10. A higher score indicates that there is sufficient context to provide a correct and reliable response to the user's query.

Instruction Quality vs Usage

The Instruction Quality vs Usage is a 2x2 matrix that helps you evaluate the effectiveness and utilization of your artifacts (prompts, agents, datasources or conversations):

Matrix legend:

By analyzing these metrics, you can make informed decisions to improve the accuracy and reliability of your Alita AI application, ensuring a better user experience.

Prompt Topics

The Prompt Topics section provides an automatic classification of the available artifacts (prompts, datasources, and agents) within the Alita AI application. This section helps you understand the distribution and focus areas of your artifacts, enabling you to identify trends and gaps in your content.

Chart Components

The Prompt Topics section displays a clustered column chart that categorizes your artifacts by topic. This visual representation allows you to quickly see how many prompts, datasources, or agents are associated with each topic. The clustered column chart includes the following components:

Practical Applications

Understanding the distribution of your artifacts across different topics can provide several benefits:

Intro-Settings_Monitoring(Part3)

By leveraging the insights provided by the Prompt Topics section, you can ensure that your application covers a comprehensive range of topics, enhancing the overall user experience and effectiveness of the application.

Topics Summary

The Topics Summary section provides an automatic classification of user inputs, categorizing the topics that users have queried or questioned about. This section helps you understand user interests and the most frequently discussed topics within ELITEA.

Chart Components

The Topics Summary section displays a clustered column chart that categorizes user inputs by topic. This visual representation allows you to quickly see how many times users have queried information for each topic within a selected timeframe. The clustered column chart includes the following components:

Practical Applications

Understanding the distribution of user queries across different topics can provide several benefits:

By leveraging the insights provided by the Topics Summary section, you can ensure that ELITEA is aligned with user interests, enhancing the overall user experience and effectiveness of the application.

Intro-Settings_Monitoring(Part4)

The Monitoring feature in ELITEA offers a robust set of tools and metrics to help you understand the performance and usage of your application. By utilizing the various charts and statistics available, you can gain valuable insights into user engagement, sentiment, accuracy, and the distribution of topics within your project. These insights are crucial for making data-driven decisions that can enhance the overall user experience and effectiveness of your ELITEA application.

Whether you are looking to improve customer's experience through sentiment analysis, optimize the relevance and reliability of LLM responses, or identify content gaps and user interests, the Monitoring feature provides the necessary data to guide your efforts. By regularly reviewing and analyzing these metrics, you can ensure that your project remains aligned with user needs and continues to perform at its best. By leveraging the comprehensive monitoring capabilities of ELITEA, you can create a more responsive, efficient, and user-friendly application, ultimately leading to higher user satisfaction and better overall performance.

Configuration

The Configuration page serves as the nucleus for managing essential technical settings that enable the smooth operation of ELITEA's features, such as Alita Code or Alita Code Chat. Note: The available settings and configurations may vary depending on the project selected. It's important to select the appropriate project from the project dropdown menu to access specific configurations relevant to that project.

To create a token:

  1. Click the + icon to create a new token.
  2. Enter a name and set an expiration date for the token.
  3. Click Generate to create the token.
  4. Immediately copy and securely store the generated token; it will not be visible again once you close the pop-up window.
  5. From the Integration Option dropdown list, select the desired LLM model (e.g., gpt-4, gpt-4o, gpt-4-0125-preview, etc.).
  6. Once the LLM model is selected, the Download VS Code Settings and the Download Jetbrains Settings icons will appear next to the created token. This allows you to download the configuration files to integrate and configure the ELITEA project with Alita Code extensions on VSCode and IntelliJ respectively. For more information about how to setup it, please refer to the Alita Code Documentation.

Intro-Settings_Configuration

Deployments

The Deployments page is all about managing and launching AI models or services that you’ve connected to your ELITEA project. Note: The available settings and configurations may vary depending on the project selected. It's important to select the appropriate project from the project dropdown menu to access specific configurations relevant to that project.

Creating a New Deployment for EPAM AI Dial

To set up a new deployment for EPAM AI Dial within your system, follow these detailed steps:

  1. Obtain API Key: Important: Before proceeding, you must obtain a separate API Key from the EPAM DIAL team. This key is essential for authenticating and enabling communication with the AI Dial services.
  2. API Key Retrieval: Once you have received the API Key and any additional required information via email, return to this page to input these details.
  3. Initiate Deployment Creation: Click the + icon to start creating a new deployment.
  4. Select Deployment Type: From the list of available deployment types, select AI Dial.
  5. Configure Deployment Details: In the configuration window, fill in the following information:
    • Name: Enter a descriptive name for the deployment. This name will be displayed alongside LLM models configured with this deployment.
    • API Base: For EPAM AI Dial, use https://ai-proxy.lab.epam.com as the API Base.
    • Secret API Key: Paste the API Key that you received from the AI Dial team.
    • API Version: Enter the API version information provided by the AI Dial team.
  6. Add Models to Deployment:
    • Click the + icon to add one or more models associated with this deployment. For each model, provide the model's name, maximum input tokens, and capabilities.
    • Important: Ensure that you enter the correct model name as used in EPAM AI DIAL. For detailed information on model specifications and configurations, refer to the EPAM AI Dial documentation.
  7. Click Save to complete the creation of the deployment.

By following these steps, you can successfully create and configure a new deployment for EPAM AI Dial, enabling you to leverage advanced capabilities within your projects.

Intro-Settings_Deployments

Integrations

The Integrations menu in ELITEA is designed to enhance the platform's functionality and flexibility by allowing users to connect with essential external tools such as Jira, Confluence, Testrail, and GitHub. These integrations enable seamless data flow and collaboration across different platforms, enhancing productivity and efficiency. Once configured, these integrations can be selected as configurations in the Agent's tool setup for each corresponding tool, allowing for streamlined operations within ELITEA. You can create integrations in both Private workspaces and Team projects, providing versatility in managing your connections.

Confluence Integration Setup

To set up a Confluence integration, follow these step-by-step instructions:

  1. Initiate New Integration: Click the + icon to start the process of creating a new integration. A pop-up window will appear, prompting you to select the type of integration you wish to create.
  2. Select Confluence: From the available options, select Confluence. This will open a new pop-up window where you can enter the necessary details for the integration.
  3. Enter Integration Details:
    • Name: Provide a descriptive name for the integration, such as "Conf_Integration".
    • URL: Enter the Confluence URL for your organization, e.g., https://kb.epam.com/.
    • Authentication Options: Choose your preferred method for secure connection:
      • API Key: Select this option if you are using an API key for authentication. You have two choices for providing the necessary credentials:
        • Password: Enter your API key value directly into the provided field.
        • Secret: Select a pre-configured secret from the dropdown list. This secret should have been set up previously in Secrets page for secure storage and retrieval.
      • Username: Additionally, you will need to input the Username associated with your Confluence account to complete the authentication process.
      • Token: Select this option if you are using a token for authentication. Similar to the API key, you have two choices for providing the necessary credentials:
        • Password: Enter the your token value directly into the provided field.
        • Secret: Select a pre-configured secret from the dropdown list. This secret should have been set up previously in Secrets page for secure storage and retrieval.
      • Hosting Option: Select the appropriate hosting type for your Jira setup:
      • Cloud: If your Confluence is hosted on Atlassian’s cloud.
      • Server: If your Confluence is hosted on your own servers or an enterprise environment. Important Note: When connecting to Epam's Confluence, ensure you select the Server option to establish the correct configuration.
    • Set as Default: Check the checkbox to set this integration as the default.
  4. Save the Integration: Click the Save button to finalize the integration setup. Your Confluence integration is now configured and ready to use.

GitHub Integration Setup

For GitHub integration, choose from the following authentication options:

  1. Initiate New Integration: Click the + icon to start the process of creating a new integration. A pop-up window will appear, prompting you to select the type of integration you wish to create.
  2. Select Confluence: From the available options, select GitHub. This will open a new pop-up window where you can enter the necessary details for the integration.
  3. Enter Integration Details:
    • Name: Provide a descriptive name for the integration, such as "GitHub_Integration".
    • Authentication Options: Choose your preferred method for secure connection:
      • Private Key: Select this option if you are using an Private key for authentication.
        • App ID: Enter the App ID associated with your GitHub integration.
        • Private Key: Enter the configured Private key. You have two choices for providing the necessary credentials:
          • Password: Enter your Private key value directly into the provided field.
          • Secret: Select a pre-configured secret from the dropdown list. This secret should have been set up previously in Secrets page for secure storage and retrieval.
      • Token: Select this option if you are using a token for authentication. Similar to the Private key, you have two choices for providing the necessary credentials:
        • Password: Enter the your token value directly into the provided field.
        • Secret: Select a pre-configured secret from the dropdown list. This secret should have been set up previously in Secrets page for secure storage and retrieval.
      • Password: Select this option if you are using your GitHub account password for authentication.
        • Password: Enter the password associated with your GitHub account. Similar to previous options, you have two choices for providing the necessary credentials:
          • Password: Enter your password value directly into the provided field.
          • Secret: Select a pre-configured secret from the dropdown list. This secret should have been set up previously in Secrets page for secure storage and retrieval.
        • Username: Additionally, you will need to input the Username associated with your GitHub account to complete the authentication process.
      • Anonymous: Select this option, if no anuthentication is required.
    • Set as Default: Check the checkbox to set this integration as the default.
  4. Save the Integration: Click the Save button to finalize the integration setup. Your GitHub integration is now configured and ready to use.

Jira Integration Setup

To set up a Jira integration, follow these step-by-step instructions:

  1. Initiate New Integration: Click the + icon to start the process of creating a new integration. A pop-up window will appear, prompting you to select the type of integration you wish to create.
  2. Select Jira: From the available options, select Jira. This will open a new pop-up window where you can enter the necessary details for the integration.
  3. Enter Integration Details:
    • Name: Provide a descriptive name for the integration, such as "Jira_Integration".
    • URL: Enter the Jira URL for your organization, e.g., https://jiraeu.epam.com/.
    • Authentication Options: Choose your preferred method for secure connection:
      • API Key: Select this option if you are using an API key for authentication. You have two choices for providing the necessary credentials:
        • Password: Enter your API key value directly into the provided field.
        • Secret: Select a pre-configured secret from the dropdown list. This secret should have been set up previously in Secrets page for secure storage and retrieval.
      • Username: Additionally, you will need to input the Username associated with your Jira account to complete the authentication process.
      • Token: Select this option if you are using a token for authentication. Similar to the API key, you have two choices for providing the necessary credentials:
        • Password: Enter the your token value directly into the provided field.
        • Secret: Select a pre-configured secret from the dropdown list. This secret should have been set up previously in Secrets page for secure storage and retrieval.
    • Hosting Option: Select the appropriate hosting type for your Jira setup:
      • Cloud: If your Jira is hosted on Atlassian’s cloud.
      • Server: If your Jira is hosted on your own servers or an enterprise environment. Important Note: When connecting to Epam's Jira, ensure you select the Server option to establish the correct configuration.
    • Set as Default: Check the checkbox to set this integration as the default.
  4. Save the Integration: Click the Save button to finalize the integration setup. Your Jira integration is now configured and ready to use.

TestRail Integration Setup

To set up a Testrail integration, provide the following details:

  1. Initiate New Integration: Click the + icon to start the process of creating a new integration. A pop-up window will appear, prompting you to select the type of integration you wish to create.
  2. Select Jira: From the available options, select TestRail. This will open a new pop-up window where you can enter the necessary details for the integration.
  3. Enter Integration Details:
    • Name: Provide a descriptive name for the integration, such as "TestRail_Integration".
    • URL: Enter the TestRail URL for your organization, e.g., https://testrail.epam.com/.
    • Email: Enter the email used for authentication.
    • Authentication Options: Choose your preferred method for secure connection:
      • Password: Enter your password value directly into the provided field.
      • Secret: Select a pre-configured secret from the dropdown list. This secret should have been set up previously in Secrets page for secure storage and retrieval.
    • Set as Default: Check the checkbox to set this integration as the default.
  4. Save the Integration: Click the Save button to finalize the integration setup. Your TestRail integration is now configured and ready to use.

By setting up these integrations, you can streamline workflows and enhance collaboration across different platforms, making ELITEA a more powerful tool for your projects.

Projects

The Projects menu is specifically designed for administrators and becomes accessible when you hold administrative privileges in any project other than your private project within ELITEA. Note: It's important to select the appropriate project from the project dropdown menu to access specific configurations relevant to that project.

Groups

The Groups feature in ELITEA is designed to facilitate efficient management and monitoring of multiple projects by admins or managers. This feature allows you to consolidate several projects under a single group, making it easier to oversee and coordinate activities across these projects. If you are an admin of two or more projects, you can leverage the Groups feature to organize and monitor your projects collectively:

This grouping functionality not only simplifies the administrative workload but also enhances the visibility and control over multiple projects, enabling more effective management and monitoring.

Teammates

The Teammates feature in ELITEA is specifically crafted to streamline the process of collaborating within projects by allowing you to invite new users (teammates) and assign them appropriate roles. These roles include system, admin, editor, and viewer, each providing different levels of access and control within the project. Note: Only users with an admin role are empowered to invite new members. This ensures that the invitation and role assignment process is managed by users with appropriate authority and understanding of the project’s needs.

Inviting New Teammates:

  1. Enter the prospective member's email address in the Email Address input field.
  2. Select their role from the Role dropdown menu.
  3. Click the Invite button.
  4. An invitation will be sent, and upon their first login, their details will be added to the Teammates, activating their account.

Intro-Settings_Projects

Notes

Managing Teammates:

The Teammates table displays all members, their roles, and their last login information. Admins can modify a user's role or revoke access by clicking the respective Edit or Delete icons next to a user's name.

Secrets

The Secrets feature in ELITEA serves as a secure vault designed to store and manage sensitive information such as passwords, tokens, API keys, and other authentication details. This centralized system allows you to configure secrets once and utilize them across various components, such as Agent's toolkits within ELITEA.

Creating a Secret:

To add a new secret to the vault, follow these steps:

  1. Click the + icon to initiate the creation of a new secret.
  2. Enter a descriptive name for the secret to help you identify its use.
  3. In the Value field, input the token, password, API key, or any other authentication details.
  4. Once configured, this secret can now be selected and used within various components of ELITEA.

Managing Secrets:

The management of secrets is straightforward and secure, facilitated by the Secrets table which displays all your configured secrets:

This feature enhances the security and efficiency of managing sensitive information within ELITEA, ensuring that authentication details are handled in a secure, centralized manner.

Intro-Settings_Secrets