ELITEA Toolkit Guide: TestRail Integration

Introduction

Purpose of this Guide

This guide is your definitive resource for integrating and utilizing the TestRail toolkit within ELITEA. It provides a step-by-step walkthrough, from setting up your TestRail API Key to configuring the toolkit in ELITEA and effectively using it within your Agents. By following this guide, you will unlock the power of automated test management, streamlined testing workflows, and enhanced team collaboration, all directly within the ELITEA platform. This integration empowers you to leverage AI-driven automation to optimize your software testing lifecycle using the combined strengths of ELITEA and TestRail.

Brief Overview of TestRail

TestRail is a comprehensive, web-based test management tool that helps teams of all sizes efficiently organize, manage, and track their software testing efforts. It provides a centralized platform to streamline your testing process, offering features for:

Integrating TestRail with ELITEA brings these robust test management capabilities directly into your AI-powered workflows. Your ELITEA Agents can then interact with your TestRail instance to automate test-related tasks, enhance testing processes, and improve team collaboration through intelligent automation.

TestRail Account Setup and Configuration

Account Setup

If you don't have a TestRail account yet, here’s how to get started with a free trial:

  1. Go to TestRail Website: Open your web browser and navigate to TestRail's official website.
  2. Start a Free Trial: On the homepage, locate and click the "Free Trial" button.
  3. Fill Out the Registration Form: Provide your details in the registration form. This typically includes your name, company email address, company name, and other relevant information. For professional use, it's highly recommended to use your company email.

  4. Submit the Form: Click on the “Start Free Trial” or similar button to submit the form. TestRail will send a confirmation email to the email address you provided.

    TestRail-Account_Form

  5. Verify Your Email: Open the confirmation email from TestRail and click on the verification link provided within the email. This step is crucial to activate your TestRail trial account.

    TestRail-Confirm_Your_Account_1

    TestRail-Confirm_Your_Account_2

  6. Access Your Account: After email verification, you will be redirected to your new TestRail instance or prompted to log in. Follow any on-screen instructions to complete the setup and access your TestRail account.

API Key Generation in TestRail

To securely integrate TestRail with ELITEA, you need to generate an API key within TestRail. This API key will be used by ELITEA to authenticate and access your TestRail instance.

Follow these steps to create an API Key:

  1. Log in to TestRail: Access your TestRail account using your credentials.

    TestRail-Welcome

  2. Access My Settings: Click on your profile name in the top right corner of the TestRail interface. From the dropdown menu, select "My Settings".

  3. Navigate to API Keys: In the "My Settings" page, find and click on the "API keys" tab.

  4. Add API Key: Click the "Add Key" button to create a new API key.

    TestRail-Add_API_Key

  5. Name the API Key: In the "Name" field, enter a descriptive name for the API key, such as "ELITEA Integration Key". This will help you identify the purpose of this key later.

  6. Generate API Key: Click the "Generate Key" button. TestRail will generate a new API key and display it to you.
  7. Securely Store Your API Key: Immediately copy the generated API key. This is the only time it will be displayed in full. Store it securely using a password manager or ELITEA's built-in Secrets feature (recommended for enhanced security within ELITEA). You will need this API Key to configure the TestRail toolkit in ELITEA.

TestRail Integration with ELITEA

Agent Creation/Configuration

To integrate TestRail, you'll need to configure it within an ELITEA Agent. You can either create a new Agent or modify an existing one.

  1. Navigate to Agents: In ELITEA, go to the Agents menu.
  2. Create or Edit Agent:
    • New Agent: Click "+ Agent" to create a new Agent. Follow the steps to define Agent details like name, description, type, and instructions.
    • Existing Agent: Select the Agent you want to integrate with TestRail and click on its name to edit.
  3. Access Tools Section: In the Agent configuration, scroll down to the "Tools" section.

Toolkit Configuration

This section details how to configure the TestRail toolkit within your ELITEA Agent.

  1. Add Toolkit: In the "Tools" section, click the "+" icon.
  2. Select TestRail: Choose "TestRail" from the dropdown list of available toolkits. This opens the "New TestRail tool" configuration panel.

  3. Configure Toolkit Details: Fill in the following fields:

    • Name: Enter a Name for this TestRail toolkit instance. Choose a descriptive name that helps you identify its purpose within your Agent's instructions (e.g., "TestRailQA", "TestManagementTool").
    • Description: Provide a Description for the toolkit, clarifying its purpose or the specific TestRail project it accesses (e.g., "Toolkit to access and manage test cases in TestRail").
    • URL: Enter the URL of your TestRail instance. It is crucial to use the correct format and your specific instance URL. For example: https://yourcompany.testrail.io/. Replace yourcompany with your actual TestRail subdomain.
    • Email: Enter the Email address associated with your TestRail account that you used to generate the API key.
    • Password/API token: Select "API token" as the authentication method.
      • Password/Secret: Choose "Secret" and select a pre-configured secret from the dropdown. You must first create and securely store your API Key as a Secret in ELITEA's Secrets feature. Using Secrets prevents hardcoding sensitive credentials directly in the toolkit configuration and enhances security. Alternatively, you can choose "Password" and paste the API Key you generated in TestRail into the "Password" field, but using Secrets is highly recommended for security best practices.

    TestRail-Toolkit_Configuration [Note: Replace with actual screenshot of TestRail Toolkit Configuration in ELITEA if available]

  4. Enable Tools: In the "Tools" section of the TestRail toolkit configuration, select the checkboxes next to the TestRail tools you want to enable for your Agent. Enable only the tools your Agent will actually use to adhere to the principle of least privilege and enhance security. Available tools are:

    • Get case: Retrieves a specific test case from TestRail.
    • Get cases: Retrieves a list of test cases from a TestRail project.
    • Get cases by filter: Retrieves test cases based on specified filter criteria.
    • Add case: Adds a new test case to TestRail.

  5. Complete Configuration: Click the arrow icon (at the top right of the toolkit configuration) to save the TestRail toolkit setup and return to the main Agent configuration.

  6. Click Save to apply configuration and changes to the Agent.

Tool Overview

The TestRail toolkit provides the following tools for your ELITEA Agents:

Instructions and Prompts for Using the Toolkit

To instruct your ELITEA Agent to use the TestRail toolkit, you need to provide clear instructions within the Agent's "Instructions" field. These instructions guide the Agent on when and how to use the available tools.

Instruction Creation for OpenAI Agents

When creating instructions for the TestRail toolkit for OpenAI-based Agents, focus on clear, action-oriented language. Break down tasks into simple steps and explicitly state the parameters required for each tool. OpenAI Agents respond best to instructions that are:

When instructing your Agent to use a TestRail toolkit, use this pattern:

1. Identify the goal: [State the objective, e.g., "To retrieve details of a specific test case"].
2. Tool Selection: Use the "[tool_name]" tool.
3. Parameter Specification: Provide the following parameters:
    - Parameter Name 1: <value or description of value>
    - Parameter Name 2: <value or description of value>
    - ...
4. Expected Outcome: [Optionally, describe what should happen after the tool is used].

Example Agent Instructions for TestRail Toolkit Tools (OpenAI Agent Friendly):

1. Goal: To get the details of test case with ID 2260 from TestRail.
2. Tool: Use the "get_case" tool.
3. Parameters:
    - Case ID: "2260"
4. Outcome: The Agent will retrieve and display the details of test case with ID 2260.

1. Goal: To create a new test case in TestRail project with ID 1, section ID 5, with title "Login Functionality Test" and priority "High".
2. Tool: Use the "add_case" tool.
3. Parameters:
    - Project ID: "1"
    - Section ID: "5"
    - Title: "Login Functionality Test"
    - Case Properties (JSON format): '{"priority_id": 1}'  (Note: priority_id 1 usually corresponds to High priority in TestRail, check your TestRail instance for specific IDs)
4. Outcome: A new test case named "Login Functionality Test" will be created in TestRail under project ID 1 and section ID 5 with High priority.

1. Goal: To get a list of all test cases from TestRail project with ID 2.
2. Tool: Use the "get_cases" tool.
3. Parameters:
    - Project ID: "2"
4. Outcome: The Agent will retrieve and display a list of all test cases in project ID 2.

1. Goal: To retrieve all test cases from project ID 1 that are of "Functional" type and have "High" priority.
2. Tool: Use the "get_cases_by_filter" tool.
3. Parameters:
    - Project ID: "1"
    - Filter Arguments (JSON format): '{"priority_id": 1, "type_id": 2}' (Note: priority_id 1 for High, type_id 2 for Functional are examples, check your TestRail instance for specific IDs)
4. Outcome: The Agent will retrieve and display a list of test cases from project ID 1 that match the specified filter criteria (Functional type and High priority).

Conversation Starters

Use these conversation starters to interact with your TestRail-integrated Agent.

1. For Testing and Troubleshooting Connection:

These starters are designed to verify if the TestRail toolkit is correctly configured and connected to your TestRail instance.

2. For General Agent Usage Scenarios:

These starters demonstrate how to initiate agent execution for common TestRail-related tasks.

These conversation starters provide a starting point for interacting with your TestRail-integrated ELITEA Agent and can be customized further based on your specific use cases and workflows.

Use Cases

The TestRail toolkit unlocks numerous automation possibilities for software testing workflows within ELITEA. Here are some key use cases, demonstrating how each tool can be applied:

Troubleshooting and Support

Troubleshooting

FAQ

  1. Q: Can I use my regular TestRail password instead of an API Key?
    • A: No, for secure integration with ELITEA, you must use a TestRail API Key. Using your regular password directly is not supported and is a security risk.
  2. Q: Where do I find Project IDs and Section IDs in TestRail?
    • A: Project IDs and Section IDs are typically visible in the URL when you navigate to a specific project or section within TestRail. For example, in the URL https://yourcompany.testrail.io/index.php?/projects/overview/1, 1 is likely the Project ID. Similarly, when viewing a section, the URL will contain a section identifier. You can also find these IDs through TestRail's API if needed, but checking the URL is usually the easiest method.
  3. Q: What if I don't know the exact IDs for priority or type when filtering test cases?
    • A: TestRail uses numerical IDs for various fields like priority and type. You need to find the correct IDs within your TestRail instance. Typically, you can find these IDs by inspecting the source code of the TestRail page when you are viewing or editing test cases, or by using TestRail's API to query available priorities and types. Contact your TestRail administrator if you are unsure about these IDs.

Support Contact

For any issues, questions, or further assistance with the TestRail integration or ELITEA Agents, please reach out to our dedicated ELITEA Support Team. We are committed to providing prompt and effective support to ensure your success with ELITEA.

Contact ELITEA Support:

Best Practices for Effective Support Requests:

To help us understand and resolve your issue as quickly as possible, please ensure you provide the following information in your support email:

Before Contacting Support:

We encourage you to first explore the resources available within this guide and the broader ELITEA documentation. You may find answers to common questions or solutions to known issues in the documentation.

To further enhance your understanding and skills in integrating TestRail with ELITEA, here are some helpful resources: