Operations Guide

Reference

Introduction

The Code Companion component is part of Software Factory as a Package (SWaaP).

Code Companion (previously known as Hermes) is a cloud-based artificial intelligence extension for Visual Studio Code developed by the LaetitiA squad to assist users in their work by generating code, unit tests or comments. It also allows the user to chat with the AI and ask for help with tasks such as explaining code, refactoring code, or detecting vulnerabilities and logic flaws, just to name a few.

Learn more about it in LLD .

Service Catalog Items

Onboarding

The onboarding process is designed to provision access and necessary configurations for new users of the tool.
It is initiated by a TDF/Engineering account participant and must be validated by the TDF/Engineering account owner.

Steps

• A list of users to be onboarded is submitted by a TDF/Engineering participant.
• The onboarding request is reviewed and validated by the TDF/Engineering account owner.
• Upon approval, access is granted, and the user is notified.

Responsibilities

• Initiator: TDF/Engineering account participant
• Validator: TDF/Engineering account owner

Offboarding

The offboarding process ensures that users who no longer require access are properly removed from the system.
This is crucial for maintaining security and compliance.

Steps

• A list of users to be offboarded is submitted by a TDF/Engineering participant.
• The request is reviewed and validated by the TDF/Engineering account owner.
• Access rights are revoked and user data is archived or deleted as per policy.

Responsibilities

• Initiator: TDF/Engineering account participant
• Validator: TDF/Engineering account owner

Support Request

The support request service handles user inquiries related to tool features, issues, or bugs. It follows a classic L1/L2/L3 support escalation model.

Support Levels

• L1 (Level 1): Basic troubleshooting and common user issues. Typically handled by front-line support.
• L2 (Level 2): More in-depth technical support requiring knowledge of system configurations or intermediate debugging. Usually managed by platform teams.
• L3 (Level 3): Advanced support for complex issues or bugs requiring code-level investigation or engineering input. Usually managed by product team.

Workflow

1. User submits a support request.
2. L1 support attempts resolution or escalates if needed.
3. If unresolved, the issue escalates to L2, and finally to L3 if necessary.

Billing and onboarding

This section aims at describing how billing related to the component is managed and at specifying offers including access to the component.

Billing & onboarding on TDP and CASTLE

This component is deprecated and will be transfered, like Offer Management, to platform operation teams.

The Ops manager can use a “Subscribe to GenAI premium offer” Post-it ticket to onboard users on a TDF account. The Ops manager can also use an “Unsubscribe to GenAI premium offer” Post-it ticket to offboard users on a TDF account. Behind the scenes, these Post-it tickets will call the onboarding and offboarding endpoints.

The onboarding or offboarding endpoints will add or removed users from production Premium Access group

The Billing team will then call the offers and consumptions endpoints to establish a bill for each TDF account / Ops manager.

As part of the offboarding process, a scheduler is set up to check every day at midnight that users who have requested to be unregistered are indeed unregistered on their “monthiversary” day.

Billing & onboarding on RTDP and RCastle

Not Available yet.

Component deployment and configuration

Requirements & Pre-requisite

• Identity Provider (IdP (e.g Azure AD))
• Kubernetes cluster v1.26 and later
• Flux v2.2.2 and later installed. Multi-tenancy lockdown supported
• LLM/SLM
• APIM (optional)
• Guardrails (optional, depending on the platform and its data sensibility)

Install Highlights for Flux managed by Kubernetes provider (e.g K8SaaS on TDP)

• Do not flux bootstrap your cluster since it will be done by your Kubernetes provider
• Just fork the reference repository and apply the Flux manifests as explained into the documentation.

More information about how to install with Flux managed by Kubernetes provider

Configuration

IdP requirement: Configuring the API Service Principal

• Naming syntax: sf[platform]-code-companion-api-[env]

Step 1: Authentication

Go to “Manage” section.

Configure Authentication Settings

To configure authentication settings for your service principal, follow these steps:

1. Add a Web Platform with Redirect URIs:

   • Redirect URI: https://jwt.ms

2. Add a Mobile and Desktop applications Platform with Redirect URIs:

   When necessary, replace {service-principal-id} with the id of the Service Principal.

   • Redirect URI: http://127.0.0.1/callback
   • Redirect URI: https://vscode.dev/redirect
   • Redirect URI: ms-appx-web://microsoft.aad.brokerplugin/{service-principal-id}

Step 2: API Permissions

Configure API Permissions

To configure API permissions for your service principal, follow these steps:

1. Add APIM permissions:

   • Browse service principals to search for the one of your APIM in the section owned by your organization.
   • Check the permission and validate. (e.g In TrustNest, this step requires admin approval.)

2. Adding Specific permissions (e.g Microsoft Graph on TrustNest)

   • In TrustNest for example, Microsoft Graph API Rights are required.
   • To configure Microsoft Graph API rights for your service principal, follow these steps:

• (TrustNest specific) Add a permission => Microsoft Graph => Delegated permissions:
  • Check the following boxes:
    • OpenID permissions:
      • email
      • offline_access
      • openid
      • profile
    • Directory:
      • Directory.Read.All (requires Admin approval)
    • User:
      • User.Read
      • User.ReadBasic.All

Step 3: Expose an API (api)

To expose an API for your service principal, follow these steps:

1. Add an “Application ID URI”:

   • Save the changes

2. Add a scope (e.g. codecompanion) for admins access only:

   • Display name: codecompanion
   • Consent description: Admin access only
   • State: Enabled

Step 4: Configure App Roles (api)

To configure app roles for your service principal, follow these steps:

1. Create Premium app role:

   • Display name: Premium
   • Allowed member types: Both
   • Value: premium
   • Description: Premium
   • Check “Do you want to enable this app role”

2. Create K6 app role:

   • Display name: k6
   • Allowed member types: Both
   • Value: k6
   • Description: Tests k6
   • Check “Do you want to enable this app role”

IdP requirement: Configuring the APP Service Principal

• Naming syntax: sf[platform]-code-companion-app-[env]

Step 1: Configure Authentication Settings

Go to “Manage” section.

To configure authentication settings for your service principal, follow these steps:

1. Add a Web Platform with Redirect URIs:

   • Redirect URI: https://jwt.ms

2. Add a Mobile and Desktop applications Platform with Redirect URIs:

   When necessary, replace {service-principal-id} with the id of the Service Principal.

   • Redirect URI: http://127.0.0.1/callback
   • Redirect URI: https://vscode.dev/redirect
   • Redirect URI: ms-appx-web://microsoft.aad.brokerplugin/{service-principal-id}

Step 2: Configure API Permissions

To configure API permissions for your service principal, follow these steps:

1. Give access the other Service Principal:

   • Select “Add a permission”, select “My APIs” and choose the one corresponding to your environment previously set (e.g: sfc3-saml-code-companion-api-dev)

     • Select “Delegated permissions

     • Check the permission “default XXXX” and validate

       This step require admin approval

2. Adding Microsoft Graph API Rights

   To configure Microsoft Graph API rights for your service principal, follow these steps:

   • Add a permission => Microsoft Graph => Delegated permissions:
     • Check the following boxes:
       • OpenID permissions:
         • email
         • offline_access
         • openid
         • profile
       • User:
         • User.Read
         • User.ReadBasic.All

Step 3: Expose an API (app)

To expose an API for your service principal, follow these steps:

1. Add an “Application ID URI”:

   • Save the changes

2. Add a scope (e.g. codecompanion) for admins access only:

   • Display name: codecompanion
   • Consent description: Admin access only
   • State: Enabled

Step 4: Configure App Roles (app)

To configure app roles for your service principal, follow these steps:

1. Create Premium app role:

   • Display name: Premium
   • Allowed member types: Both
   • Value: premium
   • Description: Premium
   • Check “Do you want to enable this app role”

IdP requirement: Configuring the APP Enterprise Application

• Naming syntax: sf[platform]-code-companion-app-[env]

Step 1: Configure users/groups Settings

Affects current user and groups:

• dp-trustnest-genai-hermes-c2-[env]-app-freemium (Freemium role)
• dp-trustnest-genai-hermes-c2-[env]-app-premium (Premium role)
• dp-trustnest-genai-hermes-c2-[env]-app-trial (Trial role)
• dp-trustnest-genai-hermes-c2-[env]-app-reader (Reader role)

Access Groups

There are four groups that can access our product, each with distinct permissions and use cases:

• Reader: Intended for the development team, management, and platform team.
• Trial: This group is used for trial periods and is managed by a script that controls trial access.
• Premium: This group offers premium access to the product (default offer), which is granted through the onboarding and offboarding script code.
• Freemium: Although not commonly used, this group allows for temporary access to be granted in cases where there are issues with premium access.

Setting Helm values

Mandatory Code Companion values are described in the SWaaP Readme - Helm values .

There’s also a list of required and optional ConfigMaps that are needed. These are described in the Code Companion package readme document

TLS Setting

We advise against setting the TLS verify option to false for production usage.

tlsVerify: false

This should be used in development environment only.

Configuring Kubernetes secrets

The required Code Companion secrets and how they are created is described in the SWaaP Readme - Code Companion secrets .

Deployment & update procedure

The deployment and update procedures are described in the SWaaP Readme - Quick start section .

Install Steps for Flux managed by Kubernetes provider (e.g K8SaaS on TDP)

1. Fork the reference repository into your GitLab group (and/or into your restricted GitLab instance) (TODO: Update branch from ‘code-companion’ to the pre-release branch)
2. Configure platform YAML specific values (link to reference section of values )
3. Create required secrets (link to reference secrets section ))
4. Apply Flux manifests into the cluster (link to flux manifests apply in reference ) by running the following command:

kubectl apply -k setup/

The following command works as well:

kubectl kustomize setup/ | kubectl apply -f-

Test your deployment

There are k6 tests available to test your deployment.

Clone the project and run the tests locally with the following command :

      k6 run
      --env TENANT_ID=${YOUR_TENANT_ID}
      --env CLIENT_ID=${YOUR_CLIENT_ID}
      --env CLIENT_SECRET=${YOUR_CLIENT_SECRET}
      --env SCOPE=${YOUR_SCOPE}
      --env SVC_USERNAME=${YOUR_SVC_USERNAME}
      --env SVC_PASSWORD=${YOUR_SVC_PASSWORD}
      --env BASE_URL=${YOUR_SERVER_URL}
      --summary-export=${FILE_PATH}.json
      ./${FILE_PATH}.js

Make sure to replace paramters with the correct values :

• ${YOUR_TENANT_ID}, ${YOUR_CLIENT_ID}, ${YOUR_CLIENT_SECRET} and ${YOUR_SERVER_URL} are variables related to your service principal.
• ${YOUR_SCOPE} is the API endpoint you want to target.
• Replace ${YOUR_SVC_USERNAME} and ${YOUR_SVC_PASSWORD} by the username and password of your K6 service account previously created.

At test completion k6 will generate a JSON file with test reports.

Settings

Capacity Planning

(configured as nano sizing but could scale up according to a load test conducted in the integration environment)

Functional Configuration

This section is optional. describe here any customization which can be configurate on platform

Monitoring

This section is optional. It aims at describing what are the metrics we suggest to use.

Metrics for supervising health of components

• GET /health - API route to ensure the backend is UP and running
• (average, p99, p95) latency
• error rate on API http status code
• (average, p99, p95) token per seconds

Business Metrics

• (DAU, MAU) number of active users per day/month
• number of prompts (total, per user)
• token consumption (total, per user)

Common Operations

Secrets roation/expiration

List of secrets requiring rotation or being expired and requiring being updated:

• Subscription key
• Service principals Client secrets

How to rotate Subscription key

• Rotate the subscription key from your API Management portal/dependency
• Update the new subscription key value (key: trustnest-apim-subscription-key) in the secrets “apim-credentials”(SWaaP Readme - Helm values .)

How to rotate Service principals client secrets

• Rotate the service principals client secrets in your IdP portal (e.g on Azure AD, go to the service princiapl, Manage -> Certificates & Secrets -> (tab) Client secrets -> Create a new client secret and remove the previous secret once expired)
• Update the new subscription key value (key: oauth-client-secret) in the secrets “apim-credentials”