Xray

1. Application

2. Introduction

2.1 Document purpose

This document is a low level design - LLD which aims to describe how the architecture evoked in high level design - HLD will be implemented. This document will describe the protocols used in the target architecture, how to implement them and any modifications made to their default behavior. Once validated by Thales, this document will then serve as a basis for the implementation of configurations on equipment.

2.2 Document scope

This document is not a manual and is not intended to replace the reference literature describing with great precision all network protocols.

The protocols used will be briefly described as well as the modifications made to their default behavior.

2.3 Referenced documentation

3. Component general description

This component is part of Software Factory as a Product (SWaaP), and it is visible in the TASD .

JFrog Xray is an advanced security and compliance analysis tool designed to provide deep insights into the vulnerabilities and license compliance issues within software artifacts. As a complementary product to JFrog Artifactory, Xray integrates seamlessly to enhance security measures across the software development lifecycle (SDLC). It offers continuous scanning and monitoring of all artifacts stored in the repository, including those within Docker images, Java archives, and more, ensuring comprehensive protection and compliance from code to production.

Xray is completely integrated with Artifactory . It can be managed using:

• Artifactory UI
• Xray APIs and JFrog CLI.

4. Functional & Business Requirements

No formal list of requirements has been expressed by clients. It is designed and developed based on business use cases.

4.1 Feature summary

JFrog Xray addresses several critical needs in the software development and operations landscape:

• Security Vulnerabilities: Early detection of security vulnerabilities in dependencies and binaries. Xray ensures that issues are identified as soon as they are declared, preventing insecure code from progressing through the SDLC.

• License Compliance: Continuous monitoring for license violations to ensure that all software components comply with legal and organizational standards, avoiding potential legal risks associated with the use of open-source software.

• Impact Analysis: Provides detailed impact analysis to understand how a vulnerability in one component affects others, allowing for informed decision-making and prioritization of remediation efforts.

• Automated Governance: Facilitates automated and continuous governance and auditing of software artifacts, ensuring compliance with security policies and standards throughout the SDLC.

• Comprehensive Scanning: Recursively scans all layers of software artifacts, including nested components, to provide a thorough analysis of potential security and compliance issues.

4.2 Target Population

JFrog Xray is targeted towards several key user groups within the software development and IT operations ecosystem:

• DevSecOps Teams: Professionals who integrate security practices within the DevOps process, ensuring security and compliance are maintained throughout the development pipeline.

• Security Analysts: Individuals responsible for identifying, analyzing, and mitigating security threats within the organization’s software environment.

• Compliance Officers: Personnel tasked with ensuring that the organization adheres to legal and regulatory requirements related to software usage and open-source licenses.

• Software Developers: Developers who need to ensure that their code and dependencies are secure and compliant from the earliest stages of development.

• IT Operations Teams: Teams managing the deployment and maintenance of software in production environments, who require assurance that deployed applications are secure and compliant.

4.3 Prerequisites

Every prerequisites of the product are applicable to this component. In detail:

• Kubernetes and Flux. See the TASD §4.1.2 Prerequisites for supported version.
• PostgreSQL 14 or later - see database integration .

4.4 Variability

SWaaP is managing no variability on Xray.

5. Architecture decision record

Here is a list of decisions:

Table 3 - List of architecture decision record.

5.1 ADR-XRAY-001: Add Xray as a component of the product

5.1.1 Status: Accepted

• Creation of sf-package in September 2022
• Recorded in SWF organization in April 2023

5.1.2 Context

• See ADR004 in TASD .

5.1.3 Decision

• Introduce Xray in the product from its current implementation on TDP C2 & TDP C3-CA

5.1.4 Consequences

• Add Xray to the product

6. Architecture description

6.1 Business architecture and allocation to services

You will find in Figure 1 business architecture for software code and CI/CD engineering allocated to services:

Figure 1 - Business architecture allocated to services.

Note: in dash, external items.

6.2 Application architecture

Physical architecture is described in Figure 2 :

Figure 2 - Logical Architecture of Xray

Xray is using these external services:

• A Software Factory or mirror for deployment (PRE_001).
• Kubernetes with Flux (PRE_002, PRE_003).
• Persistent storage to store the configuration and cache (PRE_004).
• Database (PRE_015); we recommend Managed PostgreSQL database. Alternative can be internal to namespace provided PostgreSQL database that is provided with the product.
• Integration with Artifactory.

As Xray is integrated to Artifactory, there is no new need of ingress entry point, IAM or mail server.

6.2.1 Main Features and Functionality

• Early Detection: Xray identifies security vulnerabilities and license violations as early as the dependency declaration stage and blocks builds with security issues from development. Automated and continuous governance and auditing of software artifacts and dependencies throughout the software development lifecycle from code to production.

• Self-hosted, Cloud, Hybrid or Multi-Cloud Solution: Xray is available self-hosted (self-managed) and on the cloud. Xray Cloud is hosted on your choice of Amazon Web Services, Google Cloud Platform, or Microsoft Azure, allowing you to maintain infrastructure with automated server backups, free updates, and guaranteed uptime.

• Deep Recursive Scanning: Xray recursively scans artifacts, builds, and Release Bundles in your system, drilling down to analyze even the smallest binary component that affects your software. For example, when analysing a Docker image, if Xray finds that it contains a Java application it will also analyze all the .jar files used in this application.

• Continuous Impact Analysis: Xray analyzes how an issue in one component affects all others in your company and displays the chain of impact in a component graph, allowing you to have a clear understanding of the impact one component has on another. It is continuously updated with new security vulnerabilities, performing an impact analysis to determine all artifacts affected by the issue.

• Native Integration with Artifactory: Xray is the only security scanning tool that is natively integrated with JFrog Artifactory. As a complementary product to JFrog Artifactory, Xray has access to the wealth of metadata Artifactory stores which, combined with deep recursive scanning, puts Xray in a unique position to analyze the relationships between binary artifacts and provide radical transparency into your component architecture to reveal the impact that a vulnerability in one component has on any other.

• Vulnerability Database: Xray comes with JFrog’s vulnerabilities database, which integrates data from vulnerability databases and security advisories including NVD, GitHub, Ubuntu, Debian, Red Hat, PHP, and is enriched by the JFrog Security Research team to give more specific and detailed information on the vulnerability, its use cases, and options for mitigation.

• Custom API-Driven Automation: Through an open REST API, Xray lets you define a custom regimen of automated analysis for all components in your system.

• Dependencies Scan: Scan your sources’ dependencies using the JFrog CLI for vulnerabilities and license violations.

• On-Demand Binary Scan: Point to a binary in your local file system and receive a report that contains a list of vulnerabilities and licenses for that binary using the JFrog CLI.

• SBOM: Enable DevSecOps engineers to understand and analyze the dependencies of their components. To learn more, see Xray SBOM Report.

• JFrog Security CVE Research and Enrichment: JFrog’s security research team helps you with enhanced analysis on CVE findings in a way that allows you to focus on the most important issues with the capability of finding the best resources invested in fixing them. For more information, see JFrog Security CVE Research and Enrichment.

• Component’s Operational Risk: Provides you with additional data on OSS components that will help you gain insights into the risk level of the components in use. For more information, see Components Operational Risk.

• JFrog Advanced Scans: Includes IaC security, secrets detection, contextual analysis, and detection of OSS library and services misconfiguration or misuse. For more information, see JFrog Advanced Security.

• Universal Artifact Analysis: In line with JFrog’s universal approach, JFrog Xray performs artifact analysis for all major package formats across the CI/CD pipeline. Xray understands each package type, knows how to unpack it, and what every underlying layer contains.

6.2.2 Following package formats with new formats added regularly are supported

6.2.3 Sub components

The Xray service consists of several microservices, each responsible for different aspects of artifact scanning and analysis:

Indexer

Responsible for the indexing process, which includes:

• Recursively extracting artifacts and builds
• Collecting artifact metadata from accompanying files
• Building an artifact components graph representation

Persist

Responsibilities include:

• Matching the given components graph with public component information
• Completing component naming
• Storing graph data and component metadata in PostgreSQL

Policy Enforcer

Responsible for generating violations by matching analysis data with Xray Watches and Policies.

Analysis

Responsible for enriching component metadata such as vulnerabilities, licenses, and versions.

Server

Responsibilities include:

• Generating violations by matching analysis data with watches and policies
• Hosting the API and UI endpoints
• Running scheduled jobs such as the database synchronization process

Router

Responsible for communication between all the microservices and cross-product.

6.2.4 User management

The user management for Xray is done in Artifactory. Artifactory permits to manage roles with a finer granularity. For more information please consult the Artifactory Users Management

6.2.5 RACI

This role matrix has been defined in TASD:

For Xray, a tenant is a repository.

In addition of global RACI (Responsible, Accountable, Consulted, Informed) of the SWaaP defined in TASD:

6.3 Delivery

Component is part of the Software Factory as a Product (SWaaP) delivery. See TASD for more details.

6.3.1 Latest Version

• Latest version editor
  • Component registry : TBC on main branch
  • Helm chart repository
• SWaaP integration part

6.3.2 Version Chart 103.137.31 / Xray 3.137.31

• Component registry:

  • xray-server
  • xray-persist
  • xray-indexer
  • xray-analysis
  • router
  • xray-policyenforcer
  • observability
  • rabbitmq

• Helm chart registry: in SF delivery From JFrog

• SWaaP integration part

• Component Merge Request in Reference

• Release changelog

• K6 Test report: pipeline

  • ui_light
  • load_light
  • load_heavy

• Security Report:

  • SCA Report
  • SAST

6.4 Infrastructure architecture

6.4.1 Software Factory API

Here is a list of services that can be integrated with Xray.

6.4.2 Database integration

Xray supports the following versions of PostgreSQL:

• 16.x (from version 3.107)
• 15.x (from version 3.78.9)
• 14.x
• 13.x (from version 3.18)

6.4.3 RabbitMQ integration

RabbitMQ is installed as part of the Xray installation for every node. In a High Availability (HA) architecture setup.

Below is a low-level architecture diagram showcasing all the components, their nature, their location, and technical flows.

How JFrog Xray uses RabbitMQ Xray has multiple crucial roles, such as scanning, impact analysis, and database sync. These roles require simultaneous flows to be processed by the different Xray services. Xray uses RabbitMQ to manage these different flows while tracking communication between services.

RabbitMQ manages 3 main types of queues in Xray:

6.4.3.1 New Content

Responsible for events related to new content added to the system. For example, uploading a new Artifact to a repository that is marked for indexing -> will create a message in the Index queue.

6.4.3.2 Existing Content

Responsible for the content which already exists in the system. For example, reindexing a repository will send messages to this queue.

6.4.3.3 Retry

Failed messages will be sent to this queue and will stay there for a TTL. Once the TTL has elapsed, the messages will be returned to the original queue. The queue names are defined by the service name and the queue type (new content queues name do not have any suffix).For example – ‘indexExistingContent’, and ‘alertRetry’ while ‘persist’ has no suffix. This naming convention is useful when debugging RabbitMQ queues since the queue name is predictable.

7. Operational and maintenance

In this chapter you will find strategy and policy. Detail implementation will be described in the SCOM.

7.1 Life cycle policy

Cadence of version is describe in the Product Lifecycle .

7.2 License

As Xray is integrated with Artifactory, it’s inherit Artifactory license.

7.3 Deployment

The component is deployed as a standard component using Flux and SWaaP packaging. See TASD for more details.

7.4 IAM

We support and recommend integration with IAM using SAML SSO. This configuration is done in Artifactory .

7.5 Scaling

Xray system requirements depend on the size of your environment, specifically the number of indexed artifacts and artifacts/builds processed per day.

• Processor Requirements: Specifies the number of CPU cores required for Xray, Database (DB), and JFrog Advanced Security components.
• Memory Requirements: Specifies the RAM (memory) required for Xray, Database (DB), and JFrog Advanced Security components.
• Disk Space Requirements: Specifies the disk space requirements, including SSD type and IOPS (Input/Output Operations Per Second) for Xray, Database (DB), and JFrog Advanced Security components.

Note: For environments with over 10 million indexed artifacts and 50,000 artifacts/builds per day, contact JFrog Support for specific sizing requirements.

Xray system limitation based on the maximum xray-2xlarge.yaml (biggest size) file

Note: This size is intended for large organizations. It can be increased with adding replicas:

autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 12
  targetCPUUtilizationPercentage: 200
  targetMemoryUtilizationPercentage: 800

7.6 Backup / restore

In the official JFrog documentation, the backup and restore processes for Xray are not referenced in a separate topic. JFrog Xray uses the same PostgreSQL database as the Artifactory product, for which the vendor provides detailed information on these processes.

Database Backup Steps

We recommend to have point in time restore principle set at platform level to get consistency regarding database and volumes backup at platform level.

Data corruption is managed by the COTS and can be detecting watching logs. Can contact editor support through the product support

In case of irremediable data loss/alteration, we have to restore from the same point in time backup database and volumes.

7.7 Monitoring

Please check documentation from Operations Guide

Official documentation JFrog:

• Monitor Xray Microservices
• Log Files Location and Naming

7.8 Logging

JFrog Monitoring and Logging

7.9 Operating procedures

Please check documentation from Operations Guide

Annex

References from official JFrog-Xray Docs

Supported Installation Methods:

• Linux Archive / RPM / Debian
• Docker Compose
• Helm

Other links:

• Xray Product Download and Releases
• Xray System Requirements and Platform Support
• Installing Xray
• Xray Sizing Configuration files
• Configuring Xray
• Xray Integrations
• JFrog Xray Helm Charts
• JFrog Xray User Guide
• JFrog Xray Release Notes
• JFrog Global and Project roles