This version is still in development and is not considered stable yet. For the latest stable version, please use StreamX Guides 1.0.0!

StreamX Platform architecture

StreamX provides a modern, Kubernetes-native architecture designed to support scalable, efficient, and resilient content delivery. The platform is built around distinct planes for developer operations, integration and delivery, observability, and messaging, enabling a modular and extensible architecture.

Platform overview

StreamX Platform Architecture
Figure 1. StreamX Platform Architecture

The architecture is divided into the following planes:

  1. Developer control plane

  2. Integration & delivery plane

  3. Observability plane

  4. Messaging plane

  5. Resources plane

Each plane focuses on a specific aspect of the platform to ensure separation of concerns and modularity.

Developer control plane

The Developer control plane enables developers to manage and contribute to the platform. It consists of

  • IDE: A development environment of your choice.

  • StreamX CLI: A command line interface for setting up a local deployment environment and deploying the mesh to the cloud.

  • Version control: GitHub repositories to manage the source code for

    • Service source code: Comprises microservices built with Quarkus and the StreamX Extension.

    • StreamX mesh source code: Includes YAML definitions for mesh configuration and resource setup.

Integration & delivery plane

This plane focuses on the CI/CD process and orchestrates tenant resources

  • CI pipeline: GitHub Actions is used to automate the testing, building, and deployment of container images to a registry.

  • Container registry: Artifacts Registry stores container images.

  • StreamX tenant orchestration: Managed by the StreamX Operator, which provisions and orchestrates tenant-specific resources.

Observability plane

The Observability plane ensures that the platform is monitored effectively. It supports integration with both built-in and external tools for logs, metrics, and traces:

  • Logs:

    • OpenTelemetry for log collection.

    • Loki for log storing and aggregation.

    • Grafana for filtering and visualization.

  • Metrics:

    • OpenTelemetry for traces collection.

    • Prometheus for metrics collection and storing.

    • Grafana for filtering and visualization.

  • Traces:

    • OpenTelemetry for distributed tracing.

    • Jaeger for storing and visualizing trace data.

The Observability plane is designed to be extensible, allowing users to replace built-in tools with external solutions based on their cloud ecosystem and needs.

Messaging plane

The Messaging plane is the backbone for communication within the platform:

  • Messaging system: Apache Pulsar handles event-driven messaging across the system.

Resources plane

The Resources plane encapsulates tenant-specific and StreamX Operator-managed resources within Kubernetes:

  • ConfigMaps: Stores configuration data for services.

  • Secrets: Stores sensitive data for services.

  • Workloads: Includes deployments and stateful sets running the tenant’s services.

  • Services: Exposes workloads to internal or external networks.

  • Ingresses: Handles HTTP(S) routing to services.

Deployment and extensibility

The platform is composed and deployed by using Terraform modules, ensuring infrastructure as code and repeatable deployments. Observability plane is designed to be extensible, allowing users to replace built-in tools with external solutions based on their cloud ecosystem.

Benefits

The modular architecture of StreamX ensures:

  • High scalability and resilience.

  • Observability for troubleshooting and optimization.

  • Seamless integration into existing ecosystems by Kubernetes and cloud-native practices.

  • Adaptability to customize the setup for specific requirements.