Warning

Nightly releases are generated automatically from the latest source code and are intended for experimental purposes only. These builds may contain incomplete or untested features, bugs, or security vulnerabilities, and as such, are not for production use. Users should be aware that nightly releases may cause unexpected behavior, data loss, or system instability. Use of these releases is at the user's own risk, and it is advised to have adequate backups before testing. The software is provided as is with no guarantees or support.

Autoscaling

Introduction

Note

Autoscaling is currently only supported for transcoder pods.

The system utilizes a message queue (RabbitMQ) internally to schedule transcoder (data conversion) tasks to pods. Autoscaling can use the length of one of these queues as a scaling metric.

We recommend using Keda to configure autoscaling based on these metrics. Keda can easily be installed via Helm or directly with kubectl as described in the official documentation.

Configuration

Currently a queue is created for each data format supported by instant3Dhub. We will need to create a Keda scaling configuration for each data format we want to allow to scale. The pod we scale will be the same for every format, as every transcoder pod is able to handle every format in the default configuration.

A list of available queues to scale can be found under the RabbitMQ management interface of the instant3Dhub installation under /admin/rabbitmq/#/queues (default login is i3dhub 7F7dbMgFQjVshwzQ).

Note

Only queues starting with name TranscoderService can be configured for autoscaling. Others are not yet supported.

Below is a configuration for scaling i3dhub-transcodersvc pods based on the JT queue. Pods will be scaled to a minimum of at least one and a maximum of 10.

Important

The correct namespace must be set in both the connection string of i3dhub-keda-rabbitmq-secret as well as the TriggerAuthentication and the ScaledObject.

apiVersion: v1
kind: Secret
metadata:
  name: i3dhub-keda-rabbitmq-secret
data:
  # base64 of amqp://i3dhub:7F7dbMgFQjVshwzQ@i3dhub-rabbitmq.NAMESPACE.svc.cluster.local:5672
  host: YW1xcDovL2kzZGh1Yjo3RjdkYk1nRlFqVnNod3pRQGkzZGh1Yi1yYWJiaXRtcS5OQU1FU1BBQ0Uuc3ZjLmNsdXN0ZXIubG9jYWw6NTY3Mg==
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: i3dhub-keda-trigger-auth-rabbitmq-conn
  namespace: NAMESPACE
spec:
  secretTargetRef:
    - parameter: host
      name: i3dhub-keda-rabbitmq-secret
      key: host
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: rabbitmq-scaledobject
  namespace: NAMESPACE
spec:
  scaleTargetRef:
    name: i3dhub-transcodersvc
  pollingInterval: 10  # Optional. Default: 30 seconds
  cooldownPeriod:  300 # Optional. Default: 300 seconds
  minReplicaCount: 1   # Optional. Default: 0
  maxReplicaCount: 10  # Optional. Default: 100
  triggers:
  - type: rabbitmq
    metadata:
      protocol: auto
      queueName: TranscoderService.openjt.l3d
      mode: QueueLength
      value: "40"
    authenticationRef:
      name: i3dhub-keda-trigger-auth-rabbitmq-conn