broker up

Overview

Start the broker instance of the local pipeline

Aliases: up

Usage:

quix broker up [options]

Options:

--directory <directory>: If specified, it will use that directory instead of the current one
--dry-run: Generate 'compose.yaml' without running it
-t, --create-topics: Create topics specified in the quix.yaml

How It Works

The quix broker up command sets up and runs your local broker instance using Docker Compose. When executed, it generates the necessary compose.local.yaml file that defines the broker services required for your setup. It then builds and starts the Docker containers based on this configuration by running the equivalent of docker compose up --build -d --remove-orphans.

If the dry-run option is used, the command allows you to generate the compose.local.yaml file without starting the containers, providing a preview of the configuration. This is particularly useful for validating your setup before making any changes to your running environment.

Deployments are connected using variables that ensure seamless data flow between different services. This process ensures that your local broker instance is correctly configured and running with all necessary services. For more details on the docker compose up command, refer to the official Docker documentation.

Warning

Ensure that Docker is running before executing this command. If Docker is not running, the command will fail. You can verify Docker is running by executing the following command:

docker info

Example Usage

Starting the Local Broker with Docker Compose

To start your local broker, use the following command:

$ quix broker up

This command generates the necessary compose.local.yaml file and deployment configurations:

Generating 'compose.local.yaml'
✓ Generated 'compose.local.yaml'

Next, it executes docker compose compose.local.yaml up --build -d --remove-orphans to build and run the Docker containers:

Executing 'docker compose compose.local.yaml up --build -d --remove-orphans'

Running the Containers

Once the images are built, Docker Compose will create and start the containers:

Network githubrepo_default  Creating
Network githubrepo_default  Created
Container githubrepo-kafka-broker-1  Creating
Container githubrepo-console-1  Creating
Container githubrepo-console-1  Created
Container githubrepo-kafka-broker-1  Created
Container githubrepo-console-1  Starting
Container githubrepo-kafka-broker-1  Starting
Container githubrepo-kafka-broker-1  Started
Container githubrepo-console-1  Started

✓ Open http://localhost:8080 to manage your pipeline broker

Tip

Using the --create-topics or -t" options will create the topics from quix.yaml automatically.

Generated `compose.local.yaml` File Overview

The compose.local.yaml file configures the services in your local broker instance. Here's an overview of what will be generated:

kafka-broker:
- build: Specifies the context directory and Dockerfile for the Kafka broker.
- environment: Sets environment variables for the broker configuration, including the listener address and the Zookeeper instance address.
console:
- build: Specifies the context directory and Dockerfile for the Kafka management console.
- environment: Sets environment variables for the console configuration, including the Kafka broker address.

For more details on the compose.local.yaml file and its configurations, refer to the official Docker Compose documentation.