Setup Kafka CLI Helper Scripts - theryanwatson/confluent-kafka-scripts GitHub Wiki

To use the k-* Kafka Helper Scripts:

  1. Run this command to copy the latest __k-functions and k-* scripts to your ~/bin folder. 
    mkdir -p "${HOME}/bin/"; \
for SCRIPT in __k-functions k-avro-console-consumer k-avro-console-producer k-console-consumer k-console-producer k-consumer-groups k-copy-schema k-topics; do \
  rm -f "${HOME}/bin/${SCRIPT}"; \
  wget "https://raw.githubusercontent.com/theryanwatson/confluent-kafka-helpers/main/bin/${SCRIPT}" -P "${HOME}/bin/"; \
  chmod +x "${HOME}/bin/${SCRIPT}"; \
done;
  2. If not already added, add your home bin path to your ~/.profile (and restart or source the file, as needed) 
    # set PATH so it includes user's private bin if it exists
if [ -d "${HOME}/bin" ] ; then
    PATH="${HOME}/bin:$PATH"
fi
  3. Setup your Local Confluent Platform, including adding the ${CONFLUENT_HOME} environment variable and Confluent Bin to your ~/.profile.
  4. Create ~/.ccloud/{{ env-prefix }}.config files for each environment you wish to use, according to the Confluent CLI Tools Config. e.g. qa.config, qa2.config, prod.config, etc. For example: 
    ssl.endpoint.identification.algorithm=https
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
request.timeout.ms=20000
retry.backoff.ms=500
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username\="{{ username }}" password\="{{ password }}";
bootstrap.servers={{ host }}.us-east-1.aws.confluent.cloud\:9092
#schema.registry=http://{{ path }}