After installing and connecting to the gradient fox web UI, you need to configure connections to your Kafka clusters. Below you can find the steps to configure your connection using different authentication methods and specific instructions for certain Kafka cloud providers.
To create a new connection to your Kafka cluster, navigate to the Clusters-page and click on the Add-button in the top right-hand corner. In the dialog that is displayed you need to specify the following properties and possibly additional properties depending on the security protocol you selected.
Kafka clusters are usually configured with plaintext security only in test environments. If your cluster is configured for plaintext security there are no additional security attributes to specify. Click on test Test-button to verify that your connection is working properly. When you have confirmed that gradient fox is able to communicate with your cluster, click on the Add-button to add the new connection.
If your Kafka cluster is configured with SASL security protocol, there are additional SASL-specific properties you need to configure for your connection.
If your Kafka cluster is configured to use SSL you may have to set various SSL configuration parameters, depending on if the brokers certificates are publicly trusted and if client certificates are required. Below you see a connection that uses a trusted certificate as well as a client certificate. Refer to the following sections to see how client certificates and trusted certificates can be added.
If the subject name in your server certificate does not match the hostname of the broker(s) you are connecting to, you will receive an error similar to "No subject alternative DNS name matching xxx found" when you try to connect. This can be avoided by unchecking the 'Validate SSL Endpoint Hostname' checkbox. This will set the ssl.endpoint.identification.algorithm client property to null when connecting to the brokers.
If your Kafka brokers are using a server certificate that is not issued by a public CA, you need to add your broker certificate as a trusted certificate. This can be done using the Add-button on the Trusted Certificates page. You can give the trusted certificate a user-friendly name and the certificate itself must be entered in PEM-format.
If your Kafka cluster requires a client certificate (two-way authentication) you also need to specify a client certificate that will be used to authenticate the client to your Kafka cluster. You can add the client certificate using the Add-button on the Client Certificates page. You need to specify both the private key and the certificate itself as PEM-formatted strings. If the private key is encrypted, it must contain with the correct PEM pre- and postfixes, and you need to specify the private key password in the corresponding field.
-----BEGIN ENCRYPTED PRIVATE KEY-----
If the private key is not encrypted you need to use different PEM pre- and postfixes and the private key password is not required.
-----BEGIN PRIVATE KEY-----
The Certificate Chain field contains the needed certificates, each one in a PEM-formatted block. Depending on the cluster configuration you may have to specify the entire certificate chain or just the end-entity certificate.
If your Kafka environment is hosted in Confluent Cloud, connecting gradient fox to your cluster is straightforward. Confluent Cloud uses SASL_SSL security protocol and since the broker certificates are publicly trusted you do not need to add trusted certificates. The only mandatory values you need to set are specified below.
org.apache.kafka.common.security.plain.PlainLoginModule required username="MYUSERNAME" password="MYPASSWORD";
Here is a typical setup for Confluent Cloud connection.
The quickest method to connect to an AWS MSK cluster is to use the SASL_SSL/SCRAM security protocol if your AWS MSK environment has it enabled. The broker certificates on AWS MSK brokers are publicly trusted, so there is no need to add any additional trusted certificates.
org.apache.kafka.common.security.scram.ScramLoginModule required username="MYUSERNAME" password="MYPASSWORD";
Here is a typical setup for AWS MSK connection using SCRAM.
If your AWS MSK is configured for IAM (Identity and Access Management), configuring the connection is similar to MSK SCRAM except the credentials are not specified in JAAS config. Instead, the access credentials are specified as environment variables passed to the webserver service in the Docker Compose file compose.yaml. The broker certificates on AWS MSK brokers are publicly trusted, so there is no need to add any additional trusted certificates.
webserver:
environment:
- AWS_ACCESS_KEY_ID=myaccesskey
- AWS_SECRET_ACCESS_KEY=mysecret
The other configuration variable specific to MSK IAM is the callback handler that needs to be specified.
software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="main";
Here is a typical setup for AWS MSK connection using IAM.
If your Kafka environment is hosted in Aiven, you can connect using either SASL or SSL with a client certificate. For SASL, Aiven uses the SASL_SSL security protocol. Since the broker certificates are not publicly trusted you first need to add a Trusted Certificate (referred to as CA Certificate in the Aiven console). Refer to the SSL section above on how to add a Trusted Certificate.
org.apache.kafka.common.security.scram.ScramLoginModule required username="MYUSERNAME" password="MYPASSWORD";
Here is a typical setup for Aiven connection using SASL.
For the SSL connection method you need to select a Client Certificate (referred to as Access Key/Certificate in the Aiven console) from the corresponding drop-down. You also need to select a Trusted Certificate (referred to as CA Certificate in the Aiven console) since the broker certificates are not publicly trusted. Refer to the SSL section above on how to add Trusted Certificates and Client Certificates.
Here is a typical setup for Aiven connection using SSL with client certificate.
