Kafka Connect is a toolkit included in Apache Kafka® that simplifies the process of integrating Kafka with external systems by providing a framework to move data into and out of Kafka topics. It handles the complexities of data streaming, including serialization, deserialization, and parallelization, allowing users to focus on high-level configuration instead of coding.
To add a connection to your Kafka Connect cluster, click on the Add-button on the Kafka Connect screen. This will bring up a dialog where you can enter the information required to connect to your Kafka Connect environment. If your Kafka Connect environment is using Basic Authentication (optional), you also need to enter the basic auth username and password in the API Key and API Password fields, respectively.
A Kafka Connect connector is a pre-built plugin or component in Kafka Connect that facilitates the integration between Apache Kafka® and external systems. It enables you to easily move data into Kafka (source connectors) or out of Kafka (sink connectors) without writing custom code. Source connectors pull data from external systems (e.g., databases, file systems, APIs) into Kafka topics. Sink connectors push data from Kafka topics to external systems (e.g., databases, cloud storage, analytics platforms).
In gradient fox you can view your Kafka Connect connectors by selecting a Kafka Connect cluster and navigating to the Connectors-tab. You will see a list of all connectors of your cluster in a grid with additional details about each connector.
If you want to delete a connector, click on the delete (trashcan) icon on the appropriate row.
To add a new connector, click on the Add-button on the connectors page. This will bring up a dialog where you need to enter relevant information about the connector you are about to create.
On the connector details page you can restart, pause and resume connectors. The exact available operations depend on the state of the connector.
You can start individual connector tasks under the Tasks-tab of the selected connector by clicking on the Restart-icon.
gradient fox can automatically monitor your connector tasks and restart them when a failure is detected. This can be enabled for each Kafka Connect connector individually under the Monitoring-tab of the selected connector. When monitoring is enabled, you can select if only failed tasks or all tasks are restarted when a failed task is detected. You can also change the minimum time between attempts to restart failed tasks to avoid too frequent restart attempts.
