You would use Apache Kafka over Azure Event Hubs when you need a highly customizable and extensible messaging system that can handle complex stream processing. Kafka is particularly suitable for scenarios where you have large volumes of data and require robust messaging features, such as storing data for a longer duration or performing stream processing using its ecosystem tools. For instance, in a microservices architecture where multiple services need reliable real-time data sharing and processing across many different systems, Kafka's functionalities can be crucial.
Azure Event Hubs is ideal for applications targeting the Azure ecosystem and requiring a fully managed event streaming platform. It's best used when the infrastructure overhead needs to be minimized, such as in scenarios where development teams want to focus on application development rather than server management. For example, if a company is developing an IoT solution that sends millions of telemetry events daily and is already using other Azure services (like Azure Functions, Azure Stream Analytics), Event Hubs can be quickly integrated into their architecture without the need for maintaining dedicated servers as would be necessary with Kafka.