What are Event Hubs - Kshitijkesri/WiredBrainCoffeeMachine GitHub Wiki

What Are Azure Event Hubs? Let's say you have built an application and you want to find out how your thousands of users are using your application. That means you need to implement some kind of logging, and exactly for this scenario you could set up an Event Hub in Azure and send events with log data from your application to the Event Hub when a specific user action happens. Technically, an event is a message that contains for example a little JSON document. The JSON document could contain information, like, for example, that the user navigated to the about page of your application. So, in other words, an Event Hub can be used to collect telemetry data like this log data. If you haven't heard the word telemetry before, telemetry consists of the two parts tele and metry. Tele stands for remote and metry for metering, or measuring. The word telemetry stands for remote metering, or remote measuring. And this is actually what you want to do when you send log data from your application to a remote destination like an Event Hub. But now you might think instead of using an Event Hub here, you could also build a web API with ASP.NET Core that takes the log data from your application and stores it in a database. So, what's exactly the advantage of using an Event Hub instead of a classic ASP.NET Core Web API? An advantage is that you don't have to write any code to set up an Azure Event Hub. You just go to the Azure portal and there you can create and configure an Event Hub very easily. It's a platform as a service. But there are other advantages, and a really big advantage is scaling. Azure Event Hubs can be used to collect more than one million events per second. That's a massive number of events. The total data volume of these events can be more than one gigabyte per second. So, when you think again about your application that logs some user events to the Event Hub, this means that there can be massive amounts of events logged at the same time, and the Event Hub can handle that huge amount of incoming data. But also at small scale, an Azure Event Hub is a great choice, because another advantage is that an event hub is quite cheap. In the smallest configuration, you get an Event Hub for around $11.00 per month. On top of that, you have to pay for the incoming events, and the price there is $0.028 per million of incoming events. of incoming events. That means you can set up an Event Hub that enters on average more than 100 million events per month for less than $15.00. Beside classic applications, Event Hubs are also used quite often to collect telemetry data that comes from devices and gateways. For example, GPS records could send position data periodically to an Event Hub, or a weather station could send temperatures every 5 seconds to an Event Hub. Quite often there are devices that can't connect to the internet on their own. Then usually an application grabs sensor data from these devices and pushes that sensor data to an Event Hub. Such an application grabbing sensor data from devices and pushing it to an Event Hub is called a gateway. So you see, besides sending log data from a classic application, you could use Azure Event Hubs for typical Internet of Things scenarios where you send sensor data from devices and gateways to an Event Hub. All these different senders, an application, a device, or a gateway, have in common that it's always a piece of code that sends an event to an Event Hub. And the Event Hub itself is easy to set up and scales very well. Now let's assume that events are sent to your Event Hub. Then you can consume and analyze these events.

Consume and Analyze Events After your applications or your devices have sent events to your Event Hub, you can consume and analyze these events. One way to do this is to set up a stream analytics job in Azure. With Stream Analytics, you can use a SQL-like query syntax to read and analyze the events from the Event Hub. For example, you could group events to find out how many users navigated to the information page of your application in the last minute. Stream Analytics pushes the results of your query to an output that you have defined. The output can be, for example, Power BI. In Power BI, you can build a real-time dashboard that shows the data coming from the Stream Analytics job. This means now that, for example, an application sends events to the event hub, the Stream Analytics job processes these events, and Power BI displays the data in near real time. And exactly this architecture shows another advantage of Azure Event Hubs. Besides scaling, Azure Event Hubs have a very low latency. That means that the massive amount of data goes through the Event Hub very fast. To be more concrete, the data goes through the Event Hub below one second.