E is for Event

I recently wrote a children’s book called D is for Data: The ABCs of Data Analytics. This is the fifth in a series of behind-the-scenes, companion articles that will dive a little deeper into each term. We’ll explore the illustration used to define the term, how the word is used in the data world, and other interesting (to me) trivia.

Early computer systems used statuses to record changes over time. For example, a delivery may have several statuses—DRIVER ASSIGNED, TRUCK DISPATCHED, PICKED UP, DELIVERED. This worked well for executing deliveries, but eventually we had questions like, “how long does it take on average to make a delivery?” and “What percentage of deliveries were on-time?” So, in addition to the current status, we would include some timestamps abbreviated as TS—DELIVERY_APPOINTMENT_BEGIN_TS, DELIVERY_APPOINTMENT_END_TS, DRIVER_ASSIGNED_TS, TRUCK_DISPATCHED_TS, PICKED_UP_TS, DELIVERED_TS. Now, we can look back and compare these time stamps to get the on-time delivery percentage, or figure out how long it takes between pickup and delivery.

The problem with this way of recording the data with statuses and timestamps is that it doesn’t really account for all of the things that can go wrong. For example, what if the driver calls in sick and the delivery has to be reassigned, or if the truck breaks down and it has to be redispatched. One could continue to add more and more statuses and timestamps to try to capture all of this detail, but the model was clumsy and required too many changes. To get this level of detail you need a log of everything that happened—like a captain’s log, but much more detailed.

That’s where events come in. An event is a natural way to record something that happened. For example, there could be a driverAssigned event, a truckDispatched event, a packagePickedUp event and a packageDelivered event. This model is very flexible and can even be extended to other things… arrivedAtWarehouse, deliveryRefused, suddenStopDetected. Each event has a name and a time. From these events, we can quickly calculate the correct status. And, it’s easy to add new events into a system like this as the system gets more complex.

Recording data as events really took off with the internet and the internet-of-things. Web sites would send userClicked events, phones would send phonePickedUp and phonePutDown events, smart lights would send lightTurnedOn or lightTurnedOff events, smart doorbells and cameras send personDetected events. Because of the connectivity of the internet and the relatively cheap compute and storage in the cloud, we started collecting all of these events and analyzing them to find patterns. We track everything now and build huge datasets full of all kinds of events so that data analysts can find patterns and gain insights from what has been recorded.