8 years ago
Ghosts in the Data, Constructing Data Entities

Data Entities are seldom discussed concepts that primarily hide in the shadows or are ghosts on the periphery. These entities are data constructs that are observationally defined in terms of the underlying data set that can serve as the business level aggregation. It is common in many data sets to use identifier fields to define them. These entities are used for roll ups or aggregate metrics for reporting or data modeling purposes. In a fair number of data sets these entities correspond to customers, i.e. people. However, entities need not be constrained to the notion of people. These non-people entities and people based entities share many common traits in the data. You can construct entities for any data concepts that are necessary for the business needs (e.g. retail products, music or organizations). What we really want are entities that best reflect the physical world. Often this requires more than just the identifier fields. The Entity Effect is an entity concept in the data that is realized and can be utilized to further a process through an identified data driven relationship. In this article, we will explore the definition of more complex data entities and provide a simple example to illustrate these ideas.