Materialize design
When defining a materialized view, maximize its value by adding data items or columns to it based on computation or transformation of existing data items, on values passed in the query, or on combinations of these values when appropriate. Because of that, if the view is transient and is only used to improve query performance by reflecting the current state of the data, or to improve scalability, it can be stored in a cache or in a less reliable location. It can be a subset from a few different partitions combined.Ī view can be rebuilt if lost. The view doesn't have to be located in the same store or partition as the original data. If the source data is changing at the point when the view is generated, the copy of the data in the view won't be fully consistent with the original data.Ĭonsider where you'll store the view. If many queries are used, materialized views can result in unacceptable storage capacity requirements and storage cost.Ĭonsider the impact on data consistency when generating the view, and when updating the view if this occurs on a schedule. Materialized views tend to be specifically tailored to one, or a small number of queries. If you're not using Event Sourcing, you need to consider whether a materialized view is helpful or not. Prepopulating views by examining all events to determine the current state might be the only way to obtain information from the event store. In some systems, such as when using the Event Sourcing pattern to maintain a store of only the events that modified the data, materialized views are necessary.
#Materialize design manual#
Alternatively, consider using a scheduled task, an external trigger, or a manual action to regenerate the view. Ideally it'll regenerate in response to an event indicating a change to the source data, although this can lead to excessive overhead if the source data changes rapidly.
#Materialize design how to#
The figure shows an example of how the Materialized View pattern might be used.Ĭonsider the following points when deciding how to implement this pattern: In some cases it might be necessary to regenerate the view manually. You can schedule this to happen automatically, or when the system detects a change to the original data. When the source data for the view changes, the view must be updated to include the new information. A materialized view is never updated directly by an application, and so it's a specialized cache. A materialized view can even be optimized for just a single query.Ī key point is that a materialized view and the data it contains is completely disposable because it can be entirely rebuilt from the source data stores. In addition to joining tables or combining data entities, materialized views can include the current values of calculated columns or data items, the results of combining values or executing transformations on the data items, and values specified as part of the query. These materialized views, which only contain data required by a query, allow applications to quickly obtain the information they need. The Materialized View pattern describes generating prepopulated views of data in environments where the source data isn't in a suitable format for querying, where generating a suitable query is difficult, or where query performance is poor due to the nature of the data or the data store. To support efficient querying, a common solution is to generate, in advance, a view that materializes the data in a format suited to the required results set. When a query only needs a subset of the data from some entities, such as a summary of orders for several customers without all of the order details, it must extract all of the data for the relevant entities in order to obtain the required information. However, this can have a negative effect on queries.
#Materialize design series#
For example, when using NoSQL document store, the data is often represented as a series of aggregates, each containing all of the information for that entity. The chosen storage format is usually closely related to the format of the data, requirements for managing data size and data integrity, and the kind of store in use. When storing data, the priority for developers and data administrators is often focused on how the data is stored, as opposed to how it's read. This can help support efficient querying and data extraction, and improve application performance. Generate prepopulated views over the data in one or more data stores when the data isn't ideally formatted for required query operations.