Observe
Reliability measures are the measures that include quality/health metrics such as completeness, accuracy, consistency, timeliness, and validity of the data to ensure the trustworthiness and usability of the data:
● Freshness: The freshness of the data in an asset.
● Volume: Computes the total row count in the data.
● Schema: Computes the total number of columns in the data.
● Duplicates: Counts the number of rows that are identical based on either primary key or composite keys defined by the user.
Freshness
In freshness, the trend reports give an overview of how up-to-date the data is across various timelines.
x
x
The freshness measure is by default active for all the assets and provides the timeliness information of the dataset configured. If a user wants to know when was the last time the dataset was updated or how often the data gets updated, this measure offers a comprehensive historical perspective on the dataset's trends.
Frequently, optimal data freshness is essential for evaluating dataset quality before making crucial business decisions. In the example, the system captures freshness data and visualizes the trend. Additionally, the user can set a manual threshold to get notified when the freshness number is beyond a certain period.
Volume
In Volume, the trend report shows if there are any changes or deviations in the total row count across a time period.
x
Volume measure of an asset helps the user understand the changes in the record count of an asset over a period of time. Some use cases require estimating the business checks only on certain limited records.
The completeness dimension offers users insight into the trending record count, enabling them to implement corrective actions upon detecting anomalies. Users can choose to depend on ML-based anomaly detection for alerts or establish a manual threshold at a predetermined number, triggering platform alerts accordingly.
Schema
In Schema, the graph shows the changes and deviations in the total column count across the time period.
Actual result: It is the factual outcome generated from the data observability within the platform. It signifies the real, obtained result by understanding and analyzing the data.
Expected result: It refers to the anticipated or the predicted outcome based on the previous model or predefined criteria.
Comparing the expected result to the actual result in the reliability measures is essential in ensuring accuracy in predictions and assessing the success of various operations that will be performed in PDQ.
x
Schema measure refers to the number of columns in a table or a view. Ideally, the number of columns would remain the same and the DQScore viewed at the asset level would be based on these columns.
However, when columns are added or deleted from the source, this may lead to impacting the DQscore as these attributes contribute to its overall score calculation, misleading the consumer.
By default, PDQ do not consider any new columns for score calculation unless the user enables the column in the connection page. It is a best practice to keep a check on the trend in schema.
Alerts serve to indicate when an asset, attribute, or other aspect of the data set is not performing within the given parameters, set either by the platform or the user. Alerts are the primary mode of notification within the PDQ platform.
They serve to chronicle when an asset or attribute is performing outside the set of given parameters. Alerts can be used to monitor the performance of an asset or attribute. They are also linked to incident management through Issues.
On the Home Page: Alerts are featured as the third widget under the search bar.
Click on either 'Alerts' - Assets with Alerts are displayed.
Choose & Click on an 'asset' - Patients.
Click on the Alerts option in the top summary dashboard.
Click on an Alert.
By clicking on a chosen Alert, the user will be taken to an expanded view of the alert details, with a chart that displays when the alert was initiated, and its trend.
Under Result, the user will find the Data and Time when the asset was last run, as well as the Expected result, the Actual result, and the Priority.
The user can select whether or not the action was taken regarding the Alert under Actions.
x
x
Issues can be manually raised after an Alert is automatically detected by the PDQ platform.
To create an issue, a user must first select an Alert.
x
x
x
Users can view all issues from the PDQ homepage.
Click on the Explore icon.
Select: 'Issues'.
x
Select an 'asset' that has a logged Issue -
x
Change the 'Status' of the 'Issue' to:
New - default
In Progress
Resolved
x
x
x
Last updated
Was this helpful?
