TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 February 5, 2014 Notices References in content to IBM products, software, programs, services or associated technologies do not imply that they will be available in all countries in which IBM operates. Content, including any plans contained in content, may change at any time at IBM's sole discretion, based on market opportunities or other factors, and is not intended to be a commitment to future content, including product or feature availability, in any way. Statements regarding IBM's future direction or intent are subject to change or withdrawal without notice and represent goals and objectives only. Please refer to the developerWorks terms of use for more information. TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 Document Roadmap 1 Introduction ..............................................................................................................................................................3 1.1 Dataset Preview.................................................................................................................................................3 1.2 Pre-requisites.....................................................................................................................................................3 1.2.1 Technical.......................................................................................................................................................3 1.2.2 DASH Configuration......................................................................................................................................3 1.3 2 Document Change History.................................................................................................................................3 Availability of Implementation Artifacts ...................................................................................................................4 2.1.1 Unpack the Artifact ZIP File ..........................................................................................................................4 2.1.2 Import the Artifacts ......................................................................................................................................4 2.1.3 Verifying the Artifacts...................................................................................................................................4 2.1.4 Finally, it’s DASH Time!................................................................................................................................. 4 3 A Brief Overview of TBSM “Metric History” .............................................................................................................5 4 “Metric History Metadata” Dataset..........................................................................................................................6 4.1 4.1.1 Target Widgets .............................................................................................................................................6 4.1.2 Inappropriate Widgets .................................................................................................................................6 4.2 5 Sample Use Case................................................................................................................................................7 “Metric History” Dataset...........................................................................................................................................9 5.1 6 Dataset Use Cases..............................................................................................................................................6 Metric History Dataset Use Cases .....................................................................................................................9 5.1.1 Target Widgets .............................................................................................................................................9 5.1.2 Inappropriate Widgets .................................................................................................................................9 5.1.3 But there is no TWA User Experience ........................................................................................................10 5.2 Sample Use Case..............................................................................................................................................10 5.3 Working with the Dataset................................................................................................................................11 5.3.1 Selecting the Dataset..................................................................................................................................11 5.3.2 Mandatory Dataset Parameter ..................................................................................................................12 5.3.3 Dataset Columns ........................................................................................................................................13 5.3.4 Dataset Parameters....................................................................................................................................14 Technical Notes.......................................................................................................................................................15 6.1 Design Goals of the “Metric History” Dataset .................................................................................................15 6.2 What about that “Data Source” Artifact in the Project? .................................................................................16 TBSM Development Team Page 2 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 1 Introduction This document describes how to access TBSM metric history data for visualization on a DASH-based dashboard. This is the same data visualized by TBSM’s Time Window Analyzer. Because the TBSM UI data provider does not currently provide a dataset for this purpose, Impact capabilities are used to provide access to this data. A nice side-effect of this implementation is that all of the created artifacts can be provided for your use! Complete details on this can be found on page 4. 1.1 Dataset Preview The following datasets are provided: 1. Metric_History_Policy provides the actual metric history items for a TBSM service 2. Metric_History_Metadata_DataType provides access to all of the currently defined metrics 1.2 Pre-requisites 1.2.1 Technical Experience with the following technologies is highly recommended: 1. Impact’s Data Model and Policies (JavaScript is used) 2. TBSM service model, especially templates 3. DASH dashboard creation 1.2.2 DASH Configuration The availability of a DASH connection to Impact is a pre-requisite for the remaining steps. Note: a generic ID has been used for this connection… That’s a best practice that you should be using too! 1.3 Document Change History Version Date Changes 1.1 5-Feb-14 Original document TBSM Development Team Page 3 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 2 Availability of Implementation Artifacts The complete set of Impact artifacts implementing these datasets have been exported and provided “as is” for your convenience and/or education: Dashboards-Accessing-Metric-History-Data_v1.1-Artifacts.zip The following sections explain how to import these artifacts to your TBSM/Impact system. 2.1.1 Unpack the Artifact ZIP File Unzip the artifacts file to a convenient directory on the machine where the TBSM data server has been installed. For example: my-artifacts 2.1.2 Import the Artifacts An installation script has been provided to complete the import of these artifacts into your Impact system. cd my-artifacts install (on Windows) or ./install.sh (on Unix) At the verification prompt, type ‘y’. 2.1.3 Verifying the Artifacts To verify, login to TBSM and open System Configuration > Event Automation > Data Model from the TIP navigator. You should now see the Metric_History_Project in the Project drop-down list. Once you open the project, you see the provided artifacts. One other artifact is located under System Configuration > Event Automation > Policies. 2.1.4 Finally, it’s DASH Time! You can now begin creating your own DASH pages to visualize your metric history data. TBSM Development Team Page 4 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 3 A Brief Overview of TBSM “Metric History” TBSM automatically manages a short term history database for metric values. This includes: 1. TBSM status changes, “OverallAttribute”, which is enabled OOTB 2. other template defined metrics that the customer enables for metric collection Figure 1 - Enabling a metric for historical collection Did you know? The eDayTrader sample service model, also provided on SMC, does not enable Store data for this rule for TWA for any of its provided templates/rules. You will need to use Service Administration and manually activate TWA collection for any metrics you would be interested in… Alternatively, RAD shell commands can be used: http://pic.dhe.ibm.com/infocenter/tivihelp/v3r1/index.jsp?topic=%2Fcom.ibm.tivoli.itbsm.doc_6.1.1% 2Fadminguide%2Fbsma_shlc_timewindow.html TBSM Development Team Page 5 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 4 “Metric History Metadata” Dataset Each metric activated for history collection has a set of metadata defining its characteristics. That metadata is saved in the HISTORY_METRIC_META database table. 4.1 Dataset Use Cases The primary (only?) use of this dataset is to provide specific metric context to other widgets that are using the metric history dataset. 4.1.1 Target Widgets Given this dataset’s nature, it seems only suited to these widgets: 1. list 2. table 3. Value Status Gauge – when used as a metric selector (see below) 4.1.2 Inappropriate Widgets It is not appropriate for: 1. all other scalar widgets 2. topology / tree table 3. any chart TBSM Development Team Page 6 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 4.2 Sample Use Case This page illustrates two techniques for implementing a metric context selector. Usually, you would use just one or the other on any page… Publisher 1 2 Publisher Subscriber The History of Selected Metric table displays metric history items (details coming in the next section). It’s defined to initially pull the “Status Changes” metric history for the Dallas service (2) when the page is opened. It is configured to subscribe to NodeClickedOn events so it will receive events from any publisher widgets on the page [this is not the default wiring behavior for a table; this is the kind of operational detail that a DASH page designer must be aware of to create compelling pages]. The Select a Metric list contains all of the currently defined metrics. Its content is all of the rows in the metric metadata dataset. By default, it publishes NodeClickedOn events. TBSM Development Team Page 7 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 The status gauges (Buy and Sell) are each configured to represent a specific metric. This is accomplished by requesting the specific metric name when configuring each gauge. Both gauges have also been configured to publish NodeClickedOn events [this is not the default wiring behavior for a gauge]. Note: while this gauge is not the best user experience, it is the only available scalar widget that can be reasonably used… So any of the following user interactions will result in the History of Selected Metric table updating its content based on a new metric context (1): 1. clicking a row in the Select a Metric list 2. clicking either the Buy or Sell status gauge TBSM Development Team Page 8 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 5 “Metric History” Dataset While there are multiple database tables required to maintain metric history, we’ll base our implementation on the following view: Using this view allows us to query history items using a combination of user friendly names (which is very important when it comes to configuring a widget to use this dataset): 1. TBSM service instance (HISTORYRESOURCENAME) 2. metric (METRICNAME) Each database row represents one metric data point for a specific service/metric pair and includes: • • • • 5.1 timestamp (VALUETIME) Tech Note: stored as a GMT value (i.e. local time zone never applied) metric value (VALUE) previous metric value (PREVVALUE) the ID fields are internal database bookkeeping items Metric History Dataset Use Cases 5.1.1 Target Widgets Since history implies a set of data points, our dataset is intended for use by these widgets: 1. list 2. table 3. line chart 5.1.2 Inappropriate Widgets It is not appropriate for: 1. scalar widgets 2. topology / tree table 3. other chart types TBSM Development Team Page 9 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 5.1.3 But there is no TWA User Experience While metric history data will be visualized using the above widgets, none will provide the advanced user experience and capabilities of TWA… 5.2 Sample Use Case This page illustrates the use of this dataset on an active page where the metric history based widgets react to selections of a TBSM service… Context via NodeClickedOn Line List Table Chart The Select a site… list publishes NodeClickedOn events providing TBSM service context to the rest of the page’s widgets. All of the widgets visualizing metric history data were configured to not publish but rather subscribe to NodeClickedOn events [again, overriding default behavior]. TBSM Development Team Page 10 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 5.3 Working with the Dataset These sections provide detailed information about using this dataset, including the user experience when configuring a widget. 5.3.1 Selecting the Dataset When configuring a widget to display metric history data, enter “metric_history” as the search term then click the Search button. After the dataset(s) are displayed, select the Metric_History_Policy dataset. The dataset attributes are displayed at the top of the Visualization Settings page. TBSM Development Team Page 11 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 5.3.2 Mandatory Dataset Parameter executePolicy is a standard, Impact policy-based dataset parameter (it is not unique to the metric history implementation). • • When checked, the policy is executed when the widget loads/refreshes When unchecked, the data is retrieved from Impact’s policy cache If multiple widgets on the page are using the same policy and parameter(s) then the best practice is to check this on one widget and uncheck on the other N – 1 widgets. TBSM Development Team Page 12 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 5.3.3 Dataset Columns The dataset provides just 5 data columns. Notice that these differ from the actual metric history database shown above! This table shows how the metric history database columns are mapped to the dataset’s columns (this happens within the implementation policy). HISTORY_VIEW_RESOURCE_METRIC_VALUE View Column Dataset Column Notes TBSMServiceContextsCollectionSelectedServices See 5.3.4.1.1 Context Override for why this non-obvious value was chosen HISTORYRESOURCEID HISTORYRESOURCENAME METRICID METRICNAME Metric PREVVALUE VALUE VALUETIME Due to issues with Impact 6.1.1 GA code handling of null cell values, this column has been omitted for now… It will be restored in a future Impact fix pack Value Timestamp UIObjectId This is an Impact auto-generated value that satisfies the DASH widget’s requirement that each row provide a unique ID value This column is for internal use only and should always be hidden… TBSM Development Team Page 13 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 5.3.4 Dataset Parameters These parameters allow the page designer to define the initial data displayed by the widget. 5.3.4.1 “TBSM Service” Parameter Specify the TBSM service instance name to search for in the HISTORYRESOURCENAME column. For proper dataset operation you must specify a service! There is no parameter default. To access multiple services, specify them as a comma separated list. 5.3.4.1.1 Context Override If the policy is passed a service instance name in the TBSMServiceContextsCollectionSelectedServices parameter, it is automatically used instead of this parameter. This supports receiving context from TBSM-based datasets via NodeClickedOn events… 5.3.4.1.2 Limitations • Since this is not a TBSM provided dataset, no Extended Selection support is provided (i.e. Add… button) • At the moment, a Line Chart only seems to support one service. Specifying multiple services does not yield a separate line for each service as desired (and seen in TWA)…. • For List and Table, you’ll need to display the service name to know who it belongs to when displaying multiple services 5.3.4.2 “Metric Name” Parameter Specify the metric name to search for in the METRICNAME column. To access multiple metrics, specify them as a comma separated list The parameter default is OverallAttribute, which is the standard TBSM overall status. The format for all other TBSM metric names is: template.rule Examples: • • AEDayTrader.Buy AEDayTrader.Sell 5.3.4.2.1 Context Override If the policy is passed a metric name(s) in the METRICNAME parameter, it is automatically used instead of this parameter. This supports receiving context from the Metric History Metadata dataset… 5.3.4.3 “Historical Order” Parameter This parameter allows the page designer to control the ordering of the returned history items. It is needed for widgets that do not provide sort capabilities, like charts. • • Checked = historical……………..oldest items first (i.e. ascending date order) Unchecked = reverse historical…newest items first (i.e. descending date order) 5.3.4.3.1 Context Override At this time, the policy does not support overriding this parameter. TBSM Development Team Page 14 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 6 Technical Notes 6.1 Design Goals of the “Metric History” Dataset To be a useful, well-behaved dataset, the “metric history” dataset needed to meet all of the following requirements. These evolved over the dataset’s implementation as it was used in anticipated real world scenarios. 1. Dataset Parameters: allows the page designer to define the initial metric history visualized TBSM service name – defines the service(s) that history is desired for Metric name – defines the desired metric(s) Historical Order - defines the ordering of the history items Note: • all metric history items available based on the first 2 parameters are retrieved • the ability to query for a specific “time window” like TWA is not currently included… Implementation: handled by the policy’s input parameters 2. Widget Data Requirements: satisfy all data requirements for proper collection widget operation Each row must include a "unique ID" field Implementation: handled within the policy because the metric history database does not contain a single column that uniquely identifies a particular metric history triplet (service / metric / timestamp) Without this, history data displayed erratically in the List and Table widgets 3. NodeClickedOn Event Subscriber Requirement • • • Goal: inter-operate with NodeClickedOn events originating from TBSM datasets The dataset’s initial TBSM service context (set by the TBSM service name parameter above) should automatically update to the service specified in the event’s TBSMServiceContextsCollectionSelectedServices field Implementation: handled via policy logic 4. NodeClickedOn Event Publisher Requirement Goal: publish NodeClickedOn events that are usable by TBSM dataset based widgets The TBSM dataset context should reflect the TBSM service of a selected metric history item Since this dataset will inherit the default widget NodeClickedOn event generation (i.e. the event payload simply contains all of the dataset’s columns), we’ll make the dataset’s column name be the same name as the one that TBSM places in its NodeClickedOn event payload. We do this by mapping the HISTORYRESOURCENAME database column to TBSMServiceContextsCollectionSelectedServices. Implementation: handled by the policy’s results schema and runtime logic TBSM Development Team Page 15 of 16 2/5/2014 TBSM + JazzSM UI Services (DASH) Integration: Visualizing TBSM Metric History Data Version 1.1 6.2 What about that “Data Source” Artifact in the Project? Initially, the TBSMMetricHistory data source was used. It is created during TBSM installation and used by the TBSM product code. It was discovered that when the metric history project was exported, it included its complete data source definition. So if these artifacts would have been provided, they would have overwritten the target machine’s copy which is not a great idea. The solution was to create a project specific data source, Metric_History_DataSource, that is configured exactly the same as TBSMMetricHistory. The provided install script automatically configures the Metric_History_DataSource to be exactly the same as the local TBSMMetricHistory. TBSM Development Team Page 16 of 16 2/5/2014