Data logging and analysis

Last modified by Christoph Nölle on 2017/02/03 01:13

Data Logging for Development Purposes versus User App Information

Generally data logging can be used for two purposes:

  • To collect data for development, testing and evaluation. In this case logging of a lot of data is usually feasible and required and the developers or power users involved in this process will usually want to use powerful standard tools for evaluation and analysis.
  • To present some operation results to the end user of an application. In this case the data presented has to be selected carefully as it should be limited to a complexity that can be dealt with by all (or at least most) of the app users and also the storage load has to be kept in mind here as the data from various applications on a system might add up to a large chunk of data. Here the presentation of the data should usually be integrated into the application in a very specific way to link to the app functionality and logic, so analysis by standard development tools is not the main focus here.

As you will most likely start to use data logging for your development purposes we explain the first bullet point in the directly following Sections whereas data logging for productive app purposes (seconds bullet point) is explained at the end of this page.

Data Logging on Developer / Power User Level

Configuration of data logging can be done in various ways. The Logging App provides a simple user interface for this purpose, but is not suitable for larger configurations and has the drawback that configurations are not stored across clean starts. Therefore, programmatical configuration is the recommended way.

Logging App

The logging App can be opened from the Home Screen

LoggingAppIcon.png Logging App Icon

It shows all SingleValueResources on the system and their logging status. This app allows to configure logging for alle applicable resources, but if the resource data base grows this gets very confusing. Until an updated Logging App is available this is the only way to configure logging for arbitrary values via a GUI. The app looks like this:

LoggingTool.png

At the top of the page a logging configuration can be defined. When logging is activated for any resource the logging configuration defined at the top will be given to this resource. We recommend to use the mode "ON_VALUE_UPDATE" which just logs all data coming in from a sensor or any write operation to a parameter. In this configuration the logging interval is not relevant.

Enable logging programmatically (recommended)

To manage tasks programmatically we recommend to use a separate 'configuration app' for your project that performs tasks like logging configuration, initial resource setups etc. The configuration app can be used for development purposes and be given to beta testers, but should not be required for productive use of your apps.

This way the logging setting will be recovered after a clean start; it would be tedious to configure this by hand every time. As we recommend clean starting with reloading XML files for development as standard launching this is very important. Settings made in the logging app will disappear after a clean start. Another important advantage of this method is the fact that the method below creates a ScheduleViewer configuration that allows to find the log data and view it together with predefined other data rows in the ScheduleViewer.

Note: In the example below we get access to the relevant resources not via ResorcePatterns, which should normally be used (see Event-based interaction ). As we do not know exactly in which order apps are started and resources are created we just wait for 10 seconds until we search for resources and assume that by that time everything is created. We would not find any resources created later on, but for testing this is the simplest way to get access (note that this only applies to the development system; in a productive system resources are loaded from the database before the apps start, hence there is no need to wait).

The most convienent way to enable logging for development purposes is to use the LogHelper class, which wraps the low level OGEMA loggin API, and creates a schedule viewer configuration on top:

ApplicationManager appMan = ...;
TemperatureResource temperature = ...;
LogHelper.addResourceToRoomLog(temperature, "Temperature measurement", null, appMan);

It creates a schedule viewer configuration per room, and associates the device log data to the room the device is located in.

Analysis: Schedule Viewer

From the Home Screen open the Schedule Viewer:

ScheduleViewer.svg

When you have just started up the App opening the Schedule Viewer will look like this.

ScheduleViewer.png

Note that in "all Schedules" only real schedules are shown, not logging data. Logging data can only be accessed here if configured in Resources of type ScheduleViewerConfig, which is done by the utility method in the code above. Such resources will be found by the Schedule Veiwer anywhere in the resource tree, so you can also create them yourself e.g. in an analysis OGEMA-App.

If you select a real single schedule you can also view and edit the values of the schedule (an additional table will appear).

Data Logging for Productive App Purposes

As explained above, productive apps will typically require less logging than development systems. In addition, it should not be necessary, normally, to create schedule viewer configurations in this case. Logging is enabled most conveniently using the #activateLogging method of the LoggingUtils class in the OGEMA resource-utils package:

LoggingUtils.activateLogging(temperature, -2);

The second argument is an identifier for the logging update rate. If it is set to a positive number, then a new data point will be logged periodically every X milliseconds. If it is set to -1 logging takes place every time the value changes, -2 is similar, but also stores a new point when the value of the resource is written but the new value does not differ from the old one.

Related


Tags:
Created by Jan Lapp on 2016/11/22 18:24