Analysis of EvaluationProvider Results with Python

Last modified by David Nestle on 2018/09/25 14:31

Installation of Anaconda / Spyder / Python

  • Download and install Anaconda from https://www.continuum.io/downloadsUse default settings regarding Path
  • Start the Python Console and install missing components:
    conda install qt
    conda update spyder
  • Helpful: Search for Spyder in the Windows Startmenu, create Shortcut
  • Start Spyder
  • For the following steps: Close Git repository sema.
  • Usually you should make sure that plot are opened in separate windows:
    Tools > preferences > IPython console > Graphics > Graphics backend > Backend: Automatic
    See also Stackoverflow

Perform first evaluation steps of the tutorial evaluation

For the following steps we assume that you have run the GaRo evaluations described in the OGEMA EvaluationProvider tutorial.

  • Adapt the base path in git\sema\python\semaEvalConfigLocal\initData() to your file system
  • Run semaEvalConfigLocal first to get imports.
  • Then call from console: fp = semaEvalConfigLocal.getJsonFile('GaRoBaseResult') if you created the result file GaRoBaseResult.json, otherwise adapt accordingly
    Note: If you used the method GaRoEvalHelper.performGenericMultiEvalOverAllData to get started, the result name to use is "<Name of EvaluationProvider>Result" .
  • rgr.GaRoRead.getJsonSummary(fp)
    gets you a summary of the evaluation
  • grp.GaRoPlot.hist(fp, 2)
    plots a histogramm of the second ResultType that was given with getJsonSummary
  • rgr.GaRoRead.getGwOverview(fp, 2)
    provides mean and median per Gateway. It first calculates average values per day over all rooms, then the average of the day values
  • av = rgr.GaRoRead.getAvOverTime(fp, 2)
    provides a plot-able result of the average values per day over all rooms/gateways. Furthermore it provides mean/median over the time series. See the method documentation for simple plot information.

The library contains several more functions, which should be documented there including functions to plot and process CSV export data directly.

Structure of the Python repository

In git/sema/python/lib you find several libraries. Currently all these libraries just provide static functions:

  • GaRoRead: Functions to read data from OGEMA results. Reading is supported for
    • Room-Level of evaluations (typically the low level evaluations of OGEMA that provide one result per day and per room of input data)
    • Overall-Data (in OGEMA these are the evaluations on the Multi-Evaluation-Level) that take into account the results of several Room-Level evaluations from various rooms, gateways and/or day intervals
    • Source time series data: These are usually sensor / actor values recorded from the field test, exported into a CSV files that are provided in a Zipped directory structure.
  • GaRoPlot: Functions to plot data. Currently time series plots and histograms are supported. Some functions are intended to be called with similar input data as the reading functions in GaRoRead and directly read from files, some functions take time processed data as input. Currently time series are generally represented by Python dict structures that contain lists named 'timestamps' and 'data' for the information representation.
  • GaRoUtil: Some additional utility methods
  • WinHeatExp: Methods used for specific evaluations ('experimental'): These functions are not intended for general purpose, but they generate the plots and results presented in the evaluation power point and should thus be kept available as part of the documentation.

The file semaConfigLocal directly in the sema/python directory sets variables that are specific for the development PC and thus needs to be adapted to each target hardware. The file PlotEvalDataWLib.py contains a collection of code snippets that are intended to be adapted and executed in the IPython console of the Spyer environment. Typically such a snippet could also be put into a function in WinHeatExp, but for short and highly adaptable code this is an even quicker way of documentation.

Tensor Flow

Installation

Check Python version installed and what is required by TensorFlow. TensorFlow usually does not run on the latest Python version.

Install TensorFlow on Anaconda as described in the Tutorial provided by TensorFlow. You have to type the commands into the Anaconda Prompt (search for it in the windows start menu if you do not know how to find it otherwise). After activation of the tensorflow environment you have to install jupyter there and add it to known kernels:

conda install ipykernel

python -m ipykernel install --user --name tensorflow --display-name "Python (tensorflow)"

(as decribed here).

To start Spyder with TensorFlow support you have to open it from the Anacode console (see here):

conda install -n tensorflow pandas

conda install -n tensorflow matplotlib

evtl.

conda install -n tensorflow scikit-learn

conda install -n tensorflow spyder

spyder

Tags:
Created by David Nestle on 2017/09/04 19:35