This is an annotated copy of the selftest_list file used by the
selfcheck facility of ESP-r. Its directives are scanned by prj
and used to control the assessments to be carried out.

The directives take the form of a tag followed by phrases or numbers.

The *bin_path_std and *bin_path_tst directives will need to be 
edited to specify the specific ESP-r distributions which are 
being tested. 

Alter *tollerance to specify the level of accuracy required. A '1.0'
is interpreted as 1W or 0.1W/m2 0.1GJ or 0.1C. A '2.0' is 2W 0.2W/m2 etc.

Alter *verbose to YES to see the interactions of bps and res or NO
for minimal chatter.

Alter *model_path to specify the location of the models to be tested.
For example /home/jon/alt_tester/ points to models in a users folder
while /opt/esp-r/validation/selftest would place this in the ESP-r
distribution. NOTE - permissions should be set to allow read and write
access.

As assessments are carried out bps and res generate reports which
are compared. The *report_path_std is the path to reports generated
with the standard version of ESP-r and *report_path_tst for the
test version. It is possible to use a tool such as meld to do visual
comparisons between these two sets of files.

If you include *output FILE then prj will not ask for the summary
file name. This can simplify automated use of the selftest facility.

The *label entry can be used to document the nature of the tests.

Interuptions in the file for purposes of this document are shown
with a . . . 

NOTE - the files and folders nominated in the directives must exist
prior to running the facility. Permissions should be set to allow 
read and write access.

*SELFTEST
*help	The models contained in the following sections demonstrate
*help	performance differences between two ESP-r versions.
*help   
*help	It is possible to run individual tests or multiple selections.
*bin_path_std /home/jon/esp-r_std/bin/
*bin_path_tst /opt/esp-r/bin/
*model_path  /home/jon/alt_tester/
*report_path_std /home/jon/alt_tester/std_reports/
*report_path_tst /home/jon/alt_tester/tst_reports/
*tollerance 1.0
*verbose NO
*output FILE /home/jon/alt_tester/all_16agu.txt
*label  -- standard tester models --
 . . .
 
The list of models and the assessment attributes.  Each starts
with *item

A number of models and assessments can be grouped together. For
example '*group	basic and detailed AIM' are models which test
a specific simulation feature. A *group line is followed by a
*help line with can further clarify the intent of the group.

If *item is followed by *name then it marks the start of a model
instance. The phrase following *name is what is presented in the
prj selection menu.

The *cfg line is the name of the ESP-r configuration file.

The *root line is the path to that file. The *model_path will
be pre-pended to derive a full path.

Models to be tested are assumed to include simulation parameter
sets and the *SPS line specifies which set to use. It must match
a SPS entry in the model cfg file.  By convention a model cfg
file will have a SPS entry for save level 4 and one for save
level 5. 

The *PIF line is a file which includes directives that drive
the res application to generate specific reports. A PIF file
is generated by a manual invocation of res recording the users
actions to generate a report. One or more are included in the
model folders and currently they replicate a the 'extract'
script used by tester.pl. For assesssments using 'save level 5'
the directives should take the form of '*PIF    SL5' as the
res application is not used.

The *res line should match the building results file in the
model cfg folder. The convention is to always use results.bres.

All models to tested must include an input.xml file which tells
bps to generate a 'summary' file. The *sum line specifies the name of
this file. In the examples below there is a slightly different
name for save 4 and 5 cases.

The *rep line is the file name that results.bres will be copied
to in the  *report_path_std and *report_path_tst folders. By
convention these reports generated by res end in '.dat'.

For the case of an assessment run with save level 5 a so-called
h3k file will be generated and its name is specified in the
*h3k line.

In the example below each model has one save level 4 and 5
assessment. 

 . . .
*item	
*group	basic and detailed AIM
*help	A set of models demonstrating ESP-r's functionality.
*item
*name	basic_AIM_MAX
*cfg	basic_AIM_MAX.cfg
*root	alberta_infil/cfg/
*SPS    test_s4
*PIF    basic_AIM.pif
*res    results.bres
*rep    basic_AIM_MAX_test_s4.dat
*sum    basic_AIM_MAX_test_s4.summary
*item
*name	basic_AIM_MAX_S5
*cfg	basic_AIM_MAX.cfg
*root	alberta_infil/cfg/
*SPS    test_s5
*PIF    SL5
*res    results.bres
*sum    basic_AIM_MAX_test_s5.summary
*h3k    basic_AIM_MAX_test_s5.h3k
*item
*name	basic_AIM_MIN
*cfg	basic_AIM_MIN.cfg
*root	alberta_infil/cfg/
*SPS    test_s4
*PIF    basic_AIM.pif
*res    results.bres
*rep    basic_AIM_MIN_test_s4.dat
*sum    basic_AIM_MIN_test_s4.summary
*item
*name	basic_AIM_MIN_S5
*cfg	basic_AIM_MIN.cfg
*root	alberta_infil/cfg/
*SPS    test_s5
*PIF    SL5
*res    results.bres
*sum    basic_AIM_MIN_test_s5.summary
*h3k    basic_AIM_MIN_test_s5.h3k
 . . .

In the following group there is a basic version of the
building and a detailed version of the building. Different
controls are applied to each. Each model is assessed for a winter
and summer week as well as an annual assessment. Each of
these is run with save level 4 and with save level 5.
The matrix is large so only a portion is included here.

Note you can add a visual separation in the file by starting
a line with a #

 . . .
#
*item	
*group	CCHT basic and detailed house
*help	A set of models demonstrating ESP-r's functionality.
*item
*name	basic_ctl_winter
*cfg	basic_ctl.cfg
*root	ccht_benchmark/cfg/
*SPS    win_sl4_ts1
*PIF    basic_win_sl4_ts1.pif
*res    results.bres
*rep    basic_ctl_win_sl4.dat
*sum    basic_ctl_win_sl4.summary
*item
*name	basic_ctl_summer
*cfg	basic_ctl.cfg
*root	ccht_benchmark/cfg/
*SPS    sum_sl4_ts1
*PIF    basic_sum_sl4_ts1.pif
*res    results.bres
*rep    basic_ctl_sum_sl4.dat
*sum    basic_ctl_sum_sl4.summary
*item
*name	basic_ctl_annual
*cfg	basic_ctl.cfg
*root	ccht_benchmark/cfg/
*SPS    ann_sl4_ts1
*PIF    basic_ann_sl4_ts1.pif
*res    results.bres
*rep    basic_ctl_ann_sl4.dat
*sum    basic_ctl_ann_sl4.summary
*item
*name	basic_ctl_winter_S5
*cfg	basic_ctl.cfg
*root	ccht_benchmark/cfg/
*SPS    win_sl5_ts1
*PIF    SL5
*res    results.bres
*sum    basic_ctl_win_sl5.summary
*h3k    basic_ctl_win_sl5.h3k
*item
*name	basic_ctl_summer_S5
*cfg	basic_ctl.cfg
*root	ccht_benchmark/cfg/
*SPS    sum_sl5_ts1
*PIF    SL5
*res    results.bres
*sum    basic_ctl_sum_sl5.summary
*h3k    basic_ctl_sum_sl5.h3k
 . . .

And a group can represent a mix of models. 

 . . .
#
*item	
*group	small offices
*help	Includes cellular_miso.
*item
*name	cellular_miso_s4
*cfg	cellular_miso.cfg
*root	cellular_miso/cfg/
*SPS    test_s4
*PIF    cellular_miso.pif
*res    results.bres
*rep    cellular_miso_s4.dat
*sum    cellular_miso_s4.summary
*item
*name	cellular_miso_s5
*cfg	cellular_miso.cfg
*root	cellular_miso/cfg/
*SPS    test_s5
*PIF    SL5
*res    results.bres
*sum    cellular_miso_s5.summary
*h3k    cellular_miso_s5.h3k
*item
*name	office_operations_s4
*cfg	office_operations.cfg
*root	cellular_offices/cfg/
*SPS    test_s4
*PIF    office_operations.pif
*res    results.bres
*rep    office_operations_s4.dat
*sum    office_operations_s4.summary
*item
*name	office_operations_s5
*cfg	office_operations.cfg
*root	cellular_offices/cfg/
*SPS    test_s5
*PIF    SL5
*res    results.bres
*sum    office_operations_s5.summary
*h3k    office_operations_s5.h3k
*item
*name	shading_s4
*cfg	obs_4_eachface.cfg
*root	shading/cfg/
*SPS    test_s4
*PIF    obs_4_eachface.pif
*res    results.bres
*rep    obs_4_eachface_s4.dat
*sum    obs_4_eachface_s4.summary
*item
*name	shading_s5
*cfg	obs_4_eachface.cfg
*root	shading/cfg/
*SPS    test_s5
*PIF    SL5
*res    results.bres
*sum    obs_4_eachface_s5.summary
*h3k    obs_4_eachface_s5.h3k
*end

The singnal for the end of the file is *end

When running in non-verbose mode the display in prj will look
something like this:

 . . .
  
 Assessing std: office_operations.cfg
   
 Assessing tst: office_operations.cfg
   
 Comparing office_operations_s4.dat identical
 Comparing office_operations_s4.summary within tollerance.
   
 Assessing std: office_operations.cfg
   
 Assessing tst: office_operations.cfg
   
 Comparing office_operations_s5.h3k identical
 Comparing office_operations_s5.summary within tollerance.
   
 Assessing std: obs_4_eachface.cfg
   
 Assessing tst: obs_4_eachface.cfg
   
 Comparing obs_4_eachface_s4.dat identical
 Comparing obs_4_eachface_s4.summary within tollerance.
   
 Assessing std: obs_4_eachface.cfg
   
 Assessing tst: obs_4_eachface.cfg
   
 Comparing obs_4_eachface_s5.h3k identical
 Comparing obs_4_eachface_s5.summary within tollerance.
 ASCII line differences (group)    0
 Tollerance line differences (group)    0
 ASCII line differences (overall)    0
 Tollerance line differences (overall)    0
Selftest sets:
    -- standard tester models --    e small offices                
  a basic and detailed AIM            _____________________        
  b basic and detailed basesimp     * all available models         
  c CCHT basic and detailed house     _____________________        
  d idealized HVAC                  ? help                         
  - exit menu

 Selftest sets:?> 


In this case no ASCII lines in the *dat files differed and the
difference in the values in the .summary files were within the 
tolerance requested.

Additional documentation can be found the the ESP-r Selftest
document.
