Notes on log2csv, a console application that converts
standard BuildAX and iOpt log files to a csv format (all data for all
sensors at a user defined time intervals on one line). There is also
a header line with title for each column (sensor id + unit).

To compile:

Assumes the GNU tool chain in available (gcc + gfortran).

  make -f Makefile_log2csv

This creates executable log2csv.

If you want to tidy up the folder issue the command:

  make -f Makefile_log2csv clean

All of the source is under GPL. If you change it pass back
the changes to me so that I can incorporate the improvements.

How to use:

Current command line arguments can be found via:
./log2csv -help
command line is:
 [-act {L|i|2} ]
   i is iOpt log
   2 is BuildAX LRS 2 log
   L scan and list sensors and timings
 [-tol {10|20|30|40} ]
   10 20 30 or 40 seconds tollerance for
   identifying relevant timesteps in log
 [-name <sensor name> ] if included
   restricts reporting to this sensor
 
 -nodbt exclude temperature
 -norh  exclude relative humidity
 -nolux exclude Lux
 -nopir exclude PIR
 -nosw exclude switch count
 -ex XXXXXXXX exclude sensor XXXXXXXX from report
 -ts timesteps per hour
 [-file <log file name>]
 [-help  this message
 
The input files are assumed to be a csv file in the
following format two formats:
 . . .
2018/10/23,18:09:15,4294D9B6,-58,2,0,20,3019,41.68,242,0,1,138,1
2018/10/23,18:10:00,420B14AC,-51,2,0,20,2982,41.33,243,0,1,118,1
2018/10/23,18:10:03,42724E0F,-64,1,0,20,3187,36.17,251,0,0,768,1
2018/10/23,18:10:18,4294D9B6,-70,1,1,20,2990,40.73,248,144,14,6800,1
2018/10/23,18:10:20,4294D9B6,-49,2,2,20,2990,40.73,248,144,15,7207,1
2018/10/23,18:10:34,42CBD294,-54,1,0,20,3011,38.22,249,266,0,6289,1
2018/10/23,18:10:59,420B14AC,-44,1,1,20,2980,39.53,249,288,7,3917,1
2018/10/23,18:11:06,42724E0F,-45,1,1,20,3185,37.39,255,301,0,5619,1
2018/10/23,18:11:15,420B14AC,-44,2,2,20,2980,39.53,249,288,8,4166,1
2018/10/23,18:11:22,4294D9B6,-50,1,3,20,2990,38.52,251,318,21,9745,1
2018/10/23,18:11:24,4294D9B6,-49,2,4,20,2990,38.52,251,318,21,9941,1
2018/10/23,18:11:39,42CBD294,-57,1,1,20,3012,35.97,251,268,0,8514,1

date,time-string,sensor-ID,radio-strength,...

Processing this to get 12 timestep per hour via

 ./log2csv -act 2 -ts 12  buildAx_sensors.csv
 
 yields....
BuildAX summary from LRS
number of sensors  4
include,degC
include,Lux
include,RH
include,PIR
include,SW
sensor list 420B14AC,42724E0F,4294D9B6,42CBD294
JulianTime,yyyy-mm-dd,hh:mm,420B14AC_degC,420B14AC_Lux,420B14AC_RH,420B14AC_PIR,420B14AC_SW,42724E0F_degC,42724E0F_Lux,42724E0F_RH,42724E0F_PIR,42724E0F_SW,4294D
9B6_degC,4294D9B6_Lux,4294D9B6_RH,4294D9B6_PIR,4294D9B6_SW,42CBD294_degC,42CBD294_Lux,42CBD294_RH,42CBD294_PIR,42CBD294_SW,
296.000000,2018/10/23 00:00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
296.003479,2018/10/23 00:05,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
296.006958,2018/10/23 00:10,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
296.010406,2018/10/23 00:15,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
. . .
296.756927,2018/10/23 18:09,25.600,195.00,36.080,0.00,1.0000,25.600,241.00,34.740,0.00,1.0000,24.200,0.00,41.680,1.0000,1.0000,25.300,232.00,34.430,0.00,1.0000
296.760406,2018/10/23 18:14,25.600,195.00,36.080,0.00,1.0000,25.600,241.00,34.740,0.00,1.0000,25.300,248.00,35.260,1.0000,1.0000,25.300,232.00,34.430,0.00,1.0000
296.763885,2018/10/23 18:19,23.000,14.000,36.260,0.00,1.0000,25.500,253.00,33.570,0.00,1.0000,25.500,247.00,33.080,0.00,1.0000,25.400,217.00,33.410,0.00,1.0000
296.767365,2018/10/23 18:24,22.400,7.0000,37.740,0.00,1.0000,25.500,252.00,33.230,0.00,1.0000,26.600,0.00,41.810,0.00,1.0000,26.100,289.00,31.720,0.00,1.0000
296.770844,2018/10/23 18:29,22.200,3.0000,38.180,0.00,1.0000,25.500,252.00,33.130,0.00,1.0000,29.200,13.000,36.330,0.00,1.0000,25.800,287.00,31.480,0.00,1.0000
296.774292,2018/10/23 18:34,21.900,1.0000,38.500,1.0000,1.0000,28.300,1.0000,41.950,0.00,1.0000,27.400,24.000,29.570,0.00,1.0000,25.700,288.00,31.280,0.00,1.0000
296.777771,2018/10/23 18:39,21.700,0.00,38.690,0.00,1.0000,27.200,279.00,32.620,0.00,1.0000,27.100,23.000,29.570,0.00,1.0000,25.600,288.00,31.180,0.00,1.0000
. . . 
 
Another log file format is shown below:

device,time,temperature,co2,humidity,battery,dewpoint,room
70B3D55F200001BB,2021-04-22 00:04:40+00,17.8,455,58.5,3,9.55,lounge
70B3D55F200001BC,2021-04-22 00:02:19+00,19.8,445,52,3,9.65,main bedroom
70B3D55F200001B9,2021-04-22 00:02:48+00,18.4,464,57,3,9.72,second bedroom
70B3D55F200001BC,2021-04-22 00:09:51+00,19.7,430,52,3,9.56,main bedroom
70B3D55F200001B9,2021-04-22 00:10:20+00,18.4,449,57,3,9.72,second bedroom
70B3D55F200001BC,2021-04-22 00:17:20+00,19.6,428,52,3,9.47,main bedroom
70B3D55F200001BB,2021-04-22 00:12:09+00,17.8,449,58.5,3,9.55,lounge
70B3D55F200001B9,2021-04-22 00:17:48+00,18.3,453,57,3,9.63,second bedroom
70B3D55F200001BB,2021-04-22 00:19:38+00,17.7,465,58.5,3,9.46,lounge
70B3D55F200001BC,2021-04-22 00:24:51+00,19.6,439,52.5,3,9.61,main bedroom
70B3D55F200001B9,2021-04-22 00:25:18+00,18.3,457,57,3,9.63,second bedroom
70B3D55F200001B9,2021-04-22 00:32:49+00,18.2,449,57,3,9.54,second bedroom
70B3D55F200001BB,2021-04-22 00:27:10+00,17.7,458,59,3,9.58,lounge
70B3D55F200001BB,2021-04-22 00:42:08+00,17.6,443,59,3,9.49,lounge
70B3D55F200001BC,2021-04-22 00:32:23+00,19.5,446,52.5,3,9.52,main bedroom
 . . .

Requesting this be parsed into 12 timesteps per hour via the command:
./log2csv -act i -ts 12 -file iOpt_sensors.csv

yields:
iOpt sensor log
number of sensors  3
include,degC
include,RH
sensor list 3,1,2
JulianTime,yyyy-mm-dd,hh:mm,1B9_degC,1B9_PPM,1B9_RH,1BB_degC,1BB_PPM,1BB_RH,1BC_degC,1BC_PPM,1BC_RH,
112.000000,2021-04-22 00:00,18.400,464.00,57.000,17.800,455.00,58.500,19.800,445.00,52.000
112.003471,2021-04-22 00:05,18.400,464.00,57.000,17.800,455.00,58.500,19.750,437.50,52.000
112.006943,2021-04-22 00:10,18.400,449.00,57.000,17.800,449.00,58.500,19.700,430.00,52.000
112.010414,2021-04-22 00:15,18.400,449.00,57.000,17.800,449.00,58.500,19.650,429.00,52.000
112.013885,2021-04-22 00:20,18.300,453.00,57.000,17.700,465.00,58.500,19.600,428.00,52.000
112.017365,2021-04-22 00:25,18.300,455.00,57.000,17.700,461.50,58.750,19.600,439.00,52.500
112.020836,2021-04-22 00:30,18.300,457.00,57.000,17.700,458.00,59.000,19.550,442.50,52.500
112.024307,2021-04-22 00:35,18.200,449.00,57.000,17.700,451.00,59.000,19.500,446.00,52.500
112.027779,2021-04-22 00:40,18.200,448.00,57.000,17.650,447.00,59.000,19.500,437.00,52.500
112.031250,2021-04-22 00:45,18.200,447.00,57.000,17.600,443.00,59.000,19.450,441.00,52.500
112.034721,2021-04-22 00:50,18.200,444.00,57.000,17.600,453.00,59.000,19.400,445.00,52.500
112.038193,2021-04-22 00:55,18.150,447.50,57.000,17.600,450.50,59.000,19.300,440.00,52.500
112.041664,2021-04-22 01:00,18.100,451.00,57.000,17.600,448.00,59.000,19.300,439.00,52.500
112.045135,2021-04-22 01:05,18.100,440.00,57.000,17.500,459.00,59.000,19.300,438.00,52.500
112.048615,2021-04-22 01:10,18.050,436.50,57.000,17.500,465.00,59.000,19.200,430.00,52.500
112.052086,2021-04-22 01:15,18.000,433.00,57.000,17.500,471.00,59.000,19.150,438.00,52.500
112.055557,2021-04-22 01:20,18.000,433.00,57.000,17.500,453.00,59.000,19.100,446.00,52.500
112.059029,2021-04-22 01:25,17.950,442.50,56.750,17.450,452.50,59.000,19.100,430.00,52.500
112.062500,2021-04-22 01:30,17.900,452.00,56.500,17.400,452.00,59.000,19.100,428.50,52.500
112.065971,2021-04-22 01:35,17.900,457.00,56.500,17.400,449.00,59.000,19.100,427.00,52.500
112.069443,2021-04-22 01:40,17.850,449.50,56.500,17.400,446.50,59.000,19.000,450.00,52.500
. . .

The -act L option scans the file and generates a brief report. Example:

./log2csv -act L buildAx_sensors.csv 
 
 the input file is: buildAx_sensors.csv
command line action is L
 processing...buildAx_sensors.csv                                                                                                                             
 
 In buildAx_sensors.csv error reading (log line) EOF sensed.
 Rewinding the file after scan for sensors.
 Sensor number-of-hits typical-interval (sec)
 4294D9B6  0.608795166             810   64.9381485    
 420B14AC  0.608337402             861   61.0457039    
 42724E0F  0.609100342             824   63.8668327    
 42CBD294  0.608825684             803   65.5075226    
 sensors found:            4
4294D9B6 420B14AC 42724E0F 42CBD294

 
If you want to ignore certain types of data e.g. PIR and Switch events then modify
the command line:

  ./log2csv -act 2 -nopir -nosw houses.csv

which creates:
BuildAX summary from LRS
number of sensors  4
include,degC
include,Lux
include,RH
ignore,PIR
ignore,SW
sensor list 42A45156,42CAD40D,42D5E85D,42E77A24
JulianTime,yyyy-mm-dd,hh:mm,42A45156_degC,42A45156_Lux,42A45156_RH,42CAD40D_degC,42CAD40D_Lux,42CAD40D_RH,42D5E85D_degC,42D5E85D_Lux,42D5E85D_RH,42E77A24_degC,42E77A24_Lux,42E77A24_RH,
 22.461805,2015/01/22,11:05,18.700,3.0000,46.860,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00,0.00
 22.468750,2015/01/22,11:15,18.700,3.0000,46.890,20.700,65.000,37.380,20.000,7.0000,38.560,16.800,6.0000,50.360
 22.472223,2015/01/22,11:20,18.600,4.0000,49.140,21.000,68.000,37.240,20.100,7.0000,38.240,16.800,7.0000,50.810
 22.475695,2015/01/22,11:25,18.500,5.0000,51.810,21.200,65.000,37.010,20.300,8.0000,38.000,16.800,6.0000,50.450
 22.479168,2015/01/22,11:30,18.600,4.0000,54.280,21.300,63.000,36.940,20.500,7.0000,36.970,16.700,6.0000,50.440
 22.482639,2015/01/22,11:35,18.700,7.0000,54.440,21.500,57.000,36.430,20.700,8.0000,36.540,16.700,5.0000,50.190
 22.486113,2015/01/22,11:40,18.700,7.0000,56.460,21.600,89.000,36.300,20.700,9.0000,36.410,16.600,7.0000,50.030
 22.489584,2015/01/22,11:45,18.900,21.000,58.770,21.600,89.000,36.300,21.000,12.000,36.100,16.500,8.0000,49.970

If you want to exclue BuildAX ENV sensors then the command line
takes the form:
./log2csv -act 2 -ex 42A45156 -ex 42CAD40D houses.csv

