Validation Setup

To Access: Setup -> Validation

Validation is used to automatically validate and modify the data. The Validation process runs every hour and "bad" readings could be accessible for maximum of 60 minutes until the process runs next.

NOTE: The Validation process alters the data in the database and you cannot view the Validated and Raw data together. The Raw data is still available in the Text Files and can be manually imported into the Database again if the Validation went wrong.

To set the Validation Configuration for a Site click the edit Google Maps Setup1 button for the corresponding site.

  1. The Active button must be checked for the validation to be active.
  2. Limits Select a lower and upper limit for each variable, the default Lower Limit is -6000 and the default Upper Limit is 6000. The Min and Max value for each variable is displayed to give a better understanding of the data already in the database.
  3. Gap Size The Gap Size is an important variable. When VDV validates data it first checks if the value is within the legal data range, if it is not within the legal limit it next checks if the number of consecutive values out of the legal data range are lower or equal to the Gap Size, if they are lower or equal to the gap size then the values are changed to the last legal value before, else no action is performed.

    For example: If the data values for a variable are “23.15; -100; 24.21; 23.98; -6999; -6999; -6999; 24.37; 25.14; 26.73;” and the lower limit for the variable is set to -50 and the upper limit is set to 150 and the Validation Method is Last Legal Value. If the Gap Size is set to 4 then after the validation process those same values would be: “23.15; 23.15; 24.21; 23.98; 23.98; 23.98; 23.98; 24.37; 25.14; 26.73;”. If the Gap Size had been set to 2 then after the validation process those same values would be: “23.15; 23.15; 24.21; 23.98; -6999; -6999; -6999; 24.37; 25.14; 26.73;”.
  4. Filter easily filters the variables. It is possible to write the name of the variable, part of a name or unit.
  5. Changes all the boxes at the same time. If there is a Filter on, this button only validates the variables that are Filtered.
  6. There are 3 different Validation methods that can be used.

Last Legal Value: Replaces validated values with the last legal value.

Linear Interpolation: Does a Linear Interpolation over the gap that is validated.

Replace Validated Values with NULL: This method replaces validated values with NULL resulting in a gap in the graph when plotted.

For example if the upper limit is 6000 and the lower limit is -6000, a sequence of: 10, 20, 6999, 6999, 50, with Gap Size 4 would be validated as:

  • 10, 20, 20, 20, 50 using Last Legal Value.
  • 10, 20, 30, 40, 50 using Linear Interpolation.
  • 10, 20, NULL, NULL, 50 using Replace Validated Values with NULL.