Skip to content

Validation Tests

ODetecting Technical Deviations

NDW runs dozens of daily data validation tests for real-time traffic data (AVG), such as tests for zero measurements, data errors, unrealistic speeds, and speed differences between vehicle categories. These tests fall into four categories: intensity, speed, categorization, and completeness. A more detailed description of these validation tests can be found on Mijn NDW Login is required.

The Issue is Reported to the Supplier

Suspicious locations are manually checked before being reported to the supplier for resolution. More details on this process can be found on Mijn NDW. Login is required.

Technical Deviations are Excluded

Suspicious locations are marked as technically deviating until the issue is resolved, ensuring they are excluded from use in Dexter. In Dexter exports (intensity and speed, travel time), the default setting is to exclude technical deviations. Exclusions are included in a separate Excel tab so users can see what has been excluded. The export also includes an explanation column indicating why certain data is missing or incomplete. For example, "cat" signifies incorrect vehicle categorization, resulting in an empty cell. This improves user analysis by filtering out faulty data. here for the Dexter export page for intensity and speed. An overview of all technical exclusions in JSON format (always up to date) can be found op Dexter.

Suspicious Locations Can Also Be Excluded

Technically incorrect data is excluded by default. However, some locations are flagged as suspicious through an automated process but have not yet been manually verified. Dexter users can also exclude these data points. Benefit: This increases real-time accuracy and allows users to exclude potentially faulty measurements. Typically, NDW staff manually verify error signals before adding them to the exclusions, but this is time-consuming and not all signals can be processed immediately. To address this, an automated system alerts users to potentially unreliable data.

Users can exclude technically uncertain information in Dexter exports and reports via the 'technical exclusions' button in the selection screen. A pop-up window displays the validity/reliability percentage per selected location.

Based on experience with the current tests, certain signals are almost always accurate. These locations are assigned a low reliability score as soon as a test flags them. Depending on the test, the score varies:

  • 0% – The measurement is certainly invalid and is automatically excluded from processing. This percentage is reserved for manually verified information.
  • 5% – The measurement has been fully automated in the RWS DaVinci process and is considered certainly invalid. It is automatically excluded from processing.
  • 25% – The measurement is almost certainly invalid (~25% chance the data is correct). One or more highly reliable tests have flagged it.
  • 50% – The measurement is likely invalid (~50% chance the data is correct). One or more tests have flagged it, but they are not always accurate.
  • 75% – The measurement is suspicious (~75% chance the data is correct). One test flagged it, but it is not highly accurate.

Users of open data can also use the API for technical exclusions (available on the NDW open data page) to filter out faulty or suspicious data. This value is stored in the "validity" field.

Traffic Exclusions

Some roads experience exceptionally high or low traffic. To determine an average or typical daily intensity, these exceptional days may need to be excluded from calculations. Whether a day is classified as exceptional depends on the median daily intensity and its standard deviation over the past 120 days. The calculation follows these steps:

  • Retrieve daily intensity data for the past 120 days.
  • Remove days with less than 80% data availability.
  • Remove days flagged for technical exclusion (AVG_FLOW).
  • Remove outliers (days with intensity deviations greater than four standard deviations).
  • If fewer than 80 historical days remain, a traffic exclusion cannot be determined.
  • If at least 80 historical days remain, calculate the median intensity and standard deviation.
  • Adjust the median intensity and deviations based on the day of the week.
  • Days deviating by more than 2.5 standard deviations from the median are flagged as traffic exclusions.

Storingendashboard; inzicht in aangemelde locaties

Locations reported as issues to the supplier are listed in the Incident Dashboard. This dashboard provides an up-to-date view of locations with open or resolved reports in Topdesk, including explanations. This helps users check whether a location has already been reported. The storingenkaart can be found here.

Report Suspicious Locations Yourself

Noticed a technical issue at a location that hasn’t been reported yet? Check the (check here het storingendashboard). and report it via the NDW Service Desk: mail@servicedeskndw.nu

NOTE: Dexter distinguishes between technical and traffic-related deviations. You can mark traffic-related deviations in Dexter in addition to NDW-marked technical deviations. This feature allows users to exclude data based on traffic conditions, such as large-scale roadworks. More info here.

Go back to the previous page