Compare Target#
The compare target performs a comparison of all records of two datasets in the verification phase. If some records do not match or are missing, the processing will stop. This way the compare target can be used for implementing tests.
Example#
targets:
verify_result_macro:
kind: compare
actual:
kind: relation
relation: ${relation}
expected:
kind: file
format: json
location: "${project.basedir}/test/data/results/${relation}/data.json"
schema:
kind: spark
file: "${project.basedir}/test/data/results/${relation}/schema.json"
Fields#
kind
(mandatory) (type: string):relation
description
(optional) (type: string): Optional descriptive text of the build targetactual
(mandatory) (type: dataset): Specifies the data set containing the actual data. Often you will either use a relation written to by Flowman or a mapping.expected
(mandatory) (type: dataset): Specifies the data set containing the expected data. In most cases you probably will use a file data set referencing some predefined results
Supported Execution Phases#
VERIFY
- Comparison will be performed in the verify build phase. If the comparison fails, the build will stop with an error
Read more about execution phases.