REAL TIME: Report the data validation issues

Validation warnings are typically unresolved Managed Metadata tags, non existing users and Content-Types etc,

The view is named report_datavalidationwarnings and defined on the “Internal Database”.
Use it with a DataETL Step or a For-Each Step.

From the DataETL Step you can output the reporting result to:
  • Excel table in a new or existing workbook that subsequently can be uploaded to SharePoint
  • CSV file that subsequently can be uploaded to SharePoint
  • Items in a SharePoint List
  • Records in a database (Internal database, SQL Server or generic ODBC)

Active integrated reporting

The report can be a stand alone job you run or an integrated part of your migration set up since it is based on a standard DataETL Step.
You can choose to create the report at the end of a job or for each iteration in a For-Each and for example only if any errors happen (test the Variable [<ETL-Count>] > 0 to determine if any rows were reported). Then upload an output CSV or XLSX to SharePoint and name it according to the context in which it was created (for example ProjectId or CustomerId etc). Or create Items in a SharePoint list for review. Then finally notify someone or start a workflow using PowerAutomate.

Column name Column description for database view: report_datavalidationwarnings
destination_column Name of the column that triggered the warning
destination_url The URL to the list or Library containing the the column that triggered the warning
islatest Is the record of the validation operation the latest or not.
  • Value: 1 The record of the file is the latest
  • Value: 0 The record of the file is NOT the latest.
    Please note that values that validate correctly do not update this view.
sessionlog_message The message recorded in the session log. Any specific SharePoint or file system error is also recorded in this column
sessionlog_custominfo The information specified on the Options tab in any Step where you can mix Variables with hard coded text to form a context relevant value for example CustomerId:[<MyCustomerId>].
At runtime the Variable [<SessionLog-CustomInfo>] contains the current value. Use this column in combination with the column [sessionlog_id] and the variable [<SessionLog-Id>] to reference files being processed in the current job for example a For-Each iteration.
sessionlog_id
(Job runtime only)
The id of the job session in which the file was processed. Use this column in combination with the system Variable [<SessionLog-Id>] to query the files that are being processed by the current job (at runtime). This is useful when doing reporting as part of a job.
Note that you can also use the Variable [<SessionLog-CustomInfo>] to reference files that belong to for example a specific For-Each iteration or other section of the job.
The value for the [<SessionLog-CustomInfo>] Variable is defined on the Options tab in any Step and can contain Variables mixed with hard coded text to form a context relevant value for example CustomerId:[<MyCustomerId>]
sessionlog_name The name you manually type in in the Job tab for a given job session.
sessionlog_row The row number in the session log
sessionlog_subrow The row number associated with a For-Each session that runs on a separate thread
step_id The Id of the Step that performed the file operation. You can see any Step’s id in the properties window that you can open from the Options tab or the Properties toolbar button in the top right of the Step.
step_name The Name of the FileManagement Step that performed the file operation.
validation_errormessage The message describing why the source_value did not validate
validation timestamp The date and time when the validation took place.

Feedback

Was this helpful?

Yes No
You indicated this topic was not helpful to you ...
Could you please leave a comment telling us why? Thank you!
Thanks for your feedback.

Post your comment on this topic.

Post Comment