Article

The JSON Schema tester you really need

Test a folder of JSON data against a selected schema, retain the result, and use coverage evidence to improve the data contract.

Explore JSON Schema testing Read validation Help

JSONBuddy for Windows Test collections view listing JSON Schema test entries — The established Test collections view lists repeatable JSON Schema test entries and their data folders.

Test a data collection against its schema

A JSON Schema tester checks how a set of JSON data files validates against a selected schema resource. It is useful during schema design and when repeatable data checks need more context than a one-off online validator. JSONBuddy’s established workflow can use a schema file or a schema pool and test data accessible from one local folder.

Create a repeatable test entry

Open the Testing pane and select Create new.
Name the entry so the schema and data set are clear to the next reviewer.
Select the JSON Schema resource or the schema pool that provides it.
Select the local folder containing the JSON test data, add the test, then run it from Test collections.

After a run, inspect the detailed validation errors, warnings, and processing information for each file. The production workflow records a separate log for each run; confirm log location, retention, and current command labels with the release candidate.

Compare results and inspect coverage

The retained tester records a reference result that can be compared with later runs. It also exposes schema definition coverage, showing which parts of the selected schemas were exercised by the test data. Treat a coverage result as a review aid: confirm that the intended positive and negative cases are represented before relying on it as evidence of completeness.

Verify the test configuration

Open at least one failed and one expected-valid result to confirm the intended schema resolution context.
Keep the input folder and the selected schema version identifiable for a repeatable run.
Review changed results against the reference, then investigate the relevant data or schema change.
Re-test support for specific drafts, schema pools, logs, report output, and coverage behavior against the shipping Windows release.

Continue with the closest workflow

Use the JSON Schema tester for the capability overview, the JSON Schema editor to change the contract, or validation Help for a focused correction sequence.

Keep the useful test evidence together

The legacy tester describes a collection-level summary report with the overall status, error and message counts, and coverage. Keep that report with the schema revision and test-data revision it represents. A changed summary is a signal to inspect the per-file evidence; it does not establish whether a changed diagnostic is a regression, a deliberate schema update, or a modified fixture.

The original article also names JSON Schema draft 4 through draft 2020-12, multi-file testing, reference comparison, coverage collection, and integration with the JSONBuddy editor. These are useful discovery terms and workflow evidence, but the exact draft set, report fields, result-comparison semantics, folder constraints, and edition access must be confirmed with the release candidate rather than inferred from an older article.

Use a complete set of expected cases

Include ordinary valid documents, invalid documents that each target one important constraint, and edge cases for optional fields, arrays, reused definitions, and resolved references. When a case is intentionally invalid, record the expected failure so a future reviewer can distinguish a desired diagnostic from a newly introduced one.

Review the results as a change record

A repeatable schema test becomes more useful when its result can be compared meaningfully over time. Record the test-entry name, schema resource or pool, input-folder revision, run date, product build, and the accepted reference result. Keep a note of any intentionally changed result before updating the reference, so the history tells a reviewer why the expected outcome moved.

Use detailed findings to locate the data instance and the governing schema definition. If a result changes unexpectedly, first confirm the selected schema and reference resolution context, then compare the changed fixture with the previous input. A coverage change can reveal that a new example reached a branch, but it does not remove the need to review the asserted result.

For a release gate, preserve the evidence needed to reproduce the collection and run it again after material changes. Do not promise unattended CI behavior, report formats, exit codes, or performance characteristics until the current Windows package has been verified.