Using AJV and GitHub Actions to Ensure Consistent Meta Information Across JSON Files

TL;DR#

This article is for

• Want to easily manage metadata with a Source of Truth.
• Want to automate the execution and validation of scripts on demand
• Want to know how to validate JSON data?

to validate JSON data.

This article will also help you

• You'll learn how to validate JSON files using AJV.
• You will learn how to automate the validation script using Github Actions.

Jump in#

First of all, what is metadata? Metadata is "data that serves some purpose". A classic example of metadata that we often use is package.json. It contains the name, version, description, author, dependency libraries, and other data that is essential to the project.

In my project, [Seed Design] (https://github.com/daangn/seed-design) repository, we have a docs folder that contains the documentation project. docs folder that contains the documentation project. This is the documentation project for the design system. Let's take the Avatar component as an example.

You create separate mdx documents to manage style information, usage information, overview information, etc. for Avatar. Let's say you're using a coverImage in each of these documents, then you'd have overview.mdx, style.mdx, and usage.mdx all looking like this.

1{/* We're calling coverImage directly */}
2![coverImage](./coverImage.png)

1# AS-IS
2📦avatar
3 ┣ 📜coverImage.png
4 ┣ 📜overview.mdx # coverImage in use
5 ┣ 📜style.mdx # coverImage in use
6 ┗ 📜usage.mdx # coverImage in use

But let's say the specification for coverImage changes and the extension changes from png to jpeg. We'd have to move three files around to fix the extension of coverImage. But if this happens a lot, we could end up spending all day just fixing data.

So we set aside a metadata file. Like this

1# TO-BE
2📦avatar
3 ┣ 📜coverImage.png
4 ┣ 📜component-meta.json # Metadata file
5 ┣ 📜overview.mdx
6 ┣ 📜style.mdx
7 ┗ 📜usage.mdx

And in that JSON file, it should look like this.

• Example of a metadata JSON file for an Avatar component

1{
2  "coverImage": "./coverImage.jpeg"
3}

Rather than directly importing and using coverImage from the three files overview.mdx, style.mdx, and usage.mdx. We want the three files to look at the metadata JSON file and display information based on that data. The idea is to put the metadata JSON file as the Source of Truth.

What does this do for you?

• It favors change.
• It makes it easier to communicate and more productive.
• It ensures that data is described consistently, making it easier to interpret and manage.
• Quality goes up because data becomes more accurate, complete, and relevant.

But as with everything, there are downsides.

• Because metadata is human-generated data, you need someone to manage it.
• You need to make sure it's always up to date.
• You need to have a good discussion with your team about what data to include. (Matching costs)

Most of the disadvantages are management costs. In this article, we'll work on keeping the management costs as low as possible.

What if the metadata JSON file contains the wrong values? A design system consists of dozens of components. It's impossible to check the metadata of all these components metadata for all of them manually is impossible.

**You can use a JSON Schema Validator (AJV) to validate the metadata when the metadata JSON file is modified. to validate the metadata and automate the logic behind that validation so that we can automatically check it before it becomes merged.

Create a custom JSON schema with AJV and write a validation script#

AJV is a JSON schema validator. It allows you to declaratively write down a data type for JSON and validate that a JSON file conforms to that data type.

Let's take a quick look at an example of using AJV.

1const Ajv = require("ajv");
2
3// Declare a JSON schema
4const schema = {
5  type: "object",
6  properties: {
7    name: { type: "string" },
8    age: { type: "number" },
9  },
10  required: ["name", "age"],
11};
12
13// Create a validator instance using the schema.
14const ajv = new Ajv();
15const validate = ajv.compile(schema);
16
17// Validate some JSON data against the schema.
18const data = {
19  name: "Alice",
20  age: 30,
21};
22const isValid = validate(data);
23
24if (isValid) {
25  console.log("Data is valid!");
26} else {
27  console.log("Data is invalid:");
28  console.log(validate.errors);
29}

That's it, here's what we need to do.

• Declare the JSON schema
• Create a script to validate it

Declaring the JSON schema#

I wrote a separate file to declare the schema. The file is a bit long, but you don't need to read it all. The important thing about the code below is that we can validate all the different patterns. **I can check for specific characters through regular expressions, and I can check if they are required or optional. You can also decide if only certain properties are allowed, or if additional properties are allowed.

See the official documentation

1const statusSchema = {
2  type: "string",
3  pattern: "^(todo|in-progress|done)$",
4};
5
6const stringSchema = {
7  type: "string",
8};
9
10const storybookSchema = {
11  type: "object",
12  properties: {
13    path: stringSchema,
14    height: { type: ["string"] },
15  },
16  required: ["path"],
17  additionalProperties: false,
18};
19
20const mdxSchema = {
21  type: "string",
22  pattern: "^.*.mdx$",
23};
24
25const jsonSchema = {
26  type: "string",
27  pattern: "^.*.json$",
28};
29
30const pngSchema = {
31  type: "string",
32  pattern: "^.*.png$",
33};
34
35const platformSchema = {
36  type: "object",
37  properties: {
38    ios: {
39      type: "object",
40      properties: {
41        status: statusSchema,
42        alias: { type: "string" },
43        path: { type: "string" },
44      },
45      required: ["status", "alias", "path"],
46      additionalProperties: false,
47    },
48    android: {
49      type: "object",
50      properties: {
51        status: statusSchema,
52        path: { type: "string" },
53      },
54      required: ["status", "path"],
55      additionalProperties: false,
56    },
57    react: {
58      type: "object",
59      properties: {
60        status: statusSchema,
61        path: { type: "string" },
62      },
63      required: ["status", "path"],
64      additionalProperties: false,
65    },
66    figma: {
67      type: "object",
68      properties: {
69        status: statusSchema,
70        path: { type: "string" },
71      },
72      required: ["status", "path"],
73      additionalProperties: false,
74    },
75    docs: {
76      type: "object",
77      properties: {
78        overview: {
79          type: "object",
80          properties: {
81            status: statusSchema,
82            storybook: storybookSchema,
83            mdx: mdxSchema,
84          },
85          additionalProperties: False,
86        },
87        usage: {
88          type: "object",
89          properties: {
90            status: statusSchema,
91            mdx: mdxSchema,
92          },
93          additionalProperties: False,
94        },
95        style: {
96          type: "object",
97          properties: {
98            status: statusSchema,
99            mdx: mdxSchema,
100            additionalProperties: False,
101          },
102        },
103      },
104      required: ["usage", "style", "overview"],
105      additionalProperties: true,
106    },
107  },
108  required: ["ios", "android", "react", "figma", "docs"],
109  additionalProperties: false,
110};
111
112export const componentMetaSchema = {
113  type: "object",
114  properties: {
115    name: stringSchema,
116    description: stringSchema,
117    thumbnail: pngSchema,
118    primitive: jsonSchema,
119    group: stringSchema,
120    platform: platformSchema,
121  },
122  required: ["name", "description", "thumbnail", "platform"],
123};
124
125export const primitiveMetaSchema = {
126  type: "object",
127  properties: {
128    name: stringSchema,
129    description: stringSchema,
130    thumbnail: pngSchema,
131    primitivie: mdxSchema,
132  },
133  required: ["name", "description", "thumbnail", "primitive"],
134};

Create a validation script#

I need a script that will validate through that Schema. My requirements were as follows

1📦content
2 ┣ 📂component
3 ┃ ┣ 📂action-sheet
4 ┃ ┃ ┣ 📜component-meta.json
5 ┃ ┣ 📂actionable-callout
6 ┃ ┃ ┣ 📜component-meta.json
7 ┃ ┣ 📂alert-dialog
8 ┃ ┃ ┣ 📜component-meta.json
9 ┃ ┃ ...
10 ┣ 📂primitive
11 ┃ ┣ 📂avatar
12 ┃ ┃ ┣ 📜primitive-meta.json
13 ┃ ┣ 📂button
14 ┃ ┃ ┣ 📜primitive-meta.json
15 ┃ ┣ 📂checkbox
16 ┃ ┃ ┣ 📜primitive-meta.json
17 ┃ ┣ ...

• Validate component schema by reading only the component-meta.json file inside all folders in the ./content/component folder at root
• Validate the primitive schema by reading only the primitive-meta.json file in all folders in the ./content/primitive folder in the root.
• If the validation fails, print out the name of the failed file and why it failed.

In code, this looks like this

1import Ajv from "ajv";
2import { prettify } from "awesome-ajv-errors";
3import fs from "node:fs/promises";
4import path from "node:path";
5
6import {
7  componentMetaSchema,
8  primitiveMetaSchema,
9} from "./meta-data-schemas.mjs";
10
11const ajv = new Ajv();
12
13// ---------Start the execution part--------- //
14
15console.log("Validating meta.json files...");
16
17validateJsonInDir({
18  dir: path.resolve("./content/component"),
19  validate: ajv.compile(componentMetaSchema),
20  type: "component",
21});
22validateJsonInDir({
23  dir: path.resolve("./content/primitive"),
24  validate: ajv.compile(primitiveMetaSchema),
25  type: "primitive",
26});
27
28console.log("Finished validating meta.json files");
29
30// ---------Start the implementation part--------- //
31
32async function validateJsonInDir({ dir, validate, type }) {
33  try {
34    // NOTE: Read the files and folders in the folder corresponding to dir.
35    const filesOrFolders = await fs.readdir(dir);
36
37    for (const fileOrFolder of filesOrFolders) {
38      const filePath = path.join(dir, fileOrFolder);
39
40      const stats = await fs.stat(filePath);
41
42      // NOTE: We're only looking for folders.
43      if (!stats.isDirectory()) {
44        continue;
45      }
46
47      } const subfiles = await fs.readdir(filePath);
48
49      // NOTE: Only look for json files in the folder for each component.
50      for (const subfile of subfiles) {
51        if (path.extname(subfile) !== ".json") {
52          continue;
53        }
54
55        } const data = await fs.readFile(path.join(filePath, subfile), "utf8");
56
57        const json = JSON.parse(data);
58        const isValid = validate(json);
59        const fileName = `${json.name.replaceAll(" ", "-").toLowerCase()}.json`;
60
61        if (!isValid) {
62          // NOTE: Executed when validation fails.
63          console.log(`${type}/${fileName} is invalid`);
64          console.error(prettify(validate, { data: json }));
65
66          process.exit(1);
67        } else {
68          // NOTE: You don't need to print this when it passes.
69          // console.log(`${type}/${fileName} is valid`);
70        }
71      }
72    }
73  } catch (err) {
74    console.error(err);
75  }
76}

Once you've written the script like this, write the commands to the script in package.json.

1{
2  "scripts": {
3    "validate:meta-data": "node scripts/validate-meta-data.mjs"
4  }
5}

Let's run it once, shall we?

Looks like it works!

But what if the data format isn't right? In my component's metadata JSON file, I made sure that the description was in the form of a string.

1export const componentMetaSchema = {
2  type: "object",
3  properties: {
4    name: stringSchema,
5    description: stringSchema,
6    thumbnail: pngSchema,
7    primitive: jsonSchema,
8    group: stringSchema,
9    platform: platformSchema,
10  },
11  required: ["name", "description", "thumbnail", "platform"],
12};

And let's take the description out of the action-sheet component of component for a moment. And then if we run the script again?

1{
2  "name": "Action Sheet",
3  "description": "A custom action sheet that replaces the OS system action sheet",
4  ...
5}

• Which component failed
• Which property is wrong

Looks like you're doing a good job!

See the awesome-ajv-errors library for pretty error messages.

Integrate AJV with GitHub Actions for automated validation#

If you've made it this far, you've probably gotten as far as creating your favorite validation scripts via AJV. You could manually run a script to check every metadata change, but people are too busy. It would be nice to have an Action that automatically runs a script when a PR is raised and the corresponding Pull Request changes the metadata.

The Github Action is the perfect tool to fulfill this need. Run only when there are changes in the folder you want, on the branch you want, only on the behavior you want (on push, on pull request, ...etc) and so on.

Let's go ahead and write an Action

1on:
2  # every time a push is made
3    branches: # check all branches
4      - '**'
5    paths: # action only fires when json files in the desired paths are changed
6      - 'docs/content/component/**/component-meta.json'
7      - 'docs/content/primitive/**/primitive-meta.json'
8
9name: Validate Seed Docs meta data files
10
11jobs:
12  build:
13    name: Validate meta data files
14    runs-on: ubuntu-latest
15
16    steps:
17    - uses: actions/checkout@v3
18
19    - uses: actions/setup-node@v3
20      with:
21        node-version: 18.12.1
22        cache: yarn
23
24    - name: Install Dependencies
25      run: yarn install --immutable
26
27    # Run a validation script
28    - name: Validate meta data files
29      working-directory: ./docs
30      run: |
31        yarn validate:meta-data
32
33    - name: Report success
34      run: echo "Script ran successfully!"

And let's create a PR and push it.

Looks like it's working!

But what does it look like if it fails?

In the script we wrote earlier, we had code to exit the process if it failed. Without that code, the Action will look like it succeeded even though the script failed, so it's important to put it in.

1if (!isValid) {
2  // NOTE: This is executed when validation fails.
3  console.log(`${type}/${fileName} is invalid`);
4  console.error(prettify(validate, { data: json }));
5  process.exit(1); // If it fails, we should end the process of the Github Action.
6} else {
7  // console.log(`${type}/${fileName} is valid`);
8}

Conclusion#

Using AJVs and GitHub Actions together can help ensure consistent meta information across your project's JSON files. By validating your data against your schema, you can catch errors early and avoid subsequent issues. Setting up automated validation with GitHub Actions can further streamline your workflow and reduce the chance of human error. We hope this post has helped you see how you can use these tools together to improve the quality and consistency of your data.