Schema builder is a javascript library for exporting a site configuration that can be consumed by the Web API spindle. This config is a fully declarative JSON schema-based format which not just describes the external API routes and types but also defines their valospace projections using embedded VPlot.

This library is primarily intended to be used from inside a spindle configuration library which is invoked from inside a revela.json gateway to emit the JSON configuration.

At the time of writing this document has triple responsibilities of being the authoritative format description both for schema-builder itself and for the JSON schema format that Web API spindle consumes, as well as being the testdoc for these formats.

Eventually the Web API spindle specification should be extracted to a separate document, and full test suites should be introduced.

This document is part of the spindle workspace @valos/web-spindle (of domain @valos/kernel) which has the description: `A spindle for structured ValOS Web APIs`.

§ Routes, types, and projections

The four schema builder concepts are:

§ Site configuration

Site configuration is the JSON output of this library. It can be directly assigned as the prefix configuration of the Web API spindle section of some gateway revela.json. This config contains sections for the other building blocks.

§ The testGlobalRules shared by the example testdocs

 ({
  scriptRoot: ["@$~gh.0123456789abcdef"],
  "&ofRelation": {
    tags: {
      routeRoot: ["@$~u4.aaaabbbb-cccc-dddd-eeee-ffffffffffff"],
    },
  },
  "&ofMethod": { POST: {
    "&ofRelation": { tags: {
      relationName: "TAG",
    }, },
  }, },
})

§ Example test site configuration


({
  api: {
    identity: { "!!!": "../../env/test/web-api-identity" },
    sessionDuration: 86400,
    swaggerPrefix: "/openapi"
  },
  serviceIndex: "valos://site.test.com/site?id=aaaabbbb-cccc",
  openapi: {
    openapi: "3.0.2",
    info: {
      name: "Test API", title: "Test API - Testem",
      description: "", version: "0.1.0",
    },
    externalDocs: {
      url: "https://swagger.io", description: "Find more info here",
    },
    servers: [], host: "127.0.0.1", schemes: ["http", "https"],
    consumes: ["application/json"], produces: ["application/json"],
    tags: [{ name: "user", description: "User end-points" }],
    securityDefinitions: { apiKey: {
      type: "apiKey", name: "apiKey", in: "header",
    } },
  },
  schemas: [
    sharedSchemaOf(TestTagType),
    sharedSchemaOf(TestIndividualType),
  ],
  routes: [
    sessionGETRoute(`/session`,
        { name: "session", rules: {
          clientRedirectPath: `/`,
          grantExpirationDelay: 300, tokenExpirationDelay: 86400 * 7,
        } }, testGlobalRules),
    sessionDELETERoute(`/session`,
        { name: "session", rules: {
          clientRedirectPath: `/`,
        } }, testGlobalRules),
    listingGETRoute(`/tags`, {}, testGlobalRules, TestTagType),
    resourceGETRoute(`/individuals/:resourceId`,
        { rules: {
          routeRoot: [],
          resource: ["@!ref", ["@!:request:params:resourceId"]],
        } }, testGlobalRules, TestIndividualType),
    mappingPOSTRoute(`/individuals/:resourceId/tags`, {
          enabledWithRules: ["relationName"],
          rules: {
            resource: ["@!ref", ["@!:request:params:resourceId"]],
            doCreateMappingAndTarget: ["@!new", [["@!:Relation"], {
              name: ["@!:relationName"],
              source: ["@!:resource"],
              target: ["@!new", ["@!:Entity"], [{
                name: ["@!:request:body", "$V", "target", "name"],
                owner: ["@!:routeRoot"],
                properties: { name: ["@!:request:body", "$V", "target", "name"] },
              }]],
            }]],
          },
        }, testGlobalRules, TestIndividualType, TestIndividualType.tags),
  ],
})

§ Type and property schemas

The main building block of schema-builder is object type schema. In JSON schema all object properties are listed under 'properties' field and all meta fields are outermost fields. Schema builder format for objects lists fields on the outside and properties inside the symbol field `[ObjectSchema]`. The schema expansion will then flip the type inside out to get the appropriate JSON schema layout.

§ expanded schema of simple object type

we expect
 exportSchemaOf({
  [ObjectSchema]: {
    description: "simple object type",
    valospace: {
      reflection: ["@.:forwardedFields"],
    },
  },
  name: StringType,
})
toEqual
 ({
  description: "simple object type",
  type: "object",
  valospace: {
    reflection: ["@.", ["forwardedFields"]],
  },
  properties: { name: { type: "string" } },
})

§ Extending schemas

The schemas can also be extended using extendType. The extension is a nested merge and can accept multiple base types.

Here we extend a string type with a valospace reflection path to the field VKernel:name.

§ expanded schema of an extended string

we expect
 extendType(StringType, { valospace: { reflection: ["@.$V.name@@"] } })
via
 exportSchemaOf(type)
toEqual
 ({
  type: "string",
  valospace: {
    reflection: ["@.", [["@$V", "name"]]],
  },
})

§ Shared resource type schemas

Valospace resources can be named in addition to providing them base types they extend. A resource that is given a valospace gate are primary resources which can be directly reached through routes via their projection path.

Schema builder provides a builtin object type `ResourceType` for valospace resources with following JSON schema:
{
  "$V": {
    "id": {
      "type": "string",
      "pattern": "^[a-zA-Z0-9\\-_.~]+$",
      "valospace": {
        "reflection": [
          "@.$V.rawId@@"
        ]
      }
    }
  }
}
This type contains the basic valospace selector under the key $V which contains the resource 'id' field.

§ expanded schema of a named resource

we expect
 namedResourceType("TestTag", [], {
  [ObjectSchema]: {
    description: "Test Tag resource",
    valospace: {
      gate: {
        name: "tags",
        projection: [["@-out:TAG"], ["@.$V.target"]],
      },
    },
  },
  name: extendType(StringType, { summary: "Tag name" }),
})
via
 exportSchemaOf(type)
toEqual
 ({
  schemaName: "TestTag",
  description: "Test Tag resource",
  type: "object",
  valospace: {
    gate: {
      name: "tags",
      projection: ["@@", [["@-out", ["TAG"]], ["@.", [["@$V", "target"]]]]],
    },
  },
  properties: {
    $V: { type: "object",
      valospace: {},
      properties: {
        id: { type: "string",
          pattern: "^[a-zA-Z0-9\\-_.~]+$",
          valospace: { reflection: ["@.", [["@$V", "rawId"]]] }
        },
      },
    },
    name: { summary: "Tag name", type: "string" },
  },
})

§ Automatic substitution of shared type references

The resource types are shared and can be referred to by their name with a '#'-suffix in the JSON schema. Schema builder does this automatically during schema generation.

§ expanded schema of a named type reference

we expect
 ({ tag: TestTagType })
via
 exportSchemaOf(type)
toEqual
{
  "tag": {
    "$ref": "#TestTag"
  }
}

§ Mapping schemas

A mapping is group of relations originating from a resource with a common name. The mapping relations can have properties and can be referred from the Web API also individually: their identity (ie. 'primary key') of is the unique combination of the mapping source resource and mapping name plus the individualtarget resource.

The mappings in valospace are defined by a reflection to a set of relations. Here mappingToMany defines a mapping 'tags' into outgoing TAGS relations with a mapping property 'highlight' and where the target resource is a Tag type defined earlier.

§ expanded schema of a mapping property

we expect
 mappingToManyOf("tags", TestTagType,
    ["@-out:TAG"],
    { highlight: BooleanType })
via
 exportSchemaOf(type)
toEqual
 ({
  type: "array",
  valospace: {
    mappingName: "tags",
    reflection: ["@-out", ["TAG"]],
  },
  items: { type: "object",
    properties: {
    highlight: { type: "boolean" },
      $V: { type: "object", properties: {
        href: { type: "string" }, rel: { type: "string" },
        target: { type: "object", valospace: { resourceType: "TestTag" }, properties: {
          $V: { type: "object", properties: {
            id: { type: "string",
              pattern: "^[a-zA-Z0-9\\-_.~]+$",
              valospace: { reflection: ["@.", [["@$V", "rawId"]]] },
            },
          } },
        } },
      } },
    },
  },
})

§ Putting a complex resource type together

A complex example which puts all together.

§ expanded schema of a complex resource type

we expect
 namedResourceType(
    "TestIndividual", exports.TestProfileType, {
  [ObjectSchema]: {
    description: "Test Individual resource",
    valospace: {
      gate: {
        name: "individuals",
        projection: [["@-out:INDIVIDUAL"], ["@.$V.target"]],
        filterCondition: [["@$valk.nullable"], ["@.:visible"]],
      },
    },
  },
  title: StringType,
  company: StringType,
  interests: () => mappingToManyOf("interests", exports.TestTagType,
      [["@-out:INTEREST"], ["@$valk.nullable"]],
      { [ObjectSchema]: { valospace: { filterable: true } } }),
})
via
 exportSchemaOf(type)
toEqual
 ({
  schemaName: "TestIndividual",
  type: "object",
  description: "Test Individual resource",
  valospace: {
    gate: {
      name: "individuals",
      projection: ["@@", [["@-out", ["INDIVIDUAL"]], ["@.", [["@$V", "target"]]]]],
      filterCondition: [["@$valk.nullable"], ["@.:visible"]],
    },
  },
  properties: {
    $V: { type: "object",
      valospace: {},
      properties: {
        id: { type: "string",
          pattern: "^[a-zA-Z0-9\\-_.~]+$",
          valospace: { reflection: ["@.", [["@$V", "rawId"]]], },
        },
      },
    },
    company: { type: "string" },
    contact: { type: "object", properties: {
      email: { type: "string" },
      phone: { type: "string" },
      website: { type: "string" }
    }, },
    description: { type: "string" },
    icon: { type: "string" },
    image: { type: "string",
      valospace: { reflection: ["@.", [["@$V", "name"]]] },
    },
    interests: { type: "array",
      valospace: {
        mappingName: "interests",
        reflection: ["@@", [["@-out", ["INTEREST"]], ["@$valk", "nullable"]]],
      },
      items: {
        type: "object",
        valospace: { filterable: true },
        properties: {
          $V: { type: "object", properties: {
            href: { type: "string" }, rel: { type: "string" },
            target: { type: "object", valospace: { resourceType: "TestTag" }, properties: {
              $V: { type: "object", properties: {
                id: { type: "string",
                  pattern: "^[a-zA-Z0-9\\-_.~]+$",
                  valospace: { reflection: ["@.", [["@$V", "rawId"]]], },
                },
              } },
            } },
          } },
        },
      },
    },
    name: { type: "string" },
    owned: { type: "object",
      valospace: { reflection: ["@@"] },
      properties: {
        services: { type: "array",
          valospace: {
            mappingName: "owned/services",
            reflection: ["@@", [["@-out", ["SERVICE"]], ["@$valk", "nullable"]]],
          },
          items: {
            properties: {
              highlight: { type: "boolean" },
              $V: { type: "object", properties: {
                href: { type: "string" }, rel: { type: "string" },
                target: { type: "object", valospace: { resourceType: "TestService" }, properties: {
                  $V: { type: "object", properties: {
                    id: { type: "string",
                      pattern: "^[a-zA-Z0-9\\-_.~]+$",
                      valospace: { reflection: ["@.", [["@$V", "rawId"]]], },
                    },
                  } },
                } },
              } },
            },
            type: "object",
          },
        },
      },
    },
    tags: { type: "array",
      valospace: {
        mappingName: "tags",
        reflection: ["@@", [["@-out", ["TAG"]], ["@$valk", "nullable"]]],
      },
      items: { type: "object",
        valospace: { filterable: true },
        properties: {
          highlight: { type: "boolean" },
          $V: { type: "object", properties: {
            href: { type: "string" }, rel: { type: "string" },
            target: { type: "object", valospace: { resourceType: "TestTag" }, properties: {
              $V: { type: "object", properties: {
                id: { type: "string",
                  pattern: "^[a-zA-Z0-9\\-_.~]+$",
                  valospace: { reflection: ["@.", [["@$V", "rawId"]]], },
                },
              } },
            } },
          } },
        },
      },
    },
    title: { type: "string" },
    visible: { type: "boolean" },
  },
})

§ Route definitions

Routes are exported as JSON object that is subsequently provided as a fastify route options object.

§ Route testdoc examples

Route testdoc examples share the following data:

§ Data common to all route testdoc examples

 ({
    TestTagType,
    TestIndividualType,
    gate: TestIndividualType[ObjectSchema].valospace.gate,
    mappingName: "tags",
    testThingTagsMapping: TestIndividualType.tags,
  })

Of note is the `globalRules` section, which is a JSON construct that is sourced from configuration files.

§ Basic GET resource route

Simple resource-GET route retrieves a primary TestIndividualType resource based on an id string given as a route parameter.

The route defines the reflection rule `resource` which converts the id string into a valospace resource id. The resource-GET handler (a built-in component of the Web API spindle) then uses this id to pick the correct resource from the set of resources located by the TestIndividualType gate projection.

§ route of a simple resource GET

we expect
 resourceGETRoute(`/${gate.name}/:resourceId`, {
  rules: {
    routeRoot: null,
    resource: ["@!ref", ["@!:request:params:resourceId"]],
  },
}, {}, TestIndividualType)
toEqual
 ({
  name: "individuals", method: "GET", projector: "resource",
  url: "/individuals/:resourceId",
  schema: {
    description: "Get the contents of a TestIndividual route resource",
    querystring: {
      fields: { type: "string",
        pattern: "^([a-zA-Z0-9\\-_.~/*$]*(\\,([a-zA-Z0-9\\-_.~/*$])*)*)?$"
      },
    },
    response: {
      200: { $ref: "#TestIndividual" },
      403: { type: "string" },
      404: { type: "string" },
    }
  },
  config: {
    requiredRules: ["routeRoot"],
    valueAssertedRules: ["resource"],
    runtimeRules: [],
    resource: {
      name: "TestIndividual",
      schema: { $ref: "#TestIndividual" },
      gate: {
        name: "individuals",
        projection: ["@@", [["@-out", ["INDIVIDUAL"]], ["@.", [["@$V", "target"]]]]],
        filterCondition: [["@$valk.nullable"], ["@.:visible"]],
      },
    },
    rules: {
      resource: ["@!ref", [
        ["@!", ["request", "params", "resourceId"]],
      ]],
      routeRoot: null,
    },
  },
})

§ Complex POST mapping route

Complex mapping-POST route which adds a new tags mapping to a primary thing.

§ route of a complex POST mapping

we expect
 mappingPOSTRoute(`/${gate.name}/:resourceId/${mappingName}`, {
  enabledWithRules: ["relationName"],
  rules: {
    resource: ["@!ref", ["@!:request:params:resourceId"]],
    doCreateMappingAndTarget: ["@!new", ["@!:Relation"], [{
      name: ["@!:relationName"],
      source: ["@!:resource"],
      target: ["@!new", ["@!:Entity"], [{
        name: ["@!:request:body", "$V", "target", "name"],
        owner: ["@!:routeRoot"],
        properties: { name: ["@!:request:body", "$V", "target", "name"] },
      }]],
    }]],
  },
}, testGlobalRules, TestIndividualType, testThingTagsMapping)
toEqual
 ({
  name: "individuals", method: "POST", projector: "mapping",
  url: "/individuals/:resourceId/tags",
  schema: {
    description:
`Create a new TestTag resource
*using **body.$V.target** as content* and then a new 'tags'
mapping to it from the source TestIndividual route
resource. The remaining fields of the body are set as the mapping
content. Similarily the response will contain the newly created target
resource content in *response.$V.target* with the rest of the response
containing the mapping.`,
    body: { type: "object",
      valospace: { filterable: true },
      properties: {
        highlight: { type: "boolean" },
        $V: { type: "object", properties: { target: { $ref: "#TestTag" } } },
      },
    },
    response: {
      200: { type: "object",
        valospace: { filterable: true },
        properties: {
          highlight: { type: "boolean" },
          $V: { type: "object", properties: {
            href: { type: "string" }, rel: { type: "string" }, target: { $ref: "#TestTag" },
          } },
        },
      },
      403: { type: "string" },
      404: { type: "string" },
    }
  },
  config: {
    resource: {
      name: "TestIndividual",
      schema: { $ref: "#TestIndividual" },
      gate: {
        name: "individuals",
        projection: ["@@", [["@-out", ["INDIVIDUAL"]], ["@.", [["@$V", "target"]]]]],
        filterCondition: [["@$valk.nullable"], ["@.:visible"]],
      },
    },
    relation: {
      name: "tags",
      schema: { type: "array",
        valospace: {
          mappingName: "tags",
          reflection: ["@@", [["@-out", ["TAG"]], ["@$valk", "nullable"]]],
        },
        items: { type: "object",
          valospace: { filterable: true },
          properties: {
            highlight: { type: "boolean" },
            $V: { type: "object", properties: {
              href: { type: "string" }, rel: { type: "string" },
              target: { type: "object", valospace: { resourceType: "TestTag" }, properties: {
                $V: { type: "object", properties: {
                  id: { type: "string",
                    pattern: "^[a-zA-Z0-9\\-_.~]+$",
                    valospace: { reflection: ["@.", [["@$V", "rawId"]]], },
                  },
                } },
              } },
            } },
          },
        },
      },
    },
    target: { name: "TestTag", schema: { $ref: "#TestTag" } },
    enabledWithRules: ["relationName"],
    requiredRules: ["routeRoot", "mappingName"],
    valueAssertedRules: ["resource"],
    runtimeRules: ["doCreateMappingAndTarget"],
    rules: {
      doCreateMappingAndTarget: ["@!new", [
        ["@!", ["Relation"]],
        ["@*", [
          ["@.", ["name", ["@!", ["relationName"]]]],
          ["@.", ["source", ["@!", ["resource"]]]],
          ["@.", ["target", ["@!new", [
            ["@!", ["Entity"]],
            ["@*", [
              ["@.", ["name", ["@!", ["request", "body", "$V", "target", "name"]]]],
              ["@.", ["owner", ["@!", ["routeRoot"]]]],
              ["@.", ["properties", ["@*", [
                ["@.", ["name", ["@!", ["request", "body", "$V", "target", "name"]]]],
              ]]]],
            ]],
          ]]]],
        ]],
      ]],
      mappingName: "tags",
      relationName: "TAG",
      resource: ["@!ref", [["@!", ["request", "params", "resourceId"]]]],
      routeRoot: ["@$~u4", "aaaabbbb-cccc-dddd-eeee-ffffffffffff"],
      scriptRoot: ["@$~gh", "0123456789abcdef"],
    },
  },
})

§ Projection and reflection VPlots

Projections and reflections are VPlots which are present primary type `valospace.gate.projection` fields and in type and property `valospace.reflection` fields.

§ Projections examples

Projections testdoc examples share the following data:

§ Data common to all projections examples

 ({ shared: "shared example data example" })

§ expanded schema of an extended resource type

we expect
 namedResourceType("TestNewsItem", exports.TestThingType, {
  [ObjectSchema]: {
    description: "Test News Item resource",
    valospace: {
      gate: {
        name: "news",
        projection: [["@-out:NEWSITEM"], ["@.$V.target"]],
        filterCondition: [["@$valk.nullable"], ["@.:visible"]],
      },
    },
  },
  startTime: exports.TestDateTimeType,
  endTime: exports.TestDateTimeType,
})
via
 exportSchemaOf(type)
toEqual
 ({
    schemaName: "TestNewsItem",
    type: "object",
    description: "Test News Item resource",
    valospace: {
      gate: {
        name: "news",
        projection: ["@@", [["@-out", ["NEWSITEM"]], ["@.", [["@$V", "target"]]]]],
        filterCondition: [["@$valk.nullable"], ["@.:visible"]],
      },
    },
    properties: {
      $V: { type: "object",
        valospace: {},
        properties: {
          id: { type: "string",
            pattern: "^[a-zA-Z0-9\\-_.~]+$",
            valospace: { reflection: ["@.", [["@$V", "rawId"]]], },
          },
        },
      },
      contact: { type: "object", properties: {
        email: { type: "string" },
        phone: { type: "string" },
        website: { type: "string" }
      }, },
      description: { type: "string" },
      icon: { type: "string" },
      image: { type: "string",
        valospace: { reflection: ["@.", [["@$V", "name"]]] },
      },
      name: { type: "string" },
      tags: { type: "array",
        valospace: {
          mappingName: "tags",
          reflection: ["@@", [["@-out", ["TAG"]], ["@$valk", "nullable"]]],
        },
        items: { type: "object",
          valospace: { filterable: true },
          properties: {
            highlight: { type: "boolean" },
            $V: { type: "object", properties: {
              href: { type: "string" }, rel: { type: "string" },
              target: { type: "object", valospace: { resourceType: "TestTag" }, properties: {
                $V: { type: "object", properties: {
                  id: { type: "string",
                    pattern: "^[a-zA-Z0-9\\-_.~]+$",
                    valospace: { reflection: ["@.", [["@$V", "rawId"]]], },
                  },
                } },
              } },
            } },
          },
        },
      },
      visible: { type: "boolean" },
      startTime: { type: "object",
        valospace: {},
        properties: {
          unixSeconds: { type: "number" },
          zone: { type: "string" }
        },
      },
      endTime: { type: "object",
        valospace: {},
        properties: {
          unixSeconds: { type: "number" },
          zone: { type: "string" }
        },
      },
    },
  })