#Proof of Concept HSDS API Validator

This is a very quick-and-dirty proof of concept validator for validating HSDS APIs.

It is a CLI tool which requires python3. You may need to create a virtual environment and activate it to install the requirements, depending on your set up.

It only has one command at the moment: validate, which takes in a well-formed URL which represents an API stub. It then will check that this behaves properly i.e. that it returns the expected JSON object from the GET / path set out by the HSDS api specification, and then if so it proceeds to fetch the Open API file and starts validating the endpoints.

Validation is very simple. Basically three tests are performed:

1. What is the HTTP response code for the url (base URL + path-taken-from-openapi-definition)
2. If the response code is "OK", is the response empty JSON?
3. If the JSON is not empty, is it valid data for that endpoint according to the schema?

For API paths that take an id parameter e.g. GET /services/{id}, you can provide an --ids option to the validate command which takes in a JSON file supplying ids to that path, which you can use to test. If you do not provide this option the validator will ignore all paths that require an id to validate.

Right now it outputs a JSON report to STDOUT, meaning you can pipe it to various other programs or into a file. Errors are reported to STDERR so as not to interfere with piping.

Due to limitations on effort, only the first schema error is reported per URL. If there are no schema errors the value of schema_validation_errors will be null. It is possible to get a full list of schema errors but it is messy and the messages are not useful. For this you should use an instance of CoVE, which does the hard work of dissecting the schema errors and making them much more human readable.

#Example usage

ids.json -- file containing some sample ids. Empty arrays are fine but you must include a key per path in the openapi definition else there will be an error.

{
	"/services/{id}": ["ecbfe840-8563-11ef-89e5-8386d2f86d2e", "f60bceaa-8563-11ef-99d0-cfa605714d66", "fe64cd7c-8563-11ef-9360-b7a4929d9f55"],
	"/taxonomies/{id}": [],
	"/taxonomy_terms/{id}": [],
	"/organizations/{id}": [],
	"/service_at_locations/{id}": []
}

$ ./hsds-api-validator.py validate http://localhost:8000 --ids ids.json > result.json

Result:

result.json

{
  "/": {
    "status_code": 200
  },
  "/services/{id}": [
    {
      "url": "/services/ecbfe840-8563-11ef-89e5-8386d2f86d2e",
      "report": {
        "status_code": 404
      }
    },
    {
      "url": "/services/f60bceaa-8563-11ef-99d0-cfa605714d66",
      "report": {
        "status_code": 404
      }
    },
    {
      "url": "/services/fe64cd7c-8563-11ef-9360-b7a4929d9f55",
      "report": {
        "status_code": 404
      }
    }
  ],
  "/services": {
    "status_code": 200,
    "data_report": {
      "empty_reponse": false,
      "schema_validation_errors": "'name' is a required property"
    }
  },
  "/taxonomies/{id}": [],
  "/taxonomies": {
    "status_code": 404
  },
  "/taxonomy_terms/{id}": [],
  "/taxonomy_terms": {
    "status_code": 404
  },
  "/organizations/{id}": [],
  "/organizations": {
    "status_code": 404
  },
  "/service_at_locations/{id}": [],
  "/service_at_locations": {
    "status_code": 404
  }
}

I am not intending to develop this much further, as it is proof of concept only.