QP (Queue-Processor(s)) component acts like an internal microservice that modularizes functionalities so that it can be updated and maintained more easily
-
Consists of one Queue, which has a globally unique name and simply holds items. Items can be added to the queue by anything. Queue’s contents should be persisted to restore RRD state in case of system failure.
-
A Processor processes items from the Queue. They are stateless and preferably idempotent. A Processor can be implemented in practically any language (Java, Ruby, Python, etc.), as long as it can interface with a Message Queue (such as RabbitMQ or Amazon SQS). For scalability, a Processor can be replicated to process items from the Queue in parallel — shown as “instance 1” and “instance 2” in the diagram.
QP components will be connected together using well-tested and stable Enterprise Integration Patterns (EIP) tools (such as Apache Camel) so that we can focus on RRD functionality and less on “glue code”.
-
As the RDD prototype becomes more complex, using the same QP pattern for all RRD functionalities promotes low software coupling and, as a result, simplifies debugging and maintenance.
-
Using EIPs facilitate future migration of parts of the architecture to cloud-based services if needed (see Implementing EIP with AWS and with Azure), where for example a Processor could be implemented as AWS Lambda. This is mentioned to emphasize the flexibility of the architecture to evolve in case we decide to implement some subset of the QP components in those cloud services.
-
Using the Apache Camel implementation of EIPs provides time-saving features and integrations with many existing tools and data formats.
-
Example workflow using QP components:
-
The Router QP responds to RRD API requests and determines how the claim should be processed
- For each new claim, it quickly validates the claim and Veteran to send a response back to the API caller.
- Based on the claim characteristics and any other data it gathers, it determines how the claim should proceed and adds the claim to an appropriate Queue for processing.
-
Each hypertension/asthma/apnea QP components is expected to:
- receive an item from its Queue (either pushed or pulled)
- gathers health data by querying data sources (e.g., Lighthouse)
- decides on the claim (see details in the “RRD processing” section)
- adds a new item to the QA Queue
-
A Manual QP component can be included for discovery and research of new types of claims to fast track, or for scenarios where a claim has curious characteristics that require manual intervention.
-
The QA Processor performs quality assurance; to validate the RRD processing output and prep it for downstream processing (external to RRD). (Not shown in the diagram: another Manual QP component can be added for cases where the QA Processor finds an unsupported problem with the results.)
-
If requested as part of the original RRD API request, the Notifier Processor will execute any requested callbacks to indicate completion of RRD processing on the claim.