By parsing the NIST definitions in this way, we subsequently posit three principal dimensions of ‘rely-able’ assurance.
Suitability
Suitability is intended to address the ‘security features, practices, procedures, and architecture’—i.e., the information security protections or ‘controls’ for an appropriate scope of assessment—that are the subject of the intended assurances. We can further stipulate that the controls must be reasonable and appropriate for the organization and must provide for the adequate protection of sensitive information within the context of assessment, e.g., the controls must manage risk to a level deemed acceptable by the organization.
Think about this in the context of an academic course of study such as basic algebra. One would rightfully assume that such a course would cover, at a minimum, basic concepts, the number system, monomials and polynomials, special products and factoring, linear and fractional equations, and functions. In the same way, an organization’s controls should comprehensively address reasonably anticipated threats to its sensitive information.
Unfortunately, the type of risk analysis needed to specify a custom set of controls can be difficult to do properly, which often results in some threats not being adequately addressed by an organization that takes such an approach.89 This is why control frameworks like ISO/IEC 27001/2, NIST SP 800-53, and the HITRUST CSF are so widely used; however, it should be noted the degree of coverage and the prescription and granularity of the controls can vary from one framework to the next.
Rigor
Rigor provides the “grounds for confidence that the set of intended security controls in an information system are effective in their application,” which is generally based on the accuracy and precision supported by the assurance approach.
In our academic example, a student’s understanding of the material covered by an algebra course would typically be assessed through one or more test instruments, e.g., a series of topical area tests or a single comprehensive final exam. Generally accepted best practices for test item construction and a standardized approach to scoring test items, e.g., via a rubric, help ensure accuracy and consistency of the results, the degree by which helps determine the level of confidence provided by the test results. Similar benefits for control assessment and reporting can be obtained through the use of standardized assessment guidance and scoring models; however, the level of accuracy and precision obtained will vary based on the level of detail in the guidance (e.g., general, or control-specific) and the robustness of the scoring model (e.g., a simple binary approach or a multi-level, maturity-based approach to evaluating a control’s implementation).
Impartiality
Impartiality is intended to address the ‘measure’ or ‘grounds for confidence’ needed by a relying party in an assessment—whether the assessment/audit is ‘worthy of belief or consideration for evidentiary purposes’—via the amount or level of independence between the assessor/auditor and the entity being assessed or audited. The level of impartiality can also be supported by an objective quality assurance review and automated quality checks that address consistency of responses and supporting evidence.
Issues with self and independent (i.e., instructor) assessment in an academic environment are well understood. Students that self-assess almost always tend to overestimate their academic performance when compared with instructor evaluations.90 However, when self-assessments are preceded with appropriate guidance and facilitated by an instructor, the strengths of self-assessments can be enhanced, and their weaknesses subsequently reduced.91
HITRUST has observed similar behavior with control self-assessments when compared with those conducted by trained independent assessors, which is why HITRUST recommends limiting the use of self-assessments as an interim step in providing more robust assurances, either as a rapid, low assurance assessment of an organization’s information protection and compliance program or as a way to prepare for a higher assurance third-party assessment.92
89 For more information on control framework-based risk analysis and control specification, see Cline, B. (2017, Sep).
90 Ross, J. (2006, Nov). The reliability, validity, and utility of self-assessment. Practical Assessment, Research & Evaluation, 11(10), p.3.
91 Ibid.
92 Cline, B. (2019, 11 Nov). Understanding and Improving the Role of Self-Assessment in Third-Party Risk Management [Blog Post].