HITRUST AI Security Assessment and Certification Specification

Backing up AI system assets

HITRUST CSF requirement statement [?] (16.09lAISecOrganizational.1)

Backup copies of the following AI assets are created: 
(1) training, test, and validation datasets; 
(2) code used to create, train, and/or deploy AI models; 
(3) fine-tuning data; 
(4) models; 
(5) language model tools (e.g., plugins, agents); 
(6) AI system configurations (e.g., metaprompts); and 
(7) prompt enhancement data for RAG, as applicable. 
These backups are 
(8) managed in accordance with the organization’s policies addressing backups of data and
software.

Evaluative elements in this requirement statement [?]
1. Backup copies of AI training, test, and validation datasets are created.
2. Backup copies of code used to create, train, and/or deploy AI models is created.
3. Backup copies of AI fine-tuning data is created, if applicable.
4. Backup copies of AI models are created.
5. Backup copies of language model tools (e.g., plugins, agents) are created, if applicable.
6. Backup copies of AI system configurations (e.g., metaprompts) are created.
7. Backup copies of prompt enhancement data used for RAG are created, if applicable.
8. These backups are managed in accordance with the organization’s policies addressing
backups of data and software.


Illustrative procedures for use during assessments [?]

  • Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.

  • Procedure: Examine evidence that written or undocumented procedures exist as defined in the HITRUST scoring rubric. Determine if the procedures and address the operational aspects of how to perform each evaluative element within the requirement statement.

  • Implemented: Examine evidence that all evaluative elements within the requirement statement have been implemented as defined in the HITRUST scoring rubric, using a sample based test where possible for each evaluative element. Example test(s):
    • For example, review the AI system to ensure backup copies of AI assets are created and include training, test, and validation datasets, code used to create, train, and/or deploy AI models, tuning data, models, language model tools (e.g., plugins, agents), AI system configurations (e.g., metaprompts), prompt enhancement data for RAG. Further, confirm the backups are managed in accordance with the organization’s policies addressing backups of data and software.

  • Measured: Examine measurements that formally evaluate and communicate the operation and/or performance of each evaluative element within the requirement statement. Determine the percentage of evaluative elements addressed by the organization’s operational and/or independent measure(s) or metric(s) as defined in the HITRUST scoring rubric. Determine if the measurements include independent and/or operational measure(s) or metric(s) as defined in the HITRUST scoring rubric. Example test(s):
    • For example, measures indicate if backup copies of AI assets are created and include training, test, and validation datasets, code used to create, train, and/or deploy AI models, tuning data, models, language model tools (e.g., plugins, agents), AI system configurations (e.g., metaprompts), prompt enhancement data for RAG. Reviews, tests, or audits are completed by the organization to measure the effectiveness of the implemented controls and to confirm that backups are managed in accordance with the organization’s policies addressing backups of data and software.

  • Managed: Examine evidence that a written or undocumented risk treatment process exists, as defined in the HITRUST scoring rubric. Determine the frequency that the risk treatment process was applied to issues identified for each evaluative element within the requirement statement.

Placement of this requirement in the HITRUST CSF [?]

  • Assessment domain: 16 – Business Continuity & Disaster Recovery
  • Control category: 09.0 – Communications and Operations Management
  • Control reference: 09.l – Back-up

Specific to which parts of the overall AI system? [?]
  • AI application layer:
    • AI plugins and agents
    • Prompt enhancement via RAG, and associated RAG data sources
    • Application AI safety and security systems
    • The deployed AI application (Considered in the underlying HITRUST e1, i1, or r2 assessment)
    • The AI application’s supporting IT infrastructure (Considered in the underlying HITRUST e1, i1, or r2 assessment)
  • AI platform layer
    • The AI platform and associated APIs (Considered in the underlying HITRUST e1, i1, or r2 assessment)
    • Model safety and security systems
    • Model tuning and associated datasets
    • The deployed AI model
    • Model engineering environment and model pipeline
    • AI datasets and data pipelines
    • AI compute infrastructure (Considered in the underlying HITRUST e1, i1, or r2 assessment)

Discussed in which authoritative AI security sources? [?]
  • Securing Machine Learning Algorithms
    2021, © European Union Agency for Cybersecurity (ENISA)
    • Where:
      • 4.1- Security Controls > Organizational: Integrate ML applications into the overall cyber-resilience strategy

Control functions against which AI security threats? [?]
Additional information
  • Q: When will this requirement included in an assessment? [?]
    • This requirement will always be added to HITRUST assessments which include the
      Security for AI systems regulatory factor.

  • Q: Will this requirement be externally inheritable? [?] [?]
    • Yes, partially. This may be a responsibility shared between an AI application provider and their AI platform provider (if used), performed independently on separate layers/components of the overall AI system.