KROG
🇳🇴

CWA 18398 · assessment rubric

E.6 ICT Quality Management

← All competences

Level e-3

From role 2.6 AI Quality & Evaluation Specialist · EQF EQF7

Analyse and apply AI quality management practices by selecting and combining appropriate quality indicators, governance mechanisms, and evaluation methods to assess and improve the performance, reliability, security, and fairness of AI systems, while contributing to team-based quality decisions aligned with business needs.

  1. 7.1 Select appropriate AI quality metrics and indicators using recognised AI standards and system documentation.

    Assessed by Individual assignment evaluating AI quality metrics selection with written justification, assessed for appropriate selection of AI quality metrics and indicators using recognised AI standards and system documentation.

  2. 7.2 Combine governance frameworks and monitoring tools for different types of AI systems in operational contexts.

    Assessed by Case-based exercise analysing governance and monitoring configurations, assessed for combining governance frameworks and monitoring tools for different types of AI systems in operational contexts.

  3. 7.3 Analyse AI performance, reliability, fairness, and security data to identify quality risks and improvement opportunities.

    Assessed by Data analysis report on AI performance, fairness, reliability, and security, assessed for identification of quality risks and improvement opportunities.

  4. 7.4 Apply AI quality improvement measures in team-based development or operational AI projects.

    Assessed by Group project with documented implementation of AI quality improvement measures, assessed for application of improvement measures in team-based development or operational AI projects.

  5. 7.5 Document and communicate AI quality findings using structured reports or dashboards for stakeholders.

    Assessed by Written report or dashboard presenting AI quality findings, assessed for structured documentation, clarity of communication, and relevance for stakeholders.

  6. 7.6 Evaluate AI system outcomes against business, ethical, and regulatory requirements in organisational settings.

    Assessed by Structured evaluation paper or exam question assessing AI outcomes against business, ethical, and regulatory criteria in organisational settings.

From role 2.9 AI Reliability Engineer · EQF EQF6

Analyse and apply AI quality management practices by selecting and combining appropriate quality indicators, governance mechanisms, and evaluation methods to assess and improve the performance, reliability, security, and fairness of AI systems, while contributing to team-based quality decisions aligned with business needs.

  1. 5.1 Select appropriate AI quality metrics and indicators using recognised AI standards and system documentation.

    Assessed by Case-based exercise analysing governance and monitoring configurations

  2. 5.2 Combine governance frameworks and monitoring tools for different types of AI systems in operational contexts.

    Assessed by Data analysis report on AI performance, fairness, reliability, and security

  3. 5.3 Analyse AI performance, reliability, fairness, and security data to identify quality risks and improvement opportunities.

    Assessed by Group project with documented implementation of AI quality improvement measures

  4. 5.4 Apply AI quality improvement measures in team-based development or operational AI projects.

    Assessed by Written report or dashboard presenting AI quality findings

  5. 5.5 Document and communicate AI quality findings using structured reports or dashboards for stakeholders.

    Assessed by Structured evaluation paper or exam question assessing AI outcomes against business, ethical, and regulatory criteria DEVELOPMENT & OPERATIONS [2]

  6. 5.6 Evaluate AI system outcomes against business, ethical, and regulatory requirements in organisational settings.

From role 4.6 AI Operations Manager · EQF EQF6

Analyse and apply AI quality management practices by selecting and combining appropriate quality indicators, governance mechanisms, and evaluation methods to assess and improve the performance, reliability, security, and fairness of AI systems, while contributing to team-based quality decisions aligned with business needs.

  1. 8.1 Select appropriate AI quality metrics and indicators using recognised AI standards and system documentation.

    Assessed by Individual assignment evaluating AI quality metrics selection with written justification, assessed for use of recognised AI standards and system documentation.

  2. 8.2 Combine governance frameworks and monitoring tools for different types of AI systems in operational contexts.

    Assessed by Case-based exercise analysing governance and monitoring configurations, assessed for combining governance frameworks and monitoring tools for different types of AI systems in operational contexts.

  3. 8.3 Analyse AI performance, reliability, fairness, and security data to identify quality risks and improvement opportunities.

    Assessed by Data analysis report on AI performance, fairness, reliability, and security, assessed for identification of quality risks and improvement opportunities.

  4. 8.4 Apply AI quality improvement measures in team-based development or operational AI projects.

    Assessed by Group project with documented implementation of AI quality improvement measures, assessed for application in team-based development or operational AI projects.

  5. 8.5 Document and communicate AI quality findings using structured reports or dashboards for stakeholders.

    Assessed by Written report or dashboard presenting AI quality findings, assessed for structure, clarity, stakeholder relevance, and communication of findings.

  6. 8.6 Evaluate AI system outcomes against business, ethical, and regulatory requirements in organisational settings.

    Assessed by Structured evaluation paper or exam question assessing AI system outcomes against business, ethical, and regulatory requirements in organisational settings.

From role 5.2 AI Compliance Officer · EQF EQF7

Analyse and apply AI quality management practices by selecting and combining appropriate quality indicators, governance mechanisms, and evaluation methods to assess and improve the performance, reliability, security, and fairness of AI systems, while contributing to team-based quality decisions aligned with business needs.

  1. 7.1 Select appropriate AI quality metrics and indicators using recognised AI standards and system documentation.

    Assessed by Individual assignment evaluating AI quality metrics selection with written justification, assessed for use of recognised AI standards and system documentation.

  2. 7.2 Combine governance frameworks and monitoring tools for different types of AI systems in operational contexts.

    Assessed by Case-based exercise analysing governance and monitoring configurations, assessed for combining governance frameworks and monitoring tools for different types of AI systems in operational contexts.

  3. 7.3 Analyse AI performance, reliability, fairness, and security data to identify quality risks and improvement opportunities.

    Assessed by Data analysis report on AI performance, fairness, reliability, and security, assessed for identification of quality risks and improvement opportunities.

  4. 7.4 Apply AI quality improvement measures in team-based development or operational AI projects.

    Assessed by Group project with documented implementation of AI quality improvement measures, assessed for application in team-based development or operational AI projects.

  5. 7.5 Document and communicate AI quality findings using structured reports or dashboards for stakeholders.

    Assessed by Written report or dashboard presenting AI quality findings, assessed for structure, clarity, stakeholder relevance, and communication of findings.

  6. 7.6 Evaluate AI system outcomes against business, ethical, and regulatory requirements in organisational settings.

    Assessed by Structured evaluation paper or exam question assessing AI system outcomes against business, ethical, and regulatory requirements in organisational settings.

From role 5.3 AI Risk Manager · EQF EQF7

Analyse and apply AI quality management practices by selecting and combining appropriate quality indicators, governance mechanisms, and evaluation methods to assess and improve the performance, reliability, security, and fairness of AI systems, while contributing to team-based quality decisions aligned with business needs.

  1. 7.1 Select appropriate AI quality metrics and indicators using recognised AI standards and system documentation.

    Assessed by Individual assignment evaluating AI quality metrics selection with written justification, assessed for use of recognised AI standards and system documentation.

  2. 7.2 Combine governance frameworks and monitoring tools for different types of AI systems in operational contexts.

    Assessed by Case-based exercise analysing governance and monitoring configurations, assessed for combining governance frameworks and monitoring tools for different types of AI systems in operational contexts.

  3. 7.3 Analyse AI performance, reliability, fairness, and security data to identify quality risks and improvement opportunities.

    Assessed by Data analysis report on AI performance, fairness, reliability, and security, assessed for identification of quality risks and improvement opportunities.

  4. 7.4 Apply AI quality improvement measures in team-based development or operational AI projects.

    Assessed by Group project with documented implementation of AI quality improvement measures, assessed for application in team-based development or operational AI projects.

  5. 7.5 Document and communicate AI quality findings using structured reports or dashboards for stakeholders.

    Assessed by Written report or dashboard presenting AI quality findings, assessed for structure, clarity, stakeholder relevance, and communication of findings.

  6. 7.6 Evaluate AI system outcomes against business, ethical, and regulatory requirements in organisational settings.

    Assessed by Structured evaluation paper or exam question assessing AI system outcomes against business, ethical, and regulatory requirements in organisational settings.

Level e-4

From role 1.2 Data Curation Lead · EQF EQF7

Design, evaluate, and manage AI quality management frameworks by integrating governance mechanisms, quality metrics, ethical principles, and risk-based controls to ensure and improve the quality, trustworthiness, and compliance of AI systems across complex organisational contexts.

  1. 6.1 Design AI quality management frameworks by integrating governance mechanisms, metrics, and ethical principles.

    Assessed by Design assignment producing an integrated AI quality management framework, including data quality dimensions, governance structures, and lifecycle integration; assessed for completeness, coherence, scalability, and alignment with organisational and data curation requirements.

  2. 6.2 Adapt AI quality approaches to complex organisational contexts involving multiple AI systems and stakeholders.

    Assessed by Case study analysis justifying adaptation of AI quality approaches to complex organisational contexts; evaluated for contextual understanding, appropriateness of selected approaches, and strength of justification in relation to data quality and governance challenges.

  3. 6.3 Evaluate the effectiveness of AI quality controls using performance, risk, and compliance evidence.

    Assessed by Evaluation report assessing effectiveness of AI quality and risk controls; assessed for methodological rigour, use of appropriate metrics, and ability to demonstrate the impact of controls on data quality and system performance.

  4. 6.4 Assess AI-related risks and uncertainties in relation to trustworthiness, fairness, and system robustness.

    Assessed by Risk assessment document analysing trustworthiness and uncertainty in AI systems; evaluated for identification of data-related risks, handling of uncertainty, and alignment with governance, compliance, and quality assurance standards.

  5. 6.5 Manage the implementation of AI quality assurance activities within organisational or cross- functional projects.

    Assessed by Project-based assessment with an AI quality assurance implementation plan, including processes, controls, and monitoring mechanisms; assessed for feasibility, integration with organisational workflows, and effectiveness in ensuring continuous data and system quality.

  6. 6.6 Interpret AI quality outcomes to inform strategic decisions on AI deployment, scaling, or modification.

    Assessed by Strategic decision brief based on interpreted AI quality evidence; evaluated for clarity, relevance of evidence, justification of decisions, and alignment with organisational objectives and quality strategy.

  7. 6.7 Critically reflect on limitations of AI quality practices in light of evolving technological, ethical, or regulatory conditions.

    Assessed by Critical reflective essay on limitations of AI quality management practices; assessed for depth of reflection, identification of limitations and trade-offs, and ability to propose informed improvements for data curation and quality management processes. DATA PROCESSING & ANALYSIS [1]