KROG
🇳🇴

CWA 18398 · assessment rubric

C.4 Problem Management

← All competences

Level e-2

From role 3.2 AI Service Support Specialist · EQF EQF5

Identify and analyse common ICT and AI incidents, including system failures, performance issues, or AI bias, and apply established procedures to resolve or escalate them, using standard monitoring tools and knowledge systems in structured, well-defined situations.

  1. 4.1 Identify common ICT and AI incidents, such as system failures, performance degradation, or obvious AI bias, using monitoring dashboards and log analysis tools.

    Assessed by Review of AI incident log entries and instructor-verified checklist of detected incidents, assessed for correct identification of system failures, performance degradation, or obvious AI bias using monitoring dashboards and log analysis tools.

  2. 4.2 Analyse the immediate causes of these incidents within structured troubleshooting procedures.

    Assessed by Evaluation of root cause report using a rubric-based analysis of troubleshooting steps and evidence for immediate causes within structured troubleshooting procedures.

  3. 4.3 Apply standard corrective or preventive actions to resolve incidents following predefined operational guidelines.

    Assessed by Assessment of corrective action report, compared with predefined guidelines for appropriateness and completeness of standard corrective or preventive actions.

  4. 4.4 Document incidents and solutions in knowledge management systems for future reference and team communication.

    Assessed by Review of knowledge system entries using a completeness and clarity check for documented incidents, solutions, future reference value, and team communication.

  5. 4.5 Escalate incidents appropriately according to established organisational protocols and reporting chains.

    Assessed by Verification of escalation documentation through audit against organizational protocol, including escalation trigger, reporting chain, and evidence of appropriate incident routing. SUPPORT & GUIDANCE [3]

Level e-3

From role 2.10 AI Observability & Monitoring Specialist · EQF EQF6

Investigate and evaluate ICT and AI incidents, including emerging failures, model drift, or bias, apply corrective and preventive measures, and design monitoring strategies or knowledge management practices to optimise system and AI performance, in situations with varying and partially undefined problems.

  1. 3.1 Investigate complex ICT and AI incidents, including subtle system failures, AI model drift, and emerging bias patterns, using diagnostic tools and historical performance data.

    Assessed by Assessment of AI incident investigation report, including depth and accuracy of data analysis, evaluated for investigation of complex ICT and AI incidents using diagnostic tools and historical performance data.

  2. 3.2 Evaluate root causes of incidents and their impact on system and AI performance through structured analysis and impact assessment methods.

    Assessed by Evaluation of root cause evaluation and impact analysis review, assessed for use of structured analysis and impact assessment methods to determine effects on system and AI performance.

  3. 3.3 Apply corrective and preventive measures to mitigate incidents in less predictable operational situations.

    Assessed by Review of mitigation plan, assessed for effectiveness and applicability of corrective and preventive measures in less predictable operational situations.

  4. 3.4 Design monitoring strategies or knowledge management practices to track recurring AI failures and support system optimisation.

    Assessed by Assessment of monitoring/knowledge strategy document, assessed for clarity, feasibility, and ability to track recurring AI failures and support system optimization.

  5. 3.5 Advise or collaborate with team members by sharing findings and recommending improvements in AI system performance and ethical compliance.

    Assessed by Review of advisory report, assessed for quality of recommendations, evidence of shared findings, and relevance to AI system performance and ethical compliance.

From role 2.3 AI Engineer · EQF EQF6

Investigate and evaluate ICT and AI incidents, including emerging failures, model drift, or bias, apply corrective and preventive measures, and design monitoring strategies or knowledge management practices to optimise system and AI performance, in situations with varying and partially undefined problems.

  1. 8.1 Investigate complex ICT and AI incidents, including subtle system failures, AI model drift, and emerging bias patterns, using diagnostic tools and historical performance data.

    Assessed by Assessment of AI incident investigation report, including depth and accuracy of data analysis, evaluated for investigation of complex ICT and AI incidents using diagnostic tools and historical performance data.

  2. 8.2 Evaluate root causes of incidents and their impact on system and AI performance through structured analysis and impact assessment methods.

    Assessed by Evaluation of root cause evaluation and impact analysis review, assessed for use of structured analysis and impact assessment methods to determine effects on system and AI performance.

  3. 8.3 Apply corrective and preventive measures to mitigate incidents in less predictable operational situations.

    Assessed by Review of mitigation plan, assessed for effectiveness and applicability of corrective and preventive measures in less predictable operational situations.

  4. 8.4 Design monitoring strategies or knowledge management practices to track recurring AI failures and support system optimisation.

    Assessed by Assessment of monitoring/knowledge strategy document, assessed for clarity, feasibility, and ability to track recurring AI failures and support system optimization.

  5. 8.5 Advise or collaborate with team members by sharing findings and recommending improvements in AI system performance and ethical compliance.

    Assessed by Review of advisory report, assessed for quality of recommendations, evidence of shared findings, and relevance to AI system performance and ethical compliance.

From role 3.1 AI Security Specialist · EQF EQF7

Investigate and evaluate ICT and AI incidents, including emerging failures, model drift, or bias, apply corrective and preventive measures, and design monitoring strategies or knowledge management practices to optimise system and AI performance, in situations with varying and partially undefined problems.

  1. 3.1 Investigate complex ICT and AI incidents, including subtle system failures, AI model drift, and emerging bias patterns, using diagnostic tools and historical performance data.

    Assessed by Assessment of AI incident investigation report, including depth and accuracy of data analysis, evaluated for investigation of complex ICT and AI incidents using diagnostic tools and historical performance data.

  2. 3.2 Evaluate root causes of incidents and their impact on system and AI performance through structured analysis and impact assessment methods.

    Assessed by Evaluation of root cause evaluation and impact analysis review, assessed for use of structured analysis and impact assessment methods to determine effects on system and AI performance.

  3. 3.3 Apply corrective and preventive measures to mitigate incidents in less predictable operational situations.

    Assessed by Review of mitigation plan, assessed for effectiveness and applicability of corrective and preventive measures in less predictable operational situations.

  4. 3.4 Design monitoring strategies or knowledge management practices to track recurring AI failures and support system optimisation.

    Assessed by Assessment of monitoring/knowledge strategy document, assessed for clarity, feasibility, and ability to track recurring AI failures and support system optimization.

  5. 3.5 Advise or collaborate with team members by sharing findings and recommending improvements in AI system performance and ethical compliance.

    Assessed by Review of advisory report, assessed for quality of recommendations, evidence of shared findings, and relevance to AI system performance and ethical compliance.

From role 4.6 AI Operations Manager · EQF EQF6

Investigate and evaluate ICT and AI incidents, including emerging failures, model drift, or bias, apply corrective and preventive measures, and design monitoring strategies or knowledge management practices to optimise system and AI performance, in situations with varying and partially undefined problems.

  1. 3.1 Investigate complex ICT and AI incidents, including subtle system failures, AI model drift, and emerging bias patterns, using diagnostic tools and historical performance data.

    Assessed by Assessment of AI incident investigation report for depth and accuracy of data analysis, including use of diagnostic tools and historical performance data to investigate system failures, model drift, and bias patterns.

  2. 3.2 Evaluate root causes of incidents and their impact on system and AI performance through structured analysis and impact assessment methods.

    Assessed by Evaluation of root cause evaluation and impact analysis review, assessed for structured analysis of incident causes and effects on system and AI performance.

  3. 3.3 Apply corrective and preventive measures to mitigate incidents in less predictable operational situations.

    Assessed by Review of mitigation plan, assessed for effectiveness and appropriateness of corrective and preventive measures in less predictable operational situations.

  4. 3.4 Design monitoring strategies or knowledge management practices to track recurring AI failures and support system optimisation.

    Assessed by Assessment of monitoring/knowledge strategy document for clarity, feasibility, tracking of recurring AI failures, and support for system optimization.

  5. 3.5 Advise or collaborate with team members by sharing findings and recommending improvements in AI system performance and ethical compliance.

    Assessed by Review of advisory report assessing quality of recommendations, sharing of findings, and collaboration advice for AI system performance and ethical compliance.

Level e-4

From role 2.11 AI Incident Response & Reporting Lead · EQF EQF7

Coordinate the resolution of complex ICT and AI incidents, apply advanced methods to prevent recurring failures or bias, implement monitoring and knowledge management strategies, evaluate AI system performance, and lead multi-stakeholder teams in operational decisions to ensure reliable, efficient, and ethically responsible AI and ICT services in real-world organisational settings.

  1. 3.1 Coordinate teams in resolving complex ICT and AI incidents, including failures, performance issues, and bias, using collaborative tools and operational procedures.

    Assessed by Assessment of monitoring/knowledge output; completeness and reliability of AI/system monitoring

  2. 3.2 Apply advanced preventive measures to avoid recurring AI or ICT failures by implementing proactive monitoring and mitigation strategies.

    Assessed by Review of AI performance report; correctness and interpretation of metrics (accuracy, drift, bias)

  3. 3.3 Implement monitoring systems and knowledge management tools to maintain AI and system performance across organisational units.

    Assessed by Evaluation of ethical/compliance assessment; identification and mitigation of AI operational risks

  4. 3.4 Evaluate AI system performance using metrics such as accuracy, model drift, and bias indicators.

    Assessed by Review of operational decision record; rationale and outcomes of decisions for AI/ICT incidents

  5. 3.5 Assess ethical and compliance risks in AI system operation and organisational practices.

    Assessed by Assessment of management advisory; actionable recommendations for AI system reliability and ethical compliance

  6. 3.6 Lead multi-stakeholder teams in operational decision-making to mitigate AI or ICT incidents effectively and ensure responsible AI service delivery.

  7. 3.7 Advise management on strategies to enhance efficiency, reliability, and ethical compliance in AI and ICT operations.