KROG
🇳🇴

CWA 18398 · assessment rubric

C.2 Change Support

← All competences

Level e-2

From role 3.2 AI Service Support Specialist · EQF EQF5

Apply controlled updates to AI models and datasets by following guided procedures, while ensuring reliable system operation, monitoring performance, and complying with security, data protection, and AI governance standards.

  1. 2.1 Recall and describe the basic structure and components of AI-based systems, including models, data pipelines, and software modules, using real-world examples of AI applications.

    Assessed by Review of AI system architecture diagram and oral questioning on models, data pipelines, software modules, components, and structure in a real-world AI application.

  2. 2.2 Apply defined procedures to implement updates to AI models, software, or datasets under supervision, in a controlled development or testing environment.

    Assessed by Practical evaluation of updated AI model/dataset/software in a controlled environment, supported by implementation notes or change log showing adherence to defined procedures and supervision requirements.

  3. 2.3 Monitor the operational behaviour of AI solutions after changes and report deviations from expected performance, using standard system monitoring tools or dashboards.

    Assessed by Assessment of AI monitoring report after change implementation, evaluated for accuracy, completeness of performance tracking, and clear reporting of deviations from expected operational behaviour.

  4. 2.4 Identify and follow security, data protection, and AI governance requirements when applying changes, such as organisational policies or regulatory guidelines.

    Assessed by Verification of AI compliance checklist, with audit evidence that security, data protection, and AI governance requirements were identified and followed during the change.

  5. 2.5 Demonstrate awareness of service continuity principles by performing rollback or fallback operations under guidance, in case of system failures or unexpected behaviour.

    Assessed by Review of AI rollback/fallback record, including demonstration or documentation that the learner performed the procedure under guidance and evaluated its effectiveness for service continuity.

Level e-3

From role 2.7 AI Deployment Engineer · EQF EQF6

Plan, deploy, and optimise updates to AI models and datasets by independently analysing operational, ethical, and service impacts, monitoring system behaviour, and ensuring compliance with AI governance and service standards.

  1. 5.1 Analyse the impact of AI-related changes on operational performance, service continuity, and compliance, using metrics, logs, or simulation results.

    Assessed by Evaluation of AI impact analysis report, assessed for correctness of metrics, operational insights, and analysis of AI-related change impacts on performance, service continuity, and compliance.

  2. 5.2 Plan deployment and versioning strategies for AI models and datasets to avoid conflicting updates, in multi-team or production environments.

    Assessed by Review of AI deployment/versioning plan, assessed for feasibility, completeness, conflict mitigation, and suitability for AI model and dataset versioning in multi-team or production environments.

  3. 5.3 Evaluate ethical, fairness, and explainability considerations when implementing AI changes, by reviewing decision-making outputs or bias reports.

    Assessed by Assessment of AI ethical assessment report, evaluating fairness, explainability, bias considerations, and evidence from decision-making outputs or bias reports.

  4. 5.4 Coordinate with multiple teams to implement AI changes in accordance with organisational AI governance and SLAs, using project management practices or communication tools.

    Assessed by Review of AI coordination documentation, including evidence of multi-team collaboration, project management or communication tool use, and adherence to organisational AI governance and SLAs.

  5. 5.5 Apply monitoring techniques to detect AI model drift or unexpected outputs after deployment, in live or staged AI environments.

    Assessed by Evaluation of AI post-deployment monitoring report, assessed for application of monitoring techniques to detect model drift, unexpected outputs, or anomalies in live or staged AI environments.

  6. 5.6 Recommend corrective actions or improvements based on performance and compliance assessment, by analysing operational logs, performance metrics, or audit results.

    Assessed by Assessment of AI corrective action report, focusing on clarity, rationale, practicality, and use of operational logs, performance metrics, or audit results to justify proposed improvements.

From role 2.8 MLOps Engineer · EQF EQF6

Plan, deploy, and optimise updates to AI models and datasets by independently analysing operational, ethical, and service impacts, monitoring system behaviour, and ensuring compliance with AI governance and service standards.

  1. 5.1 Analyse the impact of AI-related changes on operational performance, service continuity, and compliance, using metrics, logs, or simulation results.

    Assessed by Evaluation of AI impact analysis report, assessed for correctness of metrics, operational insights, and analysis of AI-related change impacts on performance, service continuity, and compliance.

  2. 5.2 Plan deployment and versioning strategies for AI models and datasets to avoid conflicting updates, in multi-team or production environments.

    Assessed by Review of AI deployment/versioning plan, assessed for feasibility, completeness, conflict mitigation, and suitability for AI model and dataset versioning in multi-team or production environments.

  3. 5.3 Evaluate ethical, fairness, and explainability considerations when implementing AI changes, by reviewing decision-making outputs or bias reports.

    Assessed by Assessment of AI ethical assessment report, evaluating fairness, explainability, bias considerations, and evidence from decision-making outputs or bias reports.

  4. 5.4 Coordinate with multiple teams to implement AI changes in accordance with organisational AI governance and SLAs, using project management practices or communication tools.

    Assessed by Review of AI coordination documentation, including evidence of multi-team collaboration, project management or communication tool use, and adherence to organisational AI governance and SLAs.

  5. 5.5 Apply monitoring techniques to detect AI model drift or unexpected outputs after deployment, in live or staged AI environments.

    Assessed by Evaluation of AI post-deployment monitoring report, assessed for application of monitoring techniques to detect model drift, unexpected outputs, or anomalies in live or staged AI environments.

  6. 5.6 Recommend corrective actions or improvements based on performance and compliance assessment, by analysing operational logs, performance metrics, or audit results.

    Assessed by Assessment of AI corrective action report, focusing on clarity, rationale, practicality, and use of operational logs, performance metrics, or audit results to justify proposed improvements.

From role 4.6 AI Operations Manager · EQF EQF6

Plan, deploy, and optimise updates to AI models and datasets by independently analysing operational, ethical, and service impacts, monitoring system behaviour, and ensuring compliance with AI governance and service standards.

  1. 1.1 Analyse the impact of AI-related changes on operational performance, service continuity, and compliance, using metrics, logs, or simulation results.

    Assessed by Evaluation of AI impact analysis report, assessed for correctness of metrics, operational insights, service-continuity implications, and compliance impact of AI-related changes.

  2. 1.2 Plan deployment and versioning strategies for AI models and datasets to avoid conflicting updates, in multi-team or production environments.

    Assessed by Review of AI deployment/versioning plan, with focus on feasibility, completeness, conflict mitigation, and suitability for multi-team or production environments.

  3. 1.3 Evaluate ethical, fairness, and explainability considerations when implementing AI changes, by reviewing decision-making outputs or bias reports.

    Assessed by Assessment of AI ethical assessment report, evaluating fairness, explainability, bias considerations, and evidence from decision-making outputs or bias reports.

  4. 1.4 Coordinate with multiple teams to implement AI changes in accordance with organisational AI governance and SLAs, using project management practices or communication tools.

    Assessed by Review of AI coordination documentation, including evidence of multi-team collaboration, use of project management practices or communication tools, and adherence to governance/SLAs.

  5. 1.5 Apply monitoring techniques to detect AI model drift or unexpected outputs after deployment, in live or staged AI environments.

    Assessed by Evaluation of AI post-deployment monitoring report, including applied monitoring techniques, detection of model drift or unexpected outputs, and evidence from live or staged AI environments.

  6. 1.6 Recommend corrective actions or improvements based on performance and compliance assessment, by analysing operational logs, performance metrics, or audit results.

    Assessed by Assessment of AI corrective action report, focusing on clarity, rationale, practicality, and use of operational logs, performance metrics, or audit results to justify proposed improvements.