Explainability Requirements Met

Preliminary Requirements Set

Types of Ex. needed?

Who needs explanations?

Who's responsible for providing explanations and implementing ex. techniques?

Existing systems to use as a guide?

Hazards Identified

Risks of lack of explainability?

What types of explainability needed to mitigate hazards?

Evidence to support sufficiency of explainability

Evaluation of Ex. Techniques Planned

e.g. human tests

Model Learning

Model Selection

Could a more ex. model be used?

Accuracy/explainability tradeoffs

What types of ex. are possible for potential models?

Do they meet requirements?

Explainability techniques planned

Techniques appropriate for model

Techniques Provide appropriate types of Ex.

Model Verification

Errors identified and appropriately explained/corrected

Explanations used to provide insights into operation domain

Explanations used to justify results provided by model

Acceptable safety is assured in case of misuse and unavoidable accidents

Potential harm from misuse is mitigated

Explainability allows insight to foresee and limit potential for misuse

Potential harm from accidents is mitigated

Explainability allows contestability to challenge harmful model decisions

Global Ex. allows predictability so accidents can be predicted

Potential harm from adversarial attacks is mitigated

Users trust model

Users satisfied with explanations

Legal requirements met

User Needs

Fairness

Trust

Answer causal/contrastive questions

Legal Needs

How do certain input factors affect outputs?

Users have a "right to explanation"

Explain Mistakes/accidents

Technical Needs

Explainability

Local

Can individual decisions be explained?

Analyse why a particular problem occurred

Improve/correct model

Global

Generate explanations about the whole logic of the model

Inform design

Make predictions about future outputs

Robustness

Transparency

Relate inputs to outputs

Relate parameters to human-understandable concepts

Justification

Predictability

Do explanations allow us to predict model behavior?

Contestability