General Purpose

In order to be able to develop the AI system, an approach to the designing, testing, deploying and monitoring needs to be agreed upon. Version control and documentation systems also need to be defined. First choices about hardware and software components to be used for the development process are made, decisions on sourcing the requisite resources are taken, and the User Experience Design is planned. The choices made in this process should conform with documentation, versioning, monitoring and other needs. It may be necessary to iterate several times between this process and other processes in the Design And Development stage, in particular the Architecture definition process, AI data engineering process and Implementation process.

Fairness Purpose

The design of the AI System needs to be aligned with the previously defined and use-case specific fairness requirements. This means in particular that all technologies and methods adopted should be assessed to ascertain that they allow for the implementation of the desired fairness measures. If data labelling is required, the general strategy for data labelling needs to be addressed at this stage, and if software is acquired externally, it should be assessed regarding its suitability to meet the fairness goals of the AI system. The user interface should be designed with the needs and requirements of the intended users in mind, and mechanisms for user and other stakeholder feedback defined. Stakeholders should be included in the decision-making process at all times.

General Purpose:

Based on the selected design approach, the requirements and the acquired data, the possible system architectures and algorithms that will be explored in the development phase are defined in this process. This can be one architecture, or several that will be developed and compared.

Fairness Purpose:

Before building an AI system, a procedure or mechanism to test the AI system quality and validity needs to be determined. For example, in supervised machine learning tasks such as classification, it is common to use error rates as quality measures for trained AI systems. To align the choice of AI system architecture, test designs should include tests for selected fairness measures and de-biasing measures.

To be able to test and validate your chosen AI system(s), it is important to ensure that available datasets are suitable, and reflect the context in which the AI system will be deployed. In the case of ML-based systems, ensure that training-, validation and test datasets are completely separate, with no possibility of leakage from one dataset to another, as this could invalidate any evaluations and lead to performance failures, as well as unintentional bias and possible fairness degradation. If insufficient data is available for separate training and validation datasets, designate an alternative validation testing scheme to establish the optimal AI system in terms of architecture and hyperparameters (such as cross-validation). In addition, it may be necessary to generate or augment existing datasets in order to obtain data of sufficient quality. Any such choices and procedures should be assessed for their fairness impact, and documented.

Tools:

data-science-ethics-checklist

General Purpose:

The system analysis process provides evidence and information to support technical decision-making throughout the AI system life cycle, and is often used together with the decision management process.

System analysis includes a wide range of methodologies, from mathematical modelling to simulation and experimentation, with a level of formality that is adaptable to suit the context of the decision to be made. Examples of decisions that can be supported with system analysis include definition of operational concepts, resolution of requirements conflicts, and evaluation of engineering strategies.

Fairness Purpose:

The system analysis process includes bias detection and analysis of bias mitigation strategies. This involves the precise definition of testing strategies for detecting anomalies, bias and discrimination, as well as the analysis of system behaviour and test results in order to assess if fairness requirements are met.

The test strategy should include definition of bias detection metrics and methods, and acceptance thresholds, as well as the precise collocation of each of the measures along the AI system life cycle. The test strategy needs to be implemented, and quality control measures for ensuring the validity of analyses instated. Additionally, the regularity of intervals at which particular fairness analyses are to be performed should be established, and traceable documentation implemented. The relevant stakeholders should be identified, and included in the entire fairness analysis process.

Tools:

Aequitas

General Purpose:

Knowledge and resources that are not yet available but necessary to create the AI system are acquired during the knowledge acquisition process. This includes third party software (including pre-trained models and expert systems), as well as knowledge acquisition in the form of hiring experts, or conducting research and interviews, etc. Proper documentation of the process is necessary in order to understand why certain decisions were taken.

Fairness Purpose:

During this process, it is important to ensure that all acquired knowledge adheres to the predetermined fairness requirements. This includes, for example, ensuring that any pre-trained models satisfy appropriate fairness tests, that results are available for verification, and that the necessary access to the models for further fairness verification and validation are obtained. It is also necessary to ensure fairness and diversity in the hiring of experts, and in the selection of interviewees.

General Purpose:

The purpose of the AI Data Engineering process is to obtain, understand and prepare data for use in developing and in evaluating the AI system.

This AI system life cycle process is often broken down into a data understanding, and a data preparation phase in other AI system life cycle models such as CRISP-DM. In this AI system life cycle process, we also include data acquisition : Data needs to be collected and its quality assured – in particular with respect to bias and representativeness. In case of automated data acquisition, in addition to the data quality, the data collection tools themselves must also be evaluated. Similarly with synthetic data generators, the synthetic data generation tools must be evaluated for bias and non-discrimination.

The data understanding phase follows upon the initial data collection, and includes activities that enable the system developers and data scientists to identify data quality problems, and gain first insights into the data. The data preparation phase covers all activities needed to construct the final datasets in the form that they can be used by the AI system. Data preparation tasks are likely to be performed multiple times, and can occur at several different timepoints in the data engineering process.

Fairness Purpose:

It is necessary to check the quality of the data and determine if it represents reality. Additionally, tests for all types of bias and discrimination should be performed. If the bias affects the protected groups defined in 1.2. Risk Analysis, the appropriate mitigation measures must be identified in 1.3. Requirement Definition and the 1.4. Project Planning project plan needs to be adapted accordingly. This should be done iteratively with 1.2. Risk Analysis. Following the insights might need to be considered in 2.4 Data Preparation and 2.9. Definition of Monitoring Plan.

Feature selection, and any transformations to the data should be documented, and checked for fairness. In case automated tools are in use, a post process evaluation must take place as well as detailed documentation about the tools and the setting used in the specific use case. If necessary, bias mitigation measures identified in data understanding are used.

Tools:

Fairlearn

General Purpose:

The purpose of the implementation process, is to realize AI system elements by translating the predefined requirements, architectures and designs into actionable steps. During this process the various modelling techniques that were selected in the definition of model architecture phase are applied, their parameters are tuned to optimal values, and thoroughly tested. Typically, there are several techniques for the same problem type. Some techniques have specific requirements on the form of data. Therefore, going back to the data engineering process is often necessary. Upon completion of this task the optimal model or models for the given use case and success criteria have been identified and selected.

Fairness Purpose:

As the first step in modelling, select the actual modelling techniques that are to be used. Although you may have already selected a tool during the Business Understanding phase, this task refers to the specific modelling techniques, and the hyperparameters that will be tested e.g., decision-tree building with different depths and pruning parameters, or neural network generation using different neural network architectures and different activation functions and learning rates. To ensure possible model architectures align with fairness goals, fairness metrics of each possible model architecture should be defined.

Many modelling techniques make specific assumptions about the data—for example, that all attributes have uniform distributions, no missing values allowed, class attribute must be symbolic, etc. Those assumptions may have an impact on the ability of a model to meet fairness needs and they should be documented in an accountable way.

Before building a model, a procedure or mechanism to test the model quality and validity needs to be generated. For example, in supervised machine learning tasks such as classification, it is common to use error rates as quality measures for trained models. To align the choice of model architecture, test designs should include tests for selected fairness measures and de-biasing measures.

To be able to test and validate your chosen model(s), split up any existing datasets into training-, testing-, and validation datasets (2.5.) to avoid unintentional bias and possible fairness degradation. If no dataset is available, designate an alternative testing scheme to establish the optimal model in terms of architecture and hyperparameters (such as cross-validation).

General Purpose

The Integration process aims to combine implemented system elements into a functional system that meets specified requirements, architecture, and design, with a focus on ensuring fairness in the integration process.

Fairness Purpose:

Validate that the integration meets fairness criteria, as well as other requirements like support, security and accessibility. This process needs to be documented very well to maintain traceability of each element an step.