Halconia.
Let’s talk
Back to blog

How to Choose the First AI Project in an Industrial SME

To choose the first project, it is worth keeping in mind that the risks the organization considers highest for an AI system in its context of use require the…

Andrés Naves ·

How to Choose the First AI Project in an Industrial SME

To choose the first project, it is worth keeping in mind that the risks the organization considers highest for an AI system in its context of use require the most urgent prioritization. Before comparing, define for each one its purpose, its context of use, and the value it provides. Then, it is advisable to prefer an AI case designed to interact only with computational systems and trained on non-sensitive data, for example data collected from the physical environment, which may warrant a lower initial prioritization.

How do I choose the first AI project in my industrial SME: what value, what risk, and what type of case suits me best?

For each candidate, it is advisable to document the intended purposes, the context of use, and the business value, in line with MAP 1.1 and MAP 1.4. The NIST AI RMF (MAP 1.1) calls for understanding and documenting the context of use, and among its considerations it includes the possible positive and negative impacts of the system's use on individuals, communities, organizations, society, and the planet.

Next, the organization's risk tolerances must be determined and documented, according to MAP 1.5. The framework does not prescribe risk tolerance: when no established guidelines exist, organizations must define a reasonable risk tolerance.

With that data, policies and resources should be prioritized according to the assessed risk level and the potential impact of each AI system. NIST indicates that a system that only interacts with computational systems and is trained on non-sensitive data, such as data collected from the physical environment, may warrant a lower initial priority. A higher initial priority may be required when the system is trained on large sensitive or protected datasets, such as personally identifiable information, or when its outputs have a direct or indirect impact on people.

The EU AI Act classifies as high risk the systems used to allocate tasks based on individual behavior or to monitor and evaluate working persons. The recital of the AI Act on employment and worker management states that AI systems used to allocate tasks based on individual behavior, personal traits or characteristics, or to monitor or evaluate persons in work-related contractual relationships should also be classified as high risk, because they can have an appreciable impact on their career prospects, livelihoods, and workers' rights. The recital of the AI Act on employment and worker management classifies as high risk the AI systems used, for example, to allocate tasks based on individual behavior or to monitor or evaluate persons in employment relationships, and that passage does not mention cases that only analyze process data.

What data do I need, and in what state must it be for the chosen case to be viable?

Before choosing a case, ask yourself what information the system needs to fulfill its objective. The AESIA data guide, published by the Ministry for Digital Transformation, places that definition as the first step of the data cycle.

Next, collection consists of obtaining the data that contains the information needed to develop the AI system. The precision and quality of the solution will depend, among other elements, on how representative and sufficient the data are, which is generally drawn from different sources.

The same guide defines data quality as the degree to which data are fit for the purpose for which they were collected. Measuring it and improving data that do not reach sufficient quality is an essential process of good data governance.

If there is personal data, the applicable regulations on personal data protection must be taken into account. A candidate is only viable if it can clearly answer what information it needs and whether that information is adequate for its purpose.

How do I define in advance what it means for the first project to have worked?

The performance or assurance criteria of the AI system must be measured, qualitatively or quantitatively, and demonstrated under conditions similar to those of deployment, according to MEASURE 2.3.

Once in production, MEASURE 2.4 indicates that the functionality and behavior of the AI system and its components must be monitored.

If the AI is going to increase or replace a human activity, it needs a reference against which to compare. NIST acknowledges that this baseline is difficult to systematize, because the system and people do not perform tasks in the same way. Therefore, it is advisable to set the metric, the point of comparison, and the test conditions before seeing the results.

Checks for choosing and validating the first use case

Check

What the source says

How to verify it

Business value and purpose

The business value or context of use is clearly defined, and the purpose, beneficial uses, and intended context are understood and documented.

A document exists that records the case's purpose, context, and business value.

Risk tolerance

The framework does not prescribe risk tolerance; where no established guidelines exist, the organization must define a reasonable one.

The organization's risk tolerance is determined and documented before choosing.

Impact on people and type of data

Higher risk priority is justified if the system uses sensitive or personal data or its outputs affect people. With non-sensitive physical-environment data and no human interaction, a lower initial priority may suffice.

Check whether the case processes personal data or whether its outputs affect people.

Management or evaluation of workers

Systems that allocate tasks based on individual behavior or that monitor or evaluate workers are classified as high risk.

Check whether the case allocates tasks or evaluates people in their employment relationship.

Necessary information

It must be established what information will feed the system to achieve the objective for which it is designed.

The necessary information is listed and linked to the case's objective.

Data quality

Quality is the degree to which data are fit for their purpose, and measuring and improving it is an essential process of data governance.

The fitness of the data for the case's purpose has been measured and insufficient data have been corrected.

Measurable success criterion

Performance is measured, qualitatively or quantitatively, and demonstrated under conditions similar to deployment, with the measures documented.

Performance criteria and measures are documented and tested under conditions close to the plant.

Comparison baseline

If AI increases or replaces a human activity, some reference metric is needed for comparison.

A metric of the current activity exists against which to compare the system's results.

Sources