KI-PRO

AI Competence Profile

Measure AI competence objectively –
in around ten minutes

AI has become part of everyday working life, but competence development is lagging behind. Anyone looking to manage training in a targeted way or take AI competence into account in personnel selection needs reliable data. Self-assessments do not provide this: they show how competent someone feels, not how competent they are.

KI-PRO measures AI competence as a knowledge test – objectively, standardised and in around ten minutes. The assessment was developed by the Human-AI Institute, a spin-off from the University of Würzburg, based on several years of research into measuring AI competence. It is delivered via our IONA online testing platform, with results evaluated automatically.

KI-PRO at a glance

Everything you need for fast, fair and scalable screening

Objective rather than subjective

The selection of tasks adapts to the level of performance. This improves differentiation and supports a positive test-taking experience.

Efficient to use

Around ten minutes to complete, computer-based administration and automated scoring. Practical even for large groups and repeated assessments of current status.

Four-dimensional profile

Alongside the overall score, four subscales show where strengths and development areas lie. This makes it clear which training is appropriate for whom.

Scientifically grounded

The assessment is based on research at the University of Würzburg by PD Dr. habil. Astrid Carolus and Dr. André Markus.

Fair measurement

Not a single item shows evidence of differential item functioning. The test measures the same construct regardless of gender, age or educational qualification.

Detailed norm values

Classification is based on benchmark data (percentile ranks).

What does KI-PRO measure?

KI-PRO assesses knowledge and understanding of how to work with artificial intelligence. The tasks are predominantly scenario-based, describing brief, realistic situations from work and everyday contexts. The test is constructed on the basis of an established competency model that has been extended to include aspects of generative AI. Four subscales represent the construct:

Basic understanding of AI (9 tasks)

Understanding AI as a data-driven, probabilistic technology: key mechanisms, fundamental types of AI and typical effects of data-driven systems, such as personalised information provision and the narrowing of information.

AI application competence (7 tasks)

Using AI for problem-solving, analysis and decision support: knowledge of fundamental machine-learning methods and modelling processes, as well as the ability to interpret results and transfer them to new problems.

Generative AI competence (6 tasks)

Understanding how generative systems (such as ChatGPT, Copilot etc.) work and directing them purposefully: competent prompting, suitable contexts of use and a realistic assessment of their capabilities.

AI ethics competence (5 tasks)

Recognising and critically reflecting on ethical, social and societal implications: bias, lack of transparency, data protection, information manipulation, and issues of fairness, transparency and responsibility.

Who is AI-PRO suitable for?

AI-PRO is aimed at working adults and adults in vocational training or higher education, particularly those who work with AI systems in a professional or academic capacity.

Personnel development & organisational assessment

Baseline assessment for departments, roles or the entire workforce · Evaluation of training measures through before-and-after comparisons · Identifying specific development needs rather than providing indiscriminate training

Personnel selection

Structured assessment of AI competence wherever the proficient use of AI is a job requirement · can be readily combined with interviews, work samples and assessment centre exercises · supports consistent decisions across teams and locations

How KI-PRO works

Invite

You add participants in IONA or your applicant tracking system and send out the test link.

Complete

The test is completed online in around ten minutes. The tasks are presented in a randomised order; supervised and unsupervised settings are possible.

Evaluate

Evaluation is automatic: the overall score and subscale profile are converted into norm scores and made available immediately upon completion.

Decide

HR receives a concise overview – as a basis for development decisions or as part of a multimodal selection process.

Developed with the Human-AI Institute

KI-PRO was developed by the Human-AI Institute, a spin-off from the University of Würzburg. Scientific leadership is provided by PD Dr. habil. Astrid Carolus and Dr André Markus, whose work on measuring AI competence is widely cited internationally. KI-PRO therefore combines the current state of research with the requirements of professional occupational assessment.

Quality & psychometric criteria

Reliability

A Cronbach’s alpha of α = .82 and congeneric reliability of CR = .92 demonstrate good to very good internal consistency. The indices from item response theory also indicate high measurement precision (EAP reliability r = .84, empirical reliability r = .82, average conditional reliability r = .86). At subscale level, the values range from CR = .70 to CR = .74.

Validity

Convergent: significant positive correlation with an established AI knowledge test (r = .51)
Discriminant: no meaningful correlation with self-assessed AI competence (r = .02) – objective and subjective competence are distinct constructs
Predictive: significant association with performance in a computer science test (r = .46)
Structural: confirmatory factor analysis with very good model fit for the four-factor structure (CFI = .968, TLI = .965, RMSEA = .046, SRMR = .046)

Manual & documentation

We can provide you with a detailed test manual containing full documentation of the quality criteria and norm tables on request. As the AI field is developing dynamically, the item pool is reviewed and further developed regularly.

Experience KI-PRO live

Would you like to know where your organisation stands in terms of AI competence? We will show you KI-PRO, from the invitation through to the results report, and advise you on integrating it into your selection and development processes.