Because responsible customer interaction requires quality that evolves alongside the customer

This involves simulating various conversation paths, unexpected user requests, or edge cases to test the phone AI's behavior and responsiveness. The AI is deliberately allowed to move „freely“ through different conversation scenarios. The goal is to identify and optimize potential ambiguities, incorrect statements, flawed reasoning, inappropriate responses, or workflows that are not phrased as optional. In addition, we analyze whether the AI correctly understands the stated concerns, asks strategically sound follow-up questions, and provides technically accurate answers. The focus is particularly on whether EIKI® operates within its defined scope of action and shows high consistency and controllability in its responses.

During the development and integration phase, tests are conducted at regular intervals both internally and with the customer. This helps identify early whether existing conversation flows remain consistent and high quality when prompt templates, guardrails, or the underlying workflow are adjusted, or whether „inconsistencies“ have emerged. Continuous testing stands for ongoing quality assurance through permanent tests that check response consistency, conversation structure stability, and the interplay between dialogue flow and process logic after every change.

System tests analyze the technical infrastructure of our phone AI in a real target environment and the intended setup. In particular, the following are tested:

• synchronization between individual system components, such as the voice module, knowledge base/RAG, process logic, etc.
• whether interfaces to CRM, ERP, SAP, or other systems work as intended
• response to potential interface issues, if applicable
• data transfer and processing
• response times and time-out behavior
• tenant and use-case separation, etc.

In the double-blind test, the AI's dialogue is compared directly with a human conversation. Both real and AI-based conversations are conducted and evaluated in two independent test groups, without participants knowing which variant they are experiencing. In particular, we analyze comprehensibility, naturalness, and consistency of the conversation structure, perceived expertise, the AI's role awareness, and how objections are handled, as well as trust in the dialogue. Based on this method, quality differences can be identified and bias reduced.

In this approach, the phone AI is used in a controlled pilot operation, examining real interactions with a limited user pool. The field test not only tests the AI's functionality but primarily its acceptance in real usage behavior and how well real conversation patterns match the expected conversation design. In addition, drop-off points, typical or frequent user requests, acceptance signals, comprehension weaknesses, and the right moment to hand over to a human contact are reviewed. This is followed by a gradual expansion of the rollout.

Stress tests are used to check system stability under intensive use. The AI is deliberately exposed to load peaks. Complex requests and potential error conditions are also simulated to identify and fix possible weaknesses before they occur unexpectedly in live operation. Among other things, the following are tested:

• system response under resource constraints
• stability of the conversation flow under simultaneous and complex requests
• robustness of the process chain under heavy load
• EIKI®'s performance under full load
• scalability
• prioritization and resolution of errors

The GDPR test continuously reviews compliance with data protection requirements and the data protection parameters defined in the setup. This includes, in particular, agreed configurations regarding audio/transcripts, deletion periods, access concepts, and purpose limitation. In addition, data flows between system components and connected third-party systems are permanently monitored to ensure that data is transmitted exclusively via clearly defined and secured interfaces. Furthermore, logging and audit mechanisms are analyzed to enable auditable and traceable processing of personal data.

The transparency test identifies whether the AI identifies itself as such right at the start of the conversation and transparently discloses its role in the dialogue. This ensures that the caller understands from the outset that they are interacting with a digital system and within what scope of responsibility the conversation is being conducted. In addition, it is reviewed whether further processes, such as the handover to a human contact when needed or the processing of data, are communicated clearly to the caller.

The fake test primarily examines whether the phone AI detects potential manipulation attempts by other AIs or external parties and how it handles them. It serves to test how EIKI® reacts to misleading requests, external influence, or unexpected actions:

• Are the defined guardrails maintained during „trick questions“?
• Does the role logic remain stable and consistent?
• Does the AI refuse requests that fall outside its previously defined scope of action?
• Does the phone AI identify potential prompt injection attempts or attempts to bypass internal decision rules?
• Is the caller successfully transferred to a human employee in cases of uncertainty?
• Does the argumentation structure remain compliant even under (negative) influence?