AI Procurement: What to Look for in AI Servers and AI Providers
Artificial intelligence is rapidly making its way into businesses, educational institutions, and public organizations. Whether it’s learning platforms, AI tutors, or content generators—more and more processes are being supported by AI. As a result, the central question is shifting from “Should we use AI?” to “How do we procure AI in a way that complies with data protection regulations, is legally sound, and is future-proof?” Choosing an AI provider or an AI server in Germany has long-term implications for data protection, digital sovereignty, costs, and compliance.
Why the Right AI Procurement Is Crucial
The decision to adopt an AI solution involves much more than simply choosing a language model. Companies must give equal consideration to data protection, the EU AI Act, digital sovereignty, interoperability, and cost-effectiveness. By evaluating these factors before procurement, companies can reduce risks and establish a solid foundation for the long-term use of AI.
How to Use the AI Procurement Checklist Effectively
This checklist guides users across all sectors through the key considerations for AI procurement in the education sector, with a focus on data protection and digital sovereignty, legal and compliance issues, as well as technology, integration, and costs. It is intended as a working document: to check off items, add notes, and serve as a discussion guide with vendors and within your own organization.
Go through the eight topic areas in order. Anything you cannot clearly answer with “met” should be included in the list of requirements for the provider or in the risk assessment. The list of “red lines” at the end summarizes the points where a “no” is generally an exclusion criterion.
Note: This checklist is intended to provide guidance but does not replace individual legal or data protection advice. Please review regulatory issues with the appropriate departments within your organization.
AI Procurement Checklist – An Overview of All Considerations
- Define Requirements for AI Procurement
- Using AI in Compliance with Data Protection Regulations (GDPR)
- Digital Sovereignty and AI Servers in Germany
- The EU AI Act and AI Compliance
- Integrating AI into Existing Systems
- Realistically Assessing the Total Cost of AI
- Choosing the Right AI Provider
- Successfully Implementing AI: Pilot, Rollout, and Operations
- Red Lines: Typical Exclusion Criteria
1. Define Requirements for AI Procurement
Before comparing tools, you need to define the purpose. Otherwise, you'll end up buying technology just to find a problem.
- A specific use case and measurable benefits are described (not “we want AI”).
- Success criteria and termination criteria for a pilot have been defined.
- Make-or-Buy Decided: In-House Hosting, a Ready-Made Solution, or an API Integration?
- Responsibilities have been assigned (subject-matter, technical, data protection, legal).
- An internal AI policy/terms of use either exists or is currently being developed.
- Measures to develop employees' AI competencies are planned (a requirement for operators under the EU AI Act, Art. 4).
2. Using AI in Compliance with Data Protection Regulations (GDPR)
In the education sector, personal data and data requiring special protection are often processed, frequently involving minors.
- The legal basis and purpose of the processing have been clarified.
- A Data Processing Agreement (DPA pursuant to Article 28 of the GDPR) is in place.
- Data minimization: Only the data that is truly necessary is processed.
- It has been clarified whether user input or user data is used to train AI models—and this can be excluded in the contract or prevented via an opt-out option.
- The location where the data is stored and processed is clearly documented.
- A data deletion and retention policy is in place (including data return).
- If the risk is expected to be high, a data protection impact assessment (DPIA, Art. 35 of the GDPR) is required.
- Data subjects' rights (right to access, right to erasure, right to object) can be exercised.
- Exercise particular caution when handling data on minors and health/social data.
3. Digital Sovereignty and AI Servers in Germany
Sovereignty means retaining control over data, models, and your own ability to act—even when the market, prices, or providers change.
- Hosting in Germany or the EU is possible and can be contractually guaranteed.
- Open-source options were evaluated (more control, less lock-in) and weighed against proprietary solutions.
- It is clear who controls the AI model being used and where it is operated.
- Data can be exported at any time in open, widely used formats.
- An exit strategy, including the full return and deletion of data, has been agreed upon.
- For transfers to third countries (e.g., U.S. providers): valid certification under the EU-U.S. Data Privacy Framework has been verified; standard contractual clauses are in place as a fallback; the entire subprocessor chain has been reviewed. (The framework is being challenged in court, so a residual risk remains.)
- Dependence on a single provider is deliberately evaluated (alternative providers, ability to switch).
4. The EU AI Act and AI Compliance
This is what determines whether a project may be carried out in compliance with the law at all.
EU AI Act
- Clarify your role: Are you the operator (deployer) or the provider of the AI? (In procurement, you are usually the operator.)
- Determined by the use case's risk class. Note: Many educational applications are considered high-risk, such as AI used to make decisions regarding access/admission, to assess learning outcomes, to determine educational levels, or to monitor exams (proctoring).
- Prohibited practices are excluded; in particular, emotion recognition in educational institutions is prohibited.
- Transparency requirements met (Art. 50): Users are informed about the AI interaction, and AI-generated content is labeled.
- For high-risk operations: human oversight, logging (logs, generally for at least 6 months), and, if necessary, a fundamental rights impact assessment (FRIA) must be planned; proof of compliance from the provider (including registration in the EU database) must be required.
- Keep an eye on the deadlines: The high-risk obligations were originally scheduled to take effect on August 2, 2026; under the “Digital Omnibus” (provisional agreement reached in May 2026), a postponement to December 2, 2027, is planned for Annex III systems, but this has not yet been finalized. Do not delay your preparations.
Copyright
- The legality of the training data appears reasonable (no apparent violation of the law by the provider).
- The rights to use the AI results have been clarified (including for commercial purposes and after the contract ends).
- Indemnification against third-party intellectual property claims is governed by the terms of the contract.
- Careful attention is paid to the use of sources, citations, and the risk of plagiarism in generated content.
Accessibility
- AI interfaces and outputs comply with accessibility requirements (BFSG effective June 2025, EN 301 549/WCAG)—relevant for many digital products and services.
5. Integrating AI into Existing Systems
An AI solution is only as good as its integration into the existing system landscape.
- Open interfaces are available (API; for learning platforms, e.g., LTI for integration with Moodle/LMS).
- Integration with existing systems has been tested (SSO/Single Sign-On, LMS, administration).
- Interoperability is ensured through open standards and data formats.
- Scalability and performance are sufficient for the expected number of users.
- Procedures for handling model updates are established (versioning, reproducibility, change management).
- Quality assurance is possible (handling errors/hallucinations, bias checks, human verification).
- Information security is certified (e.g., ISO 27001, BSI C5; penetration tests).
6. Realistically Assess the Total Cost of AI
The license fee is rarely the largest cost item.
- Total cost of ownership (TCO), including integration, migration, training, and operation, has been calculated.
- The pricing model is understood (perpetual license vs. usage-based/token), and cost caps can be set.
- Hidden costs reviewed (data export fees, premium support, additional modules).
- Cost trends as the system scales are transparent.
- A low-cost pilot project is planned prior to full-scale implementation.
- The expected benefits (time savings, quality, reach) are defined in measurable terms.
7. Choosing the Right AI Provider
You're entering into a long-term relationship—examine your partner as carefully as you would a product.
- The provider's experience, references, and financial stability have been verified.
- The company's location and the jurisdiction are known and acceptable.
- Service levels (availability, response times, support) are specified in the contract.
- All subcontractors and sub-processors are disclosed.
- Relevant certifications are in place (e.g., ISO 27001, BSI C5, and, in the future, ISO 42001 for AI management).
- Contractual provisions regarding data return and deletion upon termination, audit rights, liability, and the right to make changes are set forth.
- Notice periods and an orderly exit have been agreed upon.
8. Successfully Implementing AI: Pilot, Rollout, and Operations
Procurement doesn't end with a signature.
- A pilot phase involving real-world scenarios and actual users is planned.
- The evaluation criteria for the pilot decision have been established.
- Training and change management are scheduled.
- Internal usage rules and the AI policy have been communicated.
- Responsible contacts for day-to-day operations have been designated.
- A regular review (data protection, quality, legal compliance, costs) has been scheduled.
9. Red Lines: Typical Exclusion Criteria
If any of these points apply, you should take a very close look: In most cases, it’s a reason to terminate the contract:
- Input and user data are used for training without the option to opt out.
- No Data Processing Agreement (DPA) is available.
- Prohibited practices under the EU AI Act (e.g., emotion recognition in educational institutions).
- There is no contractual guarantee of data return or deletion.
- An opaque processing chain involving subcontractors in unsafe third countries.
- For high-risk operations: no proof of compliance from the provider.
- The inability to export data in open formats amounts to de facto lock-in.
Conclusion
AI procurement in the education sector is not purely a technical issue, but rather a balancing act involving data protection, legal considerations, sovereignty, integration, and cost-effectiveness. Those who clarify these points before signing a contract can avoid costly corrections and retain control over their own data and their ability to act. The most important strategy is often the simplest: first clarify the purpose and requirements, then assess the market, and then conduct a pilot—not the other way around.
Frequently Asked Questions About AI Procurement
Why Are AI Servers in Germany Important?
The location of an AI server determines where data is processed and which legal framework applies. Hosting in Germany—or at least within the EU—can facilitate compliance with the GDPR and often offers greater transparency regarding data protection and data security. However, it is not just the server location that matters, but also who has access to the data, which subcontractors are involved, and whether personal data is used to train AI models.
Is having an AI server in Germany enough to be GDPR-compliant?
No. The server location is just one component. For AI usage to be GDPR-compliant, several elements must be in place, including the legal basis for processing, a data processing agreement (DPA), technical and organizational measures, and transparent data processing procedures. Data deletion policies, data subject rights, and control over sub-processors also play an important role.
What Does Digital Sovereignty Mean in the Context of AI?
Digital sovereignty means that organizations retain control over their data, the AI models they use, and their technical dependencies. This includes open interfaces, the ability to switch providers, data export in open formats, and the freedom to use different AI models or hosting options. The goal is to avoid long-term dependencies on individual providers.
What requirements does the EU AI Act impose on companies?
The EU AI Act requires companies to assess the intended use of their AI systems and to comply with the relevant legal requirements. Different obligations apply depending on the risk class. The requirements are particularly strict for so-called high-risk AI systems, for example in areas such as education, human resources, or critical infrastructure. Operators must also ensure that employees have sufficient AI expertise and that transparency requirements are met.
What should I look for when choosing an AI provider?
In addition to the range of features, data protection, data sovereignty, and long-term flexibility should be key considerations. Key factors include transparent data handling, a data processing agreement (DPA), open APIs, clear policies on data deletion, traceable security measures, and an exit strategy in case of a provider change. It’s also important to verify whether different AI models can be integrated flexibly.
Should I use open-source AI or proprietary solutions?
There is no one-size-fits-all answer. Open-source solutions often offer more control over data and infrastructure, as well as less dependence on individual vendors. Proprietary solutions, on the other hand, often score points for easier implementation, greater ease of use, and comprehensive support services. Which option is more suitable depends on the organization’s requirements, available resources, and data protection regulations.
What are the costs associated with implementing AI?
In addition to licensing costs, companies should consider the total cost of ownership (TCO). This includes, among other things, implementation, integration into existing systems, training, ongoing operations, support, and any usage-based fees for AI models or tokens. A pilot project can help provide a realistic assessment of the costs and benefits.
Why Should Every AI Implementation Start with a Pilot Project?
A pilot project makes it possible to test the actual benefits of an AI solution under real-world conditions before rolling it out company-wide. At the same time, it allows for the evaluation of data protection, user-friendliness, integration, and cost-effectiveness. Clear success and termination criteria help in making informed decisions for or against a subsequent rollout.
How can I avoid vendor lock-in with AI solutions?
Vendor lock-in occurs when switching to another provider is either extremely difficult or impossible. To avoid this, companies should look for open standards, standardized interfaces (APIs), exportable data formats, and contractually guaranteed data return. It is also advisable to prioritize solutions that support multiple AI models or providers.
What role does AI literacy play in a company?
Technical solutions alone are not enough. Employees must understand how to use AI effectively, safely, and responsibly. The EU AI Act requires operators of certain AI systems to ensure that employees have sufficient AI literacy. Training and internal guidelines are therefore an important part of any AI implementation.
Links
More Information About AI in Moodle
eLeDia.ai Suite Information Webinar
eLeDia.academy Program
Practical Applications of AI in e-Learning: Tools, Prompting, and Creating Learning Materials
