EU Frontier AI Initiative: Advanced Scaling Frameworks for High-Performance AI Models (RIA) - Apply AI

Expected Outcome:

This topic aims to advance the scientific and technical foundations for the scaling of AI models, enhancing the performance of training and inference processes and providing European AI labs with the methodologies required to compete at the global frontier. Project results are expected to contribute to some of the following outcomes:

1. Establishment of robust scaling laws and predictive frameworks that allow for the reliable estimation of model performance before large-scale compute investment.
2. Enhanced competitiveness of European advanced AI models through optimised dataset mixtures, refined architectures, and superior hyperparameter configurations.
3. Optimised development for frontier AI models, achieved via automated and scientifically grounded scaling methodologies.

Scope:

To maintain a leading position in AI, Europe must master the methodologies required to scale AI models’ performance, training and inference processes. Establishing a predictive methodology for AI model development ensures that limited computational resources are deployed with maximum efficiency, thereby bolstering the EU and AC’s technological sovereignty.

While previous actions have focused on training High-Performance AI models, this topic addresses the research gap in the underlying theoretical basis, methodologies and tools for predictable scaling of AI models and inference. By providing the technical foundations for high-performance systems without the costs of unguided experimentation, this action complements training-centric efforts. In this light, proposals should deliver, as an outcome of the research, a toolset designed to support the scaling of AI models.

Potential research areas include, but are not limited to:

1. Automated Scaling Law Derivation: Research into scaling laws to predict performance across diverse modalities and tasks, moving beyond simple power laws.
2. Exploratory Architecture Search: Developing automated methods to discover and refine model architectures that maintain performance while scaling both in training and inference, including innovations in sparsity, memory efficiency, and long-context handling.
3. Dataset Mixture Optimisation: Research into the systematic selection, weighting, and sequencing of datasets. This includes developing tools for automated data quality assessment.
4. Advanced Hyperparameter Tuning and Stability: Developing methodologies for stable training at large scales, including automated tuning of optimisers and learning rate schedules that remain effective as models grow.

Proposals should ensure that the developed frameworks and scaling methodologies are compatible with diverse hardware architectures, demonstrating that scaling is studied across different systems to avoid optimisation for specific proprietary hardware.

The selected project should actively support Advanced AI labs and High-performance Computing Centres already engaged in the development of frontier AI, ensuring that methodological advances are immediately validated in high-stakes environments. Proposals should describe how these scaling methodologies would benefit the development of models adapted to the specific requirements of particular strategic sectors, such as Public Administration, Manufacturing, Energy, Healthcare, Cultural and Creative Industries, or Physical AI.

Part of the Frontier AI Initiative, this topic complements HORIZON-CL4-2027-04-DIGITAL-EMERGING-11 “EU Frontier AI Initiative: Developing frontier AI solutions that are safe and computationally efficient within Apply AI” by establishing the underlying methodologies required for the large-scale training and inference of High-Performance AI models. Proposals should, therefore, describe the link between their planned activities and the objectives of topic HORIZON-CL4-2027-04-DIGITAL-EMERGING-11.

All proposals are expected to incorporate mechanisms for assessing and demonstrating progress, including qualitative and quantitative KPIs, benchmarking, and progress monitoring. When possible, proposals should build on and reuse public results from relevant previous funded actions. Communicable results should be shared with the European R&D community through the AI-on-demand platform.

The project selected in this topic should link to the resources offered by the AI Factories and the Data Labs. Where relevant, links could be established to European companies developing frontier AI models to foster a pre-competitive exchange of expertise and explore ways to mobilise the wider developer community for the efficient reuse of results.

All proposals are expected to allocate tasks for cohesion activities with the European Partnership on AI, data, and robotics (ADRA) and the CSA HORIZON-CL4-2025-03-HUMAN-18: GenAI4EU central Hub.

Technology Readiness Level - Technology readiness level expected from completed projects