The NordLink project which is the third party call in the Open-WebSearch (OWS) project, part of the Horizon 2020 initiative which is aim to establish an independent and transparent internet search index, comparable to Google, for services such as LLM training in AI applications. This project involves European universities and computing centers like LRZ and CERN for web crawling.

The Primary objective of the project NordLink will be to connect our storage and HPC resources with the OWS infrastructure and participate in web crawling and indexing activities.

The aim is to demonstrate that we are technically able to participate in the crawling and further processing of the index data as a local data center. The challenge lies in the current primary phase which is not so much the provision of technical resources, but rather the technical connection of storage space and computing resources (HPC) to the distributed OWS infrastructure via specialized interfaces.

Following are the objectives of NordLink project primary phase .

• Integration of our data center's storage and HPC resources with the OWS infrastructure.
• Ensuring seamless connectivity and data flow between local systems and the OWS network.
• Implementing and maintaining interfaces for data exchange and processing.
• Conducting web crawling and indexing to support the OWS search index.
• Documenting integration processes and preparing technical reports.

01.10.2024 - 31.08.2025

• Prof. Dr. Jorge Marx Gómez

• MSc. Khurram Shehzad

• MSc. Sreedhar Kokkarachedu