MonALISA Grid Monitoring
Search Agents
Menu mode: dynamic | fixed
  HOME       CLIENTS       REPOSITORIES       DOWNLOADS       LOOKING GLASS       FAST DATA TRANSFER  
Last update on:
Dec 03, 2015

Uptime: 165 days, 18h, 33m
Number of requests: 3974855
since 28 October 2005

Distributed Agent-Based Storage and Retrieval System


People need to share their resources among themselves for extending their collaboration.

Based on the MonALISA agent platform, we have developed a storage and retrieval system. It is a distributed, completely descentralized system that is well aware of its environment's dynamism.

The system allows its clients to:

  • Store any type of resource (text, multimedia, binary files).
  • The possibility for clients to offer a description of their uploaded resources, so that other clients will be able to find them.
  • Retrieval of resources based on their attribute-based description.
  • Resource search based on their content. This feature is currently implemented for text files, and the search is semantic-based.

The system includes:

  • Security measures: secured connections, only authorized clients' access to search and storage service
  • Intelligent replication, based on locality information, as well as on system characteristics (free memory, load etc)
  • Load balancing

The general description of the described system is rendered in the figure below.

 

Figure 1: Schematic view of the Search and Storage Agents System

The system has three main components:

  • the user shell
  • - using this shell the authorized user can interact with the search and storage distributed system in a transparent way;
  • the local daemon
  • - a deamon running on the user machine that intermediates the user with the distributed agents.
  • the search and storage distributed agents
  • - agents loaded on MonALISA distributed services that cooperate to accomplish user requests.

The authorized client has three commands, ml_add, ml_list and ml_get. The ml_add command allows the user to send a file and store it in the distributed storage system. The ml_list command searches in the system files with specified properties and shows to the user the found list. The ml_get command searches in the system files with specified properties - it is a content-based search - and sends the file to the user.

On the client machine runs a daemon that connects the user with the distributed system in a transparent way. It gets the commands entered by the client in the user shell, finds the best agents to insert the command in the distributed system (this decision is made based on the published attributes of the store and search agents), creates TCP secured connections with these chosen agents and sends them the user commands.

In the next step the agents that received user command spreads the request in the whole distributed system trying to find the desired file or trying to store the file in one or more convenient places. When the operation complets, a response is sent back to the client that request the operation.