Vigne : A Grid Middleware for Dynamic Large Scale Grids

Overview

Vigne is a grid middleware targeting large scale dynamic Grids. It is based on peer-to-peer networks to deal with grid scale and grid nodes volatility. Vigne is a self-healing system: nodes can be added or removed at any time without altering the middleware working. Active replication is used to make Vigne services highly available. An application management service ensures efficient and reliable execution of users applications in the Grid.

  • A Fully Distributed Grid Middleware: Based on peer-to-peer overlays, Vigne is fully distributed to be scalable and avoid single point of failures.
    • A structured peer-to-peer overlay is used to interconnect all the nodes of the system. A DHT is build on top of this overlay to manage entities composing Vigne distributed services. All entities have virtual positions: they are identifies by ids and have to be addressed through the DHT.
    • An unstructured peer-to-peer overlay is used to implement the information service. The information service allows to find grid resources that fit grid users requirements.
  • Self-Healing Distributed Services: Vigne is based on a framework called Semias to actively replicate distributed services on top of the structured peer-to-peer overlay. Semias ensures the high availability of Vigne services despite node volatility using self-healing techniques to automatically reconfigure replicas sets according to the changes in the overlay.
  • Distributed Resource Management: An information service is in charge of finding suitable resources for the execution of users application. It is based on the RW-OGS cached-based optimized random walk protocol in an unstructured peer-to-peer overlay. RW-OGS is optimized to increase the probability of discovering free resources and to enable fast discovery of rare resources.
  • Application Management: Vigne application management service handles application life-cycle in the Grid.
    • Reliable Application Execution:  The application management service ensures the reliable execution of grid applications. It monitors applications to detect failures and restart the failed ones (from a checkpoint, if some checkpointing mechanisms are available).
    • Multi-Tasks Applications Support: The application management services can manage complex applications composed of several tasks (code coupling, workflows, …). It is able to manage spacial and temporal dependencies between tasks of such applications.
    • Support for Distributed Shared Memory Applications: A Volatile Data Management Service (VDMS) helps programmers to build distributed applications based on shared memory paradigm.

Current Status

A prototype of Vigne is implemented in C (70K LOC). Our structured overlay implementation is based on Pastry for scalable key-based routing of messages, and the algorithms of Bamboo for the overlay maintenance. We implemented Paxos consensus algorithm for active replication of services with Semias. Semias is used to make the Application Management service highly available.
Vigne has been extensively tested on Grid’5000 on hundred of nodes in both static and dynamic environments.

Main Publications

Former contributors

Current affiliations (as known in January 2016) are mentioned.