Virtualization and Grid Computing

Cloud Computing and Its Applications

Also see Ian Foster’s blog entry

This month’s OGF newsletter has an article about the Cloud Systems BoF.

If you scroll down to the bottom of the latter link, there are slides and PDFs to view.

The mailing list URL in the newsletter is currently broken, this is the right one: http://www.ogf.org/mailman/listinfo/clouds-bof

Very interesting.

Overshadow: A Virtualization-Based Approach to Retrofitting Protection in Commodity Operating Systems

Part of the abstract:

we introduce a virtual-machine-based system called Overshadow that protects the privacy and integrity of application data, even in the event of a total OS compromise. Overshadow presents an application with a normal view of its resources, but the OS with an encrypted view. This allows the operating system to carry out the complex task of managing an application’s resources, without allowing it to read or modify them

Some cool new features:

On behalf of the workspace team, I am happy to announce the TP 1.3.1 release of the Workspace Service. You can download the new release from: http://workspace.globus.org/downloads/index.html

The main new feature in this release is the implementation of the workspace pilot which provides non-invasive adaptations to batch schedulers (such as PBS) enabling sites to run virtual machines alongside jobs. The details of this approach are described in: http://workspace.globus.org/papers/workspace-pilot-paper-submitted.pdf

In addition, the release also contains the ensemble service that allows clients to create ensembles of heterogeneous virtual machines to be deployed and managed together, improvements to the client, and several bug fixes. The complete changelog can be found at: http://workspace.globus.org/vm/TP1.3.1/index.html#changelog

We welcome comments, feedback, and bug reports. Information about the project, software downloads, documentation and instructions on how to join the workspace-user mailing list for support questions can be found at: http://workspace.globus.org

Happy Valentine’s Day!

As you can read there, the main new feature is the pilot infrastructure. The paper Kate refers to in the announcement is a relatively short read and lays out the ideas (and a practical evaluation) in an organized way. But briefy: the pilot is a program the service will submit to a local site resource manager in order to obtain time on the VMM nodes. When not allocated to the workspace service, these nodes will be used for jobs as normal. Those jobs run in normal system accounts in Xen domain 0 with no guest VMs running.

Importantly, the approach leaves the site resource manager in full control of the nodes and requires no modifications to the site resource manager. Save perhaps possible configuration changes you might like to make. For example, you can mark particular nodes as able to accomodate guest VMs: the workspace service supports sending pilot requests to particular LRM queues, or providing a particular node property etc. This allows you to really organize not just when but where VMs can run.

Several extra safeguards have been added to make sure the node is returned from VM hosting mode at the proper time, including support for:

• the workspace service being down or malfunctioning
• LRM preemption (including deliberate LRM job cancellation)
• node reboot/shutdown

Also included is a one-command “kill 9″ facility for administrators as a “worst case scenario” contingency.

So as a buzzword experiment, I want to put in a particular keyword here and see how the search engine hits work out :-). I think you know what it may be…

Cloud computing

Go make a cloud!

And with the workspace pilot, you won’t have to switch over all at once. Take it for a test run and tell us about it on workspace-user.

We’ve got some exciting stuff in the pipeline for the next few months, too (see the last release announcement and the self-configuring 100 node VM cluster news). I am really happy with where the project is going and has been recently.

- Tim

This Better Know a VM entry, Virtual Cluster Appliances, gives an overview of VM contextualization technology which is scheduled to be part of the next workspace service release. This is not just relevant to classic grid computing, but any situation where you’d like to automatically launch many virtual machines that work together and want them to securely organize themselves and adapt to the deployment environment. It can even be used for one VM, we’ll look at such cases later.

Kate will give a talk today: 10:30-11:00am at the AIST booth. Go check it out!

(In the future, you should be able to find the slides here).

Borja Sotomayor, my esteemed colleague, has an interesting article No CPU Left Behind about VMs and education. Check it out.

Following on Blue Pill counter argument, I thought I’d point you at Compatibility is Not Transparency: VMM Detection Myths and Realities. Interesting read, thanks for the pointer FS.

The date of the VTDC 2007 workshop has been confirmed. It will be on Monday, November 12th. See you there!

Last year’s International Workshop on Virtualization Technology in Distributed Computing (VTDC) was an interesting and productive day and an exciting complement to Supercomputing.

This year’s call for papers has been announced!

===============================================================
CALL FOR PAPERS (VTDC 2007)

Workshop on Virtualization Technologies in Distributed Computing

held in conjunction with SC 07, the International Conference for High Performance Computing, Networking and Storage.
===============================================================

date: November, 2007, Reno, NV, USA

SC 07: http://sc07.supercomputing.org/
VTDC 07: http://workspace.globus.org/vtdc07/
Last year’s workshop: http://workspace.globus.org/vtdc06/

General Information

The convergence of virtualization technologies and distributed computing is an exciting development and the subject of much research in both academia and industry. The VTDC workshop is intended to be a forum for the exchange of ideas and experiences on the use of virtualization technologies in distributed computing, the challenges and opportunities offered by the development of virtual systems themselves, as well as case studies of application of virtualization. The scope of “virtualization technologies” includes techniques and concepts to enable virtual machines, virtual networks, virtual data, virtual storage, virtual applications and virtual instruments. The scope of “distributed computing” includes Grid-computing, cluster computing, peer-to-peer computing and mobile computing.

VTDC 2007 topics of interest include, but are not limited to:

• Using virtualization technologies for resource management and QoS assurance
• Security aspects of using virtualization in a distributed environment
• Virtual networks
• Virtual data and storage systems
• Fault tolerance in virtualization
• Virtualization in P2P
• Monitoring techniques in virtualization
• Virtualization-based adaptive/autonomic systems
• Virtual datacenters
• Virtual environment factories and services
• Environment configuration
• Virtual machine management
• Modeling (applications and systems)
• Case studies of applications using virtual technologies
• Deployment studies of virtualization technologies
• Tools relevant to virtualization
• Virtualization as vehicle for outsourcing

Paper Submission

Authors are invited to submit original and unpublished work that exposes a new problem, advocates a specific solution, or reports on actual experience.

Papers should be submitted as full-length 8 page papers of double column text using single spaces 10pt size type on an 8.5″x11″ paper, as per IEEE manuscript guidelines. Paper submission instructions will be placed on the workshop web page at http://workspace.globus.org/vtdc07/.

Presentations will be invited based on the originality, technical merit, and topical relevance of their submissions. Please contact vtdc07@mcs.anl.gov with questions.

Important dates:

September 24, 2007 - Paper submission
October 16, 2007 - Notification of acceptance
October 26, 2007 - Final version due

It’s been busy lately, attended the first dev.Globus All Hands Meeting and TeraGrid ‘07 right here in Madison.

At TG07, Kate gave a talk which is online. The paper she presented discusses among other things contextualization, the structure and mechanisms by which an appliance/workspace is “told” what it needs in order to adapt to its deployed environment. This is not just adaptation to site specific services but also to other appliances that may be deployed with it such as in a virtual cluster deployment.

Amidst the bustle we implemented a new backend to the Workspace Service, to Amazon’s Elastic Compute Cloud (EC2). We’ve deployed it to the University of Chicago’s Teraport cluster and will currently pay for usage by selected collaborators.

Besides being somewhat fun to implement (including getting the Globus and Amazon Secure Message stacks on the same wavelength), I think it’s going to be interesting.

Because grid resources are cautiously approaching the pioneering switch to virtualizing resources [1], even in part, it is going to be interesting and educational to see what people will be able to accomplish with workspaces when a large pool of resources is actually available on tap — today.

Because the same deployment protocols can be used for both native and EC2 resources, there are of course capacity overflow use cases. In the right situations, VMs are a good mechanism for providers to dynamically reach more consumers as the need arises.

For a feature list and description, see What is the EC2 backend?

——-

[1] and some would say inevitable switch, even with the performance costs. Consider also that ‘virtualizing resources’ may mean physical node re-imaging, cf. Virtual Workspaces: Achieving Quality of Service and Quality of Life in the Grid.

The interesting presentations from the 4th Xen Summit are now available:

http://www.xensource.com/xen/xensummit.html

I did not get to go this time unfortunately!