Virtualization and Grid Computing

Cool to see the mainstream press (Newsweek) mention Nimbus: Number Crunching Made Easy.

Announcement: Perspectives on Distributed Computing: Thirty People, Four User Types, and the Distributed Computing User Experience is available for download.

“This report chronicles and analyzes the responses of thirty users to questions about using the Globus Toolkit - starting with summaries of results and conclusions but also including very detailed appendices and even transcripts of the interviews. Very interesting information for those involved in distributed computing.”

http://www.mcs.anl.gov/~childers/perspectives/

The Nimbus TP2.2 release provides a standalone context broker that can be used across Nimbus and EC2 clouds and continues our work on EC2 compatibility with the introduction of EC2 metadata server. In addition, the release contains new documentation and bug fixes.

See the changelog for all the details.

It was a good feeling to pause for a moment and put a user quotes page together for Nimbus. We’ve worked hard to make Nimbus usable and useful — but the best is yet to come!

Besides the good stuff added to Nimbus, this release also introduces something called the AutoContainer which allows you to set up a Globus Java web services environment, from scratch and with security working, within about a minute (requires Linux/OSX and Java 1.5+).

The main new features provided in this release are tools facilitating the deployment, configuration and management of clouds. We also updated our implementation to match the current Amazon EC2 deployment. In addition, the release contains new documentation and bug fixes.

You can download the new release from:
http://workspace.globus.org/downloads/index.html

The full changelog can be found here:
http://workspace.globus.org/vm/changelog.html#TP2.1

Tom Scavo writes on gridshib-user:

We are pleased to announce GridShib for Globus Toolkit v0.6.1:

http://gridshib.globus.org/downloads/gridshib-gt-0_6_1-src.tar.gz
http://gridshib.globus.org/downloads/gridshib-gt-0_6_1-src.zip

Please visit the GridShib for GT home page for an introduction and links to software and documentation:

http://gridshib.globus.org/docs/gridshib-gt-0.6.1/

This version of GridShib for GT is primarily a bug fix release. There is one new feature, and that is, a refactoring of the blacklisting framework that now permits the blacklisting of identity attributes (such as e-mail addresses) in addition to IP addresses and SAML name identifiers. See the CHANGES file for a complete list of changes in this version:

http://gridshib.globus.org/docs/gridshib-gt-0.6.1/CHANGES.txt

Along with GridShib SAML Tools v0.5.0, version 0.6.1 of GridShib for GT will be included in a Capability Kit to supplement the Coordinated TeraGrid Software and Services (CTSS) stack. This is the next step in a focused effort to deploy GridShib software at both the science gateways and resource providers throughout the TeraGrid. This work is funded by the NSF TeraGrid Grid Integration Group through a sub-award to NCSA.

Thank you for your continued support of GridShib!

It would be interesting to have a lot of input on this survey if you have some time, it’s only a few questions.

From: Stephen Childs
Subject: Survey on use of virtualization in production grids

Hi all,

Apologies for the slightly off-topic post. I am giving presentations soon on the use of virtualization in production grids and was thinking it might be nice to have some data to present!

I have set up a survey on the usage of virtualization in production grids at http://www.surveymonkey.com/s.aspx?sm=Sukm0sCdun6EAtpi7IXtwA_3d_3d

The survey will remain open until Friday August 29th. It shouldn’t take more than a few minutes to complete, and I think the results should be quite interesting for our community. I get the impression virtualization usage has rocketed in the last year, but haven’t seen any figures on uptake to date.

Please feel free to forward this to anyone else who may be interested in responding.

If you have any comments, please feel free to email me.

Stephen
–
Dr. Stephen Childs,
Research Fellow, EGEE Project, phone: +353-1-8961797
Computer Architecture Group, email: Stephen.Childs @ cs.tcd.ie
Trinity College Dublin, Ireland web: http://www.cs.tcd.ie/Stephen.Childs

It’s thrilling to organize things better.

Quoting from the Nimbus features page:

There are currently two supported remote protocol sets:

• WSRF based: protocol implementation in longstanding use by previous workspace services and clients including the cloud-client.
• EC2 based: clients written for EC2 can be used with Nimbus installations. For more information, see What is the EC2 frontend?

These protocols happen to both be Web Services based and both run in the Apache Axis based GT Java container. But neither thing is a necessity:

• There is nothing specific to web services based remote protocols in the workspace service implementation, the messaging system just needs to be able to speak to Java based libraries.
• Workspace service dependencies have nothing to do with what container it is running in, they are more “straight Java” style dependencies like Spring, ehcache, backport-util-concurrent, and JDBC (currently using the embedded Derby database).

See the announcement: new strong internal interfaces and a new remote protocol implementation (compatible with EC2 clients) that can run alongside the WSRF based ones.

Check out the changelog and the new FAQ.

Go Ask ALICE, the iSGTW image of the week. (Funny headline, see Go_Ask_Alice).

Check out some screenshots here of Nimbus resources invovled in supporting this experiment. It’s a small part of things as you can see from the scope of the grid but exciting nonetheless. The AliEn based virtual cluster is now “one-click” and can be launched anywhere running a workspace cloud setup.

VWS RSS feed:

• http://workspace.globus.org/rss.xml

A lot of developments with the workspace service and science clouds recently!

The cluster technology lets you bootstrap generic images into new network and security contexts on the fly. We built a sample cluster on top of the technology that lets you create the cluster and be immediately ready to submit jobs to a Torque cluster fronted by GRAM and GridFTP that use a newly created self-signed certificate:

1. cloud-client.sh –run –hours 12 –cluster base-cluster.xml

2. Wait a few minutes, once launched note the head-node hostname

3. scp -r root@HOSTNAME:certs/*  lib/certs/

   (SSH was bootstrapped end to end already)

4. Make sure your grid tools trust this certificate and then submit work

This can be done with nearly anything that can run on a non-virtual cluster. Check out these links for more information:

• http://workspace.globus.org/clouds/clusters.html
• http://workspace.globus.org/vm/TP1.3.3/index.html